[RFC v3 00/12] Intro to Hardware timestamping engine

[RFC v3 00/12] Intro to Hardware timestamping engine

[Dipen Patel]

This patch series introduces new subsystem called hardware timestamping
engine (HTE). It offers functionality such as timestamping through hardware
means in realtime. The HTE subsystem centralizes HTE provider and consumers
where providers can register themselves and the consumers can request
interested entity which could be lines, GPIO, signals or buses. The
HTE subsystem provides timestamp in nano seconds, having said that the provider
need to convert the timestamp if its not in that unit. There was upstream
discussion about the HTE at
https://lore.kernel.org/lkml/4c46726d-fa35-1a95-4295-bca37c8b6fe3@nvidia.com/

To summarize upstream discussion:
- It was heavily favoured by Linus and Kent to extend GPIOLIB and supporting
GPIO drivers to add HTE functionality and I agreed to experiment with it.
This patch series implements and extends GPIOLIB, GPIOLIB-CDEV and GPIO tegra
driver.
- Discussed possibility to add HTE provider as irqchip instead which
was argued against as HTE devices are not necessarily event emitting
devices. This RFC version 2 however tries to emulate threaded irq style
implementation.
- Discussed other possibility if HTE device can be added as posix clock
type like PTP clocks. That was also argues against since HTE devices
are not necessarily tightly coupled with hardware clock.

Typical HTE provider does following:
- Register itself with HTE subsystem
- Provide request, release, enable, disable timestamp and
get_clk_src_info callbacks to HTE subsystem.
- Provide optional xlate callback to the subsystem which can translate
consumer provided logical ids into actual ids of the entity, where entity here
is the provider dependent and could be GPIO, in chip lines or signals, buses
etc...This converted id will be used between HTE subsystem and the provider for
below bullet point.
- Push timestamps to the subsystem. This happens when HTE provider has
timestamp data available and willing to push it to HTE subsystem.
- Unregister itself on exit.

Typical HTE consumer does following:
- Request interested entity it wishes to timestamp in realtime to the
subsystem.
- The subsystem does necessary communications with the provider to
complete the request, which includes translating logical id of the entity to
provider dependent physical/actual id and enabling hardware timestamping on
requested id.
- The request includes callbacks, it will be used to push timestamps.
Optionally, the consumer can provided threaded callback, if specified, the HTE
subsystem will create kernel thread responsible executing the threaded callback.
- Release entity and its resources.

HTE and GPIOLIB:
- For the HTE provider which can timestamp GPIO lines.
- For the GPIO consumers, either in kernel or userspace, The GPIOLIB and its
CDEV framework are extended as frontend to the HTE by introducing new APIs.
- Tegra194 AON GPIO controller has HTE support known as GTE
(Generic Timestamping Engine). The tegra gpio driver is modified to accommodate
HTE functionality.

Changes in V2:
- Removed buffer management and related APIs from the HTE core.
- Removed timestamp retrieve APIs from the HTE core.
- Modified request API with two callbacks, second callback is invoked in thread
context and is optional, while first callback is mandatory and used to push
timestamp data to consumers.
- Replaced hte with hardware-timestamping in DT bindings as hte appeared too
short according to review comments.

Changes in V3:
- Corrected grammatical errors in HTE documentation and its bindings documents
- Removed multi-plural words in the HTE DT bindings
- Reflected changes done in DT bindings in the respective source codes
- Separated previous patch 07 into two patches in this series as 07 and 08
- Corrections in MAINTAINERS file

Dipen Patel (12):
  Documentation: Add HTE subsystem guide
  drivers: Add hardware timestamp engine (HTE)
  hte: Add tegra194 HTE kernel provider
  dt-bindings: Add HTE bindings
  hte: Add Tegra194 IRQ HTE test driver
  gpiolib: Add HTE support
  dt-bindings: gpio: Add hardware-timestamp-engine property
  gpio: tegra186: Add HTE in gpio-tegra186 driver
  gpiolib: cdev: Add hardware timestamp clock type
  tools: gpio: Add new hardware clock type
  hte: Add tegra GPIO HTE test driver
  MAINTAINERS: Added HTE Subsystem

 .../devicetree/bindings/gpio/gpio.txt         |   8 +
 .../bindings/gpio/nvidia,tegra186-gpio.txt    |   7 +
 .../hte/hardware-timestamps-common.yaml       |  29 +
 .../devicetree/bindings/hte/hte-consumer.yaml |  48 +
 .../bindings/hte/nvidia,tegra194-hte.yaml     |  80 ++
 Documentation/hte/hte.rst                     |  84 ++
 Documentation/hte/index.rst                   |  22 +
 Documentation/hte/tegra194-hte.rst            |  57 ++
 Documentation/index.rst                       |   1 +
 MAINTAINERS                                   |   8 +
 drivers/Kconfig                               |   2 +
 drivers/Makefile                              |   1 +
 drivers/gpio/gpio-tegra186.c                  |  89 ++
 drivers/gpio/gpiolib-cdev.c                   | 161 +++-
 drivers/gpio/gpiolib.c                        |  73 ++
 drivers/gpio/gpiolib.h                        |  12 +
 drivers/hte/Kconfig                           |  50 +
 drivers/hte/Makefile                          |   5 +
 drivers/hte/hte-tegra194-gpio-test.c          | 252 +++++
 drivers/hte/hte-tegra194-irq-test.c           | 169 ++++
 drivers/hte/hte-tegra194.c                    | 545 +++++++++++
 drivers/hte/hte.c                             | 907 ++++++++++++++++++
 include/linux/gpio/consumer.h                 |  19 +-
 include/linux/gpio/driver.h                   |  14 +
 include/linux/hte.h                           | 248 +++++
 include/uapi/linux/gpio.h                     |   1 +
 tools/gpio/gpio-event-mon.c                   |   6 +-
 27 files changed, 2886 insertions(+), 12 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/hte/hardware-timestamps-common.yaml
 create mode 100644 Documentation/devicetree/bindings/hte/hte-consumer.yaml
 create mode 100644 Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
 create mode 100644 Documentation/hte/hte.rst
 create mode 100644 Documentation/hte/index.rst
 create mode 100644 Documentation/hte/tegra194-hte.rst
 create mode 100644 drivers/hte/Kconfig
 create mode 100644 drivers/hte/Makefile
 create mode 100644 drivers/hte/hte-tegra194-gpio-test.c
 create mode 100644 drivers/hte/hte-tegra194-irq-test.c
 create mode 100644 drivers/hte/hte-tegra194.c
 create mode 100644 drivers/hte/hte.c
 create mode 100644 include/linux/hte.h


base-commit: 5191249f880367a4cd675825cd721a8d78f26a45
-- 
2.17.1

[RFC v3 01/12] Documentation: Add HTE subsystem guide

[Dipen Patel]

Adding hte document which can help understand various APIs implemented
in HTE framework for the HTE producers and the consumers.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
Changes in v2:
- Removed explanation, instead added kernel-doc references.

Changes in v3:
- Addressed grammatical errors.

 Documentation/hte/hte.rst | 84 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 84 insertions(+)
 create mode 100644 Documentation/hte/hte.rst

diff --git a/Documentation/hte/hte.rst b/Documentation/hte/hte.rst
new file mode 100644
index 000000000000..c39e57303d12
--- /dev/null
+++ b/Documentation/hte/hte.rst
@@ -0,0 +1,84 @@
+============================================
+The Linux Hardware Timestamping Engine (HTE)
+============================================
+
+:Author: Dipen Patel
+
+Introduction
+------------
+
+Certain devices have built in hardware timestamping engines which can
+monitor sets of system signals, lines, buses etc... in realtime for state
+change; upon detecting the change they can automatically store the timestamp at
+the moment of occurrence. Such functionality may help achieve better accuracy
+in obtaining timestamps than using software counterparts i.e. ktime and
+friends.
+
+This document describes the API that can be used by hardware timestamping
+engine provider and consumer drivers that want to use the hardware timestamping
+engine (HTE) framework. Both consumers and providers must
+#include <linux/hte.h>.
+
+The HTE framework APIs for the providers
+----------------------------------------
+
+.. kernel-doc:: drivers/hte/hte.c
+   :functions: devm_hte_register_chip hte_push_ts_ns
+
+The HTE framework APIs for the consumers
+----------------------------------------
+
+.. kernel-doc:: drivers/hte/hte.c
+   :functions: devm_of_hte_request_ts hte_req_ts_by_hte_name hte_release_ts hte_enable_ts hte_disable_ts hte_get_clk_src_info
+
+The HTE framework public structures
+-----------------------------------
+.. kernel-doc:: include/linux/hte.h
+
+
+More on the HTE timestamp data
+------------------------------
+The struct hte_ts_data is used to pass timestamp details between the consumers
+and the providers. It expresses timestamp data in nanoseconds in u64 data
+type. For now all the HTE APIs using struct hte_ts_data require tsc to be in
+nanoseconds. An example of the typical hte_ts_data data life cycle, for the
+GPIO line is as follows::
+
+ - Monitors GPIO line change.
+ - Detects the state change on GPIO line.
+ - Converts timestamps in nanoseconds and stores it in tsc.
+ - Stores GPIO direction in dir variable if the provider has that hardware
+ capability.
+ - Pushes this hte_ts_data object to HTE subsystem.
+ - HTE subsystem increments seq counter and invokes consumer provided callback.
+ Based on callback return value, the HTE starts a kernel thread and invokes
+ secondary callback in the thread context.
+
+HTE subsystem debugfs attributes
+--------------------------------
+HTE subsystem creates debugfs attributes at ``/sys/kernel/debug/hte/``.
+It also creates line/signal-related debugfs attributes at
+``/sys/kernel/debug/hte/<provider>/<label or line id>/``.
+
+`ts_requested`
+		The total number of entities requested from the given provider,
+		where entity is specified by the provider and could represent
+		lines, GPIO, chip signals, buses etc...
+                The attribute will be available at
+		``/sys/kernel/debug/hte/<provider>/``.
+
+		Read-only value
+
+`total_ts`
+		The total number of entities supported by the provider.
+                The attribute will be available at
+		``/sys/kernel/debug/hte/<provider>/``.
+
+		Read-only value
+
+`dropped_timestamps`
+		The dropped timestamps for a given line.
+                The attribute will be available at
+		``/sys/kernel/debug/hte/<provider>/<label or line id>/``.
+
+		Read-only value
-- 
2.17.1

[RFC v3 02/12] drivers: Add hardware timestamp engine (HTE)

[Dipen Patel]

Some devices can timestamp system lines/signals/Buses in real-time
using the hardware counter or other hardware means which can give
finer granularity and help avoid jitter introduced by software
timestamping. To utilize such functionality, this patchset creates
HTE subsystem where devices can register themselves as providers so
that the consumers devices can request specific line from the
providers.

It provides below APIs for the provider:
- devm_hte_register_chip -- To register the HTE provider.
- hte_push_ts_ns() -- To push timestamp data into HTE subsystem.

It provides below APIs for the consumer:
- devm_of_hte_request_ts() -- Request timestamp functionality.
- hte_req_ts_by_hte_name() -- To request timestamp functionality by
using HTE provider dt node.
- hte_enable_ts() -- To disable timestamp functionality.
- hte_disable_ts() -- To enable timestamp functionality.
- hte_release_ts() -- To release timestamp functionality and its
associated resources.
- hte_get_clk_src_info() -- To query clock source information from
the provider

The detail about parameters and API usage are described in each
functions definitions in drivers/hte/hte.c file.

The patch adds compilation support in Makefile and menu options in
Kconfig.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
Reported-by: kernel test robot <lkp@intel.com>
---
Changes in v2:
- Removed buffer abstraction layer as well related APIs, HTE now will not store
any data, instead will pass to consumer as soon as it is available.
- Removed unnecessary dynamical allocations
- Removed timestamp retrieve API
- Removed release, unregister related APIs as their counterpart are resource
managed.
- Added kernel thread implementation if consumer indicates threaded callback
during request API time.
- Changed hte_req_ts_by_dt_node API to remove device node exposure from the
interface, instead consumer will their device node with property name that
indicates the provider it wants to use.

Changes in v3:
- Addressed grammatical/spelling errors.

 drivers/Kconfig      |   2 +
 drivers/Makefile     |   1 +
 drivers/hte/Kconfig  |  22 ++
 drivers/hte/Makefile |   2 +
 drivers/hte/hte.c    | 907 +++++++++++++++++++++++++++++++++++++++++++
 include/linux/hte.h  | 248 ++++++++++++
 6 files changed, 1182 insertions(+)
 create mode 100644 drivers/hte/Kconfig
 create mode 100644 drivers/hte/Makefile
 create mode 100644 drivers/hte/hte.c
 create mode 100644 include/linux/hte.h

diff --git a/drivers/Kconfig b/drivers/Kconfig
index 0d399ddaa185..b3c8332bf5c7 100644
--- a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -236,4 +236,6 @@ source "drivers/interconnect/Kconfig"
 source "drivers/counter/Kconfig"
 
 source "drivers/most/Kconfig"
+
+source "drivers/hte/Kconfig"
 endmenu
diff --git a/drivers/Makefile b/drivers/Makefile
index be5d40ae1488..087c08e846e6 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -188,3 +188,4 @@ obj-$(CONFIG_GNSS)		+= gnss/
 obj-$(CONFIG_INTERCONNECT)	+= interconnect/
 obj-$(CONFIG_COUNTER)		+= counter/
 obj-$(CONFIG_MOST)		+= most/
+obj-$(CONFIG_HTE)		+= hte/
diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
new file mode 100644
index 000000000000..1fcfe17cf28a
--- /dev/null
+++ b/drivers/hte/Kconfig
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: GPL-2.0-only
+menuconfig HTE
+	bool "Hardware Timestamping Engine (HTE) Support"
+	help
+	  Hardware Timestamping Engine (HTE) Support.
+
+	  Some devices provide a hardware timestamping engine which can
+	  timestamp certain device lines/signals in realtime. This provides a
+	  hardware-assisted timestamp to generic signals like GPIOs or IRQs
+	  lines. It comes with a benefit for applications like autonomous
+	  machines needing accurate timestamping event with less jitter.
+
+	  This framework provides a generic interface to such HTE devices
+	  within the Linux kernel. It provides an API to register and
+	  unregister a HTE provider chip, configurable software buffer to
+	  store the timestamps, push the timestamp from the HTE providers and
+	  retrieve timestamps for the consumers. It also provides means for the
+	  consumers to request signals it wishes to hardware timestamp and
+	  release them if not required.
+
+	  If unsure, say no.
+
diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
new file mode 100644
index 000000000000..fc03bdf44427
--- /dev/null
+++ b/drivers/hte/Makefile
@@ -0,0 +1,2 @@
+obj-$(CONFIG_HTE)		+= hte.o
+
diff --git a/drivers/hte/hte.c b/drivers/hte/hte.c
new file mode 100644
index 000000000000..ca91d48f48b9
--- /dev/null
+++ b/drivers/hte/hte.c
@@ -0,0 +1,907 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 NVIDIA Corporation
+ *
+ * Author: Dipen Patel <dipenp@nvidia.com>
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/err.h>
+#include <linux/slab.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/mutex.h>
+#include <linux/sched.h>
+#include <linux/uaccess.h>
+#include <linux/hte.h>
+#include <linux/delay.h>
+#include <linux/debugfs.h>
+#include <linux/kthread.h>
+
+#define HTE_TS_NAME_LEN		10
+
+/* Global list of the HTE devices */
+static DEFINE_SPINLOCK(hte_lock);
+static LIST_HEAD(hte_devices);
+
+enum {
+	HTE_TS_REGISTERED,
+	HTE_TS_DISABLE,
+};
+
+enum {
+	HTE_CB_RUN_THREAD,
+	HTE_CB_NUM,
+};
+
+/**
+ * struct hte_ts_info - Information related to requested timestamp.
+ *
+ * @xlated_id: Timestamp ID as understood between HTE subsys and HTE provider,
+ * See xlate callback API.
+ * @flags: Flags holding state informations.
+ * @hte_cb_flags: Callback related flags.
+ * @seq: Timestamp sequence counter.
+ * @hte_name: Indicates if HTE core has set name for this timestamp entity.
+ * @cb: Callback function provided by clients.
+ * @tcb: Threaded callback function provided by clients.
+ * @dropped_ts: Dropped timestamps.
+ * @slock: Spin lock.
+ * @thread: Thread task when tcb is provided.
+ * @req_mlock: Lock during timestamp request/release APIs.
+ * @ts_dbg_root: Root for the debug fs.
+ * @gdev: HTE abstract device that this timestamp belongs to.
+ * @cl_data: Client specific data.
+ */
+struct hte_ts_info {
+	u32 xlated_id;
+	unsigned long flags;
+	unsigned long hte_cb_flags;
+	u64 seq;
+	bool hte_name;
+	hte_ts_cb_t cb;
+	hte_ts_threaded_cb_t tcb;
+	atomic_t dropped_ts;
+	spinlock_t slock;
+	struct task_struct *thread;
+	struct mutex req_mlock;
+	struct dentry *ts_dbg_root;
+	struct hte_device *gdev;
+	void *cl_data;
+};
+
+/**
+ * struct hte_device - HTE abstract device
+ * @nlines: Number of entities this device supports.
+ * @ts_req: Total number of entities requested.
+ * @sdev: Device used at various debug prints.
+ * @dbg_root: Root directory for debug fs.
+ * @list: List node to store hte_device for each provider.
+ * @chip: HTE chip providing this HTE device.
+ * @owner: helps prevent removal of modules when in use.
+ * @ei: Timestamp information.
+ */
+struct hte_device {
+	u32 nlines;
+	atomic_t ts_req;
+	struct device *sdev;
+	struct dentry *dbg_root;
+	struct list_head list;
+	struct hte_chip *chip;
+	struct module *owner;
+	struct hte_ts_info ei[];
+};
+
+#ifdef CONFIG_DEBUG_FS
+
+static struct dentry *hte_root;
+
+static int __init hte_subsys_dbgfs_init(void)
+{
+	/* creates /sys/kernel/debug/hte/ */
+	hte_root = debugfs_create_dir("hte", NULL);
+
+	return 0;
+}
+subsys_initcall(hte_subsys_dbgfs_init);
+
+static void hte_chip_dbgfs_init(struct hte_device *gdev)
+{
+	const struct hte_chip *chip = gdev->chip;
+	const char *name = chip->name ? chip->name : dev_name(chip->dev);
+
+	gdev->dbg_root = debugfs_create_dir(name, hte_root);
+
+	debugfs_create_atomic_t("ts_requested", 0444, gdev->dbg_root,
+				&gdev->ts_req);
+	debugfs_create_u32("total_ts", 0444, gdev->dbg_root,
+			   &gdev->nlines);
+}
+
+static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
+{
+	if (!ei->gdev->dbg_root || !name)
+		return;
+
+	ei->ts_dbg_root = debugfs_create_dir(name, ei->gdev->dbg_root);
+
+	debugfs_create_atomic_t("dropped_timestamps", 0444, ei->ts_dbg_root,
+				&ei->dropped_ts);
+}
+
+#else
+
+static void hte_chip_dbgfs_init(struct hte_device *gdev)
+{
+}
+
+static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
+{
+}
+
+#endif
+
+/**
+ * hte_release_ts() - Consumer calls this API to release the entity, where
+ * entity could be anything providers support, like lines, signals, buses,
+ * etc...
+ *
+ * @desc: timestamp descriptor, this is the same as returned by the request API.
+ *
+ * Context: debugfs_remove_recursive() function call may use sleeping locks,
+ *	    not suitable from atomic context.
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int hte_release_ts(struct hte_ts_desc *desc)
+{
+	u32 id;
+	int ret = 0;
+	unsigned long flag;
+	struct hte_device *gdev;
+	struct hte_ts_info *ei;
+
+	if (!desc)
+		return -EINVAL;
+
+	ei = desc->hte_data;
+
+	if (!ei || !ei->gdev)
+		return -EINVAL;
+
+	gdev = ei->gdev;
+	id = desc->con_id;
+
+	mutex_lock(&ei->req_mlock);
+
+	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
+		dev_info(gdev->sdev, "id:%d is not registered", id);
+		ret = -EUSERS;
+		goto unlock;
+	}
+
+	ret = gdev->chip->ops->release(gdev->chip, ei->xlated_id);
+	if (ret) {
+		dev_err(gdev->sdev, "id: %d free failed\n", id);
+		goto unlock;
+	}
+
+	if (ei->hte_name)
+		kfree(desc->name);
+
+	debugfs_remove_recursive(ei->ts_dbg_root);
+
+	spin_lock_irqsave(&ei->slock, flag);
+
+	atomic_dec(&gdev->ts_req);
+	atomic_set(&ei->dropped_ts, 0);
+
+	ei->seq = 0;
+	desc->hte_data = NULL;
+
+	clear_bit(HTE_TS_REGISTERED, &ei->flags);
+
+	spin_unlock_irqrestore(&ei->slock, flag);
+
+	if (ei->tcb) {
+		kthread_stop(ei->thread);
+		put_task_struct(ei->thread);
+	}
+
+	ei->cb = NULL;
+	ei->tcb = NULL;
+	ei->thread = NULL;
+	ei->cl_data = NULL;
+
+	module_put(gdev->owner);
+unlock:
+	mutex_unlock(&ei->req_mlock);
+	dev_dbg(gdev->sdev, "release id: %d\n", id);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(hte_release_ts);
+
+static int hte_ts_dis_en_common(struct hte_ts_desc *desc, bool en)
+{
+	u32 ts_id;
+	struct hte_device *gdev;
+	struct hte_ts_info *ei;
+	int ret;
+	unsigned long flag;
+
+	if (!desc)
+		return -EINVAL;
+
+	ei = desc->hte_data;
+
+	if (!ei || !ei->gdev)
+		return -EINVAL;
+
+	gdev = ei->gdev;
+	ts_id = desc->con_id;
+
+	mutex_lock(&ei->req_mlock);
+
+	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
+		dev_dbg(gdev->sdev, "id:%d is not registered", ts_id);
+		ret = -EUSERS;
+		goto out;
+	}
+
+	spin_lock_irqsave(&ei->slock, flag);
+
+	if (en) {
+		if (!test_bit(HTE_TS_DISABLE, &ei->flags)) {
+			ret = 0;
+			goto out_unlock;
+		}
+
+		spin_unlock_irqrestore(&ei->slock, flag);
+		ret = gdev->chip->ops->enable(gdev->chip, ei->xlated_id);
+		if (ret) {
+			dev_warn(gdev->sdev, "id: %d enable failed\n",
+				 ts_id);
+			goto out;
+		}
+
+		spin_lock_irqsave(&ei->slock, flag);
+		clear_bit(HTE_TS_DISABLE, &ei->flags);
+	} else {
+		if (test_bit(HTE_TS_DISABLE, &ei->flags)) {
+			ret = 0;
+			goto out_unlock;
+		}
+
+		spin_unlock_irqrestore(&ei->slock, flag);
+		ret = gdev->chip->ops->disable(gdev->chip, ei->xlated_id);
+		if (ret) {
+			dev_warn(gdev->sdev, "id: %d disable failed\n",
+				 ts_id);
+			goto out;
+		}
+
+		spin_lock_irqsave(&ei->slock, flag);
+		set_bit(HTE_TS_DISABLE, &ei->flags);
+	}
+
+out_unlock:
+	spin_unlock_irqrestore(&ei->slock, flag);
+out:
+	mutex_unlock(&ei->req_mlock);
+	return ret;
+}
+
+/**
+ * hte_disable_ts() - Disable timestamp on given descriptor.
+ *
+ * The API does not release any resources associated with desc.
+ *
+ * @desc: ts descriptor, this is the same as returned by the request API.
+ *
+ * Context: Holds mutex lock, not suitable from atomic context.
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int hte_disable_ts(struct hte_ts_desc *desc)
+{
+	return hte_ts_dis_en_common(desc, false);
+}
+EXPORT_SYMBOL_GPL(hte_disable_ts);
+
+/**
+ * hte_enable_ts() - Enable timestamp on given descriptor.
+ *
+ * @desc: ts descriptor, this is the same as returned by the request API.
+ *
+ * Context: Holds mutex lock, not suitable from atomic context.
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int hte_enable_ts(struct hte_ts_desc *desc)
+{
+	return hte_ts_dis_en_common(desc, true);
+}
+EXPORT_SYMBOL_GPL(hte_enable_ts);
+
+static int hte_simple_xlate(struct hte_chip *gc,
+			    const struct of_phandle_args *args,
+			    struct hte_ts_desc *desc,
+			    u32 *id)
+{
+	if (!id || !desc || !gc)
+		return -EINVAL;
+
+	/*
+	 * For the providers which do not have any internal mappings between
+	 * logically exposed ids and actual ids, will set both
+	 * the same.
+	 *
+	 * In case there is a internal mapping needed, providers will need to
+	 * provide its own xlate function where con_id will be sent as
+	 * args[0] and it will return xlated id. Later xlated id will be
+	 * used for any future exchanges between provider and subsystems.
+	 */
+
+	if (args) {
+		if (gc->of_hte_n_cells < 1)
+			return -EINVAL;
+
+		if (args->args_count != gc->of_hte_n_cells)
+			return -EINVAL;
+
+		*id = args->args[0];
+		desc->con_id = *id;
+	} else {
+		*id = desc->con_id;
+	}
+
+	if (desc->con_id > gc->nlines)
+		return -EINVAL;
+
+	desc->hte_data = NULL;
+
+	return 0;
+}
+
+static int _hte_wait_for_ts_data(struct hte_ts_info *ei)
+{
+	for (;;) {
+		set_current_state(TASK_INTERRUPTIBLE);
+
+		if (kthread_should_stop()) {
+			if (test_and_clear_bit(HTE_CB_RUN_THREAD,
+			    &ei->hte_cb_flags)) {
+				__set_current_state(TASK_RUNNING);
+				return 0;
+			}
+			__set_current_state(TASK_RUNNING);
+			return -1;
+		}
+
+		if (test_and_clear_bit(HTE_CB_RUN_THREAD,
+				       &ei->hte_cb_flags)) {
+			__set_current_state(TASK_RUNNING);
+			return 0;
+		}
+		schedule();
+	}
+}
+
+static int _hte_threadfn(void *data)
+{
+	struct hte_ts_info *ei = data;
+
+	while (!_hte_wait_for_ts_data(ei))
+		ei->tcb(ei->cl_data);
+
+	return 0;
+}
+
+static int _hte_setup_thread(struct hte_ts_info *ei, u32 id)
+{
+	struct task_struct *t;
+
+	t = kthread_create(_hte_threadfn, ei, "hte-%u", id);
+	if (IS_ERR(t))
+		return PTR_ERR(t);
+
+	ei->thread = get_task_struct(t);
+
+	return 0;
+}
+
+static int ___hte_req_ts(struct hte_device *gdev, struct hte_ts_desc *desc,
+			 u32 xlated_id, hte_ts_cb_t cb,
+			 hte_ts_threaded_cb_t tcb, void *data)
+{
+	struct hte_ts_info *ei;
+	int ret;
+	u32 con_id = desc->con_id;
+
+	if (!try_module_get(gdev->owner))
+		return -ENODEV;
+
+	ei = &gdev->ei[xlated_id];
+	ei->xlated_id = xlated_id;
+
+	/*
+	 * There is a chance that multiple consumers requesting same entity,
+	 * lock here.
+	 */
+	mutex_lock(&ei->req_mlock);
+
+	if (test_bit(HTE_TS_REGISTERED, &ei->flags)) {
+		dev_dbg(gdev->chip->dev, "id:%u is already registered",
+			xlated_id);
+		ret = -EUSERS;
+		goto unlock;
+	}
+
+	ei->cb = cb;
+	ei->tcb = tcb;
+	if (tcb) {
+		ret = _hte_setup_thread(ei, xlated_id);
+		if (ret < 0) {
+			dev_err(gdev->chip->dev, "setting thread failed\n");
+			goto unlock;
+		}
+	}
+
+	ret = gdev->chip->ops->request(gdev->chip, xlated_id);
+	if (ret < 0) {
+		dev_err(gdev->chip->dev, "ts request failed\n");
+		goto unlock;
+	}
+
+	desc->hte_data = ei;
+	ei->cl_data = data;
+
+	atomic_inc(&gdev->ts_req);
+
+	ei->hte_name = false;
+	if (!desc->name) {
+		desc->name = kzalloc(HTE_TS_NAME_LEN, GFP_KERNEL);
+		if (desc->name) {
+			scnprintf(desc->name, HTE_TS_NAME_LEN, "ts_%u",
+				  con_id);
+			ei->hte_name = true;
+		}
+	}
+
+	hte_ts_dbgfs_init(desc->name, ei);
+	set_bit(HTE_TS_REGISTERED, &ei->flags);
+
+	mutex_unlock(&ei->req_mlock);
+
+	dev_dbg(gdev->chip->dev, "id: %u, xlated id:%u", con_id, xlated_id);
+
+	return 0;
+
+unlock:
+	module_put(gdev->owner);
+	mutex_unlock(&ei->req_mlock);
+
+	return ret;
+}
+
+static struct hte_device *of_node_to_htedevice(struct device_node *np)
+{
+	struct hte_device *gdev;
+
+	spin_lock(&hte_lock);
+
+	list_for_each_entry(gdev, &hte_devices, list)
+		if (gdev->chip && gdev->chip->dev &&
+		    gdev->chip->dev->of_node == np) {
+			spin_unlock(&hte_lock);
+			return gdev;
+		}
+
+	spin_unlock(&hte_lock);
+
+	return ERR_PTR(-ENODEV);
+}
+
+static struct hte_device *of_hte_dev_get(struct device *dev,
+					 struct device_node *np,
+					 const char *label,
+					 struct of_phandle_args *args)
+{
+	struct hte_device *gdev = NULL;
+	int index = 0;
+	int err;
+
+	if (label) {
+		index = of_property_match_string(np,
+						 "hardware-timestamp-names",
+						 label);
+		if (index < 0)
+			return ERR_PTR(index);
+	}
+
+	err = of_parse_phandle_with_args(np, "hardware-timestamps",
+					 "#hardware-timestamp-cells", index,
+					 args);
+	if (err) {
+		pr_err("%s(): can't parse \"hardware-timestamps\" property\n",
+		       __func__);
+		return ERR_PTR(err);
+	}
+
+	gdev = of_node_to_htedevice(args->np);
+	if (IS_ERR(gdev)) {
+		pr_err("%s(): HTE chip not found\n", __func__);
+		of_node_put(args->np);
+		return gdev;
+	}
+
+	return gdev;
+}
+
+static int __hte_req_ts(struct device *dev, struct hte_ts_desc *desc,
+			hte_ts_cb_t cb, hte_ts_threaded_cb_t tcb, void *data)
+{
+	struct hte_device *gdev = NULL;
+	struct of_phandle_args args;
+	int ret;
+	u32 xlated_id;
+
+	gdev = of_hte_dev_get(dev, dev->of_node, desc->name, &args);
+	if (IS_ERR(gdev))
+		return PTR_ERR(gdev);
+
+	if (!gdev->chip) {
+		pr_debug("requested id does not have provider\n");
+		return -ENODEV;
+	}
+
+	ret = gdev->chip->xlate(gdev->chip, &args, desc, &xlated_id);
+	if (ret < 0)
+		goto put;
+
+	ret = ___hte_req_ts(gdev, desc, xlated_id, cb, tcb, data);
+	if (ret < 0)
+		goto put;
+
+	return 0;
+
+put:
+	of_node_put(args.np);
+
+	return ret;
+}
+
+static void __devm_hte_release_ts(void *res)
+{
+	hte_release_ts(res);
+}
+
+/**
+ * devm_of_hte_request_ts() - Resource managed API to request the HTE facility
+ * on the specified entity, where entity is provider specific for example,
+ * GPIO lines, signals, buses etc...
+ *
+ * The API allocates necessary resources and enables the timestamp. So calling
+ * hte_enable_ts is not needed. The consumer does not need to call
+ * hte_release_ts since it will be called upon consumer exit.
+ *
+ * @dev: HTE consumer/client device.
+ * @desc: Pre-allocated timestamp descriptor. HTE core will fill out necessary
+ * details. Optionally the consumer can set name field of desc, if not
+ * specified HTE core will set it as ts_con_id. It will be the consumer's
+ * job to free any allocation related to this structure as well name field
+ * in case it has set that field.
+ * @cb: Callback to push the timestamp data to consumer.
+ * @tcb: Optional callback. If its provided, subsystem will create
+ * thread. This will be called when cb returns HTE_RUN_THREADED_CB.
+ * @data: Client data, will be sent back during cb and tcb callbacks.
+ *
+ * Context: Holds mutex lock.
+ * Returns: Returns 0 on success or negative error code on failure.
+ */
+int devm_of_hte_request_ts(struct device *dev, struct hte_ts_desc *desc,
+			   hte_ts_cb_t cb, hte_ts_threaded_cb_t tcb,
+			   void *data)
+{
+	int err;
+
+	if (!dev || !dev->of_node || !desc || !cb)
+		return -EINVAL;
+
+	err = __hte_req_ts(dev, desc, cb, tcb, data);
+	if (err)
+		return err;
+
+	err = devm_add_action_or_reset(dev, __devm_hte_release_ts, desc);
+	if (err)
+		return err;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(devm_of_hte_request_ts);
+
+/**
+ * hte_req_ts_by_hte_name() - Request entity to timestamp realtime by passing
+ * property name that contains HTE provider phandle, meaning of the entity
+ * is HTE provider specific, for example lines, signals, GPIOs, buses etc...
+ *
+ * This API is designed to address below uses cases:
+ *
+ * 1) For the consumer device which acts as a central device for secondary
+ * consumers. For example, GPIO controller driver acts as a primary consumer
+ * on behalf of in kernel and userspace GPIO HTE consumers. The GPIO controller
+ * driver specifies HTE provider that it supports/wants and it becomes opaque
+ * for the secondary consumers requesting GPIO and hardware timestamp through
+ * that GPIO controller.
+ *
+ * 2) For the providers which are dependent on other hardware modules. In that
+ * case it forces consumers to go through other subsystem or driver making them
+ * secondary consumers. Same example as above applies here as well.
+ *
+ * The API allocates necessary resources and enables the timestamp. So calling
+ * hte_enable_ts is not needed.
+ *
+ * @dev: HTE consumer/client device.
+ * @propname: Name of property holding a HTE provider phandle value
+ * @desc: Pre-allocated timestamp descriptor with con_id set by the consumer.
+ * HTE core will fill out the rest. Optionally the consumer can set name
+ * field of desc, if not specified HTE core will set it as ts_con_id. It will
+ * be the consumer's job to free any allocation related to this structure as
+ * well name field in case it has set that field.
+ * @cb: Callback to push the timestamp data to consumer.
+ * @tcb: Optional callback. If its provided, subsystem will create
+ * thread. This will be called when cb returns HTE_RUN_THREADED_CB.
+ * @data: Client data, will be sent back during cb and tcb callbacks.
+ *
+ * Context: Holds mutex lock, can not be called from atomic context. The mutex
+ * lock is used to serialize multiple consumers.
+ * Returns: returns 0 on success or negative error code on failure.
+ */
+int hte_req_ts_by_hte_name(struct device *dev, const char *propname,
+			   struct hte_ts_desc *desc, hte_ts_cb_t cb,
+			   hte_ts_threaded_cb_t tcb, void *data)
+{
+	struct hte_device *gdev;
+	struct device_node *np = NULL;
+	int ret;
+	u32 xlated_id;
+
+	if (!dev->of_node || !propname || !desc)
+		return -EINVAL;
+
+	np = of_parse_phandle(dev->of_node, propname, 0);
+	if (!np)
+		return -ENODEV;
+
+	of_node_put(np);
+
+	gdev = of_node_to_htedevice(np);
+	if (IS_ERR(gdev))
+		return -ENOTSUPP;
+
+	if (!gdev->chip || !gdev->chip->ops)
+		return -ENOTSUPP;
+
+	ret = gdev->chip->xlate(gdev->chip, NULL, desc, &xlated_id);
+	if (ret < 0) {
+		dev_err(gdev->chip->dev,
+			"failed to xlate id: %d\n", desc->con_id);
+		return ret;
+	}
+
+	ret = ___hte_req_ts(gdev, desc, xlated_id, cb, tcb, data);
+	if (ret < 0) {
+		dev_err(gdev->chip->dev,
+			"failed to request id: %d\n", desc->con_id);
+		return ret;
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(hte_req_ts_by_hte_name);
+
+/**
+ * hte_get_clk_src_info() - Consumer calls this API to query clock source
+ * information of the desc.
+ *
+ * @desc: ts descriptor, same as returned from request API.
+ * @ci: The API fills this structure with the clock information data.
+ *
+ * Context: Any context.
+ * Returns: 0 on success else negative error code on failure.
+ */
+int hte_get_clk_src_info(const struct hte_ts_desc *desc,
+			 struct hte_clk_info *ci)
+{
+	struct hte_chip *chip;
+	struct hte_ts_info *ei;
+
+	if (!desc || !desc->hte_data || !ci) {
+		pr_debug("%s:%d\n", __func__, __LINE__);
+		return -EINVAL;
+	}
+
+	ei = desc->hte_data;
+	if (!ei || !ei->gdev || !ei->gdev->chip)
+		return -EINVAL;
+
+	chip = ei->gdev->chip;
+	if (!chip->ops->get_clk_src_info)
+		return -ENOTSUPP;
+
+	return chip->ops->get_clk_src_info(chip, ci);
+}
+EXPORT_SYMBOL_GPL(hte_get_clk_src_info);
+
+/**
+ * hte_push_ts_ns() - Used by the provider to push timestamp in nano
+ * seconds i.e data->tsc will be in ns.
+ *
+ * @chip: The HTE chip, used during the registration.
+ * @xlated_id: entity id understood by both subsystem and provider, usually this
+ * is obtained from xlate callback during request API.
+ * @data: timestamp data.
+ *
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int hte_push_ts_ns(const struct hte_chip *chip, u32 xlated_id,
+		   struct hte_ts_data *data)
+{
+	hte_return_t ret;
+	int st = 0;
+	struct hte_ts_info *ei;
+	unsigned long flag;
+
+	if (!chip || !data || !chip->gdev)
+		return -EINVAL;
+
+	if (xlated_id > chip->nlines)
+		return -EINVAL;
+
+	ei = &chip->gdev->ei[xlated_id];
+
+	spin_lock_irqsave(&ei->slock, flag);
+
+	/* timestamp sequence counter */
+	data->seq = ei->seq++;
+
+	if (!test_bit(HTE_TS_REGISTERED, &ei->flags) ||
+	    test_bit(HTE_TS_DISABLE, &ei->flags)) {
+		dev_dbg(chip->dev, "Unknown timestamp push\n");
+		st = -EINVAL;
+		goto unlock;
+	}
+
+	ret = ei->cb(data, ei->cl_data);
+	if (ret == HTE_RUN_THREADED_CB && ei->thread) {
+		if (test_and_set_bit(HTE_CB_RUN_THREAD, &ei->hte_cb_flags))
+			goto unlock;
+		else
+			wake_up_process(ei->thread);
+	} else if (ret == HTE_CB_TS_DROPPED) {
+		atomic_inc(&ei->dropped_ts);
+	} else if (ret == HTE_CB_ERROR) {
+		dev_dbg(chip->dev, "cb error\n");
+	}
+
+unlock:
+	spin_unlock_irqrestore(&ei->slock, flag);
+
+	return st;
+}
+EXPORT_SYMBOL_GPL(hte_push_ts_ns);
+
+static int hte_register_chip(struct hte_chip *chip)
+{
+	struct hte_device *gdev;
+	u32 i;
+
+	if (!chip || !chip->dev || !chip->dev->of_node)
+		return -EINVAL;
+
+	if (!chip->ops || !chip->ops->request || !chip->ops->release) {
+		dev_err(chip->dev, "Driver needs to provide ops\n");
+		return -EINVAL;
+	}
+
+	gdev = kzalloc(struct_size(gdev, ei, chip->nlines), GFP_KERNEL);
+	if (!gdev)
+		return -ENOMEM;
+
+	gdev->chip = chip;
+	chip->gdev = gdev;
+	gdev->nlines = chip->nlines;
+	gdev->sdev = chip->dev;
+
+	for (i = 0; i < chip->nlines; i++) {
+		gdev->ei[i].gdev = gdev;
+		mutex_init(&gdev->ei[i].req_mlock);
+		spin_lock_init(&gdev->ei[i].slock);
+	}
+
+	if (chip->dev->driver)
+		gdev->owner = chip->dev->driver->owner;
+	else
+		gdev->owner = THIS_MODULE;
+
+	if (!chip->xlate) {
+		chip->xlate = hte_simple_xlate;
+		/* Just a id number to monitor */
+		chip->of_hte_n_cells = 1;
+	}
+
+	of_node_get(chip->dev->of_node);
+
+	INIT_LIST_HEAD(&gdev->list);
+
+	spin_lock(&hte_lock);
+	list_add_tail(&gdev->list, &hte_devices);
+	spin_unlock(&hte_lock);
+
+	hte_chip_dbgfs_init(gdev);
+
+	dev_dbg(chip->dev, "Added hte chip\n");
+
+	return 0;
+}
+
+/**
+ * hte_unregister_chip() - Used by the provider to remove a HTE chip.
+ * @chip: the HTE chip to remove.
+ *
+ * Context: Can not be called from atomic context.
+ * Returns: 0 on success or a negative error code on failure.
+ */
+static int hte_unregister_chip(struct hte_chip *chip)
+{
+	struct hte_device *gdev;
+
+	if (!chip)
+		return -EINVAL;
+
+	gdev = chip->gdev;
+
+	spin_lock(&hte_lock);
+	list_del(&gdev->list);
+	spin_unlock(&hte_lock);
+
+	gdev->chip = NULL;
+
+	of_node_put(chip->dev->of_node);
+	debugfs_remove_recursive(gdev->dbg_root);
+	kfree(gdev);
+
+	dev_dbg(chip->dev, "Removed hte chip\n");
+
+	return 0;
+}
+
+static void _hte_devm_unregister_chip(void *chip)
+{
+	hte_unregister_chip(chip);
+}
+
+/**
+ * devm_hte_register_chip() - Used by provider to register a HTE chip.
+ * @chip: the HTE chip to add to subsystem.
+ *
+ * The API is resource managed and  _hte_devm_unregister_chip will be called
+ * automatically when the provider exits.
+ *
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int devm_hte_register_chip(struct hte_chip *chip)
+{
+	int err;
+
+	err = hte_register_chip(chip);
+	if (err)
+		return err;
+
+	err = devm_add_action_or_reset(chip->dev, _hte_devm_unregister_chip,
+				       chip);
+	if (err)
+		return err;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(devm_hte_register_chip);
diff --git a/include/linux/hte.h b/include/linux/hte.h
new file mode 100644
index 000000000000..f4dd5415a493
--- /dev/null
+++ b/include/linux/hte.h
@@ -0,0 +1,248 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 NVIDIA Corporation
+ *
+ * Author: Dipen Patel <dipenp@nvidia.com>
+ */
+
+#ifndef __LINUX_HTE_H
+#define __LINUX_HTE_H
+
+#include <linux/errno.h>
+
+struct hte_chip;
+struct hte_device;
+struct of_phandle_args;
+struct device_node;
+
+/**
+ * enum hte_dir - HTE edge timestamp direction.
+ *
+ * @HTE_RISING_EDGE_TS: Timestamps is for rising edge.
+ * @HTE_FALLING_EDGE_TS: Timestamps is for falling edge.
+ * @HTE_DIR_NOSUPP: Direction is not supported.
+ */
+enum hte_dir {
+	HTE_RISING_EDGE_TS,
+	HTE_FALLING_EDGE_TS,
+	HTE_DIR_NOSUPP,
+};
+
+/**
+ * struct hte_ts_data - HTE timestamp data.
+ * The provider uses and fills timestamp related details during push_timestamp
+ * API call. The consumer uses during retrieve_timestamp API call.
+ *
+ * @tsc: Timestamp value.
+ * @seq: Sequence counter of the timestamps.
+ * @dir: Direction of the event at the time of timestamp.
+ */
+struct hte_ts_data {
+	u64 tsc;
+	u64 seq;
+	enum hte_dir dir;
+};
+
+/**
+ * struct hte_clk_info - Clock source info that HTE provider uses to timestamp
+ * The provider uses hardware clock as a source to timestamp real time. This
+ * structure presents the clock information to consumers during
+ * hte_get_clk_src_info call.
+ *
+ * @hz: Supported clock rate in HZ, for example 1KHz clock = 1000.
+ * @type: Supported clock type. CLOCK_* types.
+ */
+struct hte_clk_info {
+	u64 hz;
+	clockid_t type;
+};
+
+/**
+ * enum hte_return- HTE subsystem return values used during callback.
+ *
+ * @HTE_CB_HANDLED: The consumer handled the data successfully.
+ * @HTE_RUN_THREADED_CB: The consumer needs further processing, in that case HTE
+ * subsystem will invoke kernel thread and call secondary callback provided by
+ * the consumer during devm_of_hte_request_ts and hte_req_ts_by_dt_node call.
+ * @HTE_CB_TS_DROPPED: The client returns when it can not store ts data.
+ * @HTE_CB_ERROR: The client returns error if anything goes wrong.
+ */
+enum hte_return {
+	HTE_CB_HANDLED,
+	HTE_RUN_THREADED_CB,
+	HTE_CB_TS_DROPPED,
+	HTE_CB_ERROR,
+};
+typedef enum hte_return hte_return_t;
+
+/**
+ * typedef hte_ts_cb_t - Callback provided during devm_of_hte_request_ts and
+ * hte_req_ts_by_dt_node APIs call.
+ *
+ * The callback is used to push timestamp data to client.
+ * @ts: HW timestamp data.
+ * @data: Client supplied data.
+ */
+typedef hte_return_t (*hte_ts_cb_t)(struct hte_ts_data *ts, void *data);
+
+/**
+ * typedef hte_ts_threaded_cb_t - Threaded callback provided during
+ * devm_of_hte_request_ts and hte_req_ts_by_dt_node APIs call.
+ *
+ * @data: Client supplied data.
+ *
+ * It will be called when client return HTE_RUN_THREADED_CB from hte_ts_cb_t.
+ * The callback will be called from thread context.
+ *
+ */
+typedef hte_return_t (*hte_ts_threaded_cb_t)(void *data);
+
+/**
+ * struct hte_ts_desc - HTE timestamp descriptor, this structure will be
+ * communication token between consumers to subsystem and subsystem to
+ * providers.
+ *
+ * @con_id: This is the same id sent in request APIs.
+ * @name: Descriptive name of the entity that is being monitored for the
+ * realtime timestamping. The consumer can set any name it likes. If null
+ * HTE core will construct name as ts_con_id. It will be the consumer's
+ * job to free any allocation if name is set by the consumer.
+ * @hte_data: Subsystem's private data relate to requested con_id.
+ */
+struct hte_ts_desc {
+	u32 con_id;
+	char *name;
+	void *hte_data;
+};
+
+/**
+ * struct hte_ops - HTE operations set by providers.
+ *
+ * @request: Hook for requesting a HTE timestamp. Returns 0 on success,
+ * non-zero for failures.
+ * @release: Hook for releasing a HTE timestamp. Returns 0 on success,
+ * non-zero for failures.
+ * @enable: Hook to enable the specified timestamp. Returns 0 on success,
+ * non-zero for failures.
+ * @disable: Hook to disable specified timestamp. Returns 0 on success,
+ * non-zero for failures.
+ * @get_clk_src_info: Hook to get the clock information the provider uses
+ * to timestamp. Returns 0 for success and negative error code for failure. On
+ * success HTE subsystem fills up provided struct hte_clk_info.
+ *
+ * xlated_id parameter is used to communicate between HTE subsystem and the
+ * providers. It is the same id returned during xlate API call and translated
+ * by the provider. This may be helpful as both subsystem and provider locate
+ * the requested entity in constant time, where entity could be anything from
+ * lines, signals, events, buses etc.. that providers support.
+ */
+struct hte_ops {
+	int (*request)(struct hte_chip *chip, u32 xlated_id);
+	int (*release)(struct hte_chip *chip, u32 xlated_id);
+	int (*enable)(struct hte_chip *chip, u32 xlated_id);
+	int (*disable)(struct hte_chip *chip, u32 xlated_id);
+	int (*get_clk_src_info)(struct hte_chip *chip,
+				struct hte_clk_info *ci);
+};
+
+/**
+ * struct hte_chip - Abstract HTE chip structure.
+ * @name: functional name of the HTE IP block.
+ * @dev: device providing the HTE.
+ * @ops: callbacks for this HTE.
+ * @nlines: number of lines/signals supported by this chip.
+ * @xlate: Callback which translates consumer supplied logical ids to
+ * physical ids, return from 0 for the success and negative for the
+ * failures. It stores (0 to @nlines) in xlated_id parameter for the success.
+ * @of_hte_n_cells: Number of cells used to form the HTE specifier.
+ * @gdev: HTE subsystem abstract device, internal to the HTE subsystem.
+ * @data: chip specific private data.
+ */
+struct hte_chip {
+	const char *name;
+	struct device *dev;
+	const struct hte_ops *ops;
+	u32 nlines;
+	int (*xlate)(struct hte_chip *gc,
+		     const struct of_phandle_args *args,
+		     struct hte_ts_desc *desc, u32 *xlated_id);
+	u8 of_hte_n_cells;
+
+	struct hte_device *gdev;
+	void *data;
+};
+
+#if IS_ENABLED(CONFIG_HTE)
+/* HTE APIs for the providers */
+int devm_hte_register_chip(struct hte_chip *chip);
+int hte_push_ts_ns(const struct hte_chip *chip, u32 xlated_id,
+		   struct hte_ts_data *data);
+
+/* HTE APIs for the consumers */
+
+int hte_release_ts(struct hte_ts_desc *desc);
+int devm_of_hte_request_ts(struct device *dev, struct hte_ts_desc *desc,
+			   hte_ts_cb_t cb, hte_ts_threaded_cb_t tcb,
+			   void *data);
+int hte_req_ts_by_hte_name(struct device *dev, const char *propname,
+			   struct hte_ts_desc *desc, hte_ts_cb_t cb,
+			   hte_ts_threaded_cb_t tcb, void *data);
+int hte_enable_ts(struct hte_ts_desc *desc);
+int hte_disable_ts(struct hte_ts_desc *desc);
+int hte_get_clk_src_info(const struct hte_ts_desc *desc,
+			 struct hte_clk_info *ci);
+
+#else /* !CONFIG_HTE */
+static inline int devm_hte_register_chip(struct hte_chip *chip)
+{
+	return -ENOTSUPP;
+}
+
+static inline int hte_push_ts_ns(const struct hte_chip *chip,
+				 u32 xlated_id,
+				 const struct hte_ts_data *data)
+{
+	return -ENOTSUPP;
+}
+
+static inline int hte_release_ts(struct hte_ts_desc *desc)
+{
+	return -ENOTSUPP;
+}
+
+static inline int devm_of_hte_request_ts(struct device *dev,
+					 struct hte_ts_desc *desc,
+					 hte_ts_cb_t cb,
+					 hte_ts_threaded_cb_t threaded_cb,
+					 void *data)
+{
+	return -ENOTSUPP;
+}
+
+static inline int hte_req_ts_by_hte_name(struct device *dev,
+					 const char *propname,
+					 struct hte_ts_desc *desc,
+					 hte_ts_cb_t cb,
+					 hte_ts_threaded_cb_t tcb, void *data)
+{
+	return -ENOTSUPP;
+}
+
+static inline int hte_enable_ts(struct hte_ts_desc *desc)
+{
+	return -ENOTSUPP;
+}
+
+static inline int hte_disable_ts(struct hte_ts_desc *desc)
+{
+	return -ENOTSUPP;
+}
+
+static inline int hte_get_clk_src_info(const struct hte_ts_desc *desc,
+				       struct hte_clk_info *ci)
+{
+	return -ENOTSUPP;
+}
+#endif /* !CONFIG_HTE */
+
+#endif
-- 
2.17.1

[RFC v3 03/12] hte: Add tegra194 HTE kernel provider

[Dipen Patel]

Tegra194 device has multiple HTE instances also known as GTE
(Generic hardware Timestamping Engine) which can timestamp subset of
SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
and exposes timestamping ability on those lines to the consumers
through HTE subsystem.

Also, with this patch, added:
- documentation about this provider and its capabilities at
Documentation/hte.
- Compilation support in Makefile and Kconfig

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
Changes in v3:
- Addressed grammatical/spelling errors.

 Documentation/hte/index.rst        |  22 ++
 Documentation/hte/tegra194-hte.rst |  57 +++
 Documentation/index.rst            |   1 +
 drivers/hte/Kconfig                |  12 +
 drivers/hte/Makefile               |   1 +
 drivers/hte/hte-tegra194.c         | 545 +++++++++++++++++++++++++++++
 6 files changed, 638 insertions(+)
 create mode 100644 Documentation/hte/index.rst
 create mode 100644 Documentation/hte/tegra194-hte.rst
 create mode 100644 drivers/hte/hte-tegra194.c

diff --git a/Documentation/hte/index.rst b/Documentation/hte/index.rst
new file mode 100644
index 000000000000..9f43301c05dc
--- /dev/null
+++ b/Documentation/hte/index.rst
@@ -0,0 +1,22 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+============================================
+The Linux Hardware Timestamping Engine (HTE)
+============================================
+
+The HTE Subsystem
+=================
+
+.. toctree::
+   :maxdepth: 1
+
+   hte
+
+HTE Tegra Provider
+==================
+
+.. toctree::
+   :maxdepth: 1
+
+   tegra194-hte
+
diff --git a/Documentation/hte/tegra194-hte.rst b/Documentation/hte/tegra194-hte.rst
new file mode 100644
index 000000000000..3bebcbac5847
--- /dev/null
+++ b/Documentation/hte/tegra194-hte.rst
@@ -0,0 +1,57 @@
+HTE Kernel provider driver
+==========================
+
+Description
+-----------
+The Nvidia tegra194 HTE provider driver implements two GTE
+(Generic Timestamping Engine) instances: 1) GPIO GTE and 2) LIC
+(Legacy Interrupt Controller) IRQ GTE. Both GTEs instances get the
+timestamp from the system counter TSC which has 31.25MHz clock rate, and the
+driver converts clock tick rate to nanoseconds before storing it as timestamp
+value.
+
+GPIO GTE
+--------
+
+This GTE instance timestamps GPIO in real time. For that to happen GPIO
+needs to be configured as input and IRQ needs to be enabled. The only always on
+(AON) GPIO controller instance supports timestamping GPIOs in real time and it
+has 39 GPIO lines. The GPIO GTE and AON GPIO controller are tightly coupled as
+it requires very specific bits to be set in GPIO config register before GPIO
+GTE can be used. The GPIO GTE functionality is accessed from the GPIOLIB
+framework for the in-kernel and userspace consumers. In the latter case,
+requests go through GPIOLIB CDEV framework. The below APIs are added in GPIOLIB
+framework to access HTE subsystem and GPIO GTE.
+
+.. kernel-doc:: drivers/gpio/gpiolib.c
+   :functions: gpiod_req_hw_timestamp_ns gpiod_rel_hw_timestamp_ns
+
+There is hte-tegra194-gpio-test.c, located in ``drivers/hte/`` directory, test
+driver which demonstrates above APIs for the Jetson AGX platform.
+
+For userspace consumers, GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE flag must be
+specified during IOCTL calls. Refer to ``tools/gpio/gpio-event-mon.c``, which
+returns the timestamp in nanoseconds.
+
+LIC (Legacy Interrupt Controller) IRQ GTE
+-----------------------------------------
+
+This GTE instance timestamps LIC IRQ lines in real time. There are 352 IRQ
+lines which this instance can add timestamps to in real time. The hte
+devicetree binding described at ``Documentation/devicetree/bindings/hte/``
+provides an example of how a consumer can request an IRQ line. Since it is a
+one-to-one mapping, consumers can simply specify the IRQ number that they are
+interested in. There is no userspace consumer support for this GTE instance in
+the hte framework. The sample test code hte-tegra194-irq-test.c, located in
+the ``drivers/hte/`` directory, demonstrates how to use an IRQ GTE instance.
+The below is sample device tree snippet code for the test driver::
+
+ tegra_hte_irq_test {
+        compatible = "nvidia,tegra194-hte-irq-test";
+        htes = <&tegra_hte_lic 0x19>;
+        hte-names = "hte-lic";
+ };
+
+The provider source code of both IRQ and GPIO GTE instances is located at
+``drivers/hte/hte-tegra194.c``.
+
diff --git a/Documentation/index.rst b/Documentation/index.rst
index 1b13c2445e87..b41118577fe6 100644
--- a/Documentation/index.rst
+++ b/Documentation/index.rst
@@ -138,6 +138,7 @@ needed).
    misc-devices/index
    scheduler/index
    mhi/index
+   hte/index
 
 Architecture-agnostic documentation
 -----------------------------------
diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
index 1fcfe17cf28a..ebd9817651c2 100644
--- a/drivers/hte/Kconfig
+++ b/drivers/hte/Kconfig
@@ -20,3 +20,15 @@ menuconfig HTE
 
 	  If unsure, say no.
 
+if HTE
+
+config HTE_TEGRA194
+	tristate "NVIDIA Tegra194 HTE Support"
+	depends on ARCH_TEGRA_194_SOC
+	help
+	  Enable this option for integrated hardware timestamping engine also
+	  known as generic timestamping engine (GTE) support on NVIDIA Tegra194
+	  systems-on-chip. The driver supports 352 LIC IRQs and 39 AON GPIOs
+	  lines for timestamping in realtime.
+
+endif
diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
index fc03bdf44427..3ae7c4029991 100644
--- a/drivers/hte/Makefile
+++ b/drivers/hte/Makefile
@@ -1,2 +1,3 @@
 obj-$(CONFIG_HTE)		+= hte.o
+obj-$(CONFIG_HTE_TEGRA194)	+= hte-tegra194.o
 
diff --git a/drivers/hte/hte-tegra194.c b/drivers/hte/hte-tegra194.c
new file mode 100644
index 000000000000..ffe9d0d20b94
--- /dev/null
+++ b/drivers/hte/hte-tegra194.c
@@ -0,0 +1,545 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 NVIDIA Corporation
+ *
+ * Author: Dipen Patel <dipenp@nvidia.com>
+ */
+
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/stat.h>
+#include <linux/interrupt.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/hte.h>
+#include <linux/uaccess.h>
+
+#define HTE_SUSPEND	0
+
+/* HTE source clock TSC is 31.25MHz */
+#define HTE_TS_CLK_RATE_HZ	31250000ULL
+#define HTE_CLK_RATE_NS		32
+#define HTE_TS_NS_SHIFT	__builtin_ctz(HTE_CLK_RATE_NS)
+
+#define NV_AON_SLICE_INVALID	-1
+#define NV_LINES_IN_SLICE	32
+
+/* AON HTE line map For slice 1 */
+#define NV_AON_HTE_SLICE1_IRQ_GPIO_28	12
+#define NV_AON_HTE_SLICE1_IRQ_GPIO_29	13
+
+/* AON HTE line map For slice 2 */
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_0	0
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_1	1
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_2	2
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_3	3
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_4	4
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_5	5
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_6	6
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_7	7
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_8	8
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_9	9
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_10	10
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_11	11
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_12	12
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_13	13
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_14	14
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_15	15
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_16	16
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_17	17
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_18	18
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_19	19
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_20	20
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_21	21
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_22	22
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_23	23
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_24	24
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_25	25
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_26	26
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_27	27
+
+/* AON GPIO port AA pins */
+#define NV_AON_GPIO_PORT_AA_0		0
+#define NV_AON_GPIO_PORT_AA_1		1
+#define NV_AON_GPIO_PORT_AA_2		2
+#define NV_AON_GPIO_PORT_AA_3		3
+#define NV_AON_GPIO_PORT_AA_4		4
+#define NV_AON_GPIO_PORT_AA_5		5
+#define NV_AON_GPIO_PORT_AA_6		6
+#define NV_AON_GPIO_PORT_AA_7		7
+
+/* AON GPIO port BB pins */
+#define NV_AON_GPIO_PORT_BB_0		8
+#define NV_AON_GPIO_PORT_BB_1		9
+#define NV_AON_GPIO_PORT_BB_2		10
+#define NV_AON_GPIO_PORT_BB_3		11
+
+/* AON GPIO port CC pins */
+#define NV_AON_GPIO_PORT_CC_0		12
+#define NV_AON_GPIO_PORT_CC_1		13
+#define NV_AON_GPIO_PORT_CC_2		14
+#define NV_AON_GPIO_PORT_CC_3		15
+#define NV_AON_GPIO_PORT_CC_4		16
+#define NV_AON_GPIO_PORT_CC_5		17
+#define NV_AON_GPIO_PORT_CC_6		18
+#define NV_AON_GPIO_PORT_CC_7		19
+
+/* AON GPIO port DD pins */
+#define NV_AON_GPIO_PORT_DD_0		20
+#define NV_AON_GPIO_PORT_DD_1		21
+#define NV_AON_GPIO_PORT_DD_2		22
+
+/* AON GPIO port EE pins */
+#define NV_AON_GPIO_PORT_EE_0		23
+#define NV_AON_GPIO_PORT_EE_1		24
+#define NV_AON_GPIO_PORT_EE_2		25
+#define NV_AON_GPIO_PORT_EE_3		26
+#define NV_AON_GPIO_PORT_EE_4		27
+#define NV_AON_GPIO_PORT_EE_5		28
+#define NV_AON_GPIO_PORT_EE_6		29
+
+
+#define HTE_TECTRL		0x0
+#define HTE_TETSCH		0x4
+#define HTE_TETSCL		0x8
+#define HTE_TESRC		0xC
+#define HTE_TECCV		0x10
+#define HTE_TEPCV		0x14
+#define HTE_TECMD		0x1C
+#define HTE_TESTATUS		0x20
+#define HTE_SLICE0_TETEN	0x40
+#define HTE_SLICE1_TETEN	0x60
+
+#define HTE_SLICE_SIZE		(HTE_SLICE1_TETEN - HTE_SLICE0_TETEN)
+
+#define HTE_TECTRL_ENABLE_ENABLE	0x1
+
+#define HTE_TECTRL_OCCU_SHIFT		0x8
+#define HTE_TECTRL_INTR_SHIFT		0x1
+#define HTE_TECTRL_INTR_ENABLE		0x1
+
+#define HTE_TESRC_SLICE_SHIFT		16
+#define HTE_TESRC_SLICE_DEFAULT_MASK	0xFF
+
+#define HTE_TECMD_CMD_POP		0x1
+
+#define HTE_TESTATUS_OCCUPANCY_SHIFT	8
+#define HTE_TESTATUS_OCCUPANCY_MASK	0xFF
+
+struct hte_slices {
+	u32 r_val;
+	unsigned long flags;
+	/* to prevent lines mapped to same slice updating its register */
+	spinlock_t s_lock;
+};
+
+struct tegra_hte_line_mapped {
+	int slice;
+	u32 bit_index;
+};
+
+struct tegra_hte_line_table {
+	u32 map_sz;
+	const struct tegra_hte_line_mapped *map;
+};
+
+struct tegra_hte_soc {
+	int hte_irq;
+	u32 itr_thrshld;
+	u32 conf_rval;
+	struct hte_slices *sl;
+	const struct tegra_hte_line_table *line_map;
+	struct hte_chip *chip;
+	void __iomem *regs;
+};
+
+static const struct tegra_hte_line_mapped tegra194_aon_gpio_map[] = {
+	/* gpio, slice, bit_index */
+	[NV_AON_GPIO_PORT_AA_0]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_11},
+	[NV_AON_GPIO_PORT_AA_1]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_10},
+	[NV_AON_GPIO_PORT_AA_2]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_9},
+	[NV_AON_GPIO_PORT_AA_3]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_8},
+	[NV_AON_GPIO_PORT_AA_4]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_7},
+	[NV_AON_GPIO_PORT_AA_5]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_6},
+	[NV_AON_GPIO_PORT_AA_6]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_5},
+	[NV_AON_GPIO_PORT_AA_7]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_4},
+	[NV_AON_GPIO_PORT_BB_0]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_3},
+	[NV_AON_GPIO_PORT_BB_1]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_2},
+	[NV_AON_GPIO_PORT_BB_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_1},
+	[NV_AON_GPIO_PORT_BB_3] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_0},
+	[NV_AON_GPIO_PORT_CC_0] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_22},
+	[NV_AON_GPIO_PORT_CC_1] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_21},
+	[NV_AON_GPIO_PORT_CC_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_20},
+	[NV_AON_GPIO_PORT_CC_3] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_19},
+	[NV_AON_GPIO_PORT_CC_4] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_18},
+	[NV_AON_GPIO_PORT_CC_5] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_17},
+	[NV_AON_GPIO_PORT_CC_6] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_16},
+	[NV_AON_GPIO_PORT_CC_7] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_15},
+	[NV_AON_GPIO_PORT_DD_0] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_14},
+	[NV_AON_GPIO_PORT_DD_1] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_13},
+	[NV_AON_GPIO_PORT_DD_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_12},
+	[NV_AON_GPIO_PORT_EE_0] = {1, NV_AON_HTE_SLICE1_IRQ_GPIO_29},
+	[NV_AON_GPIO_PORT_EE_1] = {1, NV_AON_HTE_SLICE1_IRQ_GPIO_28},
+	[NV_AON_GPIO_PORT_EE_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_27},
+	[NV_AON_GPIO_PORT_EE_3] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_26},
+	[NV_AON_GPIO_PORT_EE_4] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_25},
+	[NV_AON_GPIO_PORT_EE_5] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_24},
+	[NV_AON_GPIO_PORT_EE_6] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_23},
+};
+
+static const struct tegra_hte_line_table aon_hte_map = {
+	.map_sz = ARRAY_SIZE(tegra194_aon_gpio_map),
+	.map = tegra194_aon_gpio_map,
+};
+
+static inline u32 tegra_hte_readl(struct tegra_hte_soc *hte, u32 reg)
+{
+	return readl(hte->regs + reg);
+}
+
+static inline void tegra_hte_writel(struct tegra_hte_soc *hte, u32 reg,
+				    u32 val)
+{
+	writel(val, hte->regs + reg);
+}
+
+static inline int tegra_hte_map_to_line_id(u32 eid, struct tegra_hte_soc *gs,
+					  u32 *mapped)
+{
+	const struct tegra_hte_line_mapped *m;
+
+	if (gs->line_map) {
+		m = gs->line_map->map;
+		if (eid > gs->line_map->map_sz)
+			return -EINVAL;
+		if (m[eid].slice == NV_AON_SLICE_INVALID)
+			return -EINVAL;
+
+		*mapped = (m[eid].slice << 5) + m[eid].bit_index;
+	} else {
+		*mapped = eid;
+	}
+
+	return 0;
+}
+
+static int tegra_hte_line_xlate(struct hte_chip *gc,
+				 const struct of_phandle_args *args,
+				 struct hte_ts_desc *desc, u32 *xlated_id)
+{
+	int ret = 0;
+
+	if (!gc || !desc || !xlated_id)
+		return -EINVAL;
+
+	if (args) {
+		if (gc->of_hte_n_cells < 1)
+			return -EINVAL;
+
+		if (args->args_count != gc->of_hte_n_cells)
+			return -EINVAL;
+
+		desc->con_id = args->args[0];
+	}
+
+	ret = tegra_hte_map_to_line_id(desc->con_id, gc->data,
+				       xlated_id);
+	if (ret < 0) {
+		dev_dbg(gc->dev, "con_id:%u mapping failed\n",
+			desc->con_id);
+		return ret;
+	}
+
+	if (*xlated_id > gc->nlines)
+		return -EINVAL;
+
+	dev_dbg(gc->dev, "requested id:%u, xlated id:%u\n",
+		desc->con_id, *xlated_id);
+
+	return 0;
+}
+
+static int tegra_hte_en_dis_common(struct hte_chip *chip, u32 line_id, bool en)
+{
+	u32 slice, sl_bit_shift, line_bit, val, reg;
+	struct tegra_hte_soc *gs;
+
+	sl_bit_shift = __builtin_ctz(HTE_SLICE_SIZE);
+
+	if (!chip)
+		return -EINVAL;
+
+	gs = (struct tegra_hte_soc *)chip->data;
+
+	if (line_id > chip->nlines) {
+		dev_err(chip->dev,
+			"line id: %u is not supported by this controller\n",
+			line_id);
+		return -EINVAL;
+	}
+
+	slice = line_id >> sl_bit_shift;
+	line_bit = line_id & (HTE_SLICE_SIZE - 1);
+	reg = (slice << sl_bit_shift) + HTE_SLICE0_TETEN;
+
+	spin_lock(&gs->sl[slice].s_lock);
+
+	if (test_bit(HTE_SUSPEND, &gs->sl[slice].flags)) {
+		spin_unlock(&gs->sl[slice].s_lock);
+		dev_dbg(chip->dev, "device suspended");
+		return -EBUSY;
+	}
+
+	val = tegra_hte_readl(gs, reg);
+	if (en)
+		val = val | (1 << line_bit);
+	else
+		val = val & (~(1 << line_bit));
+	tegra_hte_writel(gs, reg, val);
+
+	spin_unlock(&gs->sl[slice].s_lock);
+
+	dev_dbg(chip->dev, "line: %u, slice %u, line_bit %u, reg:0x%x\n",
+		line_id, slice, line_bit, reg);
+
+	return 0;
+}
+
+static int tegra_hte_request(struct hte_chip *chip, u32 line_id)
+{
+	return tegra_hte_en_dis_common(chip, line_id, true);
+}
+
+static int tegra_hte_release(struct hte_chip *chip, u32 line_id)
+{
+	return tegra_hte_en_dis_common(chip, line_id, false);
+}
+
+static int tegra_hte_clk_src_info(struct hte_chip *chip,
+				  struct hte_clk_info *ci)
+{
+	(void)chip;
+
+	if (!ci)
+		return -EINVAL;
+
+	ci->hz = HTE_TS_CLK_RATE_HZ;
+	ci->type = CLOCK_MONOTONIC;
+
+	return 0;
+}
+
+static void tegra_hte_read_fifo(struct tegra_hte_soc *gs)
+{
+	u32 tsh, tsl, src, pv, cv, acv, slice, bit_index, line_id;
+	u64 tsc;
+	struct hte_ts_data el;
+
+	while ((tegra_hte_readl(gs, HTE_TESTATUS) >>
+		HTE_TESTATUS_OCCUPANCY_SHIFT) &
+		HTE_TESTATUS_OCCUPANCY_MASK) {
+		tsh = tegra_hte_readl(gs, HTE_TETSCH);
+		tsl = tegra_hte_readl(gs, HTE_TETSCL);
+		tsc = (((u64)tsh << 32) | tsl);
+
+		src = tegra_hte_readl(gs, HTE_TESRC);
+		slice = (src >> HTE_TESRC_SLICE_SHIFT) &
+			    HTE_TESRC_SLICE_DEFAULT_MASK;
+
+		pv = tegra_hte_readl(gs, HTE_TEPCV);
+		cv = tegra_hte_readl(gs, HTE_TECCV);
+		acv = pv ^ cv;
+		while (acv) {
+			bit_index = __builtin_ctz(acv);
+			el.dir = HTE_DIR_NOSUPP;
+			line_id = bit_index + (slice << 5);
+			el.tsc = tsc << HTE_TS_NS_SHIFT;
+			hte_push_ts_ns(gs->chip, line_id, &el);
+			acv &= ~BIT(bit_index);
+		}
+		tegra_hte_writel(gs, HTE_TECMD, HTE_TECMD_CMD_POP);
+	}
+}
+
+static irqreturn_t tegra_hte_isr(int irq, void *dev_id)
+{
+	struct tegra_hte_soc *gs = dev_id;
+	(void)irq;
+
+	tegra_hte_read_fifo(gs);
+
+	return IRQ_HANDLED;
+}
+
+static const struct of_device_id tegra_hte_of_match[] = {
+	{ .compatible = "nvidia,tegra194-gte-lic"},
+	{ .compatible = "nvidia,tegra194-gte-aon", .data = &aon_hte_map},
+	{ }
+};
+MODULE_DEVICE_TABLE(of, tegra_hte_of_match);
+
+static const struct hte_ops g_ops = {
+	.request = tegra_hte_request,
+	.release = tegra_hte_release,
+	.enable = tegra_hte_request,
+	.disable = tegra_hte_release,
+	.get_clk_src_info = tegra_hte_clk_src_info,
+};
+
+static void tegra_gte_disable(void *data)
+{
+	struct platform_device *pdev = data;
+	struct tegra_hte_soc *gs = dev_get_drvdata(&pdev->dev);
+
+	tegra_hte_writel(gs, HTE_TECTRL, 0);
+}
+
+static int tegra_hte_probe(struct platform_device *pdev)
+{
+	int ret;
+	u32 i, slices, val = 0;
+	struct device *dev;
+	struct tegra_hte_soc *hte_dev;
+	struct hte_chip *gc;
+
+	dev = &pdev->dev;
+
+	hte_dev = devm_kzalloc(dev, sizeof(*hte_dev), GFP_KERNEL);
+	if (!hte_dev)
+		return -ENOMEM;
+
+	gc = devm_kzalloc(dev, sizeof(*gc), GFP_KERNEL);
+	if (!gc)
+		return -ENOMEM;
+
+	dev_set_drvdata(&pdev->dev, hte_dev);
+	hte_dev->line_map = of_device_get_match_data(&pdev->dev);
+
+	hte_dev->regs = devm_platform_ioremap_resource(pdev, 0);
+	if (IS_ERR(hte_dev->regs))
+		return PTR_ERR(hte_dev->regs);
+
+	ret = of_property_read_u32(dev->of_node, "nvidia,int-threshold",
+				   &hte_dev->itr_thrshld);
+	if (ret != 0)
+		hte_dev->itr_thrshld = 1;
+
+	ret = of_property_read_u32(dev->of_node, "nvidia,slices", &slices);
+	if (ret != 0) {
+		dev_err(dev, "Could not read slices\n");
+		return -EINVAL;
+	}
+
+	hte_dev->sl = devm_kcalloc(dev, slices, sizeof(*hte_dev->sl),
+				   GFP_KERNEL);
+	if (!hte_dev->sl)
+		return -ENOMEM;
+
+	ret = platform_get_irq(pdev, 0);
+	if (ret < 0) {
+		dev_err_probe(dev, ret, "failed to get irq\n");
+		return ret;
+	}
+	hte_dev->hte_irq = ret;
+	ret = devm_request_irq(dev, hte_dev->hte_irq, tegra_hte_isr, 0,
+			       dev_name(dev), hte_dev);
+	if (ret < 0) {
+		dev_err(dev, "request irq failed.\n");
+		return ret;
+	}
+
+	gc->nlines = slices << 5;
+	gc->ops = &g_ops;
+	gc->dev = dev;
+	hte_dev->chip = gc;
+	gc->data = hte_dev;
+	gc->xlate = tegra_hte_line_xlate;
+	gc->of_hte_n_cells = 1;
+
+	ret = devm_hte_register_chip(hte_dev->chip);
+	if (ret) {
+		dev_err(gc->dev, "hte chip register failed");
+		return ret;
+	}
+
+	for (i = 0; i < slices; i++) {
+		hte_dev->sl[i].flags = 0;
+		spin_lock_init(&hte_dev->sl[i].s_lock);
+	}
+
+	val = HTE_TECTRL_ENABLE_ENABLE |
+	      (HTE_TECTRL_INTR_ENABLE << HTE_TECTRL_INTR_SHIFT) |
+	      (hte_dev->itr_thrshld << HTE_TECTRL_OCCU_SHIFT);
+	tegra_hte_writel(hte_dev, HTE_TECTRL, val);
+
+	ret = devm_add_action_or_reset(&pdev->dev, tegra_gte_disable, pdev);
+	if (ret)
+		return ret;
+
+	dev_dbg(gc->dev, "lines: %d, slices:%d", gc->nlines, slices);
+
+	return 0;
+}
+
+static int __maybe_unused tegra_hte_resume_early(struct device *dev)
+{
+	u32 i;
+	struct tegra_hte_soc *gs = dev_get_drvdata(dev);
+	u32 slices = gs->chip->nlines / NV_LINES_IN_SLICE;
+	u32 sl_bit_shift = __builtin_ctz(HTE_SLICE_SIZE);
+
+	tegra_hte_writel(gs, HTE_TECTRL, gs->conf_rval);
+
+	for (i = 0; i < slices; i++) {
+		spin_lock(&gs->sl[i].s_lock);
+		tegra_hte_writel(gs,
+				 ((i << sl_bit_shift) + HTE_SLICE0_TETEN),
+				 gs->sl[i].r_val);
+		clear_bit(HTE_SUSPEND, &gs->sl[i].flags);
+		spin_unlock(&gs->sl[i].s_lock);
+	}
+
+	return 0;
+}
+
+static int __maybe_unused tegra_hte_suspend_late(struct device *dev)
+{
+	u32 i;
+	struct tegra_hte_soc *gs = dev_get_drvdata(dev);
+	u32 slices = gs->chip->nlines / NV_LINES_IN_SLICE;
+	u32 sl_bit_shift = __builtin_ctz(HTE_SLICE_SIZE);
+
+	gs->conf_rval = tegra_hte_readl(gs, HTE_TECTRL);
+	for (i = 0; i < slices; i++) {
+		spin_lock(&gs->sl[i].s_lock);
+		gs->sl[i].r_val = tegra_hte_readl(gs,
+				((i << sl_bit_shift) + HTE_SLICE0_TETEN));
+		set_bit(HTE_SUSPEND, &gs->sl[i].flags);
+		spin_unlock(&gs->sl[i].s_lock);
+	}
+
+	return 0;
+}
+
+static const struct dev_pm_ops tegra_hte_pm = {
+	SET_LATE_SYSTEM_SLEEP_PM_OPS(tegra_hte_suspend_late,
+				     tegra_hte_resume_early)
+};
+
+static struct platform_driver tegra_hte_driver = {
+	.probe = tegra_hte_probe,
+	.driver = {
+		.name = "tegra_hte",
+		.pm = &tegra_hte_pm,
+		.of_match_table = tegra_hte_of_match,
+	},
+};
+
+module_platform_driver(tegra_hte_driver);
+
+MODULE_AUTHOR("Dipen Patel <dipenp@nvidia.com>");
+MODULE_DESCRIPTION("NVIDIA Tegra HTE (Hardware Timestamping Engine) driver");
+MODULE_LICENSE("GPL v2");
-- 
2.17.1

[RFC v3 04/12] dt-bindings: Add HTE bindings

[Dipen Patel]

Introduces HTE devicetree binding details for the HTE subsystem. It
includes examples for the consumers, binding details for the providers
and specific binding details for the Tegra194 based HTE providers.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
---
Changes in v2:
- Replace hte with hardware-timestamp for property names
- Renamed file
- Removed example from the common dt binding file.

Changes in v3:
- Addressed grammatical errors.
- Removed double plural from the respective properties.
- Added dual license.
- Prefixed "nvidia" in nvidia specific properties. 

 .../hte/hardware-timestamps-common.yaml       | 29 +++++++
 .../devicetree/bindings/hte/hte-consumer.yaml | 48 +++++++++++
 .../bindings/hte/nvidia,tegra194-hte.yaml     | 80 +++++++++++++++++++
 3 files changed, 157 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/hte/hardware-timestamps-common.yaml
 create mode 100644 Documentation/devicetree/bindings/hte/hte-consumer.yaml
 create mode 100644 Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml

diff --git a/Documentation/devicetree/bindings/hte/hardware-timestamps-common.yaml b/Documentation/devicetree/bindings/hte/hardware-timestamps-common.yaml
new file mode 100644
index 000000000000..ee6f94890695
--- /dev/null
+++ b/Documentation/devicetree/bindings/hte/hardware-timestamps-common.yaml
@@ -0,0 +1,29 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/hte/hardware-timestamps-common.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Hardware timestamp providers
+
+maintainers:
+  - Dipen Patel <dipenp@nvidia.com>
+
+description: |
+  Some devices/SoCs have hardware time stamping engines which can use hardware
+  means to timestamp entity in realtime. The entity could be anything from
+  GPIOs, IRQs, Bus and so on. The hardware timestamp engine (HTE) present
+  itself as a provider with the bindings described in this document.
+
+properties:
+  $nodename:
+    pattern: "^hardware-timestamp(@.*|-[0-9a-f])?$"
+
+  "#hardware-timestamp-cells":
+    description:
+      Number of cells in a HTE specifier.
+
+required:
+  - "#hardware-timestamp-cells"
+
+additionalProperties: true
diff --git a/Documentation/devicetree/bindings/hte/hte-consumer.yaml b/Documentation/devicetree/bindings/hte/hte-consumer.yaml
new file mode 100644
index 000000000000..75914e1c8ee1
--- /dev/null
+++ b/Documentation/devicetree/bindings/hte/hte-consumer.yaml
@@ -0,0 +1,48 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/hte/hte-consumer.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: HTE Consumer Device Tree Bindings
+
+maintainers:
+  - Dipen Patel <dipenp@nvidia.com>
+
+description: |
+  HTE properties should be named "hardware-timestamps". The exact meaning of
+  each hardware-timestamps property must be documented in the device tree
+  binding for each device. An optional property "hardware-timestamp-names" may
+  contain a list of strings to label each of the HTE devices listed in the
+  "hardware-timestamps" property.
+
+  The "hardware-timestamp-names" property if specified is used to map the name
+  of the HTE device requested by the devm_of_hte_request_ts() call to an index
+  into the list given by the "hardware-timestamps" property.
+
+properties:
+  hardware-timestamps:
+    $ref: /schemas/types.yaml#/definitions/phandle-array
+    description:
+      The list of HTE provider phandle. The provider must document the number
+      of cell that must be passed in this property along with phandle.
+
+  hardware-timestamp-names:
+    $ref: /schemas/types.yaml#/definitions/string-array
+    description:
+      An optional string property.
+
+required:
+  - hardware-timestamps
+
+dependencies:
+  hardware-timestamp-names: [ hardware-timestamps ]
+
+additionalProperties: true
+
+examples:
+  - |
+    hte_irq_consumer {
+              hardware-timestamps = <&tegra_hte_lic 0x19>;
+              hardware-timestamp-names = "hte-irq";
+    };
diff --git a/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml b/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
new file mode 100644
index 000000000000..675d4bb6ad3b
--- /dev/null
+++ b/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
@@ -0,0 +1,80 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/hte/nvidia,tegra194-hte.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Tegra194 on chip generic hardware timestamping engine (HTE)
+
+maintainers:
+  - Dipen Patel <dipenp@nvidia.com>
+
+description: |
+  Tegra194 SoC has multiple generic hardware timestamping engines which can
+  monitor subset of GPIO and on chip IRQ lines for the state change, upon
+  detection it will record timestamp (taken from system counter) in its
+  internal hardware FIFO. It has a bitmap array arranged in 32bit slices where
+  each bit represent signal/line to enable or disable for the hardware
+  timestamping.
+
+properties:
+  compatible:
+    enum:
+      - nvidia,tegra194-gte-aon
+      - nvidia,tegra194-gte-lic
+
+  reg:
+    maxItems: 1
+
+  interrupts:
+    maxItems: 1
+
+  nvidia,int-threshold:
+    description:
+      HTE device generates its interrupt based on this u32 FIFO threshold
+      value. The recommended value is 1.
+    minimum: 1
+    maximum: 256
+
+  nvidia,slices:
+    type: int
+    description:
+      HTE lines are arranged in 32 bit slice where each bit represents different
+      line/signal that it can enable/configure for the timestamp. It is u32
+      property and depends on the HTE instance in the chip.
+    enum: [3, 11]
+
+  '#hardware-timestamp-cells':
+    const: 1
+
+required:
+  - compatible
+  - reg
+  - interrupts
+  - nvidia,slices
+  - "#hardware-timestamp-cells"
+
+additionalProperties: false
+
+examples:
+  - |
+    tegra_hte_aon: hardware-timestamp@c1e0000 {
+              compatible = "nvidia,tegra194-gte-aon";
+              reg = <0xc1e0000 0x10000>;
+              interrupts = <0 13 0x4>;
+              nvidia,int-threshold = <1>;
+              nvidia,slices = <3>;
+              #hardware-timestamp-cells = <1>;
+    };
+
+  - |
+    tegra_hte_lic: hardware-timestamp@3aa0000 {
+              compatible = "nvidia,tegra194-gte-lic";
+              reg = <0x3aa0000 0x10000>;
+              interrupts = <0 11 0x4>;
+              nvidia,int-threshold = <1>;
+              nvidia,slices = <11>;
+              #hardware-timestamp-cells = <1>;
+    };
+
+...
-- 
2.17.1

[RFC v3 05/12] hte: Add Tegra194 IRQ HTE test driver

[Dipen Patel]

Tegra194 has IRQ HTE provider which can timestamp IRQ lines in realtime
, this test driver implements consumer side which tests such provider
through HTE subsystem. During its probe, it registers sysfs interface
to easily navigate from userspace as below.

All the files are at /sys/kernel/tegra_hte_irq_test/.

- en_dis - Write only, Value 1 enables HTE line, 0 disables it

Its devicetree detail can be accessed from
Documentation/hte/tegra194-hte.rst.

This driver can be compiled as loadable module and is tested on Jetson
AGX platform using 0x19 IRQ line which belongs to i2c controller
3160000.i2c.

i2cdetect -y 1 from the userspace on this platform should be enough to
generate LIC I2C IRQ. The HTE should be able to generate
timestamps in realtime for each interrupts.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
 drivers/hte/Kconfig                 |   7 ++
 drivers/hte/Makefile                |   1 +
 drivers/hte/hte-tegra194-irq-test.c | 169 ++++++++++++++++++++++++++++
 3 files changed, 177 insertions(+)
 create mode 100644 drivers/hte/hte-tegra194-irq-test.c

diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
index ebd9817651c2..dee8b7a2b980 100644
--- a/drivers/hte/Kconfig
+++ b/drivers/hte/Kconfig
@@ -31,4 +31,11 @@ config HTE_TEGRA194
 	  systems-on-chip. The driver supports 352 LIC IRQs and 39 AON GPIOs
 	  lines for timestamping in realtime.
 
+config HTE_TEGRA194_IRQ_TEST
+        tristate "NVIDIA Tegra194 HTE LIC IRQ Test"
+        depends on HTE_TEGRA194
+        help
+	  The NVIDIA Tegra194 GTE IRQ test driver demonstrates HTE subsystem
+	  usage for the LIC IRQ hardware timestamp.
+
 endif
diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
index 3ae7c4029991..75b7932c2ffc 100644
--- a/drivers/hte/Makefile
+++ b/drivers/hte/Makefile
@@ -1,3 +1,4 @@
 obj-$(CONFIG_HTE)		+= hte.o
 obj-$(CONFIG_HTE_TEGRA194)	+= hte-tegra194.o
+obj-$(CONFIG_HTE_TEGRA194_IRQ_TEST) += hte-tegra194-irq-test.o
 
diff --git a/drivers/hte/hte-tegra194-irq-test.c b/drivers/hte/hte-tegra194-irq-test.c
new file mode 100644
index 000000000000..e43991321172
--- /dev/null
+++ b/drivers/hte/hte-tegra194-irq-test.c
@@ -0,0 +1,169 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 NVIDIA Corporation
+ *
+ * Author: Dipen Patel <dipenp@nvidia.com>
+ */
+
+#include <linux/version.h>
+#include <linux/err.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/interrupt.h>
+#include <linux/hte.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/workqueue.h>
+
+/*
+ * Tegra194 On chip HTE (hardware timestamping engine) also known as GTE
+ * (generic timestamping engine) can monitor LIC (Legacy interrupt controller)
+ * IRQ lines for the event and timestamp accordingly in realtime. Follow
+ * technical reference manual for the IRQ numbers and descriptions.
+ *
+ * This sample HTE IRQ test driver demonstrating HTE API usage by enabling
+ * lic irq line in HTE to monitor and timestamp.
+ */
+
+static struct tegra_hte_test {
+	struct hte_ts_desc desc;
+	struct kobject *kobj;
+	struct device *pdev;
+} hte;
+
+static hte_return_t process_hw_ts(struct hte_ts_data *ts, void *p)
+{
+	char *edge;
+	(void)p;
+
+	if (!ts)
+		return HTE_CB_ERROR;
+
+	switch (ts->dir) {
+	case HTE_FALLING_EDGE_TS:
+		edge = "falling";
+		break;
+	case HTE_RISING_EDGE_TS:
+		edge = "rising";
+		break;
+	default:
+		edge = "unknown";
+		break;
+	}
+
+	dev_info(hte.pdev, "IRQ HW timestamp(%llu): %llu, edge: %s\n",
+		 ts->seq, ts->tsc, edge);
+
+	return HTE_CB_HANDLED;
+}
+
+/*
+ * Sysfs attribute to request/release HTE IRQ line.
+ */
+static ssize_t store_en_dis(struct kobject *kobj,
+			    struct kobj_attribute *attr,
+			    const char *buf, size_t count)
+{
+	int ret = count;
+	unsigned long val = 0;
+	struct hte_clk_info ci;
+	(void)kobj;
+	(void)attr;
+
+	if (kstrtoul(buf, 10, &val) < 0) {
+		ret = -EINVAL;
+		goto error;
+	}
+
+	if (val == 1) {
+		ret = devm_of_hte_request_ts(hte.pdev, &hte.desc,
+					     process_hw_ts, NULL, NULL);
+		if (ret)
+			goto error;
+
+		hte_get_clk_src_info(&hte.desc, &ci);
+		dev_info(hte.pdev, "clk rate:%llu, clk type: %d\n",
+			 ci.hz, ci.type);
+	} else if (val == 0) {
+		ret = hte_release_ts(&hte.desc);
+		if (ret)
+			goto error;
+	}
+
+	ret = count;
+
+error:
+	return ret;
+}
+
+struct kobj_attribute en_dis_attr =
+		__ATTR(en_dis, 0220, NULL, store_en_dis);
+
+static struct attribute *attrs[] = {
+	&en_dis_attr.attr,
+	NULL,
+};
+
+static struct attribute_group tegra_hte_test_attr_group = {
+	.attrs = attrs,
+};
+
+static int tegra_hte_test_sysfs_create(void)
+{
+	int ret;
+
+	/* Creates /sys/kernel/tegra_hte_irq_test */
+	hte.kobj = kobject_create_and_add("tegra_hte_irq_test", kernel_kobj);
+	if (!hte.kobj)
+		return -ENOMEM;
+
+	ret = sysfs_create_group(hte.kobj, &tegra_hte_test_attr_group);
+	if (ret)
+		kobject_put(hte.kobj);
+
+	return ret;
+}
+
+static const struct of_device_id tegra_hte_irq_test_of_match[] = {
+	{ .compatible = "nvidia,tegra194-hte-irq-test"},
+	{ }
+};
+MODULE_DEVICE_TABLE(of, tegra_hte_irq_test_of_match);
+
+static int tegra_hte_test_probe(struct platform_device *pdev)
+{
+	int ret;
+
+	dev_set_drvdata(&pdev->dev, &hte);
+	hte.pdev = &pdev->dev;
+
+	ret = tegra_hte_test_sysfs_create();
+	if (ret != 0) {
+		dev_err(hte.pdev, "sysfs creation failed\n");
+		return -ENXIO;
+	}
+
+	return 0;
+}
+
+static int tegra_hte_test_remove(struct platform_device *pdev)
+{
+	(void)pdev;
+
+	kobject_put(hte.kobj);
+
+	return 0;
+}
+
+static struct platform_driver tegra_hte_irq_test_driver = {
+	.probe = tegra_hte_test_probe,
+	.remove = tegra_hte_test_remove,
+	.driver = {
+		.name = "tegra_hte_irq_test",
+		.of_match_table = tegra_hte_irq_test_of_match,
+	},
+};
+module_platform_driver(tegra_hte_irq_test_driver);
+
+MODULE_AUTHOR("Dipen Patel <dipenp@nvidia.com>");
+MODULE_LICENSE("GPL v2");
-- 
2.17.1

[RFC v3 06/12] gpiolib: Add HTE support

[Dipen Patel]

Some GPIO chip can provide hardware timestamp support on its GPIO lines
, in order to support that additional API needs to be added which
can talk to both GPIO chip and HTE (hardware timestamping engine)
subsystem. This patch introduces APIs which gpio consumer can use
to request hardware assisted timestamping. Below is the list of the APIs
that are added in gpiolib subsystem.

- gpiod_req_hw_timestamp_ns - Request HTE on specified GPIO line.
- gpiod_rel_hw_timestamp_ns - Release HTE functionality on GPIO line.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Reported-by: kernel test robot <lkp@intel.com>
---
Changes in v2:
- removed get timestamp and is timestamp enabled APIs

 drivers/gpio/gpiolib.c        | 73 +++++++++++++++++++++++++++++++++++
 drivers/gpio/gpiolib.h        | 12 ++++++
 include/linux/gpio/consumer.h | 19 ++++++++-
 include/linux/gpio/driver.h   | 14 +++++++
 4 files changed, 116 insertions(+), 2 deletions(-)

diff --git a/drivers/gpio/gpiolib.c b/drivers/gpio/gpiolib.c
index abfbf546d159..46cba75c80e8 100644
--- a/drivers/gpio/gpiolib.c
+++ b/drivers/gpio/gpiolib.c
@@ -1976,6 +1976,10 @@ static bool gpiod_free_commit(struct gpio_desc *desc)
 			gc->free(gc, gpio_chip_hwgpio(desc));
 			spin_lock_irqsave(&gpio_lock, flags);
 		}
+		spin_unlock_irqrestore(&gpio_lock, flags);
+		gpiod_rel_hw_timestamp_ns(desc);
+		spin_lock_irqsave(&gpio_lock, flags);
+
 		kfree_const(desc->label);
 		desc_set_label(desc, NULL);
 		clear_bit(FLAG_ACTIVE_LOW, &desc->flags);
@@ -2388,6 +2392,75 @@ int gpiod_direction_output(struct gpio_desc *desc, int value)
 }
 EXPORT_SYMBOL_GPL(gpiod_direction_output);
 
+/**
+ * gpiod_req_hw_timestamp_ns - Enable the hardware assisted timestamp in
+ * nano second.
+ *
+ * @desc: GPIO to enable
+ * @cb:	Callback, will be called when HTE pushes timestamp data.
+ * @tcb: Threaeded callback, it gets called from kernel thread context and when
+ * cb returns with HTE_RUN_THREADED_CB return value.
+ * @data: Client data, will be sent back with tcb and cb.
+ *
+ * Certain GPIO chip can rely on hardware assisted timestamp engines which can
+ * record timestamp at the occurance of the configured events
+ * i.e. rising/falling on specified GPIO lines. This is helper API to enable hw
+ * assisted timestamp in nano second.
+ *
+ * Return 0 in case of success, else an error code.
+ */
+int gpiod_req_hw_timestamp_ns(struct gpio_desc *desc, hte_ts_cb_t cb,
+			      hte_ts_threaded_cb_t tcb, void *data)
+{
+	struct gpio_chip *gc;
+	int ret = 0;
+
+	VALIDATE_DESC(desc);
+	gc = desc->gdev->chip;
+
+	if (!gc->req_hw_timestamp) {
+		gpiod_warn(desc, "%s: hw ts not supported\n", __func__);
+		return -ENOTSUPP;
+	}
+
+	ret = gc->req_hw_timestamp(gc, gpio_chip_hwgpio(desc), cb, tcb,
+				   &desc->hdesc, data);
+	if (ret)
+		gpiod_warn(desc, "%s: hw ts request failed\n", __func__);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(gpiod_req_hw_timestamp_ns);
+
+/**
+ * gpiod_rel_hw_timestamp_ns - Release and disable the hardware assisted
+ * timestamp.
+ *
+ * @desc: GPIO to disable
+ *
+ * Return 0 in case of success, else an error code.
+ */
+int gpiod_rel_hw_timestamp_ns(struct gpio_desc *desc)
+{
+	struct gpio_chip *gc;
+	int ret = 0;
+
+	VALIDATE_DESC(desc);
+	gc = desc->gdev->chip;
+
+	if (!gc->rel_hw_timestamp) {
+		gpiod_warn(desc, "%s: hw ts not supported\n", __func__);
+		return -ENOTSUPP;
+	}
+
+	ret = gc->rel_hw_timestamp(gc, gpio_chip_hwgpio(desc), &desc->hdesc);
+	if (ret)
+		gpiod_warn(desc, "%s: hw ts release failed\n", __func__);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(gpiod_rel_hw_timestamp_ns);
+
 /**
  * gpiod_set_config - sets @config for a GPIO
  * @desc: descriptor of the GPIO for which to set the configuration
diff --git a/drivers/gpio/gpiolib.h b/drivers/gpio/gpiolib.h
index 30bc3f80f83e..f634b9de3756 100644
--- a/drivers/gpio/gpiolib.h
+++ b/drivers/gpio/gpiolib.h
@@ -15,6 +15,7 @@
 #include <linux/device.h>
 #include <linux/module.h>
 #include <linux/cdev.h>
+#include <linux/hte.h>
 
 #define GPIOCHIP_NAME	"gpiochip"
 
@@ -117,6 +118,7 @@ struct gpio_desc {
 #define FLAG_EDGE_RISING     16	/* GPIO CDEV detects rising edge events */
 #define FLAG_EDGE_FALLING    17	/* GPIO CDEV detects falling edge events */
 #define FLAG_EVENT_CLOCK_REALTIME	18 /* GPIO CDEV reports REALTIME timestamps in events */
+#define FLAG_EVENT_CLOCK_HARDWARE	19 /* GPIO CDEV reports hardware timestamps in events */
 
 	/* Connection label */
 	const char		*label;
@@ -129,6 +131,16 @@ struct gpio_desc {
 	/* debounce period in microseconds */
 	unsigned int		debounce_period_us;
 #endif
+	/*
+	 * Hardware timestamp engine related internal data structure.
+	 * This gets filled out when the consumer calls
+	 * gpiod_req_hw_timestamp_ns to enable hardware timestamping on the
+	 * specified GPIO line. The API calls into HTE subsystem which
+	 * initializes appropriate field of the hdesc. The hdesc will be later
+	 * used with gpiod_rel_hw_timestamp to release hw timestamp
+	 * functionality.
+	 */
+	struct hte_ts_desc hdesc;
 };
 
 #define gpiod_not_found(desc)		(IS_ERR(desc) && PTR_ERR(desc) == -ENOENT)
diff --git a/include/linux/gpio/consumer.h b/include/linux/gpio/consumer.h
index 3ad67b4a72be..680867e66fb6 100644
--- a/include/linux/gpio/consumer.h
+++ b/include/linux/gpio/consumer.h
@@ -6,6 +6,7 @@
 #include <linux/bug.h>
 #include <linux/compiler_types.h>
 #include <linux/err.h>
+#include <linux/hte.h>
 
 struct device;
 
@@ -112,6 +113,9 @@ int gpiod_get_direction(struct gpio_desc *desc);
 int gpiod_direction_input(struct gpio_desc *desc);
 int gpiod_direction_output(struct gpio_desc *desc, int value);
 int gpiod_direction_output_raw(struct gpio_desc *desc, int value);
+int gpiod_req_hw_timestamp_ns(struct gpio_desc *desc, hte_ts_cb_t cb,
+			      hte_ts_threaded_cb_t tcb, void *data);
+int gpiod_rel_hw_timestamp_ns(struct gpio_desc *desc);
 
 /* Value get/set from non-sleeping context */
 int gpiod_get_value(const struct gpio_desc *desc);
@@ -353,8 +357,19 @@ static inline int gpiod_direction_output_raw(struct gpio_desc *desc, int value)
 	WARN_ON(desc);
 	return -ENOSYS;
 }
-
-
+static inline int gpiod_req_hw_timestamp_ns(struct gpio_desc *desc,
+					    hte_ts_cb_t cb,
+					    hte_ts_threaded_cb_t tcb,
+					    void *data)
+{
+	WARN_ON(desc);
+	return -ENOSYS;
+}
+static inline int gpiod_rel_hw_timestamp_ns(struct gpio_desc *desc)
+{
+	WARN_ON(desc);
+	return -ENOSYS;
+}
 static inline int gpiod_get_value(const struct gpio_desc *desc)
 {
 	/* GPIO can never have been requested */
diff --git a/include/linux/gpio/driver.h b/include/linux/gpio/driver.h
index a673a359e20b..fda5500ea60a 100644
--- a/include/linux/gpio/driver.h
+++ b/include/linux/gpio/driver.h
@@ -10,6 +10,7 @@
 #include <linux/lockdep.h>
 #include <linux/pinctrl/pinctrl.h>
 #include <linux/pinctrl/pinconf-generic.h>
+#include <linux/hte.h> /* For hardware timestamping */
 
 struct gpio_desc;
 struct of_phandle_args;
@@ -319,6 +320,10 @@ struct gpio_irq_chip {
  * @add_pin_ranges: optional routine to initialize pin ranges, to be used when
  *	requires special mapping of the pins that provides GPIO functionality.
  *	It is called after adding GPIO chip and before adding IRQ chip.
+ * @req_hw_timestamp: Dependent on GPIO chip, an optional routine to
+ *	enable hardware assisted timestamp.
+ * @rel_hw_timestamp: Dependent on GPIO chip, an optional routine to
+ *	disable/release hardware assisted timestamp.
  * @base: identifies the first GPIO number handled by this chip;
  *	or, if negative during registration, requests dynamic ID allocation.
  *	DEPRECATION: providing anything non-negative and nailing the base
@@ -414,6 +419,15 @@ struct gpio_chip {
 
 	int			(*add_pin_ranges)(struct gpio_chip *gc);
 
+	int			(*req_hw_timestamp)(struct gpio_chip *gc,
+						    unsigned int offset,
+						    hte_ts_cb_t cb,
+						    hte_ts_threaded_cb_t tcb,
+						    struct hte_ts_desc *hdesc,
+						    void *data);
+	int			(*rel_hw_timestamp)(struct gpio_chip *chip,
+						    unsigned int offset,
+						    struct hte_ts_desc *hdesc);
 	int			base;
 	u16			ngpio;
 	u16			offset;
-- 
2.17.1

[RFC v3 07/12] dt-bindings: gpio: Add hardware-timestamp-engine property

[Dipen Patel]

Some GPIO controllers can timestamp GPIO lines in real time using
hardware timestamp engine. The optional property is added to facilitate
this feature.

The nvidia GPIO controller has hardware timestamp engine as a backend
to timestamp its GPIO lines in a real time. This change set adds
hardware-timestamp-engine property to reflect that fact.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
 Documentation/devicetree/bindings/gpio/gpio.txt           | 8 ++++++++
 .../devicetree/bindings/gpio/nvidia,tegra186-gpio.txt     | 7 +++++++
 2 files changed, 15 insertions(+)

diff --git a/Documentation/devicetree/bindings/gpio/gpio.txt b/Documentation/devicetree/bindings/gpio/gpio.txt
index a8895d339bfe..c6ae2eaf5762 100644
--- a/Documentation/devicetree/bindings/gpio/gpio.txt
+++ b/Documentation/devicetree/bindings/gpio/gpio.txt
@@ -167,6 +167,13 @@ left to right from the passed array. An incomplete array (where the number
 of passed named are less than ngpios) will still be used up until the last
 provided valid line index.
 
+Optionally, a GPIO controller may have a "hardware-timestamp-engine" property.
+This specifies the timestamp engine node. The property is only useful if the
+hardware timestamp engine (HTE) serves as a back-end to support hardware
+timestamp GPIO lines in a real time and GPIO controller has dependency on the
+engine. It will be up to GPIO controller and HTE provider drivers to figure out
+the mapping between GPIO controller and HTE namespaces of a given GPIO line.
+
 Example:
 
 gpio-controller@00000000 {
@@ -180,6 +187,7 @@ gpio-controller@00000000 {
 		"LED G", "LED B", "Col A", "Col B", "Col C", "Col D",
 		"Row A", "Row B", "Row C", "Row D", "NMI button",
 		"poweroff", "reset";
+	hardware-timestamp-engine = <&hadrware_ts_engine_node>;
 }
 
 The GPIO chip may contain GPIO hog definitions. GPIO hogging is a mechanism
diff --git a/Documentation/devicetree/bindings/gpio/nvidia,tegra186-gpio.txt b/Documentation/devicetree/bindings/gpio/nvidia,tegra186-gpio.txt
index adff16c71d21..20f6c9e69839 100644
--- a/Documentation/devicetree/bindings/gpio/nvidia,tegra186-gpio.txt
+++ b/Documentation/devicetree/bindings/gpio/nvidia,tegra186-gpio.txt
@@ -127,6 +127,12 @@ Required properties:
             - 8: Active low level-sensitive.
             Valid combinations are 1, 2, 3, 4, 8.
 
+Optional properties:
+- hardware-timestamp-engine
+    AON GPIO controller has timestamp engine which can hardware timestamp
+    GPIO configured as input and IRQ. This property specifies hardware
+    timestamp engine (HTE) device-tree node.
+
 Example:
 
 #include <dt-bindings/interrupt-controller/irq.h>
@@ -162,4 +168,5 @@ gpio@c2f0000 {
 	#gpio-cells = <2>;
 	interrupt-controller;
 	#interrupt-cells = <2>;
+	hardware-timestamp-engine = <&tegra_hte_aon>;
 };
-- 
2.17.1

[RFC v3 08/12] gpio: tegra186: Add HTE in gpio-tegra186 driver

[Dipen Patel]

Tegra194 AON GPIO controller with the use of its internal hardware
timestamping engine (HTE) also known as GTE can timestamp GPIO
lines through system counter. This patch implements two callbacks
which are essential for the gpio consumers that want such HTE
functionality. The callbacks details can be found at
include/gpio/driver.h.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
 drivers/gpio/gpio-tegra186.c | 89 ++++++++++++++++++++++++++++++++++++
 1 file changed, 89 insertions(+)

diff --git a/drivers/gpio/gpio-tegra186.c b/drivers/gpio/gpio-tegra186.c
index c026e7141e4e..53b7daebb89f 100644
--- a/drivers/gpio/gpio-tegra186.c
+++ b/drivers/gpio/gpio-tegra186.c
@@ -11,6 +11,7 @@
 #include <linux/module.h>
 #include <linux/of_device.h>
 #include <linux/platform_device.h>
+#include <linux/hte.h>
 
 #include <dt-bindings/gpio/tegra186-gpio.h>
 #include <dt-bindings/gpio/tegra194-gpio.h>
@@ -34,6 +35,7 @@
 #define  TEGRA186_GPIO_ENABLE_CONFIG_TRIGGER_LEVEL BIT(4)
 #define  TEGRA186_GPIO_ENABLE_CONFIG_DEBOUNCE BIT(5)
 #define  TEGRA186_GPIO_ENABLE_CONFIG_INTERRUPT BIT(6)
+#define  TEGRA186_GPIO_ENABLE_CONFIG_TIMESTAMP_FUNC BIT(7)
 
 #define TEGRA186_GPIO_DEBOUNCE_CONTROL 0x04
 #define  TEGRA186_GPIO_DEBOUNCE_CONTROL_THRESHOLD(x) ((x) & 0xff)
@@ -81,6 +83,7 @@ struct tegra_gpio {
 	struct irq_chip intc;
 	unsigned int num_irq;
 	unsigned int *irq;
+	struct device *dev;
 
 	const struct tegra_gpio_soc *soc;
 	unsigned int num_irqs_per_bank;
@@ -192,6 +195,86 @@ static int tegra186_gpio_direction_output(struct gpio_chip *chip,
 	return 0;
 }
 
+static int tegra186_gpio_req_hw_ts(struct gpio_chip *chip, unsigned int offset,
+				   hte_ts_cb_t cb, hte_ts_threaded_cb_t tcb,
+				   struct hte_ts_desc *hdesc, void *data)
+{
+	struct tegra_gpio *gpio;
+	void __iomem *base;
+	int value, ret;
+
+	if (!chip || !hdesc)
+		return -EINVAL;
+
+	gpio = gpiochip_get_data(chip);
+	if (!gpio)
+		return -ENODEV;
+
+	base = tegra186_gpio_get_base(gpio, offset);
+	if (WARN_ON(base == NULL))
+		return -EINVAL;
+
+	/*
+	 * HTE provider of this gpio controller does not support below gpio
+	 * configs:
+	 * 1. gpio as output
+	 * 2. gpio as input
+	 *
+	 * HTE provider supports below gpio config:
+	 * a. gpio as input with irq enabled
+	 */
+
+	if (tegra186_gpio_get_direction(chip, offset) ==
+	    GPIO_LINE_DIRECTION_OUT)
+		return -ENOTSUPP;
+
+	if (!gpiochip_line_is_irq(chip, offset))
+		return -ENOTSUPP;
+
+	hdesc->con_id = offset;
+
+	ret = hte_req_ts_by_hte_name(gpio->dev, "hardware-timestamp-engine",
+				     hdesc, cb, tcb, data);
+	if (ret)
+		return ret;
+
+	value = readl(base + TEGRA186_GPIO_ENABLE_CONFIG);
+	value |= TEGRA186_GPIO_ENABLE_CONFIG_TIMESTAMP_FUNC;
+	writel(value, base + TEGRA186_GPIO_ENABLE_CONFIG);
+
+	return 0;
+}
+
+static int tegra186_gpio_rel_hw_ts(struct gpio_chip *chip,
+				   unsigned int offset,
+				   struct hte_ts_desc *hdesc)
+{
+	struct tegra_gpio *gpio;
+	void __iomem *base;
+	int value, ret;
+
+	if (!hdesc || !chip)
+		return -EINVAL;
+
+	gpio = gpiochip_get_data(chip);
+	if (!gpio)
+		return -ENODEV;
+
+	base = tegra186_gpio_get_base(gpio, offset);
+	if (WARN_ON(base == NULL))
+		return -EINVAL;
+
+	ret = hte_release_ts(hdesc);
+	if (ret)
+		return ret;
+
+	value = readl(base + TEGRA186_GPIO_ENABLE_CONFIG);
+	value &= ~TEGRA186_GPIO_ENABLE_CONFIG_TIMESTAMP_FUNC;
+	writel(value, base + TEGRA186_GPIO_ENABLE_CONFIG);
+
+	return 0;
+}
+
 static int tegra186_gpio_get(struct gpio_chip *chip, unsigned int offset)
 {
 	struct tegra_gpio *gpio = gpiochip_get_data(chip);
@@ -821,6 +904,12 @@ static int tegra186_gpio_probe(struct platform_device *pdev)
 		offset += port->pins;
 	}
 
+	gpio->dev = &pdev->dev;
+	if (device_property_present(gpio->dev, "hardware-timestamp-engine")) {
+		gpio->gpio.req_hw_timestamp = tegra186_gpio_req_hw_ts;
+		gpio->gpio.rel_hw_timestamp = tegra186_gpio_rel_hw_ts;
+	}
+
 	return devm_gpiochip_add_data(&pdev->dev, &gpio->gpio, gpio);
 }
 
-- 
2.17.1

[RFC v3 09/12] gpiolib: cdev: Add hardware timestamp clock type

[Dipen Patel]

This patch adds new clock type for the GPIO controller which can
timestamp gpio lines in realtime using hardware means. To expose such
functionalities to the userspace, code has been added in this patch
where during line create call, it checks for new clock type and if
requested, calls hardware timestamp related API from gpiolib.c.
During line change event, the HTE subsystem pushes timestamp data
through callbacks.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
---
Changes in v2:
- Added hte_dir and static structure hte_ts_desc.
- Added callbacks which get invoked by HTE when new data is available.
- Better use of hte_dir and seq from hte_ts_desc.
- Modified sw debounce function to accommodate hardware timestamping.

 drivers/gpio/gpiolib-cdev.c | 161 ++++++++++++++++++++++++++++++++++--
 include/uapi/linux/gpio.h   |   1 +
 2 files changed, 153 insertions(+), 9 deletions(-)

diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
index c7b5446d01fd..1736ad54e3ec 100644
--- a/drivers/gpio/gpiolib-cdev.c
+++ b/drivers/gpio/gpiolib-cdev.c
@@ -464,6 +464,12 @@ struct line {
 	 * stale value.
 	 */
 	unsigned int level;
+	/*
+	 * dir will be touched in HTE callbacks hte_ts_cb_t and
+	 * hte_ts_threaded_cb_t and they are mutually exclusive. This will be
+	 * unused when HTE is not supported/disabled.
+	 */
+	enum hte_dir dir;
 };
 
 /**
@@ -518,6 +524,7 @@ struct linereq {
 	 GPIO_V2_LINE_DRIVE_FLAGS | \
 	 GPIO_V2_LINE_EDGE_FLAGS | \
 	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
+	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
 	 GPIO_V2_LINE_BIAS_FLAGS)
 
 static void linereq_put_event(struct linereq *lr,
@@ -546,6 +553,94 @@ static u64 line_event_timestamp(struct line *line)
 	return ktime_get_ns();
 }
 
+static hte_return_t process_hw_ts_thread(void *p)
+{
+	struct line *line = p;
+	struct linereq *lr = line->req;
+	struct gpio_v2_line_event le;
+	u64 eflags;
+
+	memset(&le, 0, sizeof(le));
+
+	le.timestamp_ns = line->timestamp_ns;
+	line->timestamp_ns = 0;
+
+	if (line->dir >= HTE_DIR_NOSUPP) {
+		eflags = READ_ONCE(line->eflags);
+		if (eflags == GPIO_V2_LINE_FLAG_EDGE_BOTH) {
+			int level = gpiod_get_value_cansleep(line->desc);
+
+			if (level)
+				/* Emit low-to-high event */
+				le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
+			else
+				/* Emit high-to-low event */
+				le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
+		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_RISING) {
+			/* Emit low-to-high event */
+			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
+		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_FALLING) {
+			/* Emit high-to-low event */
+			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
+		} else {
+			return HTE_CB_ERROR;
+		}
+	} else {
+		if (line->dir == HTE_RISING_EDGE_TS)
+			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
+		else
+			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
+	}
+
+	le.line_seqno = line->line_seqno;
+	le.seqno = (lr->num_lines == 1) ? le.line_seqno : line->req_seqno;
+	le.offset = gpio_chip_hwgpio(line->desc);
+
+	linereq_put_event(lr, &le);
+
+	return HTE_CB_HANDLED;
+}
+
+static hte_return_t process_hw_ts(struct hte_ts_data *ts, void *p)
+{
+	struct line *line = p;
+	struct linereq *lr = line->req;
+
+	if (!ts)
+		return HTE_CB_ERROR;
+
+	line->timestamp_ns = ts->tsc;
+	line->dir = ts->dir;
+
+	/*
+	 * It is possible that HTE engine detects spurious edges for the
+	 * lines where software debounce is enabled. This primary callback
+	 * will be called multiple times in that case. It will be better to
+	 * let debounce_work_func handle instead of process_hw_ts_thread.
+	 * The timestamp_ns will be overwritten here which is fine as we are
+	 * interested in the last value anyway. The debounce_work_func will
+	 * then just read whatever last line->timestamp_ns is stored. Because
+	 * this callback can be called multiple times, we are not really
+	 * interested in ts->seq.
+	 */
+	if (!READ_ONCE(line->sw_debounced)) {
+		line->line_seqno = ts->seq;
+
+		/*
+		 * Increment in this callback incase all the lines in linereq
+		 * are enabled for hw timestamping. This will work even if
+		 * subset of lines are enabled for hw timestamping as
+		 * edge_irq_* callbacks will proceed as usual for them.
+		 */
+		if (lr->num_lines != 1)
+			line->req_seqno = atomic_inc_return(&lr->seqno);
+
+		return HTE_RUN_THREADED_CB;
+	}
+
+	return HTE_CB_HANDLED;
+}
+
 static irqreturn_t edge_irq_thread(int irq, void *p)
 {
 	struct line *line = p;
@@ -553,6 +648,10 @@ static irqreturn_t edge_irq_thread(int irq, void *p)
 	struct gpio_v2_line_event le;
 	u64 eflags;
 
+	/* Let process_hw_ts_thread handle */
+	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
+		return IRQ_HANDLED;
+
 	/* Do not leak kernel stack to userspace */
 	memset(&le, 0, sizeof(le));
 
@@ -604,6 +703,10 @@ static irqreturn_t edge_irq_handler(int irq, void *p)
 	struct line *line = p;
 	struct linereq *lr = line->req;
 
+	/* Let HTE supplied callbacks handle */
+	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
+		return IRQ_HANDLED;
+
 	/*
 	 * Just store the timestamp in hardirq context so we get it as
 	 * close in time as possible to the actual event.
@@ -682,14 +785,6 @@ static void debounce_work_func(struct work_struct *work)
 	/* Do not leak kernel stack to userspace */
 	memset(&le, 0, sizeof(le));
 
-	lr = line->req;
-	le.timestamp_ns = line_event_timestamp(line);
-	le.offset = gpio_chip_hwgpio(line->desc);
-	line->line_seqno++;
-	le.line_seqno = line->line_seqno;
-	le.seqno = (lr->num_lines == 1) ?
-		le.line_seqno : atomic_inc_return(&lr->seqno);
-
 	if (level)
 		/* Emit low-to-high event */
 		le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
@@ -697,6 +792,23 @@ static void debounce_work_func(struct work_struct *work)
 		/* Emit high-to-low event */
 		le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
 
+	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
+		le.timestamp_ns = line->timestamp_ns;
+		if (line->dir < HTE_DIR_NOSUPP)
+			le.id = (line->dir == HTE_RISING_EDGE_TS) ?
+				 GPIO_V2_LINE_EVENT_RISING_EDGE :
+				 GPIO_V2_LINE_EVENT_FALLING_EDGE;
+	} else {
+		le.timestamp_ns = line_event_timestamp(line);
+	}
+
+	lr = line->req;
+	le.offset = gpio_chip_hwgpio(line->desc);
+	line->line_seqno++;
+	le.line_seqno = line->line_seqno;
+	le.seqno = (lr->num_lines == 1) ?
+		le.line_seqno : atomic_inc_return(&lr->seqno);
+
 	linereq_put_event(lr, &le);
 }
 
@@ -891,7 +1003,6 @@ static int gpio_v2_line_flags_validate(u64 flags)
 	/* Return an error if an unknown flag is set */
 	if (flags & ~GPIO_V2_LINE_VALID_FLAGS)
 		return -EINVAL;
-
 	/*
 	 * Do not allow both INPUT and OUTPUT flags to be set as they are
 	 * contradictory.
@@ -900,6 +1011,11 @@ static int gpio_v2_line_flags_validate(u64 flags)
 	    (flags & GPIO_V2_LINE_FLAG_OUTPUT))
 		return -EINVAL;
 
+	/* Only allow one event clock source */
+	if ((flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME) &&
+	    (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE))
+		return -EINVAL;
+
 	/* Edge detection requires explicit input. */
 	if ((flags & GPIO_V2_LINE_EDGE_FLAGS) &&
 	    !(flags & GPIO_V2_LINE_FLAG_INPUT))
@@ -992,6 +1108,8 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 flags,
 
 	assign_bit(FLAG_EVENT_CLOCK_REALTIME, flagsp,
 		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME);
+	assign_bit(FLAG_EVENT_CLOCK_HARDWARE, flagsp,
+		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE);
 }
 
 static long linereq_get_values(struct linereq *lr, void __user *ip)
@@ -1154,6 +1272,21 @@ static long linereq_set_config_unlocked(struct linereq *lr,
 				return ret;
 		}
 
+		/* Check if new config sets hardware assisted clock */
+		if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
+			ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
+							process_hw_ts_thread,
+							&lr->lines[i]);
+			if (ret)
+				return ret;
+		} else {
+			/*
+			 * HTE subsys will do nothing if there is nothing to
+			 * release.
+			 */
+			gpiod_rel_hw_timestamp_ns(desc);
+		}
+
 		blocking_notifier_call_chain(&desc->gdev->notifier,
 					     GPIO_V2_LINE_CHANGED_CONFIG,
 					     desc);
@@ -1409,6 +1542,14 @@ static int linereq_create(struct gpio_device *gdev, void __user *ip)
 					flags & GPIO_V2_LINE_EDGE_FLAGS);
 			if (ret)
 				goto out_free_linereq;
+
+			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
+				ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
+							process_hw_ts_thread,
+							&lr->lines[i]);
+				if (ret)
+					goto out_free_linereq;
+			}
 		}
 
 		blocking_notifier_call_chain(&desc->gdev->notifier,
@@ -1959,6 +2100,8 @@ static void gpio_desc_to_lineinfo(struct gpio_desc *desc,
 
 	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &desc->flags))
 		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
+	else if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &desc->flags))
+		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
 
 	debounce_period_us = READ_ONCE(desc->debounce_period_us);
 	if (debounce_period_us) {
diff --git a/include/uapi/linux/gpio.h b/include/uapi/linux/gpio.h
index eaaea3d8e6b4..d360545b4c21 100644
--- a/include/uapi/linux/gpio.h
+++ b/include/uapi/linux/gpio.h
@@ -80,6 +80,7 @@ enum gpio_v2_line_flag {
 	GPIO_V2_LINE_FLAG_BIAS_PULL_DOWN	= _BITULL(9),
 	GPIO_V2_LINE_FLAG_BIAS_DISABLED		= _BITULL(10),
 	GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME	= _BITULL(11),
+	GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE	= _BITULL(12),
 };
 
 /**
-- 
2.17.1

[RFC v3 10/12] tools: gpio: Add new hardware clock type

[Dipen Patel]

gpiolib-cdev is extended to support hardware clock type, this
patch reflects that fact.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
---
 tools/gpio/gpio-event-mon.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/tools/gpio/gpio-event-mon.c b/tools/gpio/gpio-event-mon.c
index a2b233fdb572..bed52333698d 100644
--- a/tools/gpio/gpio-event-mon.c
+++ b/tools/gpio/gpio-event-mon.c
@@ -149,6 +149,7 @@ void print_usage(void)
 		"  -r         Listen for rising edges\n"
 		"  -f         Listen for falling edges\n"
 		"  -w         Report the wall-clock time for events\n"
+		"  -t         Report the hardware timestamp for events\n"
 		"  -b <n>     Debounce the line with period n microseconds\n"
 		" [-c <n>]    Do <n> loops (optional, infinite loop if not stated)\n"
 		"  -?         This helptext\n"
@@ -174,7 +175,7 @@ int main(int argc, char **argv)
 
 	memset(&config, 0, sizeof(config));
 	config.flags = GPIO_V2_LINE_FLAG_INPUT;
-	while ((c = getopt(argc, argv, "c:n:o:b:dsrfw?")) != -1) {
+	while ((c = getopt(argc, argv, "c:n:o:b:dsrfwt?")) != -1) {
 		switch (c) {
 		case 'c':
 			loops = strtoul(optarg, NULL, 10);
@@ -208,6 +209,9 @@ int main(int argc, char **argv)
 		case 'w':
 			config.flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
 			break;
+		case 't':
+			config.flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
+			break;
 		case '?':
 			print_usage();
 			return -1;
-- 
2.17.1

[RFC v3 11/12] hte: Add tegra GPIO HTE test driver

[Dipen Patel]

Tegra194 GPIO controller and HTE supports AON GPIO lines for realtime
timestamp using hardware means. This in kernel gpio consumer test
driver demonstrates that functionality indirectly using HTE subsytem
through GPIOLIB framework. During probe it also registers sysfs
interface /sys/kernel/tegra_hte_gpio_test/gpio_en_dis. The value 1
enables gpio line for the HTE functionality while the value 0
disables that.

The test driver can be compiled as a module and takes optional
parameters gpio_out and gpio_in of type uint that specifies GPIO
numbers.

This patch also adds compilation support in Kconfig and Makefile.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
Changes in v3:
- Addressed grammatical errors.

 drivers/hte/Kconfig                  |   9 +
 drivers/hte/Makefile                 |   1 +
 drivers/hte/hte-tegra194-gpio-test.c | 252 +++++++++++++++++++++++++++
 3 files changed, 262 insertions(+)
 create mode 100644 drivers/hte/hte-tegra194-gpio-test.c

diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
index dee8b7a2b980..f20f38ae172b 100644
--- a/drivers/hte/Kconfig
+++ b/drivers/hte/Kconfig
@@ -38,4 +38,13 @@ config HTE_TEGRA194_IRQ_TEST
 	  The NVIDIA Tegra194 GTE IRQ test driver demonstrates HTE subsystem
 	  usage for the LIC IRQ hardware timestamp.
 
+config HTE_TEGRA194_GPIO_TEST
+        tristate "NVIDIA Tegra194 HTE GPIO Test"
+        depends on HTE_TEGRA194
+        help
+	  The NVIDIA Tegra194 GTE GPIO test driver demonstrates how to use the HTE
+	  subsystem indirectly through gpiolib API calls for GPIO lines for the
+	  hardware-assisted timestamping.
+
 endif
+
diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
index 75b7932c2ffc..1d12f3a36709 100644
--- a/drivers/hte/Makefile
+++ b/drivers/hte/Makefile
@@ -1,4 +1,5 @@
 obj-$(CONFIG_HTE)		+= hte.o
 obj-$(CONFIG_HTE_TEGRA194)	+= hte-tegra194.o
 obj-$(CONFIG_HTE_TEGRA194_IRQ_TEST) += hte-tegra194-irq-test.o
+obj-$(CONFIG_HTE_TEGRA194_GPIO_TEST) += hte-tegra194-gpio-test.o
 
diff --git a/drivers/hte/hte-tegra194-gpio-test.c b/drivers/hte/hte-tegra194-gpio-test.c
new file mode 100644
index 000000000000..418f7df7b1b8
--- /dev/null
+++ b/drivers/hte/hte-tegra194-gpio-test.c
@@ -0,0 +1,252 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 NVIDIA Corporation
+ *
+ * Author: Dipen Patel <dipenp@nvidia.com>
+ */
+
+#include <linux/version.h>
+#include <linux/err.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/interrupt.h>
+#include <linux/gpio.h>
+#include <linux/timer.h>
+#include <linux/platform_device.h>
+#include <linux/workqueue.h>
+
+/*
+ * Tegra194 On chip HTE (hardware timestamping engine) also known as GTE
+ * (generic timestamping engine) can monitor subset of GPIO lines for the event
+ * and timestamp accordingly.
+ *
+ * This sample HTE GPIO test driver demonstrates HTE API usage indirectly
+ * through GPIOLIB framework. It enables hardware timestamp on gpio_in line.
+ * Also the gpio_in is configured to be a rising edge interrupt.
+ *
+ * Note: gpio_out and gpio_in need to be shorted externally in order for this
+ * test driver to work for the GPIO monitoring. The test driver has been
+ * tested on Jetson AGX platform by shorting pin 32 and 16 on 40 pin header.
+ */
+
+static unsigned int gpio_in = 322;
+module_param(gpio_in, uint, 0660);
+
+static unsigned int gpio_out = 321;
+module_param(gpio_out, uint, 0660);
+
+static struct tegra_hte_test {
+	bool is_ts_en;
+	int gpio_in_irq;
+	struct gpio_desc *gpio_in;
+	struct gpio_desc *gpio_out;
+	struct timer_list timer;
+	struct kobject *kobj;
+} hte;
+
+static hte_return_t process_hw_ts(struct hte_ts_data *ts, void *p)
+{
+	char *edge;
+	(void)p;
+
+	if (!ts)
+		return HTE_CB_ERROR;
+
+	if (ts->dir >= HTE_DIR_NOSUPP)
+		edge = "rising";
+
+	pr_info("GPIO HW timestamp(%llu): %llu, edge: %s\n", ts->seq, ts->tsc,
+		(ts->dir < HTE_DIR_NOSUPP) ? ((ts->dir == HTE_FALLING_EDGE_TS) ?
+					      "falling" : "rising") : edge);
+
+	return HTE_CB_HANDLED;
+}
+
+/*
+ * Sysfs attribute to request/release HTE gpio line
+ */
+static ssize_t store_gpio_en_dis(struct kobject *kobj,
+				 struct kobj_attribute *attr,
+				 const char *buf, size_t count)
+{
+	int ret = count;
+	unsigned long val = 0;
+	(void)kobj;
+	(void)attr;
+
+	if (kstrtoul(buf, 10, &val) < 0) {
+		ret = -EINVAL;
+		goto error;
+	}
+
+	if (val == 1) {
+		if (hte.is_ts_en) {
+			ret = -EEXIST;
+			goto error;
+		}
+
+		ret = gpiod_req_hw_timestamp_ns(hte.gpio_in, process_hw_ts,
+						NULL, NULL);
+		if (ret)
+			goto error;
+
+		hte.is_ts_en = true;
+	} else if (val == 0) {
+		if (!hte.is_ts_en) {
+			ret = -EINVAL;
+			goto error;
+		}
+		ret = gpiod_rel_hw_timestamp_ns(hte.gpio_in);
+		if (ret)
+			goto error;
+
+		hte.is_ts_en = false;
+	}
+
+	ret = count;
+
+error:
+	return ret;
+}
+
+struct kobj_attribute gpio_en_dis_attr =
+		__ATTR(gpio_en_dis, 0220, NULL, store_gpio_en_dis);
+
+static struct attribute *attrs[] = {
+	&gpio_en_dis_attr.attr,
+	NULL,
+};
+
+static struct attribute_group tegra_hte_test_attr_group = {
+	.attrs = attrs,
+};
+
+static int tegra_hte_test_sysfs_create(void)
+{
+	int ret;
+
+	/* Creates /sys/kernel/tegra_hte_gpio_test */
+	hte.kobj = kobject_create_and_add("tegra_hte_gpio_test", kernel_kobj);
+	if (!hte.kobj)
+		return -ENOMEM;
+
+	ret = sysfs_create_group(hte.kobj, &tegra_hte_test_attr_group);
+	if (ret)
+		kobject_put(hte.kobj);
+	return ret;
+}
+
+static void gpio_timer_cb(struct timer_list *t)
+{
+	(void)t;
+
+	gpiod_set_value(hte.gpio_out, !gpiod_get_value(hte.gpio_out));
+	mod_timer(&hte.timer, jiffies + msecs_to_jiffies(8000));
+}
+
+static irqreturn_t tegra_hte_test_gpio_isr(int irq, void *data)
+{
+	(void)irq;
+	(void)data;
+
+	return IRQ_HANDLED;
+}
+
+static int __init tegra_hte_gpio_test_init(void)
+{
+	int ret = 0;
+
+	ret = gpio_request(gpio_out, "gte_test_gpio_out");
+	if (ret) {
+		pr_err("failed request gpio out\n");
+		return -EINVAL;
+	}
+
+	ret = gpio_request(gpio_in, "gte_test_gpio_in");
+	if (ret) {
+		pr_err("failed request gpio in\n");
+		ret = -EINVAL;
+		goto free_gpio_out;
+	}
+
+	hte.gpio_out = gpio_to_desc(gpio_out);
+	if (!hte.gpio_out) {
+		pr_err("failed convert gpio out to desc\n");
+		ret = -EINVAL;
+		goto free_gpio_in;
+	}
+
+	hte.gpio_in = gpio_to_desc(gpio_in);
+	if (!hte.gpio_in) {
+		pr_err("failed convert gpio in to desc\n");
+		ret = -EINVAL;
+		goto free_gpio_in;
+	}
+
+	ret = gpiod_direction_output(hte.gpio_out, 0);
+	if (ret) {
+		pr_err("failed to set output\n");
+		ret = -EINVAL;
+		goto free_gpio_in;
+	}
+
+	ret = gpiod_direction_input(hte.gpio_in);
+	if (ret) {
+		pr_err("failed to set input\n");
+		ret = -EINVAL;
+		goto free_gpio_in;
+	}
+
+	ret = gpiod_to_irq(hte.gpio_in);
+	if (ret < 0) {
+		pr_err("failed to map GPIO to IRQ: %d\n", ret);
+		ret = -ENXIO;
+		goto free_gpio_in;
+	}
+
+	hte.gpio_in_irq = ret;
+
+	ret = request_irq(ret, tegra_hte_test_gpio_isr,
+			  IRQF_TRIGGER_RISING,
+			  "tegra_hte_gpio_test_isr", &hte);
+	if (ret) {
+		pr_err("failed to acquire IRQ\n");
+		ret = -ENXIO;
+		goto free_irq;
+	}
+
+	ret = tegra_hte_test_sysfs_create();
+	if (ret != 0) {
+		pr_err("sysfs creation failed\n");
+		ret = -ENXIO;
+		goto free_irq;
+	}
+
+	timer_setup(&hte.timer, gpio_timer_cb, 0);
+	mod_timer(&hte.timer, jiffies + msecs_to_jiffies(5000));
+
+	return 0;
+
+free_irq:
+	free_irq(hte.gpio_in_irq, &hte);
+free_gpio_in:
+	gpio_free(gpio_in);
+free_gpio_out:
+	gpio_free(gpio_out);
+
+	return ret;
+}
+
+static void __exit tegra_hte_gpio_test_exit(void)
+{
+	free_irq(hte.gpio_in_irq, &hte);
+	gpio_free(gpio_in);
+	gpio_free(gpio_out);
+	kobject_put(hte.kobj);
+	del_timer(&hte.timer);
+}
+
+module_init(tegra_hte_gpio_test_init);
+module_exit(tegra_hte_gpio_test_exit);
+MODULE_AUTHOR("Dipen Patel <dipenp@nvidia.com>");
+MODULE_LICENSE("GPL v2");
-- 
2.17.1

[RFC v3 12/12] MAINTAINERS: Added HTE Subsystem

[Dipen Patel]

Added myself as a maintainer for this new Hardware Timestamping Engine
(HTE) subsystem.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
Changes in v3:
- Followed guidelines for the "M" field.
- Removed "*" in "F" field.

 MAINTAINERS | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index f32c7d733255..7296ae6110cd 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8759,6 +8759,14 @@ L:	linux-input@vger.kernel.org
 S:	Maintained
 F:	drivers/input/touchscreen/htcpen.c
 
+HTE SUBSYSTEM
+M:	Dipen Patel <dipenp@nvidia.com>
+S:	Maintained
+F:	drivers/hte/
+F:	include/linux/hte.h
+F:	Documentation/hte/
+F:	Documentation/devicetree/bindings/hte/
+
 HTS221 TEMPERATURE-HUMIDITY IIO DRIVER
 M:	Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
 L:	linux-iio@vger.kernel.org
-- 
2.17.1

Re: [RFC v3 03/12] hte: Add tegra194 HTE kernel provider

[Randy Dunlap]

Hi--

On 11/23/21 11:30 AM, Dipen Patel wrote:
> Tegra194 device has multiple HTE instances also known as GTE
> (Generic hardware Timestamping Engine) which can timestamp subset of
> SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
> and exposes timestamping ability on those lines to the consumers
> through HTE subsystem.
> 
> Also, with this patch, added:
> - documentation about this provider and its capabilities at
> Documentation/hte.
> - Compilation support in Makefile and Kconfig
> 
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> ---
> Changes in v3:
> - Addressed grammatical/spelling errors.
> 
>   Documentation/hte/index.rst        |  22 ++
>   Documentation/hte/tegra194-hte.rst |  57 +++
>   Documentation/index.rst            |   1 +
>   drivers/hte/Kconfig                |  12 +
>   drivers/hte/Makefile               |   1 +
>   drivers/hte/hte-tegra194.c         | 545 +++++++++++++++++++++++++++++
>   6 files changed, 638 insertions(+)
>   create mode 100644 Documentation/hte/index.rst
>   create mode 100644 Documentation/hte/tegra194-hte.rst
>   create mode 100644 drivers/hte/hte-tegra194.c
> 

> diff --git a/Documentation/hte/tegra194-hte.rst b/Documentation/hte/tegra194-hte.rst
> new file mode 100644
> index 000000000000..3bebcbac5847
> --- /dev/null
> +++ b/Documentation/hte/tegra194-hte.rst
> @@ -0,0 +1,57 @@
> +HTE Kernel provider driver
> +==========================
> +
> +Description
> +-----------
> +The Nvidia tegra194 HTE provider driver implements two GTE
> +(Generic Timestamping Engine) instances: 1) GPIO GTE and 2) LIC
> +(Legacy Interrupt Controller) IRQ GTE. Both GTEs instances get the

                                           Both GTE instances

> +timestamp from the system counter TSC which has 31.25MHz clock rate, and the
> +driver converts clock tick rate to nanoseconds before storing it as timestamp
> +value.
> +
> +GPIO GTE
> +--------
> +
> +This GTE instance timestamps GPIO in real time. For that to happen GPIO
> +needs to be configured as input and IRQ needs to be enabled. The only always on

                         Is "only" supposed to be here? ----------- ^^^^^^

> +(AON) GPIO controller instance supports timestamping GPIOs in real time and it
> +has 39 GPIO lines. The GPIO GTE and AON GPIO controller are tightly coupled as
> +it requires very specific bits to be set in GPIO config register before GPIO
> +GTE can be used. The GPIO GTE functionality is accessed from the GPIOLIB
> +framework for the in-kernel and userspace consumers. In the latter case,
> +requests go through GPIOLIB CDEV framework. The below APIs are added in GPIOLIB
> +framework to access HTE subsystem and GPIO GTE.
> +
> +.. kernel-doc:: drivers/gpio/gpiolib.c
> +   :functions: gpiod_req_hw_timestamp_ns gpiod_rel_hw_timestamp_ns
> +
> +There is hte-tegra194-gpio-test.c, located in ``drivers/hte/`` directory, test
> +driver which demonstrates above APIs for the Jetson AGX platform.
> +
> +For userspace consumers, GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE flag must be
> +specified during IOCTL calls. Refer to ``tools/gpio/gpio-event-mon.c``, which
> +returns the timestamp in nanoseconds.
> +
> +LIC (Legacy Interrupt Controller) IRQ GTE
> +-----------------------------------------
> +
> +This GTE instance timestamps LIC IRQ lines in real time. There are 352 IRQ
> +lines which this instance can add timestamps to in real time. The hte
> +devicetree binding described at ``Documentation/devicetree/bindings/hte/``
> +provides an example of how a consumer can request an IRQ line. Since it is a
> +one-to-one mapping, consumers can simply specify the IRQ number that they are
> +interested in. There is no userspace consumer support for this GTE instance in
> +the hte framework. The sample test code hte-tegra194-irq-test.c, located in
> +the ``drivers/hte/`` directory, demonstrates how to use an IRQ GTE instance.
> +The below is sample device tree snippet code for the test driver::
> +
> + tegra_hte_irq_test {
> +        compatible = "nvidia,tegra194-hte-irq-test";
> +        htes = <&tegra_hte_lic 0x19>;
> +        hte-names = "hte-lic";
> + };
> +
> +The provider source code of both IRQ and GPIO GTE instances is located at
> +``drivers/hte/hte-tegra194.c``.
> +
> diff --git a/Documentation/index.rst b/Documentation/index.rst
> index 1b13c2445e87..b41118577fe6 100644
> --- a/Documentation/index.rst
> +++ b/Documentation/index.rst
> @@ -138,6 +138,7 @@ needed).
>      misc-devices/index
>      scheduler/index
>      mhi/index
> +   hte/index

Looks like a text alignment problem there.

>   
>   Architecture-agnostic documentation
>   -----------------------------------


-- 
~Randy

Re: [RFC v3 04/12] dt-bindings: Add HTE bindings

[Rob Herring]

On Tue, 23 Nov 2021 11:30:31 -0800, Dipen Patel wrote:
> Introduces HTE devicetree binding details for the HTE subsystem. It
> includes examples for the consumers, binding details for the providers
> and specific binding details for the Tegra194 based HTE providers.
> 
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
> ---
> Changes in v2:
> - Replace hte with hardware-timestamp for property names
> - Renamed file
> - Removed example from the common dt binding file.
> 
> Changes in v3:
> - Addressed grammatical errors.
> - Removed double plural from the respective properties.
> - Added dual license.
> - Prefixed "nvidia" in nvidia specific properties.
> 
>  .../hte/hardware-timestamps-common.yaml       | 29 +++++++
>  .../devicetree/bindings/hte/hte-consumer.yaml | 48 +++++++++++
>  .../bindings/hte/nvidia,tegra194-hte.yaml     | 80 +++++++++++++++++++
>  3 files changed, 157 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/hte/hardware-timestamps-common.yaml
>  create mode 100644 Documentation/devicetree/bindings/hte/hte-consumer.yaml
>  create mode 100644 Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
> 

My bot found errors running 'make DT_CHECKER_FLAGS=-m dt_binding_check'
on your patch (DT_CHECKER_FLAGS is new in v5.13):

yamllint warnings/errors:

dtschema/dtc warnings/errors:
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml: properties:nvidia,slices:type: 'anyOf' conditional failed, one must be fixed:
	'int' is not one of ['array', 'boolean', 'integer', 'null', 'number', 'object', 'string']
	'int' is not of type 'array'
	from schema $id: http://json-schema.org/draft-07/schema#
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml: properties:nvidia,slices:type: 'int' is not one of ['boolean', 'object']
	from schema $id: http://devicetree.org/meta-schemas/core.yaml#
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml: properties:nvidia,slices: 'oneOf' conditional failed, one must be fixed:
	Additional properties are not allowed ('enum' was unexpected)
		hint: A vendor boolean property can use "type: boolean"
	Additional properties are not allowed ('type' was unexpected)
		hint: A vendor string property with exact values has an implicit type
	/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml: properties:nvidia,slices: 'oneOf' conditional failed, one must be fixed:
		'$ref' is a required property
		'allOf' is a required property
		hint: A vendor property needs a $ref to types.yaml
		from schema $id: http://devicetree.org/meta-schemas/vendor-props.yaml#
	3 is not of type 'string'
		hint: A vendor string property with exact values has an implicit type
	11 is not of type 'string'
		hint: A vendor string property with exact values has an implicit type
	'boolean' was expected
		hint: A vendor boolean property can use "type: boolean"
	hint: Vendor specific properties must have a type and description unless they have a defined, common suffix.
	from schema $id: http://devicetree.org/meta-schemas/vendor-props.yaml#
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml: properties:nvidia,int-threshold: 'oneOf' conditional failed, one must be fixed:
	'type' is a required property
		hint: A vendor boolean property can use "type: boolean"
	Additional properties are not allowed ('minimum', 'maximum' were unexpected)
		hint: A vendor boolean property can use "type: boolean"
	/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml: properties:nvidia,int-threshold: 'oneOf' conditional failed, one must be fixed:
		'enum' is a required property
		'const' is a required property
		hint: A vendor string property with exact values has an implicit type
		from schema $id: http://devicetree.org/meta-schemas/vendor-props.yaml#
	/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml: properties:nvidia,int-threshold: 'oneOf' conditional failed, one must be fixed:
		'$ref' is a required property
		'allOf' is a required property
		hint: A vendor property needs a $ref to types.yaml
		from schema $id: http://devicetree.org/meta-schemas/vendor-props.yaml#
	hint: Vendor specific properties must have a type and description unless they have a defined, common suffix.
	from schema $id: http://devicetree.org/meta-schemas/vendor-props.yaml#
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml: ignoring, error in schema: properties: nvidia,slices: type
warning: no schema found in file: ./Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.example.dt.yaml:0:0: /example-0/hardware-timestamp@c1e0000: failed to match any schema with compatible: ['nvidia,tegra194-gte-aon']
Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.example.dt.yaml:0:0: /example-1/hardware-timestamp@3aa0000: failed to match any schema with compatible: ['nvidia,tegra194-gte-lic']

doc reference errors (make refcheckdocs):

See https://patchwork.ozlabs.org/patch/1558766

This check can fail if there are any dependencies. The base for a patch
series is generally the most recent rc1.

If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:

pip3 install dtschema --upgrade

Please check and re-submit.

Re: [RFC v3 02/12] drivers: Add hardware timestamp engine (HTE)

[Kent Gibson]

On Tue, Nov 23, 2021 at 11:30:29AM -0800, Dipen Patel wrote:
> Some devices can timestamp system lines/signals/Buses in real-time
> using the hardware counter or other hardware means which can give
> finer granularity and help avoid jitter introduced by software
> timestamping. To utilize such functionality, this patchset creates
> HTE subsystem where devices can register themselves as providers so
> that the consumers devices can request specific line from the
> providers.
> 
> It provides below APIs for the provider:
> - devm_hte_register_chip -- To register the HTE provider.
> - hte_push_ts_ns() -- To push timestamp data into HTE subsystem.
> 
> It provides below APIs for the consumer:
> - devm_of_hte_request_ts() -- Request timestamp functionality.
> - hte_req_ts_by_hte_name() -- To request timestamp functionality by
> using HTE provider dt node.
> - hte_enable_ts() -- To disable timestamp functionality.
> - hte_disable_ts() -- To enable timestamp functionality.
> - hte_release_ts() -- To release timestamp functionality and its
> associated resources.
> - hte_get_clk_src_info() -- To query clock source information from
> the provider
> 
> The detail about parameters and API usage are described in each
> functions definitions in drivers/hte/hte.c file.
> 
> The patch adds compilation support in Makefile and menu options in
> Kconfig.
> 
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> Reported-by: kernel test robot <lkp@intel.com>
> ---
> Changes in v2:
> - Removed buffer abstraction layer as well related APIs, HTE now will not store
> any data, instead will pass to consumer as soon as it is available.
> - Removed unnecessary dynamical allocations
> - Removed timestamp retrieve API
> - Removed release, unregister related APIs as their counterpart are resource
> managed.
> - Added kernel thread implementation if consumer indicates threaded callback
> during request API time.
> - Changed hte_req_ts_by_dt_node API to remove device node exposure from the
> interface, instead consumer will their device node with property name that
> indicates the provider it wants to use.
> 
> Changes in v3:
> - Addressed grammatical/spelling errors.
> 
>  drivers/Kconfig      |   2 +
>  drivers/Makefile     |   1 +
>  drivers/hte/Kconfig  |  22 ++
>  drivers/hte/Makefile |   2 +
>  drivers/hte/hte.c    | 907 +++++++++++++++++++++++++++++++++++++++++++
>  include/linux/hte.h  | 248 ++++++++++++
>  6 files changed, 1182 insertions(+)
>  create mode 100644 drivers/hte/Kconfig
>  create mode 100644 drivers/hte/Makefile
>  create mode 100644 drivers/hte/hte.c
>  create mode 100644 include/linux/hte.h
> 
> diff --git a/drivers/Kconfig b/drivers/Kconfig
> index 0d399ddaa185..b3c8332bf5c7 100644
> --- a/drivers/Kconfig
> +++ b/drivers/Kconfig
> @@ -236,4 +236,6 @@ source "drivers/interconnect/Kconfig"
>  source "drivers/counter/Kconfig"
>  
>  source "drivers/most/Kconfig"
> +
> +source "drivers/hte/Kconfig"
>  endmenu
> diff --git a/drivers/Makefile b/drivers/Makefile
> index be5d40ae1488..087c08e846e6 100644
> --- a/drivers/Makefile
> +++ b/drivers/Makefile
> @@ -188,3 +188,4 @@ obj-$(CONFIG_GNSS)		+= gnss/
>  obj-$(CONFIG_INTERCONNECT)	+= interconnect/
>  obj-$(CONFIG_COUNTER)		+= counter/
>  obj-$(CONFIG_MOST)		+= most/
> +obj-$(CONFIG_HTE)		+= hte/
> diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
> new file mode 100644
> index 000000000000..1fcfe17cf28a
> --- /dev/null
> +++ b/drivers/hte/Kconfig
> @@ -0,0 +1,22 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +menuconfig HTE
> +	bool "Hardware Timestamping Engine (HTE) Support"
> +	help
> +	  Hardware Timestamping Engine (HTE) Support.
> +
> +	  Some devices provide a hardware timestamping engine which can
> +	  timestamp certain device lines/signals in realtime. This provides a
> +	  hardware-assisted timestamp to generic signals like GPIOs or IRQs
> +	  lines. It comes with a benefit for applications like autonomous
> +	  machines needing accurate timestamping event with less jitter.
> +
> +	  This framework provides a generic interface to such HTE devices
> +	  within the Linux kernel. It provides an API to register and
> +	  unregister a HTE provider chip, configurable software buffer to
> +	  store the timestamps, push the timestamp from the HTE providers and
> +	  retrieve timestamps for the consumers. It also provides means for the
> +	  consumers to request signals it wishes to hardware timestamp and
> +	  release them if not required.
> +
> +	  If unsure, say no.
> +
> diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
> new file mode 100644
> index 000000000000..fc03bdf44427
> --- /dev/null
> +++ b/drivers/hte/Makefile
> @@ -0,0 +1,2 @@
> +obj-$(CONFIG_HTE)		+= hte.o
> +
> diff --git a/drivers/hte/hte.c b/drivers/hte/hte.c
> new file mode 100644
> index 000000000000..ca91d48f48b9
> --- /dev/null
> +++ b/drivers/hte/hte.c
> @@ -0,0 +1,907 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2021 NVIDIA Corporation
> + *
> + * Author: Dipen Patel <dipenp@nvidia.com>
> + */
> +
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/err.h>
> +#include <linux/slab.h>
> +#include <linux/of.h>
> +#include <linux/of_device.h>
> +#include <linux/mutex.h>
> +#include <linux/sched.h>
> +#include <linux/uaccess.h>
> +#include <linux/hte.h>
> +#include <linux/delay.h>
> +#include <linux/debugfs.h>
> +#include <linux/kthread.h>
> +
> +#define HTE_TS_NAME_LEN		10
> +
> +/* Global list of the HTE devices */
> +static DEFINE_SPINLOCK(hte_lock);
> +static LIST_HEAD(hte_devices);
> +
> +enum {
> +	HTE_TS_REGISTERED,
> +	HTE_TS_DISABLE,
> +};
> +
> +enum {
> +	HTE_CB_RUN_THREAD,
> +	HTE_CB_NUM,
> +};
> +
> +/**
> + * struct hte_ts_info - Information related to requested timestamp.
> + *
> + * @xlated_id: Timestamp ID as understood between HTE subsys and HTE provider,
> + * See xlate callback API.
> + * @flags: Flags holding state informations.
> + * @hte_cb_flags: Callback related flags.
> + * @seq: Timestamp sequence counter.
> + * @hte_name: Indicates if HTE core has set name for this timestamp entity.
> + * @cb: Callback function provided by clients.
> + * @tcb: Threaded callback function provided by clients.
> + * @dropped_ts: Dropped timestamps.
> + * @slock: Spin lock.
> + * @thread: Thread task when tcb is provided.
> + * @req_mlock: Lock during timestamp request/release APIs.
> + * @ts_dbg_root: Root for the debug fs.
> + * @gdev: HTE abstract device that this timestamp belongs to.
> + * @cl_data: Client specific data.
> + */
> +struct hte_ts_info {
> +	u32 xlated_id;
> +	unsigned long flags;
> +	unsigned long hte_cb_flags;
> +	u64 seq;
> +	bool hte_name;
> +	hte_ts_cb_t cb;
> +	hte_ts_threaded_cb_t tcb;
> +	atomic_t dropped_ts;
> +	spinlock_t slock;
> +	struct task_struct *thread;
> +	struct mutex req_mlock;
> +	struct dentry *ts_dbg_root;
> +	struct hte_device *gdev;
> +	void *cl_data;
> +};
> +
> +/**
> + * struct hte_device - HTE abstract device
> + * @nlines: Number of entities this device supports.
> + * @ts_req: Total number of entities requested.
> + * @sdev: Device used at various debug prints.
> + * @dbg_root: Root directory for debug fs.
> + * @list: List node to store hte_device for each provider.
> + * @chip: HTE chip providing this HTE device.
> + * @owner: helps prevent removal of modules when in use.
> + * @ei: Timestamp information.
> + */
> +struct hte_device {
> +	u32 nlines;
> +	atomic_t ts_req;
> +	struct device *sdev;
> +	struct dentry *dbg_root;
> +	struct list_head list;
> +	struct hte_chip *chip;
> +	struct module *owner;
> +	struct hte_ts_info ei[];
> +};
> +
> +#ifdef CONFIG_DEBUG_FS
> +
> +static struct dentry *hte_root;
> +
> +static int __init hte_subsys_dbgfs_init(void)
> +{
> +	/* creates /sys/kernel/debug/hte/ */
> +	hte_root = debugfs_create_dir("hte", NULL);
> +
> +	return 0;
> +}
> +subsys_initcall(hte_subsys_dbgfs_init);
> +
> +static void hte_chip_dbgfs_init(struct hte_device *gdev)
> +{
> +	const struct hte_chip *chip = gdev->chip;
> +	const char *name = chip->name ? chip->name : dev_name(chip->dev);
> +
> +	gdev->dbg_root = debugfs_create_dir(name, hte_root);
> +
> +	debugfs_create_atomic_t("ts_requested", 0444, gdev->dbg_root,
> +				&gdev->ts_req);
> +	debugfs_create_u32("total_ts", 0444, gdev->dbg_root,
> +			   &gdev->nlines);
> +}
> +
> +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
> +{
> +	if (!ei->gdev->dbg_root || !name)
> +		return;
> +
> +	ei->ts_dbg_root = debugfs_create_dir(name, ei->gdev->dbg_root);
> +
> +	debugfs_create_atomic_t("dropped_timestamps", 0444, ei->ts_dbg_root,
> +				&ei->dropped_ts);
> +}
> +
> +#else
> +
> +static void hte_chip_dbgfs_init(struct hte_device *gdev)
> +{
> +}
> +
> +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
> +{
> +}
> +
> +#endif
> +
> +/**
> + * hte_release_ts() - Consumer calls this API to release the entity, where
> + * entity could be anything providers support, like lines, signals, buses,
> + * etc...
> + *
> + * @desc: timestamp descriptor, this is the same as returned by the request API.
> + *
> + * Context: debugfs_remove_recursive() function call may use sleeping locks,
> + *	    not suitable from atomic context.
> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int hte_release_ts(struct hte_ts_desc *desc)
> +{
> +	u32 id;
> +	int ret = 0;
> +	unsigned long flag;
> +	struct hte_device *gdev;
> +	struct hte_ts_info *ei;
> +
> +	if (!desc)
> +		return -EINVAL;
> +
> +	ei = desc->hte_data;
> +
> +	if (!ei || !ei->gdev)
> +		return -EINVAL;
> +
> +	gdev = ei->gdev;
> +	id = desc->con_id;
> +
> +	mutex_lock(&ei->req_mlock);
> +
> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
> +		dev_info(gdev->sdev, "id:%d is not registered", id);
> +		ret = -EUSERS;
> +		goto unlock;
> +	}
> +
> +	ret = gdev->chip->ops->release(gdev->chip, ei->xlated_id);
> +	if (ret) {
> +		dev_err(gdev->sdev, "id: %d free failed\n", id);
> +		goto unlock;
> +	}
> +
> +	if (ei->hte_name)
> +		kfree(desc->name);
> +
> +	debugfs_remove_recursive(ei->ts_dbg_root);
> +
> +	spin_lock_irqsave(&ei->slock, flag);
> +
> +	atomic_dec(&gdev->ts_req);
> +	atomic_set(&ei->dropped_ts, 0);
> +
> +	ei->seq = 0;
> +	desc->hte_data = NULL;
> +
> +	clear_bit(HTE_TS_REGISTERED, &ei->flags);
> +
> +	spin_unlock_irqrestore(&ei->slock, flag);
> +
> +	if (ei->tcb) {
> +		kthread_stop(ei->thread);
> +		put_task_struct(ei->thread);
> +	}
> +
> +	ei->cb = NULL;
> +	ei->tcb = NULL;
> +	ei->thread = NULL;
> +	ei->cl_data = NULL;
> +
> +	module_put(gdev->owner);
> +unlock:
> +	mutex_unlock(&ei->req_mlock);
> +	dev_dbg(gdev->sdev, "release id: %d\n", id);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(hte_release_ts);
> +
> +static int hte_ts_dis_en_common(struct hte_ts_desc *desc, bool en)
> +{
> +	u32 ts_id;
> +	struct hte_device *gdev;
> +	struct hte_ts_info *ei;
> +	int ret;
> +	unsigned long flag;
> +
> +	if (!desc)
> +		return -EINVAL;
> +
> +	ei = desc->hte_data;
> +
> +	if (!ei || !ei->gdev)
> +		return -EINVAL;
> +
> +	gdev = ei->gdev;
> +	ts_id = desc->con_id;
> +
> +	mutex_lock(&ei->req_mlock);
> +
> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
> +		dev_dbg(gdev->sdev, "id:%d is not registered", ts_id);
> +		ret = -EUSERS;
> +		goto out;
> +	}
> +
> +	spin_lock_irqsave(&ei->slock, flag);
> +
> +	if (en) {
> +		if (!test_bit(HTE_TS_DISABLE, &ei->flags)) {
> +			ret = 0;
> +			goto out_unlock;
> +		}
> +
> +		spin_unlock_irqrestore(&ei->slock, flag);
> +		ret = gdev->chip->ops->enable(gdev->chip, ei->xlated_id);
> +		if (ret) {
> +			dev_warn(gdev->sdev, "id: %d enable failed\n",
> +				 ts_id);
> +			goto out;
> +		}
> +
> +		spin_lock_irqsave(&ei->slock, flag);
> +		clear_bit(HTE_TS_DISABLE, &ei->flags);
> +	} else {
> +		if (test_bit(HTE_TS_DISABLE, &ei->flags)) {
> +			ret = 0;
> +			goto out_unlock;
> +		}
> +
> +		spin_unlock_irqrestore(&ei->slock, flag);
> +		ret = gdev->chip->ops->disable(gdev->chip, ei->xlated_id);
> +		if (ret) {
> +			dev_warn(gdev->sdev, "id: %d disable failed\n",
> +				 ts_id);
> +			goto out;
> +		}
> +
> +		spin_lock_irqsave(&ei->slock, flag);
> +		set_bit(HTE_TS_DISABLE, &ei->flags);
> +	}
> +
> +out_unlock:
> +	spin_unlock_irqrestore(&ei->slock, flag);
> +out:
> +	mutex_unlock(&ei->req_mlock);
> +	return ret;
> +}
> +
> +/**
> + * hte_disable_ts() - Disable timestamp on given descriptor.
> + *
> + * The API does not release any resources associated with desc.
> + *
> + * @desc: ts descriptor, this is the same as returned by the request API.
> + *
> + * Context: Holds mutex lock, not suitable from atomic context.
> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int hte_disable_ts(struct hte_ts_desc *desc)
> +{
> +	return hte_ts_dis_en_common(desc, false);
> +}
> +EXPORT_SYMBOL_GPL(hte_disable_ts);
> +
> +/**
> + * hte_enable_ts() - Enable timestamp on given descriptor.
> + *
> + * @desc: ts descriptor, this is the same as returned by the request API.
> + *
> + * Context: Holds mutex lock, not suitable from atomic context.
> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int hte_enable_ts(struct hte_ts_desc *desc)
> +{
> +	return hte_ts_dis_en_common(desc, true);
> +}
> +EXPORT_SYMBOL_GPL(hte_enable_ts);
> +
> +static int hte_simple_xlate(struct hte_chip *gc,
> +			    const struct of_phandle_args *args,
> +			    struct hte_ts_desc *desc,
> +			    u32 *id)
> +{
> +	if (!id || !desc || !gc)
> +		return -EINVAL;
> +
> +	/*
> +	 * For the providers which do not have any internal mappings between
> +	 * logically exposed ids and actual ids, will set both
> +	 * the same.
> +	 *
> +	 * In case there is a internal mapping needed, providers will need to
> +	 * provide its own xlate function where con_id will be sent as
> +	 * args[0] and it will return xlated id. Later xlated id will be
> +	 * used for any future exchanges between provider and subsystems.
> +	 */
> +
> +	if (args) {
> +		if (gc->of_hte_n_cells < 1)
> +			return -EINVAL;
> +
> +		if (args->args_count != gc->of_hte_n_cells)
> +			return -EINVAL;
> +
> +		*id = args->args[0];
> +		desc->con_id = *id;
> +	} else {
> +		*id = desc->con_id;
> +	}
> +
> +	if (desc->con_id > gc->nlines)
> +		return -EINVAL;
> +
> +	desc->hte_data = NULL;
> +
> +	return 0;
> +}
> +
> +static int _hte_wait_for_ts_data(struct hte_ts_info *ei)
> +{
> +	for (;;) {
> +		set_current_state(TASK_INTERRUPTIBLE);
> +
> +		if (kthread_should_stop()) {
> +			if (test_and_clear_bit(HTE_CB_RUN_THREAD,
> +			    &ei->hte_cb_flags)) {
> +				__set_current_state(TASK_RUNNING);
> +				return 0;
> +			}
> +			__set_current_state(TASK_RUNNING);
> +			return -1;
> +		}
> +
> +		if (test_and_clear_bit(HTE_CB_RUN_THREAD,
> +				       &ei->hte_cb_flags)) {
> +			__set_current_state(TASK_RUNNING);
> +			return 0;
> +		}
> +		schedule();
> +	}
> +}
> +
> +static int _hte_threadfn(void *data)
> +{
> +	struct hte_ts_info *ei = data;
> +
> +	while (!_hte_wait_for_ts_data(ei))
> +		ei->tcb(ei->cl_data);
> +
> +	return 0;
> +}
> +
> +static int _hte_setup_thread(struct hte_ts_info *ei, u32 id)
> +{
> +	struct task_struct *t;
> +
> +	t = kthread_create(_hte_threadfn, ei, "hte-%u", id);
> +	if (IS_ERR(t))
> +		return PTR_ERR(t);
> +
> +	ei->thread = get_task_struct(t);
> +
> +	return 0;
> +}
> +
> +static int ___hte_req_ts(struct hte_device *gdev, struct hte_ts_desc *desc,
> +			 u32 xlated_id, hte_ts_cb_t cb,
> +			 hte_ts_threaded_cb_t tcb, void *data)
> +{
> +	struct hte_ts_info *ei;
> +	int ret;
> +	u32 con_id = desc->con_id;
> +
> +	if (!try_module_get(gdev->owner))
> +		return -ENODEV;
> +
> +	ei = &gdev->ei[xlated_id];
> +	ei->xlated_id = xlated_id;
> +
> +	/*
> +	 * There is a chance that multiple consumers requesting same entity,
> +	 * lock here.
> +	 */
> +	mutex_lock(&ei->req_mlock);
> +
> +	if (test_bit(HTE_TS_REGISTERED, &ei->flags)) {
> +		dev_dbg(gdev->chip->dev, "id:%u is already registered",
> +			xlated_id);
> +		ret = -EUSERS;
> +		goto unlock;
> +	}
> +
> +	ei->cb = cb;
> +	ei->tcb = tcb;
> +	if (tcb) {
> +		ret = _hte_setup_thread(ei, xlated_id);
> +		if (ret < 0) {
> +			dev_err(gdev->chip->dev, "setting thread failed\n");
> +			goto unlock;
> +		}
> +	}
> +
> +	ret = gdev->chip->ops->request(gdev->chip, xlated_id);
> +	if (ret < 0) {
> +		dev_err(gdev->chip->dev, "ts request failed\n");
> +		goto unlock;
> +	}
> +
> +	desc->hte_data = ei;
> +	ei->cl_data = data;
> +
> +	atomic_inc(&gdev->ts_req);
> +
> +	ei->hte_name = false;
> +	if (!desc->name) {
> +		desc->name = kzalloc(HTE_TS_NAME_LEN, GFP_KERNEL);
> +		if (desc->name) {
> +			scnprintf(desc->name, HTE_TS_NAME_LEN, "ts_%u",
> +				  con_id);
> +			ei->hte_name = true;
> +		}
> +	}
> +
> +	hte_ts_dbgfs_init(desc->name, ei);
> +	set_bit(HTE_TS_REGISTERED, &ei->flags);
> +
> +	mutex_unlock(&ei->req_mlock);
> +
> +	dev_dbg(gdev->chip->dev, "id: %u, xlated id:%u", con_id, xlated_id);
> +
> +	return 0;
> +
> +unlock:
> +	module_put(gdev->owner);
> +	mutex_unlock(&ei->req_mlock);
> +
> +	return ret;
> +}
> +
> +static struct hte_device *of_node_to_htedevice(struct device_node *np)
> +{
> +	struct hte_device *gdev;
> +
> +	spin_lock(&hte_lock);
> +
> +	list_for_each_entry(gdev, &hte_devices, list)
> +		if (gdev->chip && gdev->chip->dev &&
> +		    gdev->chip->dev->of_node == np) {
> +			spin_unlock(&hte_lock);
> +			return gdev;
> +		}
> +
> +	spin_unlock(&hte_lock);
> +
> +	return ERR_PTR(-ENODEV);
> +}
> +
> +static struct hte_device *of_hte_dev_get(struct device *dev,
> +					 struct device_node *np,
> +					 const char *label,
> +					 struct of_phandle_args *args)
> +{
> +	struct hte_device *gdev = NULL;
> +	int index = 0;
> +	int err;
> +
> +	if (label) {
> +		index = of_property_match_string(np,
> +						 "hardware-timestamp-names",
> +						 label);
> +		if (index < 0)
> +			return ERR_PTR(index);
> +	}
> +
> +	err = of_parse_phandle_with_args(np, "hardware-timestamps",
> +					 "#hardware-timestamp-cells", index,
> +					 args);
> +	if (err) {
> +		pr_err("%s(): can't parse \"hardware-timestamps\" property\n",
> +		       __func__);
> +		return ERR_PTR(err);
> +	}
> +
> +	gdev = of_node_to_htedevice(args->np);
> +	if (IS_ERR(gdev)) {
> +		pr_err("%s(): HTE chip not found\n", __func__);
> +		of_node_put(args->np);
> +		return gdev;
> +	}
> +
> +	return gdev;
> +}
> +
> +static int __hte_req_ts(struct device *dev, struct hte_ts_desc *desc,
> +			hte_ts_cb_t cb, hte_ts_threaded_cb_t tcb, void *data)
> +{
> +	struct hte_device *gdev = NULL;
> +	struct of_phandle_args args;
> +	int ret;
> +	u32 xlated_id;
> +
> +	gdev = of_hte_dev_get(dev, dev->of_node, desc->name, &args);
> +	if (IS_ERR(gdev))
> +		return PTR_ERR(gdev);
> +
> +	if (!gdev->chip) {
> +		pr_debug("requested id does not have provider\n");
> +		return -ENODEV;
> +	}
> +
> +	ret = gdev->chip->xlate(gdev->chip, &args, desc, &xlated_id);
> +	if (ret < 0)
> +		goto put;
> +
> +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb, tcb, data);
> +	if (ret < 0)
> +		goto put;
> +
> +	return 0;
> +
> +put:
> +	of_node_put(args.np);
> +
> +	return ret;
> +}
> +
> +static void __devm_hte_release_ts(void *res)
> +{
> +	hte_release_ts(res);
> +}
> +
> +/**
> + * devm_of_hte_request_ts() - Resource managed API to request the HTE facility
> + * on the specified entity, where entity is provider specific for example,
> + * GPIO lines, signals, buses etc...
> + *
> + * The API allocates necessary resources and enables the timestamp. So calling
> + * hte_enable_ts is not needed. The consumer does not need to call
> + * hte_release_ts since it will be called upon consumer exit.
> + *
> + * @dev: HTE consumer/client device.
> + * @desc: Pre-allocated timestamp descriptor. HTE core will fill out necessary
> + * details. Optionally the consumer can set name field of desc, if not
> + * specified HTE core will set it as ts_con_id. It will be the consumer's
> + * job to free any allocation related to this structure as well name field
> + * in case it has set that field.
> + * @cb: Callback to push the timestamp data to consumer.
> + * @tcb: Optional callback. If its provided, subsystem will create
> + * thread. This will be called when cb returns HTE_RUN_THREADED_CB.
> + * @data: Client data, will be sent back during cb and tcb callbacks.
> + *
> + * Context: Holds mutex lock.
> + * Returns: Returns 0 on success or negative error code on failure.
> + */
> +int devm_of_hte_request_ts(struct device *dev, struct hte_ts_desc *desc,
> +			   hte_ts_cb_t cb, hte_ts_threaded_cb_t tcb,
> +			   void *data)
> +{
> +	int err;
> +
> +	if (!dev || !dev->of_node || !desc || !cb)
> +		return -EINVAL;
> +
> +	err = __hte_req_ts(dev, desc, cb, tcb, data);
> +	if (err)
> +		return err;
> +
> +	err = devm_add_action_or_reset(dev, __devm_hte_release_ts, desc);
> +	if (err)
> +		return err;
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(devm_of_hte_request_ts);
> +
> +/**
> + * hte_req_ts_by_hte_name() - Request entity to timestamp realtime by passing
> + * property name that contains HTE provider phandle, meaning of the entity
> + * is HTE provider specific, for example lines, signals, GPIOs, buses etc...
> + *
> + * This API is designed to address below uses cases:
> + *
> + * 1) For the consumer device which acts as a central device for secondary
> + * consumers. For example, GPIO controller driver acts as a primary consumer
> + * on behalf of in kernel and userspace GPIO HTE consumers. The GPIO controller
> + * driver specifies HTE provider that it supports/wants and it becomes opaque
> + * for the secondary consumers requesting GPIO and hardware timestamp through
> + * that GPIO controller.
> + *
> + * 2) For the providers which are dependent on other hardware modules. In that
> + * case it forces consumers to go through other subsystem or driver making them
> + * secondary consumers. Same example as above applies here as well.
> + *
> + * The API allocates necessary resources and enables the timestamp. So calling
> + * hte_enable_ts is not needed.
> + *
> + * @dev: HTE consumer/client device.
> + * @propname: Name of property holding a HTE provider phandle value
> + * @desc: Pre-allocated timestamp descriptor with con_id set by the consumer.
> + * HTE core will fill out the rest. Optionally the consumer can set name
> + * field of desc, if not specified HTE core will set it as ts_con_id. It will
> + * be the consumer's job to free any allocation related to this structure as
> + * well name field in case it has set that field.
> + * @cb: Callback to push the timestamp data to consumer.
> + * @tcb: Optional callback. If its provided, subsystem will create
> + * thread. This will be called when cb returns HTE_RUN_THREADED_CB.
> + * @data: Client data, will be sent back during cb and tcb callbacks.
> + *
> + * Context: Holds mutex lock, can not be called from atomic context. The mutex
> + * lock is used to serialize multiple consumers.
> + * Returns: returns 0 on success or negative error code on failure.
> + */
> +int hte_req_ts_by_hte_name(struct device *dev, const char *propname,
> +			   struct hte_ts_desc *desc, hte_ts_cb_t cb,
> +			   hte_ts_threaded_cb_t tcb, void *data)
> +{
> +	struct hte_device *gdev;
> +	struct device_node *np = NULL;
> +	int ret;
> +	u32 xlated_id;
> +
> +	if (!dev->of_node || !propname || !desc)
> +		return -EINVAL;
> +
> +	np = of_parse_phandle(dev->of_node, propname, 0);
> +	if (!np)
> +		return -ENODEV;
> +
> +	of_node_put(np);
> +
> +	gdev = of_node_to_htedevice(np);
> +	if (IS_ERR(gdev))
> +		return -ENOTSUPP;
> +
> +	if (!gdev->chip || !gdev->chip->ops)
> +		return -ENOTSUPP;
> +
> +	ret = gdev->chip->xlate(gdev->chip, NULL, desc, &xlated_id);
> +	if (ret < 0) {
> +		dev_err(gdev->chip->dev,
> +			"failed to xlate id: %d\n", desc->con_id);
> +		return ret;
> +	}
> +
> +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb, tcb, data);
> +	if (ret < 0) {
> +		dev_err(gdev->chip->dev,
> +			"failed to request id: %d\n", desc->con_id);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(hte_req_ts_by_hte_name);
> +
> +/**
> + * hte_get_clk_src_info() - Consumer calls this API to query clock source
> + * information of the desc.
> + *

                               ^^^^^^^^^^
Get the clock source information for a ts descriptor.

> + * @desc: ts descriptor, same as returned from request API.
> + * @ci: The API fills this structure with the clock information data.
> + *
> + * Context: Any context.
> + * Returns: 0 on success else negative error code on failure.
> + */
> +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
> +			 struct hte_clk_info *ci)
> +{
> +	struct hte_chip *chip;
> +	struct hte_ts_info *ei;
> +
> +	if (!desc || !desc->hte_data || !ci) {
> +		pr_debug("%s:%d\n", __func__, __LINE__);
> +		return -EINVAL;
> +	}
> +
> +	ei = desc->hte_data;
> +	if (!ei || !ei->gdev || !ei->gdev->chip)
> +		return -EINVAL;
> +
> +	chip = ei->gdev->chip;
> +	if (!chip->ops->get_clk_src_info)
> +		return -ENOTSUPP;
> +
> +	return chip->ops->get_clk_src_info(chip, ci);
> +}
> +EXPORT_SYMBOL_GPL(hte_get_clk_src_info);
> +
> +/**
> + * hte_push_ts_ns() - Used by the provider to push timestamp in nano
> + * seconds i.e data->tsc will be in ns.
> + *
                         ^^^^^^^^
Push timestamp in nanoseconds.

> + * @chip: The HTE chip, used during the registration.
> + * @xlated_id: entity id understood by both subsystem and provider, usually this
> + * is obtained from xlate callback during request API.
> + * @data: timestamp data.
> + *
> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int hte_push_ts_ns(const struct hte_chip *chip, u32 xlated_id,
> +		   struct hte_ts_data *data)
> +{
> +	hte_return_t ret;
> +	int st = 0;
> +	struct hte_ts_info *ei;
> +	unsigned long flag;
> +
> +	if (!chip || !data || !chip->gdev)
> +		return -EINVAL;
> +
> +	if (xlated_id > chip->nlines)
> +		return -EINVAL;
> +
> +	ei = &chip->gdev->ei[xlated_id];
> +
> +	spin_lock_irqsave(&ei->slock, flag);
> +
> +	/* timestamp sequence counter */
> +	data->seq = ei->seq++;
> +
> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags) ||
> +	    test_bit(HTE_TS_DISABLE, &ei->flags)) {
> +		dev_dbg(chip->dev, "Unknown timestamp push\n");
> +		st = -EINVAL;
> +		goto unlock;
> +	}
> +
> +	ret = ei->cb(data, ei->cl_data);
> +	if (ret == HTE_RUN_THREADED_CB && ei->thread) {
> +		if (test_and_set_bit(HTE_CB_RUN_THREAD, &ei->hte_cb_flags))
> +			goto unlock;
> +		else
> +			wake_up_process(ei->thread);
> +	} else if (ret == HTE_CB_TS_DROPPED) {
> +		atomic_inc(&ei->dropped_ts);
> +	} else if (ret == HTE_CB_ERROR) {
> +		dev_dbg(chip->dev, "cb error\n");
> +	}
> +
> +unlock:
> +	spin_unlock_irqrestore(&ei->slock, flag);
> +
> +	return st;
> +}
> +EXPORT_SYMBOL_GPL(hte_push_ts_ns);
> +
> +static int hte_register_chip(struct hte_chip *chip)
> +{
> +	struct hte_device *gdev;
> +	u32 i;
> +
> +	if (!chip || !chip->dev || !chip->dev->of_node)
> +		return -EINVAL;
> +
> +	if (!chip->ops || !chip->ops->request || !chip->ops->release) {
> +		dev_err(chip->dev, "Driver needs to provide ops\n");
> +		return -EINVAL;
> +	}
> +
> +	gdev = kzalloc(struct_size(gdev, ei, chip->nlines), GFP_KERNEL);
> +	if (!gdev)
> +		return -ENOMEM;
> +
> +	gdev->chip = chip;
> +	chip->gdev = gdev;
> +	gdev->nlines = chip->nlines;
> +	gdev->sdev = chip->dev;
> +
> +	for (i = 0; i < chip->nlines; i++) {
> +		gdev->ei[i].gdev = gdev;
> +		mutex_init(&gdev->ei[i].req_mlock);
> +		spin_lock_init(&gdev->ei[i].slock);
> +	}
> +
> +	if (chip->dev->driver)
> +		gdev->owner = chip->dev->driver->owner;
> +	else
> +		gdev->owner = THIS_MODULE;
> +
> +	if (!chip->xlate) {
> +		chip->xlate = hte_simple_xlate;
> +		/* Just a id number to monitor */
> +		chip->of_hte_n_cells = 1;
> +	}
> +
> +	of_node_get(chip->dev->of_node);
> +
> +	INIT_LIST_HEAD(&gdev->list);
> +
> +	spin_lock(&hte_lock);
> +	list_add_tail(&gdev->list, &hte_devices);
> +	spin_unlock(&hte_lock);
> +
> +	hte_chip_dbgfs_init(gdev);
> +
> +	dev_dbg(chip->dev, "Added hte chip\n");
> +
> +	return 0;
> +}
> +
> +/**
> + * hte_unregister_chip() - Used by the provider to remove a HTE chip.
> + * @chip: the HTE chip to remove.
> + *
> + * Context: Can not be called from atomic context.
> + * Returns: 0 on success or a negative error code on failure.
> + */
> +static int hte_unregister_chip(struct hte_chip *chip)
> +{
> +	struct hte_device *gdev;
> +
> +	if (!chip)
> +		return -EINVAL;
> +
> +	gdev = chip->gdev;
> +
> +	spin_lock(&hte_lock);
> +	list_del(&gdev->list);
> +	spin_unlock(&hte_lock);
> +
> +	gdev->chip = NULL;
> +
> +	of_node_put(chip->dev->of_node);
> +	debugfs_remove_recursive(gdev->dbg_root);
> +	kfree(gdev);
> +
> +	dev_dbg(chip->dev, "Removed hte chip\n");
> +
> +	return 0;
> +}
> +
> +static void _hte_devm_unregister_chip(void *chip)
> +{
> +	hte_unregister_chip(chip);
> +}
> +
> +/**
> + * devm_hte_register_chip() - Used by provider to register a HTE chip.
> + * @chip: the HTE chip to add to subsystem.
> + *
> + * The API is resource managed and  _hte_devm_unregister_chip will be called
> + * automatically when the provider exits.
> + *
> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int devm_hte_register_chip(struct hte_chip *chip)
> +{
> +	int err;
> +
> +	err = hte_register_chip(chip);
> +	if (err)
> +		return err;
> +
> +	err = devm_add_action_or_reset(chip->dev, _hte_devm_unregister_chip,
> +				       chip);
> +	if (err)
> +		return err;
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(devm_hte_register_chip);
> diff --git a/include/linux/hte.h b/include/linux/hte.h
> new file mode 100644
> index 000000000000..f4dd5415a493
> --- /dev/null
> +++ b/include/linux/hte.h
> @@ -0,0 +1,248 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2021 NVIDIA Corporation
> + *
> + * Author: Dipen Patel <dipenp@nvidia.com>
> + */
> +
> +#ifndef __LINUX_HTE_H
> +#define __LINUX_HTE_H
> +
> +#include <linux/errno.h>
> +
> +struct hte_chip;
> +struct hte_device;
> +struct of_phandle_args;
> +struct device_node;
> +
> +/**
> + * enum hte_dir - HTE edge timestamp direction.
> + *
> + * @HTE_RISING_EDGE_TS: Timestamps is for rising edge.
> + * @HTE_FALLING_EDGE_TS: Timestamps is for falling edge.
> + * @HTE_DIR_NOSUPP: Direction is not supported.
> + */
> +enum hte_dir {
> +	HTE_RISING_EDGE_TS,
> +	HTE_FALLING_EDGE_TS,
> +	HTE_DIR_NOSUPP,
> +};
> +

Your tegra194 driver doesn't support direction - it always reports
HTE_DIR_NOSUPP.  Do you have an example of hardware that does support
direction?

This might be better as a raw line level rather than an edge direction.
See my related comments on patch 09.

And the third option should be "unknown", rather than "not supported" as
this is a reading being returned, not a feature being requested.
It may be unknown because the feature is not supported, but that is a
separate issue.

> +/**
> + * struct hte_ts_data - HTE timestamp data.
> + * The provider uses and fills timestamp related details during push_timestamp
> + * API call. The consumer uses during retrieve_timestamp API call.
> + *

What push_timestamp() and retrieve_timestamp() calls?

General comment:
Documenting how a struct is used in an API, rather than what it is for or
what it represents, is a maintenance nightmare.  If it is subsequently
used elsewhere or names are changed then the user is now obliged to update
your documentation.
And they will forget to, or overlook it, as you yourself have done here.

Descriptions of what the API calls do should be in the documentation of
the API calls themselves, not the structs they use.

> + * @tsc: Timestamp value.
> + * @seq: Sequence counter of the timestamps.
> + * @dir: Direction of the event at the time of timestamp.
> + */
> +struct hte_ts_data {
> +	u64 tsc;
> +	u64 seq;
> +	enum hte_dir dir;
> +};
> +
> +/**
> + * struct hte_clk_info - Clock source info that HTE provider uses to timestamp
> + * The provider uses hardware clock as a source to timestamp real time. This
> + * structure presents the clock information to consumers during
> + * hte_get_clk_src_info call.
> + *
> + * @hz: Supported clock rate in HZ, for example 1KHz clock = 1000.
> + * @type: Supported clock type. CLOCK_* types.
> + */
> +struct hte_clk_info {
> +	u64 hz;
> +	clockid_t type;
> +};
> +
> +/**
> + * enum hte_return- HTE subsystem return values used during callback.
> + *
> + * @HTE_CB_HANDLED: The consumer handled the data successfully.
> + * @HTE_RUN_THREADED_CB: The consumer needs further processing, in that case HTE
> + * subsystem will invoke kernel thread and call secondary callback provided by
> + * the consumer during devm_of_hte_request_ts and hte_req_ts_by_dt_node call.
> + * @HTE_CB_TS_DROPPED: The client returns when it can not store ts data.
> + * @HTE_CB_ERROR: The client returns error if anything goes wrong.
> + */
> +enum hte_return {
> +	HTE_CB_HANDLED,
> +	HTE_RUN_THREADED_CB,
> +	HTE_CB_TS_DROPPED,
> +	HTE_CB_ERROR,
> +};
> +typedef enum hte_return hte_return_t;
> +

Wrt HTE_CB_TS_DROPPED, why is the client dropping data any of hte's
business?  It is also confusing in that I would expect the dropped_ts
gauge, that you increment when this code is returned, to indicate the
events dropped by the hardware, not the client.  But then you have no
indication of events dropped by hardware at all, though you could
determine that from gaps in the sequence numbers.
Anyway, the client can do the math in both cases if they care to, so not
sure what its purpose is here.

Similarly HTE_CB_ERROR, what can hte do here other than log it - and the
client can do that themselves if they have the need.

> +/**
> + * typedef hte_ts_cb_t - Callback provided during devm_of_hte_request_ts and
> + * hte_req_ts_by_dt_node APIs call.
> + *
> + * The callback is used to push timestamp data to client.
> + * @ts: HW timestamp data.
> + * @data: Client supplied data.
> + */
> +typedef hte_return_t (*hte_ts_cb_t)(struct hte_ts_data *ts, void *data);
> +
> +/**
> + * typedef hte_ts_threaded_cb_t - Threaded callback provided during
> + * devm_of_hte_request_ts and hte_req_ts_by_dt_node APIs call.
> + *
> + * @data: Client supplied data.
> + *
> + * It will be called when client return HTE_RUN_THREADED_CB from hte_ts_cb_t.
> + * The callback will be called from thread context.
> + *
> + */
> +typedef hte_return_t (*hte_ts_threaded_cb_t)(void *data);
> +
> +/**
> + * struct hte_ts_desc - HTE timestamp descriptor, this structure will be
> + * communication token between consumers to subsystem and subsystem to
> + * providers.
> + *
> + * @con_id: This is the same id sent in request APIs.
> + * @name: Descriptive name of the entity that is being monitored for the
> + * realtime timestamping. The consumer can set any name it likes. If null
> + * HTE core will construct name as ts_con_id. It will be the consumer's
> + * job to free any allocation if name is set by the consumer.
> + * @hte_data: Subsystem's private data relate to requested con_id.
> + */
> +struct hte_ts_desc {
> +	u32 con_id;
> +	char *name;
> +	void *hte_data;
> +};
> +
> +/**
> + * struct hte_ops - HTE operations set by providers.
> + *
> + * @request: Hook for requesting a HTE timestamp. Returns 0 on success,
> + * non-zero for failures.
> + * @release: Hook for releasing a HTE timestamp. Returns 0 on success,
> + * non-zero for failures.
> + * @enable: Hook to enable the specified timestamp. Returns 0 on success,
> + * non-zero for failures.
> + * @disable: Hook to disable specified timestamp. Returns 0 on success,
> + * non-zero for failures.
> + * @get_clk_src_info: Hook to get the clock information the provider uses
> + * to timestamp. Returns 0 for success and negative error code for failure. On
> + * success HTE subsystem fills up provided struct hte_clk_info.
> + *
> + * xlated_id parameter is used to communicate between HTE subsystem and the
> + * providers. It is the same id returned during xlate API call and translated
> + * by the provider. This may be helpful as both subsystem and provider locate
> + * the requested entity in constant time, where entity could be anything from
> + * lines, signals, events, buses etc.. that providers support.
> + */
> +struct hte_ops {
> +	int (*request)(struct hte_chip *chip, u32 xlated_id);
> +	int (*release)(struct hte_chip *chip, u32 xlated_id);
> +	int (*enable)(struct hte_chip *chip, u32 xlated_id);
> +	int (*disable)(struct hte_chip *chip, u32 xlated_id);
> +	int (*get_clk_src_info)(struct hte_chip *chip,
> +				struct hte_clk_info *ci);
> +};
> +
> +/**
> + * struct hte_chip - Abstract HTE chip structure.
> + * @name: functional name of the HTE IP block.
> + * @dev: device providing the HTE.
> + * @ops: callbacks for this HTE.
> + * @nlines: number of lines/signals supported by this chip.
> + * @xlate: Callback which translates consumer supplied logical ids to
> + * physical ids, return from 0 for the success and negative for the
> + * failures. It stores (0 to @nlines) in xlated_id parameter for the success.
> + * @of_hte_n_cells: Number of cells used to form the HTE specifier.
> + * @gdev: HTE subsystem abstract device, internal to the HTE subsystem.
> + * @data: chip specific private data.
> + */
> +struct hte_chip {
> +	const char *name;
> +	struct device *dev;
> +	const struct hte_ops *ops;
> +	u32 nlines;
> +	int (*xlate)(struct hte_chip *gc,
> +		     const struct of_phandle_args *args,
> +		     struct hte_ts_desc *desc, u32 *xlated_id);
> +	u8 of_hte_n_cells;
> +
> +	struct hte_device *gdev;
> +	void *data;
> +};
> +
> +#if IS_ENABLED(CONFIG_HTE)
> +/* HTE APIs for the providers */
> +int devm_hte_register_chip(struct hte_chip *chip);
> +int hte_push_ts_ns(const struct hte_chip *chip, u32 xlated_id,
> +		   struct hte_ts_data *data);
> +
> +/* HTE APIs for the consumers */
> +
> +int hte_release_ts(struct hte_ts_desc *desc);
> +int devm_of_hte_request_ts(struct device *dev, struct hte_ts_desc *desc,
> +			   hte_ts_cb_t cb, hte_ts_threaded_cb_t tcb,
> +			   void *data);
> +int hte_req_ts_by_hte_name(struct device *dev, const char *propname,
> +			   struct hte_ts_desc *desc, hte_ts_cb_t cb,
> +			   hte_ts_threaded_cb_t tcb, void *data);
> +int hte_enable_ts(struct hte_ts_desc *desc);
> +int hte_disable_ts(struct hte_ts_desc *desc);
> +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
> +			 struct hte_clk_info *ci);
> +
> +#else /* !CONFIG_HTE */
> +static inline int devm_hte_register_chip(struct hte_chip *chip)
> +{
> +	return -ENOTSUPP;
> +}
> +
> +static inline int hte_push_ts_ns(const struct hte_chip *chip,
> +				 u32 xlated_id,
> +				 const struct hte_ts_data *data)
> +{
> +	return -ENOTSUPP;
> +}
> +
> +static inline int hte_release_ts(struct hte_ts_desc *desc)
> +{
> +	return -ENOTSUPP;
> +}
> +
> +static inline int devm_of_hte_request_ts(struct device *dev,
> +					 struct hte_ts_desc *desc,
> +					 hte_ts_cb_t cb,
> +					 hte_ts_threaded_cb_t threaded_cb,
> +					 void *data)
> +{
> +	return -ENOTSUPP;
> +}
> +

Rename threaded_cb to tcb to be consistent with other functions.

> +static inline int hte_req_ts_by_hte_name(struct device *dev,
> +					 const char *propname,
> +					 struct hte_ts_desc *desc,
> +					 hte_ts_cb_t cb,
> +					 hte_ts_threaded_cb_t tcb, void *data)
> +{
> +	return -ENOTSUPP;
> +}
> +

Make line wrapping of hte_req_ts_by_hte_name() consistent with
devm_of_hte_request_ts().

Cheers,
Kent.

Re: [RFC v3 06/12] gpiolib: Add HTE support

[Kent Gibson]

On Tue, Nov 23, 2021 at 11:30:33AM -0800, Dipen Patel wrote:
> Some GPIO chip can provide hardware timestamp support on its GPIO lines
> , in order to support that additional API needs to be added which
> can talk to both GPIO chip and HTE (hardware timestamping engine)
> subsystem. This patch introduces APIs which gpio consumer can use
> to request hardware assisted timestamping. Below is the list of the APIs
> that are added in gpiolib subsystem.
> 
> - gpiod_req_hw_timestamp_ns - Request HTE on specified GPIO line.
> - gpiod_rel_hw_timestamp_ns - Release HTE functionality on GPIO line.
> 
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
> Reported-by: kernel test robot <lkp@intel.com>
> ---
> Changes in v2:
> - removed get timestamp and is timestamp enabled APIs
> 
>  drivers/gpio/gpiolib.c        | 73 +++++++++++++++++++++++++++++++++++
>  drivers/gpio/gpiolib.h        | 12 ++++++
>  include/linux/gpio/consumer.h | 19 ++++++++-
>  include/linux/gpio/driver.h   | 14 +++++++
>  4 files changed, 116 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpio/gpiolib.c b/drivers/gpio/gpiolib.c
> index abfbf546d159..46cba75c80e8 100644
> --- a/drivers/gpio/gpiolib.c
> +++ b/drivers/gpio/gpiolib.c
> @@ -1976,6 +1976,10 @@ static bool gpiod_free_commit(struct gpio_desc *desc)
>  			gc->free(gc, gpio_chip_hwgpio(desc));
>  			spin_lock_irqsave(&gpio_lock, flags);
>  		}
> +		spin_unlock_irqrestore(&gpio_lock, flags);
> +		gpiod_rel_hw_timestamp_ns(desc);
> +		spin_lock_irqsave(&gpio_lock, flags);
> +
>  		kfree_const(desc->label);
>  		desc_set_label(desc, NULL);
>  		clear_bit(FLAG_ACTIVE_LOW, &desc->flags);
> @@ -2388,6 +2392,75 @@ int gpiod_direction_output(struct gpio_desc *desc, int value)
>  }
>  EXPORT_SYMBOL_GPL(gpiod_direction_output);
>  
> +/**
> + * gpiod_req_hw_timestamp_ns - Enable the hardware assisted timestamp in
> + * nano second.
> + *

s/nano second/nanoseconds/g

> + * @desc: GPIO to enable
> + * @cb:	Callback, will be called when HTE pushes timestamp data.
> + * @tcb: Threaeded callback, it gets called from kernel thread context and when

s/Threaeded/Threaded/

> + * cb returns with HTE_RUN_THREADED_CB return value.
> + * @data: Client data, will be sent back with tcb and cb.
> + *
> + * Certain GPIO chip can rely on hardware assisted timestamp engines which can

Either drop the 'assisted' and refer to them as "hardware timestamping
engines" throughout, or rename your subsystem 'hate'?
Either way, be consistent.

> + * record timestamp at the occurance of the configured events
> + * i.e. rising/falling on specified GPIO lines. This is helper API to enable hw
> + * assisted timestamp in nano second.
> + *

Not sure this comment block adds anything.

> + * Return 0 in case of success, else an error code.
> + */
> +int gpiod_req_hw_timestamp_ns(struct gpio_desc *desc, hte_ts_cb_t cb,
> +			      hte_ts_threaded_cb_t tcb, void *data)
> +{
> +	struct gpio_chip *gc;
> +	int ret = 0;
> +
> +	VALIDATE_DESC(desc);
> +	gc = desc->gdev->chip;
> +
> +	if (!gc->req_hw_timestamp) {
> +		gpiod_warn(desc, "%s: hw ts not supported\n", __func__);
> +		return -ENOTSUPP;
> +	}
> +
> +	ret = gc->req_hw_timestamp(gc, gpio_chip_hwgpio(desc), cb, tcb,
> +				   &desc->hdesc, data);
> +	if (ret)
> +		gpiod_warn(desc, "%s: hw ts request failed\n", __func__);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(gpiod_req_hw_timestamp_ns);
> +
> +/**
> + * gpiod_rel_hw_timestamp_ns - Release and disable the hardware assisted
> + * timestamp.
> + *

Are gpiod_req_hw_timestamp_ns() and gpiod_rel_hw_timestamp_ns()
request/release or enable/disable?
You are using both descriptions in the documentation.
request/release implies resource allocation, while enable/disable does
not.  Which is it?

> + * @desc: GPIO to disable
> + *
> + * Return 0 in case of success, else an error code.
> + */
> +int gpiod_rel_hw_timestamp_ns(struct gpio_desc *desc)

Cheers,
Kent.

Re: [RFC v3 09/12] gpiolib: cdev: Add hardware timestamp clock type

[Kent Gibson]

On Tue, Nov 23, 2021 at 11:30:36AM -0800, Dipen Patel wrote:
> This patch adds new clock type for the GPIO controller which can
> timestamp gpio lines in realtime using hardware means. To expose such
> functionalities to the userspace, code has been added in this patch
> where during line create call, it checks for new clock type and if
> requested, calls hardware timestamp related API from gpiolib.c.
> During line change event, the HTE subsystem pushes timestamp data
> through callbacks.
> 
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> Acked-by: Linus Walleij <linus.walleij@linaro.org>
> ---
> Changes in v2:
> - Added hte_dir and static structure hte_ts_desc.
> - Added callbacks which get invoked by HTE when new data is available.
> - Better use of hte_dir and seq from hte_ts_desc.
> - Modified sw debounce function to accommodate hardware timestamping.
> 
>  drivers/gpio/gpiolib-cdev.c | 161 ++++++++++++++++++++++++++++++++++--
>  include/uapi/linux/gpio.h   |   1 +
>  2 files changed, 153 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
> index c7b5446d01fd..1736ad54e3ec 100644
> --- a/drivers/gpio/gpiolib-cdev.c
> +++ b/drivers/gpio/gpiolib-cdev.c
> @@ -464,6 +464,12 @@ struct line {
>  	 * stale value.
>  	 */
>  	unsigned int level;
> +	/*
> +	 * dir will be touched in HTE callbacks hte_ts_cb_t and
> +	 * hte_ts_threaded_cb_t and they are mutually exclusive. This will be
> +	 * unused when HTE is not supported/disabled.
> +	 */
> +	enum hte_dir dir;
>  };
>  

Documentation should be in present tense, so 

s/will be/is/g

Same applies to other patches.

Also

s/touched/accessed/

dir is a poor name for the field.  It is the hte edge direction and
effectively the line level, so call it hte_edge_dirn or
hte_edge_direction or hte_level.

And it is placed in a section of the struct documented as "debouncer specific
fields", but it is not specfic to the debouncer.  Add a "hte specific
fields" section if nothing else is suitable.

>  /**
> @@ -518,6 +524,7 @@ struct linereq {
>  	 GPIO_V2_LINE_DRIVE_FLAGS | \
>  	 GPIO_V2_LINE_EDGE_FLAGS | \
>  	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
> +	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
>  	 GPIO_V2_LINE_BIAS_FLAGS)
>  

>  static void linereq_put_event(struct linereq *lr,
> @@ -546,6 +553,94 @@ static u64 line_event_timestamp(struct line *line)
>  	return ktime_get_ns();
>  }
>  
> +static hte_return_t process_hw_ts_thread(void *p)
> +{
> +	struct line *line = p;
> +	struct linereq *lr = line->req;
> +	struct gpio_v2_line_event le;
> +	u64 eflags;
> +
> +	memset(&le, 0, sizeof(le));
> +
> +	le.timestamp_ns = line->timestamp_ns;
> +	line->timestamp_ns = 0;
> +

What is the purpose of this zeroing?

> +	if (line->dir >= HTE_DIR_NOSUPP) {
> +		eflags = READ_ONCE(line->eflags);
> +		if (eflags == GPIO_V2_LINE_FLAG_EDGE_BOTH) {
> +			int level = gpiod_get_value_cansleep(line->desc);
> +
> +			if (level)
> +				/* Emit low-to-high event */
> +				le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> +			else
> +				/* Emit high-to-low event */
> +				le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_RISING) {
> +			/* Emit low-to-high event */
> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_FALLING) {
> +			/* Emit high-to-low event */
> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> +		} else {
> +			return HTE_CB_ERROR;
> +		}
> +	} else {
> +		if (line->dir == HTE_RISING_EDGE_TS)
> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> +		else
> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> +	}

The mapping from line->dir to le.id needs to take into account the active
low setting for the line.

And it might be simpler if the hte_ts_data provided the level, equivalent
to gpiod_get_raw_value_cansleep(), rather than an edge direction, so you
can provide a common helper to determine the edge given the raw level.

> +
> +	le.line_seqno = line->line_seqno;
> +	le.seqno = (lr->num_lines == 1) ? le.line_seqno : line->req_seqno;
> +	le.offset = gpio_chip_hwgpio(line->desc);
> +
> +	linereq_put_event(lr, &le);
> +
> +	return HTE_CB_HANDLED;
> +}
> +
> +static hte_return_t process_hw_ts(struct hte_ts_data *ts, void *p)
> +{
> +	struct line *line = p;
> +	struct linereq *lr = line->req;
> +
> +	if (!ts)
> +		return HTE_CB_ERROR;
> +
> +	line->timestamp_ns = ts->tsc;
> +	line->dir = ts->dir;
> +

The doc for timestamp_ns states:

	 * timestamp_ns and req_seqno are accessed only by
	 * edge_irq_handler() and edge_irq_thread(), which are themselves
	 * mutually exclusive, so no additional protection is necessary.

That no longer holds.  It is now also accessed here, and in
process_hw_ts_thread(), which wont run concurrently with each other or
the edge_irq_* handlers, but also in debounce_work_func() which may run
concurrently with the others.
So timestamp_ns now requires protection from concurrent access.

> +	/*
> +	 * It is possible that HTE engine detects spurious edges for the
> +	 * lines where software debounce is enabled. This primary callback
> +	 * will be called multiple times in that case. It will be better to
> +	 * let debounce_work_func handle instead of process_hw_ts_thread.
> +	 * The timestamp_ns will be overwritten here which is fine as we are
> +	 * interested in the last value anyway. The debounce_work_func will
> +	 * then just read whatever last line->timestamp_ns is stored. Because
> +	 * this callback can be called multiple times, we are not really
> +	 * interested in ts->seq.
> +	 */

Not sure what this is trying to say.
Is this the primary callback? Or debounce_irq_handler()?
You say you really aren't interested in ts->seq, but the code immediately
uses it.

Reword to clarify.
And add braces after function names to highlight them, so
debounce_work_func().

> +	if (!READ_ONCE(line->sw_debounced)) {
> +		line->line_seqno = ts->seq;
> +
> +		/*
> +		 * Increment in this callback incase all the lines in linereq
> +		 * are enabled for hw timestamping. This will work even if
> +		 * subset of lines are enabled for hw timestamping as
> +		 * edge_irq_* callbacks will proceed as usual for them.
> +		 */

s/incase/in case/

Not sure what the comment is trying to say. There is no check here that
the other lines have HTE enabled.  And that is not relevant anyway.
The edge_irq_* handlers will proceed as usual for those lines NOT
enabled for hw timestamping.

To clarify, the line_seqno indicates where this event lies in the
sequence of events for the line.
The request seqno indicates where this event lines in the sequence of
events for the request.
For a single line request these are the same, hence the minor
optimisation of not updating lr->seqno below.

> +		if (lr->num_lines != 1)
> +			line->req_seqno = atomic_inc_return(&lr->seqno);
> +

The req_seqno should be updated corresponding to the change in the
line_reqno.  That always used to be 1, but no longer if hte can discard
events, i.e. skip over line_seqnos.
To be consistent, i.e. if events were lost for this line then they were
also lost for the requested lines, the lr->seqno should be incremented by
the change in line_seqno.  Probably with some sanity checks.

> +		return HTE_RUN_THREADED_CB;
> +	}
> +
> +	return HTE_CB_HANDLED;
> +}
> +
>  static irqreturn_t edge_irq_thread(int irq, void *p)
>  {
>  	struct line *line = p;
> @@ -553,6 +648,10 @@ static irqreturn_t edge_irq_thread(int irq, void *p)
>  	struct gpio_v2_line_event le;
>  	u64 eflags;
>  
> +	/* Let process_hw_ts_thread handle */
> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
> +		return IRQ_HANDLED;
> +

This adds pointless runtime overhead, and for everyone not just hte users.
Don't stub out a handler in the handler - stub it out where it is
registered by registering a stub handler.  Or don't request it at all.

So why would gpiolib-cdev be requesting the irq, only to stub out
the handlers?
If that has a side-effect that hte requires then hte should be taking
care of it - it is not gpiolib-cdev's problem.

And speaking as to how the whole hte/gpiolib-cdev interface should work,
hte should be an edge event generator alternative to irq.  So lines with
hte enabled should work without any irq calls from gpiolib-cdev.
That includes the sw debouncer - more on that below.

>  	/* Do not leak kernel stack to userspace */
>  	memset(&le, 0, sizeof(le));
>  
> @@ -604,6 +703,10 @@ static irqreturn_t edge_irq_handler(int irq, void *p)
>  	struct line *line = p;
>  	struct linereq *lr = line->req;
>  
> +	/* Let HTE supplied callbacks handle */
> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
> +		return IRQ_HANDLED;
> +
>  	/*
>  	 * Just store the timestamp in hardirq context so we get it as
>  	 * close in time as possible to the actual event.
> @@ -682,14 +785,6 @@ static void debounce_work_func(struct work_struct *work)
>  	/* Do not leak kernel stack to userspace */
>  	memset(&le, 0, sizeof(le));
>  
> -	lr = line->req;
> -	le.timestamp_ns = line_event_timestamp(line);
> -	le.offset = gpio_chip_hwgpio(line->desc);
> -	line->line_seqno++;
> -	le.line_seqno = line->line_seqno;
> -	le.seqno = (lr->num_lines == 1) ?
> -		le.line_seqno : atomic_inc_return(&lr->seqno);
> -
>  	if (level)
>  		/* Emit low-to-high event */
>  		le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> @@ -697,6 +792,23 @@ static void debounce_work_func(struct work_struct *work)
>  		/* Emit high-to-low event */
>  		le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>  
> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
> +		le.timestamp_ns = line->timestamp_ns;
> +		if (line->dir < HTE_DIR_NOSUPP)
> +			le.id = (line->dir == HTE_RISING_EDGE_TS) ?
> +				 GPIO_V2_LINE_EVENT_RISING_EDGE :
> +				 GPIO_V2_LINE_EVENT_FALLING_EDGE;
> +	} else {
> +		le.timestamp_ns = line_event_timestamp(line);
> +	}
> +

Move the FLAG_EVENT_CLOCK_HARDWARE check into line_event_timestamp().

And the id fudging is necessary because the level returned by
gpiod_get_raw_value_cansleep() can disagree with the level from hte?
So you are still trying to synchronise events from two streams.
And that is still broken.
If a hte event occurs between the level being sampled by
gpiod_get_raw_value_cansleep() and the line->dir being read then the line
will have toggled and you will be reporting the opposite state than the
one the debouncer determined was stable.  And maybe the wrong timestamp as
well.

For lines where hte is enabled, the hte should be the source of level for
the debouncer, not the raw value.  And the mod_delayed_work() that
drives the debouncer should be called by a hte handler, not an irq handler.

There is also a race on reading the hte timestamp (line->timestamp_ns) and
the hte level (line->dir), such that you can get the level from one event
the timestamp from another.

> +	lr = line->req;
> +	le.offset = gpio_chip_hwgpio(line->desc);
> +	line->line_seqno++;
> +	le.line_seqno = line->line_seqno;
> +	le.seqno = (lr->num_lines == 1) ?
> +		le.line_seqno : atomic_inc_return(&lr->seqno);
> +

What is the purpose of moving this block of code moved from before the
if (level)?


>  	linereq_put_event(lr, &le);
>  }
>  
> @@ -891,7 +1003,6 @@ static int gpio_v2_line_flags_validate(u64 flags)
>  	/* Return an error if an unknown flag is set */
>  	if (flags & ~GPIO_V2_LINE_VALID_FLAGS)
>  		return -EINVAL;
> -

Gratuitous whitespace change.

>  	/*
>  	 * Do not allow both INPUT and OUTPUT flags to be set as they are
>  	 * contradictory.
> @@ -900,6 +1011,11 @@ static int gpio_v2_line_flags_validate(u64 flags)
>  	    (flags & GPIO_V2_LINE_FLAG_OUTPUT))
>  		return -EINVAL;
>  
> +	/* Only allow one event clock source */
> +	if ((flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME) &&
> +	    (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE))
> +		return -EINVAL;
> +
>  	/* Edge detection requires explicit input. */
>  	if ((flags & GPIO_V2_LINE_EDGE_FLAGS) &&
>  	    !(flags & GPIO_V2_LINE_FLAG_INPUT))
> @@ -992,6 +1108,8 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 flags,
>  
>  	assign_bit(FLAG_EVENT_CLOCK_REALTIME, flagsp,
>  		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME);
> +	assign_bit(FLAG_EVENT_CLOCK_HARDWARE, flagsp,
> +		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE);
>  }
>  
>  static long linereq_get_values(struct linereq *lr, void __user *ip)
> @@ -1154,6 +1272,21 @@ static long linereq_set_config_unlocked(struct linereq *lr,
>  				return ret;
>  		}
>  
> +		/* Check if new config sets hardware assisted clock */
> +		if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
> +			ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
> +							process_hw_ts_thread,
> +							&lr->lines[i]);
> +			if (ret)
> +				return ret;

Note that the line config is the complete line config, not a delta.

What happens when a line that already has hte enabled is reconfigured
and still has hte enabled?  i.e. what happens when
gpiod_req_hw_timestamp_ns() is called for the second time?

You provide a comment for the release case below, what of the request
case?

If you need to check for change then compare the old and new flags, as
the polarity_change check does (not visible in the diff here).

> +		} else {
> +			/*
> +			 * HTE subsys will do nothing if there is nothing to
> +			 * release.
> +			 */
> +			gpiod_rel_hw_timestamp_ns(desc);
> +		}
> +

Comment will fit on one line.

And it would be better to document that the function is idempotent in the
function documentation, not everywhere it is used.

>  		blocking_notifier_call_chain(&desc->gdev->notifier,
>  					     GPIO_V2_LINE_CHANGED_CONFIG,
>  					     desc);
> @@ -1409,6 +1542,14 @@ static int linereq_create(struct gpio_device *gdev, void __user *ip)
>  					flags & GPIO_V2_LINE_EDGE_FLAGS);
>  			if (ret)
>  				goto out_free_linereq;
> +
> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
> +				ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
> +							process_hw_ts_thread,
> +							&lr->lines[i]);
> +				if (ret)
> +					goto out_free_linereq;
> +			}
>  		}
>  
>  		blocking_notifier_call_chain(&desc->gdev->notifier,
> @@ -1959,6 +2100,8 @@ static void gpio_desc_to_lineinfo(struct gpio_desc *desc,
>  
>  	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &desc->flags))
>  		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
> +	else if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &desc->flags))
> +		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
>  
>  	debounce_period_us = READ_ONCE(desc->debounce_period_us);
>  	if (debounce_period_us) {
> diff --git a/include/uapi/linux/gpio.h b/include/uapi/linux/gpio.h
> index eaaea3d8e6b4..d360545b4c21 100644
> --- a/include/uapi/linux/gpio.h
> +++ b/include/uapi/linux/gpio.h
> @@ -80,6 +80,7 @@ enum gpio_v2_line_flag {
>  	GPIO_V2_LINE_FLAG_BIAS_PULL_DOWN	= _BITULL(9),
>  	GPIO_V2_LINE_FLAG_BIAS_DISABLED		= _BITULL(10),
>  	GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME	= _BITULL(11),
> +	GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE	= _BITULL(12),
>  };
>  

I'm now thinking this name, "HARDWARE" is too vague, in case other
timestamp source alternatives join the fray, and so should be "HTE".

Cheers,
Kent.

Re: [RFC v3 07/12] dt-bindings: gpio: Add hardware-timestamp-engine property

[Rob Herring]

On Tue, Nov 23, 2021 at 11:30:34AM -0800, Dipen Patel wrote:
> Some GPIO controllers can timestamp GPIO lines in real time using
> hardware timestamp engine. The optional property is added to facilitate
> this feature.
> 
> The nvidia GPIO controller has hardware timestamp engine as a backend
> to timestamp its GPIO lines in a real time. This change set adds
> hardware-timestamp-engine property to reflect that fact.
> 
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> ---
>  Documentation/devicetree/bindings/gpio/gpio.txt           | 8 ++++++++
>  .../devicetree/bindings/gpio/nvidia,tegra186-gpio.txt     | 7 +++++++
>  2 files changed, 15 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/gpio/gpio.txt b/Documentation/devicetree/bindings/gpio/gpio.txt
> index a8895d339bfe..c6ae2eaf5762 100644
> --- a/Documentation/devicetree/bindings/gpio/gpio.txt
> +++ b/Documentation/devicetree/bindings/gpio/gpio.txt
> @@ -167,6 +167,13 @@ left to right from the passed array. An incomplete array (where the number
>  of passed named are less than ngpios) will still be used up until the last
>  provided valid line index.
>  
> +Optionally, a GPIO controller may have a "hardware-timestamp-engine" property.
> +This specifies the timestamp engine node. The property is only useful if the
> +hardware timestamp engine (HTE) serves as a back-end to support hardware
> +timestamp GPIO lines in a real time and GPIO controller has dependency on the
> +engine. It will be up to GPIO controller and HTE provider drivers to figure out
> +the mapping between GPIO controller and HTE namespaces of a given GPIO line.

This needs to be added to schema instead (in dtschema).

I find the 'hardware-' part to be redundant. What other type of 
timestamp engine are we going to have in the h/w description?

> +
>  Example:
>  
>  gpio-controller@00000000 {
> @@ -180,6 +187,7 @@ gpio-controller@00000000 {
>  		"LED G", "LED B", "Col A", "Col B", "Col C", "Col D",
>  		"Row A", "Row B", "Row C", "Row D", "NMI button",
>  		"poweroff", "reset";
> +	hardware-timestamp-engine = <&hadrware_ts_engine_node>;
>  }
>  
>  The GPIO chip may contain GPIO hog definitions. GPIO hogging is a mechanism
> diff --git a/Documentation/devicetree/bindings/gpio/nvidia,tegra186-gpio.txt b/Documentation/devicetree/bindings/gpio/nvidia,tegra186-gpio.txt
> index adff16c71d21..20f6c9e69839 100644
> --- a/Documentation/devicetree/bindings/gpio/nvidia,tegra186-gpio.txt
> +++ b/Documentation/devicetree/bindings/gpio/nvidia,tegra186-gpio.txt
> @@ -127,6 +127,12 @@ Required properties:
>              - 8: Active low level-sensitive.
>              Valid combinations are 1, 2, 3, 4, 8.
>  
> +Optional properties:
> +- hardware-timestamp-engine
> +    AON GPIO controller has timestamp engine which can hardware timestamp
> +    GPIO configured as input and IRQ. This property specifies hardware
> +    timestamp engine (HTE) device-tree node.
> +
>  Example:
>  
>  #include <dt-bindings/interrupt-controller/irq.h>
> @@ -162,4 +168,5 @@ gpio@c2f0000 {
>  	#gpio-cells = <2>;
>  	interrupt-controller;
>  	#interrupt-cells = <2>;
> +	hardware-timestamp-engine = <&tegra_hte_aon>;
>  };
> -- 
> 2.17.1
> 
> 

Re: [RFC v3 09/12] gpiolib: cdev: Add hardware timestamp clock type

[Dipen Patel]

Hi,

On 11/25/21 5:31 PM, Kent Gibson wrote:
> On Tue, Nov 23, 2021 at 11:30:36AM -0800, Dipen Patel wrote:
>> This patch adds new clock type for the GPIO controller which can
>> timestamp gpio lines in realtime using hardware means. To expose such
>> functionalities to the userspace, code has been added in this patch
>> where during line create call, it checks for new clock type and if
>> requested, calls hardware timestamp related API from gpiolib.c.
>> During line change event, the HTE subsystem pushes timestamp data
>> through callbacks.
>>
>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>> Acked-by: Linus Walleij <linus.walleij@linaro.org>
>> ---
>> Changes in v2:
>> - Added hte_dir and static structure hte_ts_desc.
>> - Added callbacks which get invoked by HTE when new data is available.
>> - Better use of hte_dir and seq from hte_ts_desc.
>> - Modified sw debounce function to accommodate hardware timestamping.
>>
>>  drivers/gpio/gpiolib-cdev.c | 161 ++++++++++++++++++++++++++++++++++--
>>  include/uapi/linux/gpio.h   |   1 +
>>  2 files changed, 153 insertions(+), 9 deletions(-)
>>
>> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
>> index c7b5446d01fd..1736ad54e3ec 100644
>> --- a/drivers/gpio/gpiolib-cdev.c
>> +++ b/drivers/gpio/gpiolib-cdev.c
>> @@ -464,6 +464,12 @@ struct line {
>>  	 * stale value.
>>  	 */
>>  	unsigned int level;
>> +	/*
>> +	 * dir will be touched in HTE callbacks hte_ts_cb_t and
>> +	 * hte_ts_threaded_cb_t and they are mutually exclusive. This will be
>> +	 * unused when HTE is not supported/disabled.
>> +	 */
>> +	enum hte_dir dir;
>>  };
>>  
> Documentation should be in present tense, so 
>
> s/will be/is/g
>
> Same applies to other patches.
>
> Also
>
> s/touched/accessed/
>
> dir is a poor name for the field.  It is the hte edge direction and
> effectively the line level, so call it hte_edge_dirn or
> hte_edge_direction or hte_level.
>
> And it is placed in a section of the struct documented as "debouncer specific
> fields", but it is not specfic to the debouncer.  Add a "hte specific
> fields" section if nothing else is suitable.
>
>>  /**
>> @@ -518,6 +524,7 @@ struct linereq {
>>  	 GPIO_V2_LINE_DRIVE_FLAGS | \
>>  	 GPIO_V2_LINE_EDGE_FLAGS | \
>>  	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
>> +	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
>>  	 GPIO_V2_LINE_BIAS_FLAGS)
>>  
>>  static void linereq_put_event(struct linereq *lr,
>> @@ -546,6 +553,94 @@ static u64 line_event_timestamp(struct line *line)
>>  	return ktime_get_ns();
>>  }
>>  
>> +static hte_return_t process_hw_ts_thread(void *p)
>> +{
>> +	struct line *line = p;
>> +	struct linereq *lr = line->req;
>> +	struct gpio_v2_line_event le;
>> +	u64 eflags;
>> +
>> +	memset(&le, 0, sizeof(le));
>> +
>> +	le.timestamp_ns = line->timestamp_ns;
>> +	line->timestamp_ns = 0;
>> +
> What is the purpose of this zeroing?
>
>> +	if (line->dir >= HTE_DIR_NOSUPP) {
>> +		eflags = READ_ONCE(line->eflags);
>> +		if (eflags == GPIO_V2_LINE_FLAG_EDGE_BOTH) {
>> +			int level = gpiod_get_value_cansleep(line->desc);
>> +
>> +			if (level)
>> +				/* Emit low-to-high event */
>> +				le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>> +			else
>> +				/* Emit high-to-low event */
>> +				le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_RISING) {
>> +			/* Emit low-to-high event */
>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_FALLING) {
>> +			/* Emit high-to-low event */
>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>> +		} else {
>> +			return HTE_CB_ERROR;
>> +		}
>> +	} else {
>> +		if (line->dir == HTE_RISING_EDGE_TS)
>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>> +		else
>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>> +	}
> The mapping from line->dir to le.id needs to take into account the active
> low setting for the line.
>
> And it might be simpler if the hte_ts_data provided the level, equivalent
> to gpiod_get_raw_value_cansleep(), rather than an edge direction, so you
> can provide a common helper to determine the edge given the raw level.

(So from the level determine the edge?) that sound right specially when

HTE provider has capability to record the edge in that case why bother

getting the level and determine edge?

Calculating the edge from the level makes sense when hte provider does not

have that feature and that is what if (line->dir >= HTE_DIR_NOSUPP) does.

>
>> +
>> +	le.line_seqno = line->line_seqno;
>> +	le.seqno = (lr->num_lines == 1) ? le.line_seqno : line->req_seqno;
>> +	le.offset = gpio_chip_hwgpio(line->desc);
>> +
>> +	linereq_put_event(lr, &le);
>> +
>> +	return HTE_CB_HANDLED;
>> +}
>> +
>> +static hte_return_t process_hw_ts(struct hte_ts_data *ts, void *p)
>> +{
>> +	struct line *line = p;
>> +	struct linereq *lr = line->req;
>> +
>> +	if (!ts)
>> +		return HTE_CB_ERROR;
>> +
>> +	line->timestamp_ns = ts->tsc;
>> +	line->dir = ts->dir;
>> +
> The doc for timestamp_ns states:
>
> 	 * timestamp_ns and req_seqno are accessed only by
> 	 * edge_irq_handler() and edge_irq_thread(), which are themselves
> 	 * mutually exclusive, so no additional protection is necessary.
>
> That no longer holds.  It is now also accessed here, and in
> process_hw_ts_thread(), which wont run concurrently with each other or
> the edge_irq_* handlers, but also in debounce_work_func() which may run
> concurrently with the others.
> So timestamp_ns now requires protection from concurrent access.
>
>> +	/*
>> +	 * It is possible that HTE engine detects spurious edges for the
>> +	 * lines where software debounce is enabled. This primary callback
>> +	 * will be called multiple times in that case. It will be better to
>> +	 * let debounce_work_func handle instead of process_hw_ts_thread.
>> +	 * The timestamp_ns will be overwritten here which is fine as we are
>> +	 * interested in the last value anyway. The debounce_work_func will
>> +	 * then just read whatever last line->timestamp_ns is stored. Because
>> +	 * this callback can be called multiple times, we are not really
>> +	 * interested in ts->seq.
>> +	 */
> Not sure what this is trying to say.
> Is this the primary callback? Or debounce_irq_handler()?

This is primary callback called from HTE when it pushes new TS data per line, it

also says so in the second line.

> You say you really aren't interested in ts->seq, but the code immediately
> uses it.

That is when sw_debounced is not set and whole paragraph is about when

sw_debounced is set.

>
> Reword to clarify.
> And add braces after function names to highlight them, so
> debounce_work_func().
Will do.
>
>> +	if (!READ_ONCE(line->sw_debounced)) {
>> +		line->line_seqno = ts->seq;
>> +
>> +		/*
>> +		 * Increment in this callback incase all the lines in linereq
>> +		 * are enabled for hw timestamping. This will work even if
>> +		 * subset of lines are enabled for hw timestamping as
>> +		 * edge_irq_* callbacks will proceed as usual for them.
>> +		 */
> s/incase/in case/
>
> Not sure what the comment is trying to say. There is no check here that
> the other lines have HTE enabled.  And that is not relevant anyway.
> The edge_irq_* handlers will proceed as usual for those lines NOT
> enabled for hw timestamping.
>
> To clarify, the line_seqno indicates where this event lies in the
> sequence of events for the line.
> The request seqno indicates where this event lines in the sequence of
> events for the request.
> For a single line request these are the same, hence the minor
> optimisation of not updating lr->seqno below.
>
>> +		if (lr->num_lines != 1)
>> +			line->req_seqno = atomic_inc_return(&lr->seqno);
>> +
> The req_seqno should be updated corresponding to the change in the
> line_reqno.  That always used to be 1, but no longer if hte can discard
> events, i.e. skip over line_seqnos.

HTE does not discard any events, it pushes to clients as soon as its

available through primary callback.

> To be consistent, i.e. if events were lost for this line then they were
> also lost for the requested lines, the lr->seqno should be incremented by
> the change in line_seqno.  Probably with some sanity checks.
>
>> +		return HTE_RUN_THREADED_CB;
>> +	}
>> +
>> +	return HTE_CB_HANDLED;
>> +}
>> +
>>  static irqreturn_t edge_irq_thread(int irq, void *p)
>>  {
>>  	struct line *line = p;
>> @@ -553,6 +648,10 @@ static irqreturn_t edge_irq_thread(int irq, void *p)
>>  	struct gpio_v2_line_event le;
>>  	u64 eflags;
>>  
>> +	/* Let process_hw_ts_thread handle */
>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
>> +		return IRQ_HANDLED;
>> +
> This adds pointless runtime overhead, and for everyone not just hte users.
> Don't stub out a handler in the handler - stub it out where it is
> registered by registering a stub handler.  Or don't request it at all.
>
> So why would gpiolib-cdev be requesting the irq, only to stub out
> the handlers?
> If that has a side-effect that hte requires then hte should be taking
> care of it - it is not gpiolib-cdev's problem.

- Why stop at moving irq and debounce related stuff to hte then?

I mean if there is hte provider which can TS GPIO output/input

does it mean hte is responsible for parsing the GPIO line configs, setting them up

(i.e. input or output) as well? Are we not duplicating logic instead of

leveraging gpio-cdev? Does it make sense for the HTE subsystem which not

only TS the GPIOs but other SoC lines?

- What happens to in kernel GPIO HTE client (for example, hte-tegra194-gpio-test.c)?

some clients do more in their IRQ handler than what edge_irq_handler does in which

case it would make sense to have them request irq in their code than through HTE.

>
> And speaking as to how the whole hte/gpiolib-cdev interface should work,
> hte should be an edge event generator alternative to irq.  So lines with
> hte enabled should work without any irq calls from gpiolib-cdev.
> That includes the sw debouncer - more on that below.
>
>>  	/* Do not leak kernel stack to userspace */
>>  	memset(&le, 0, sizeof(le));
>>  
>> @@ -604,6 +703,10 @@ static irqreturn_t edge_irq_handler(int irq, void *p)
>>  	struct line *line = p;
>>  	struct linereq *lr = line->req;
>>  
>> +	/* Let HTE supplied callbacks handle */
>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
>> +		return IRQ_HANDLED;
>> +
>>  	/*
>>  	 * Just store the timestamp in hardirq context so we get it as
>>  	 * close in time as possible to the actual event.
>> @@ -682,14 +785,6 @@ static void debounce_work_func(struct work_struct *work)
>>  	/* Do not leak kernel stack to userspace */
>>  	memset(&le, 0, sizeof(le));
>>  
>> -	lr = line->req;
>> -	le.timestamp_ns = line_event_timestamp(line);
>> -	le.offset = gpio_chip_hwgpio(line->desc);
>> -	line->line_seqno++;
>> -	le.line_seqno = line->line_seqno;
>> -	le.seqno = (lr->num_lines == 1) ?
>> -		le.line_seqno : atomic_inc_return(&lr->seqno);
>> -
>>  	if (level)
>>  		/* Emit low-to-high event */
>>  		le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>> @@ -697,6 +792,23 @@ static void debounce_work_func(struct work_struct *work)
>>  		/* Emit high-to-low event */
>>  		le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>  
>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
>> +		le.timestamp_ns = line->timestamp_ns;
>> +		if (line->dir < HTE_DIR_NOSUPP)
>> +			le.id = (line->dir == HTE_RISING_EDGE_TS) ?
>> +				 GPIO_V2_LINE_EVENT_RISING_EDGE :
>> +				 GPIO_V2_LINE_EVENT_FALLING_EDGE;
>> +	} else {
>> +		le.timestamp_ns = line_event_timestamp(line);
>> +	}
>> +
> Move the FLAG_EVENT_CLOCK_HARDWARE check into line_event_timestamp().
>
> And the id fudging is necessary because the level returned by
> gpiod_get_raw_value_cansleep() can disagree with the level from hte?

> So you are still trying to synchronise events from two streams.
> And that is still broken.
> If a hte event occurs between the level being sampled by
> gpiod_get_raw_value_cansleep() and the line->dir being read then the line
> will have toggled and you will be reporting the opposite state than the
> one the debouncer determined was stable.  And maybe the wrong timestamp as
> well.
>
> For lines where hte is enabled, the hte should be the source of level for
> the debouncer, not the raw value.  And the mod_delayed_work() that
> drives the debouncer should be called by a hte handler, not an irq handler.
>
> There is also a race on reading the hte timestamp (line->timestamp_ns) and
> the hte level (line->dir), such that you can get the level from one event
> the timestamp from another.
>
>> +	lr = line->req;
>> +	le.offset = gpio_chip_hwgpio(line->desc);
>> +	line->line_seqno++;
>> +	le.line_seqno = line->line_seqno;
>> +	le.seqno = (lr->num_lines == 1) ?
>> +		le.line_seqno : atomic_inc_return(&lr->seqno);
>> +
> What is the purpose of moving this block of code moved from before the
> if (level)?
>
>
>>  	linereq_put_event(lr, &le);
>>  }
>>  
>> @@ -891,7 +1003,6 @@ static int gpio_v2_line_flags_validate(u64 flags)
>>  	/* Return an error if an unknown flag is set */
>>  	if (flags & ~GPIO_V2_LINE_VALID_FLAGS)
>>  		return -EINVAL;
>> -
> Gratuitous whitespace change.
>
>>  	/*
>>  	 * Do not allow both INPUT and OUTPUT flags to be set as they are
>>  	 * contradictory.
>> @@ -900,6 +1011,11 @@ static int gpio_v2_line_flags_validate(u64 flags)
>>  	    (flags & GPIO_V2_LINE_FLAG_OUTPUT))
>>  		return -EINVAL;
>>  
>> +	/* Only allow one event clock source */
>> +	if ((flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME) &&
>> +	    (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE))
>> +		return -EINVAL;
>> +
>>  	/* Edge detection requires explicit input. */
>>  	if ((flags & GPIO_V2_LINE_EDGE_FLAGS) &&
>>  	    !(flags & GPIO_V2_LINE_FLAG_INPUT))
>> @@ -992,6 +1108,8 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 flags,
>>  
>>  	assign_bit(FLAG_EVENT_CLOCK_REALTIME, flagsp,
>>  		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME);
>> +	assign_bit(FLAG_EVENT_CLOCK_HARDWARE, flagsp,
>> +		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE);
>>  }
>>  
>>  static long linereq_get_values(struct linereq *lr, void __user *ip)
>> @@ -1154,6 +1272,21 @@ static long linereq_set_config_unlocked(struct linereq *lr,
>>  				return ret;
>>  		}
>>  
>> +		/* Check if new config sets hardware assisted clock */
>> +		if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
>> +			ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
>> +							process_hw_ts_thread,
>> +							&lr->lines[i]);
>> +			if (ret)
>> +				return ret;
> Note that the line config is the complete line config, not a delta.
>
> What happens when a line that already has hte enabled is reconfigured
> and still has hte enabled?  i.e. what happens when
> gpiod_req_hw_timestamp_ns() is called for the second time?

HTE will return without doing anything with error code.

>
> You provide a comment for the release case below, what of the request
> case?
>
> If you need to check for change then compare the old and new flags, as
> the polarity_change check does (not visible in the diff here).
>
>> +		} else {
>> +			/*
>> +			 * HTE subsys will do nothing if there is nothing to
>> +			 * release.
>> +			 */
>> +			gpiod_rel_hw_timestamp_ns(desc);
>> +		}
>> +
> Comment will fit on one line.
>
> And it would be better to document that the function is idempotent in the
> function documentation, not everywhere it is used.
>
>>  		blocking_notifier_call_chain(&desc->gdev->notifier,
>>  					     GPIO_V2_LINE_CHANGED_CONFIG,
>>  					     desc);
>> @@ -1409,6 +1542,14 @@ static int linereq_create(struct gpio_device *gdev, void __user *ip)
>>  					flags & GPIO_V2_LINE_EDGE_FLAGS);
>>  			if (ret)
>>  				goto out_free_linereq;
>> +
>> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
>> +				ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
>> +							process_hw_ts_thread,
>> +							&lr->lines[i]);
>> +				if (ret)
>> +					goto out_free_linereq;
>> +			}
>>  		}
>>  
>>  		blocking_notifier_call_chain(&desc->gdev->notifier,
>> @@ -1959,6 +2100,8 @@ static void gpio_desc_to_lineinfo(struct gpio_desc *desc,
>>  
>>  	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &desc->flags))
>>  		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
>> +	else if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &desc->flags))
>> +		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
>>  
>>  	debounce_period_us = READ_ONCE(desc->debounce_period_us);
>>  	if (debounce_period_us) {
>> diff --git a/include/uapi/linux/gpio.h b/include/uapi/linux/gpio.h
>> index eaaea3d8e6b4..d360545b4c21 100644
>> --- a/include/uapi/linux/gpio.h
>> +++ b/include/uapi/linux/gpio.h
>> @@ -80,6 +80,7 @@ enum gpio_v2_line_flag {
>>  	GPIO_V2_LINE_FLAG_BIAS_PULL_DOWN	= _BITULL(9),
>>  	GPIO_V2_LINE_FLAG_BIAS_DISABLED		= _BITULL(10),
>>  	GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME	= _BITULL(11),
>> +	GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE	= _BITULL(12),
>>  };
>>  
> I'm now thinking this name, "HARDWARE" is too vague, in case other
> timestamp source alternatives join the fray, and so should be "HTE".
>
> Cheers,
> Kent.

Re: [RFC v3 09/12] gpiolib: cdev: Add hardware timestamp clock type

[Kent Gibson]

On Tue, Nov 30, 2021 at 07:29:20PM -0800, Dipen Patel wrote:
> Hi,
> 
> On 11/25/21 5:31 PM, Kent Gibson wrote:
> > On Tue, Nov 23, 2021 at 11:30:36AM -0800, Dipen Patel wrote:
> >> This patch adds new clock type for the GPIO controller which can
> >> timestamp gpio lines in realtime using hardware means. To expose such
> >> functionalities to the userspace, code has been added in this patch
> >> where during line create call, it checks for new clock type and if
> >> requested, calls hardware timestamp related API from gpiolib.c.
> >> During line change event, the HTE subsystem pushes timestamp data
> >> through callbacks.
> >>
> >> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> >> Acked-by: Linus Walleij <linus.walleij@linaro.org>
> >> ---
> >> Changes in v2:
> >> - Added hte_dir and static structure hte_ts_desc.
> >> - Added callbacks which get invoked by HTE when new data is available.
> >> - Better use of hte_dir and seq from hte_ts_desc.
> >> - Modified sw debounce function to accommodate hardware timestamping.
> >>
> >>  drivers/gpio/gpiolib-cdev.c | 161 ++++++++++++++++++++++++++++++++++--
> >>  include/uapi/linux/gpio.h   |   1 +
> >>  2 files changed, 153 insertions(+), 9 deletions(-)
> >>
> >> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
> >> index c7b5446d01fd..1736ad54e3ec 100644
> >> --- a/drivers/gpio/gpiolib-cdev.c
> >> +++ b/drivers/gpio/gpiolib-cdev.c
> >> @@ -464,6 +464,12 @@ struct line {
> >>  	 * stale value.
> >>  	 */
> >>  	unsigned int level;
> >> +	/*
> >> +	 * dir will be touched in HTE callbacks hte_ts_cb_t and
> >> +	 * hte_ts_threaded_cb_t and they are mutually exclusive. This will be
> >> +	 * unused when HTE is not supported/disabled.
> >> +	 */
> >> +	enum hte_dir dir;
> >>  };
> >>  
> > Documentation should be in present tense, so 
> >
> > s/will be/is/g
> >
> > Same applies to other patches.
> >
> > Also
> >
> > s/touched/accessed/
> >
> > dir is a poor name for the field.  It is the hte edge direction and
> > effectively the line level, so call it hte_edge_dirn or
> > hte_edge_direction or hte_level.
> >
> > And it is placed in a section of the struct documented as "debouncer specific
> > fields", but it is not specfic to the debouncer.  Add a "hte specific
> > fields" section if nothing else is suitable.
> >
> >>  /**
> >> @@ -518,6 +524,7 @@ struct linereq {
> >>  	 GPIO_V2_LINE_DRIVE_FLAGS | \
> >>  	 GPIO_V2_LINE_EDGE_FLAGS | \
> >>  	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
> >> +	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
> >>  	 GPIO_V2_LINE_BIAS_FLAGS)
> >>  
> >>  static void linereq_put_event(struct linereq *lr,
> >> @@ -546,6 +553,94 @@ static u64 line_event_timestamp(struct line *line)
> >>  	return ktime_get_ns();
> >>  }
> >>  
> >> +static hte_return_t process_hw_ts_thread(void *p)
> >> +{
> >> +	struct line *line = p;
> >> +	struct linereq *lr = line->req;
> >> +	struct gpio_v2_line_event le;
> >> +	u64 eflags;
> >> +
> >> +	memset(&le, 0, sizeof(le));
> >> +
> >> +	le.timestamp_ns = line->timestamp_ns;
> >> +	line->timestamp_ns = 0;
> >> +
> > What is the purpose of this zeroing?
> >
> >> +	if (line->dir >= HTE_DIR_NOSUPP) {
> >> +		eflags = READ_ONCE(line->eflags);
> >> +		if (eflags == GPIO_V2_LINE_FLAG_EDGE_BOTH) {
> >> +			int level = gpiod_get_value_cansleep(line->desc);
> >> +
> >> +			if (level)
> >> +				/* Emit low-to-high event */
> >> +				le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> >> +			else
> >> +				/* Emit high-to-low event */
> >> +				le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_RISING) {
> >> +			/* Emit low-to-high event */
> >> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> >> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_FALLING) {
> >> +			/* Emit high-to-low event */
> >> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >> +		} else {
> >> +			return HTE_CB_ERROR;
> >> +		}
> >> +	} else {
> >> +		if (line->dir == HTE_RISING_EDGE_TS)
> >> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> >> +		else
> >> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >> +	}
> > The mapping from line->dir to le.id needs to take into account the active
> > low setting for the line.
> >
> > And it might be simpler if the hte_ts_data provided the level, equivalent
> > to gpiod_get_raw_value_cansleep(), rather than an edge direction, so you
> > can provide a common helper to determine the edge given the raw level.
> 
> (So from the level determine the edge?) that sound right specially when
> 
> HTE provider has capability to record the edge in that case why bother
> 
> getting the level and determine edge?
> 
> Calculating the edge from the level makes sense when hte provider does not
> 
> have that feature and that is what if (line->dir >= HTE_DIR_NOSUPP) does.
> 

As asked in the review of patch 02, do you have an example of hardware that
reports an edge direction rather than NOSUPP?

Anyway, this is just a naming thing - the information content being passed
is the the same, be it high/low/unknown or rising/falling/unknown.

If the hardware does report edge direction then it is just one bit, and
that also corresponds to the physical level immediately following the
edge, so no additional conversion required there.

It would be clearer to pass a level than an edge, as
 - a hte edge (hte_dir) could be confused with a cdev edge
   (gpio_v2_line_event_id), in mails like this if not in the code.
 - cdev will fallback to using the physical level to determine the edge
   if the hte can't provide it
 - cdev has to perform inversion if active low and it already does that
   based on levels

> >
> >> +
> >> +	le.line_seqno = line->line_seqno;
> >> +	le.seqno = (lr->num_lines == 1) ? le.line_seqno : line->req_seqno;
> >> +	le.offset = gpio_chip_hwgpio(line->desc);
> >> +
> >> +	linereq_put_event(lr, &le);
> >> +
> >> +	return HTE_CB_HANDLED;
> >> +}
> >> +
> >> +static hte_return_t process_hw_ts(struct hte_ts_data *ts, void *p)
> >> +{
> >> +	struct line *line = p;
> >> +	struct linereq *lr = line->req;
> >> +
> >> +	if (!ts)
> >> +		return HTE_CB_ERROR;
> >> +
> >> +	line->timestamp_ns = ts->tsc;
> >> +	line->dir = ts->dir;
> >> +
> > The doc for timestamp_ns states:
> >
> > 	 * timestamp_ns and req_seqno are accessed only by
> > 	 * edge_irq_handler() and edge_irq_thread(), which are themselves
> > 	 * mutually exclusive, so no additional protection is necessary.
> >
> > That no longer holds.  It is now also accessed here, and in
> > process_hw_ts_thread(), which wont run concurrently with each other or
> > the edge_irq_* handlers, but also in debounce_work_func() which may run
> > concurrently with the others.
> > So timestamp_ns now requires protection from concurrent access.
> >
> >> +	/*
> >> +	 * It is possible that HTE engine detects spurious edges for the
> >> +	 * lines where software debounce is enabled. This primary callback
> >> +	 * will be called multiple times in that case. It will be better to
> >> +	 * let debounce_work_func handle instead of process_hw_ts_thread.
> >> +	 * The timestamp_ns will be overwritten here which is fine as we are
> >> +	 * interested in the last value anyway. The debounce_work_func will
> >> +	 * then just read whatever last line->timestamp_ns is stored. Because
> >> +	 * this callback can be called multiple times, we are not really
> >> +	 * interested in ts->seq.
> >> +	 */
> > Not sure what this is trying to say.
> > Is this the primary callback? Or debounce_irq_handler()?
> 
> This is primary callback called from HTE when it pushes new TS data per line, it
> 
> also says so in the second line.
> 

Yeah, I probably read that as "The primary callback", but it is
confusing anyway. "This primary callback" implies there is another
primary callback. 
Just say "This handler" instead of "This primary callback".

> > You say you really aren't interested in ts->seq, but the code immediately
> > uses it.
> 
> That is when sw_debounced is not set and whole paragraph is about when
> 
> sw_debounced is set.
> 

So your whole comment here is about the else case?
Then either put the comment where the else would be, or better yet invert
the logic and return immediately if sw_debounced.

> >
> > Reword to clarify.
> > And add braces after function names to highlight them, so
> > debounce_work_func().
> Will do.
> >
> >> +	if (!READ_ONCE(line->sw_debounced)) {
> >> +		line->line_seqno = ts->seq;
> >> +
> >> +		/*
> >> +		 * Increment in this callback incase all the lines in linereq
> >> +		 * are enabled for hw timestamping. This will work even if
> >> +		 * subset of lines are enabled for hw timestamping as
> >> +		 * edge_irq_* callbacks will proceed as usual for them.
> >> +		 */
> > s/incase/in case/
> >
> > Not sure what the comment is trying to say. There is no check here that
> > the other lines have HTE enabled.  And that is not relevant anyway.
> > The edge_irq_* handlers will proceed as usual for those lines NOT
> > enabled for hw timestamping.
> >
> > To clarify, the line_seqno indicates where this event lies in the
> > sequence of events for the line.
> > The request seqno indicates where this event lines in the sequence of
> > events for the request.
> > For a single line request these are the same, hence the minor
> > optimisation of not updating lr->seqno below.
> >
> >> +		if (lr->num_lines != 1)
> >> +			line->req_seqno = atomic_inc_return(&lr->seqno);
> >> +
> > The req_seqno should be updated corresponding to the change in the
> > line_reqno.  That always used to be 1, but no longer if hte can discard
> > events, i.e. skip over line_seqnos.
> 
> HTE does not discard any events, it pushes to clients as soon as its
> 
> available through primary callback.

The discarding of events that I am referring to is from your previous
answers that indicated, to me anyway, that there could be gaps in the
ts->seq numbers if the hardware event FIFO overflowed.
Is that not the case?

And when you say "primary callback", both here and elsewhere, you mean the
cb parameter to gpiod_req_hw_timestamp_ns() and gc->req_hw_timestamp()?
Either way, be specific and name the function or parameter, or find a
better term than "primary callback".
In this case "the client's event handler" would be much clearer.

> 
> > To be consistent, i.e. if events were lost for this line then they were
> > also lost for the requested lines, the lr->seqno should be incremented by
> > the change in line_seqno.  Probably with some sanity checks.
> >
> >> +		return HTE_RUN_THREADED_CB;
> >> +	}
> >> +
> >> +	return HTE_CB_HANDLED;
> >> +}
> >> +
> >>  static irqreturn_t edge_irq_thread(int irq, void *p)
> >>  {
> >>  	struct line *line = p;
> >> @@ -553,6 +648,10 @@ static irqreturn_t edge_irq_thread(int irq, void *p)
> >>  	struct gpio_v2_line_event le;
> >>  	u64 eflags;
> >>  
> >> +	/* Let process_hw_ts_thread handle */
> >> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
> >> +		return IRQ_HANDLED;
> >> +
> > This adds pointless runtime overhead, and for everyone not just hte users.
> > Don't stub out a handler in the handler - stub it out where it is
> > registered by registering a stub handler.  Or don't request it at all.
> >
> > So why would gpiolib-cdev be requesting the irq, only to stub out
> > the handlers?
> > If that has a side-effect that hte requires then hte should be taking
> > care of it - it is not gpiolib-cdev's problem.
> 
> - Why stop at moving irq and debounce related stuff to hte then?
> 

How about you answer my question before asking your own?
Here I am only questioning why gpiolib-cdev is requesting an interrupt
for a hte line at all.  It has no reason to, so hte must have?
And where did I suggest moving the "debounce stuff" to hte?

What I am suggesting is called separation of concerns.  In this context
the intent is to look towards abstracting the edge event generation.
Having hte and irq entangled for no apparent reason makes doing that more
difficult than it needs to be, whereas keeping them separate greatly
simplifies identification of common code suitable for refactoring
subsequently.

Not sure what to call what you are suggesting.

> I mean if there is hte provider which can TS GPIO output/input
> 
> does it mean hte is responsible for parsing the GPIO line configs, setting them up
> 
> (i.e. input or output) as well? Are we not duplicating logic instead of
> 
> leveraging gpio-cdev? Does it make sense for the HTE subsystem which not
> 
> only TS the GPIOs but other SoC lines?
> 
> - What happens to in kernel GPIO HTE client (for example, hte-tegra194-gpio-test.c)?
> 
> some clients do more in their IRQ handler than what edge_irq_handler does in which
> 
> case it would make sense to have them request irq in their code than through HTE.
> 
> >
> > And speaking as to how the whole hte/gpiolib-cdev interface should work,
> > hte should be an edge event generator alternative to irq.  So lines with
> > hte enabled should work without any irq calls from gpiolib-cdev.
> > That includes the sw debouncer - more on that below.
> >
> >>  	/* Do not leak kernel stack to userspace */
> >>  	memset(&le, 0, sizeof(le));
> >>  
> >> @@ -604,6 +703,10 @@ static irqreturn_t edge_irq_handler(int irq, void *p)
> >>  	struct line *line = p;
> >>  	struct linereq *lr = line->req;
> >>  
> >> +	/* Let HTE supplied callbacks handle */
> >> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
> >> +		return IRQ_HANDLED;
> >> +
> >>  	/*
> >>  	 * Just store the timestamp in hardirq context so we get it as
> >>  	 * close in time as possible to the actual event.
> >> @@ -682,14 +785,6 @@ static void debounce_work_func(struct work_struct *work)
> >>  	/* Do not leak kernel stack to userspace */
> >>  	memset(&le, 0, sizeof(le));
> >>  
> >> -	lr = line->req;
> >> -	le.timestamp_ns = line_event_timestamp(line);
> >> -	le.offset = gpio_chip_hwgpio(line->desc);
> >> -	line->line_seqno++;
> >> -	le.line_seqno = line->line_seqno;
> >> -	le.seqno = (lr->num_lines == 1) ?
> >> -		le.line_seqno : atomic_inc_return(&lr->seqno);
> >> -
> >>  	if (level)
> >>  		/* Emit low-to-high event */
> >>  		le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> >> @@ -697,6 +792,23 @@ static void debounce_work_func(struct work_struct *work)
> >>  		/* Emit high-to-low event */
> >>  		le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >>  
> >> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
> >> +		le.timestamp_ns = line->timestamp_ns;
> >> +		if (line->dir < HTE_DIR_NOSUPP)
> >> +			le.id = (line->dir == HTE_RISING_EDGE_TS) ?
> >> +				 GPIO_V2_LINE_EVENT_RISING_EDGE :
> >> +				 GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >> +	} else {
> >> +		le.timestamp_ns = line_event_timestamp(line);
> >> +	}
> >> +
> > Move the FLAG_EVENT_CLOCK_HARDWARE check into line_event_timestamp().
> >
> > And the id fudging is necessary because the level returned by
> > gpiod_get_raw_value_cansleep() can disagree with the level from hte?
> 
> > So you are still trying to synchronise events from two streams.
> > And that is still broken.
> > If a hte event occurs between the level being sampled by
> > gpiod_get_raw_value_cansleep() and the line->dir being read then the line
> > will have toggled and you will be reporting the opposite state than the
> > one the debouncer determined was stable.  And maybe the wrong timestamp as
> > well.
> >
> > For lines where hte is enabled, the hte should be the source of level for
> > the debouncer, not the raw value.  And the mod_delayed_work() that
> > drives the debouncer should be called by a hte handler, not an irq handler.
> >
> > There is also a race on reading the hte timestamp (line->timestamp_ns) and
> > the hte level (line->dir), such that you can get the level from one event
> > the timestamp from another.
> >
> >> +	lr = line->req;
> >> +	le.offset = gpio_chip_hwgpio(line->desc);
> >> +	line->line_seqno++;
> >> +	le.line_seqno = line->line_seqno;
> >> +	le.seqno = (lr->num_lines == 1) ?
> >> +		le.line_seqno : atomic_inc_return(&lr->seqno);
> >> +
> > What is the purpose of moving this block of code moved from before the
> > if (level)?
> >
> >
> >>  	linereq_put_event(lr, &le);
> >>  }
> >>  
> >> @@ -891,7 +1003,6 @@ static int gpio_v2_line_flags_validate(u64 flags)
> >>  	/* Return an error if an unknown flag is set */
> >>  	if (flags & ~GPIO_V2_LINE_VALID_FLAGS)
> >>  		return -EINVAL;
> >> -
> > Gratuitous whitespace change.
> >
> >>  	/*
> >>  	 * Do not allow both INPUT and OUTPUT flags to be set as they are
> >>  	 * contradictory.
> >> @@ -900,6 +1011,11 @@ static int gpio_v2_line_flags_validate(u64 flags)
> >>  	    (flags & GPIO_V2_LINE_FLAG_OUTPUT))
> >>  		return -EINVAL;
> >>  
> >> +	/* Only allow one event clock source */
> >> +	if ((flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME) &&
> >> +	    (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE))
> >> +		return -EINVAL;
> >> +
> >>  	/* Edge detection requires explicit input. */
> >>  	if ((flags & GPIO_V2_LINE_EDGE_FLAGS) &&
> >>  	    !(flags & GPIO_V2_LINE_FLAG_INPUT))
> >> @@ -992,6 +1108,8 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 flags,
> >>  
> >>  	assign_bit(FLAG_EVENT_CLOCK_REALTIME, flagsp,
> >>  		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME);
> >> +	assign_bit(FLAG_EVENT_CLOCK_HARDWARE, flagsp,
> >> +		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE);
> >>  }
> >>  
> >>  static long linereq_get_values(struct linereq *lr, void __user *ip)
> >> @@ -1154,6 +1272,21 @@ static long linereq_set_config_unlocked(struct linereq *lr,
> >>  				return ret;
> >>  		}
> >>  
> >> +		/* Check if new config sets hardware assisted clock */
> >> +		if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
> >> +			ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
> >> +							process_hw_ts_thread,
> >> +							&lr->lines[i]);
> >> +			if (ret)
> >> +				return ret;
> > Note that the line config is the complete line config, not a delta.
> >
> > What happens when a line that already has hte enabled is reconfigured
> > and still has hte enabled?  i.e. what happens when
> > gpiod_req_hw_timestamp_ns() is called for the second time?
> 
> HTE will return without doing anything with error code.
> 

But this is not an error case, it is a normal reconfigure of an
attribute other than the hte flag.
And that will now return an error to userspace?

Cheers,
Kent.

> >
> > You provide a comment for the release case below, what of the request
> > case?
> >
> > If you need to check for change then compare the old and new flags, as
> > the polarity_change check does (not visible in the diff here).
> >
> >> +		} else {
> >> +			/*
> >> +			 * HTE subsys will do nothing if there is nothing to
> >> +			 * release.
> >> +			 */
> >> +			gpiod_rel_hw_timestamp_ns(desc);
> >> +		}
> >> +
> > Comment will fit on one line.
> >
> > And it would be better to document that the function is idempotent in the
> > function documentation, not everywhere it is used.
> >
> >>  		blocking_notifier_call_chain(&desc->gdev->notifier,
> >>  					     GPIO_V2_LINE_CHANGED_CONFIG,
> >>  					     desc);
> >> @@ -1409,6 +1542,14 @@ static int linereq_create(struct gpio_device *gdev, void __user *ip)
> >>  					flags & GPIO_V2_LINE_EDGE_FLAGS);
> >>  			if (ret)
> >>  				goto out_free_linereq;
> >> +
> >> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
> >> +				ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
> >> +							process_hw_ts_thread,
> >> +							&lr->lines[i]);
> >> +				if (ret)
> >> +					goto out_free_linereq;
> >> +			}
> >>  		}
> >>  
> >>  		blocking_notifier_call_chain(&desc->gdev->notifier,
> >> @@ -1959,6 +2100,8 @@ static void gpio_desc_to_lineinfo(struct gpio_desc *desc,
> >>  
> >>  	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &desc->flags))
> >>  		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
> >> +	else if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &desc->flags))
> >> +		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
> >>  
> >>  	debounce_period_us = READ_ONCE(desc->debounce_period_us);
> >>  	if (debounce_period_us) {
> >> diff --git a/include/uapi/linux/gpio.h b/include/uapi/linux/gpio.h
> >> index eaaea3d8e6b4..d360545b4c21 100644
> >> --- a/include/uapi/linux/gpio.h
> >> +++ b/include/uapi/linux/gpio.h
> >> @@ -80,6 +80,7 @@ enum gpio_v2_line_flag {
> >>  	GPIO_V2_LINE_FLAG_BIAS_PULL_DOWN	= _BITULL(9),
> >>  	GPIO_V2_LINE_FLAG_BIAS_DISABLED		= _BITULL(10),
> >>  	GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME	= _BITULL(11),
> >> +	GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE	= _BITULL(12),
> >>  };
> >>  
> > I'm now thinking this name, "HARDWARE" is too vague, in case other
> > timestamp source alternatives join the fray, and so should be "HTE".
> >
> > Cheers,
> > Kent.

Re: [RFC v3 09/12] gpiolib: cdev: Add hardware timestamp clock type

[Dipen Patel]

On 11/30/21 7:29 PM, Dipen Patel wrote:
> Hi,
>
> On 11/25/21 5:31 PM, Kent Gibson wrote:
>> On Tue, Nov 23, 2021 at 11:30:36AM -0800, Dipen Patel wrote:
>>> This patch adds new clock type for the GPIO controller which can
>>> timestamp gpio lines in realtime using hardware means. To expose such
>>> functionalities to the userspace, code has been added in this patch
>>> where during line create call, it checks for new clock type and if
>>> requested, calls hardware timestamp related API from gpiolib.c.
>>> During line change event, the HTE subsystem pushes timestamp data
>>> through callbacks.
>>>
>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>>> Acked-by: Linus Walleij <linus.walleij@linaro.org>
>>> ---
>>> Changes in v2:
>>> - Added hte_dir and static structure hte_ts_desc.
>>> - Added callbacks which get invoked by HTE when new data is available.
>>> - Better use of hte_dir and seq from hte_ts_desc.
>>> - Modified sw debounce function to accommodate hardware timestamping.
>>>
>>>  drivers/gpio/gpiolib-cdev.c | 161 ++++++++++++++++++++++++++++++++++--
>>>  include/uapi/linux/gpio.h   |   1 +
>>>  2 files changed, 153 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
>>> index c7b5446d01fd..1736ad54e3ec 100644
>>> --- a/drivers/gpio/gpiolib-cdev.c
>>> +++ b/drivers/gpio/gpiolib-cdev.c
>>> @@ -464,6 +464,12 @@ struct line {
>>>  	 * stale value.
>>>  	 */
>>>  	unsigned int level;
>>> +	/*
>>> +	 * dir will be touched in HTE callbacks hte_ts_cb_t and
>>> +	 * hte_ts_threaded_cb_t and they are mutually exclusive. This will be
>>> +	 * unused when HTE is not supported/disabled.
>>> +	 */
>>> +	enum hte_dir dir;
>>>  };
>>>  
>> Documentation should be in present tense, so 
>>
>> s/will be/is/g
>>
>> Same applies to other patches.
>>
>> Also
>>
>> s/touched/accessed/
>>
>> dir is a poor name for the field.  It is the hte edge direction and
>> effectively the line level, so call it hte_edge_dirn or
>> hte_edge_direction or hte_level.
>>
>> And it is placed in a section of the struct documented as "debouncer specific
>> fields", but it is not specfic to the debouncer.  Add a "hte specific
>> fields" section if nothing else is suitable.
>>
>>>  /**
>>> @@ -518,6 +524,7 @@ struct linereq {
>>>  	 GPIO_V2_LINE_DRIVE_FLAGS | \
>>>  	 GPIO_V2_LINE_EDGE_FLAGS | \
>>>  	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
>>> +	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
>>>  	 GPIO_V2_LINE_BIAS_FLAGS)
>>>  
>>>  static void linereq_put_event(struct linereq *lr,
>>> @@ -546,6 +553,94 @@ static u64 line_event_timestamp(struct line *line)
>>>  	return ktime_get_ns();
>>>  }
>>>  
>>> +static hte_return_t process_hw_ts_thread(void *p)
>>> +{
>>> +	struct line *line = p;
>>> +	struct linereq *lr = line->req;
>>> +	struct gpio_v2_line_event le;
>>> +	u64 eflags;
>>> +
>>> +	memset(&le, 0, sizeof(le));
>>> +
>>> +	le.timestamp_ns = line->timestamp_ns;
>>> +	line->timestamp_ns = 0;
>>> +
>> What is the purpose of this zeroing?
>>
>>> +	if (line->dir >= HTE_DIR_NOSUPP) {
>>> +		eflags = READ_ONCE(line->eflags);
>>> +		if (eflags == GPIO_V2_LINE_FLAG_EDGE_BOTH) {
>>> +			int level = gpiod_get_value_cansleep(line->desc);
>>> +
>>> +			if (level)
>>> +				/* Emit low-to-high event */
>>> +				le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>> +			else
>>> +				/* Emit high-to-low event */
>>> +				le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_RISING) {
>>> +			/* Emit low-to-high event */
>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_FALLING) {
>>> +			/* Emit high-to-low event */
>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>> +		} else {
>>> +			return HTE_CB_ERROR;
>>> +		}
>>> +	} else {
>>> +		if (line->dir == HTE_RISING_EDGE_TS)
>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>> +		else
>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>> +	}
>> The mapping from line->dir to le.id needs to take into account the active
>> low setting for the line.
>>
>> And it might be simpler if the hte_ts_data provided the level, equivalent
>> to gpiod_get_raw_value_cansleep(), rather than an edge direction, so you
>> can provide a common helper to determine the edge given the raw level.
> (So from the level determine the edge?) that sound right specially when

                                                                ^^^

                                                                does not*

>
> HTE provider has capability to record the edge in that case why bother
>
> getting the level and determine edge?
>
> Calculating the edge from the level makes sense when hte provider does not
>
> have that feature and that is what if (line->dir >= HTE_DIR_NOSUPP) does.
>
>>> +
>>> +	le.line_seqno = line->line_seqno;
>>> +	le.seqno = (lr->num_lines == 1) ? le.line_seqno : line->req_seqno;
>>> +	le.offset = gpio_chip_hwgpio(line->desc);
>>> +
>>> +	linereq_put_event(lr, &le);
>>> +
>>> +	return HTE_CB_HANDLED;
>>> +}
>>> +
>>> +static hte_return_t process_hw_ts(struct hte_ts_data *ts, void *p)
>>> +{
>>> +	struct line *line = p;
>>> +	struct linereq *lr = line->req;
>>> +
>>> +	if (!ts)
>>> +		return HTE_CB_ERROR;
>>> +
>>> +	line->timestamp_ns = ts->tsc;
>>> +	line->dir = ts->dir;
>>> +
>> The doc for timestamp_ns states:
>>
>> 	 * timestamp_ns and req_seqno are accessed only by
>> 	 * edge_irq_handler() and edge_irq_thread(), which are themselves
>> 	 * mutually exclusive, so no additional protection is necessary.
>>
>> That no longer holds.  It is now also accessed here, and in
>> process_hw_ts_thread(), which wont run concurrently with each other or
>> the edge_irq_* handlers, but also in debounce_work_func() which may run
>> concurrently with the others.
>> So timestamp_ns now requires protection from concurrent access.
>>
>>> +	/*
>>> +	 * It is possible that HTE engine detects spurious edges for the
>>> +	 * lines where software debounce is enabled. This primary callback
>>> +	 * will be called multiple times in that case. It will be better to
>>> +	 * let debounce_work_func handle instead of process_hw_ts_thread.
>>> +	 * The timestamp_ns will be overwritten here which is fine as we are
>>> +	 * interested in the last value anyway. The debounce_work_func will
>>> +	 * then just read whatever last line->timestamp_ns is stored. Because
>>> +	 * this callback can be called multiple times, we are not really
>>> +	 * interested in ts->seq.
>>> +	 */
>> Not sure what this is trying to say.
>> Is this the primary callback? Or debounce_irq_handler()?
> This is primary callback called from HTE when it pushes new TS data per line, it
>
> also says so in the second line.
>
>> You say you really aren't interested in ts->seq, but the code immediately
>> uses it.
> That is when sw_debounced is not set and whole paragraph is about when
>
> sw_debounced is set.
>
>> Reword to clarify.
>> And add braces after function names to highlight them, so
>> debounce_work_func().
> Will do.
>>> +	if (!READ_ONCE(line->sw_debounced)) {
>>> +		line->line_seqno = ts->seq;
>>> +
>>> +		/*
>>> +		 * Increment in this callback incase all the lines in linereq
>>> +		 * are enabled for hw timestamping. This will work even if
>>> +		 * subset of lines are enabled for hw timestamping as
>>> +		 * edge_irq_* callbacks will proceed as usual for them.
>>> +		 */
>> s/incase/in case/
>>
>> Not sure what the comment is trying to say. There is no check here that
>> the other lines have HTE enabled.  And that is not relevant anyway.
>> The edge_irq_* handlers will proceed as usual for those lines NOT
>> enabled for hw timestamping.
>>
>> To clarify, the line_seqno indicates where this event lies in the
>> sequence of events for the line.
>> The request seqno indicates where this event lines in the sequence of
>> events for the request.
>> For a single line request these are the same, hence the minor
>> optimisation of not updating lr->seqno below.
>>
>>> +		if (lr->num_lines != 1)
>>> +			line->req_seqno = atomic_inc_return(&lr->seqno);
>>> +
>> The req_seqno should be updated corresponding to the change in the
>> line_reqno.  That always used to be 1, but no longer if hte can discard
>> events, i.e. skip over line_seqnos.
> HTE does not discard any events, it pushes to clients as soon as its
>
> available through primary callback.
>
>> To be consistent, i.e. if events were lost for this line then they were
>> also lost for the requested lines, the lr->seqno should be incremented by
>> the change in line_seqno.  Probably with some sanity checks.
>>
>>> +		return HTE_RUN_THREADED_CB;
>>> +	}
>>> +
>>> +	return HTE_CB_HANDLED;
>>> +}
>>> +
>>>  static irqreturn_t edge_irq_thread(int irq, void *p)
>>>  {
>>>  	struct line *line = p;
>>> @@ -553,6 +648,10 @@ static irqreturn_t edge_irq_thread(int irq, void *p)
>>>  	struct gpio_v2_line_event le;
>>>  	u64 eflags;
>>>  
>>> +	/* Let process_hw_ts_thread handle */
>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
>>> +		return IRQ_HANDLED;
>>> +
>> This adds pointless runtime overhead, and for everyone not just hte users.
>> Don't stub out a handler in the handler - stub it out where it is
>> registered by registering a stub handler.  Or don't request it at all.
>>
>> So why would gpiolib-cdev be requesting the irq, only to stub out
>> the handlers?
>> If that has a side-effect that hte requires then hte should be taking
>> care of it - it is not gpiolib-cdev's problem.
> - Why stop at moving irq and debounce related stuff to hte then?
>
> I mean if there is hte provider which can TS GPIO output/input
>
> does it mean hte is responsible for parsing the GPIO line configs, setting them up
>
> (i.e. input or output) as well? Are we not duplicating logic instead of
>
> leveraging gpio-cdev? Does it make sense for the HTE subsystem which not
>
> only TS the GPIOs but other SoC lines?
>
> - What happens to in kernel GPIO HTE client (for example, hte-tegra194-gpio-test.c)?
>
> some clients do more in their IRQ handler than what edge_irq_handler does in which
>
> case it would make sense to have them request irq in their code than through HTE.
>
>> And speaking as to how the whole hte/gpiolib-cdev interface should work,
>> hte should be an edge event generator alternative to irq.  So lines with
>> hte enabled should work without any irq calls from gpiolib-cdev.
>> That includes the sw debouncer - more on that below.
>>
>>>  	/* Do not leak kernel stack to userspace */
>>>  	memset(&le, 0, sizeof(le));
>>>  
>>> @@ -604,6 +703,10 @@ static irqreturn_t edge_irq_handler(int irq, void *p)
>>>  	struct line *line = p;
>>>  	struct linereq *lr = line->req;
>>>  
>>> +	/* Let HTE supplied callbacks handle */
>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
>>> +		return IRQ_HANDLED;
>>> +
>>>  	/*
>>>  	 * Just store the timestamp in hardirq context so we get it as
>>>  	 * close in time as possible to the actual event.
>>> @@ -682,14 +785,6 @@ static void debounce_work_func(struct work_struct *work)
>>>  	/* Do not leak kernel stack to userspace */
>>>  	memset(&le, 0, sizeof(le));
>>>  
>>> -	lr = line->req;
>>> -	le.timestamp_ns = line_event_timestamp(line);
>>> -	le.offset = gpio_chip_hwgpio(line->desc);
>>> -	line->line_seqno++;
>>> -	le.line_seqno = line->line_seqno;
>>> -	le.seqno = (lr->num_lines == 1) ?
>>> -		le.line_seqno : atomic_inc_return(&lr->seqno);
>>> -
>>>  	if (level)
>>>  		/* Emit low-to-high event */
>>>  		le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>> @@ -697,6 +792,23 @@ static void debounce_work_func(struct work_struct *work)
>>>  		/* Emit high-to-low event */
>>>  		le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>  
>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
>>> +		le.timestamp_ns = line->timestamp_ns;
>>> +		if (line->dir < HTE_DIR_NOSUPP)
>>> +			le.id = (line->dir == HTE_RISING_EDGE_TS) ?
>>> +				 GPIO_V2_LINE_EVENT_RISING_EDGE :
>>> +				 GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>> +	} else {
>>> +		le.timestamp_ns = line_event_timestamp(line);
>>> +	}
>>> +
>> Move the FLAG_EVENT_CLOCK_HARDWARE check into line_event_timestamp().
>>
>> And the id fudging is necessary because the level returned by
>> gpiod_get_raw_value_cansleep() can disagree with the level from hte?
>> So you are still trying to synchronise events from two streams.
>> And that is still broken.
>> If a hte event occurs between the level being sampled by
>> gpiod_get_raw_value_cansleep() and the line->dir being read then the line
>> will have toggled and you will be reporting the opposite state than the
>> one the debouncer determined was stable.  And maybe the wrong timestamp as
>> well.
>>
>> For lines where hte is enabled, the hte should be the source of level for
>> the debouncer, not the raw value.  And the mod_delayed_work() that
>> drives the debouncer should be called by a hte handler, not an irq handler.
>>
>> There is also a race on reading the hte timestamp (line->timestamp_ns) and
>> the hte level (line->dir), such that you can get the level from one event
>> the timestamp from another.
>>
>>> +	lr = line->req;
>>> +	le.offset = gpio_chip_hwgpio(line->desc);
>>> +	line->line_seqno++;
>>> +	le.line_seqno = line->line_seqno;
>>> +	le.seqno = (lr->num_lines == 1) ?
>>> +		le.line_seqno : atomic_inc_return(&lr->seqno);
>>> +
>> What is the purpose of moving this block of code moved from before the
>> if (level)?
>>
>>
>>>  	linereq_put_event(lr, &le);
>>>  }
>>>  
>>> @@ -891,7 +1003,6 @@ static int gpio_v2_line_flags_validate(u64 flags)
>>>  	/* Return an error if an unknown flag is set */
>>>  	if (flags & ~GPIO_V2_LINE_VALID_FLAGS)
>>>  		return -EINVAL;
>>> -
>> Gratuitous whitespace change.
>>
>>>  	/*
>>>  	 * Do not allow both INPUT and OUTPUT flags to be set as they are
>>>  	 * contradictory.
>>> @@ -900,6 +1011,11 @@ static int gpio_v2_line_flags_validate(u64 flags)
>>>  	    (flags & GPIO_V2_LINE_FLAG_OUTPUT))
>>>  		return -EINVAL;
>>>  
>>> +	/* Only allow one event clock source */
>>> +	if ((flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME) &&
>>> +	    (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE))
>>> +		return -EINVAL;
>>> +
>>>  	/* Edge detection requires explicit input. */
>>>  	if ((flags & GPIO_V2_LINE_EDGE_FLAGS) &&
>>>  	    !(flags & GPIO_V2_LINE_FLAG_INPUT))
>>> @@ -992,6 +1108,8 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 flags,
>>>  
>>>  	assign_bit(FLAG_EVENT_CLOCK_REALTIME, flagsp,
>>>  		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME);
>>> +	assign_bit(FLAG_EVENT_CLOCK_HARDWARE, flagsp,
>>> +		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE);
>>>  }
>>>  
>>>  static long linereq_get_values(struct linereq *lr, void __user *ip)
>>> @@ -1154,6 +1272,21 @@ static long linereq_set_config_unlocked(struct linereq *lr,
>>>  				return ret;
>>>  		}
>>>  
>>> +		/* Check if new config sets hardware assisted clock */
>>> +		if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
>>> +			ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
>>> +							process_hw_ts_thread,
>>> +							&lr->lines[i]);
>>> +			if (ret)
>>> +				return ret;
>> Note that the line config is the complete line config, not a delta.
>>
>> What happens when a line that already has hte enabled is reconfigured
>> and still has hte enabled?  i.e. what happens when
>> gpiod_req_hw_timestamp_ns() is called for the second time?
> HTE will return without doing anything with error code.
>
>> You provide a comment for the release case below, what of the request
>> case?
>>
>> If you need to check for change then compare the old and new flags, as
>> the polarity_change check does (not visible in the diff here).
>>
>>> +		} else {
>>> +			/*
>>> +			 * HTE subsys will do nothing if there is nothing to
>>> +			 * release.
>>> +			 */
>>> +			gpiod_rel_hw_timestamp_ns(desc);
>>> +		}
>>> +
>> Comment will fit on one line.
>>
>> And it would be better to document that the function is idempotent in the
>> function documentation, not everywhere it is used.
>>
>>>  		blocking_notifier_call_chain(&desc->gdev->notifier,
>>>  					     GPIO_V2_LINE_CHANGED_CONFIG,
>>>  					     desc);
>>> @@ -1409,6 +1542,14 @@ static int linereq_create(struct gpio_device *gdev, void __user *ip)
>>>  					flags & GPIO_V2_LINE_EDGE_FLAGS);
>>>  			if (ret)
>>>  				goto out_free_linereq;
>>> +
>>> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
>>> +				ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
>>> +							process_hw_ts_thread,
>>> +							&lr->lines[i]);
>>> +				if (ret)
>>> +					goto out_free_linereq;
>>> +			}
>>>  		}
>>>  
>>>  		blocking_notifier_call_chain(&desc->gdev->notifier,
>>> @@ -1959,6 +2100,8 @@ static void gpio_desc_to_lineinfo(struct gpio_desc *desc,
>>>  
>>>  	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &desc->flags))
>>>  		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
>>> +	else if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &desc->flags))
>>> +		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
>>>  
>>>  	debounce_period_us = READ_ONCE(desc->debounce_period_us);
>>>  	if (debounce_period_us) {
>>> diff --git a/include/uapi/linux/gpio.h b/include/uapi/linux/gpio.h
>>> index eaaea3d8e6b4..d360545b4c21 100644
>>> --- a/include/uapi/linux/gpio.h
>>> +++ b/include/uapi/linux/gpio.h
>>> @@ -80,6 +80,7 @@ enum gpio_v2_line_flag {
>>>  	GPIO_V2_LINE_FLAG_BIAS_PULL_DOWN	= _BITULL(9),
>>>  	GPIO_V2_LINE_FLAG_BIAS_DISABLED		= _BITULL(10),
>>>  	GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME	= _BITULL(11),
>>> +	GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE	= _BITULL(12),
>>>  };
>>>  
>> I'm now thinking this name, "HARDWARE" is too vague, in case other
>> timestamp source alternatives join the fray, and so should be "HTE".
>>
>> Cheers,
>> Kent.

Re: [RFC v3 09/12] gpiolib: cdev: Add hardware timestamp clock type

[Dipen Patel]

Hi,


On 12/1/21 9:16 AM, Kent Gibson wrote:
> On Tue, Nov 30, 2021 at 07:29:20PM -0800, Dipen Patel wrote:
>> Hi,
>>
>> On 11/25/21 5:31 PM, Kent Gibson wrote:
>>> On Tue, Nov 23, 2021 at 11:30:36AM -0800, Dipen Patel wrote:
>>>> This patch adds new clock type for the GPIO controller which can
>>>> timestamp gpio lines in realtime using hardware means. To expose such
>>>> functionalities to the userspace, code has been added in this patch
>>>> where during line create call, it checks for new clock type and if
>>>> requested, calls hardware timestamp related API from gpiolib.c.
>>>> During line change event, the HTE subsystem pushes timestamp data
>>>> through callbacks.
>>>>
>>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>>>> Acked-by: Linus Walleij <linus.walleij@linaro.org>
>>>> ---
>>>> Changes in v2:
>>>> - Added hte_dir and static structure hte_ts_desc.
>>>> - Added callbacks which get invoked by HTE when new data is available.
>>>> - Better use of hte_dir and seq from hte_ts_desc.
>>>> - Modified sw debounce function to accommodate hardware timestamping.
>>>>
>>>>  drivers/gpio/gpiolib-cdev.c | 161 ++++++++++++++++++++++++++++++++++--
>>>>  include/uapi/linux/gpio.h   |   1 +
>>>>  2 files changed, 153 insertions(+), 9 deletions(-)
>>>>
>>>> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
>>>> index c7b5446d01fd..1736ad54e3ec 100644
>>>> --- a/drivers/gpio/gpiolib-cdev.c
>>>> +++ b/drivers/gpio/gpiolib-cdev.c
>>>> @@ -464,6 +464,12 @@ struct line {
>>>>  	 * stale value.
>>>>  	 */
>>>>  	unsigned int level;
>>>> +	/*
>>>> +	 * dir will be touched in HTE callbacks hte_ts_cb_t and
>>>> +	 * hte_ts_threaded_cb_t and they are mutually exclusive. This will be
>>>> +	 * unused when HTE is not supported/disabled.
>>>> +	 */
>>>> +	enum hte_dir dir;
>>>>  };
>>>>  
>>> Documentation should be in present tense, so 
>>>
>>> s/will be/is/g
>>>
>>> Same applies to other patches.
>>>
>>> Also
>>>
>>> s/touched/accessed/
>>>
>>> dir is a poor name for the field.  It is the hte edge direction and
>>> effectively the line level, so call it hte_edge_dirn or
>>> hte_edge_direction or hte_level.
>>>
>>> And it is placed in a section of the struct documented as "debouncer specific
>>> fields", but it is not specfic to the debouncer.  Add a "hte specific
>>> fields" section if nothing else is suitable.
>>>
>>>>  /**
>>>> @@ -518,6 +524,7 @@ struct linereq {
>>>>  	 GPIO_V2_LINE_DRIVE_FLAGS | \
>>>>  	 GPIO_V2_LINE_EDGE_FLAGS | \
>>>>  	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
>>>> +	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
>>>>  	 GPIO_V2_LINE_BIAS_FLAGS)
>>>>  
>>>>  static void linereq_put_event(struct linereq *lr,
>>>> @@ -546,6 +553,94 @@ static u64 line_event_timestamp(struct line *line)
>>>>  	return ktime_get_ns();
>>>>  }
>>>>  
>>>> +static hte_return_t process_hw_ts_thread(void *p)
>>>> +{
>>>> +	struct line *line = p;
>>>> +	struct linereq *lr = line->req;
>>>> +	struct gpio_v2_line_event le;
>>>> +	u64 eflags;
>>>> +
>>>> +	memset(&le, 0, sizeof(le));
>>>> +
>>>> +	le.timestamp_ns = line->timestamp_ns;
>>>> +	line->timestamp_ns = 0;
>>>> +
>>> What is the purpose of this zeroing?
>>>
>>>> +	if (line->dir >= HTE_DIR_NOSUPP) {
>>>> +		eflags = READ_ONCE(line->eflags);
>>>> +		if (eflags == GPIO_V2_LINE_FLAG_EDGE_BOTH) {
>>>> +			int level = gpiod_get_value_cansleep(line->desc);
>>>> +
>>>> +			if (level)
>>>> +				/* Emit low-to-high event */
>>>> +				le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>> +			else
>>>> +				/* Emit high-to-low event */
>>>> +				le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_RISING) {
>>>> +			/* Emit low-to-high event */
>>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_FALLING) {
>>>> +			/* Emit high-to-low event */
>>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>> +		} else {
>>>> +			return HTE_CB_ERROR;
>>>> +		}
>>>> +	} else {
>>>> +		if (line->dir == HTE_RISING_EDGE_TS)
>>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>> +		else
>>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>> +	}
>>> The mapping from line->dir to le.id needs to take into account the active
>>> low setting for the line.
>>>
>>> And it might be simpler if the hte_ts_data provided the level, equivalent
>>> to gpiod_get_raw_value_cansleep(), rather than an edge direction, so you
>>> can provide a common helper to determine the edge given the raw level.
>> (So from the level determine the edge?) that sound right specially when
>>
>> HTE provider has capability to record the edge in that case why bother
>>
>> getting the level and determine edge?
>>
>> Calculating the edge from the level makes sense when hte provider does not
>>
>> have that feature and that is what if (line->dir >= HTE_DIR_NOSUPP) does.
>>
> As asked in the review of patch 02, do you have an example of hardware that
> reports an edge direction rather than NOSUPP?
No...
>
> Anyway, this is just a naming thing - the information content being passed
> is the the same, be it high/low/unknown or rising/falling/unknown.
>
> If the hardware does report edge direction then it is just one bit, and
> that also corresponds to the physical level immediately following the
> edge, so no additional conversion required there.
>
> It would be clearer to pass a level than an edge, as
>  - a hte edge (hte_dir) could be confused with a cdev edge
>    (gpio_v2_line_event_id), in mails like this if not in the code.
>  - cdev will fallback to using the physical level to determine the edge
>    if the hte can't provide it
I believe I have retained all the fallback logic in such case.
>  - cdev has to perform inversion if active low and it already does that
>    based on levels
>
>>>> +
>>>> +	le.line_seqno = line->line_seqno;
>>>> +	le.seqno = (lr->num_lines == 1) ? le.line_seqno : line->req_seqno;
>>>> +	le.offset = gpio_chip_hwgpio(line->desc);
>>>> +
>>>> +	linereq_put_event(lr, &le);
>>>> +
>>>> +	return HTE_CB_HANDLED;
>>>> +}
>>>> +
>>>> +static hte_return_t process_hw_ts(struct hte_ts_data *ts, void *p)
>>>> +{
>>>> +	struct line *line = p;
>>>> +	struct linereq *lr = line->req;
>>>> +
>>>> +	if (!ts)
>>>> +		return HTE_CB_ERROR;
>>>> +
>>>> +	line->timestamp_ns = ts->tsc;
>>>> +	line->dir = ts->dir;
>>>> +
>>> The doc for timestamp_ns states:
>>>
>>> 	 * timestamp_ns and req_seqno are accessed only by
>>> 	 * edge_irq_handler() and edge_irq_thread(), which are themselves
>>> 	 * mutually exclusive, so no additional protection is necessary.
>>>
>>> That no longer holds.  It is now also accessed here, and in
>>> process_hw_ts_thread(), which wont run concurrently with each other or
>>> the edge_irq_* handlers, but also in debounce_work_func() which may run
>>> concurrently with the others.
>>> So timestamp_ns now requires protection from concurrent access.
>>>
>>>> +	/*
>>>> +	 * It is possible that HTE engine detects spurious edges for the
>>>> +	 * lines where software debounce is enabled. This primary callback
>>>> +	 * will be called multiple times in that case. It will be better to
>>>> +	 * let debounce_work_func handle instead of process_hw_ts_thread.
>>>> +	 * The timestamp_ns will be overwritten here which is fine as we are
>>>> +	 * interested in the last value anyway. The debounce_work_func will
>>>> +	 * then just read whatever last line->timestamp_ns is stored. Because
>>>> +	 * this callback can be called multiple times, we are not really
>>>> +	 * interested in ts->seq.
>>>> +	 */
>>> Not sure what this is trying to say.
>>> Is this the primary callback? Or debounce_irq_handler()?
>> This is primary callback called from HTE when it pushes new TS data per line, it
>>
>> also says so in the second line.
>>
> Yeah, I probably read that as "The primary callback", but it is
> confusing anyway. "This primary callback" implies there is another
> primary callback. 
> Just say "This handler" instead of "This primary callback".
Noted...
>
>>> You say you really aren't interested in ts->seq, but the code immediately
>>> uses it.
>> That is when sw_debounced is not set and whole paragraph is about when
>>
>> sw_debounced is set.
>>
> So your whole comment here is about the else case?
> Then either put the comment where the else would be, or better yet invert
> the logic and return immediately if sw_debounced.
Sure...
>
>>> Reword to clarify.
>>> And add braces after function names to highlight them, so
>>> debounce_work_func().
>> Will do.
>>>> +	if (!READ_ONCE(line->sw_debounced)) {
>>>> +		line->line_seqno = ts->seq;
>>>> +
>>>> +		/*
>>>> +		 * Increment in this callback incase all the lines in linereq
>>>> +		 * are enabled for hw timestamping. This will work even if
>>>> +		 * subset of lines are enabled for hw timestamping as
>>>> +		 * edge_irq_* callbacks will proceed as usual for them.
>>>> +		 */
>>> s/incase/in case/
>>>
>>> Not sure what the comment is trying to say. There is no check here that
>>> the other lines have HTE enabled.  And that is not relevant anyway.
>>> The edge_irq_* handlers will proceed as usual for those lines NOT
>>> enabled for hw timestamping.
>>>
>>> To clarify, the line_seqno indicates where this event lies in the
>>> sequence of events for the line.
>>> The request seqno indicates where this event lines in the sequence of
>>> events for the request.
>>> For a single line request these are the same, hence the minor
>>> optimisation of not updating lr->seqno below.
>>>
>>>> +		if (lr->num_lines != 1)
>>>> +			line->req_seqno = atomic_inc_return(&lr->seqno);
>>>> +
>>> The req_seqno should be updated corresponding to the change in the
>>> line_reqno.  That always used to be 1, but no longer if hte can discard
>>> events, i.e. skip over line_seqnos.
>> HTE does not discard any events, it pushes to clients as soon as its
>>
>> available through primary callback.
> The discarding of events that I am referring to is from your previous
> answers that indicated, to me anyway, that there could be gaps in the
> ts->seq numbers if the hardware event FIFO overflowed.
> Is that not the case?

Not in this patch series as the provider I have dealt with does not such

feature. ts->seq is software counter in that case.

>
> And when you say "primary callback", both here and elsewhere, you mean the
> cb parameter to gpiod_req_hw_timestamp_ns() and gc->req_hw_timestamp()?
yes, cb is primary and tcb (threaded cb) is optional secondary.
> Either way, be specific and name the function or parameter, or find a
> better term than "primary callback".
> In this case "the client's event handler" would be much clearer.
Noted...
>
>>> To be consistent, i.e. if events were lost for this line then they were
>>> also lost for the requested lines, the lr->seqno should be incremented by
>>> the change in line_seqno.  Probably with some sanity checks.
>>>
>>>> +		return HTE_RUN_THREADED_CB;
>>>> +	}
>>>> +
>>>> +	return HTE_CB_HANDLED;
>>>> +}
>>>> +
>>>>  static irqreturn_t edge_irq_thread(int irq, void *p)
>>>>  {
>>>>  	struct line *line = p;
>>>> @@ -553,6 +648,10 @@ static irqreturn_t edge_irq_thread(int irq, void *p)
>>>>  	struct gpio_v2_line_event le;
>>>>  	u64 eflags;
>>>>  
>>>> +	/* Let process_hw_ts_thread handle */
>>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
>>>> +		return IRQ_HANDLED;
>>>> +
>>> This adds pointless runtime overhead, and for everyone not just hte users.
>>> Don't stub out a handler in the handler - stub it out where it is
>>> registered by registering a stub handler.  Or don't request it at all.
>>>
>>> So why would gpiolib-cdev be requesting the irq, only to stub out
>>> the handlers?
>>> If that has a side-effect that hte requires then hte should be taking
>>> care of it - it is not gpiolib-cdev's problem.
>> - Why stop at moving irq and debounce related stuff to hte then?
>>
> How about you answer my question before asking your own?
> Here I am only questioning why gpiolib-cdev is requesting an interrupt
> for a hte line at all.  It has no reason to, so hte must have?

Commented below in "separation of concern" paragraph.

> And where did I suggest moving the "debounce stuff" to hte?

Perhaps I misunderstood "That includes the sw debouncer - more on that

below." comment.

>
> What I am suggesting is called separation of concerns.  In this context
> the intent is to look towards abstracting the edge event generation.
> Having hte and irq entangled for no apparent reason makes doing that more

But in this patch series (i.e. HTE provider), they are entangled as hte provider

will only ts the GPIO line which is configured as input irq. That is why I have

separated GPIO line config part and TS part. In other words, let cdev handle

any line config that userspace has asked and let hte handle ts part if the

line config is supported (If line config is not suitable, it will not enable hte, see

gpio-tegra186.c function tegra186_gpio_req_hw_ts() in this patch series where

check happens for the tegra hte provider).

> difficult than it needs to be, whereas keeping them separate greatly
> simplifies identification of common code suitable for refactoring
> subsequently.
>
> Not sure what to call what you are suggesting.
>
>> I mean if there is hte provider which can TS GPIO output/input
>>
>> does it mean hte is responsible for parsing the GPIO line configs, setting them up
>>
>> (i.e. input or output) as well? Are we not duplicating logic instead of
>>
>> leveraging gpio-cdev? Does it make sense for the HTE subsystem which not
>>
>> only TS the GPIOs but other SoC lines?
>>
>> - What happens to in kernel GPIO HTE client (for example, hte-tegra194-gpio-test.c)?
>>
>> some clients do more in their IRQ handler than what edge_irq_handler does in which
>>
>> case it would make sense to have them request irq in their code than through HTE.
>>
>>> And speaking as to how the whole hte/gpiolib-cdev interface should work,
>>> hte should be an edge event generator alternative to irq.  So lines with
>>> hte enabled should work without any irq calls from gpiolib-cdev.
>>> That includes the sw debouncer - more on that below.
>>>
>>>>  	/* Do not leak kernel stack to userspace */
>>>>  	memset(&le, 0, sizeof(le));
>>>>  
>>>> @@ -604,6 +703,10 @@ static irqreturn_t edge_irq_handler(int irq, void *p)
>>>>  	struct line *line = p;
>>>>  	struct linereq *lr = line->req;
>>>>  
>>>> +	/* Let HTE supplied callbacks handle */
>>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
>>>> +		return IRQ_HANDLED;
>>>> +
>>>>  	/*
>>>>  	 * Just store the timestamp in hardirq context so we get it as
>>>>  	 * close in time as possible to the actual event.
>>>> @@ -682,14 +785,6 @@ static void debounce_work_func(struct work_struct *work)
>>>>  	/* Do not leak kernel stack to userspace */
>>>>  	memset(&le, 0, sizeof(le));
>>>>  
>>>> -	lr = line->req;
>>>> -	le.timestamp_ns = line_event_timestamp(line);
>>>> -	le.offset = gpio_chip_hwgpio(line->desc);
>>>> -	line->line_seqno++;
>>>> -	le.line_seqno = line->line_seqno;
>>>> -	le.seqno = (lr->num_lines == 1) ?
>>>> -		le.line_seqno : atomic_inc_return(&lr->seqno);
>>>> -
>>>>  	if (level)
>>>>  		/* Emit low-to-high event */
>>>>  		le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>> @@ -697,6 +792,23 @@ static void debounce_work_func(struct work_struct *work)
>>>>  		/* Emit high-to-low event */
>>>>  		le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>>  
>>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
>>>> +		le.timestamp_ns = line->timestamp_ns;
>>>> +		if (line->dir < HTE_DIR_NOSUPP)
>>>> +			le.id = (line->dir == HTE_RISING_EDGE_TS) ?
>>>> +				 GPIO_V2_LINE_EVENT_RISING_EDGE :
>>>> +				 GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>> +	} else {
>>>> +		le.timestamp_ns = line_event_timestamp(line);
>>>> +	}
>>>> +
>>> Move the FLAG_EVENT_CLOCK_HARDWARE check into line_event_timestamp().
>>>
>>> And the id fudging is necessary because the level returned by
>>> gpiod_get_raw_value_cansleep() can disagree with the level from hte?
>>> So you are still trying to synchronise events from two streams.
>>> And that is still broken.
>>> If a hte event occurs between the level being sampled by
>>> gpiod_get_raw_value_cansleep() and the line->dir being read then the line
>>> will have toggled and you will be reporting the opposite state than the
>>> one the debouncer determined was stable.  And maybe the wrong timestamp as
>>> well.
>>>
>>> For lines where hte is enabled, the hte should be the source of level for
>>> the debouncer, not the raw value.  And the mod_delayed_work() that
>>> drives the debouncer should be called by a hte handler, not an irq handler.
>>>
>>> There is also a race on reading the hte timestamp (line->timestamp_ns) and
>>> the hte level (line->dir), such that you can get the level from one event
>>> the timestamp from another.
>>>
>>>> +	lr = line->req;
>>>> +	le.offset = gpio_chip_hwgpio(line->desc);
>>>> +	line->line_seqno++;
>>>> +	le.line_seqno = line->line_seqno;
>>>> +	le.seqno = (lr->num_lines == 1) ?
>>>> +		le.line_seqno : atomic_inc_return(&lr->seqno);
>>>> +
>>> What is the purpose of moving this block of code moved from before the
>>> if (level)?
>>>
>>>
>>>>  	linereq_put_event(lr, &le);
>>>>  }
>>>>  
>>>> @@ -891,7 +1003,6 @@ static int gpio_v2_line_flags_validate(u64 flags)
>>>>  	/* Return an error if an unknown flag is set */
>>>>  	if (flags & ~GPIO_V2_LINE_VALID_FLAGS)
>>>>  		return -EINVAL;
>>>> -
>>> Gratuitous whitespace change.
>>>
>>>>  	/*
>>>>  	 * Do not allow both INPUT and OUTPUT flags to be set as they are
>>>>  	 * contradictory.
>>>> @@ -900,6 +1011,11 @@ static int gpio_v2_line_flags_validate(u64 flags)
>>>>  	    (flags & GPIO_V2_LINE_FLAG_OUTPUT))
>>>>  		return -EINVAL;
>>>>  
>>>> +	/* Only allow one event clock source */
>>>> +	if ((flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME) &&
>>>> +	    (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE))
>>>> +		return -EINVAL;
>>>> +
>>>>  	/* Edge detection requires explicit input. */
>>>>  	if ((flags & GPIO_V2_LINE_EDGE_FLAGS) &&
>>>>  	    !(flags & GPIO_V2_LINE_FLAG_INPUT))
>>>> @@ -992,6 +1108,8 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 flags,
>>>>  
>>>>  	assign_bit(FLAG_EVENT_CLOCK_REALTIME, flagsp,
>>>>  		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME);
>>>> +	assign_bit(FLAG_EVENT_CLOCK_HARDWARE, flagsp,
>>>> +		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE);
>>>>  }
>>>>  
>>>>  static long linereq_get_values(struct linereq *lr, void __user *ip)
>>>> @@ -1154,6 +1272,21 @@ static long linereq_set_config_unlocked(struct linereq *lr,
>>>>  				return ret;
>>>>  		}
>>>>  
>>>> +		/* Check if new config sets hardware assisted clock */
>>>> +		if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
>>>> +			ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
>>>> +							process_hw_ts_thread,
>>>> +							&lr->lines[i]);
>>>> +			if (ret)
>>>> +				return ret;
>>> Note that the line config is the complete line config, not a delta.
>>>
>>> What happens when a line that already has hte enabled is reconfigured
>>> and still has hte enabled?  i.e. what happens when
>>> gpiod_req_hw_timestamp_ns() is called for the second time?
>> HTE will return without doing anything with error code.
>>
> But this is not an error case, it is a normal reconfigure of an
> attribute other than the hte flag.

I assumed when this function is called it will "reset" and not  update the configs.

If this assumption is wrong, I will correct the logic here.

> And that will now return an error to userspace?
>
> Cheers,
> Kent.
>
>>> You provide a comment for the release case below, what of the request
>>> case?
>>>
>>> If you need to check for change then compare the old and new flags, as
>>> the polarity_change check does (not visible in the diff here).
>>>
>>>> +		} else {
>>>> +			/*
>>>> +			 * HTE subsys will do nothing if there is nothing to
>>>> +			 * release.
>>>> +			 */
>>>> +			gpiod_rel_hw_timestamp_ns(desc);
>>>> +		}
>>>> +
>>> Comment will fit on one line.
>>>
>>> And it would be better to document that the function is idempotent in the
>>> function documentation, not everywhere it is used.
>>>
>>>>  		blocking_notifier_call_chain(&desc->gdev->notifier,
>>>>  					     GPIO_V2_LINE_CHANGED_CONFIG,
>>>>  					     desc);
>>>> @@ -1409,6 +1542,14 @@ static int linereq_create(struct gpio_device *gdev, void __user *ip)
>>>>  					flags & GPIO_V2_LINE_EDGE_FLAGS);
>>>>  			if (ret)
>>>>  				goto out_free_linereq;
>>>> +
>>>> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
>>>> +				ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
>>>> +							process_hw_ts_thread,
>>>> +							&lr->lines[i]);
>>>> +				if (ret)
>>>> +					goto out_free_linereq;
>>>> +			}
>>>>  		}
>>>>  
>>>>  		blocking_notifier_call_chain(&desc->gdev->notifier,
>>>> @@ -1959,6 +2100,8 @@ static void gpio_desc_to_lineinfo(struct gpio_desc *desc,
>>>>  
>>>>  	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &desc->flags))
>>>>  		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
>>>> +	else if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &desc->flags))
>>>> +		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
>>>>  
>>>>  	debounce_period_us = READ_ONCE(desc->debounce_period_us);
>>>>  	if (debounce_period_us) {
>>>> diff --git a/include/uapi/linux/gpio.h b/include/uapi/linux/gpio.h
>>>> index eaaea3d8e6b4..d360545b4c21 100644
>>>> --- a/include/uapi/linux/gpio.h
>>>> +++ b/include/uapi/linux/gpio.h
>>>> @@ -80,6 +80,7 @@ enum gpio_v2_line_flag {
>>>>  	GPIO_V2_LINE_FLAG_BIAS_PULL_DOWN	= _BITULL(9),
>>>>  	GPIO_V2_LINE_FLAG_BIAS_DISABLED		= _BITULL(10),
>>>>  	GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME	= _BITULL(11),
>>>> +	GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE	= _BITULL(12),
>>>>  };
>>>>  
>>> I'm now thinking this name, "HARDWARE" is too vague, in case other
>>> timestamp source alternatives join the fray, and so should be "HTE".
>>>
>>> Cheers,
>>> Kent.

Re: [RFC v3 09/12] gpiolib: cdev: Add hardware timestamp clock type

[Kent Gibson]

On Wed, Dec 01, 2021 at 10:01:46AM -0800, Dipen Patel wrote:
> Hi,
> 
> 
> On 12/1/21 9:16 AM, Kent Gibson wrote:
> > On Tue, Nov 30, 2021 at 07:29:20PM -0800, Dipen Patel wrote:
> >> Hi,
> >>
> >> On 11/25/21 5:31 PM, Kent Gibson wrote:
> >>> On Tue, Nov 23, 2021 at 11:30:36AM -0800, Dipen Patel wrote:
> >>>> This patch adds new clock type for the GPIO controller which can
> >>>> timestamp gpio lines in realtime using hardware means. To expose such
> >>>> functionalities to the userspace, code has been added in this patch
> >>>> where during line create call, it checks for new clock type and if
> >>>> requested, calls hardware timestamp related API from gpiolib.c.
> >>>> During line change event, the HTE subsystem pushes timestamp data
> >>>> through callbacks.
> >>>>
> >>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> >>>> Acked-by: Linus Walleij <linus.walleij@linaro.org>
> >>>> ---
> >>>> Changes in v2:
> >>>> - Added hte_dir and static structure hte_ts_desc.
> >>>> - Added callbacks which get invoked by HTE when new data is available.
> >>>> - Better use of hte_dir and seq from hte_ts_desc.
> >>>> - Modified sw debounce function to accommodate hardware timestamping.
> >>>>
> >>>>  drivers/gpio/gpiolib-cdev.c | 161 ++++++++++++++++++++++++++++++++++--
> >>>>  include/uapi/linux/gpio.h   |   1 +
> >>>>  2 files changed, 153 insertions(+), 9 deletions(-)
> >>>>
> >>>> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
> >>>> index c7b5446d01fd..1736ad54e3ec 100644
> >>>> --- a/drivers/gpio/gpiolib-cdev.c
> >>>> +++ b/drivers/gpio/gpiolib-cdev.c
> >>>> @@ -464,6 +464,12 @@ struct line {
> >>>>  	 * stale value.
> >>>>  	 */
> >>>>  	unsigned int level;
> >>>> +	/*
> >>>> +	 * dir will be touched in HTE callbacks hte_ts_cb_t and
> >>>> +	 * hte_ts_threaded_cb_t and they are mutually exclusive. This will be
> >>>> +	 * unused when HTE is not supported/disabled.
> >>>> +	 */
> >>>> +	enum hte_dir dir;
> >>>>  };
> >>>>  
> >>> Documentation should be in present tense, so 
> >>>
> >>> s/will be/is/g
> >>>
> >>> Same applies to other patches.
> >>>
> >>> Also
> >>>
> >>> s/touched/accessed/
> >>>
> >>> dir is a poor name for the field.  It is the hte edge direction and
> >>> effectively the line level, so call it hte_edge_dirn or
> >>> hte_edge_direction or hte_level.
> >>>
> >>> And it is placed in a section of the struct documented as "debouncer specific
> >>> fields", but it is not specfic to the debouncer.  Add a "hte specific
> >>> fields" section if nothing else is suitable.
> >>>
> >>>>  /**
> >>>> @@ -518,6 +524,7 @@ struct linereq {
> >>>>  	 GPIO_V2_LINE_DRIVE_FLAGS | \
> >>>>  	 GPIO_V2_LINE_EDGE_FLAGS | \
> >>>>  	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
> >>>> +	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
> >>>>  	 GPIO_V2_LINE_BIAS_FLAGS)
> >>>>  
> >>>>  static void linereq_put_event(struct linereq *lr,
> >>>> @@ -546,6 +553,94 @@ static u64 line_event_timestamp(struct line *line)
> >>>>  	return ktime_get_ns();
> >>>>  }
> >>>>  
> >>>> +static hte_return_t process_hw_ts_thread(void *p)
> >>>> +{
> >>>> +	struct line *line = p;
> >>>> +	struct linereq *lr = line->req;
> >>>> +	struct gpio_v2_line_event le;
> >>>> +	u64 eflags;
> >>>> +
> >>>> +	memset(&le, 0, sizeof(le));
> >>>> +
> >>>> +	le.timestamp_ns = line->timestamp_ns;
> >>>> +	line->timestamp_ns = 0;
> >>>> +
> >>> What is the purpose of this zeroing?
> >>>
> >>>> +	if (line->dir >= HTE_DIR_NOSUPP) {
> >>>> +		eflags = READ_ONCE(line->eflags);
> >>>> +		if (eflags == GPIO_V2_LINE_FLAG_EDGE_BOTH) {
> >>>> +			int level = gpiod_get_value_cansleep(line->desc);
> >>>> +
> >>>> +			if (level)
> >>>> +				/* Emit low-to-high event */
> >>>> +				le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> >>>> +			else
> >>>> +				/* Emit high-to-low event */
> >>>> +				le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_RISING) {
> >>>> +			/* Emit low-to-high event */
> >>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> >>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_FALLING) {
> >>>> +			/* Emit high-to-low event */
> >>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >>>> +		} else {
> >>>> +			return HTE_CB_ERROR;
> >>>> +		}
> >>>> +	} else {
> >>>> +		if (line->dir == HTE_RISING_EDGE_TS)
> >>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> >>>> +		else
> >>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >>>> +	}
> >>> The mapping from line->dir to le.id needs to take into account the active
> >>> low setting for the line.
> >>>
> >>> And it might be simpler if the hte_ts_data provided the level, equivalent
> >>> to gpiod_get_raw_value_cansleep(), rather than an edge direction, so you
> >>> can provide a common helper to determine the edge given the raw level.
> >> (So from the level determine the edge?) that sound right specially when
> >>
> >> HTE provider has capability to record the edge in that case why bother
> >>
> >> getting the level and determine edge?
> >>
> >> Calculating the edge from the level makes sense when hte provider does not
> >>
> >> have that feature and that is what if (line->dir >= HTE_DIR_NOSUPP) does.
> >>
> > As asked in the review of patch 02, do you have an example of hardware that
> > reports an edge direction rather than NOSUPP?
> No...

So you are adding an interface that nothing will currently use.
Are there plans for hardware that will report the edge, and you are
laying the groundwork here?

> >
> > Anyway, this is just a naming thing - the information content being passed
> > is the the same, be it high/low/unknown or rising/falling/unknown.
> >
> > If the hardware does report edge direction then it is just one bit, and
> > that also corresponds to the physical level immediately following the
> > edge, so no additional conversion required there.
> >
> > It would be clearer to pass a level than an edge, as
> >  - a hte edge (hte_dir) could be confused with a cdev edge
> >    (gpio_v2_line_event_id), in mails like this if not in the code.
> >  - cdev will fallback to using the physical level to determine the edge
> >    if the hte can't provide it
> I believe I have retained all the fallback logic in such case.
> >  - cdev has to perform inversion if active low and it already does that
> >    based on levels
> >
> >>>> +
> >>>> +	le.line_seqno = line->line_seqno;
> >>>> +	le.seqno = (lr->num_lines == 1) ? le.line_seqno : line->req_seqno;
> >>>> +	le.offset = gpio_chip_hwgpio(line->desc);
> >>>> +
> >>>> +	linereq_put_event(lr, &le);
> >>>> +
> >>>> +	return HTE_CB_HANDLED;
> >>>> +}
> >>>> +
> >>>> +static hte_return_t process_hw_ts(struct hte_ts_data *ts, void *p)
> >>>> +{
> >>>> +	struct line *line = p;
> >>>> +	struct linereq *lr = line->req;
> >>>> +
> >>>> +	if (!ts)
> >>>> +		return HTE_CB_ERROR;
> >>>> +
> >>>> +	line->timestamp_ns = ts->tsc;
> >>>> +	line->dir = ts->dir;
> >>>> +
> >>> The doc for timestamp_ns states:
> >>>
> >>> 	 * timestamp_ns and req_seqno are accessed only by
> >>> 	 * edge_irq_handler() and edge_irq_thread(), which are themselves
> >>> 	 * mutually exclusive, so no additional protection is necessary.
> >>>
> >>> That no longer holds.  It is now also accessed here, and in
> >>> process_hw_ts_thread(), which wont run concurrently with each other or
> >>> the edge_irq_* handlers, but also in debounce_work_func() which may run
> >>> concurrently with the others.
> >>> So timestamp_ns now requires protection from concurrent access.
> >>>
> >>>> +	/*
> >>>> +	 * It is possible that HTE engine detects spurious edges for the
> >>>> +	 * lines where software debounce is enabled. This primary callback
> >>>> +	 * will be called multiple times in that case. It will be better to
> >>>> +	 * let debounce_work_func handle instead of process_hw_ts_thread.
> >>>> +	 * The timestamp_ns will be overwritten here which is fine as we are
> >>>> +	 * interested in the last value anyway. The debounce_work_func will
> >>>> +	 * then just read whatever last line->timestamp_ns is stored. Because
> >>>> +	 * this callback can be called multiple times, we are not really
> >>>> +	 * interested in ts->seq.
> >>>> +	 */
> >>> Not sure what this is trying to say.
> >>> Is this the primary callback? Or debounce_irq_handler()?
> >> This is primary callback called from HTE when it pushes new TS data per line, it
> >>
> >> also says so in the second line.
> >>
> > Yeah, I probably read that as "The primary callback", but it is
> > confusing anyway. "This primary callback" implies there is another
> > primary callback. 
> > Just say "This handler" instead of "This primary callback".
> Noted...
> >
> >>> You say you really aren't interested in ts->seq, but the code immediately
> >>> uses it.
> >> That is when sw_debounced is not set and whole paragraph is about when
> >>
> >> sw_debounced is set.
> >>
> > So your whole comment here is about the else case?
> > Then either put the comment where the else would be, or better yet invert
> > the logic and return immediately if sw_debounced.
> Sure...

Understand, you wont be maintaining this code, even if you intend to.
Consider the poor unfortunate who will have to deal with your code in
the future.  This is not and should not be a minor consideration.

Sometimes long winded comments only add confusion rather than clarity.
If the code alone is confusing and requires more than a line or two of
explanatory comments, excluding function documentation, then you might
want to rethink your code.

In this case the clearest is probably to restructure the if condition as
I suggested and simplify or even drop the comment.

> >
> >>> Reword to clarify.
> >>> And add braces after function names to highlight them, so
> >>> debounce_work_func().
> >> Will do.
> >>>> +	if (!READ_ONCE(line->sw_debounced)) {
> >>>> +		line->line_seqno = ts->seq;
> >>>> +
> >>>> +		/*
> >>>> +		 * Increment in this callback incase all the lines in linereq
> >>>> +		 * are enabled for hw timestamping. This will work even if
> >>>> +		 * subset of lines are enabled for hw timestamping as
> >>>> +		 * edge_irq_* callbacks will proceed as usual for them.
> >>>> +		 */
> >>> s/incase/in case/
> >>>
> >>> Not sure what the comment is trying to say. There is no check here that
> >>> the other lines have HTE enabled.  And that is not relevant anyway.
> >>> The edge_irq_* handlers will proceed as usual for those lines NOT
> >>> enabled for hw timestamping.
> >>>
> >>> To clarify, the line_seqno indicates where this event lies in the
> >>> sequence of events for the line.
> >>> The request seqno indicates where this event lines in the sequence of
> >>> events for the request.
> >>> For a single line request these are the same, hence the minor
> >>> optimisation of not updating lr->seqno below.
> >>>
> >>>> +		if (lr->num_lines != 1)
> >>>> +			line->req_seqno = atomic_inc_return(&lr->seqno);
> >>>> +
> >>> The req_seqno should be updated corresponding to the change in the
> >>> line_reqno.  That always used to be 1, but no longer if hte can discard
> >>> events, i.e. skip over line_seqnos.
> >> HTE does not discard any events, it pushes to clients as soon as its
> >>
> >> available through primary callback.
> > The discarding of events that I am referring to is from your previous
> > answers that indicated, to me anyway, that there could be gaps in the
> > ts->seq numbers if the hardware event FIFO overflowed.
> > Is that not the case?
> 
> Not in this patch series as the provider I have dealt with does not such
> 
> feature. ts->seq is software counter in that case.
> 

The code here has to deal with the general case, not just the one example
driver you have provided.  So in general there COULD be gaps in the
ts->seq, right?

I do see that using the ts-seq for sw debounced lines is problematic
though. The debouncer itself will be discarding hte events, but that
shouldn't be considered a lost event to the user.  You could track
how many events are discarded by the debouncer and subtract those from
the sequence numbers reported to userspace?

> >
> > And when you say "primary callback", both here and elsewhere, you mean the
> > cb parameter to gpiod_req_hw_timestamp_ns() and gc->req_hw_timestamp()?
> yes, cb is primary and tcb (threaded cb) is optional secondary.
> > Either way, be specific and name the function or parameter, or find a
> > better term than "primary callback".
> > In this case "the client's event handler" would be much clearer.
> Noted...
> >
> >>> To be consistent, i.e. if events were lost for this line then they were
> >>> also lost for the requested lines, the lr->seqno should be incremented by
> >>> the change in line_seqno.  Probably with some sanity checks.
> >>>
> >>>> +		return HTE_RUN_THREADED_CB;
> >>>> +	}
> >>>> +
> >>>> +	return HTE_CB_HANDLED;
> >>>> +}
> >>>> +
> >>>>  static irqreturn_t edge_irq_thread(int irq, void *p)
> >>>>  {
> >>>>  	struct line *line = p;
> >>>> @@ -553,6 +648,10 @@ static irqreturn_t edge_irq_thread(int irq, void *p)
> >>>>  	struct gpio_v2_line_event le;
> >>>>  	u64 eflags;
> >>>>  
> >>>> +	/* Let process_hw_ts_thread handle */
> >>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
> >>>> +		return IRQ_HANDLED;
> >>>> +
> >>> This adds pointless runtime overhead, and for everyone not just hte users.
> >>> Don't stub out a handler in the handler - stub it out where it is
> >>> registered by registering a stub handler.  Or don't request it at all.
> >>>
> >>> So why would gpiolib-cdev be requesting the irq, only to stub out
> >>> the handlers?
> >>> If that has a side-effect that hte requires then hte should be taking
> >>> care of it - it is not gpiolib-cdev's problem.
> >> - Why stop at moving irq and debounce related stuff to hte then?
> >>
> > How about you answer my question before asking your own?
> > Here I am only questioning why gpiolib-cdev is requesting an interrupt
> > for a hte line at all.  It has no reason to, so hte must have?
> 
> Commented below in "separation of concern" paragraph.
> 
> > And where did I suggest moving the "debounce stuff" to hte?
> 
> Perhaps I misunderstood "That includes the sw debouncer - more on that
> 
> below." comment.
> 

The point I was trying to make there was that, for hte enabled lines, the
sw debouncer should be driven by hte events, not irq events.
That does not imply nor require moving the debouncer to hte.

> >
> > What I am suggesting is called separation of concerns.  In this context
> > the intent is to look towards abstracting the edge event generation.
> > Having hte and irq entangled for no apparent reason makes doing that more
> 
> But in this patch series (i.e. HTE provider), they are entangled as hte provider
> 
> will only ts the GPIO line which is configured as input irq. That is why I have
> 
> separated GPIO line config part and TS part. In other words, let cdev handle
> 
> any line config that userspace has asked and let hte handle ts part if the
> 
> line config is supported (If line config is not suitable, it will not enable hte, see
> 
> gpio-tegra186.c function tegra186_gpio_req_hw_ts() in this patch series where
> 
> check happens for the tegra hte provider).
> 

Can you explain "hte provider will only ts the GPIO line which is
configured as input irq"?  That sounds like a hte concern to me, or even
worse a particular hte provider problem.

The user requests a line with edge event detection enabled and with hte
timestamps.
Nothing in that requires irq from gpiolib-cdev.
gpiolib-cdev should use hte as the event source where it would usually
use irq. If hte requires irq to allow it to generate those events then
hte should be responsible for requesting the irq.
In a hypothetical extreme case where gpiolib-cdev only supported hte
lines there should be no reference to irq in gpiolib-cdev.
Yet you continue to insist that gpiolib-cdev should request the irq for
hte.
What am I missing here?

> > difficult than it needs to be, whereas keeping them separate greatly
> > simplifies identification of common code suitable for refactoring
> > subsequently.
> >
> > Not sure what to call what you are suggesting.
> >
> >> I mean if there is hte provider which can TS GPIO output/input
> >>
> >> does it mean hte is responsible for parsing the GPIO line configs, setting them up
> >>
> >> (i.e. input or output) as well? Are we not duplicating logic instead of
> >>
> >> leveraging gpio-cdev? Does it make sense for the HTE subsystem which not
> >>
> >> only TS the GPIOs but other SoC lines?
> >>
> >> - What happens to in kernel GPIO HTE client (for example, hte-tegra194-gpio-test.c)?
> >>
> >> some clients do more in their IRQ handler than what edge_irq_handler does in which
> >>
> >> case it would make sense to have them request irq in their code than through HTE.
> >>
> >>> And speaking as to how the whole hte/gpiolib-cdev interface should work,
> >>> hte should be an edge event generator alternative to irq.  So lines with
> >>> hte enabled should work without any irq calls from gpiolib-cdev.
> >>> That includes the sw debouncer - more on that below.
> >>>
> >>>>  	/* Do not leak kernel stack to userspace */
> >>>>  	memset(&le, 0, sizeof(le));
> >>>>  
> >>>> @@ -604,6 +703,10 @@ static irqreturn_t edge_irq_handler(int irq, void *p)
> >>>>  	struct line *line = p;
> >>>>  	struct linereq *lr = line->req;
> >>>>  
> >>>> +	/* Let HTE supplied callbacks handle */
> >>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
> >>>> +		return IRQ_HANDLED;
> >>>> +
> >>>>  	/*
> >>>>  	 * Just store the timestamp in hardirq context so we get it as
> >>>>  	 * close in time as possible to the actual event.
> >>>> @@ -682,14 +785,6 @@ static void debounce_work_func(struct work_struct *work)
> >>>>  	/* Do not leak kernel stack to userspace */
> >>>>  	memset(&le, 0, sizeof(le));
> >>>>  
> >>>> -	lr = line->req;
> >>>> -	le.timestamp_ns = line_event_timestamp(line);
> >>>> -	le.offset = gpio_chip_hwgpio(line->desc);
> >>>> -	line->line_seqno++;
> >>>> -	le.line_seqno = line->line_seqno;
> >>>> -	le.seqno = (lr->num_lines == 1) ?
> >>>> -		le.line_seqno : atomic_inc_return(&lr->seqno);
> >>>> -
> >>>>  	if (level)
> >>>>  		/* Emit low-to-high event */
> >>>>  		le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> >>>> @@ -697,6 +792,23 @@ static void debounce_work_func(struct work_struct *work)
> >>>>  		/* Emit high-to-low event */
> >>>>  		le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >>>>  
> >>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
> >>>> +		le.timestamp_ns = line->timestamp_ns;
> >>>> +		if (line->dir < HTE_DIR_NOSUPP)
> >>>> +			le.id = (line->dir == HTE_RISING_EDGE_TS) ?
> >>>> +				 GPIO_V2_LINE_EVENT_RISING_EDGE :
> >>>> +				 GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >>>> +	} else {
> >>>> +		le.timestamp_ns = line_event_timestamp(line);
> >>>> +	}
> >>>> +
> >>> Move the FLAG_EVENT_CLOCK_HARDWARE check into line_event_timestamp().
> >>>
> >>> And the id fudging is necessary because the level returned by
> >>> gpiod_get_raw_value_cansleep() can disagree with the level from hte?
> >>> So you are still trying to synchronise events from two streams.
> >>> And that is still broken.
> >>> If a hte event occurs between the level being sampled by
> >>> gpiod_get_raw_value_cansleep() and the line->dir being read then the line
> >>> will have toggled and you will be reporting the opposite state than the
> >>> one the debouncer determined was stable.  And maybe the wrong timestamp as
> >>> well.
> >>>
> >>> For lines where hte is enabled, the hte should be the source of level for
> >>> the debouncer, not the raw value.  And the mod_delayed_work() that
> >>> drives the debouncer should be called by a hte handler, not an irq handler.
> >>>
> >>> There is also a race on reading the hte timestamp (line->timestamp_ns) and
> >>> the hte level (line->dir), such that you can get the level from one event
> >>> the timestamp from another.
> >>>
> >>>> +	lr = line->req;
> >>>> +	le.offset = gpio_chip_hwgpio(line->desc);
> >>>> +	line->line_seqno++;
> >>>> +	le.line_seqno = line->line_seqno;
> >>>> +	le.seqno = (lr->num_lines == 1) ?
> >>>> +		le.line_seqno : atomic_inc_return(&lr->seqno);
> >>>> +
> >>> What is the purpose of moving this block of code moved from before the
> >>> if (level)?
> >>>
> >>>
> >>>>  	linereq_put_event(lr, &le);
> >>>>  }
> >>>>  
> >>>> @@ -891,7 +1003,6 @@ static int gpio_v2_line_flags_validate(u64 flags)
> >>>>  	/* Return an error if an unknown flag is set */
> >>>>  	if (flags & ~GPIO_V2_LINE_VALID_FLAGS)
> >>>>  		return -EINVAL;
> >>>> -
> >>> Gratuitous whitespace change.
> >>>
> >>>>  	/*
> >>>>  	 * Do not allow both INPUT and OUTPUT flags to be set as they are
> >>>>  	 * contradictory.
> >>>> @@ -900,6 +1011,11 @@ static int gpio_v2_line_flags_validate(u64 flags)
> >>>>  	    (flags & GPIO_V2_LINE_FLAG_OUTPUT))
> >>>>  		return -EINVAL;
> >>>>  
> >>>> +	/* Only allow one event clock source */
> >>>> +	if ((flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME) &&
> >>>> +	    (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE))
> >>>> +		return -EINVAL;
> >>>> +
> >>>>  	/* Edge detection requires explicit input. */
> >>>>  	if ((flags & GPIO_V2_LINE_EDGE_FLAGS) &&
> >>>>  	    !(flags & GPIO_V2_LINE_FLAG_INPUT))
> >>>> @@ -992,6 +1108,8 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 flags,
> >>>>  
> >>>>  	assign_bit(FLAG_EVENT_CLOCK_REALTIME, flagsp,
> >>>>  		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME);
> >>>> +	assign_bit(FLAG_EVENT_CLOCK_HARDWARE, flagsp,
> >>>> +		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE);
> >>>>  }
> >>>>  
> >>>>  static long linereq_get_values(struct linereq *lr, void __user *ip)
> >>>> @@ -1154,6 +1272,21 @@ static long linereq_set_config_unlocked(struct linereq *lr,
> >>>>  				return ret;
> >>>>  		}
> >>>>  
> >>>> +		/* Check if new config sets hardware assisted clock */
> >>>> +		if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
> >>>> +			ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
> >>>> +							process_hw_ts_thread,
> >>>> +							&lr->lines[i]);
> >>>> +			if (ret)
> >>>> +				return ret;
> >>> Note that the line config is the complete line config, not a delta.
> >>>
> >>> What happens when a line that already has hte enabled is reconfigured
> >>> and still has hte enabled?  i.e. what happens when
> >>> gpiod_req_hw_timestamp_ns() is called for the second time?
> >> HTE will return without doing anything with error code.
> >>
> > But this is not an error case, it is a normal reconfigure of an
> > attribute other than the hte flag.
> 
> I assumed when this function is called it will "reset" and not  update the configs.
> 
> If this assumption is wrong, I will correct the logic here.
> 

The set_config does whatever is required to change the line request
config from the old to the new.  Both old and new are complete snapshots
of the line request config.

The change should be seamless except for the attributes being changed.
We certainly do not reset the config and start from scratch - that would
be little better than the user releasing and re-requesting the lines.

For many attributes we can just apply the new config.  But some cases,
such as a change to active low polarity, are a lot more involved.
And if hte considers re-requesting the line to be an error then you
should only make the hte request when the hte flag changes to set.

Cheers,
Kent.

> > And that will now return an error to userspace?
> >
> > Cheers,
> > Kent.
> >
> >>> You provide a comment for the release case below, what of the request
> >>> case?
> >>>
> >>> If you need to check for change then compare the old and new flags, as
> >>> the polarity_change check does (not visible in the diff here).
> >>>
> >>>> +		} else {
> >>>> +			/*
> >>>> +			 * HTE subsys will do nothing if there is nothing to
> >>>> +			 * release.
> >>>> +			 */
> >>>> +			gpiod_rel_hw_timestamp_ns(desc);
> >>>> +		}
> >>>> +
> >>> Comment will fit on one line.
> >>>
> >>> And it would be better to document that the function is idempotent in the
> >>> function documentation, not everywhere it is used.
> >>>
> >>>>  		blocking_notifier_call_chain(&desc->gdev->notifier,
> >>>>  					     GPIO_V2_LINE_CHANGED_CONFIG,
> >>>>  					     desc);
> >>>> @@ -1409,6 +1542,14 @@ static int linereq_create(struct gpio_device *gdev, void __user *ip)
> >>>>  					flags & GPIO_V2_LINE_EDGE_FLAGS);
> >>>>  			if (ret)
> >>>>  				goto out_free_linereq;
> >>>> +
> >>>> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
> >>>> +				ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
> >>>> +							process_hw_ts_thread,
> >>>> +							&lr->lines[i]);
> >>>> +				if (ret)
> >>>> +					goto out_free_linereq;
> >>>> +			}
> >>>>  		}
> >>>>  
> >>>>  		blocking_notifier_call_chain(&desc->gdev->notifier,
> >>>> @@ -1959,6 +2100,8 @@ static void gpio_desc_to_lineinfo(struct gpio_desc *desc,
> >>>>  
> >>>>  	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &desc->flags))
> >>>>  		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
> >>>> +	else if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &desc->flags))
> >>>> +		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
> >>>>  
> >>>>  	debounce_period_us = READ_ONCE(desc->debounce_period_us);
> >>>>  	if (debounce_period_us) {
> >>>> diff --git a/include/uapi/linux/gpio.h b/include/uapi/linux/gpio.h
> >>>> index eaaea3d8e6b4..d360545b4c21 100644
> >>>> --- a/include/uapi/linux/gpio.h
> >>>> +++ b/include/uapi/linux/gpio.h
> >>>> @@ -80,6 +80,7 @@ enum gpio_v2_line_flag {
> >>>>  	GPIO_V2_LINE_FLAG_BIAS_PULL_DOWN	= _BITULL(9),
> >>>>  	GPIO_V2_LINE_FLAG_BIAS_DISABLED		= _BITULL(10),
> >>>>  	GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME	= _BITULL(11),
> >>>> +	GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE	= _BITULL(12),
> >>>>  };
> >>>>  
> >>> I'm now thinking this name, "HARDWARE" is too vague, in case other
> >>> timestamp source alternatives join the fray, and so should be "HTE".
> >>>
> >>> Cheers,
> >>> Kent.

Re: [RFC v3 09/12] gpiolib: cdev: Add hardware timestamp clock type

[Dipen Patel]

On 12/1/21 4:53 PM, Kent Gibson wrote:
> On Wed, Dec 01, 2021 at 10:01:46AM -0800, Dipen Patel wrote:
>> Hi,
>>
>>
>> On 12/1/21 9:16 AM, Kent Gibson wrote:
>>> On Tue, Nov 30, 2021 at 07:29:20PM -0800, Dipen Patel wrote:
>>>> Hi,
>>>>
>>>> On 11/25/21 5:31 PM, Kent Gibson wrote:
>>>>> On Tue, Nov 23, 2021 at 11:30:36AM -0800, Dipen Patel wrote:
>>>>>> This patch adds new clock type for the GPIO controller which can
>>>>>> timestamp gpio lines in realtime using hardware means. To expose such
>>>>>> functionalities to the userspace, code has been added in this patch
>>>>>> where during line create call, it checks for new clock type and if
>>>>>> requested, calls hardware timestamp related API from gpiolib.c.
>>>>>> During line change event, the HTE subsystem pushes timestamp data
>>>>>> through callbacks.
>>>>>>
>>>>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>>>>>> Acked-by: Linus Walleij <linus.walleij@linaro.org>
>>>>>> ---
>>>>>> Changes in v2:
>>>>>> - Added hte_dir and static structure hte_ts_desc.
>>>>>> - Added callbacks which get invoked by HTE when new data is available.
>>>>>> - Better use of hte_dir and seq from hte_ts_desc.
>>>>>> - Modified sw debounce function to accommodate hardware timestamping.
>>>>>>
>>>>>>  drivers/gpio/gpiolib-cdev.c | 161 ++++++++++++++++++++++++++++++++++--
>>>>>>  include/uapi/linux/gpio.h   |   1 +
>>>>>>  2 files changed, 153 insertions(+), 9 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
>>>>>> index c7b5446d01fd..1736ad54e3ec 100644
>>>>>> --- a/drivers/gpio/gpiolib-cdev.c
>>>>>> +++ b/drivers/gpio/gpiolib-cdev.c
>>>>>> @@ -464,6 +464,12 @@ struct line {
>>>>>>  	 * stale value.
>>>>>>  	 */
>>>>>>  	unsigned int level;
>>>>>> +	/*
>>>>>> +	 * dir will be touched in HTE callbacks hte_ts_cb_t and
>>>>>> +	 * hte_ts_threaded_cb_t and they are mutually exclusive. This will be
>>>>>> +	 * unused when HTE is not supported/disabled.
>>>>>> +	 */
>>>>>> +	enum hte_dir dir;
>>>>>>  };
>>>>>>  
>>>>> Documentation should be in present tense, so 
>>>>>
>>>>> s/will be/is/g
>>>>>
>>>>> Same applies to other patches.
>>>>>
>>>>> Also
>>>>>
>>>>> s/touched/accessed/
>>>>>
>>>>> dir is a poor name for the field.  It is the hte edge direction and
>>>>> effectively the line level, so call it hte_edge_dirn or
>>>>> hte_edge_direction or hte_level.
>>>>>
>>>>> And it is placed in a section of the struct documented as "debouncer specific
>>>>> fields", but it is not specfic to the debouncer.  Add a "hte specific
>>>>> fields" section if nothing else is suitable.
>>>>>
>>>>>>  /**
>>>>>> @@ -518,6 +524,7 @@ struct linereq {
>>>>>>  	 GPIO_V2_LINE_DRIVE_FLAGS | \
>>>>>>  	 GPIO_V2_LINE_EDGE_FLAGS | \
>>>>>>  	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
>>>>>> +	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
>>>>>>  	 GPIO_V2_LINE_BIAS_FLAGS)
>>>>>>  
>>>>>>  static void linereq_put_event(struct linereq *lr,
>>>>>> @@ -546,6 +553,94 @@ static u64 line_event_timestamp(struct line *line)
>>>>>>  	return ktime_get_ns();
>>>>>>  }
>>>>>>  
>>>>>> +static hte_return_t process_hw_ts_thread(void *p)
>>>>>> +{
>>>>>> +	struct line *line = p;
>>>>>> +	struct linereq *lr = line->req;
>>>>>> +	struct gpio_v2_line_event le;
>>>>>> +	u64 eflags;
>>>>>> +
>>>>>> +	memset(&le, 0, sizeof(le));
>>>>>> +
>>>>>> +	le.timestamp_ns = line->timestamp_ns;
>>>>>> +	line->timestamp_ns = 0;
>>>>>> +
>>>>> What is the purpose of this zeroing?
>>>>>
>>>>>> +	if (line->dir >= HTE_DIR_NOSUPP) {
>>>>>> +		eflags = READ_ONCE(line->eflags);
>>>>>> +		if (eflags == GPIO_V2_LINE_FLAG_EDGE_BOTH) {
>>>>>> +			int level = gpiod_get_value_cansleep(line->desc);
>>>>>> +
>>>>>> +			if (level)
>>>>>> +				/* Emit low-to-high event */
>>>>>> +				le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>>>> +			else
>>>>>> +				/* Emit high-to-low event */
>>>>>> +				le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_RISING) {
>>>>>> +			/* Emit low-to-high event */
>>>>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_FALLING) {
>>>>>> +			/* Emit high-to-low event */
>>>>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>>>> +		} else {
>>>>>> +			return HTE_CB_ERROR;
>>>>>> +		}
>>>>>> +	} else {
>>>>>> +		if (line->dir == HTE_RISING_EDGE_TS)
>>>>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>>>> +		else
>>>>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>>>> +	}
>>>>> The mapping from line->dir to le.id needs to take into account the active
>>>>> low setting for the line.
>>>>>
>>>>> And it might be simpler if the hte_ts_data provided the level, equivalent
>>>>> to gpiod_get_raw_value_cansleep(), rather than an edge direction, so you
>>>>> can provide a common helper to determine the edge given the raw level.
>>>> (So from the level determine the edge?) that sound right specially when
>>>>
>>>> HTE provider has capability to record the edge in that case why bother
>>>>
>>>> getting the level and determine edge?
>>>>
>>>> Calculating the edge from the level makes sense when hte provider does not
>>>>
>>>> have that feature and that is what if (line->dir >= HTE_DIR_NOSUPP) does.
>>>>
>>> As asked in the review of patch 02, do you have an example of hardware that
>>> reports an edge direction rather than NOSUPP?
>> No...
> So you are adding an interface that nothing will currently use.
> Are there plans for hardware that will report the edge, and you are
> laying the groundwork here?
>
>>> Anyway, this is just a naming thing - the information content being passed
>>> is the the same, be it high/low/unknown or rising/falling/unknown.
>>>
>>> If the hardware does report edge direction then it is just one bit, and
>>> that also corresponds to the physical level immediately following the
>>> edge, so no additional conversion required there.
>>>
>>> It would be clearer to pass a level than an edge, as
>>>  - a hte edge (hte_dir) could be confused with a cdev edge
>>>    (gpio_v2_line_event_id), in mails like this if not in the code.
>>>  - cdev will fallback to using the physical level to determine the edge
>>>    if the hte can't provide it
>> I believe I have retained all the fallback logic in such case.
>>>  - cdev has to perform inversion if active low and it already does that
>>>    based on levels
>>>
>>>>>> +
>>>>>> +	le.line_seqno = line->line_seqno;
>>>>>> +	le.seqno = (lr->num_lines == 1) ? le.line_seqno : line->req_seqno;
>>>>>> +	le.offset = gpio_chip_hwgpio(line->desc);
>>>>>> +
>>>>>> +	linereq_put_event(lr, &le);
>>>>>> +
>>>>>> +	return HTE_CB_HANDLED;
>>>>>> +}
>>>>>> +
>>>>>> +static hte_return_t process_hw_ts(struct hte_ts_data *ts, void *p)
>>>>>> +{
>>>>>> +	struct line *line = p;
>>>>>> +	struct linereq *lr = line->req;
>>>>>> +
>>>>>> +	if (!ts)
>>>>>> +		return HTE_CB_ERROR;
>>>>>> +
>>>>>> +	line->timestamp_ns = ts->tsc;
>>>>>> +	line->dir = ts->dir;
>>>>>> +
>>>>> The doc for timestamp_ns states:
>>>>>
>>>>> 	 * timestamp_ns and req_seqno are accessed only by
>>>>> 	 * edge_irq_handler() and edge_irq_thread(), which are themselves
>>>>> 	 * mutually exclusive, so no additional protection is necessary.
>>>>>
>>>>> That no longer holds.  It is now also accessed here, and in
>>>>> process_hw_ts_thread(), which wont run concurrently with each other or
>>>>> the edge_irq_* handlers, but also in debounce_work_func() which may run
>>>>> concurrently with the others.
>>>>> So timestamp_ns now requires protection from concurrent access.
>>>>>
>>>>>> +	/*
>>>>>> +	 * It is possible that HTE engine detects spurious edges for the
>>>>>> +	 * lines where software debounce is enabled. This primary callback
>>>>>> +	 * will be called multiple times in that case. It will be better to
>>>>>> +	 * let debounce_work_func handle instead of process_hw_ts_thread.
>>>>>> +	 * The timestamp_ns will be overwritten here which is fine as we are
>>>>>> +	 * interested in the last value anyway. The debounce_work_func will
>>>>>> +	 * then just read whatever last line->timestamp_ns is stored. Because
>>>>>> +	 * this callback can be called multiple times, we are not really
>>>>>> +	 * interested in ts->seq.
>>>>>> +	 */
>>>>> Not sure what this is trying to say.
>>>>> Is this the primary callback? Or debounce_irq_handler()?
>>>> This is primary callback called from HTE when it pushes new TS data per line, it
>>>>
>>>> also says so in the second line.
>>>>
>>> Yeah, I probably read that as "The primary callback", but it is
>>> confusing anyway. "This primary callback" implies there is another
>>> primary callback. 
>>> Just say "This handler" instead of "This primary callback".
>> Noted...
>>>>> You say you really aren't interested in ts->seq, but the code immediately
>>>>> uses it.
>>>> That is when sw_debounced is not set and whole paragraph is about when
>>>>
>>>> sw_debounced is set.
>>>>
>>> So your whole comment here is about the else case?
>>> Then either put the comment where the else would be, or better yet invert
>>> the logic and return immediately if sw_debounced.
>> Sure...
> Understand, you wont be maintaining this code, even if you intend to.
> Consider the poor unfortunate who will have to deal with your code in
> the future.  This is not and should not be a minor consideration.
>
> Sometimes long winded comments only add confusion rather than clarity.
> If the code alone is confusing and requires more than a line or two of
> explanatory comments, excluding function documentation, then you might
> want to rethink your code.
>
> In this case the clearest is probably to restructure the if condition as
> I suggested and simplify or even drop the comment.
Agreed...
>
>>>>> Reword to clarify.
>>>>> And add braces after function names to highlight them, so
>>>>> debounce_work_func().
>>>> Will do.
>>>>>> +	if (!READ_ONCE(line->sw_debounced)) {
>>>>>> +		line->line_seqno = ts->seq;
>>>>>> +
>>>>>> +		/*
>>>>>> +		 * Increment in this callback incase all the lines in linereq
>>>>>> +		 * are enabled for hw timestamping. This will work even if
>>>>>> +		 * subset of lines are enabled for hw timestamping as
>>>>>> +		 * edge_irq_* callbacks will proceed as usual for them.
>>>>>> +		 */
>>>>> s/incase/in case/
>>>>>
>>>>> Not sure what the comment is trying to say. There is no check here that
>>>>> the other lines have HTE enabled.  And that is not relevant anyway.
>>>>> The edge_irq_* handlers will proceed as usual for those lines NOT
>>>>> enabled for hw timestamping.
>>>>>
>>>>> To clarify, the line_seqno indicates where this event lies in the
>>>>> sequence of events for the line.
>>>>> The request seqno indicates where this event lines in the sequence of
>>>>> events for the request.
>>>>> For a single line request these are the same, hence the minor
>>>>> optimisation of not updating lr->seqno below.
>>>>>
>>>>>> +		if (lr->num_lines != 1)
>>>>>> +			line->req_seqno = atomic_inc_return(&lr->seqno);
>>>>>> +
>>>>> The req_seqno should be updated corresponding to the change in the
>>>>> line_reqno.  That always used to be 1, but no longer if hte can discard
>>>>> events, i.e. skip over line_seqnos.
>>>> HTE does not discard any events, it pushes to clients as soon as its
>>>>
>>>> available through primary callback.
>>> The discarding of events that I am referring to is from your previous
>>> answers that indicated, to me anyway, that there could be gaps in the
>>> ts->seq numbers if the hardware event FIFO overflowed.
>>> Is that not the case?
>> Not in this patch series as the provider I have dealt with does not such
>>
>> feature. ts->seq is software counter in that case.
>>
> The code here has to deal with the general case, not just the one example
> driver you have provided.  So in general there COULD be gaps in the
> ts->seq, right?

Good point. I agree for the general case and also agree that it could have gap when such

providers exist. I will revisit this logic in next version 4.

>
> I do see that using the ts-seq for sw debounced lines is problematic
> though. The debouncer itself will be discarding hte events, but that
> shouldn't be considered a lost event to the user.  You could track
> how many events are discarded by the debouncer and subtract those from
> the sequence numbers reported to userspace?
I agree, I will correct the logic in next version 4.
>
>>> And when you say "primary callback", both here and elsewhere, you mean the
>>> cb parameter to gpiod_req_hw_timestamp_ns() and gc->req_hw_timestamp()?
>> yes, cb is primary and tcb (threaded cb) is optional secondary.
>>> Either way, be specific and name the function or parameter, or find a
>>> better term than "primary callback".
>>> In this case "the client's event handler" would be much clearer.
>> Noted...
>>>>> To be consistent, i.e. if events were lost for this line then they were
>>>>> also lost for the requested lines, the lr->seqno should be incremented by
>>>>> the change in line_seqno.  Probably with some sanity checks.
>>>>>
>>>>>> +		return HTE_RUN_THREADED_CB;
>>>>>> +	}
>>>>>> +
>>>>>> +	return HTE_CB_HANDLED;
>>>>>> +}
>>>>>> +
>>>>>>  static irqreturn_t edge_irq_thread(int irq, void *p)
>>>>>>  {
>>>>>>  	struct line *line = p;
>>>>>> @@ -553,6 +648,10 @@ static irqreturn_t edge_irq_thread(int irq, void *p)
>>>>>>  	struct gpio_v2_line_event le;
>>>>>>  	u64 eflags;
>>>>>>  
>>>>>> +	/* Let process_hw_ts_thread handle */
>>>>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
>>>>>> +		return IRQ_HANDLED;
>>>>>> +
>>>>> This adds pointless runtime overhead, and for everyone not just hte users.
>>>>> Don't stub out a handler in the handler - stub it out where it is
>>>>> registered by registering a stub handler.  Or don't request it at all.
>>>>>
>>>>> So why would gpiolib-cdev be requesting the irq, only to stub out
>>>>> the handlers?
>>>>> If that has a side-effect that hte requires then hte should be taking
>>>>> care of it - it is not gpiolib-cdev's problem.
>>>> - Why stop at moving irq and debounce related stuff to hte then?
>>>>
>>> How about you answer my question before asking your own?
>>> Here I am only questioning why gpiolib-cdev is requesting an interrupt
>>> for a hte line at all.  It has no reason to, so hte must have?
>> Commented below in "separation of concern" paragraph.
>>
>>> And where did I suggest moving the "debounce stuff" to hte?
>> Perhaps I misunderstood "That includes the sw debouncer - more on that
>>
>> below." comment.
>>
> The point I was trying to make there was that, for hte enabled lines, the
> sw debouncer should be driven by hte events, not irq events.
> That does not imply nor require moving the debouncer to hte.
Agreed...
>
>>> What I am suggesting is called separation of concerns.  In this context
>>> the intent is to look towards abstracting the edge event generation.
>>> Having hte and irq entangled for no apparent reason makes doing that more
>> But in this patch series (i.e. HTE provider), they are entangled as hte provider
>>
>> will only ts the GPIO line which is configured as input irq. That is why I have
>>
>> separated GPIO line config part and TS part. In other words, let cdev handle
>>
>> any line config that userspace has asked and let hte handle ts part if the
>>
>> line config is supported (If line config is not suitable, it will not enable hte, see
>>
>> gpio-tegra186.c function tegra186_gpio_req_hw_ts() in this patch series where
>>
>> check happens for the tegra hte provider).
>>
> Can you explain "hte provider will only ts the GPIO line which is
> configured as input irq"?

This specific HTE provider hardware is designed to detect edges to ts only on

input lines and which also has IRQ enabled. So for an example, if

userspace requests both OUTPUT and hardware clock config on a line

HTE provider will not generate any TS event on them.

>   That sounds like a hte concern to me, or even
> worse a particular hte provider problem.
>
> The user requests a line with edge event detection enabled and with hte
> timestamps.
> Nothing in that requires irq from gpiolib-cdev.
> gpiolib-cdev should use hte as the event source where it would usually
> use irq. If hte requires irq to allow it to generate those events then
> hte should be responsible for requesting the irq.
> In a hypothetical extreme case where gpiolib-cdev only supported hte
> lines there should be no reference to irq in gpiolib-cdev.
> Yet you continue to insist that gpiolib-cdev should request the irq for
> hte.
> What am I missing here?

I have to be clear, I am not insisting anything here. I am only trying

to understand what else could fall into HTE's plate so that I can get it right.

I am agreeing everything in above comment. I will revisit this logic in next

revision 4.


>
>>> difficult than it needs to be, whereas keeping them separate greatly
>>> simplifies identification of common code suitable for refactoring
>>> subsequently.
>>>
>>> Not sure what to call what you are suggesting.
>>>
>>>> I mean if there is hte provider which can TS GPIO output/input
>>>>
>>>> does it mean hte is responsible for parsing the GPIO line configs, setting them up
>>>>
>>>> (i.e. input or output) as well? Are we not duplicating logic instead of
>>>>
>>>> leveraging gpio-cdev? Does it make sense for the HTE subsystem which not
>>>>
>>>> only TS the GPIOs but other SoC lines?
>>>>
>>>> - What happens to in kernel GPIO HTE client (for example, hte-tegra194-gpio-test.c)?
>>>>
>>>> some clients do more in their IRQ handler than what edge_irq_handler does in which
>>>>
>>>> case it would make sense to have them request irq in their code than through HTE.
>>>>
>>>>> And speaking as to how the whole hte/gpiolib-cdev interface should work,
>>>>> hte should be an edge event generator alternative to irq.  So lines with
>>>>> hte enabled should work without any irq calls from gpiolib-cdev.
>>>>> That includes the sw debouncer - more on that below.
>>>>>
>>>>>>  	/* Do not leak kernel stack to userspace */
>>>>>>  	memset(&le, 0, sizeof(le));
>>>>>>  
>>>>>> @@ -604,6 +703,10 @@ static irqreturn_t edge_irq_handler(int irq, void *p)
>>>>>>  	struct line *line = p;
>>>>>>  	struct linereq *lr = line->req;
>>>>>>  
>>>>>> +	/* Let HTE supplied callbacks handle */
>>>>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
>>>>>> +		return IRQ_HANDLED;
>>>>>> +
>>>>>>  	/*
>>>>>>  	 * Just store the timestamp in hardirq context so we get it as
>>>>>>  	 * close in time as possible to the actual event.
>>>>>> @@ -682,14 +785,6 @@ static void debounce_work_func(struct work_struct *work)
>>>>>>  	/* Do not leak kernel stack to userspace */
>>>>>>  	memset(&le, 0, sizeof(le));
>>>>>>  
>>>>>> -	lr = line->req;
>>>>>> -	le.timestamp_ns = line_event_timestamp(line);
>>>>>> -	le.offset = gpio_chip_hwgpio(line->desc);
>>>>>> -	line->line_seqno++;
>>>>>> -	le.line_seqno = line->line_seqno;
>>>>>> -	le.seqno = (lr->num_lines == 1) ?
>>>>>> -		le.line_seqno : atomic_inc_return(&lr->seqno);
>>>>>> -
>>>>>>  	if (level)
>>>>>>  		/* Emit low-to-high event */
>>>>>>  		le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>>>> @@ -697,6 +792,23 @@ static void debounce_work_func(struct work_struct *work)
>>>>>>  		/* Emit high-to-low event */
>>>>>>  		le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>>>>  
>>>>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
>>>>>> +		le.timestamp_ns = line->timestamp_ns;
>>>>>> +		if (line->dir < HTE_DIR_NOSUPP)
>>>>>> +			le.id = (line->dir == HTE_RISING_EDGE_TS) ?
>>>>>> +				 GPIO_V2_LINE_EVENT_RISING_EDGE :
>>>>>> +				 GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>>>> +	} else {
>>>>>> +		le.timestamp_ns = line_event_timestamp(line);
>>>>>> +	}
>>>>>> +
>>>>> Move the FLAG_EVENT_CLOCK_HARDWARE check into line_event_timestamp().
>>>>>
>>>>> And the id fudging is necessary because the level returned by
>>>>> gpiod_get_raw_value_cansleep() can disagree with the level from hte?
>>>>> So you are still trying to synchronise events from two streams.
>>>>> And that is still broken.
>>>>> If a hte event occurs between the level being sampled by
>>>>> gpiod_get_raw_value_cansleep() and the line->dir being read then the line
>>>>> will have toggled and you will be reporting the opposite state than the
>>>>> one the debouncer determined was stable.  And maybe the wrong timestamp as
>>>>> well.
>>>>>
>>>>> For lines where hte is enabled, the hte should be the source of level for
>>>>> the debouncer, not the raw value.  And the mod_delayed_work() that
>>>>> drives the debouncer should be called by a hte handler, not an irq handler.
>>>>>
>>>>> There is also a race on reading the hte timestamp (line->timestamp_ns) and
>>>>> the hte level (line->dir), such that you can get the level from one event
>>>>> the timestamp from another.
>>>>>
>>>>>> +	lr = line->req;
>>>>>> +	le.offset = gpio_chip_hwgpio(line->desc);
>>>>>> +	line->line_seqno++;
>>>>>> +	le.line_seqno = line->line_seqno;
>>>>>> +	le.seqno = (lr->num_lines == 1) ?
>>>>>> +		le.line_seqno : atomic_inc_return(&lr->seqno);
>>>>>> +
>>>>> What is the purpose of moving this block of code moved from before the
>>>>> if (level)?
>>>>>
>>>>>
>>>>>>  	linereq_put_event(lr, &le);
>>>>>>  }
>>>>>>  
>>>>>> @@ -891,7 +1003,6 @@ static int gpio_v2_line_flags_validate(u64 flags)
>>>>>>  	/* Return an error if an unknown flag is set */
>>>>>>  	if (flags & ~GPIO_V2_LINE_VALID_FLAGS)
>>>>>>  		return -EINVAL;
>>>>>> -
>>>>> Gratuitous whitespace change.
>>>>>
>>>>>>  	/*
>>>>>>  	 * Do not allow both INPUT and OUTPUT flags to be set as they are
>>>>>>  	 * contradictory.
>>>>>> @@ -900,6 +1011,11 @@ static int gpio_v2_line_flags_validate(u64 flags)
>>>>>>  	    (flags & GPIO_V2_LINE_FLAG_OUTPUT))
>>>>>>  		return -EINVAL;
>>>>>>  
>>>>>> +	/* Only allow one event clock source */
>>>>>> +	if ((flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME) &&
>>>>>> +	    (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE))
>>>>>> +		return -EINVAL;
>>>>>> +
>>>>>>  	/* Edge detection requires explicit input. */
>>>>>>  	if ((flags & GPIO_V2_LINE_EDGE_FLAGS) &&
>>>>>>  	    !(flags & GPIO_V2_LINE_FLAG_INPUT))
>>>>>> @@ -992,6 +1108,8 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 flags,
>>>>>>  
>>>>>>  	assign_bit(FLAG_EVENT_CLOCK_REALTIME, flagsp,
>>>>>>  		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME);
>>>>>> +	assign_bit(FLAG_EVENT_CLOCK_HARDWARE, flagsp,
>>>>>> +		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE);
>>>>>>  }
>>>>>>  
>>>>>>  static long linereq_get_values(struct linereq *lr, void __user *ip)
>>>>>> @@ -1154,6 +1272,21 @@ static long linereq_set_config_unlocked(struct linereq *lr,
>>>>>>  				return ret;
>>>>>>  		}
>>>>>>  
>>>>>> +		/* Check if new config sets hardware assisted clock */
>>>>>> +		if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
>>>>>> +			ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
>>>>>> +							process_hw_ts_thread,
>>>>>> +							&lr->lines[i]);
>>>>>> +			if (ret)
>>>>>> +				return ret;
>>>>> Note that the line config is the complete line config, not a delta.
>>>>>
>>>>> What happens when a line that already has hte enabled is reconfigured
>>>>> and still has hte enabled?  i.e. what happens when
>>>>> gpiod_req_hw_timestamp_ns() is called for the second time?
>>>> HTE will return without doing anything with error code.
>>>>
>>> But this is not an error case, it is a normal reconfigure of an
>>> attribute other than the hte flag.
>> I assumed when this function is called it will "reset" and not  update the configs.
>>
>> If this assumption is wrong, I will correct the logic here.
>>
> The set_config does whatever is required to change the line request
> config from the old to the new.  Both old and new are complete snapshots
> of the line request config.
>
> The change should be seamless except for the attributes being changed.
> We certainly do not reset the config and start from scratch - that would
> be little better than the user releasing and re-requesting the lines.
>
> For many attributes we can just apply the new config.  But some cases,
> such as a change to active low polarity, are a lot more involved.
> And if hte considers re-requesting the line to be an error then you
> should only make the hte request when the hte flag changes to set.
Thanks for the explanation, will correct it in next version 4.
>
> Cheers,
> Kent.
>
>>> And that will now return an error to userspace?
>>>
>>> Cheers,
>>> Kent.
>>>
>>>>> You provide a comment for the release case below, what of the request
>>>>> case?
>>>>>
>>>>> If you need to check for change then compare the old and new flags, as
>>>>> the polarity_change check does (not visible in the diff here).
>>>>>
>>>>>> +		} else {
>>>>>> +			/*
>>>>>> +			 * HTE subsys will do nothing if there is nothing to
>>>>>> +			 * release.
>>>>>> +			 */
>>>>>> +			gpiod_rel_hw_timestamp_ns(desc);
>>>>>> +		}
>>>>>> +
>>>>> Comment will fit on one line.
>>>>>
>>>>> And it would be better to document that the function is idempotent in the
>>>>> function documentation, not everywhere it is used.
>>>>>
>>>>>>  		blocking_notifier_call_chain(&desc->gdev->notifier,
>>>>>>  					     GPIO_V2_LINE_CHANGED_CONFIG,
>>>>>>  					     desc);
>>>>>> @@ -1409,6 +1542,14 @@ static int linereq_create(struct gpio_device *gdev, void __user *ip)
>>>>>>  					flags & GPIO_V2_LINE_EDGE_FLAGS);
>>>>>>  			if (ret)
>>>>>>  				goto out_free_linereq;
>>>>>> +
>>>>>> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
>>>>>> +				ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
>>>>>> +							process_hw_ts_thread,
>>>>>> +							&lr->lines[i]);
>>>>>> +				if (ret)
>>>>>> +					goto out_free_linereq;
>>>>>> +			}
>>>>>>  		}
>>>>>>  
>>>>>>  		blocking_notifier_call_chain(&desc->gdev->notifier,
>>>>>> @@ -1959,6 +2100,8 @@ static void gpio_desc_to_lineinfo(struct gpio_desc *desc,
>>>>>>  
>>>>>>  	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &desc->flags))
>>>>>>  		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
>>>>>> +	else if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &desc->flags))
>>>>>> +		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
>>>>>>  
>>>>>>  	debounce_period_us = READ_ONCE(desc->debounce_period_us);
>>>>>>  	if (debounce_period_us) {
>>>>>> diff --git a/include/uapi/linux/gpio.h b/include/uapi/linux/gpio.h
>>>>>> index eaaea3d8e6b4..d360545b4c21 100644
>>>>>> --- a/include/uapi/linux/gpio.h
>>>>>> +++ b/include/uapi/linux/gpio.h
>>>>>> @@ -80,6 +80,7 @@ enum gpio_v2_line_flag {
>>>>>>  	GPIO_V2_LINE_FLAG_BIAS_PULL_DOWN	= _BITULL(9),
>>>>>>  	GPIO_V2_LINE_FLAG_BIAS_DISABLED		= _BITULL(10),
>>>>>>  	GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME	= _BITULL(11),
>>>>>> +	GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE	= _BITULL(12),
>>>>>>  };
>>>>>>  
>>>>> I'm now thinking this name, "HARDWARE" is too vague, in case other
>>>>> timestamp source alternatives join the fray, and so should be "HTE".
>>>>>
>>>>> Cheers,
>>>>> Kent.

Re: [RFC v3 02/12] drivers: Add hardware timestamp engine (HTE)

[Dipen Patel]

Hi,

On 11/25/21 5:30 PM, Kent Gibson wrote:
> On Tue, Nov 23, 2021 at 11:30:29AM -0800, Dipen Patel wrote:
>> Some devices can timestamp system lines/signals/Buses in real-time
>> using the hardware counter or other hardware means which can give
>> finer granularity and help avoid jitter introduced by software
>> timestamping. To utilize such functionality, this patchset creates
>> HTE subsystem where devices can register themselves as providers so
>> that the consumers devices can request specific line from the
>> providers.
>>
>> It provides below APIs for the provider:
>> - devm_hte_register_chip -- To register the HTE provider.
>> - hte_push_ts_ns() -- To push timestamp data into HTE subsystem.
>>
>> It provides below APIs for the consumer:
>> - devm_of_hte_request_ts() -- Request timestamp functionality.
>> - hte_req_ts_by_hte_name() -- To request timestamp functionality by
>> using HTE provider dt node.
>> - hte_enable_ts() -- To disable timestamp functionality.
>> - hte_disable_ts() -- To enable timestamp functionality.
>> - hte_release_ts() -- To release timestamp functionality and its
>> associated resources.
>> - hte_get_clk_src_info() -- To query clock source information from
>> the provider
>>
>> The detail about parameters and API usage are described in each
>> functions definitions in drivers/hte/hte.c file.
>>
>> The patch adds compilation support in Makefile and menu options in
>> Kconfig.
>>
>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>> Reported-by: kernel test robot <lkp@intel.com>
>> ---
>> Changes in v2:
>> - Removed buffer abstraction layer as well related APIs, HTE now will not store
>> any data, instead will pass to consumer as soon as it is available.
>> - Removed unnecessary dynamical allocations
>> - Removed timestamp retrieve API
>> - Removed release, unregister related APIs as their counterpart are resource
>> managed.
>> - Added kernel thread implementation if consumer indicates threaded callback
>> during request API time.
>> - Changed hte_req_ts_by_dt_node API to remove device node exposure from the
>> interface, instead consumer will their device node with property name that
>> indicates the provider it wants to use.
>>
>> Changes in v3:
>> - Addressed grammatical/spelling errors.
>>
>>  drivers/Kconfig      |   2 +
>>  drivers/Makefile     |   1 +
>>  drivers/hte/Kconfig  |  22 ++
>>  drivers/hte/Makefile |   2 +
>>  drivers/hte/hte.c    | 907 +++++++++++++++++++++++++++++++++++++++++++
>>  include/linux/hte.h  | 248 ++++++++++++
>>  6 files changed, 1182 insertions(+)
>>  create mode 100644 drivers/hte/Kconfig
>>  create mode 100644 drivers/hte/Makefile
>>  create mode 100644 drivers/hte/hte.c
>>  create mode 100644 include/linux/hte.h
>>
>> diff --git a/drivers/Kconfig b/drivers/Kconfig
>> index 0d399ddaa185..b3c8332bf5c7 100644
>> --- a/drivers/Kconfig
>> +++ b/drivers/Kconfig
>> @@ -236,4 +236,6 @@ source "drivers/interconnect/Kconfig"
>>  source "drivers/counter/Kconfig"
>>  
>>  source "drivers/most/Kconfig"
>> +
>> +source "drivers/hte/Kconfig"
>>  endmenu
>> diff --git a/drivers/Makefile b/drivers/Makefile
>> index be5d40ae1488..087c08e846e6 100644
>> --- a/drivers/Makefile
>> +++ b/drivers/Makefile
>> @@ -188,3 +188,4 @@ obj-$(CONFIG_GNSS)		+= gnss/
>>  obj-$(CONFIG_INTERCONNECT)	+= interconnect/
>>  obj-$(CONFIG_COUNTER)		+= counter/
>>  obj-$(CONFIG_MOST)		+= most/
>> +obj-$(CONFIG_HTE)		+= hte/
>> diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
>> new file mode 100644
>> index 000000000000..1fcfe17cf28a
>> --- /dev/null
>> +++ b/drivers/hte/Kconfig
>> @@ -0,0 +1,22 @@
>> +# SPDX-License-Identifier: GPL-2.0-only
>> +menuconfig HTE
>> +	bool "Hardware Timestamping Engine (HTE) Support"
>> +	help
>> +	  Hardware Timestamping Engine (HTE) Support.
>> +
>> +	  Some devices provide a hardware timestamping engine which can
>> +	  timestamp certain device lines/signals in realtime. This provides a
>> +	  hardware-assisted timestamp to generic signals like GPIOs or IRQs
>> +	  lines. It comes with a benefit for applications like autonomous
>> +	  machines needing accurate timestamping event with less jitter.
>> +
>> +	  This framework provides a generic interface to such HTE devices
>> +	  within the Linux kernel. It provides an API to register and
>> +	  unregister a HTE provider chip, configurable software buffer to
>> +	  store the timestamps, push the timestamp from the HTE providers and
>> +	  retrieve timestamps for the consumers. It also provides means for the
>> +	  consumers to request signals it wishes to hardware timestamp and
>> +	  release them if not required.
>> +
>> +	  If unsure, say no.
>> +
>> diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
>> new file mode 100644
>> index 000000000000..fc03bdf44427
>> --- /dev/null
>> +++ b/drivers/hte/Makefile
>> @@ -0,0 +1,2 @@
>> +obj-$(CONFIG_HTE)		+= hte.o
>> +
>> diff --git a/drivers/hte/hte.c b/drivers/hte/hte.c
>> new file mode 100644
>> index 000000000000..ca91d48f48b9
>> --- /dev/null
>> +++ b/drivers/hte/hte.c
>> @@ -0,0 +1,907 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Copyright (c) 2021 NVIDIA Corporation
>> + *
>> + * Author: Dipen Patel <dipenp@nvidia.com>
>> + */
>> +
>> +#include <linux/kernel.h>
>> +#include <linux/module.h>
>> +#include <linux/err.h>
>> +#include <linux/slab.h>
>> +#include <linux/of.h>
>> +#include <linux/of_device.h>
>> +#include <linux/mutex.h>
>> +#include <linux/sched.h>
>> +#include <linux/uaccess.h>
>> +#include <linux/hte.h>
>> +#include <linux/delay.h>
>> +#include <linux/debugfs.h>
>> +#include <linux/kthread.h>
>> +
>> +#define HTE_TS_NAME_LEN		10
>> +
>> +/* Global list of the HTE devices */
>> +static DEFINE_SPINLOCK(hte_lock);
>> +static LIST_HEAD(hte_devices);
>> +
>> +enum {
>> +	HTE_TS_REGISTERED,
>> +	HTE_TS_DISABLE,
>> +};
>> +
>> +enum {
>> +	HTE_CB_RUN_THREAD,
>> +	HTE_CB_NUM,
>> +};
>> +
>> +/**
>> + * struct hte_ts_info - Information related to requested timestamp.
>> + *
>> + * @xlated_id: Timestamp ID as understood between HTE subsys and HTE provider,
>> + * See xlate callback API.
>> + * @flags: Flags holding state informations.
>> + * @hte_cb_flags: Callback related flags.
>> + * @seq: Timestamp sequence counter.
>> + * @hte_name: Indicates if HTE core has set name for this timestamp entity.
>> + * @cb: Callback function provided by clients.
>> + * @tcb: Threaded callback function provided by clients.
>> + * @dropped_ts: Dropped timestamps.
>> + * @slock: Spin lock.
>> + * @thread: Thread task when tcb is provided.
>> + * @req_mlock: Lock during timestamp request/release APIs.
>> + * @ts_dbg_root: Root for the debug fs.
>> + * @gdev: HTE abstract device that this timestamp belongs to.
>> + * @cl_data: Client specific data.
>> + */
>> +struct hte_ts_info {
>> +	u32 xlated_id;
>> +	unsigned long flags;
>> +	unsigned long hte_cb_flags;
>> +	u64 seq;
>> +	bool hte_name;
>> +	hte_ts_cb_t cb;
>> +	hte_ts_threaded_cb_t tcb;
>> +	atomic_t dropped_ts;
>> +	spinlock_t slock;
>> +	struct task_struct *thread;
>> +	struct mutex req_mlock;
>> +	struct dentry *ts_dbg_root;
>> +	struct hte_device *gdev;
>> +	void *cl_data;
>> +};
>> +
>> +/**
>> + * struct hte_device - HTE abstract device
>> + * @nlines: Number of entities this device supports.
>> + * @ts_req: Total number of entities requested.
>> + * @sdev: Device used at various debug prints.
>> + * @dbg_root: Root directory for debug fs.
>> + * @list: List node to store hte_device for each provider.
>> + * @chip: HTE chip providing this HTE device.
>> + * @owner: helps prevent removal of modules when in use.
>> + * @ei: Timestamp information.
>> + */
>> +struct hte_device {
>> +	u32 nlines;
>> +	atomic_t ts_req;
>> +	struct device *sdev;
>> +	struct dentry *dbg_root;
>> +	struct list_head list;
>> +	struct hte_chip *chip;
>> +	struct module *owner;
>> +	struct hte_ts_info ei[];
>> +};
>> +
>> +#ifdef CONFIG_DEBUG_FS
>> +
>> +static struct dentry *hte_root;
>> +
>> +static int __init hte_subsys_dbgfs_init(void)
>> +{
>> +	/* creates /sys/kernel/debug/hte/ */
>> +	hte_root = debugfs_create_dir("hte", NULL);
>> +
>> +	return 0;
>> +}
>> +subsys_initcall(hte_subsys_dbgfs_init);
>> +
>> +static void hte_chip_dbgfs_init(struct hte_device *gdev)
>> +{
>> +	const struct hte_chip *chip = gdev->chip;
>> +	const char *name = chip->name ? chip->name : dev_name(chip->dev);
>> +
>> +	gdev->dbg_root = debugfs_create_dir(name, hte_root);
>> +
>> +	debugfs_create_atomic_t("ts_requested", 0444, gdev->dbg_root,
>> +				&gdev->ts_req);
>> +	debugfs_create_u32("total_ts", 0444, gdev->dbg_root,
>> +			   &gdev->nlines);
>> +}
>> +
>> +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
>> +{
>> +	if (!ei->gdev->dbg_root || !name)
>> +		return;
>> +
>> +	ei->ts_dbg_root = debugfs_create_dir(name, ei->gdev->dbg_root);
>> +
>> +	debugfs_create_atomic_t("dropped_timestamps", 0444, ei->ts_dbg_root,
>> +				&ei->dropped_ts);
>> +}
>> +
>> +#else
>> +
>> +static void hte_chip_dbgfs_init(struct hte_device *gdev)
>> +{
>> +}
>> +
>> +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
>> +{
>> +}
>> +
>> +#endif
>> +
>> +/**
>> + * hte_release_ts() - Consumer calls this API to release the entity, where
>> + * entity could be anything providers support, like lines, signals, buses,
>> + * etc...
>> + *
>> + * @desc: timestamp descriptor, this is the same as returned by the request API.
>> + *
>> + * Context: debugfs_remove_recursive() function call may use sleeping locks,
>> + *	    not suitable from atomic context.
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int hte_release_ts(struct hte_ts_desc *desc)
>> +{
>> +	u32 id;
>> +	int ret = 0;
>> +	unsigned long flag;
>> +	struct hte_device *gdev;
>> +	struct hte_ts_info *ei;
>> +
>> +	if (!desc)
>> +		return -EINVAL;
>> +
>> +	ei = desc->hte_data;
>> +
>> +	if (!ei || !ei->gdev)
>> +		return -EINVAL;
>> +
>> +	gdev = ei->gdev;
>> +	id = desc->con_id;
>> +
>> +	mutex_lock(&ei->req_mlock);
>> +
>> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
>> +		dev_info(gdev->sdev, "id:%d is not registered", id);
>> +		ret = -EUSERS;
>> +		goto unlock;
>> +	}
>> +
>> +	ret = gdev->chip->ops->release(gdev->chip, ei->xlated_id);
>> +	if (ret) {
>> +		dev_err(gdev->sdev, "id: %d free failed\n", id);
>> +		goto unlock;
>> +	}
>> +
>> +	if (ei->hte_name)
>> +		kfree(desc->name);
>> +
>> +	debugfs_remove_recursive(ei->ts_dbg_root);
>> +
>> +	spin_lock_irqsave(&ei->slock, flag);
>> +
>> +	atomic_dec(&gdev->ts_req);
>> +	atomic_set(&ei->dropped_ts, 0);
>> +
>> +	ei->seq = 0;
>> +	desc->hte_data = NULL;
>> +
>> +	clear_bit(HTE_TS_REGISTERED, &ei->flags);
>> +
>> +	spin_unlock_irqrestore(&ei->slock, flag);
>> +
>> +	if (ei->tcb) {
>> +		kthread_stop(ei->thread);
>> +		put_task_struct(ei->thread);
>> +	}
>> +
>> +	ei->cb = NULL;
>> +	ei->tcb = NULL;
>> +	ei->thread = NULL;
>> +	ei->cl_data = NULL;
>> +
>> +	module_put(gdev->owner);
>> +unlock:
>> +	mutex_unlock(&ei->req_mlock);
>> +	dev_dbg(gdev->sdev, "release id: %d\n", id);
>> +
>> +	return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(hte_release_ts);
>> +
>> +static int hte_ts_dis_en_common(struct hte_ts_desc *desc, bool en)
>> +{
>> +	u32 ts_id;
>> +	struct hte_device *gdev;
>> +	struct hte_ts_info *ei;
>> +	int ret;
>> +	unsigned long flag;
>> +
>> +	if (!desc)
>> +		return -EINVAL;
>> +
>> +	ei = desc->hte_data;
>> +
>> +	if (!ei || !ei->gdev)
>> +		return -EINVAL;
>> +
>> +	gdev = ei->gdev;
>> +	ts_id = desc->con_id;
>> +
>> +	mutex_lock(&ei->req_mlock);
>> +
>> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
>> +		dev_dbg(gdev->sdev, "id:%d is not registered", ts_id);
>> +		ret = -EUSERS;
>> +		goto out;
>> +	}
>> +
>> +	spin_lock_irqsave(&ei->slock, flag);
>> +
>> +	if (en) {
>> +		if (!test_bit(HTE_TS_DISABLE, &ei->flags)) {
>> +			ret = 0;
>> +			goto out_unlock;
>> +		}
>> +
>> +		spin_unlock_irqrestore(&ei->slock, flag);
>> +		ret = gdev->chip->ops->enable(gdev->chip, ei->xlated_id);
>> +		if (ret) {
>> +			dev_warn(gdev->sdev, "id: %d enable failed\n",
>> +				 ts_id);
>> +			goto out;
>> +		}
>> +
>> +		spin_lock_irqsave(&ei->slock, flag);
>> +		clear_bit(HTE_TS_DISABLE, &ei->flags);
>> +	} else {
>> +		if (test_bit(HTE_TS_DISABLE, &ei->flags)) {
>> +			ret = 0;
>> +			goto out_unlock;
>> +		}
>> +
>> +		spin_unlock_irqrestore(&ei->slock, flag);
>> +		ret = gdev->chip->ops->disable(gdev->chip, ei->xlated_id);
>> +		if (ret) {
>> +			dev_warn(gdev->sdev, "id: %d disable failed\n",
>> +				 ts_id);
>> +			goto out;
>> +		}
>> +
>> +		spin_lock_irqsave(&ei->slock, flag);
>> +		set_bit(HTE_TS_DISABLE, &ei->flags);
>> +	}
>> +
>> +out_unlock:
>> +	spin_unlock_irqrestore(&ei->slock, flag);
>> +out:
>> +	mutex_unlock(&ei->req_mlock);
>> +	return ret;
>> +}
>> +
>> +/**
>> + * hte_disable_ts() - Disable timestamp on given descriptor.
>> + *
>> + * The API does not release any resources associated with desc.
>> + *
>> + * @desc: ts descriptor, this is the same as returned by the request API.
>> + *
>> + * Context: Holds mutex lock, not suitable from atomic context.
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int hte_disable_ts(struct hte_ts_desc *desc)
>> +{
>> +	return hte_ts_dis_en_common(desc, false);
>> +}
>> +EXPORT_SYMBOL_GPL(hte_disable_ts);
>> +
>> +/**
>> + * hte_enable_ts() - Enable timestamp on given descriptor.
>> + *
>> + * @desc: ts descriptor, this is the same as returned by the request API.
>> + *
>> + * Context: Holds mutex lock, not suitable from atomic context.
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int hte_enable_ts(struct hte_ts_desc *desc)
>> +{
>> +	return hte_ts_dis_en_common(desc, true);
>> +}
>> +EXPORT_SYMBOL_GPL(hte_enable_ts);
>> +
>> +static int hte_simple_xlate(struct hte_chip *gc,
>> +			    const struct of_phandle_args *args,
>> +			    struct hte_ts_desc *desc,
>> +			    u32 *id)
>> +{
>> +	if (!id || !desc || !gc)
>> +		return -EINVAL;
>> +
>> +	/*
>> +	 * For the providers which do not have any internal mappings between
>> +	 * logically exposed ids and actual ids, will set both
>> +	 * the same.
>> +	 *
>> +	 * In case there is a internal mapping needed, providers will need to
>> +	 * provide its own xlate function where con_id will be sent as
>> +	 * args[0] and it will return xlated id. Later xlated id will be
>> +	 * used for any future exchanges between provider and subsystems.
>> +	 */
>> +
>> +	if (args) {
>> +		if (gc->of_hte_n_cells < 1)
>> +			return -EINVAL;
>> +
>> +		if (args->args_count != gc->of_hte_n_cells)
>> +			return -EINVAL;
>> +
>> +		*id = args->args[0];
>> +		desc->con_id = *id;
>> +	} else {
>> +		*id = desc->con_id;
>> +	}
>> +
>> +	if (desc->con_id > gc->nlines)
>> +		return -EINVAL;
>> +
>> +	desc->hte_data = NULL;
>> +
>> +	return 0;
>> +}
>> +
>> +static int _hte_wait_for_ts_data(struct hte_ts_info *ei)
>> +{
>> +	for (;;) {
>> +		set_current_state(TASK_INTERRUPTIBLE);
>> +
>> +		if (kthread_should_stop()) {
>> +			if (test_and_clear_bit(HTE_CB_RUN_THREAD,
>> +			    &ei->hte_cb_flags)) {
>> +				__set_current_state(TASK_RUNNING);
>> +				return 0;
>> +			}
>> +			__set_current_state(TASK_RUNNING);
>> +			return -1;
>> +		}
>> +
>> +		if (test_and_clear_bit(HTE_CB_RUN_THREAD,
>> +				       &ei->hte_cb_flags)) {
>> +			__set_current_state(TASK_RUNNING);
>> +			return 0;
>> +		}
>> +		schedule();
>> +	}
>> +}
>> +
>> +static int _hte_threadfn(void *data)
>> +{
>> +	struct hte_ts_info *ei = data;
>> +
>> +	while (!_hte_wait_for_ts_data(ei))
>> +		ei->tcb(ei->cl_data);
>> +
>> +	return 0;
>> +}
>> +
>> +static int _hte_setup_thread(struct hte_ts_info *ei, u32 id)
>> +{
>> +	struct task_struct *t;
>> +
>> +	t = kthread_create(_hte_threadfn, ei, "hte-%u", id);
>> +	if (IS_ERR(t))
>> +		return PTR_ERR(t);
>> +
>> +	ei->thread = get_task_struct(t);
>> +
>> +	return 0;
>> +}
>> +
>> +static int ___hte_req_ts(struct hte_device *gdev, struct hte_ts_desc *desc,
>> +			 u32 xlated_id, hte_ts_cb_t cb,
>> +			 hte_ts_threaded_cb_t tcb, void *data)
>> +{
>> +	struct hte_ts_info *ei;
>> +	int ret;
>> +	u32 con_id = desc->con_id;
>> +
>> +	if (!try_module_get(gdev->owner))
>> +		return -ENODEV;
>> +
>> +	ei = &gdev->ei[xlated_id];
>> +	ei->xlated_id = xlated_id;
>> +
>> +	/*
>> +	 * There is a chance that multiple consumers requesting same entity,
>> +	 * lock here.
>> +	 */
>> +	mutex_lock(&ei->req_mlock);
>> +
>> +	if (test_bit(HTE_TS_REGISTERED, &ei->flags)) {
>> +		dev_dbg(gdev->chip->dev, "id:%u is already registered",
>> +			xlated_id);
>> +		ret = -EUSERS;
>> +		goto unlock;
>> +	}
>> +
>> +	ei->cb = cb;
>> +	ei->tcb = tcb;
>> +	if (tcb) {
>> +		ret = _hte_setup_thread(ei, xlated_id);
>> +		if (ret < 0) {
>> +			dev_err(gdev->chip->dev, "setting thread failed\n");
>> +			goto unlock;
>> +		}
>> +	}
>> +
>> +	ret = gdev->chip->ops->request(gdev->chip, xlated_id);
>> +	if (ret < 0) {
>> +		dev_err(gdev->chip->dev, "ts request failed\n");
>> +		goto unlock;
>> +	}
>> +
>> +	desc->hte_data = ei;
>> +	ei->cl_data = data;
>> +
>> +	atomic_inc(&gdev->ts_req);
>> +
>> +	ei->hte_name = false;
>> +	if (!desc->name) {
>> +		desc->name = kzalloc(HTE_TS_NAME_LEN, GFP_KERNEL);
>> +		if (desc->name) {
>> +			scnprintf(desc->name, HTE_TS_NAME_LEN, "ts_%u",
>> +				  con_id);
>> +			ei->hte_name = true;
>> +		}
>> +	}
>> +
>> +	hte_ts_dbgfs_init(desc->name, ei);
>> +	set_bit(HTE_TS_REGISTERED, &ei->flags);
>> +
>> +	mutex_unlock(&ei->req_mlock);
>> +
>> +	dev_dbg(gdev->chip->dev, "id: %u, xlated id:%u", con_id, xlated_id);
>> +
>> +	return 0;
>> +
>> +unlock:
>> +	module_put(gdev->owner);
>> +	mutex_unlock(&ei->req_mlock);
>> +
>> +	return ret;
>> +}
>> +
>> +static struct hte_device *of_node_to_htedevice(struct device_node *np)
>> +{
>> +	struct hte_device *gdev;
>> +
>> +	spin_lock(&hte_lock);
>> +
>> +	list_for_each_entry(gdev, &hte_devices, list)
>> +		if (gdev->chip && gdev->chip->dev &&
>> +		    gdev->chip->dev->of_node == np) {
>> +			spin_unlock(&hte_lock);
>> +			return gdev;
>> +		}
>> +
>> +	spin_unlock(&hte_lock);
>> +
>> +	return ERR_PTR(-ENODEV);
>> +}
>> +
>> +static struct hte_device *of_hte_dev_get(struct device *dev,
>> +					 struct device_node *np,
>> +					 const char *label,
>> +					 struct of_phandle_args *args)
>> +{
>> +	struct hte_device *gdev = NULL;
>> +	int index = 0;
>> +	int err;
>> +
>> +	if (label) {
>> +		index = of_property_match_string(np,
>> +						 "hardware-timestamp-names",
>> +						 label);
>> +		if (index < 0)
>> +			return ERR_PTR(index);
>> +	}
>> +
>> +	err = of_parse_phandle_with_args(np, "hardware-timestamps",
>> +					 "#hardware-timestamp-cells", index,
>> +					 args);
>> +	if (err) {
>> +		pr_err("%s(): can't parse \"hardware-timestamps\" property\n",
>> +		       __func__);
>> +		return ERR_PTR(err);
>> +	}
>> +
>> +	gdev = of_node_to_htedevice(args->np);
>> +	if (IS_ERR(gdev)) {
>> +		pr_err("%s(): HTE chip not found\n", __func__);
>> +		of_node_put(args->np);
>> +		return gdev;
>> +	}
>> +
>> +	return gdev;
>> +}
>> +
>> +static int __hte_req_ts(struct device *dev, struct hte_ts_desc *desc,
>> +			hte_ts_cb_t cb, hte_ts_threaded_cb_t tcb, void *data)
>> +{
>> +	struct hte_device *gdev = NULL;
>> +	struct of_phandle_args args;
>> +	int ret;
>> +	u32 xlated_id;
>> +
>> +	gdev = of_hte_dev_get(dev, dev->of_node, desc->name, &args);
>> +	if (IS_ERR(gdev))
>> +		return PTR_ERR(gdev);
>> +
>> +	if (!gdev->chip) {
>> +		pr_debug("requested id does not have provider\n");
>> +		return -ENODEV;
>> +	}
>> +
>> +	ret = gdev->chip->xlate(gdev->chip, &args, desc, &xlated_id);
>> +	if (ret < 0)
>> +		goto put;
>> +
>> +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb, tcb, data);
>> +	if (ret < 0)
>> +		goto put;
>> +
>> +	return 0;
>> +
>> +put:
>> +	of_node_put(args.np);
>> +
>> +	return ret;
>> +}
>> +
>> +static void __devm_hte_release_ts(void *res)
>> +{
>> +	hte_release_ts(res);
>> +}
>> +
>> +/**
>> + * devm_of_hte_request_ts() - Resource managed API to request the HTE facility
>> + * on the specified entity, where entity is provider specific for example,
>> + * GPIO lines, signals, buses etc...
>> + *
>> + * The API allocates necessary resources and enables the timestamp. So calling
>> + * hte_enable_ts is not needed. The consumer does not need to call
>> + * hte_release_ts since it will be called upon consumer exit.
>> + *
>> + * @dev: HTE consumer/client device.
>> + * @desc: Pre-allocated timestamp descriptor. HTE core will fill out necessary
>> + * details. Optionally the consumer can set name field of desc, if not
>> + * specified HTE core will set it as ts_con_id. It will be the consumer's
>> + * job to free any allocation related to this structure as well name field
>> + * in case it has set that field.
>> + * @cb: Callback to push the timestamp data to consumer.
>> + * @tcb: Optional callback. If its provided, subsystem will create
>> + * thread. This will be called when cb returns HTE_RUN_THREADED_CB.
>> + * @data: Client data, will be sent back during cb and tcb callbacks.
>> + *
>> + * Context: Holds mutex lock.
>> + * Returns: Returns 0 on success or negative error code on failure.
>> + */
>> +int devm_of_hte_request_ts(struct device *dev, struct hte_ts_desc *desc,
>> +			   hte_ts_cb_t cb, hte_ts_threaded_cb_t tcb,
>> +			   void *data)
>> +{
>> +	int err;
>> +
>> +	if (!dev || !dev->of_node || !desc || !cb)
>> +		return -EINVAL;
>> +
>> +	err = __hte_req_ts(dev, desc, cb, tcb, data);
>> +	if (err)
>> +		return err;
>> +
>> +	err = devm_add_action_or_reset(dev, __devm_hte_release_ts, desc);
>> +	if (err)
>> +		return err;
>> +
>> +	return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(devm_of_hte_request_ts);
>> +
>> +/**
>> + * hte_req_ts_by_hte_name() - Request entity to timestamp realtime by passing
>> + * property name that contains HTE provider phandle, meaning of the entity
>> + * is HTE provider specific, for example lines, signals, GPIOs, buses etc...
>> + *
>> + * This API is designed to address below uses cases:
>> + *
>> + * 1) For the consumer device which acts as a central device for secondary
>> + * consumers. For example, GPIO controller driver acts as a primary consumer
>> + * on behalf of in kernel and userspace GPIO HTE consumers. The GPIO controller
>> + * driver specifies HTE provider that it supports/wants and it becomes opaque
>> + * for the secondary consumers requesting GPIO and hardware timestamp through
>> + * that GPIO controller.
>> + *
>> + * 2) For the providers which are dependent on other hardware modules. In that
>> + * case it forces consumers to go through other subsystem or driver making them
>> + * secondary consumers. Same example as above applies here as well.
>> + *
>> + * The API allocates necessary resources and enables the timestamp. So calling
>> + * hte_enable_ts is not needed.
>> + *
>> + * @dev: HTE consumer/client device.
>> + * @propname: Name of property holding a HTE provider phandle value
>> + * @desc: Pre-allocated timestamp descriptor with con_id set by the consumer.
>> + * HTE core will fill out the rest. Optionally the consumer can set name
>> + * field of desc, if not specified HTE core will set it as ts_con_id. It will
>> + * be the consumer's job to free any allocation related to this structure as
>> + * well name field in case it has set that field.
>> + * @cb: Callback to push the timestamp data to consumer.
>> + * @tcb: Optional callback. If its provided, subsystem will create
>> + * thread. This will be called when cb returns HTE_RUN_THREADED_CB.
>> + * @data: Client data, will be sent back during cb and tcb callbacks.
>> + *
>> + * Context: Holds mutex lock, can not be called from atomic context. The mutex
>> + * lock is used to serialize multiple consumers.
>> + * Returns: returns 0 on success or negative error code on failure.
>> + */
>> +int hte_req_ts_by_hte_name(struct device *dev, const char *propname,
>> +			   struct hte_ts_desc *desc, hte_ts_cb_t cb,
>> +			   hte_ts_threaded_cb_t tcb, void *data)
>> +{
>> +	struct hte_device *gdev;
>> +	struct device_node *np = NULL;
>> +	int ret;
>> +	u32 xlated_id;
>> +
>> +	if (!dev->of_node || !propname || !desc)
>> +		return -EINVAL;
>> +
>> +	np = of_parse_phandle(dev->of_node, propname, 0);
>> +	if (!np)
>> +		return -ENODEV;
>> +
>> +	of_node_put(np);
>> +
>> +	gdev = of_node_to_htedevice(np);
>> +	if (IS_ERR(gdev))
>> +		return -ENOTSUPP;
>> +
>> +	if (!gdev->chip || !gdev->chip->ops)
>> +		return -ENOTSUPP;
>> +
>> +	ret = gdev->chip->xlate(gdev->chip, NULL, desc, &xlated_id);
>> +	if (ret < 0) {
>> +		dev_err(gdev->chip->dev,
>> +			"failed to xlate id: %d\n", desc->con_id);
>> +		return ret;
>> +	}
>> +
>> +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb, tcb, data);
>> +	if (ret < 0) {
>> +		dev_err(gdev->chip->dev,
>> +			"failed to request id: %d\n", desc->con_id);
>> +		return ret;
>> +	}
>> +
>> +	return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(hte_req_ts_by_hte_name);
>> +
>> +/**
>> + * hte_get_clk_src_info() - Consumer calls this API to query clock source
>> + * information of the desc.
>> + *
>                                ^^^^^^^^^^
> Get the clock source information for a ts descriptor.
>
>> + * @desc: ts descriptor, same as returned from request API.
>> + * @ci: The API fills this structure with the clock information data.
>> + *
>> + * Context: Any context.
>> + * Returns: 0 on success else negative error code on failure.
>> + */
>> +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
>> +			 struct hte_clk_info *ci)
>> +{
>> +	struct hte_chip *chip;
>> +	struct hte_ts_info *ei;
>> +
>> +	if (!desc || !desc->hte_data || !ci) {
>> +		pr_debug("%s:%d\n", __func__, __LINE__);
>> +		return -EINVAL;
>> +	}
>> +
>> +	ei = desc->hte_data;
>> +	if (!ei || !ei->gdev || !ei->gdev->chip)
>> +		return -EINVAL;
>> +
>> +	chip = ei->gdev->chip;
>> +	if (!chip->ops->get_clk_src_info)
>> +		return -ENOTSUPP;
>> +
>> +	return chip->ops->get_clk_src_info(chip, ci);
>> +}
>> +EXPORT_SYMBOL_GPL(hte_get_clk_src_info);
>> +
>> +/**
>> + * hte_push_ts_ns() - Used by the provider to push timestamp in nano
>> + * seconds i.e data->tsc will be in ns.
>> + *
>                          ^^^^^^^^
> Push timestamp in nanoseconds.
>
>> + * @chip: The HTE chip, used during the registration.
>> + * @xlated_id: entity id understood by both subsystem and provider, usually this
>> + * is obtained from xlate callback during request API.
>> + * @data: timestamp data.
>> + *
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int hte_push_ts_ns(const struct hte_chip *chip, u32 xlated_id,
>> +		   struct hte_ts_data *data)
>> +{
>> +	hte_return_t ret;
>> +	int st = 0;
>> +	struct hte_ts_info *ei;
>> +	unsigned long flag;
>> +
>> +	if (!chip || !data || !chip->gdev)
>> +		return -EINVAL;
>> +
>> +	if (xlated_id > chip->nlines)
>> +		return -EINVAL;
>> +
>> +	ei = &chip->gdev->ei[xlated_id];
>> +
>> +	spin_lock_irqsave(&ei->slock, flag);
>> +
>> +	/* timestamp sequence counter */
>> +	data->seq = ei->seq++;
>> +
>> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags) ||
>> +	    test_bit(HTE_TS_DISABLE, &ei->flags)) {
>> +		dev_dbg(chip->dev, "Unknown timestamp push\n");
>> +		st = -EINVAL;
>> +		goto unlock;
>> +	}
>> +
>> +	ret = ei->cb(data, ei->cl_data);
>> +	if (ret == HTE_RUN_THREADED_CB && ei->thread) {
>> +		if (test_and_set_bit(HTE_CB_RUN_THREAD, &ei->hte_cb_flags))
>> +			goto unlock;
>> +		else
>> +			wake_up_process(ei->thread);
>> +	} else if (ret == HTE_CB_TS_DROPPED) {
>> +		atomic_inc(&ei->dropped_ts);
>> +	} else if (ret == HTE_CB_ERROR) {
>> +		dev_dbg(chip->dev, "cb error\n");
>> +	}
>> +
>> +unlock:
>> +	spin_unlock_irqrestore(&ei->slock, flag);
>> +
>> +	return st;
>> +}
>> +EXPORT_SYMBOL_GPL(hte_push_ts_ns);
>> +
>> +static int hte_register_chip(struct hte_chip *chip)
>> +{
>> +	struct hte_device *gdev;
>> +	u32 i;
>> +
>> +	if (!chip || !chip->dev || !chip->dev->of_node)
>> +		return -EINVAL;
>> +
>> +	if (!chip->ops || !chip->ops->request || !chip->ops->release) {
>> +		dev_err(chip->dev, "Driver needs to provide ops\n");
>> +		return -EINVAL;
>> +	}
>> +
>> +	gdev = kzalloc(struct_size(gdev, ei, chip->nlines), GFP_KERNEL);
>> +	if (!gdev)
>> +		return -ENOMEM;
>> +
>> +	gdev->chip = chip;
>> +	chip->gdev = gdev;
>> +	gdev->nlines = chip->nlines;
>> +	gdev->sdev = chip->dev;
>> +
>> +	for (i = 0; i < chip->nlines; i++) {
>> +		gdev->ei[i].gdev = gdev;
>> +		mutex_init(&gdev->ei[i].req_mlock);
>> +		spin_lock_init(&gdev->ei[i].slock);
>> +	}
>> +
>> +	if (chip->dev->driver)
>> +		gdev->owner = chip->dev->driver->owner;
>> +	else
>> +		gdev->owner = THIS_MODULE;
>> +
>> +	if (!chip->xlate) {
>> +		chip->xlate = hte_simple_xlate;
>> +		/* Just a id number to monitor */
>> +		chip->of_hte_n_cells = 1;
>> +	}
>> +
>> +	of_node_get(chip->dev->of_node);
>> +
>> +	INIT_LIST_HEAD(&gdev->list);
>> +
>> +	spin_lock(&hte_lock);
>> +	list_add_tail(&gdev->list, &hte_devices);
>> +	spin_unlock(&hte_lock);
>> +
>> +	hte_chip_dbgfs_init(gdev);
>> +
>> +	dev_dbg(chip->dev, "Added hte chip\n");
>> +
>> +	return 0;
>> +}
>> +
>> +/**
>> + * hte_unregister_chip() - Used by the provider to remove a HTE chip.
>> + * @chip: the HTE chip to remove.
>> + *
>> + * Context: Can not be called from atomic context.
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +static int hte_unregister_chip(struct hte_chip *chip)
>> +{
>> +	struct hte_device *gdev;
>> +
>> +	if (!chip)
>> +		return -EINVAL;
>> +
>> +	gdev = chip->gdev;
>> +
>> +	spin_lock(&hte_lock);
>> +	list_del(&gdev->list);
>> +	spin_unlock(&hte_lock);
>> +
>> +	gdev->chip = NULL;
>> +
>> +	of_node_put(chip->dev->of_node);
>> +	debugfs_remove_recursive(gdev->dbg_root);
>> +	kfree(gdev);
>> +
>> +	dev_dbg(chip->dev, "Removed hte chip\n");
>> +
>> +	return 0;
>> +}
>> +
>> +static void _hte_devm_unregister_chip(void *chip)
>> +{
>> +	hte_unregister_chip(chip);
>> +}
>> +
>> +/**
>> + * devm_hte_register_chip() - Used by provider to register a HTE chip.
>> + * @chip: the HTE chip to add to subsystem.
>> + *
>> + * The API is resource managed and  _hte_devm_unregister_chip will be called
>> + * automatically when the provider exits.
>> + *
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int devm_hte_register_chip(struct hte_chip *chip)
>> +{
>> +	int err;
>> +
>> +	err = hte_register_chip(chip);
>> +	if (err)
>> +		return err;
>> +
>> +	err = devm_add_action_or_reset(chip->dev, _hte_devm_unregister_chip,
>> +				       chip);
>> +	if (err)
>> +		return err;
>> +
>> +	return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(devm_hte_register_chip);
>> diff --git a/include/linux/hte.h b/include/linux/hte.h
>> new file mode 100644
>> index 000000000000..f4dd5415a493
>> --- /dev/null
>> +++ b/include/linux/hte.h
>> @@ -0,0 +1,248 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Copyright (c) 2021 NVIDIA Corporation
>> + *
>> + * Author: Dipen Patel <dipenp@nvidia.com>
>> + */
>> +
>> +#ifndef __LINUX_HTE_H
>> +#define __LINUX_HTE_H
>> +
>> +#include <linux/errno.h>
>> +
>> +struct hte_chip;
>> +struct hte_device;
>> +struct of_phandle_args;
>> +struct device_node;
>> +
>> +/**
>> + * enum hte_dir - HTE edge timestamp direction.
>> + *
>> + * @HTE_RISING_EDGE_TS: Timestamps is for rising edge.
>> + * @HTE_FALLING_EDGE_TS: Timestamps is for falling edge.
>> + * @HTE_DIR_NOSUPP: Direction is not supported.
>> + */
>> +enum hte_dir {
>> +	HTE_RISING_EDGE_TS,
>> +	HTE_FALLING_EDGE_TS,
>> +	HTE_DIR_NOSUPP,
>> +};
>> +
> Your tegra194 driver doesn't support direction - it always reports
> HTE_DIR_NOSUPP.  Do you have an example of hardware that does support
> direction?
No
>
> This might be better as a raw line level rather than an edge direction.
> See my related comments on patch 09.

We are discussing this on 09 patch, will follow up there should I need

clarifications if that is ok.

>
> And the third option should be "unknown", rather than "not supported" as
> this is a reading being returned, not a feature being requested.
> It may be unknown because the feature is not supported, but that is a
> separate issue.
>
>> +/**
>> + * struct hte_ts_data - HTE timestamp data.
>> + * The provider uses and fills timestamp related details during push_timestamp
>> + * API call. The consumer uses during retrieve_timestamp API call.
>> + *
> What push_timestamp() and retrieve_timestamp() calls?
>
> General comment:
> Documenting how a struct is used in an API, rather than what it is for or
> what it represents, is a maintenance nightmare.  If it is subsequently
> used elsewhere or names are changed then the user is now obliged to update
> your documentation.
> And they will forget to, or overlook it, as you yourself have done here.
>
> Descriptions of what the API calls do should be in the documentation of
> the API calls themselves, not the structs they use.
I agree, will correct in v4.
>
>> + * @tsc: Timestamp value.
>> + * @seq: Sequence counter of the timestamps.
>> + * @dir: Direction of the event at the time of timestamp.
>> + */
>> +struct hte_ts_data {
>> +	u64 tsc;
>> +	u64 seq;
>> +	enum hte_dir dir;
>> +};
>> +
>> +/**
>> + * struct hte_clk_info - Clock source info that HTE provider uses to timestamp
>> + * The provider uses hardware clock as a source to timestamp real time. This
>> + * structure presents the clock information to consumers during
>> + * hte_get_clk_src_info call.
>> + *
>> + * @hz: Supported clock rate in HZ, for example 1KHz clock = 1000.
>> + * @type: Supported clock type. CLOCK_* types.
>> + */
>> +struct hte_clk_info {
>> +	u64 hz;
>> +	clockid_t type;
>> +};
>> +
>> +/**
>> + * enum hte_return- HTE subsystem return values used during callback.
>> + *
>> + * @HTE_CB_HANDLED: The consumer handled the data successfully.
>> + * @HTE_RUN_THREADED_CB: The consumer needs further processing, in that case HTE
>> + * subsystem will invoke kernel thread and call secondary callback provided by
>> + * the consumer during devm_of_hte_request_ts and hte_req_ts_by_dt_node call.
>> + * @HTE_CB_TS_DROPPED: The client returns when it can not store ts data.
>> + * @HTE_CB_ERROR: The client returns error if anything goes wrong.
>> + */
>> +enum hte_return {
>> +	HTE_CB_HANDLED,
>> +	HTE_RUN_THREADED_CB,
>> +	HTE_CB_TS_DROPPED,
>> +	HTE_CB_ERROR,
>> +};
>> +typedef enum hte_return hte_return_t;
>> +
> Wrt HTE_CB_TS_DROPPED, why is the client dropping data any of hte's
> business?  It is also confusing in that I would expect the dropped_ts
> gauge, that you increment when this code is returned, to indicate the
> events dropped by the hardware, not the client.  But then you have no
> indication of events dropped by hardware at all, though you could
> determine that from gaps in the sequence numbers.
> Anyway, the client can do the math in both cases if they care to, so not
> sure what its purpose is here.

It is used for statistical purpose and hte being subsytem it can provide

standard interface in debugfs (so that clients do not have to) to anyone interested.

The dropped_ts could represent total dropped ts by both hardware and

client. I can add debugfs interface to break it down further if it helps in statistics.

>
> Similarly HTE_CB_ERROR, what can hte do here other than log it - and the
> client can do that themselves if they have the need.
Ok, I will remove it in v4.
>
>> +/**
>> + * typedef hte_ts_cb_t - Callback provided during devm_of_hte_request_ts and
>> + * hte_req_ts_by_dt_node APIs call.
>> + *
>> + * The callback is used to push timestamp data to client.
>> + * @ts: HW timestamp data.
>> + * @data: Client supplied data.
>> + */
>> +typedef hte_return_t (*hte_ts_cb_t)(struct hte_ts_data *ts, void *data);
>> +
>> +/**
>> + * typedef hte_ts_threaded_cb_t - Threaded callback provided during
>> + * devm_of_hte_request_ts and hte_req_ts_by_dt_node APIs call.
>> + *
>> + * @data: Client supplied data.
>> + *
>> + * It will be called when client return HTE_RUN_THREADED_CB from hte_ts_cb_t.
>> + * The callback will be called from thread context.
>> + *
>> + */
>> +typedef hte_return_t (*hte_ts_threaded_cb_t)(void *data);
>> +
>> +/**
>> + * struct hte_ts_desc - HTE timestamp descriptor, this structure will be
>> + * communication token between consumers to subsystem and subsystem to
>> + * providers.
>> + *
>> + * @con_id: This is the same id sent in request APIs.
>> + * @name: Descriptive name of the entity that is being monitored for the
>> + * realtime timestamping. The consumer can set any name it likes. If null
>> + * HTE core will construct name as ts_con_id. It will be the consumer's
>> + * job to free any allocation if name is set by the consumer.
>> + * @hte_data: Subsystem's private data relate to requested con_id.
>> + */
>> +struct hte_ts_desc {
>> +	u32 con_id;
>> +	char *name;
>> +	void *hte_data;
>> +};
>> +
>> +/**
>> + * struct hte_ops - HTE operations set by providers.
>> + *
>> + * @request: Hook for requesting a HTE timestamp. Returns 0 on success,
>> + * non-zero for failures.
>> + * @release: Hook for releasing a HTE timestamp. Returns 0 on success,
>> + * non-zero for failures.
>> + * @enable: Hook to enable the specified timestamp. Returns 0 on success,
>> + * non-zero for failures.
>> + * @disable: Hook to disable specified timestamp. Returns 0 on success,
>> + * non-zero for failures.
>> + * @get_clk_src_info: Hook to get the clock information the provider uses
>> + * to timestamp. Returns 0 for success and negative error code for failure. On
>> + * success HTE subsystem fills up provided struct hte_clk_info.
>> + *
>> + * xlated_id parameter is used to communicate between HTE subsystem and the
>> + * providers. It is the same id returned during xlate API call and translated
>> + * by the provider. This may be helpful as both subsystem and provider locate
>> + * the requested entity in constant time, where entity could be anything from
>> + * lines, signals, events, buses etc.. that providers support.
>> + */
>> +struct hte_ops {
>> +	int (*request)(struct hte_chip *chip, u32 xlated_id);
>> +	int (*release)(struct hte_chip *chip, u32 xlated_id);
>> +	int (*enable)(struct hte_chip *chip, u32 xlated_id);
>> +	int (*disable)(struct hte_chip *chip, u32 xlated_id);
>> +	int (*get_clk_src_info)(struct hte_chip *chip,
>> +				struct hte_clk_info *ci);
>> +};
>> +
>> +/**
>> + * struct hte_chip - Abstract HTE chip structure.
>> + * @name: functional name of the HTE IP block.
>> + * @dev: device providing the HTE.
>> + * @ops: callbacks for this HTE.
>> + * @nlines: number of lines/signals supported by this chip.
>> + * @xlate: Callback which translates consumer supplied logical ids to
>> + * physical ids, return from 0 for the success and negative for the
>> + * failures. It stores (0 to @nlines) in xlated_id parameter for the success.
>> + * @of_hte_n_cells: Number of cells used to form the HTE specifier.
>> + * @gdev: HTE subsystem abstract device, internal to the HTE subsystem.
>> + * @data: chip specific private data.
>> + */
>> +struct hte_chip {
>> +	const char *name;
>> +	struct device *dev;
>> +	const struct hte_ops *ops;
>> +	u32 nlines;
>> +	int (*xlate)(struct hte_chip *gc,
>> +		     const struct of_phandle_args *args,
>> +		     struct hte_ts_desc *desc, u32 *xlated_id);
>> +	u8 of_hte_n_cells;
>> +
>> +	struct hte_device *gdev;
>> +	void *data;
>> +};
>> +
>> +#if IS_ENABLED(CONFIG_HTE)
>> +/* HTE APIs for the providers */
>> +int devm_hte_register_chip(struct hte_chip *chip);
>> +int hte_push_ts_ns(const struct hte_chip *chip, u32 xlated_id,
>> +		   struct hte_ts_data *data);
>> +
>> +/* HTE APIs for the consumers */
>> +
>> +int hte_release_ts(struct hte_ts_desc *desc);
>> +int devm_of_hte_request_ts(struct device *dev, struct hte_ts_desc *desc,
>> +			   hte_ts_cb_t cb, hte_ts_threaded_cb_t tcb,
>> +			   void *data);
>> +int hte_req_ts_by_hte_name(struct device *dev, const char *propname,
>> +			   struct hte_ts_desc *desc, hte_ts_cb_t cb,
>> +			   hte_ts_threaded_cb_t tcb, void *data);
>> +int hte_enable_ts(struct hte_ts_desc *desc);
>> +int hte_disable_ts(struct hte_ts_desc *desc);
>> +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
>> +			 struct hte_clk_info *ci);
>> +
>> +#else /* !CONFIG_HTE */
>> +static inline int devm_hte_register_chip(struct hte_chip *chip)
>> +{
>> +	return -ENOTSUPP;
>> +}
>> +
>> +static inline int hte_push_ts_ns(const struct hte_chip *chip,
>> +				 u32 xlated_id,
>> +				 const struct hte_ts_data *data)
>> +{
>> +	return -ENOTSUPP;
>> +}
>> +
>> +static inline int hte_release_ts(struct hte_ts_desc *desc)
>> +{
>> +	return -ENOTSUPP;
>> +}
>> +
>> +static inline int devm_of_hte_request_ts(struct device *dev,
>> +					 struct hte_ts_desc *desc,
>> +					 hte_ts_cb_t cb,
>> +					 hte_ts_threaded_cb_t threaded_cb,
>> +					 void *data)
>> +{
>> +	return -ENOTSUPP;
>> +}
>> +
> Rename threaded_cb to tcb to be consistent with other functions.
>
>> +static inline int hte_req_ts_by_hte_name(struct device *dev,
>> +					 const char *propname,
>> +					 struct hte_ts_desc *desc,
>> +					 hte_ts_cb_t cb,
>> +					 hte_ts_threaded_cb_t tcb, void *data)
>> +{
>> +	return -ENOTSUPP;
>> +}
>> +
> Make line wrapping of hte_req_ts_by_hte_name() consistent with
> devm_of_hte_request_ts().
Sure, will do.
>
> Cheers,
> Kent.
>

Re: [RFC v3 02/12] drivers: Add hardware timestamp engine (HTE)

[Kent Gibson]

On Tue, Dec 07, 2021 at 04:36:35PM -0800, Dipen Patel wrote:
> Hi,
> 

[snip]

> >> +/**
> >> + * enum hte_return- HTE subsystem return values used during callback.
> >> + *
> >> + * @HTE_CB_HANDLED: The consumer handled the data successfully.
> >> + * @HTE_RUN_THREADED_CB: The consumer needs further processing, in that case HTE
> >> + * subsystem will invoke kernel thread and call secondary callback provided by
> >> + * the consumer during devm_of_hte_request_ts and hte_req_ts_by_dt_node call.
> >> + * @HTE_CB_TS_DROPPED: The client returns when it can not store ts data.
> >> + * @HTE_CB_ERROR: The client returns error if anything goes wrong.
> >> + */
> >> +enum hte_return {
> >> +	HTE_CB_HANDLED,
> >> +	HTE_RUN_THREADED_CB,
> >> +	HTE_CB_TS_DROPPED,
> >> +	HTE_CB_ERROR,
> >> +};
> >> +typedef enum hte_return hte_return_t;
> >> +
> > Wrt HTE_CB_TS_DROPPED, why is the client dropping data any of hte's
> > business?  It is also confusing in that I would expect the dropped_ts
> > gauge, that you increment when this code is returned, to indicate the
> > events dropped by the hardware, not the client.  But then you have no
> > indication of events dropped by hardware at all, though you could
> > determine that from gaps in the sequence numbers.
> > Anyway, the client can do the math in both cases if they care to, so not
> > sure what its purpose is here.
> 
> It is used for statistical purpose and hte being subsytem it can provide
> 
> standard interface in debugfs (so that clients do not have to) to anyone interested.
> 
> The dropped_ts could represent total dropped ts by both hardware and
> 
> client. I can add debugfs interface to break it down further if it helps in statistics.
> 

Updating stats is not what the return code here is for.

And what if the client discards the event AFTER returning from the
handler, say in the threaded cb?

If you want stats fedback then provide a function for the client to call
to update stats, rather than piggy-backing it on the callback return.
I'm unconvinced that stats are a worthwhile addition, and you certainly
don't need to bake it into your core api.

Cheers,
Kent.

Re: [RFC v3 09/12] gpiolib: cdev: Add hardware timestamp clock type

[Dipen Patel]

On 12/1/21 4:53 PM, Kent Gibson wrote:
> On Wed, Dec 01, 2021 at 10:01:46AM -0800, Dipen Patel wrote:
>> Hi,
>>
>>
>> On 12/1/21 9:16 AM, Kent Gibson wrote:
>>> On Tue, Nov 30, 2021 at 07:29:20PM -0800, Dipen Patel wrote:
>>>> Hi,
>>>>
>>>> On 11/25/21 5:31 PM, Kent Gibson wrote:
>>>>> On Tue, Nov 23, 2021 at 11:30:36AM -0800, Dipen Patel wrote:
>>>>>> This patch adds new clock type for the GPIO controller which can
>>>>>> timestamp gpio lines in realtime using hardware means. To expose such
>>>>>> functionalities to the userspace, code has been added in this patch
>>>>>> where during line create call, it checks for new clock type and if
>>>>>> requested, calls hardware timestamp related API from gpiolib.c.
>>>>>> During line change event, the HTE subsystem pushes timestamp data
>>>>>> through callbacks.
>>>>>>
>>>>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>>>>>> Acked-by: Linus Walleij <linus.walleij@linaro.org>
>>>>>> ---
>>>>>> Changes in v2:
>>>>>> - Added hte_dir and static structure hte_ts_desc.
>>>>>> - Added callbacks which get invoked by HTE when new data is available.
>>>>>> - Better use of hte_dir and seq from hte_ts_desc.
>>>>>> - Modified sw debounce function to accommodate hardware timestamping.
>>>>>>
>>>>>>  drivers/gpio/gpiolib-cdev.c | 161 ++++++++++++++++++++++++++++++++++--
>>>>>>  include/uapi/linux/gpio.h   |   1 +
>>>>>>  2 files changed, 153 insertions(+), 9 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
>>>>>> index c7b5446d01fd..1736ad54e3ec 100644
>>>>>> --- a/drivers/gpio/gpiolib-cdev.c
>>>>>> +++ b/drivers/gpio/gpiolib-cdev.c
>>>>>> @@ -464,6 +464,12 @@ struct line {
>>>>>>  	 * stale value.
>>>>>>  	 */
>>>>>>  	unsigned int level;
>>>>>> +	/*
>>>>>> +	 * dir will be touched in HTE callbacks hte_ts_cb_t and
>>>>>> +	 * hte_ts_threaded_cb_t and they are mutually exclusive. This will be
>>>>>> +	 * unused when HTE is not supported/disabled.
>>>>>> +	 */
>>>>>> +	enum hte_dir dir;
>>>>>>  };
>>>>>>  
>>>>> Documentation should be in present tense, so 
>>>>>
>>>>> s/will be/is/g
>>>>>
>>>>> Same applies to other patches.
>>>>>
>>>>> Also
>>>>>
>>>>> s/touched/accessed/
>>>>>
>>>>> dir is a poor name for the field.  It is the hte edge direction and
>>>>> effectively the line level, so call it hte_edge_dirn or
>>>>> hte_edge_direction or hte_level.
>>>>>
>>>>> And it is placed in a section of the struct documented as "debouncer specific
>>>>> fields", but it is not specfic to the debouncer.  Add a "hte specific
>>>>> fields" section if nothing else is suitable.
>>>>>
>>>>>>  /**
>>>>>> @@ -518,6 +524,7 @@ struct linereq {
>>>>>>  	 GPIO_V2_LINE_DRIVE_FLAGS | \
>>>>>>  	 GPIO_V2_LINE_EDGE_FLAGS | \
>>>>>>  	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
>>>>>> +	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
>>>>>>  	 GPIO_V2_LINE_BIAS_FLAGS)
>>>>>>  
>>>>>>  static void linereq_put_event(struct linereq *lr,
>>>>>> @@ -546,6 +553,94 @@ static u64 line_event_timestamp(struct line *line)
>>>>>>  	return ktime_get_ns();
>>>>>>  }
>>>>>>  
>>>>>> +static hte_return_t process_hw_ts_thread(void *p)
>>>>>> +{
>>>>>> +	struct line *line = p;
>>>>>> +	struct linereq *lr = line->req;
>>>>>> +	struct gpio_v2_line_event le;
>>>>>> +	u64 eflags;
>>>>>> +
>>>>>> +	memset(&le, 0, sizeof(le));
>>>>>> +
>>>>>> +	le.timestamp_ns = line->timestamp_ns;
>>>>>> +	line->timestamp_ns = 0;
>>>>>> +
>>>>> What is the purpose of this zeroing?
>>>>>
>>>>>> +	if (line->dir >= HTE_DIR_NOSUPP) {
>>>>>> +		eflags = READ_ONCE(line->eflags);
>>>>>> +		if (eflags == GPIO_V2_LINE_FLAG_EDGE_BOTH) {
>>>>>> +			int level = gpiod_get_value_cansleep(line->desc);
>>>>>> +
>>>>>> +			if (level)
>>>>>> +				/* Emit low-to-high event */
>>>>>> +				le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>>>> +			else
>>>>>> +				/* Emit high-to-low event */
>>>>>> +				le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_RISING) {
>>>>>> +			/* Emit low-to-high event */
>>>>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_FALLING) {
>>>>>> +			/* Emit high-to-low event */
>>>>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>>>> +		} else {
>>>>>> +			return HTE_CB_ERROR;
>>>>>> +		}
>>>>>> +	} else {
>>>>>> +		if (line->dir == HTE_RISING_EDGE_TS)
>>>>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>>>> +		else
>>>>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>>>> +	}
>>>>> The mapping from line->dir to le.id needs to take into account the active
>>>>> low setting for the line.
>>>>>
>>>>> And it might be simpler if the hte_ts_data provided the level, equivalent
>>>>> to gpiod_get_raw_value_cansleep(), rather than an edge direction, so you
>>>>> can provide a common helper to determine the edge given the raw level.
>>>> (So from the level determine the edge?) that sound right specially when
>>>>
>>>> HTE provider has capability to record the edge in that case why bother
>>>>
>>>> getting the level and determine edge?
>>>>
>>>> Calculating the edge from the level makes sense when hte provider does not
>>>>
>>>> have that feature and that is what if (line->dir >= HTE_DIR_NOSUPP) does.
>>>>
>>> As asked in the review of patch 02, do you have an example of hardware that
>>> reports an edge direction rather than NOSUPP?
>> No...
> So you are adding an interface that nothing will currently use.
> Are there plans for hardware that will report the edge, and you are
> laying the groundwork here?

Adding here for the general case should there be provider

available with such feature.

>
>>> Anyway, this is just a naming thing - the information content being passed
>>> is the the same, be it high/low/unknown or rising/falling/unknown.
>>>
>>> If the hardware does report edge direction then it is just one bit, and
>>> that also corresponds to the physical level immediately following the
>>> edge, so no additional conversion required there.
>>>
>>> It would be clearer to pass a level than an edge, as
>>>  - a hte edge (hte_dir) could be confused with a cdev edge
>>>    (gpio_v2_line_event_id), in mails like this if not in the code.
>>>  - cdev will fallback to using the physical level to determine the edge
>>>    if the hte can't provide it
>> I believe I have retained all the fallback logic in such case.
>>>  - cdev has to perform inversion if active low and it already does that
>>>    based on levels
>>>
>>>>>> +
>>>>>> +	le.line_seqno = line->line_seqno;
>>>>>> +	le.seqno = (lr->num_lines == 1) ? le.line_seqno : line->req_seqno;
>>>>>> +	le.offset = gpio_chip_hwgpio(line->desc);
>>>>>> +
>>>>>> +	linereq_put_event(lr, &le);
>>>>>> +
>>>>>> +	return HTE_CB_HANDLED;
>>>>>> +}
>>>>>> +
>>>>>> +static hte_return_t process_hw_ts(struct hte_ts_data *ts, void *p)
>>>>>> +{
>>>>>> +	struct line *line = p;
>>>>>> +	struct linereq *lr = line->req;
>>>>>> +
>>>>>> +	if (!ts)
>>>>>> +		return HTE_CB_ERROR;
>>>>>> +
>>>>>> +	line->timestamp_ns = ts->tsc;
>>>>>> +	line->dir = ts->dir;
>>>>>> +
>>>>> The doc for timestamp_ns states:
>>>>>
>>>>> 	 * timestamp_ns and req_seqno are accessed only by
>>>>> 	 * edge_irq_handler() and edge_irq_thread(), which are themselves
>>>>> 	 * mutually exclusive, so no additional protection is necessary.
>>>>>
>>>>> That no longer holds.  It is now also accessed here, and in
>>>>> process_hw_ts_thread(), which wont run concurrently with each other or
>>>>> the edge_irq_* handlers, but also in debounce_work_func() which may run
>>>>> concurrently with the others.
>>>>> So timestamp_ns now requires protection from concurrent access.
>>>>>
>>>>>> +	/*
>>>>>> +	 * It is possible that HTE engine detects spurious edges for the
>>>>>> +	 * lines where software debounce is enabled. This primary callback
>>>>>> +	 * will be called multiple times in that case. It will be better to
>>>>>> +	 * let debounce_work_func handle instead of process_hw_ts_thread.
>>>>>> +	 * The timestamp_ns will be overwritten here which is fine as we are
>>>>>> +	 * interested in the last value anyway. The debounce_work_func will
>>>>>> +	 * then just read whatever last line->timestamp_ns is stored. Because
>>>>>> +	 * this callback can be called multiple times, we are not really
>>>>>> +	 * interested in ts->seq.
>>>>>> +	 */
>>>>> Not sure what this is trying to say.
>>>>> Is this the primary callback? Or debounce_irq_handler()?
>>>> This is primary callback called from HTE when it pushes new TS data per line, it
>>>>
>>>> also says so in the second line.
>>>>
>>> Yeah, I probably read that as "The primary callback", but it is
>>> confusing anyway. "This primary callback" implies there is another
>>> primary callback. 
>>> Just say "This handler" instead of "This primary callback".
>> Noted...
>>>>> You say you really aren't interested in ts->seq, but the code immediately
>>>>> uses it.
>>>> That is when sw_debounced is not set and whole paragraph is about when
>>>>
>>>> sw_debounced is set.
>>>>
>>> So your whole comment here is about the else case?
>>> Then either put the comment where the else would be, or better yet invert
>>> the logic and return immediately if sw_debounced.
>> Sure...
> Understand, you wont be maintaining this code, even if you intend to.
> Consider the poor unfortunate who will have to deal with your code in
> the future.  This is not and should not be a minor consideration.
>
> Sometimes long winded comments only add confusion rather than clarity.
> If the code alone is confusing and requires more than a line or two of
> explanatory comments, excluding function documentation, then you might
> want to rethink your code.
>
> In this case the clearest is probably to restructure the if condition as
> I suggested and simplify or even drop the comment.
>
>>>>> Reword to clarify.
>>>>> And add braces after function names to highlight them, so
>>>>> debounce_work_func().
>>>> Will do.
>>>>>> +	if (!READ_ONCE(line->sw_debounced)) {
>>>>>> +		line->line_seqno = ts->seq;
>>>>>> +
>>>>>> +		/*
>>>>>> +		 * Increment in this callback incase all the lines in linereq
>>>>>> +		 * are enabled for hw timestamping. This will work even if
>>>>>> +		 * subset of lines are enabled for hw timestamping as
>>>>>> +		 * edge_irq_* callbacks will proceed as usual for them.
>>>>>> +		 */
>>>>> s/incase/in case/
>>>>>
>>>>> Not sure what the comment is trying to say. There is no check here that
>>>>> the other lines have HTE enabled.  And that is not relevant anyway.
>>>>> The edge_irq_* handlers will proceed as usual for those lines NOT
>>>>> enabled for hw timestamping.
>>>>>
>>>>> To clarify, the line_seqno indicates where this event lies in the
>>>>> sequence of events for the line.
>>>>> The request seqno indicates where this event lines in the sequence of
>>>>> events for the request.
>>>>> For a single line request these are the same, hence the minor
>>>>> optimisation of not updating lr->seqno below.
>>>>>
>>>>>> +		if (lr->num_lines != 1)
>>>>>> +			line->req_seqno = atomic_inc_return(&lr->seqno);
>>>>>> +
>>>>> The req_seqno should be updated corresponding to the change in the
>>>>> line_reqno.  That always used to be 1, but no longer if hte can discard
>>>>> events, i.e. skip over line_seqnos.
>>>> HTE does not discard any events, it pushes to clients as soon as its
>>>>
>>>> available through primary callback.
>>> The discarding of events that I am referring to is from your previous
>>> answers that indicated, to me anyway, that there could be gaps in the
>>> ts->seq numbers if the hardware event FIFO overflowed.
>>> Is that not the case?
>> Not in this patch series as the provider I have dealt with does not such
>>
>> feature. ts->seq is software counter in that case.
>>
> The code here has to deal with the general case, not just the one example
> driver you have provided.  So in general there COULD be gaps in the
> ts->seq, right?
>
> I do see that using the ts-seq for sw debounced lines is problematic
> though. The debouncer itself will be discarding hte events, but that
> shouldn't be considered a lost event to the user.  You could track
> how many events are discarded by the debouncer and subtract those from
> the sequence numbers reported to userspace?
>
>>> And when you say "primary callback", both here and elsewhere, you mean the
>>> cb parameter to gpiod_req_hw_timestamp_ns() and gc->req_hw_timestamp()?
>> yes, cb is primary and tcb (threaded cb) is optional secondary.
>>> Either way, be specific and name the function or parameter, or find a
>>> better term than "primary callback".
>>> In this case "the client's event handler" would be much clearer.
>> Noted...
>>>>> To be consistent, i.e. if events were lost for this line then they were
>>>>> also lost for the requested lines, the lr->seqno should be incremented by
>>>>> the change in line_seqno.  Probably with some sanity checks.
>>>>>
>>>>>> +		return HTE_RUN_THREADED_CB;
>>>>>> +	}
>>>>>> +
>>>>>> +	return HTE_CB_HANDLED;
>>>>>> +}
>>>>>> +
>>>>>>  static irqreturn_t edge_irq_thread(int irq, void *p)
>>>>>>  {
>>>>>>  	struct line *line = p;
>>>>>> @@ -553,6 +648,10 @@ static irqreturn_t edge_irq_thread(int irq, void *p)
>>>>>>  	struct gpio_v2_line_event le;
>>>>>>  	u64 eflags;
>>>>>>  
>>>>>> +	/* Let process_hw_ts_thread handle */
>>>>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
>>>>>> +		return IRQ_HANDLED;
>>>>>> +
>>>>> This adds pointless runtime overhead, and for everyone not just hte users.
>>>>> Don't stub out a handler in the handler - stub it out where it is
>>>>> registered by registering a stub handler.  Or don't request it at all.
>>>>>
>>>>> So why would gpiolib-cdev be requesting the irq, only to stub out
>>>>> the handlers?
>>>>> If that has a side-effect that hte requires then hte should be taking
>>>>> care of it - it is not gpiolib-cdev's problem.
>>>> - Why stop at moving irq and debounce related stuff to hte then?
>>>>
>>> How about you answer my question before asking your own?
>>> Here I am only questioning why gpiolib-cdev is requesting an interrupt
>>> for a hte line at all.  It has no reason to, so hte must have?
>> Commented below in "separation of concern" paragraph.
>>
>>> And where did I suggest moving the "debounce stuff" to hte?
>> Perhaps I misunderstood "That includes the sw debouncer - more on that
>>
>> below." comment.
>>
> The point I was trying to make there was that, for hte enabled lines, the
> sw debouncer should be driven by hte events, not irq events.
> That does not imply nor require moving the debouncer to hte.
>
>>> What I am suggesting is called separation of concerns.  In this context
>>> the intent is to look towards abstracting the edge event generation.
>>> Having hte and irq entangled for no apparent reason makes doing that more
>> But in this patch series (i.e. HTE provider), they are entangled as hte provider
>>
>> will only ts the GPIO line which is configured as input irq. That is why I have
>>
>> separated GPIO line config part and TS part. In other words, let cdev handle
>>
>> any line config that userspace has asked and let hte handle ts part if the
>>
>> line config is supported (If line config is not suitable, it will not enable hte, see
>>
>> gpio-tegra186.c function tegra186_gpio_req_hw_ts() in this patch series where
>>
>> check happens for the tegra hte provider).
>>
> Can you explain "hte provider will only ts the GPIO line which is
> configured as input irq"?  That sounds like a hte concern to me, or even
> worse a particular hte provider problem.
>
> The user requests a line with edge event detection enabled and with hte
> timestamps.
> Nothing in that requires irq from gpiolib-cdev.
> gpiolib-cdev should use hte as the event source where it would usually
> use irq. If hte requires irq to allow it to generate those events then
> hte should be responsible for requesting the irq.
> In a hypothetical extreme case where gpiolib-cdev only supported hte
> lines there should be no reference to irq in gpiolib-cdev.
> Yet you continue to insist that gpiolib-cdev should request the irq for
> hte.
> What am I missing here?
>
>>> difficult than it needs to be, whereas keeping them separate greatly
>>> simplifies identification of common code suitable for refactoring
>>> subsequently.
>>>
>>> Not sure what to call what you are suggesting.
>>>
>>>> I mean if there is hte provider which can TS GPIO output/input
>>>>
>>>> does it mean hte is responsible for parsing the GPIO line configs, setting them up
>>>>
>>>> (i.e. input or output) as well? Are we not duplicating logic instead of
>>>>
>>>> leveraging gpio-cdev? Does it make sense for the HTE subsystem which not
>>>>
>>>> only TS the GPIOs but other SoC lines?
>>>>
>>>> - What happens to in kernel GPIO HTE client (for example, hte-tegra194-gpio-test.c)?
>>>>
>>>> some clients do more in their IRQ handler than what edge_irq_handler does in which
>>>>
>>>> case it would make sense to have them request irq in their code than through HTE.
>>>>
>>>>> And speaking as to how the whole hte/gpiolib-cdev interface should work,
>>>>> hte should be an edge event generator alternative to irq.  So lines with
>>>>> hte enabled should work without any irq calls from gpiolib-cdev.
>>>>> That includes the sw debouncer - more on that below.
>>>>>
>>>>>>  	/* Do not leak kernel stack to userspace */
>>>>>>  	memset(&le, 0, sizeof(le));
>>>>>>  
>>>>>> @@ -604,6 +703,10 @@ static irqreturn_t edge_irq_handler(int irq, void *p)
>>>>>>  	struct line *line = p;
>>>>>>  	struct linereq *lr = line->req;
>>>>>>  
>>>>>> +	/* Let HTE supplied callbacks handle */
>>>>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags))
>>>>>> +		return IRQ_HANDLED;
>>>>>> +
>>>>>>  	/*
>>>>>>  	 * Just store the timestamp in hardirq context so we get it as
>>>>>>  	 * close in time as possible to the actual event.
>>>>>> @@ -682,14 +785,6 @@ static void debounce_work_func(struct work_struct *work)
>>>>>>  	/* Do not leak kernel stack to userspace */
>>>>>>  	memset(&le, 0, sizeof(le));
>>>>>>  
>>>>>> -	lr = line->req;
>>>>>> -	le.timestamp_ns = line_event_timestamp(line);
>>>>>> -	le.offset = gpio_chip_hwgpio(line->desc);
>>>>>> -	line->line_seqno++;
>>>>>> -	le.line_seqno = line->line_seqno;
>>>>>> -	le.seqno = (lr->num_lines == 1) ?
>>>>>> -		le.line_seqno : atomic_inc_return(&lr->seqno);
>>>>>> -
>>>>>>  	if (level)
>>>>>>  		/* Emit low-to-high event */
>>>>>>  		le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>>>> @@ -697,6 +792,23 @@ static void debounce_work_func(struct work_struct *work)
>>>>>>  		/* Emit high-to-low event */
>>>>>>  		le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>>>>  
>>>>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
>>>>>> +		le.timestamp_ns = line->timestamp_ns;
>>>>>> +		if (line->dir < HTE_DIR_NOSUPP)
>>>>>> +			le.id = (line->dir == HTE_RISING_EDGE_TS) ?
>>>>>> +				 GPIO_V2_LINE_EVENT_RISING_EDGE :
>>>>>> +				 GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>>>> +	} else {
>>>>>> +		le.timestamp_ns = line_event_timestamp(line);
>>>>>> +	}
>>>>>> +
>>>>> Move the FLAG_EVENT_CLOCK_HARDWARE check into line_event_timestamp().
>>>>>
>>>>> And the id fudging is necessary because the level returned by
>>>>> gpiod_get_raw_value_cansleep() can disagree with the level from hte?
>>>>> So you are still trying to synchronise events from two streams.
>>>>> And that is still broken.
>>>>> If a hte event occurs between the level being sampled by
>>>>> gpiod_get_raw_value_cansleep() and the line->dir being read then the line
>>>>> will have toggled and you will be reporting the opposite state than the
>>>>> one the debouncer determined was stable.  And maybe the wrong timestamp as
>>>>> well.
>>>>>
>>>>> For lines where hte is enabled, the hte should be the source of level for
>>>>> the debouncer, not the raw value.  And the mod_delayed_work() that
>>>>> drives the debouncer should be called by a hte handler, not an irq handler.
>>>>>
>>>>> There is also a race on reading the hte timestamp (line->timestamp_ns) and
>>>>> the hte level (line->dir), such that you can get the level from one event
>>>>> the timestamp from another.
>>>>>
>>>>>> +	lr = line->req;
>>>>>> +	le.offset = gpio_chip_hwgpio(line->desc);
>>>>>> +	line->line_seqno++;
>>>>>> +	le.line_seqno = line->line_seqno;
>>>>>> +	le.seqno = (lr->num_lines == 1) ?
>>>>>> +		le.line_seqno : atomic_inc_return(&lr->seqno);
>>>>>> +
>>>>> What is the purpose of moving this block of code moved from before the
>>>>> if (level)?
>>>>>
>>>>>
>>>>>>  	linereq_put_event(lr, &le);
>>>>>>  }
>>>>>>  
>>>>>> @@ -891,7 +1003,6 @@ static int gpio_v2_line_flags_validate(u64 flags)
>>>>>>  	/* Return an error if an unknown flag is set */
>>>>>>  	if (flags & ~GPIO_V2_LINE_VALID_FLAGS)
>>>>>>  		return -EINVAL;
>>>>>> -
>>>>> Gratuitous whitespace change.
>>>>>
>>>>>>  	/*
>>>>>>  	 * Do not allow both INPUT and OUTPUT flags to be set as they are
>>>>>>  	 * contradictory.
>>>>>> @@ -900,6 +1011,11 @@ static int gpio_v2_line_flags_validate(u64 flags)
>>>>>>  	    (flags & GPIO_V2_LINE_FLAG_OUTPUT))
>>>>>>  		return -EINVAL;
>>>>>>  
>>>>>> +	/* Only allow one event clock source */
>>>>>> +	if ((flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME) &&
>>>>>> +	    (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE))
>>>>>> +		return -EINVAL;
>>>>>> +
>>>>>>  	/* Edge detection requires explicit input. */
>>>>>>  	if ((flags & GPIO_V2_LINE_EDGE_FLAGS) &&
>>>>>>  	    !(flags & GPIO_V2_LINE_FLAG_INPUT))
>>>>>> @@ -992,6 +1108,8 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 flags,
>>>>>>  
>>>>>>  	assign_bit(FLAG_EVENT_CLOCK_REALTIME, flagsp,
>>>>>>  		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME);
>>>>>> +	assign_bit(FLAG_EVENT_CLOCK_HARDWARE, flagsp,
>>>>>> +		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE);
>>>>>>  }
>>>>>>  
>>>>>>  static long linereq_get_values(struct linereq *lr, void __user *ip)
>>>>>> @@ -1154,6 +1272,21 @@ static long linereq_set_config_unlocked(struct linereq *lr,
>>>>>>  				return ret;
>>>>>>  		}
>>>>>>  
>>>>>> +		/* Check if new config sets hardware assisted clock */
>>>>>> +		if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
>>>>>> +			ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
>>>>>> +							process_hw_ts_thread,
>>>>>> +							&lr->lines[i]);
>>>>>> +			if (ret)
>>>>>> +				return ret;
>>>>> Note that the line config is the complete line config, not a delta.
>>>>>
>>>>> What happens when a line that already has hte enabled is reconfigured
>>>>> and still has hte enabled?  i.e. what happens when
>>>>> gpiod_req_hw_timestamp_ns() is called for the second time?
>>>> HTE will return without doing anything with error code.
>>>>
>>> But this is not an error case, it is a normal reconfigure of an
>>> attribute other than the hte flag.
>> I assumed when this function is called it will "reset" and not  update the configs.
>>
>> If this assumption is wrong, I will correct the logic here.
>>
> The set_config does whatever is required to change the line request
> config from the old to the new.  Both old and new are complete snapshots
> of the line request config.
>
> The change should be seamless except for the attributes being changed.
> We certainly do not reset the config and start from scratch - that would
> be little better than the user releasing and re-requesting the lines.
>
> For many attributes we can just apply the new config.  But some cases,
> such as a change to active low polarity, are a lot more involved.
> And if hte considers re-requesting the line to be an error then you
> should only make the hte request when the hte flag changes to set.
>
> Cheers,
> Kent.
>
>>> And that will now return an error to userspace?
>>>
>>> Cheers,
>>> Kent.
>>>
>>>>> You provide a comment for the release case below, what of the request
>>>>> case?
>>>>>
>>>>> If you need to check for change then compare the old and new flags, as
>>>>> the polarity_change check does (not visible in the diff here).
>>>>>
>>>>>> +		} else {
>>>>>> +			/*
>>>>>> +			 * HTE subsys will do nothing if there is nothing to
>>>>>> +			 * release.
>>>>>> +			 */
>>>>>> +			gpiod_rel_hw_timestamp_ns(desc);
>>>>>> +		}
>>>>>> +
>>>>> Comment will fit on one line.
>>>>>
>>>>> And it would be better to document that the function is idempotent in the
>>>>> function documentation, not everywhere it is used.
>>>>>
>>>>>>  		blocking_notifier_call_chain(&desc->gdev->notifier,
>>>>>>  					     GPIO_V2_LINE_CHANGED_CONFIG,
>>>>>>  					     desc);
>>>>>> @@ -1409,6 +1542,14 @@ static int linereq_create(struct gpio_device *gdev, void __user *ip)
>>>>>>  					flags & GPIO_V2_LINE_EDGE_FLAGS);
>>>>>>  			if (ret)
>>>>>>  				goto out_free_linereq;
>>>>>> +
>>>>>> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
>>>>>> +				ret = gpiod_req_hw_timestamp_ns(desc, process_hw_ts,
>>>>>> +							process_hw_ts_thread,
>>>>>> +							&lr->lines[i]);
>>>>>> +				if (ret)
>>>>>> +					goto out_free_linereq;
>>>>>> +			}
>>>>>>  		}
>>>>>>  
>>>>>>  		blocking_notifier_call_chain(&desc->gdev->notifier,
>>>>>> @@ -1959,6 +2100,8 @@ static void gpio_desc_to_lineinfo(struct gpio_desc *desc,
>>>>>>  
>>>>>>  	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &desc->flags))
>>>>>>  		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
>>>>>> +	else if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &desc->flags))
>>>>>> +		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
>>>>>>  
>>>>>>  	debounce_period_us = READ_ONCE(desc->debounce_period_us);
>>>>>>  	if (debounce_period_us) {
>>>>>> diff --git a/include/uapi/linux/gpio.h b/include/uapi/linux/gpio.h
>>>>>> index eaaea3d8e6b4..d360545b4c21 100644
>>>>>> --- a/include/uapi/linux/gpio.h
>>>>>> +++ b/include/uapi/linux/gpio.h
>>>>>> @@ -80,6 +80,7 @@ enum gpio_v2_line_flag {
>>>>>>  	GPIO_V2_LINE_FLAG_BIAS_PULL_DOWN	= _BITULL(9),
>>>>>>  	GPIO_V2_LINE_FLAG_BIAS_DISABLED		= _BITULL(10),
>>>>>>  	GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME	= _BITULL(11),
>>>>>> +	GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE	= _BITULL(12),
>>>>>>  };
>>>>>>  
>>>>> I'm now thinking this name, "HARDWARE" is too vague, in case other
>>>>> timestamp source alternatives join the fray, and so should be "HTE".
>>>>>
>>>>> Cheers,
>>>>> Kent.

Re: [RFC v3 02/12] drivers: Add hardware timestamp engine (HTE)

[Dipen Patel]

On 12/7/21 5:21 PM, Kent Gibson wrote:
> On Tue, Dec 07, 2021 at 04:36:35PM -0800, Dipen Patel wrote:
>> Hi,
>>
> [snip]
>
>>>> +/**
>>>> + * enum hte_return- HTE subsystem return values used during callback.
>>>> + *
>>>> + * @HTE_CB_HANDLED: The consumer handled the data successfully.
>>>> + * @HTE_RUN_THREADED_CB: The consumer needs further processing, in that case HTE
>>>> + * subsystem will invoke kernel thread and call secondary callback provided by
>>>> + * the consumer during devm_of_hte_request_ts and hte_req_ts_by_dt_node call.
>>>> + * @HTE_CB_TS_DROPPED: The client returns when it can not store ts data.
>>>> + * @HTE_CB_ERROR: The client returns error if anything goes wrong.
>>>> + */
>>>> +enum hte_return {
>>>> +	HTE_CB_HANDLED,
>>>> +	HTE_RUN_THREADED_CB,
>>>> +	HTE_CB_TS_DROPPED,
>>>> +	HTE_CB_ERROR,
>>>> +};
>>>> +typedef enum hte_return hte_return_t;
>>>> +
>>> Wrt HTE_CB_TS_DROPPED, why is the client dropping data any of hte's
>>> business?  It is also confusing in that I would expect the dropped_ts
>>> gauge, that you increment when this code is returned, to indicate the
>>> events dropped by the hardware, not the client.  But then you have no
>>> indication of events dropped by hardware at all, though you could
>>> determine that from gaps in the sequence numbers.
>>> Anyway, the client can do the math in both cases if they care to, so not
>>> sure what its purpose is here.
>> It is used for statistical purpose and hte being subsytem it can provide
>>
>> standard interface in debugfs (so that clients do not have to) to anyone interested.
>>
>> The dropped_ts could represent total dropped ts by both hardware and
>>
>> client. I can add debugfs interface to break it down further if it helps in statistics.
>>
> Updating stats is not what the return code here is for.
>
> And what if the client discards the event AFTER returning from the
> handler, say in the threaded cb?
>
> If you want stats fedback then provide a function for the client to call
> to update stats, rather than piggy-backing it on the callback return.
I agree, will work that in v4.
> I'm unconvinced that stats are a worthwhile addition, and you certainly
> don't need to bake it into your core api.

Wouldn't it help in debugging i.e. quickly check using this interface

if there are drops for given application setup?

>
> Cheers,
> Kent.

Re: [RFC v3 09/12] gpiolib: cdev: Add hardware timestamp clock type

[Dipen Patel]

Hi,

On 12/7/21 5:42 PM, Dipen Patel wrote:
> On 12/1/21 4:53 PM, Kent Gibson wrote:
>> On Wed, Dec 01, 2021 at 10:01:46AM -0800, Dipen Patel wrote:
>>> Hi,
>>>
>>>
>>> On 12/1/21 9:16 AM, Kent Gibson wrote:
>>>> On Tue, Nov 30, 2021 at 07:29:20PM -0800, Dipen Patel wrote:
>>>>>
>>>>>> [snip]
>>>>>>> +	if (line->dir >= HTE_DIR_NOSUPP) {
>>>>>>> +		eflags = READ_ONCE(line->eflags);
>>>>>>> +		if (eflags == GPIO_V2_LINE_FLAG_EDGE_BOTH) {
>>>>>>> +			int level = gpiod_get_value_cansleep(line->desc);
>>>>>>> +
>>>>>>> +			if (level)
>>>>>>> +				/* Emit low-to-high event */
>>>>>>> +				le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>>>>> +			else
>>>>>>> +				/* Emit high-to-low event */
>>>>>>> +				le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>>>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_RISING) {
>>>>>>> +			/* Emit low-to-high event */
>>>>>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>>>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_FALLING) {
>>>>>>> +			/* Emit high-to-low event */
>>>>>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>>>>> +		} else {
>>>>>>> +			return HTE_CB_ERROR;
>>>>>>> +		}
>>>>>>> +	} else {
>>>>>>> +		if (line->dir == HTE_RISING_EDGE_TS)
>>>>>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
>>>>>>> +		else
>>>>>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
>>>>>>> +	}
>>>>>> The mapping from line->dir to le.id needs to take into account the active
>>>>>> low setting for the line.
>>>>>>
>>>>>> And it might be simpler if the hte_ts_data provided the level, equivalent
>>>>>> to gpiod_get_raw_value_cansleep(), rather than an edge direction, so you
>>>>>> can provide a common helper to determine the edge given the raw level.
>>>>> (So from the level determine the edge?) that sound right specially when
>>>>>
>>>>> HTE provider has capability to record the edge in that case why bother
>>>>>
>>>>> getting the level and determine edge?
>>>>>
>>>>> Calculating the edge from the level makes sense when hte provider does not
>>>>>
>>>>> have that feature and that is what if (line->dir >= HTE_DIR_NOSUPP) does.
>>>>>
>>>> As asked in the review of patch 02, do you have an example of hardware that
>>>> reports an edge direction rather than NOSUPP?
>>> No...
>> So you are adding an interface that nothing will currently use.
>> Are there plans for hardware that will report the edge, and you are
>> laying the groundwork here?
> Adding here for the general case should there be provider
>
> available with such feature.

I have a doubt as below on how edge_irq_thread calculates le.id (Only for

gpiod_get_value_cansleep case), i believe clearing that doubt will help me properly

address this issue:

- Does it have potential to read level which might have changed by the time thread is run?

- Does it make sense to read it in edge_irq_handler instead at least of the chip which can

fetch the level without needing to sleep?

>
>>

Re: [RFC v3 09/12] gpiolib: cdev: Add hardware timestamp clock type

[Kent Gibson]

On Wed, Dec 08, 2021 at 12:14:36PM -0800, Dipen Patel wrote:
> Hi,
> 
> On 12/7/21 5:42 PM, Dipen Patel wrote:
> > On 12/1/21 4:53 PM, Kent Gibson wrote:
> >> On Wed, Dec 01, 2021 at 10:01:46AM -0800, Dipen Patel wrote:
> >>> Hi,
> >>>
> >>>
> >>> On 12/1/21 9:16 AM, Kent Gibson wrote:
> >>>> On Tue, Nov 30, 2021 at 07:29:20PM -0800, Dipen Patel wrote:
> >>>>>
> >>>>>> [snip]
> >>>>>>> +	if (line->dir >= HTE_DIR_NOSUPP) {
> >>>>>>> +		eflags = READ_ONCE(line->eflags);
> >>>>>>> +		if (eflags == GPIO_V2_LINE_FLAG_EDGE_BOTH) {
> >>>>>>> +			int level = gpiod_get_value_cansleep(line->desc);
> >>>>>>> +
> >>>>>>> +			if (level)
> >>>>>>> +				/* Emit low-to-high event */
> >>>>>>> +				le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> >>>>>>> +			else
> >>>>>>> +				/* Emit high-to-low event */
> >>>>>>> +				le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >>>>>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_RISING) {
> >>>>>>> +			/* Emit low-to-high event */
> >>>>>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> >>>>>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_FALLING) {
> >>>>>>> +			/* Emit high-to-low event */
> >>>>>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >>>>>>> +		} else {
> >>>>>>> +			return HTE_CB_ERROR;
> >>>>>>> +		}
> >>>>>>> +	} else {
> >>>>>>> +		if (line->dir == HTE_RISING_EDGE_TS)
> >>>>>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> >>>>>>> +		else
> >>>>>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >>>>>>> +	}
> >>>>>> The mapping from line->dir to le.id needs to take into account the active
> >>>>>> low setting for the line.
> >>>>>>
> >>>>>> And it might be simpler if the hte_ts_data provided the level, equivalent
> >>>>>> to gpiod_get_raw_value_cansleep(), rather than an edge direction, so you
> >>>>>> can provide a common helper to determine the edge given the raw level.
> >>>>> (So from the level determine the edge?) that sound right specially when
> >>>>>
> >>>>> HTE provider has capability to record the edge in that case why bother
> >>>>>
> >>>>> getting the level and determine edge?
> >>>>>
> >>>>> Calculating the edge from the level makes sense when hte provider does not
> >>>>>
> >>>>> have that feature and that is what if (line->dir >= HTE_DIR_NOSUPP) does.
> >>>>>
> >>>> As asked in the review of patch 02, do you have an example of hardware that
> >>>> reports an edge direction rather than NOSUPP?
> >>> No...
> >> So you are adding an interface that nothing will currently use.
> >> Are there plans for hardware that will report the edge, and you are
> >> laying the groundwork here?
> > Adding here for the general case should there be provider
> >
> > available with such feature.
> 
> I have a doubt as below on how edge_irq_thread calculates le.id (Only for
> 
> gpiod_get_value_cansleep case), i believe clearing that doubt will help me properly
> 
> address this issue:
> 
> - Does it have potential to read level which might have changed by the time thread is run?
> 

Yes it does.

> - Does it make sense to read it in edge_irq_handler instead at least of the chip which can
> 
> fetch the level without needing to sleep?
> 

That would not make it any more valid.  There is an inherent race there
- that is the nature of the irq interface.
The existing code does the best it can in the circumstances - for the
more likely case that there isn't another edge between the interrupt
handler and thread.

The hte can do better - assumung it has hardware capable of providing the
edge as well as the timestamp.

Cheers,
Kent.

Re: [RFC v3 09/12] gpiolib: cdev: Add hardware timestamp clock type

[Kent Gibson]

On Tue, Dec 07, 2021 at 05:42:35PM -0800, Dipen Patel wrote:
> 
> On 12/1/21 4:53 PM, Kent Gibson wrote:
> > On Wed, Dec 01, 2021 at 10:01:46AM -0800, Dipen Patel wrote:
> >> Hi,
> >>
> >>
> >> On 12/1/21 9:16 AM, Kent Gibson wrote:
> >>> On Tue, Nov 30, 2021 at 07:29:20PM -0800, Dipen Patel wrote:
> >>>> Hi,
> >>>>
> >>>> On 11/25/21 5:31 PM, Kent Gibson wrote:
> >>>>> On Tue, Nov 23, 2021 at 11:30:36AM -0800, Dipen Patel wrote:
> >>>>>> This patch adds new clock type for the GPIO controller which can
> >>>>>> timestamp gpio lines in realtime using hardware means. To expose such
> >>>>>> functionalities to the userspace, code has been added in this patch
> >>>>>> where during line create call, it checks for new clock type and if
> >>>>>> requested, calls hardware timestamp related API from gpiolib.c.
> >>>>>> During line change event, the HTE subsystem pushes timestamp data
> >>>>>> through callbacks.
> >>>>>>
> >>>>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> >>>>>> Acked-by: Linus Walleij <linus.walleij@linaro.org>
> >>>>>> ---
> >>>>>> Changes in v2:
> >>>>>> - Added hte_dir and static structure hte_ts_desc.
> >>>>>> - Added callbacks which get invoked by HTE when new data is available.
> >>>>>> - Better use of hte_dir and seq from hte_ts_desc.
> >>>>>> - Modified sw debounce function to accommodate hardware timestamping.
> >>>>>>
> >>>>>>  drivers/gpio/gpiolib-cdev.c | 161 ++++++++++++++++++++++++++++++++++--
> >>>>>>  include/uapi/linux/gpio.h   |   1 +
> >>>>>>  2 files changed, 153 insertions(+), 9 deletions(-)
> >>>>>>
> >>>>>> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
> >>>>>> index c7b5446d01fd..1736ad54e3ec 100644
> >>>>>> --- a/drivers/gpio/gpiolib-cdev.c
> >>>>>> +++ b/drivers/gpio/gpiolib-cdev.c
> >>>>>> @@ -464,6 +464,12 @@ struct line {
> >>>>>>  	 * stale value.
> >>>>>>  	 */
> >>>>>>  	unsigned int level;
> >>>>>> +	/*
> >>>>>> +	 * dir will be touched in HTE callbacks hte_ts_cb_t and
> >>>>>> +	 * hte_ts_threaded_cb_t and they are mutually exclusive. This will be
> >>>>>> +	 * unused when HTE is not supported/disabled.
> >>>>>> +	 */
> >>>>>> +	enum hte_dir dir;
> >>>>>>  };
> >>>>>>  
> >>>>> Documentation should be in present tense, so 
> >>>>>
> >>>>> s/will be/is/g
> >>>>>
> >>>>> Same applies to other patches.
> >>>>>
> >>>>> Also
> >>>>>
> >>>>> s/touched/accessed/
> >>>>>
> >>>>> dir is a poor name for the field.  It is the hte edge direction and
> >>>>> effectively the line level, so call it hte_edge_dirn or
> >>>>> hte_edge_direction or hte_level.
> >>>>>
> >>>>> And it is placed in a section of the struct documented as "debouncer specific
> >>>>> fields", but it is not specfic to the debouncer.  Add a "hte specific
> >>>>> fields" section if nothing else is suitable.
> >>>>>
> >>>>>>  /**
> >>>>>> @@ -518,6 +524,7 @@ struct linereq {
> >>>>>>  	 GPIO_V2_LINE_DRIVE_FLAGS | \
> >>>>>>  	 GPIO_V2_LINE_EDGE_FLAGS | \
> >>>>>>  	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
> >>>>>> +	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
> >>>>>>  	 GPIO_V2_LINE_BIAS_FLAGS)
> >>>>>>  
> >>>>>>  static void linereq_put_event(struct linereq *lr,
> >>>>>> @@ -546,6 +553,94 @@ static u64 line_event_timestamp(struct line *line)
> >>>>>>  	return ktime_get_ns();
> >>>>>>  }
> >>>>>>  
> >>>>>> +static hte_return_t process_hw_ts_thread(void *p)
> >>>>>> +{
> >>>>>> +	struct line *line = p;
> >>>>>> +	struct linereq *lr = line->req;
> >>>>>> +	struct gpio_v2_line_event le;
> >>>>>> +	u64 eflags;
> >>>>>> +
> >>>>>> +	memset(&le, 0, sizeof(le));
> >>>>>> +
> >>>>>> +	le.timestamp_ns = line->timestamp_ns;
> >>>>>> +	line->timestamp_ns = 0;
> >>>>>> +
> >>>>> What is the purpose of this zeroing?
> >>>>>
> >>>>>> +	if (line->dir >= HTE_DIR_NOSUPP) {
> >>>>>> +		eflags = READ_ONCE(line->eflags);
> >>>>>> +		if (eflags == GPIO_V2_LINE_FLAG_EDGE_BOTH) {
> >>>>>> +			int level = gpiod_get_value_cansleep(line->desc);
> >>>>>> +
> >>>>>> +			if (level)
> >>>>>> +				/* Emit low-to-high event */
> >>>>>> +				le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> >>>>>> +			else
> >>>>>> +				/* Emit high-to-low event */
> >>>>>> +				le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >>>>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_RISING) {
> >>>>>> +			/* Emit low-to-high event */
> >>>>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> >>>>>> +		} else if (eflags == GPIO_V2_LINE_FLAG_EDGE_FALLING) {
> >>>>>> +			/* Emit high-to-low event */
> >>>>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >>>>>> +		} else {
> >>>>>> +			return HTE_CB_ERROR;
> >>>>>> +		}
> >>>>>> +	} else {
> >>>>>> +		if (line->dir == HTE_RISING_EDGE_TS)
> >>>>>> +			le.id = GPIO_V2_LINE_EVENT_RISING_EDGE;
> >>>>>> +		else
> >>>>>> +			le.id = GPIO_V2_LINE_EVENT_FALLING_EDGE;
> >>>>>> +	}
> >>>>> The mapping from line->dir to le.id needs to take into account the active
> >>>>> low setting for the line.
> >>>>>
> >>>>> And it might be simpler if the hte_ts_data provided the level, equivalent
> >>>>> to gpiod_get_raw_value_cansleep(), rather than an edge direction, so you
> >>>>> can provide a common helper to determine the edge given the raw level.
> >>>> (So from the level determine the edge?) that sound right specially when
> >>>>
> >>>> HTE provider has capability to record the edge in that case why bother
> >>>>
> >>>> getting the level and determine edge?
> >>>>
> >>>> Calculating the edge from the level makes sense when hte provider does not
> >>>>
> >>>> have that feature and that is what if (line->dir >= HTE_DIR_NOSUPP) does.
> >>>>
> >>> As asked in the review of patch 02, do you have an example of hardware that
> >>> reports an edge direction rather than NOSUPP?
> >> No...
> > So you are adding an interface that nothing will currently use.
> > Are there plans for hardware that will report the edge, and you are
> > laying the groundwork here?
> 
> Adding here for the general case should there be provider
> 
> available with such feature.
> 

Then you are adding dead code, and you should remove that aspect of your
interface until you have hardware that does support it.

Cheers,
Kent.

Re: [RFC v3 09/12] gpiolib: cdev: Add hardware timestamp clock type

[Dipen Patel]

Hi,


On 12/1/21 4:53 PM, Kent Gibson wrote:
> On Wed, Dec 01, 2021 at 10:01:46AM -0800, Dipen Patel wrote:
>> Hi,
>>
>>
>> On 12/1/21 9:16 AM, Kent Gibson wrote:
>>> On Tue, Nov 30, 2021 at 07:29:20PM -0800, Dipen Patel wrote:
>>>> Hi,
>>>>
>>>> On 11/25/21 5:31 PM, Kent Gibson wrote:
>>>>> On Tue, Nov 23, 2021 at 11:30:36AM -0800, Dipen Patel wrote:
>>>>>> This patch adds new clock type for the GPIO controller which can
>>>>>> timestamp gpio lines in realtime using hardware means. To expose such
>>>>>> functionalities to the userspace, code has been added in this patch
>>>>>> where during line create call, it checks for new clock type and if
>>>>>> requested, calls hardware timestamp related API from gpiolib.c.
>>>>>> During line change event, the HTE subsystem pushes timestamp data
>>>>>> through callbacks.
>>>>>>
>>>>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>>>>>> Acked-by: Linus Walleij <linus.walleij@linaro.org>
>>>>>> ---
>>>>>> Changes in v2:
>>>>>> - Added hte_dir and static structure hte_ts_desc.
>>>>>> - Added callbacks which get invoked by HTE when new data is available.
>>>>>> - Better use of hte_dir and seq from hte_ts_desc.
>>>>>> - Modified sw debounce function to accommodate hardware timestamping.
>>>>>>
>>>>>>  drivers/gpio/gpiolib-cdev.c | 161 ++++++++++++++++++++++++++++++++++--
>>>>>>  include/uapi/linux/gpio.h   |   1 +
>>>>>>  2 files changed, 153 insertions(+), 9 deletions(-)
>>>>>>
>>>>>> [snip]
> The code here has to deal with the general case, not just the one example
> driver you have provided.  So in general there COULD be gaps in the
> ts->seq, right?
>
> I do see that using the ts-seq for sw debounced lines is problematic
> though. The debouncer itself will be discarding hte events, but that
> shouldn't be considered a lost event to the user.  You could track
> how many events are discarded by the debouncer and subtract those from
> the sequence numbers reported to userspace?

This could be little complicated, especially for "hybrid" scenario where

cdev debouncer receives partial events to discard and rest is dropped in hw fifo

in hte core. For example, if there were 5 events to discard before debounce period,

cdev receives only 3 events to discard and rest 2 is dropped in hte core. In this

case, when debounce period ends and work func gets executed, it will take 3rd event

as that was the last seen in cdev debouncer. This makes both ts and seq unstable and

out of sync. In the absence of actual hardware which can drop, its probably hard to

simulate and implement complete solution to tackle this.


>
>