[PATCH 0/9] serial: Add Tegra Combined UART driver

[PATCH 0/9] serial: Add Tegra Combined UART driver

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

Hi everyone,

this is a reworked version of Mikko's earlier proposal[0]. I've reworked
the TCU driver itself so that it relies less on global variables as well
as added a Kconfig option to allow the console support to be selected. I
also fixed a couple of issues that manifested themselves as I was moving
towards the IRQ driven mode (TCU was passing a pointer to a local
variable which was getting stored in the mailbox's ring buffer and the
data pointed at was becoming stale by the time the mailbox got around to
dequeue it).

The biggest bulk of the changes is in the mailbox driver. This series
addresses all of Jassi's comments from back at the time. One notable
additional change is that shared mailboxes are now interrupt driven,
which removes the need for polling mode.

Unfortunately there is still an issue because the TCU uses the mailbox
in atomic context for both TTY and console modes, so we get a sleeping-
while-atomic BUG when using mbox_client->tx_block = true in order to
rate-limit mbox_send_message(). In order to work around this, I added a
mechanism to mbox_send_message() that will allow blocking from atomic
context if the mailbox controller implements the new ->flush() callback.
For Tegra HSP shared mailboxes this is done by spinning on the shared
mailbox register until the receiver has marked the mailbox as empty. I
have been running this locally for a couple of days now and it works
perfectly.

Furthermore this series incorporates Mikko's work in progress on
splitting up the mailbox controllers and allowing multiple controllers
to match on the same device tree node during mbox_request_channel().

Last but not least there are no build-time dependencies between the
mailbox and serial drivers, so I think the easiest way to merge this is
if Jassi picks up patches 1-5, Greg takes patches 6 & 7 and I pick up
patches 8-9 into the Tegra tree.

Thanks,
Thierry

[0]: https://lore.kernel.org/patchwork/project/lkml/list/?series=357641

Mikko Perttunen (5):
  mailbox: Allow multiple controllers per device
  dt-bindings: tegra186-hsp: Add shared interrupts
  dt-bindings: serial: Add bindings for nvidia,tegra194-tcu
  arm64: tegra: Add nodes for TCU on Tegra194
  arm64: tegra: Mark TCU as primary serial port on Tegra194 P2888

Thierry Reding (4):
  mailbox: Support blocking transfers in atomic context
  mailbox: tegra-hsp: Add support for shared mailboxes
  mailbox: tegra-hsp: Add suspend/resume support
  serial: Add Tegra Combined UART driver

 .../bindings/mailbox/nvidia,tegra186-hsp.txt  |   3 +
 .../bindings/serial/nvidia,tegra194-tcu.txt   |  35 ++
 .../arm64/boot/dts/nvidia/tegra194-p2888.dtsi |   2 +-
 arch/arm64/boot/dts/nvidia/tegra194.dtsi      |  38 +-
 drivers/mailbox/mailbox.c                     |  11 +-
 drivers/mailbox/tegra-hsp.c                   | 495 +++++++++++++++---
 drivers/tty/serial/Kconfig                    |  22 +
 drivers/tty/serial/Makefile                   |   1 +
 drivers/tty/serial/tegra-tcu.c                | 299 +++++++++++
 include/linux/mailbox_controller.h            |   4 +
 include/uapi/linux/serial_core.h              |   3 +
 11 files changed, 847 insertions(+), 66 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/serial/nvidia,tegra194-tcu.txt
 create mode 100644 drivers/tty/serial/tegra-tcu.c

-- 
2.19.1

[PATCH 1/9] mailbox: Support blocking transfers in atomic context

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

The mailbox framework supports blocking transfers via completions for
clients that can sleep. In order to support blocking transfers in cases
where the transmission is not permitted to sleep, add a new ->flush()
callback that controller drivers can implement to busy loop until the
transmission has been completed. This will automatically be called when
available and interrupts are disabled for clients that request blocking
transfers.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 drivers/mailbox/mailbox.c          | 8 ++++++++
 include/linux/mailbox_controller.h | 4 ++++
 2 files changed, 12 insertions(+)

diff --git a/drivers/mailbox/mailbox.c b/drivers/mailbox/mailbox.c
index 674b35f402f5..0eaf21259874 100644
--- a/drivers/mailbox/mailbox.c
+++ b/drivers/mailbox/mailbox.c
@@ -267,6 +267,14 @@ int mbox_send_message(struct mbox_chan *chan, void *mssg)
 		unsigned long wait;
 		int ret;
 
+		if (irqs_disabled() && chan->mbox->ops->flush) {
+			ret = chan->mbox->ops->flush(chan, chan->cl->tx_tout);
+			if (ret < 0)
+				tx_tick(chan, ret);
+
+			return ret;
+		}
+
 		if (!chan->cl->tx_tout) /* wait forever */
 			wait = msecs_to_jiffies(3600000);
 		else
diff --git a/include/linux/mailbox_controller.h b/include/linux/mailbox_controller.h
index 74deadb42d76..2a07d93f781a 100644
--- a/include/linux/mailbox_controller.h
+++ b/include/linux/mailbox_controller.h
@@ -24,6 +24,9 @@ struct mbox_chan;
  *		transmission of data is reported by the controller via
  *		mbox_chan_txdone (if it has some TX ACK irq). It must not
  *		sleep.
+ * @flush:	Called when a client requests transmissions to be blocking but
+ *		the context doesn't allow sleeping. Typically the controller
+ *		will implement a busy loop waiting for the data to flush out.
  * @startup:	Called when a client requests the chan. The controller
  *		could ask clients for additional parameters of communication
  *		to be provided via client's chan_data. This call may
@@ -46,6 +49,7 @@ struct mbox_chan;
  */
 struct mbox_chan_ops {
 	int (*send_data)(struct mbox_chan *chan, void *data);
+	int (*flush)(struct mbox_chan *chan, unsigned long timeout);
 	int (*startup)(struct mbox_chan *chan);
 	void (*shutdown)(struct mbox_chan *chan);
 	bool (*last_tx_done)(struct mbox_chan *chan);
-- 
2.19.1

[PATCH 2/9] mailbox: Allow multiple controllers per device

[Thierry Reding]

From: Mikko Perttunen <mperttunen@nvidia.com>

Look through the whole controller list when mapping device tree
phandles to controllers instead of stopping at the first one.

Each controller is intended to only contain one kind of mailbox,
but some devices (like Tegra HSP) implement multiple kinds and use
the same device tree node for all of them. As such, we need to allow
multiple mbox_controllers per device tree node.

Signed-off-by: Mikko Perttunen <mperttunen@nvidia.com>
---
 drivers/mailbox/mailbox.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/mailbox/mailbox.c b/drivers/mailbox/mailbox.c
index 0eaf21259874..f34820bcf33c 100644
--- a/drivers/mailbox/mailbox.c
+++ b/drivers/mailbox/mailbox.c
@@ -335,7 +335,8 @@ struct mbox_chan *mbox_request_channel(struct mbox_client *cl, int index)
 	list_for_each_entry(mbox, &mbox_cons, node)
 		if (mbox->dev->of_node == spec.np) {
 			chan = mbox->of_xlate(mbox, &spec);
-			break;
+			if (!IS_ERR(chan))
+				break;
 		}
 
 	of_node_put(spec.np);
-- 
2.19.1

[PATCH 3/9] dt-bindings: tegra186-hsp: Add shared interrupts

[Thierry Reding]

From: Mikko Perttunen <mperttunen@nvidia.com>

HSP interrupts can be routed through exposed "shared interrupts". These
interrupts can be mapped to various internal interrupt lines. Add
interrupt properties for shared interrupts to the tegra186-hsp device
tree bindings.

Signed-off-by: Mikko Perttunen <mperttunen@nvidia.com>
Acked-by: Jon Hunter <jonathanh@nvidia.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Acked-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 .../devicetree/bindings/mailbox/nvidia,tegra186-hsp.txt        | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Documentation/devicetree/bindings/mailbox/nvidia,tegra186-hsp.txt b/Documentation/devicetree/bindings/mailbox/nvidia,tegra186-hsp.txt
index b99d25fc2f26..620c249363ca 100644
--- a/Documentation/devicetree/bindings/mailbox/nvidia,tegra186-hsp.txt
+++ b/Documentation/devicetree/bindings/mailbox/nvidia,tegra186-hsp.txt
@@ -15,12 +15,15 @@ Required properties:
     Array of strings.
     one of:
     - "nvidia,tegra186-hsp"
+    - "nvidia,tegra194-hsp", "nvidia,tegra186-hsp"
 - reg : Offset and length of the register set for the device.
 - interrupt-names
     Array of strings.
     Contains a list of names for the interrupts described by the interrupt
     property. May contain the following entries, in any order:
     - "doorbell"
+    - "sharedN", where 'N' is a number from zero up to the number of
+      external interrupts supported by the HSP instance minus one.
     Users of this binding MUST look up entries in the interrupt property
     by name, using this interrupt-names property to do so.
 - interrupts
-- 
2.19.1

[PATCH 4/9] mailbox: tegra-hsp: Add support for shared mailboxes

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

The Tegra HSP block supports 'shared mailboxes' that are simple 32-bit
registers consisting of a FULL bit in MSB position and 31 bits of data.
The hardware can be configured to trigger interrupts when a mailbox
is empty or full. Add support for these shared mailboxes to the HSP
driver.

The initial use for the mailboxes is the Tegra Combined UART. For this
purpose, we use interrupts to receive data, and spinning to wait for
the transmit mailbox to be emptied to minimize unnecessary overhead.

Based on work by Mikko Perttunen <mperttunen@nvidia.com>.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 drivers/mailbox/tegra-hsp.c | 476 +++++++++++++++++++++++++++++++-----
 1 file changed, 415 insertions(+), 61 deletions(-)

diff --git a/drivers/mailbox/tegra-hsp.c b/drivers/mailbox/tegra-hsp.c
index 0cde356c11ab..d070c8e38375 100644
--- a/drivers/mailbox/tegra-hsp.c
+++ b/drivers/mailbox/tegra-hsp.c
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016, NVIDIA CORPORATION.  All rights reserved.
+ * Copyright (c) 2016-2018, NVIDIA CORPORATION.  All rights reserved.
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms and conditions of the GNU General Public License,
@@ -11,6 +11,7 @@
  * more details.
  */
 
+#include <linux/delay.h>
 #include <linux/interrupt.h>
 #include <linux/io.h>
 #include <linux/mailbox_controller.h>
@@ -21,6 +22,17 @@
 
 #include <dt-bindings/mailbox/tegra186-hsp.h>
 
+#include "mailbox.h"
+
+#define HSP_INT_IE(x)		(0x100 + ((x) * 4))
+#define HSP_INT_IV		0x300
+#define HSP_INT_IR		0x304
+
+#define HSP_INT_EMPTY_SHIFT	0
+#define HSP_INT_EMPTY_MASK	0xff
+#define HSP_INT_FULL_SHIFT	8
+#define HSP_INT_FULL_MASK	0xff
+
 #define HSP_INT_DIMENSIONING	0x380
 #define HSP_nSM_SHIFT		0
 #define HSP_nSS_SHIFT		4
@@ -34,6 +46,11 @@
 #define HSP_DB_RAW	0x8
 #define HSP_DB_PENDING	0xc
 
+#define HSP_SM_SHRD_MBOX	0x0
+#define HSP_SM_SHRD_MBOX_FULL	BIT(31)
+#define HSP_SM_SHRD_MBOX_FULL_INT_IE	0x04
+#define HSP_SM_SHRD_MBOX_EMPTY_INT_IE	0x08
+
 #define HSP_DB_CCPLEX		1
 #define HSP_DB_BPMP		3
 #define HSP_DB_MAX		7
@@ -55,6 +72,12 @@ struct tegra_hsp_doorbell {
 	unsigned int index;
 };
 
+struct tegra_hsp_mailbox {
+	struct tegra_hsp_channel channel;
+	unsigned int index;
+	bool sending;
+};
+
 struct tegra_hsp_db_map {
 	const char *name;
 	unsigned int master;
@@ -66,10 +89,13 @@ struct tegra_hsp_soc {
 };
 
 struct tegra_hsp {
+	struct device *dev;
 	const struct tegra_hsp_soc *soc;
-	struct mbox_controller mbox;
+	struct mbox_controller mbox_db;
+	struct mbox_controller mbox_sm;
 	void __iomem *regs;
-	unsigned int irq;
+	unsigned int doorbell_irq;
+	unsigned int *shared_irqs;
 	unsigned int num_sm;
 	unsigned int num_as;
 	unsigned int num_ss;
@@ -78,14 +104,9 @@ struct tegra_hsp {
 	spinlock_t lock;
 
 	struct list_head doorbells;
+	struct tegra_hsp_mailbox *mailboxes;
 };
 
-static inline struct tegra_hsp *
-to_tegra_hsp(struct mbox_controller *mbox)
-{
-	return container_of(mbox, struct tegra_hsp, mbox);
-}
-
 static inline u32 tegra_hsp_readl(struct tegra_hsp *hsp, unsigned int offset)
 {
 	return readl(hsp->regs + offset);
@@ -158,7 +179,7 @@ static irqreturn_t tegra_hsp_doorbell_irq(int irq, void *data)
 
 	spin_lock(&hsp->lock);
 
-	for_each_set_bit(master, &value, hsp->mbox.num_chans) {
+	for_each_set_bit(master, &value, hsp->mbox_db.num_chans) {
 		struct tegra_hsp_doorbell *db;
 
 		db = __tegra_hsp_doorbell_get(hsp, master);
@@ -182,6 +203,84 @@ static irqreturn_t tegra_hsp_doorbell_irq(int irq, void *data)
 	return IRQ_HANDLED;
 }
 
+static irqreturn_t tegra_hsp_shared_full_irq(int irq, void *data)
+{
+	struct tegra_hsp *hsp = data;
+	unsigned long bit, mask;
+	u32 status, value;
+	void *msg;
+
+	status = tegra_hsp_readl(hsp, HSP_INT_IR);
+
+	/* only interested in FULL interrupts */
+	mask = (status >> HSP_INT_FULL_SHIFT) & HSP_INT_FULL_MASK;
+
+	if (!mask)
+		return IRQ_NONE;
+
+	for_each_set_bit(bit, &mask, hsp->num_sm) {
+		struct tegra_hsp_mailbox *mb = &hsp->mailboxes[bit];
+
+		if (!mb->sending) {
+			value = tegra_hsp_channel_readl(&mb->channel,
+							HSP_SM_SHRD_MBOX);
+			value &= ~HSP_SM_SHRD_MBOX_FULL;
+			msg = (void *)(unsigned long)value;
+			mbox_chan_received_data(mb->channel.chan, msg);
+
+			/*
+			 * Need to clear all bits here since some producers,
+			 * such as TCU, depend on fields in the register
+			 * getting cleared by the consumer.
+			 *
+			 * The mailbox API doesn't give the consumers a way
+			 * of doing that explicitly, so we have to make sure
+			 * we cover all possible cases.
+			 */
+			tegra_hsp_channel_writel(&mb->channel, 0x0,
+						 HSP_SM_SHRD_MBOX);
+		}
+	}
+
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t tegra_hsp_shared_empty_irq(int irq, void *data)
+{
+	struct tegra_hsp *hsp = data;
+	unsigned long bit, mask;
+	u32 status, value;
+
+	status = tegra_hsp_readl(hsp, HSP_INT_IR);
+
+	/* only interested in EMPTY interrupts */
+	mask = (status >> HSP_INT_EMPTY_SHIFT) & HSP_INT_EMPTY_MASK;
+
+	if (!mask)
+		return IRQ_NONE;
+
+	for_each_set_bit(bit, &mask, hsp->num_sm) {
+		struct tegra_hsp_mailbox *mb = &hsp->mailboxes[bit];
+
+		if (mb->sending) {
+			/*
+			 * Disable EMPTY interrupts until data is sent
+			 * with the next message. These interrupts are
+			 * level-triggered, so if we kept them enabled
+			 * they would constantly trigger until we next
+			 * write data into the message.
+			 */
+			value = tegra_hsp_readl(hsp, HSP_INT_IE(1));
+			value &= ~BIT(HSP_INT_EMPTY_SHIFT + mb->index);
+			tegra_hsp_writel(hsp, value, HSP_INT_IE(1));
+
+			mbox_chan_txdone(mb->channel.chan, 0);
+		}
+	}
+
+	return IRQ_HANDLED;
+}
+
 static struct tegra_hsp_channel *
 tegra_hsp_doorbell_create(struct tegra_hsp *hsp, const char *name,
 			  unsigned int master, unsigned int index)
@@ -194,7 +293,7 @@ tegra_hsp_doorbell_create(struct tegra_hsp *hsp, const char *name,
 	if (!db)
 		return ERR_PTR(-ENOMEM);
 
-	offset = (1 + (hsp->num_sm / 2) + hsp->num_ss + hsp->num_as) << 16;
+	offset = (1 + (hsp->num_sm / 2) + hsp->num_ss + hsp->num_as) * SZ_64K;
 	offset += index * 0x100;
 
 	db->channel.regs = hsp->regs + offset;
@@ -235,8 +334,8 @@ static int tegra_hsp_doorbell_startup(struct mbox_chan *chan)
 	unsigned long flags;
 	u32 value;
 
-	if (db->master >= hsp->mbox.num_chans) {
-		dev_err(hsp->mbox.dev,
+	if (db->master >= chan->mbox->num_chans) {
+		dev_err(chan->mbox->dev,
 			"invalid master ID %u for HSP channel\n",
 			db->master);
 		return -EINVAL;
@@ -281,46 +380,147 @@ static void tegra_hsp_doorbell_shutdown(struct mbox_chan *chan)
 	spin_unlock_irqrestore(&hsp->lock, flags);
 }
 
-static const struct mbox_chan_ops tegra_hsp_doorbell_ops = {
+static const struct mbox_chan_ops tegra_hsp_db_ops = {
 	.send_data = tegra_hsp_doorbell_send_data,
 	.startup = tegra_hsp_doorbell_startup,
 	.shutdown = tegra_hsp_doorbell_shutdown,
 };
 
-static struct mbox_chan *of_tegra_hsp_xlate(struct mbox_controller *mbox,
+static int tegra_hsp_mailbox_send_data(struct mbox_chan *chan, void *data)
+{
+	struct tegra_hsp_mailbox *mb = chan->con_priv;
+	struct tegra_hsp *hsp = mb->channel.hsp;
+	u32 value;
+
+	mb->sending = true;
+
+	/* copy data and mark mailbox full */
+	value = (u32)(unsigned long)data;
+	value |= HSP_SM_SHRD_MBOX_FULL;
+
+	tegra_hsp_channel_writel(&mb->channel, value, HSP_SM_SHRD_MBOX);
+
+	if (!irqs_disabled()) {
+		/* enable EMPTY interrupt for the shared mailbox */
+		value = tegra_hsp_readl(hsp, HSP_INT_IE(1));
+		value |= BIT(HSP_INT_EMPTY_SHIFT + mb->index);
+		tegra_hsp_writel(hsp, value, HSP_INT_IE(1));
+	}
+
+	return 0;
+}
+
+static int tegra_hsp_mailbox_flush(struct mbox_chan *chan,
+				   unsigned long timeout)
+{
+	struct tegra_hsp_mailbox *mb = chan->con_priv;
+	struct tegra_hsp_channel *ch = &mb->channel;
+	u32 value;
+
+	timeout = jiffies + msecs_to_jiffies(timeout);
+
+	while (time_before(jiffies, timeout)) {
+		value = tegra_hsp_channel_readl(ch, HSP_SM_SHRD_MBOX);
+		if ((value & HSP_SM_SHRD_MBOX_FULL) == 0) {
+			mbox_chan_txdone(chan, 0);
+			return 0;
+		}
+
+		udelay(1);
+	}
+
+	return -ETIME;
+}
+
+static int tegra_hsp_mailbox_startup(struct mbox_chan *chan)
+{
+	struct tegra_hsp_mailbox *mb = chan->con_priv;
+	struct tegra_hsp_channel *ch = &mb->channel;
+	struct tegra_hsp *hsp = mb->channel.hsp;
+	u32 value;
+
+	chan->txdone_method = TXDONE_BY_IRQ;
+
+	/*
+	 * Shared mailboxes start out as consumers by default. FULL interrupts
+	 * are coalesced at shared interrupt 0, while EMPTY interrupts will be
+	 * coalesced at shared interrupt 1.
+	 *
+	 * Keep EMPTY interrupts disabled at startup and only enable them when
+	 * the mailbox is actually full. This is required because the FULL and
+	 * EMPTY interrupts are level-triggered, so keeping EMPTY interrupts
+	 * enabled all the time would cause an interrupt storm while mailboxes
+	 * are idle.
+	 */
+
+	value = tegra_hsp_readl(hsp, HSP_INT_IE(0));
+	value |= BIT(HSP_INT_FULL_SHIFT + mb->index);
+	tegra_hsp_writel(hsp, value, HSP_INT_IE(0));
+
+	value = tegra_hsp_readl(hsp, HSP_INT_IE(1));
+	value &= ~BIT(HSP_INT_EMPTY_SHIFT + mb->index);
+	tegra_hsp_writel(hsp, value, HSP_INT_IE(1));
+
+	tegra_hsp_channel_writel(ch, 0x1, HSP_SM_SHRD_MBOX_FULL_INT_IE);
+	tegra_hsp_channel_writel(ch, 0x1, HSP_SM_SHRD_MBOX_EMPTY_INT_IE);
+
+	return 0;
+}
+
+static void tegra_hsp_mailbox_shutdown(struct mbox_chan *chan)
+{
+	struct tegra_hsp_mailbox *mb = chan->con_priv;
+	struct tegra_hsp_channel *ch = &mb->channel;
+	struct tegra_hsp *hsp = mb->channel.hsp;
+	u32 value;
+
+	tegra_hsp_channel_writel(ch, 0x0, HSP_SM_SHRD_MBOX_EMPTY_INT_IE);
+	tegra_hsp_channel_writel(ch, 0x0, HSP_SM_SHRD_MBOX_FULL_INT_IE);
+
+	value = tegra_hsp_readl(hsp, HSP_INT_IE(1));
+	value &= ~BIT(HSP_INT_EMPTY_SHIFT + mb->index);
+	tegra_hsp_writel(hsp, value, HSP_INT_IE(1));
+
+	value = tegra_hsp_readl(hsp, HSP_INT_IE(0));
+	value &= ~BIT(HSP_INT_FULL_SHIFT + mb->index);
+	tegra_hsp_writel(hsp, value, HSP_INT_IE(0));
+}
+
+static const struct mbox_chan_ops tegra_hsp_sm_ops = {
+	.send_data = tegra_hsp_mailbox_send_data,
+	.flush = tegra_hsp_mailbox_flush,
+	.startup = tegra_hsp_mailbox_startup,
+	.shutdown = tegra_hsp_mailbox_shutdown,
+};
+
+static struct mbox_chan *tegra_hsp_db_xlate(struct mbox_controller *mbox,
 					    const struct of_phandle_args *args)
 {
+	struct tegra_hsp *hsp = container_of(mbox, struct tegra_hsp, mbox_db);
+	unsigned int type = args->args[0], master = args->args[1];
 	struct tegra_hsp_channel *channel = ERR_PTR(-ENODEV);
-	struct tegra_hsp *hsp = to_tegra_hsp(mbox);
-	unsigned int type = args->args[0];
-	unsigned int master = args->args[1];
 	struct tegra_hsp_doorbell *db;
 	struct mbox_chan *chan;
 	unsigned long flags;
 	unsigned int i;
 
-	switch (type) {
-	case TEGRA_HSP_MBOX_TYPE_DB:
-		db = tegra_hsp_doorbell_get(hsp, master);
-		if (db)
-			channel = &db->channel;
+	if (type != TEGRA_HSP_MBOX_TYPE_DB || !hsp->doorbell_irq)
+		return ERR_PTR(-ENODEV);
 
-		break;
-
-	default:
-		break;
-	}
+	db = tegra_hsp_doorbell_get(hsp, master);
+	if (db)
+		channel = &db->channel;
 
 	if (IS_ERR(channel))
 		return ERR_CAST(channel);
 
 	spin_lock_irqsave(&hsp->lock, flags);
 
-	for (i = 0; i < hsp->mbox.num_chans; i++) {
-		chan = &hsp->mbox.chans[i];
+	for (i = 0; i < mbox->num_chans; i++) {
+		chan = &mbox->chans[i];
 		if (!chan->con_priv) {
-			chan->con_priv = channel;
 			channel->chan = chan;
+			chan->con_priv = db;
 			break;
 		}
 
@@ -332,6 +532,19 @@ static struct mbox_chan *of_tegra_hsp_xlate(struct mbox_controller *mbox,
 	return chan ?: ERR_PTR(-EBUSY);
 }
 
+static struct mbox_chan *tegra_hsp_sm_xlate(struct mbox_controller *mbox,
+					    const struct of_phandle_args *args)
+{
+	struct tegra_hsp *hsp = container_of(mbox, struct tegra_hsp, mbox_sm);
+	unsigned int type = args->args[0], index = args->args[1];
+
+	if (type != TEGRA_HSP_MBOX_TYPE_SM || !hsp->shared_irqs ||
+	    index >= hsp->num_sm)
+		return ERR_PTR(-ENODEV);
+
+	return hsp->mailboxes[index].channel.chan;
+}
+
 static void tegra_hsp_remove_doorbells(struct tegra_hsp *hsp)
 {
 	struct tegra_hsp_doorbell *db, *tmp;
@@ -364,10 +577,72 @@ static int tegra_hsp_add_doorbells(struct tegra_hsp *hsp)
 	return 0;
 }
 
+static int tegra_hsp_add_mailboxes(struct tegra_hsp *hsp, struct device *dev)
+{
+	int i;
+
+	hsp->mailboxes = devm_kcalloc(dev, hsp->num_sm, sizeof(*hsp->mailboxes),
+				      GFP_KERNEL);
+	if (!hsp->mailboxes)
+		return -ENOMEM;
+
+	for (i = 0; i < hsp->num_sm; i++) {
+		struct tegra_hsp_mailbox *mb = &hsp->mailboxes[i];
+
+		mb->index = i;
+		mb->sending = false;
+
+		mb->channel.hsp = hsp;
+		mb->channel.regs = hsp->regs + SZ_64K + i * SZ_32K;
+		mb->channel.chan = &hsp->mbox_sm.chans[i];
+		mb->channel.chan->con_priv = mb;
+	}
+
+	return 0;
+}
+
+static int tegra_hsp_request_shared_irqs(struct tegra_hsp *hsp)
+{
+	int err;
+
+	if (hsp->shared_irqs[0] > 0) {
+		err = devm_request_irq(hsp->dev, hsp->shared_irqs[0],
+				       tegra_hsp_shared_full_irq, 0,
+				       dev_name(hsp->dev), hsp);
+		if (err < 0) {
+			dev_err(hsp->dev,
+				"failed to request full interrupt: %d\n",
+				err);
+			return err;
+		}
+
+		dev_dbg(hsp->dev, "full interrupt requested: %u\n",
+			hsp->shared_irqs[0]);
+	}
+
+	if (hsp->shared_irqs[1] > 0) {
+		err = devm_request_irq(hsp->dev, hsp->shared_irqs[1],
+				       tegra_hsp_shared_empty_irq, 0,
+				       dev_name(hsp->dev), hsp);
+		if (err < 0) {
+			dev_err(hsp->dev,
+				"failed to request empty interrupt: %d\n",
+				err);
+			return err;
+		}
+
+		dev_dbg(hsp->dev, "empty interrupt requested: %u\n",
+			hsp->shared_irqs[1]);
+	}
+
+	return 0;
+}
+
 static int tegra_hsp_probe(struct platform_device *pdev)
 {
 	struct tegra_hsp *hsp;
 	struct resource *res;
+	unsigned int i;
 	u32 value;
 	int err;
 
@@ -375,6 +650,7 @@ static int tegra_hsp_probe(struct platform_device *pdev)
 	if (!hsp)
 		return -ENOMEM;
 
+	hsp->dev = &pdev->dev;
 	hsp->soc = of_device_get_match_data(&pdev->dev);
 	INIT_LIST_HEAD(&hsp->doorbells);
 	spin_lock_init(&hsp->lock);
@@ -392,58 +668,136 @@ static int tegra_hsp_probe(struct platform_device *pdev)
 	hsp->num_si = (value >> HSP_nSI_SHIFT) & HSP_nINT_MASK;
 
 	err = platform_get_irq_byname(pdev, "doorbell");
-	if (err < 0) {
-		dev_err(&pdev->dev, "failed to get doorbell IRQ: %d\n", err);
-		return err;
+	if (err >= 0)
+		hsp->doorbell_irq = err;
+
+	if (hsp->num_si > 0) {
+		unsigned int count = 0;
+
+		hsp->shared_irqs = devm_kcalloc(&pdev->dev, hsp->num_si,
+						sizeof(*hsp->shared_irqs),
+						GFP_KERNEL);
+		if (!hsp->shared_irqs)
+			return -ENOMEM;
+
+		for (i = 0; i < hsp->num_si; i++) {
+			char *name;
+
+			name = kasprintf(GFP_KERNEL, "shared%u", i);
+			if (!name)
+				return -ENOMEM;
+
+			err = platform_get_irq_byname(pdev, name);
+			if (err >= 0) {
+				hsp->shared_irqs[i] = err;
+				count++;
+			}
+
+			kfree(name);
+		}
+
+		if (count == 0) {
+			devm_kfree(&pdev->dev, hsp->shared_irqs);
+			hsp->shared_irqs = NULL;
+		}
 	}
 
-	hsp->irq = err;
+	/* setup the doorbell controller */
+	hsp->mbox_db.of_xlate = tegra_hsp_db_xlate;
+	hsp->mbox_db.num_chans = 32;
+	hsp->mbox_db.dev = &pdev->dev;
+	hsp->mbox_db.ops = &tegra_hsp_db_ops;
 
-	hsp->mbox.of_xlate = of_tegra_hsp_xlate;
-	hsp->mbox.num_chans = 32;
-	hsp->mbox.dev = &pdev->dev;
-	hsp->mbox.txdone_irq = false;
-	hsp->mbox.txdone_poll = false;
-	hsp->mbox.ops = &tegra_hsp_doorbell_ops;
+	hsp->mbox_db.chans = devm_kcalloc(&pdev->dev, hsp->mbox_db.num_chans,
+					  sizeof(*hsp->mbox_db.chans),
+					  GFP_KERNEL);
+	if (!hsp->mbox_db.chans)
+		return -ENOMEM;
 
-	hsp->mbox.chans = devm_kcalloc(&pdev->dev, hsp->mbox.num_chans,
-					sizeof(*hsp->mbox.chans),
-					GFP_KERNEL);
-	if (!hsp->mbox.chans)
+	if (hsp->doorbell_irq) {
+		err = tegra_hsp_add_doorbells(hsp);
+		if (err < 0) {
+			dev_err(&pdev->dev, "failed to add doorbells: %d\n",
+			        err);
+			return err;
+		}
+	}
+
+	err = mbox_controller_register(&hsp->mbox_db);
+	if (err < 0) {
+		dev_err(&pdev->dev, "failed to register doorbell mailbox: %d\n", err);
+		goto remove_doorbells;
+	}
+
+	/* setup the shared mailbox controller */
+	hsp->mbox_sm.of_xlate = tegra_hsp_sm_xlate;
+	hsp->mbox_sm.num_chans = hsp->num_sm;
+	hsp->mbox_sm.dev = &pdev->dev;
+	hsp->mbox_sm.ops = &tegra_hsp_sm_ops;
+
+	hsp->mbox_sm.chans = devm_kcalloc(&pdev->dev, hsp->mbox_sm.num_chans,
+					  sizeof(*hsp->mbox_sm.chans),
+					  GFP_KERNEL);
+	if (!hsp->mbox_sm.chans)
 		return -ENOMEM;
 
-	err = tegra_hsp_add_doorbells(hsp);
+	if (hsp->shared_irqs) {
+		err = tegra_hsp_add_mailboxes(hsp, &pdev->dev);
+		if (err < 0) {
+			dev_err(&pdev->dev, "failed to add mailboxes: %d\n",
+			        err);
+			goto unregister_mbox_db;
+		}
+	}
+
+	err = mbox_controller_register(&hsp->mbox_sm);
 	if (err < 0) {
-		dev_err(&pdev->dev, "failed to add doorbells: %d\n", err);
-		return err;
+		dev_err(&pdev->dev, "failed to register shared mailbox: %d\n", err);
+		goto unregister_mbox_db;
 	}
 
 	platform_set_drvdata(pdev, hsp);
 
-	err = mbox_controller_register(&hsp->mbox);
-	if (err) {
-		dev_err(&pdev->dev, "failed to register mailbox: %d\n", err);
-		tegra_hsp_remove_doorbells(hsp);
-		return err;
+	if (hsp->doorbell_irq) {
+		err = devm_request_irq(&pdev->dev, hsp->doorbell_irq,
+				       tegra_hsp_doorbell_irq, IRQF_NO_SUSPEND,
+				       dev_name(&pdev->dev), hsp);
+		if (err < 0) {
+			dev_err(&pdev->dev,
+			        "failed to request doorbell IRQ#%u: %d\n",
+				hsp->doorbell_irq, err);
+			goto unregister_mbox_sm;
+		}
 	}
 
-	err = devm_request_irq(&pdev->dev, hsp->irq, tegra_hsp_doorbell_irq,
-			       IRQF_NO_SUSPEND, dev_name(&pdev->dev), hsp);
-	if (err < 0) {
-		dev_err(&pdev->dev, "failed to request IRQ#%u: %d\n",
-			hsp->irq, err);
-		return err;
+	if (hsp->shared_irqs) {
+		err = tegra_hsp_request_shared_irqs(hsp);
+		if (err < 0)
+			goto unregister_mbox_sm;
 	}
 
 	return 0;
+
+unregister_mbox_sm:
+	mbox_controller_unregister(&hsp->mbox_sm);
+unregister_mbox_db:
+	mbox_controller_unregister(&hsp->mbox_db);
+remove_doorbells:
+	if (hsp->doorbell_irq)
+		tegra_hsp_remove_doorbells(hsp);
+
+	return err;
 }
 
 static int tegra_hsp_remove(struct platform_device *pdev)
 {
 	struct tegra_hsp *hsp = platform_get_drvdata(pdev);
 
-	mbox_controller_unregister(&hsp->mbox);
-	tegra_hsp_remove_doorbells(hsp);
+	mbox_controller_unregister(&hsp->mbox_sm);
+	mbox_controller_unregister(&hsp->mbox_db);
+
+	if (hsp->doorbell_irq)
+		tegra_hsp_remove_doorbells(hsp);
 
 	return 0;
 }
-- 
2.19.1

[PATCH 5/9] mailbox: tegra-hsp: Add suspend/resume support

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

Upon resuming from a system sleep state, the interrupts for all active
shared mailboxes need to be reenabled, otherwise they will not work.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 drivers/mailbox/tegra-hsp.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/drivers/mailbox/tegra-hsp.c b/drivers/mailbox/tegra-hsp.c
index d070c8e38375..043f7173af8f 100644
--- a/drivers/mailbox/tegra-hsp.c
+++ b/drivers/mailbox/tegra-hsp.c
@@ -18,6 +18,7 @@
 #include <linux/of.h>
 #include <linux/of_device.h>
 #include <linux/platform_device.h>
+#include <linux/pm.h>
 #include <linux/slab.h>
 
 #include <dt-bindings/mailbox/tegra186-hsp.h>
@@ -802,6 +803,23 @@ static int tegra_hsp_remove(struct platform_device *pdev)
 	return 0;
 }
 
+static int tegra_hsp_resume(struct device *dev)
+{
+	struct tegra_hsp *hsp = dev_get_drvdata(dev);
+	unsigned int i;
+
+	for (i = 0; i < hsp->num_sm; i++) {
+		struct tegra_hsp_mailbox *mb = &hsp->mailboxes[i];
+
+		if (mb->channel.chan->cl)
+			tegra_hsp_mailbox_startup(mb->channel.chan);
+	}
+
+	return 0;
+}
+
+static SIMPLE_DEV_PM_OPS(tegra_hsp_pm_ops, NULL, tegra_hsp_resume);
+
 static const struct tegra_hsp_db_map tegra186_hsp_db_map[] = {
 	{ "ccplex", TEGRA_HSP_DB_MASTER_CCPLEX, HSP_DB_CCPLEX, },
 	{ "bpmp",   TEGRA_HSP_DB_MASTER_BPMP,   HSP_DB_BPMP,   },
@@ -821,6 +839,7 @@ static struct platform_driver tegra_hsp_driver = {
 	.driver = {
 		.name = "tegra-hsp",
 		.of_match_table = tegra_hsp_match,
+		.pm = &tegra_hsp_pm_ops,
 	},
 	.probe = tegra_hsp_probe,
 	.remove = tegra_hsp_remove,
-- 
2.19.1

[PATCH 6/9] dt-bindings: serial: Add bindings for nvidia,tegra194-tcu

[Thierry Reding]

From: Mikko Perttunen <mperttunen@nvidia.com>

Add bindings for the Tegra Combined UART device used to talk to the
UART console on Tegra194 systems.

Signed-off-by: Mikko Perttunen <mperttunen@nvidia.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Acked-by: Jon Hunter <jonathanh@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 .../bindings/serial/nvidia,tegra194-tcu.txt   | 35 +++++++++++++++++++
 1 file changed, 35 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/serial/nvidia,tegra194-tcu.txt

diff --git a/Documentation/devicetree/bindings/serial/nvidia,tegra194-tcu.txt b/Documentation/devicetree/bindings/serial/nvidia,tegra194-tcu.txt
new file mode 100644
index 000000000000..a8becf6efd2a
--- /dev/null
+++ b/Documentation/devicetree/bindings/serial/nvidia,tegra194-tcu.txt
@@ -0,0 +1,35 @@
+NVIDIA Tegra Combined UART (TCU)
+
+The TCU is a system for sharing a hardware UART instance among multiple
+systems within the Tegra SoC. It is implemented through a mailbox-
+based protocol where each "virtual UART" has a pair of mailboxes, one
+for transmitting and one for receiving, that is used to communicate
+with the hardware implementing the TCU.
+
+Required properties:
+- name : Should be tcu
+- compatible
+    Array of strings
+    One of:
+    - "nvidia,tegra194-tcu"
+- mbox-names:
+    "rx" - Mailbox for receiving data from hardware UART
+    "tx" - Mailbox for transmitting data to hardware UART
+- mboxes: Mailboxes corresponding to the mbox-names. 
+
+This node is a mailbox consumer. See the following files for details of
+the mailbox subsystem, and the specifiers implemented by the relevant
+provider(s):
+
+- .../mailbox/mailbox.txt
+- .../mailbox/nvidia,tegra186-hsp.txt
+
+Example bindings:
+-----------------
+
+tcu: tcu {
+	compatible = "nvidia,tegra194-tcu";
+	mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_SM 0>,
+	         <&hsp_aon TEGRA_HSP_MBOX_TYPE_SM 1>;
+	mbox-names = "rx", "tx";
+};
-- 
2.19.1

[PATCH 7/9] serial: Add Tegra Combined UART driver

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

The Tegra Combined UART (TCU) is a mailbox-based mechanism that allows
multiplexing multiple "virtual UARTs" into a single hardware serial
port. The TCU is the primary serial port on Tegra194 devices.

Add a TCU driver utilizing the mailbox framework, as the used mailboxes
are part of Tegra HSP blocks that are already controlled by the Tegra
HSP mailbox driver.

Based on work by  Mikko Perttunen <mperttunen@nvidia.com>.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 drivers/tty/serial/Kconfig       |  22 +++
 drivers/tty/serial/Makefile      |   1 +
 drivers/tty/serial/tegra-tcu.c   | 299 +++++++++++++++++++++++++++++++
 include/uapi/linux/serial_core.h |   3 +
 4 files changed, 325 insertions(+)
 create mode 100644 drivers/tty/serial/tegra-tcu.c

diff --git a/drivers/tty/serial/Kconfig b/drivers/tty/serial/Kconfig
index 32886c304641..785306388aa4 100644
--- a/drivers/tty/serial/Kconfig
+++ b/drivers/tty/serial/Kconfig
@@ -323,6 +323,28 @@ config SERIAL_TEGRA
 	  are enabled). This driver uses the APB DMA to achieve higher baudrate
 	  and better performance.
 
+config SERIAL_TEGRA_TCU
+	tristate "NVIDIA Tegra Combined UART"
+	depends on ARCH_TEGRA && TEGRA_HSP_MBOX
+	select SERIAL_CORE
+	help
+	  Support for the mailbox-based TCU (Tegra Combined UART) serial port.
+	  TCU is a virtual serial port that allows multiplexing multiple data
+	  streams into a single hardware serial port.
+
+config SERIAL_TEGRA_TCU_CONSOLE
+	bool "Support for console on a Tegra TCU serial port"
+	depends on SERIAL_TEGRA_TCU=y
+	select SERIAL_CORE_CONSOLE
+	default y
+	---help---
+	  If you say Y here, it will be possible to use a the Tegra TCU as the
+	  system console (the system console is the device which receives all
+	  kernel messages and warnings and which allows logins in single user
+	  mode).
+
+	  If unsure, say Y.
+
 config SERIAL_MAX3100
 	tristate "MAX3100 support"
 	depends on SPI
diff --git a/drivers/tty/serial/Makefile b/drivers/tty/serial/Makefile
index daac675612df..4ad82231ff8a 100644
--- a/drivers/tty/serial/Makefile
+++ b/drivers/tty/serial/Makefile
@@ -76,6 +76,7 @@ obj-$(CONFIG_SERIAL_LANTIQ)	+= lantiq.o
 obj-$(CONFIG_SERIAL_XILINX_PS_UART) += xilinx_uartps.o
 obj-$(CONFIG_SERIAL_SIRFSOC) += sirfsoc_uart.o
 obj-$(CONFIG_SERIAL_TEGRA) += serial-tegra.o
+obj-$(CONFIG_SERIAL_TEGRA_TCU) += tegra-tcu.o
 obj-$(CONFIG_SERIAL_AR933X)   += ar933x_uart.o
 obj-$(CONFIG_SERIAL_EFM32_UART) += efm32-uart.o
 obj-$(CONFIG_SERIAL_ARC)	+= arc_uart.o
diff --git a/drivers/tty/serial/tegra-tcu.c b/drivers/tty/serial/tegra-tcu.c
new file mode 100644
index 000000000000..1d360cd03b18
--- /dev/null
+++ b/drivers/tty/serial/tegra-tcu.c
@@ -0,0 +1,299 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018, NVIDIA CORPORATION.  All rights reserved.
+ */
+
+#include <linux/console.h>
+#include <linux/mailbox_client.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/serial.h>
+#include <linux/serial_core.h>
+#include <linux/slab.h>
+#include <linux/tty.h>
+#include <linux/tty_flip.h>
+
+#define TCU_MBOX_BYTE(i, x)			((x) << (i * 8))
+#define TCU_MBOX_BYTE_V(x, i)			(((x) >> (i * 8)) & 0xff)
+#define TCU_MBOX_NUM_BYTES(x)			((x) << 24)
+#define TCU_MBOX_NUM_BYTES_V(x)			(((x) >> 24) & 0x3)
+
+struct tegra_tcu {
+	struct uart_driver driver;
+#if IS_ENABLED(CONFIG_SERIAL_TEGRA_TCU_CONSOLE)
+	struct console console;
+#endif
+	struct uart_port port;
+
+	struct mbox_client tx_client, rx_client;
+	struct mbox_chan *tx, *rx;
+};
+
+static unsigned int tegra_tcu_uart_tx_empty(struct uart_port *port)
+{
+	return TIOCSER_TEMT;
+}
+
+static void tegra_tcu_uart_set_mctrl(struct uart_port *port, unsigned int mctrl)
+{
+}
+
+static unsigned int tegra_tcu_uart_get_mctrl(struct uart_port *port)
+{
+	return 0;
+}
+
+static void tegra_tcu_uart_stop_tx(struct uart_port *port)
+{
+}
+
+static void tegra_tcu_write_one(struct tegra_tcu *tcu, u32 value,
+			       unsigned int count)
+{
+	void *msg;
+
+	value |= TCU_MBOX_NUM_BYTES(count);
+	msg = (void *)(unsigned long)value;
+	mbox_send_message(tcu->tx, msg);
+}
+
+static void tegra_tcu_write(struct tegra_tcu *tcu, const char *s,
+			    unsigned int count)
+{
+	unsigned int written = 0, i = 0;
+	bool insert_nl = false;
+	u32 value = 0;
+
+	while (i < count) {
+		if (insert_nl) {
+			value |= TCU_MBOX_BYTE(written++, '\n');
+			insert_nl = false;
+			i++;
+		} else if (s[i] == '\n') {
+			value |= TCU_MBOX_BYTE(written++, '\r');
+			insert_nl = true;
+		} else {
+			value |= TCU_MBOX_BYTE(written++, s[i++]);
+		}
+
+		if (written == 3) {
+			tegra_tcu_write_one(tcu, value, 3);
+			value = written = 0;
+		}
+	}
+
+	if (written)
+		tegra_tcu_write_one(tcu, value, written);
+}
+
+static void tegra_tcu_uart_start_tx(struct uart_port *port)
+{
+	struct tegra_tcu *tcu = port->private_data;
+	struct circ_buf *xmit = &port->state->xmit;
+	unsigned long count;
+
+	for (;;) {
+		count = CIRC_CNT_TO_END(xmit->head, xmit->tail, UART_XMIT_SIZE);
+		if (!count)
+			break;
+
+		tegra_tcu_write(tcu, &xmit->buf[xmit->tail], count);
+		xmit->tail = (xmit->tail + count) & (UART_XMIT_SIZE - 1);
+	}
+
+	uart_write_wakeup(port);
+}
+
+static void tegra_tcu_uart_stop_rx(struct uart_port *port)
+{
+}
+
+static void tegra_tcu_uart_break_ctl(struct uart_port *port, int ctl)
+{
+}
+
+static int tegra_tcu_uart_startup(struct uart_port *port)
+{
+	return 0;
+}
+
+static void tegra_tcu_uart_shutdown(struct uart_port *port)
+{
+}
+
+static void tegra_tcu_uart_set_termios(struct uart_port *port,
+				       struct ktermios *new,
+				       struct ktermios *old)
+{
+}
+
+static const struct uart_ops tegra_tcu_uart_ops = {
+	.tx_empty = tegra_tcu_uart_tx_empty,
+	.set_mctrl = tegra_tcu_uart_set_mctrl,
+	.get_mctrl = tegra_tcu_uart_get_mctrl,
+	.stop_tx = tegra_tcu_uart_stop_tx,
+	.start_tx = tegra_tcu_uart_start_tx,
+	.stop_rx = tegra_tcu_uart_stop_rx,
+	.break_ctl = tegra_tcu_uart_break_ctl,
+	.startup = tegra_tcu_uart_startup,
+	.shutdown = tegra_tcu_uart_shutdown,
+	.set_termios = tegra_tcu_uart_set_termios,
+};
+
+#if IS_ENABLED(CONFIG_SERIAL_TEGRA_TCU_CONSOLE)
+static void tegra_tcu_console_write(struct console *cons, const char *s,
+				    unsigned int count)
+{
+	struct tegra_tcu *tcu = container_of(cons, struct tegra_tcu, console);
+
+	tegra_tcu_write(tcu, s, count);
+}
+
+static int tegra_tcu_console_setup(struct console *cons, char *options)
+{
+	return 0;
+}
+#endif
+
+static void tegra_tcu_receive(struct mbox_client *cl, void *msg)
+{
+	struct tegra_tcu *tcu = container_of(cl, struct tegra_tcu, rx_client);
+	struct tty_port *port = &tcu->port.state->port;
+	u32 value = (u32)(unsigned long)msg;
+	unsigned int num_bytes, i;
+
+	num_bytes = TCU_MBOX_NUM_BYTES_V(value);
+
+	for (i = 0; i < num_bytes; i++)
+		tty_insert_flip_char(port, TCU_MBOX_BYTE_V(value, i),
+				     TTY_NORMAL);
+
+	tty_flip_buffer_push(port);
+}
+
+static int tegra_tcu_probe(struct platform_device *pdev)
+{
+	struct uart_port *port;
+	struct tegra_tcu *tcu;
+	int err;
+
+	tcu = devm_kzalloc(&pdev->dev, sizeof(*tcu), GFP_KERNEL);
+	if (!tcu)
+		return -ENOMEM;
+
+	tcu->tx_client.dev = &pdev->dev;
+	tcu->tx_client.tx_block = true;
+	tcu->tx_client.tx_tout = 10000;
+	tcu->rx_client.dev = &pdev->dev;
+	tcu->rx_client.rx_callback = tegra_tcu_receive;
+
+	tcu->tx = mbox_request_channel_byname(&tcu->tx_client, "tx");
+	if (IS_ERR(tcu->tx)) {
+		err = PTR_ERR(tcu->tx);
+		dev_err(&pdev->dev, "failed to get tx mailbox: %d\n", err);
+		return err;
+	}
+
+	tcu->rx = mbox_request_channel_byname(&tcu->rx_client, "rx");
+	if (IS_ERR(tcu->rx)) {
+		err = PTR_ERR(tcu->rx);
+		dev_err(&pdev->dev, "failed to get rx mailbox: %d\n", err);
+		goto free_tx;
+	}
+
+#if IS_ENABLED(CONFIG_SERIAL_TEGRA_TCU_CONSOLE)
+	/* setup the console */
+	strcpy(tcu->console.name, "ttyTCU");
+	tcu->console.device = uart_console_device;
+	tcu->console.flags = CON_PRINTBUFFER | CON_ANYTIME;
+	tcu->console.index = -1;
+	tcu->console.write = tegra_tcu_console_write;
+	tcu->console.setup = tegra_tcu_console_setup;
+	tcu->console.data = &tcu->driver;
+#endif
+
+	/* setup the driver */
+	tcu->driver.owner = THIS_MODULE;
+	tcu->driver.driver_name = "tegra-tcu";
+	tcu->driver.dev_name = "ttyTCU";
+#if IS_ENABLED(CONFIG_SERIAL_TEGRA_TCU_CONSOLE)
+	tcu->driver.cons = &tcu->console;
+#endif
+	tcu->driver.nr = 1;
+
+	err = uart_register_driver(&tcu->driver);
+	if (err) {
+		dev_err(&pdev->dev, "failed to register UART driver: %d\n",
+			err);
+		goto free_rx;
+	}
+
+	/* setup the port */
+	port = &tcu->port;
+	spin_lock_init(&port->lock);
+	port->dev = &pdev->dev;
+	port->type = PORT_TEGRA_TCU;
+	port->ops = &tegra_tcu_uart_ops;
+	port->fifosize = 1;
+	port->iotype = UPIO_MEM;
+	port->flags = UPF_BOOT_AUTOCONF;
+	port->private_data = tcu;
+
+	err = uart_add_one_port(&tcu->driver, port);
+	if (err) {
+		dev_err(&pdev->dev, "failed to add UART port: %d\n", err);
+		goto unregister_uart;
+	}
+
+	platform_set_drvdata(pdev, tcu);
+#if IS_ENABLED(CONFIG_SERIAL_TEGRA_TCU_CONSOLE)
+	register_console(&tcu->console);
+#endif
+
+	return 0;
+
+unregister_uart:
+	uart_unregister_driver(&tcu->driver);
+free_rx:
+	mbox_free_channel(tcu->rx);
+free_tx:
+	mbox_free_channel(tcu->tx);
+
+	return err;
+}
+
+static int tegra_tcu_remove(struct platform_device *pdev)
+{
+	struct tegra_tcu *tcu = platform_get_drvdata(pdev);
+
+#if IS_ENABLED(CONFIG_SERIAL_TEGRA_TCU_CONSOLE)
+	unregister_console(&tcu->console);
+#endif
+	uart_remove_one_port(&tcu->driver, &tcu->port);
+	uart_unregister_driver(&tcu->driver);
+	mbox_free_channel(tcu->rx);
+	mbox_free_channel(tcu->tx);
+
+	return 0;
+}
+
+static const struct of_device_id tegra_tcu_match[] = {
+	{ .compatible = "nvidia,tegra194-tcu" },
+	{ }
+};
+
+static struct platform_driver tegra_tcu_driver = {
+	.driver = {
+		.name = "tegra-tcu",
+		.of_match_table = tegra_tcu_match,
+	},
+	.probe = tegra_tcu_probe,
+	.remove = tegra_tcu_remove,
+};
+module_platform_driver(tegra_tcu_driver);
+
+MODULE_AUTHOR("Mikko Perttunen <mperttunen@nvidia.com>");
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("NVIDIA Tegra Combined UART driver");
diff --git a/include/uapi/linux/serial_core.h b/include/uapi/linux/serial_core.h
index dce5f9dae121..69883c32cb98 100644
--- a/include/uapi/linux/serial_core.h
+++ b/include/uapi/linux/serial_core.h
@@ -79,6 +79,9 @@
 /* Nuvoton UART */
 #define PORT_NPCM	40
 
+/* NVIDIA Tegra Combined UART */
+#define PORT_TEGRA_TCU	41
+
 /* Intel EG20 */
 #define PORT_PCH_8LINE	44
 #define PORT_PCH_2LINE	45
-- 
2.19.1

[PATCH 8/9] arm64: tegra: Add nodes for TCU on Tegra194

[Thierry Reding]

From: Mikko Perttunen <mperttunen@nvidia.com>

Add nodes required for communication through the Tegra Combined UART.
This includes the AON HSP instance, addition of shared interrupts
for the TOP0 HSP instance, and finally the TCU node itself. Also
mark the HSP instances as compatible to tegra194-hsp, as the hardware
is not identical but is compatible to tegra186-hsp.

Signed-off-by: Mikko Perttunen <mperttunen@nvidia.com>
Acked-by: Jon Hunter <jonathanh@nvidia.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 arch/arm64/boot/dts/nvidia/tegra194.dtsi | 38 ++++++++++++++++++++++--
 1 file changed, 35 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/boot/dts/nvidia/tegra194.dtsi b/arch/arm64/boot/dts/nvidia/tegra194.dtsi
index c2091bb16546..521d13be0457 100644
--- a/arch/arm64/boot/dts/nvidia/tegra194.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra194.dtsi
@@ -340,10 +340,35 @@
 		};
 
 		hsp_top0: hsp@3c00000 {
-			compatible = "nvidia,tegra186-hsp";
+			compatible = "nvidia,tegra194-hsp", "nvidia,tegra186-hsp";
 			reg = <0x03c00000 0xa0000>;
-			interrupts = <GIC_SPI 176 IRQ_TYPE_LEVEL_HIGH>;
-			interrupt-names = "doorbell";
+			interrupts = <GIC_SPI 176 IRQ_TYPE_LEVEL_HIGH>,
+			             <GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>,
+			             <GIC_SPI 121 IRQ_TYPE_LEVEL_HIGH>,
+			             <GIC_SPI 122 IRQ_TYPE_LEVEL_HIGH>,
+			             <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH>,
+			             <GIC_SPI 124 IRQ_TYPE_LEVEL_HIGH>,
+			             <GIC_SPI 125 IRQ_TYPE_LEVEL_HIGH>,
+			             <GIC_SPI 126 IRQ_TYPE_LEVEL_HIGH>,
+			             <GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH>;
+			interrupt-names = "doorbell", "shared0", "shared1", "shared2",
+			                  "shared3", "shared4", "shared5", "shared6",
+			                  "shared7";
+			#mbox-cells = <2>;
+		};
+
+		hsp_aon: hsp@c150000 {
+			compatible = "nvidia,tegra194-hsp", "nvidia,tegra186-hsp";
+			reg = <0x0c150000 0xa0000>;
+			interrupts = <GIC_SPI 133 IRQ_TYPE_LEVEL_HIGH>,
+			             <GIC_SPI 134 IRQ_TYPE_LEVEL_HIGH>,
+			             <GIC_SPI 135 IRQ_TYPE_LEVEL_HIGH>,
+			             <GIC_SPI 136 IRQ_TYPE_LEVEL_HIGH>;
+			/*
+			 * Shared interrupt 0 is routed only to AON/SPE, so
+			 * we only have 4 shared interrupts for the CCPLEX.
+			 */
+			interrupt-names = "shared1", "shared2", "shared3", "shared4";
 			#mbox-cells = <2>;
 		};
 
@@ -531,6 +556,13 @@
 		method = "smc";
 	};
 
+	tcu: tcu {
+		compatible = "nvidia,tegra194-tcu";
+		mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_SM 0>,
+		         <&hsp_aon TEGRA_HSP_MBOX_TYPE_SM 1>;
+		mbox-names = "rx", "tx";
+	};
+
 	timer {
 		compatible = "arm,armv8-timer";
 		interrupts = <GIC_PPI 13
-- 
2.19.1

[PATCH 9/9] arm64: tegra: Mark TCU as primary serial port on Tegra194 P2888

[Thierry Reding]

From: Mikko Perttunen <mperttunen@nvidia.com>

The Tegra Combined UART is the proper primary serial port on P2888,
so use it.

Signed-off-by: Mikko Perttunen <mperttunen@nvidia.com>
Acked-by: Jon Hunter <jonathanh@nvidia.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 arch/arm64/boot/dts/nvidia/tegra194-p2888.dtsi | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/boot/dts/nvidia/tegra194-p2888.dtsi b/arch/arm64/boot/dts/nvidia/tegra194-p2888.dtsi
index 57d3f00464ce..fcbe2a88f8db 100644
--- a/arch/arm64/boot/dts/nvidia/tegra194-p2888.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra194-p2888.dtsi
@@ -10,7 +10,7 @@
 	aliases {
 		sdhci0 = "/cbb/sdhci@3460000";
 		sdhci1 = "/cbb/sdhci@3400000";
-		serial0 = &uartb;
+		serial0 = &tcu;
 		i2c0 = "/bpmp/i2c";
 		i2c1 = "/cbb/i2c@3160000";
 		i2c2 = "/cbb/i2c@c240000";
-- 
2.19.1

Re: [PATCH 0/9] serial: Add Tegra Combined UART driver

[Pekka Pessi]

Hi Thierry,

What is the use case for the mailbox driver? What kind of entity will be 
there consuming sent messages and sending messages to kernel?

--Pekka

On 10/26/2018 02:16 PM, Thierry Reding wrote:
> From: Thierry Reding <treding@nvidia.com>
>
> Hi everyone,
>
> this is a reworked version of Mikko's earlier proposal[0]. I've reworked
> the TCU driver itself so that it relies less on global variables as well
> as added a Kconfig option to allow the console support to be selected. I
> also fixed a couple of issues that manifested themselves as I was moving
> towards the IRQ driven mode (TCU was passing a pointer to a local
> variable which was getting stored in the mailbox's ring buffer and the
> data pointed at was becoming stale by the time the mailbox got around to
> dequeue it).
>
> The biggest bulk of the changes is in the mailbox driver. This series
> addresses all of Jassi's comments from back at the time. One notable
> additional change is that shared mailboxes are now interrupt driven,
> which removes the need for polling mode.
>
> Unfortunately there is still an issue because the TCU uses the mailbox
> in atomic context for both TTY and console modes, so we get a sleeping-
> while-atomic BUG when using mbox_client->tx_block = true in order to
> rate-limit mbox_send_message(). In order to work around this, I added a
> mechanism to mbox_send_message() that will allow blocking from atomic
> context if the mailbox controller implements the new ->flush() callback.
> For Tegra HSP shared mailboxes this is done by spinning on the shared
> mailbox register until the receiver has marked the mailbox as empty. I
> have been running this locally for a couple of days now and it works
> perfectly.
>
> Furthermore this series incorporates Mikko's work in progress on
> splitting up the mailbox controllers and allowing multiple controllers
> to match on the same device tree node during mbox_request_channel().
>
> Last but not least there are no build-time dependencies between the
> mailbox and serial drivers, so I think the easiest way to merge this is
> if Jassi picks up patches 1-5, Greg takes patches 6 & 7 and I pick up
> patches 8-9 into the Tegra tree.
>
> Thanks,
> Thierry
>
> [0]: https://lore.kernel.org/patchwork/project/lkml/list/?series=357641
>
> Mikko Perttunen (5):
>    mailbox: Allow multiple controllers per device
>    dt-bindings: tegra186-hsp: Add shared interrupts
>    dt-bindings: serial: Add bindings for nvidia,tegra194-tcu
>    arm64: tegra: Add nodes for TCU on Tegra194
>    arm64: tegra: Mark TCU as primary serial port on Tegra194 P2888
>
> Thierry Reding (4):
>    mailbox: Support blocking transfers in atomic context
>    mailbox: tegra-hsp: Add support for shared mailboxes
>    mailbox: tegra-hsp: Add suspend/resume support
>    serial: Add Tegra Combined UART driver
>
>   .../bindings/mailbox/nvidia,tegra186-hsp.txt  |   3 +
>   .../bindings/serial/nvidia,tegra194-tcu.txt   |  35 ++
>   .../arm64/boot/dts/nvidia/tegra194-p2888.dtsi |   2 +-
>   arch/arm64/boot/dts/nvidia/tegra194.dtsi      |  38 +-
>   drivers/mailbox/mailbox.c                     |  11 +-
>   drivers/mailbox/tegra-hsp.c                   | 495 +++++++++++++++---
>   drivers/tty/serial/Kconfig                    |  22 +
>   drivers/tty/serial/Makefile                   |   1 +
>   drivers/tty/serial/tegra-tcu.c                | 299 +++++++++++
>   include/linux/mailbox_controller.h            |   4 +
>   include/uapi/linux/serial_core.h              |   3 +
>   11 files changed, 847 insertions(+), 66 deletions(-)
>   create mode 100644 Documentation/devicetree/bindings/serial/nvidia,tegra194-tcu.txt
>   create mode 100644 drivers/tty/serial/tegra-tcu.c
>

Re: [PATCH 4/9] mailbox: tegra-hsp: Add support for shared mailboxes

[Pekka Pessi]

Hi Thierry,

There is typically one entity (aux cpu or a VM running on CCPLEX) owning 
the "empty" or producer side of mailbox (iow, waking up on empty) and 
another entity owning the "full" or consumer side of mailbox (waking up 
on full). An entity should not muck with the interrupts used by the 
opposite side.

One entity typically owns one shared interrupt only.  For the 
BPMP/SCE/RCE/SPE HSP blocks the shared interrupt 0 is owned by the 
auxiliary processor itself, the shared interrupts 1..4 are connected to 
LIC and are available to other entities. The convention is to go through 
the interrupts 0..4 and then using the first available shared interrupt 
for both full and empty.

The interrupt functions should use a mask for mailboxes owned by kernel 
(in essence what the IE register should be for the HSP shared interrupt 
owned by the kernel) and serve only those mailboxes owned by kernel. 
Note that there is no reset for HSP in Xavier, and the IE register 
contents may be stale.

And lastly, if we want to support only Xavier and later, perhaps we 
should be more clear in the bindings? There are no mailbox-specific 
interrupt enable registers available on Parker and your design relies on 
them.

--Pekka

On 10/26/2018 02:16 PM, Thierry Reding wrote:
> From: Thierry Reding <treding@nvidia.com>
>
> The Tegra HSP block supports 'shared mailboxes' that are simple 32-bit
> registers consisting of a FULL bit in MSB position and 31 bits of data.
> The hardware can be configured to trigger interrupts when a mailbox
> is empty or full. Add support for these shared mailboxes to the HSP
> driver.
>
> The initial use for the mailboxes is the Tegra Combined UART. For this
> purpose, we use interrupts to receive data, and spinning to wait for
> the transmit mailbox to be emptied to minimize unnecessary overhead.
>
> Based on work by Mikko Perttunen <mperttunen@nvidia.com>.
>
> Signed-off-by: Thierry Reding <treding@nvidia.com>
> ---
>   drivers/mailbox/tegra-hsp.c | 476 +++++++++++++++++++++++++++++++-----
>   1 file changed, 415 insertions(+), 61 deletions(-)
>
> diff --git a/drivers/mailbox/tegra-hsp.c b/drivers/mailbox/tegra-hsp.c
> index 0cde356c11ab..d070c8e38375 100644
> --- a/drivers/mailbox/tegra-hsp.c
> +++ b/drivers/mailbox/tegra-hsp.c
> @@ -1,5 +1,5 @@
>   /*
> - * Copyright (c) 2016, NVIDIA CORPORATION.  All rights reserved.
> + * Copyright (c) 2016-2018, NVIDIA CORPORATION.  All rights reserved.
>    *
>    * This program is free software; you can redistribute it and/or modify it
>    * under the terms and conditions of the GNU General Public License,
> @@ -11,6 +11,7 @@
>    * more details.
>    */
>   
> +#include <linux/delay.h>
>   #include <linux/interrupt.h>
>   #include <linux/io.h>
>   #include <linux/mailbox_controller.h>
> @@ -21,6 +22,17 @@
>   
>   #include <dt-bindings/mailbox/tegra186-hsp.h>
>   
> +#include "mailbox.h"
> +
> +#define HSP_INT_IE(x)		(0x100 + ((x) * 4))
> +#define HSP_INT_IV		0x300
> +#define HSP_INT_IR		0x304
> +
> +#define HSP_INT_EMPTY_SHIFT	0
> +#define HSP_INT_EMPTY_MASK	0xff
> +#define HSP_INT_FULL_SHIFT	8
> +#define HSP_INT_FULL_MASK	0xff
> +
>   #define HSP_INT_DIMENSIONING	0x380
>   #define HSP_nSM_SHIFT		0
>   #define HSP_nSS_SHIFT		4
> @@ -34,6 +46,11 @@
>   #define HSP_DB_RAW	0x8
>   #define HSP_DB_PENDING	0xc
>   
> +#define HSP_SM_SHRD_MBOX	0x0
> +#define HSP_SM_SHRD_MBOX_FULL	BIT(31)
> +#define HSP_SM_SHRD_MBOX_FULL_INT_IE	0x04
> +#define HSP_SM_SHRD_MBOX_EMPTY_INT_IE	0x08
> +
>   #define HSP_DB_CCPLEX		1
>   #define HSP_DB_BPMP		3
>   #define HSP_DB_MAX		7
> @@ -55,6 +72,12 @@ struct tegra_hsp_doorbell {
>   	unsigned int index;
>   };
>   
> +struct tegra_hsp_mailbox {
> +	struct tegra_hsp_channel channel;
> +	unsigned int index;
> +	bool sending;
> +};
> +
>   struct tegra_hsp_db_map {
>   	const char *name;
>   	unsigned int master;
> @@ -66,10 +89,13 @@ struct tegra_hsp_soc {
>   };
>   
>   struct tegra_hsp {
> +	struct device *dev;
>   	const struct tegra_hsp_soc *soc;
> -	struct mbox_controller mbox;
> +	struct mbox_controller mbox_db;
> +	struct mbox_controller mbox_sm;
>   	void __iomem *regs;
> -	unsigned int irq;
> +	unsigned int doorbell_irq;
> +	unsigned int *shared_irqs;
>   	unsigned int num_sm;
>   	unsigned int num_as;
>   	unsigned int num_ss;
> @@ -78,14 +104,9 @@ struct tegra_hsp {
>   	spinlock_t lock;
>   
>   	struct list_head doorbells;
> +	struct tegra_hsp_mailbox *mailboxes;
>   };
>   
> -static inline struct tegra_hsp *
> -to_tegra_hsp(struct mbox_controller *mbox)
> -{
> -	return container_of(mbox, struct tegra_hsp, mbox);
> -}
> -
>   static inline u32 tegra_hsp_readl(struct tegra_hsp *hsp, unsigned int offset)
>   {
>   	return readl(hsp->regs + offset);
> @@ -158,7 +179,7 @@ static irqreturn_t tegra_hsp_doorbell_irq(int irq, void *data)
>   
>   	spin_lock(&hsp->lock);
>   
> -	for_each_set_bit(master, &value, hsp->mbox.num_chans) {
> +	for_each_set_bit(master, &value, hsp->mbox_db.num_chans) {
>   		struct tegra_hsp_doorbell *db;
>   
>   		db = __tegra_hsp_doorbell_get(hsp, master);
> @@ -182,6 +203,84 @@ static irqreturn_t tegra_hsp_doorbell_irq(int irq, void *data)
>   	return IRQ_HANDLED;
>   }
>   
> +static irqreturn_t tegra_hsp_shared_full_irq(int irq, void *data)
> +{
> +	struct tegra_hsp *hsp = data;
> +	unsigned long bit, mask;
> +	u32 status, value;
> +	void *msg;
> +
> +	status = tegra_hsp_readl(hsp, HSP_INT_IR);
> +
> +	/* only interested in FULL interrupts */
> +	mask = (status >> HSP_INT_FULL_SHIFT) & HSP_INT_FULL_MASK;
> +
> +	if (!mask)
> +		return IRQ_NONE;
> +
> +	for_each_set_bit(bit, &mask, hsp->num_sm) {
> +		struct tegra_hsp_mailbox *mb = &hsp->mailboxes[bit];
> +
> +		if (!mb->sending) {
> +			value = tegra_hsp_channel_readl(&mb->channel,
> +							HSP_SM_SHRD_MBOX);
> +			value &= ~HSP_SM_SHRD_MBOX_FULL;
> +			msg = (void *)(unsigned long)value;
> +			mbox_chan_received_data(mb->channel.chan, msg);
> +
> +			/*
> +			 * Need to clear all bits here since some producers,
> +			 * such as TCU, depend on fields in the register
> +			 * getting cleared by the consumer.
> +			 *
> +			 * The mailbox API doesn't give the consumers a way
> +			 * of doing that explicitly, so we have to make sure
> +			 * we cover all possible cases.
> +			 */
> +			tegra_hsp_channel_writel(&mb->channel, 0x0,
> +						 HSP_SM_SHRD_MBOX);
> +		}
> +	}
> +
> +	return IRQ_HANDLED;
> +}
> +
> +static irqreturn_t tegra_hsp_shared_empty_irq(int irq, void *data)
> +{
> +	struct tegra_hsp *hsp = data;
> +	unsigned long bit, mask;
> +	u32 status, value;
> +
> +	status = tegra_hsp_readl(hsp, HSP_INT_IR);
> +
> +	/* only interested in EMPTY interrupts */
> +	mask = (status >> HSP_INT_EMPTY_SHIFT) & HSP_INT_EMPTY_MASK;
> +
> +	if (!mask)
> +		return IRQ_NONE;
> +
> +	for_each_set_bit(bit, &mask, hsp->num_sm) {
> +		struct tegra_hsp_mailbox *mb = &hsp->mailboxes[bit];
> +
> +		if (mb->sending) {
> +			/*
> +			 * Disable EMPTY interrupts until data is sent
> +			 * with the next message. These interrupts are
> +			 * level-triggered, so if we kept them enabled
> +			 * they would constantly trigger until we next
> +			 * write data into the message.
> +			 */
> +			value = tegra_hsp_readl(hsp, HSP_INT_IE(1));
> +			value &= ~BIT(HSP_INT_EMPTY_SHIFT + mb->index);
> +			tegra_hsp_writel(hsp, value, HSP_INT_IE(1));
> +
> +			mbox_chan_txdone(mb->channel.chan, 0);
> +		}
> +	}
> +
> +	return IRQ_HANDLED;
> +}
> +
>   static struct tegra_hsp_channel *
>   tegra_hsp_doorbell_create(struct tegra_hsp *hsp, const char *name,
>   			  unsigned int master, unsigned int index)
> @@ -194,7 +293,7 @@ tegra_hsp_doorbell_create(struct tegra_hsp *hsp, const char *name,
>   	if (!db)
>   		return ERR_PTR(-ENOMEM);
>   
> -	offset = (1 + (hsp->num_sm / 2) + hsp->num_ss + hsp->num_as) << 16;
> +	offset = (1 + (hsp->num_sm / 2) + hsp->num_ss + hsp->num_as) * SZ_64K;
>   	offset += index * 0x100;
>   
>   	db->channel.regs = hsp->regs + offset;
> @@ -235,8 +334,8 @@ static int tegra_hsp_doorbell_startup(struct mbox_chan *chan)
>   	unsigned long flags;
>   	u32 value;
>   
> -	if (db->master >= hsp->mbox.num_chans) {
> -		dev_err(hsp->mbox.dev,
> +	if (db->master >= chan->mbox->num_chans) {
> +		dev_err(chan->mbox->dev,
>   			"invalid master ID %u for HSP channel\n",
>   			db->master);
>   		return -EINVAL;
> @@ -281,46 +380,147 @@ static void tegra_hsp_doorbell_shutdown(struct mbox_chan *chan)
>   	spin_unlock_irqrestore(&hsp->lock, flags);
>   }
>   
> -static const struct mbox_chan_ops tegra_hsp_doorbell_ops = {
> +static const struct mbox_chan_ops tegra_hsp_db_ops = {
>   	.send_data = tegra_hsp_doorbell_send_data,
>   	.startup = tegra_hsp_doorbell_startup,
>   	.shutdown = tegra_hsp_doorbell_shutdown,
>   };
>   
> -static struct mbox_chan *of_tegra_hsp_xlate(struct mbox_controller *mbox,
> +static int tegra_hsp_mailbox_send_data(struct mbox_chan *chan, void *data)
> +{
> +	struct tegra_hsp_mailbox *mb = chan->con_priv;
> +	struct tegra_hsp *hsp = mb->channel.hsp;
> +	u32 value;
> +
> +	mb->sending = true;
> +
> +	/* copy data and mark mailbox full */
> +	value = (u32)(unsigned long)data;
> +	value |= HSP_SM_SHRD_MBOX_FULL;
> +
> +	tegra_hsp_channel_writel(&mb->channel, value, HSP_SM_SHRD_MBOX);
> +
> +	if (!irqs_disabled()) {
> +		/* enable EMPTY interrupt for the shared mailbox */
> +		value = tegra_hsp_readl(hsp, HSP_INT_IE(1));
> +		value |= BIT(HSP_INT_EMPTY_SHIFT + mb->index);
> +		tegra_hsp_writel(hsp, value, HSP_INT_IE(1));
> +	}
> +
> +	return 0;
> +}
> +
> +static int tegra_hsp_mailbox_flush(struct mbox_chan *chan,
> +				   unsigned long timeout)
> +{
> +	struct tegra_hsp_mailbox *mb = chan->con_priv;
> +	struct tegra_hsp_channel *ch = &mb->channel;
> +	u32 value;
> +
> +	timeout = jiffies + msecs_to_jiffies(timeout);
> +
> +	while (time_before(jiffies, timeout)) {
> +		value = tegra_hsp_channel_readl(ch, HSP_SM_SHRD_MBOX);
> +		if ((value & HSP_SM_SHRD_MBOX_FULL) == 0) {
> +			mbox_chan_txdone(chan, 0);
> +			return 0;
> +		}
> +
> +		udelay(1);
> +	}
> +
> +	return -ETIME;
> +}
> +
> +static int tegra_hsp_mailbox_startup(struct mbox_chan *chan)
> +{
> +	struct tegra_hsp_mailbox *mb = chan->con_priv;
> +	struct tegra_hsp_channel *ch = &mb->channel;
> +	struct tegra_hsp *hsp = mb->channel.hsp;
> +	u32 value;
> +
> +	chan->txdone_method = TXDONE_BY_IRQ;
> +
> +	/*
> +	 * Shared mailboxes start out as consumers by default. FULL interrupts
> +	 * are coalesced at shared interrupt 0, while EMPTY interrupts will be
> +	 * coalesced at shared interrupt 1.
> +	 *
> +	 * Keep EMPTY interrupts disabled at startup and only enable them when
> +	 * the mailbox is actually full. This is required because the FULL and
> +	 * EMPTY interrupts are level-triggered, so keeping EMPTY interrupts
> +	 * enabled all the time would cause an interrupt storm while mailboxes
> +	 * are idle.
> +	 */
> +
> +	value = tegra_hsp_readl(hsp, HSP_INT_IE(0));
> +	value |= BIT(HSP_INT_FULL_SHIFT + mb->index);
> +	tegra_hsp_writel(hsp, value, HSP_INT_IE(0));
> +
> +	value = tegra_hsp_readl(hsp, HSP_INT_IE(1));
> +	value &= ~BIT(HSP_INT_EMPTY_SHIFT + mb->index);
> +	tegra_hsp_writel(hsp, value, HSP_INT_IE(1));
> +
> +	tegra_hsp_channel_writel(ch, 0x1, HSP_SM_SHRD_MBOX_FULL_INT_IE);
> +	tegra_hsp_channel_writel(ch, 0x1, HSP_SM_SHRD_MBOX_EMPTY_INT_IE);
> +
> +	return 0;
> +}
> +
> +static void tegra_hsp_mailbox_shutdown(struct mbox_chan *chan)
> +{
> +	struct tegra_hsp_mailbox *mb = chan->con_priv;
> +	struct tegra_hsp_channel *ch = &mb->channel;
> +	struct tegra_hsp *hsp = mb->channel.hsp;
> +	u32 value;
> +
> +	tegra_hsp_channel_writel(ch, 0x0, HSP_SM_SHRD_MBOX_EMPTY_INT_IE);
> +	tegra_hsp_channel_writel(ch, 0x0, HSP_SM_SHRD_MBOX_FULL_INT_IE);
> +
> +	value = tegra_hsp_readl(hsp, HSP_INT_IE(1));
> +	value &= ~BIT(HSP_INT_EMPTY_SHIFT + mb->index);
> +	tegra_hsp_writel(hsp, value, HSP_INT_IE(1));
> +
> +	value = tegra_hsp_readl(hsp, HSP_INT_IE(0));
> +	value &= ~BIT(HSP_INT_FULL_SHIFT + mb->index);
> +	tegra_hsp_writel(hsp, value, HSP_INT_IE(0));
> +}
> +
> +static const struct mbox_chan_ops tegra_hsp_sm_ops = {
> +	.send_data = tegra_hsp_mailbox_send_data,
> +	.flush = tegra_hsp_mailbox_flush,
> +	.startup = tegra_hsp_mailbox_startup,
> +	.shutdown = tegra_hsp_mailbox_shutdown,
> +};
> +
> +static struct mbox_chan *tegra_hsp_db_xlate(struct mbox_controller *mbox,
>   					    const struct of_phandle_args *args)
>   {
> +	struct tegra_hsp *hsp = container_of(mbox, struct tegra_hsp, mbox_db);
> +	unsigned int type = args->args[0], master = args->args[1];
>   	struct tegra_hsp_channel *channel = ERR_PTR(-ENODEV);
> -	struct tegra_hsp *hsp = to_tegra_hsp(mbox);
> -	unsigned int type = args->args[0];
> -	unsigned int master = args->args[1];
>   	struct tegra_hsp_doorbell *db;
>   	struct mbox_chan *chan;
>   	unsigned long flags;
>   	unsigned int i;
>   
> -	switch (type) {
> -	case TEGRA_HSP_MBOX_TYPE_DB:
> -		db = tegra_hsp_doorbell_get(hsp, master);
> -		if (db)
> -			channel = &db->channel;
> +	if (type != TEGRA_HSP_MBOX_TYPE_DB || !hsp->doorbell_irq)
> +		return ERR_PTR(-ENODEV);
>   
> -		break;
> -
> -	default:
> -		break;
> -	}
> +	db = tegra_hsp_doorbell_get(hsp, master);
> +	if (db)
> +		channel = &db->channel;
>   
>   	if (IS_ERR(channel))
>   		return ERR_CAST(channel);
>   
>   	spin_lock_irqsave(&hsp->lock, flags);
>   
> -	for (i = 0; i < hsp->mbox.num_chans; i++) {
> -		chan = &hsp->mbox.chans[i];
> +	for (i = 0; i < mbox->num_chans; i++) {
> +		chan = &mbox->chans[i];
>   		if (!chan->con_priv) {
> -			chan->con_priv = channel;
>   			channel->chan = chan;
> +			chan->con_priv = db;
>   			break;
>   		}
>   
> @@ -332,6 +532,19 @@ static struct mbox_chan *of_tegra_hsp_xlate(struct mbox_controller *mbox,
>   	return chan ?: ERR_PTR(-EBUSY);
>   }
>   
> +static struct mbox_chan *tegra_hsp_sm_xlate(struct mbox_controller *mbox,
> +					    const struct of_phandle_args *args)
> +{
> +	struct tegra_hsp *hsp = container_of(mbox, struct tegra_hsp, mbox_sm);
> +	unsigned int type = args->args[0], index = args->args[1];
> +
> +	if (type != TEGRA_HSP_MBOX_TYPE_SM || !hsp->shared_irqs ||
> +	    index >= hsp->num_sm)
> +		return ERR_PTR(-ENODEV);
> +
> +	return hsp->mailboxes[index].channel.chan;
> +}
> +
>   static void tegra_hsp_remove_doorbells(struct tegra_hsp *hsp)
>   {
>   	struct tegra_hsp_doorbell *db, *tmp;
> @@ -364,10 +577,72 @@ static int tegra_hsp_add_doorbells(struct tegra_hsp *hsp)
>   	return 0;
>   }
>   
> +static int tegra_hsp_add_mailboxes(struct tegra_hsp *hsp, struct device *dev)
> +{
> +	int i;
> +
> +	hsp->mailboxes = devm_kcalloc(dev, hsp->num_sm, sizeof(*hsp->mailboxes),
> +				      GFP_KERNEL);
> +	if (!hsp->mailboxes)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < hsp->num_sm; i++) {
> +		struct tegra_hsp_mailbox *mb = &hsp->mailboxes[i];
> +
> +		mb->index = i;
> +		mb->sending = false;
> +
> +		mb->channel.hsp = hsp;
> +		mb->channel.regs = hsp->regs + SZ_64K + i * SZ_32K;
> +		mb->channel.chan = &hsp->mbox_sm.chans[i];
> +		mb->channel.chan->con_priv = mb;
> +	}
> +
> +	return 0;
> +}
> +
> +static int tegra_hsp_request_shared_irqs(struct tegra_hsp *hsp)
> +{
> +	int err;
> +
> +	if (hsp->shared_irqs[0] > 0) {
> +		err = devm_request_irq(hsp->dev, hsp->shared_irqs[0],
> +				       tegra_hsp_shared_full_irq, 0,
> +				       dev_name(hsp->dev), hsp);
> +		if (err < 0) {
> +			dev_err(hsp->dev,
> +				"failed to request full interrupt: %d\n",
> +				err);
> +			return err;
> +		}
> +
> +		dev_dbg(hsp->dev, "full interrupt requested: %u\n",
> +			hsp->shared_irqs[0]);
> +	}
> +
> +	if (hsp->shared_irqs[1] > 0) {
> +		err = devm_request_irq(hsp->dev, hsp->shared_irqs[1],
> +				       tegra_hsp_shared_empty_irq, 0,
> +				       dev_name(hsp->dev), hsp);
> +		if (err < 0) {
> +			dev_err(hsp->dev,
> +				"failed to request empty interrupt: %d\n",
> +				err);
> +			return err;
> +		}
> +
> +		dev_dbg(hsp->dev, "empty interrupt requested: %u\n",
> +			hsp->shared_irqs[1]);
> +	}
> +
> +	return 0;
> +}
> +
>   static int tegra_hsp_probe(struct platform_device *pdev)
>   {
>   	struct tegra_hsp *hsp;
>   	struct resource *res;
> +	unsigned int i;
>   	u32 value;
>   	int err;
>   
> @@ -375,6 +650,7 @@ static int tegra_hsp_probe(struct platform_device *pdev)
>   	if (!hsp)
>   		return -ENOMEM;
>   
> +	hsp->dev = &pdev->dev;
>   	hsp->soc = of_device_get_match_data(&pdev->dev);
>   	INIT_LIST_HEAD(&hsp->doorbells);
>   	spin_lock_init(&hsp->lock);
> @@ -392,58 +668,136 @@ static int tegra_hsp_probe(struct platform_device *pdev)
>   	hsp->num_si = (value >> HSP_nSI_SHIFT) & HSP_nINT_MASK;
>   
>   	err = platform_get_irq_byname(pdev, "doorbell");
> -	if (err < 0) {
> -		dev_err(&pdev->dev, "failed to get doorbell IRQ: %d\n", err);
> -		return err;
> +	if (err >= 0)
> +		hsp->doorbell_irq = err;
> +
> +	if (hsp->num_si > 0) {
> +		unsigned int count = 0;
> +
> +		hsp->shared_irqs = devm_kcalloc(&pdev->dev, hsp->num_si,
> +						sizeof(*hsp->shared_irqs),
> +						GFP_KERNEL);
> +		if (!hsp->shared_irqs)
> +			return -ENOMEM;
> +
> +		for (i = 0; i < hsp->num_si; i++) {
> +			char *name;
> +
> +			name = kasprintf(GFP_KERNEL, "shared%u", i);
> +			if (!name)
> +				return -ENOMEM;
> +
> +			err = platform_get_irq_byname(pdev, name);
> +			if (err >= 0) {
> +				hsp->shared_irqs[i] = err;
> +				count++;
> +			}
> +
> +			kfree(name);
> +		}
> +
> +		if (count == 0) {
> +			devm_kfree(&pdev->dev, hsp->shared_irqs);
> +			hsp->shared_irqs = NULL;
> +		}
>   	}
>   
> -	hsp->irq = err;
> +	/* setup the doorbell controller */
> +	hsp->mbox_db.of_xlate = tegra_hsp_db_xlate;
> +	hsp->mbox_db.num_chans = 32;
> +	hsp->mbox_db.dev = &pdev->dev;
> +	hsp->mbox_db.ops = &tegra_hsp_db_ops;
>   
> -	hsp->mbox.of_xlate = of_tegra_hsp_xlate;
> -	hsp->mbox.num_chans = 32;
> -	hsp->mbox.dev = &pdev->dev;
> -	hsp->mbox.txdone_irq = false;
> -	hsp->mbox.txdone_poll = false;
> -	hsp->mbox.ops = &tegra_hsp_doorbell_ops;
> +	hsp->mbox_db.chans = devm_kcalloc(&pdev->dev, hsp->mbox_db.num_chans,
> +					  sizeof(*hsp->mbox_db.chans),
> +					  GFP_KERNEL);
> +	if (!hsp->mbox_db.chans)
> +		return -ENOMEM;
>   
> -	hsp->mbox.chans = devm_kcalloc(&pdev->dev, hsp->mbox.num_chans,
> -					sizeof(*hsp->mbox.chans),
> -					GFP_KERNEL);
> -	if (!hsp->mbox.chans)
> +	if (hsp->doorbell_irq) {
> +		err = tegra_hsp_add_doorbells(hsp);
> +		if (err < 0) {
> +			dev_err(&pdev->dev, "failed to add doorbells: %d\n",
> +			        err);
> +			return err;
> +		}
> +	}
> +
> +	err = mbox_controller_register(&hsp->mbox_db);
> +	if (err < 0) {
> +		dev_err(&pdev->dev, "failed to register doorbell mailbox: %d\n", err);
> +		goto remove_doorbells;
> +	}
> +
> +	/* setup the shared mailbox controller */
> +	hsp->mbox_sm.of_xlate = tegra_hsp_sm_xlate;
> +	hsp->mbox_sm.num_chans = hsp->num_sm;
> +	hsp->mbox_sm.dev = &pdev->dev;
> +	hsp->mbox_sm.ops = &tegra_hsp_sm_ops;
> +
> +	hsp->mbox_sm.chans = devm_kcalloc(&pdev->dev, hsp->mbox_sm.num_chans,
> +					  sizeof(*hsp->mbox_sm.chans),
> +					  GFP_KERNEL);
> +	if (!hsp->mbox_sm.chans)
>   		return -ENOMEM;
>   
> -	err = tegra_hsp_add_doorbells(hsp);
> +	if (hsp->shared_irqs) {
> +		err = tegra_hsp_add_mailboxes(hsp, &pdev->dev);
> +		if (err < 0) {
> +			dev_err(&pdev->dev, "failed to add mailboxes: %d\n",
> +			        err);
> +			goto unregister_mbox_db;
> +		}
> +	}
> +
> +	err = mbox_controller_register(&hsp->mbox_sm);
>   	if (err < 0) {
> -		dev_err(&pdev->dev, "failed to add doorbells: %d\n", err);
> -		return err;
> +		dev_err(&pdev->dev, "failed to register shared mailbox: %d\n", err);
> +		goto unregister_mbox_db;
>   	}
>   
>   	platform_set_drvdata(pdev, hsp);
>   
> -	err = mbox_controller_register(&hsp->mbox);
> -	if (err) {
> -		dev_err(&pdev->dev, "failed to register mailbox: %d\n", err);
> -		tegra_hsp_remove_doorbells(hsp);
> -		return err;
> +	if (hsp->doorbell_irq) {
> +		err = devm_request_irq(&pdev->dev, hsp->doorbell_irq,
> +				       tegra_hsp_doorbell_irq, IRQF_NO_SUSPEND,
> +				       dev_name(&pdev->dev), hsp);
> +		if (err < 0) {
> +			dev_err(&pdev->dev,
> +			        "failed to request doorbell IRQ#%u: %d\n",
> +				hsp->doorbell_irq, err);
> +			goto unregister_mbox_sm;
> +		}
>   	}
>   
> -	err = devm_request_irq(&pdev->dev, hsp->irq, tegra_hsp_doorbell_irq,
> -			       IRQF_NO_SUSPEND, dev_name(&pdev->dev), hsp);
> -	if (err < 0) {
> -		dev_err(&pdev->dev, "failed to request IRQ#%u: %d\n",
> -			hsp->irq, err);
> -		return err;
> +	if (hsp->shared_irqs) {
> +		err = tegra_hsp_request_shared_irqs(hsp);
> +		if (err < 0)
> +			goto unregister_mbox_sm;
>   	}
>   
>   	return 0;
> +
> +unregister_mbox_sm:
> +	mbox_controller_unregister(&hsp->mbox_sm);
> +unregister_mbox_db:
> +	mbox_controller_unregister(&hsp->mbox_db);
> +remove_doorbells:
> +	if (hsp->doorbell_irq)
> +		tegra_hsp_remove_doorbells(hsp);
> +
> +	return err;
>   }
>   
>   static int tegra_hsp_remove(struct platform_device *pdev)
>   {
>   	struct tegra_hsp *hsp = platform_get_drvdata(pdev);
>   
> -	mbox_controller_unregister(&hsp->mbox);
> -	tegra_hsp_remove_doorbells(hsp);
> +	mbox_controller_unregister(&hsp->mbox_sm);
> +	mbox_controller_unregister(&hsp->mbox_db);
> +
> +	if (hsp->doorbell_irq)
> +		tegra_hsp_remove_doorbells(hsp);
>   
>   	return 0;
>   }

Re: [PATCH 0/9] serial: Add Tegra Combined UART driver

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 430 bytes --]

On Mon, Oct 29, 2018 at 11:04:06AM +0200, Pekka Pessi wrote:
> Hi Thierry,
> 
> What is the use case for the mailbox driver? What kind of entity will be
> there consuming sent messages and sending messages to kernel?

This is currently only used for the TCU. I'm sure there could be other
cases, but we support none of the upstream at the moment. Are you aware
of any others that we need to take into account?

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 4/9] mailbox: tegra-hsp: Add support for shared mailboxes

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 4879 bytes --]

On Mon, Oct 29, 2018 at 12:04:22PM +0200, Pekka Pessi wrote:
> Hi Thierry,
> 
> There is typically one entity (aux cpu or a VM running on CCPLEX) owning the
> "empty" or producer side of mailbox (iow, waking up on empty) and another
> entity owning the "full" or consumer side of mailbox (waking up on full). An
> entity should not muck with the interrupts used by the opposite side.

Okay, that explains some of my observations. I was initially trying to
program interrupt enables for both FULL and EMPTY interrupts for all
mailboxes, but then I'd usually get timeouts because the consumer wasn't
responding (i.e. the SPE wasn't getting FULL interrupts for the CCPLEX's
TX mailbox).

If I understand correctly, you're saying that the CPU should only be
using the EMPTY interrupts for it's TX mailbox (while leaving the FULL
interrupts completely untouched) and only the FULL interrupt for it's
RX mailbox (while leaving the EMPTY interrupts untouched). This is a bit
of a problem because the mailbox driver doesn't really know anything
about the direction when it starts up, so how would it make the decision
about how to program the registers?

> One entity typically owns one shared interrupt only.  For the
> BPMP/SCE/RCE/SPE HSP blocks the shared interrupt 0 is owned by the auxiliary
> processor itself, the shared interrupts 1..4 are connected to LIC and are
> available to other entities. The convention is to go through the interrupts
> 0..4 and then using the first available shared interrupt for both full and
> empty.

That partially matches another of my observations. It seems like we
can't use the shared interrupt 0 at all on at least the AON HSP. That's
fine because that HSP instance contains the TX mailbox for TCU and by
the current convention in the HSP driver, shared interrupt 0 would be
aggregating the FULL interrupts, which according to the above we don't
need for TX mailboxes.

What's somewhat surprising is that you're saying that both FULL and
EMPTY interrupts should be handled by the same shared interrupt. That's
the opposite of what the recommended programming sequence is that the
TRM specifies. Why is it better to handle both FULL and EMPTY interrupts
with the same shared interrupt?

> The interrupt functions should use a mask for mailboxes owned by kernel (in
> essence what the IE register should be for the HSP shared interrupt owned by
> the kernel) and serve only those mailboxes owned by kernel. Note that there
> is no reset for HSP in Xavier, and the IE register contents may be stale.

Would it be safe to clear all of the IE registers to 0 on driver probe?
I seem to remember trying to do that and getting similar behaviour to
what I describe above, namely that interrupts on the SPE weren't working
anymore. I concluded that the IE register must be shared between the
various processors, even though that's somewhat suprising given that
there is no way to synchronize accesses to those registers, so their
programming would be somewhat up to chance.

Do you know any more about these registers? If they are indeed separate
for each processor, it should be fairly easy to keep track of the
mailboxes used by the kernel and process only those. Again I don't know
how exactly to distinguish between TX and RX mailboxes because they all
start out the same and only their use defines which direction they go.
Currently this works because we program them as consumers by default.
That means we enable the FULL interrupts but keep EMPTY interrupts
disabled until a message in transmitted on the mailbox, at which point
we enable the EMPTY interrupt. I suppose at that point we should also
disable the FULL interrupt, given the above discussion.

> And lastly, if we want to support only Xavier and later, perhaps we should
> be more clear in the bindings? There are no mailbox-specific interrupt
> enable registers available on Parker and your design relies on them.

That was certainly not the intention. I thought I had seen the per-
mailbox interrupt enable registers also in Tegra186 documentation, but
after double-checking they're indeed not there. I don't think the driver
currently "relies" on them because it uses them in addition to the
HSP_IE registers. I suppose that accessing them might cause aborts on
Tegra186 if they don't exist, though.

I'm not entirely clear on what the advantages are of using the per-
mailbox registers, or how they are supposed to be used. The existing
documentation doesn't really explain how these are supposed to be used
either, so I was mostly just going by trial and error.

Do you know anything more on how to use these registers? I can easily
make them Tegra194 specific in the code, but if they're aren't any clear
advantages, it might just be easier to stick with HSP_IE programming
only.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 4/9] mailbox: tegra-hsp: Add support for shared mailboxes

[Pekka Pessi]

Hi Thierry,

 From the 0/9:
> Are you aware of any others that we need to take into account?
We would like to use upstream driver for RCE (and probably AON and SCE) 
mailbox handling, too. Eventually.
> This is a bit
> of a problem because the mailbox driver doesn't really know anything
> about the direction when it starts up, so how would it make the decision
> about how to program the registers?
I'm afraid that information must be stored in the device tree.
> What's somewhat surprising is that you're saying that both FULL and
> EMPTY interrupts should be handled by the same shared interrupt. That's
> the opposite of what the recommended programming sequence is that the
> TRM specifies.
Which TRM you mean? Something here?

https://p4viewer.nvidia.com/get///hw/ar/doc/Tech_Pubs/Xavier/TRM/

I browsed through he Xavier_TRM_DP09253001v1.pdf HSP section and I could 
not find any recommendations? But see below.
> Why is it better to handle both FULL and EMPTY interrupts
> with the same shared interrupt?
Only top0 has 8 interrupts, rest of the HSP blocks have only 4 (or 5 in 
case of top1) interrupts per HSP available through LIC. Hogging two of 
them means that only two VMs can access a HSP.
> Would it be safe to clear all of the IE registers to 0 on driver probe?
Nope, the driver should clear only the IE register for the shared 
interrupt that the driver uses. Other IEs are used by other entities.
> If they are indeed separate
> for each processor, it should be fairly easy to keep track of the
> mailboxes used by the kernel and process only those.
Yes, that is what we should do. Again the directionality should be 
specified in DT.
> I'm not entirely clear on what the advantages are of using the per-
> mailbox registers, or how they are supposed to be used. The existing
> documentation doesn't really explain how these are supposed to be used
> either, so I was mostly just going by trial and error.
On virtualized configs, multiple VMs can use same HSP block but each VM 
must use its own interrupt. Because all the IE registers are on the same 
64 KB page, the writes to the page are trapped by hypervisor, which 
means that enabling or disabling an interrupt via the shared IE register 
is pretty heavy operation. The per-mailbox IE registers are on a page 
accessed freely by the IE, no need to trap.

If a VM uses only one empty mailbox interrupt, using a dedicated 
"shared" interrupt and disabling that in GIC could be a lighter 
operation, we used to do that in Parker.

--Pekka

On 10/29/2018 12:39 PM, Thierry Reding wrote:
> On Mon, Oct 29, 2018 at 12:04:22PM +0200, Pekka Pessi wrote:
>> Hi Thierry,
>>
>> There is typically one entity (aux cpu or a VM running on CCPLEX) owning the
>> "empty" or producer side of mailbox (iow, waking up on empty) and another
>> entity owning the "full" or consumer side of mailbox (waking up on full). An
>> entity should not muck with the interrupts used by the opposite side.
> Okay, that explains some of my observations. I was initially trying to
> program interrupt enables for both FULL and EMPTY interrupts for all
> mailboxes, but then I'd usually get timeouts because the consumer wasn't
> responding (i.e. the SPE wasn't getting FULL interrupts for the CCPLEX's
> TX mailbox).
>
> If I understand correctly, you're saying that the CPU should only be
> using the EMPTY interrupts for it's TX mailbox (while leaving the FULL
> interrupts completely untouched) and only the FULL interrupt for it's
> RX mailbox (while leaving the EMPTY interrupts untouched). This is a bit
> of a problem because the mailbox driver doesn't really know anything
> about the direction when it starts up, so how would it make the decision
> about how to program the registers?
>
>> One entity typically owns one shared interrupt only.  For the
>> BPMP/SCE/RCE/SPE HSP blocks the shared interrupt 0 is owned by the auxiliary
>> processor itself, the shared interrupts 1..4 are connected to LIC and are
>> available to other entities. The convention is to go through the interrupts
>> 0..4 and then using the first available shared interrupt for both full and
>> empty.
> That partially matches another of my observations. It seems like we
> can't use the shared interrupt 0 at all on at least the AON HSP. That's
> fine because that HSP instance contains the TX mailbox for TCU and by
> the current convention in the HSP driver, shared interrupt 0 would be
> aggregating the FULL interrupts, which according to the above we don't
> need for TX mailboxes.
>
> What's somewhat surprising is that you're saying that both FULL and
> EMPTY interrupts should be handled by the same shared interrupt. That's
> the opposite of what the recommended programming sequence is that the
> TRM specifies. Why is it better to handle both FULL and EMPTY interrupts
> with the same shared interrupt?
>
>> The interrupt functions should use a mask for mailboxes owned by kernel (in
>> essence what the IE register should be for the HSP shared interrupt owned by
>> the kernel) and serve only those mailboxes owned by kernel. Note that there
>> is no reset for HSP in Xavier, and the IE register contents may be stale.
> Would it be safe to clear all of the IE registers to 0 on driver probe?
> I seem to remember trying to do that and getting similar behaviour to
> what I describe above, namely that interrupts on the SPE weren't working
> anymore. I concluded that the IE register must be shared between the
> various processors, even though that's somewhat suprising given that
> there is no way to synchronize accesses to those registers, so their
> programming would be somewhat up to chance.
>
> Do you know any more about these registers? If they are indeed separate
> for each processor, it should be fairly easy to keep track of the
> mailboxes used by the kernel and process only those. Again I don't know
> how exactly to distinguish between TX and RX mailboxes because they all
> start out the same and only their use defines which direction they go.
> Currently this works because we program them as consumers by default.
> That means we enable the FULL interrupts but keep EMPTY interrupts
> disabled until a message in transmitted on the mailbox, at which point
> we enable the EMPTY interrupt. I suppose at that point we should also
> disable the FULL interrupt, given the above discussion.
>
>> And lastly, if we want to support only Xavier and later, perhaps we should
>> be more clear in the bindings? There are no mailbox-specific interrupt
>> enable registers available on Parker and your design relies on them.
> That was certainly not the intention. I thought I had seen the per-
> mailbox interrupt enable registers also in Tegra186 documentation, but
> after double-checking they're indeed not there. I don't think the driver
> currently "relies" on them because it uses them in addition to the
> HSP_IE registers. I suppose that accessing them might cause aborts on
> Tegra186 if they don't exist, though.
>
> I'm not entirely clear on what the advantages are of using the per-
> mailbox registers, or how they are supposed to be used. The existing
> documentation doesn't really explain how these are supposed to be used
> either, so I was mostly just going by trial and error.
>
> Do you know anything more on how to use these registers? I can easily
> make them Tegra194 specific in the code, but if they're aren't any clear
> advantages, it might just be easier to stick with HSP_IE programming
> only.
>
> Thierry

Re: [PATCH 4/9] mailbox: tegra-hsp: Add support for shared mailboxes

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 10812 bytes --]

On Mon, Oct 29, 2018 at 02:25:42PM +0200, Pekka Pessi wrote:
> Hi Thierry,
> 
> From the 0/9:
> > Are you aware of any others that we need to take into account?
> We would like to use upstream driver for RCE (and probably AON and SCE)
> mailbox handling, too. Eventually.
> > This is a bit
> > of a problem because the mailbox driver doesn't really know anything
> > about the direction when it starts up, so how would it make the decision
> > about how to program the registers?
> I'm afraid that information must be stored in the device tree.
> > What's somewhat surprising is that you're saying that both FULL and
> > EMPTY interrupts should be handled by the same shared interrupt. That's
> > the opposite of what the recommended programming sequence is that the
> > TRM specifies.
> Which TRM you mean? Something here?
> 
> https://p4viewer.nvidia.com/get///hw/ar/doc/Tech_Pubs/Xavier/TRM/
> 
> I browsed through he Xavier_TRM_DP09253001v1.pdf HSP section and I could not
> find any recommendations? But see below.

I'm referring to Xavier_TRM_Public.pdf in that location. If you look at
page 4422, "Shared Interrupts Configuration", the example has aggregated
EMPTY and FULL interrupts on shared interrupts 0 and 1, respectively. I
guess it being an example doesn't make it strictly a recommendation, but
I wonder if we should avoid giving examples that use mappings which we
discourage.

> > Why is it better to handle both FULL and EMPTY interrupts
> > with the same shared interrupt?
> Only top0 has 8 interrupts, rest of the HSP blocks have only 4 (or 5 in case
> of top1) interrupts per HSP available through LIC. Hogging two of them means
> that only two VMs can access a HSP.

Virtualization isn't something that we're very concerned about upstream,
but I'll take that under consideration.

> > Would it be safe to clear all of the IE registers to 0 on driver probe?
> Nope, the driver should clear only the IE register for the shared interrupt
> that the driver uses. Other IEs are used by other entities.

Right, that makes sense. But within that IE register it can consider all
bits fair game, right? One thing I wonder, though, is whether there
should be some external mechanism to set the shared interrupt to use. If
we go purely by convention we'll eventually get this wrong. So currently
we list all available interrupts in DT and then we could pick just the
first. But what if somebody else already picked the first and "owns" it?

I'm not sure we have any practical mechanism to rewrite the DTB, but
perhaps something to keep in mind if ever we need to support other
entities down the road.

> > If they are indeed separate
> > for each processor, it should be fairly easy to keep track of the
> > mailboxes used by the kernel and process only those.
> Yes, that is what we should do. Again the directionality should be specified
> in DT.

Currently we encode the shared mailboxes in DT like this:

	mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_SM 0>,
		 <&hsp_aon TEGRA_HSP_MBOX_TYPE_SM 1>;
	mbox-names = "rx", "tx";

I suppose we could change that to something like:

	mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_SM (0 << 31 | 0)>,
		 <&hsp_aon TEGRA_HSP_MBOX_TYPE_SM (1 << 31 | 0)>;

Where the MSB of the mailbox index would indicate the direction. We
could maybe add some eye-candy to make it easier to read:

	mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_SM TEGRA_HSP_MBOX_SM_RX(0)>,
		 <&hsp_aon TEGRA_HSP_MBOX_TYPE_SM TEGRA_HSP_MBOX_SM_TX(1)>;

That said, I wonder if perhaps it is safe to treat all mailboxes as
consumers by default and omit and direction in DT. If a mailbox is used
as consumer, then the CCPLEX is only interested in FULL interrupts, so
we enable those. The entity that shares the mailbox will write data to
it and is only interested in the EMPTY interrupt, which we don't touch
from the CCPLEX.

For mailboxes that the CCPLEX writes to, we can reconfigure them as
producers by disabling the FULL interrupt and enabling the EMPTY
interrupt instead. We can do that the first time somebody calls the
mbox_send_message() on the mailbox.

Do you see any problems with that approach?

> > I'm not entirely clear on what the advantages are of using the per-
> > mailbox registers, or how they are supposed to be used. The existing
> > documentation doesn't really explain how these are supposed to be used
> > either, so I was mostly just going by trial and error.
> On virtualized configs, multiple VMs can use same HSP block but each VM must
> use its own interrupt. Because all the IE registers are on the same 64 KB
> page, the writes to the page are trapped by hypervisor, which means that
> enabling or disabling an interrupt via the shared IE register is pretty
> heavy operation. The per-mailbox IE registers are on a page accessed freely
> by the IE, no need to trap.

Do the per-mailbox IE registers act as a second level enable, then? The
HSP driver would still have to set the IE register to aggregate FULL and
EMPTY interrupts as needed, but could then use the per-mailbox IE
registers to actually enable and disable (that is, unmask and mask) the
interrupts?

> If a VM uses only one empty mailbox interrupt, using a dedicated "shared"
> interrupt and disabling that in GIC could be a lighter operation, we used to
> do that in Parker.

Okay, that seems like it would somewhat complicate things and we don't
really support VM uses yet, so I think it best to leave that out for
now.

Thierry

> On 10/29/2018 12:39 PM, Thierry Reding wrote:
> > On Mon, Oct 29, 2018 at 12:04:22PM +0200, Pekka Pessi wrote:
> > > Hi Thierry,
> > > 
> > > There is typically one entity (aux cpu or a VM running on CCPLEX) owning the
> > > "empty" or producer side of mailbox (iow, waking up on empty) and another
> > > entity owning the "full" or consumer side of mailbox (waking up on full). An
> > > entity should not muck with the interrupts used by the opposite side.
> > Okay, that explains some of my observations. I was initially trying to
> > program interrupt enables for both FULL and EMPTY interrupts for all
> > mailboxes, but then I'd usually get timeouts because the consumer wasn't
> > responding (i.e. the SPE wasn't getting FULL interrupts for the CCPLEX's
> > TX mailbox).
> > 
> > If I understand correctly, you're saying that the CPU should only be
> > using the EMPTY interrupts for it's TX mailbox (while leaving the FULL
> > interrupts completely untouched) and only the FULL interrupt for it's
> > RX mailbox (while leaving the EMPTY interrupts untouched). This is a bit
> > of a problem because the mailbox driver doesn't really know anything
> > about the direction when it starts up, so how would it make the decision
> > about how to program the registers?
> > 
> > > One entity typically owns one shared interrupt only.  For the
> > > BPMP/SCE/RCE/SPE HSP blocks the shared interrupt 0 is owned by the auxiliary
> > > processor itself, the shared interrupts 1..4 are connected to LIC and are
> > > available to other entities. The convention is to go through the interrupts
> > > 0..4 and then using the first available shared interrupt for both full and
> > > empty.
> > That partially matches another of my observations. It seems like we
> > can't use the shared interrupt 0 at all on at least the AON HSP. That's
> > fine because that HSP instance contains the TX mailbox for TCU and by
> > the current convention in the HSP driver, shared interrupt 0 would be
> > aggregating the FULL interrupts, which according to the above we don't
> > need for TX mailboxes.
> > 
> > What's somewhat surprising is that you're saying that both FULL and
> > EMPTY interrupts should be handled by the same shared interrupt. That's
> > the opposite of what the recommended programming sequence is that the
> > TRM specifies. Why is it better to handle both FULL and EMPTY interrupts
> > with the same shared interrupt?
> > 
> > > The interrupt functions should use a mask for mailboxes owned by kernel (in
> > > essence what the IE register should be for the HSP shared interrupt owned by
> > > the kernel) and serve only those mailboxes owned by kernel. Note that there
> > > is no reset for HSP in Xavier, and the IE register contents may be stale.
> > Would it be safe to clear all of the IE registers to 0 on driver probe?
> > I seem to remember trying to do that and getting similar behaviour to
> > what I describe above, namely that interrupts on the SPE weren't working
> > anymore. I concluded that the IE register must be shared between the
> > various processors, even though that's somewhat suprising given that
> > there is no way to synchronize accesses to those registers, so their
> > programming would be somewhat up to chance.
> > 
> > Do you know any more about these registers? If they are indeed separate
> > for each processor, it should be fairly easy to keep track of the
> > mailboxes used by the kernel and process only those. Again I don't know
> > how exactly to distinguish between TX and RX mailboxes because they all
> > start out the same and only their use defines which direction they go.
> > Currently this works because we program them as consumers by default.
> > That means we enable the FULL interrupts but keep EMPTY interrupts
> > disabled until a message in transmitted on the mailbox, at which point
> > we enable the EMPTY interrupt. I suppose at that point we should also
> > disable the FULL interrupt, given the above discussion.
> > 
> > > And lastly, if we want to support only Xavier and later, perhaps we should
> > > be more clear in the bindings? There are no mailbox-specific interrupt
> > > enable registers available on Parker and your design relies on them.
> > That was certainly not the intention. I thought I had seen the per-
> > mailbox interrupt enable registers also in Tegra186 documentation, but
> > after double-checking they're indeed not there. I don't think the driver
> > currently "relies" on them because it uses them in addition to the
> > HSP_IE registers. I suppose that accessing them might cause aborts on
> > Tegra186 if they don't exist, though.
> > 
> > I'm not entirely clear on what the advantages are of using the per-
> > mailbox registers, or how they are supposed to be used. The existing
> > documentation doesn't really explain how these are supposed to be used
> > either, so I was mostly just going by trial and error.
> > 
> > Do you know anything more on how to use these registers? I can easily
> > make them Tegra194 specific in the code, but if they're aren't any clear
> > advantages, it might just be easier to stick with HSP_IE programming
> > only.
> > 
> > Thierry
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 4/9] mailbox: tegra-hsp: Add support for shared mailboxes

[Pekka Pessi]

> I guess it being an example doesn't make it strictly a recommendation, but
> I wonder if we should avoid giving examples that use mappings which we
> discourage.

Well, yes, that is example for top0 – but I agree, giving examples that 
do not make much sense is not very productive.

> So currently
> we list all available interrupts in DT and then we could pick just the
> first. But what if somebody else already picked the first and "owns" it

We have rather static hardware allocation in device trees, if an another 
VM instance or another aux cpu owns interrupt, we just remove it from 
the DTS or DTSI.  So the native Linux would see all the possible 
interrupts (that are not used by SPE or RCE, for instance), but a 
virtualized one would see only one.

> For mailboxes that the CCPLEX writes to, we can reconfigure them as
> producers by disabling the FULL interrupt and enabling the EMPTY
> interrupt instead. We can do that the first time somebody calls the
> mbox_send_message() on the mailbox.

The problem here is "disabling the FULL interrupt". The main problem are 
the mailbox-specific IE registers, they are not really shared, but the 
consumer owns the full_ie and producer empty_ie. We can not enable or 
disable interrupts without interfering with the interrupt settings of 
the remote end, when the interrupt is enabled, we either modify 
full_ie/empty_ie register or run with a risk that it has incorrect value.

Can we just leave the mbox channel without interrupts? How the consumer 
indicates that it is interested in receiving messages? mbox_peek_data()?

--Pekka



On 10/29/2018 03:16 PM, Thierry Reding wrote:
> On Mon, Oct 29, 2018 at 02:25:42PM +0200, Pekka Pessi wrote:
>> Hi Thierry,
>>
>>  From the 0/9:
>>> Are you aware of any others that we need to take into account?
>> We would like to use upstream driver for RCE (and probably AON and SCE)
>> mailbox handling, too. Eventually.
>>> This is a bit
>>> of a problem because the mailbox driver doesn't really know anything
>>> about the direction when it starts up, so how would it make the decision
>>> about how to program the registers?
>> I'm afraid that information must be stored in the device tree.
>>> What's somewhat surprising is that you're saying that both FULL and
>>> EMPTY interrupts should be handled by the same shared interrupt. That's
>>> the opposite of what the recommended programming sequence is that the
>>> TRM specifies.
>> Which TRM you mean? Something here?
>>
>> https://p4viewer.nvidia.com/get///hw/ar/doc/Tech_Pubs/Xavier/TRM/
>>
>> I browsed through he Xavier_TRM_DP09253001v1.pdf HSP section and I could not
>> find any recommendations? But see below.
> I'm referring to Xavier_TRM_Public.pdf in that location. If you look at
> page 4422, "Shared Interrupts Configuration", the example has aggregated
> EMPTY and FULL interrupts on shared interrupts 0 and 1, respectively. I
> guess it being an example doesn't make it strictly a recommendation, but
> I wonder if we should avoid giving examples that use mappings which we
> discourage.
>
>>> Why is it better to handle both FULL and EMPTY interrupts
>>> with the same shared interrupt?
>> Only top0 has 8 interrupts, rest of the HSP blocks have only 4 (or 5 in case
>> of top1) interrupts per HSP available through LIC. Hogging two of them means
>> that only two VMs can access a HSP.
> Virtualization isn't something that we're very concerned about upstream,
> but I'll take that under consideration.
>
>>> Would it be safe to clear all of the IE registers to 0 on driver probe?
>> Nope, the driver should clear only the IE register for the shared interrupt
>> that the driver uses. Other IEs are used by other entities.
> Right, that makes sense. But within that IE register it can consider all
> bits fair game, right? One thing I wonder, though, is whether there
> should be some external mechanism to set the shared interrupt to use. If
> we go purely by convention we'll eventually get this wrong. So currently
> we list all available interrupts in DT and then we could pick just the
> first. But what if somebody else already picked the first and "owns" it?
>
> I'm not sure we have any practical mechanism to rewrite the DTB, but
> perhaps something to keep in mind if ever we need to support other
> entities down the road.
>
>>> If they are indeed separate
>>> for each processor, it should be fairly easy to keep track of the
>>> mailboxes used by the kernel and process only those.
>> Yes, that is what we should do. Again the directionality should be specified
>> in DT.
> Currently we encode the shared mailboxes in DT like this:
>
> 	mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_SM 0>,
> 		 <&hsp_aon TEGRA_HSP_MBOX_TYPE_SM 1>;
> 	mbox-names = "rx", "tx";
>
> I suppose we could change that to something like:
>
> 	mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_SM (0 << 31 | 0)>,
> 		 <&hsp_aon TEGRA_HSP_MBOX_TYPE_SM (1 << 31 | 0)>;
>
> Where the MSB of the mailbox index would indicate the direction. We
> could maybe add some eye-candy to make it easier to read:
>
> 	mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_SM TEGRA_HSP_MBOX_SM_RX(0)>,
> 		 <&hsp_aon TEGRA_HSP_MBOX_TYPE_SM TEGRA_HSP_MBOX_SM_TX(1)>;
>
> That said, I wonder if perhaps it is safe to treat all mailboxes as
> consumers by default and omit and direction in DT. If a mailbox is used
> as consumer, then the CCPLEX is only interested in FULL interrupts, so
> we enable those. The entity that shares the mailbox will write data to
> it and is only interested in the EMPTY interrupt, which we don't touch
> from the CCPLEX.
>
> For mailboxes that the CCPLEX writes to, we can reconfigure them as
> producers by disabling the FULL interrupt and enabling the EMPTY
> interrupt instead. We can do that the first time somebody calls the
> mbox_send_message() on the mailbox.
>
> Do you see any problems with that approach?
>
>>> I'm not entirely clear on what the advantages are of using the per-
>>> mailbox registers, or how they are supposed to be used. The existing
>>> documentation doesn't really explain how these are supposed to be used
>>> either, so I was mostly just going by trial and error.
>> On virtualized configs, multiple VMs can use same HSP block but each VM must
>> use its own interrupt. Because all the IE registers are on the same 64 KB
>> page, the writes to the page are trapped by hypervisor, which means that
>> enabling or disabling an interrupt via the shared IE register is pretty
>> heavy operation. The per-mailbox IE registers are on a page accessed freely
>> by the IE, no need to trap.
> Do the per-mailbox IE registers act as a second level enable, then? The
> HSP driver would still have to set the IE register to aggregate FULL and
> EMPTY interrupts as needed, but could then use the per-mailbox IE
> registers to actually enable and disable (that is, unmask and mask) the
> interrupts?
>
>> If a VM uses only one empty mailbox interrupt, using a dedicated "shared"
>> interrupt and disabling that in GIC could be a lighter operation, we used to
>> do that in Parker.
> Okay, that seems like it would somewhat complicate things and we don't
> really support VM uses yet, so I think it best to leave that out for
> now.
>
> Thierry
>
>> On 10/29/2018 12:39 PM, Thierry Reding wrote:
>>> On Mon, Oct 29, 2018 at 12:04:22PM +0200, Pekka Pessi wrote:
>>>> Hi Thierry,
>>>>
>>>> There is typically one entity (aux cpu or a VM running on CCPLEX) owning the
>>>> "empty" or producer side of mailbox (iow, waking up on empty) and another
>>>> entity owning the "full" or consumer side of mailbox (waking up on full). An
>>>> entity should not muck with the interrupts used by the opposite side.
>>> Okay, that explains some of my observations. I was initially trying to
>>> program interrupt enables for both FULL and EMPTY interrupts for all
>>> mailboxes, but then I'd usually get timeouts because the consumer wasn't
>>> responding (i.e. the SPE wasn't getting FULL interrupts for the CCPLEX's
>>> TX mailbox).
>>>
>>> If I understand correctly, you're saying that the CPU should only be
>>> using the EMPTY interrupts for it's TX mailbox (while leaving the FULL
>>> interrupts completely untouched) and only the FULL interrupt for it's
>>> RX mailbox (while leaving the EMPTY interrupts untouched). This is a bit
>>> of a problem because the mailbox driver doesn't really know anything
>>> about the direction when it starts up, so how would it make the decision
>>> about how to program the registers?
>>>
>>>> One entity typically owns one shared interrupt only.  For the
>>>> BPMP/SCE/RCE/SPE HSP blocks the shared interrupt 0 is owned by the auxiliary
>>>> processor itself, the shared interrupts 1..4 are connected to LIC and are
>>>> available to other entities. The convention is to go through the interrupts
>>>> 0..4 and then using the first available shared interrupt for both full and
>>>> empty.
>>> That partially matches another of my observations. It seems like we
>>> can't use the shared interrupt 0 at all on at least the AON HSP. That's
>>> fine because that HSP instance contains the TX mailbox for TCU and by
>>> the current convention in the HSP driver, shared interrupt 0 would be
>>> aggregating the FULL interrupts, which according to the above we don't
>>> need for TX mailboxes.
>>>
>>> What's somewhat surprising is that you're saying that both FULL and
>>> EMPTY interrupts should be handled by the same shared interrupt. That's
>>> the opposite of what the recommended programming sequence is that the
>>> TRM specifies. Why is it better to handle both FULL and EMPTY interrupts
>>> with the same shared interrupt?
>>>
>>>> The interrupt functions should use a mask for mailboxes owned by kernel (in
>>>> essence what the IE register should be for the HSP shared interrupt owned by
>>>> the kernel) and serve only those mailboxes owned by kernel. Note that there
>>>> is no reset for HSP in Xavier, and the IE register contents may be stale.
>>> Would it be safe to clear all of the IE registers to 0 on driver probe?
>>> I seem to remember trying to do that and getting similar behaviour to
>>> what I describe above, namely that interrupts on the SPE weren't working
>>> anymore. I concluded that the IE register must be shared between the
>>> various processors, even though that's somewhat suprising given that
>>> there is no way to synchronize accesses to those registers, so their
>>> programming would be somewhat up to chance.
>>>
>>> Do you know any more about these registers? If they are indeed separate
>>> for each processor, it should be fairly easy to keep track of the
>>> mailboxes used by the kernel and process only those. Again I don't know
>>> how exactly to distinguish between TX and RX mailboxes because they all
>>> start out the same and only their use defines which direction they go.
>>> Currently this works because we program them as consumers by default.
>>> That means we enable the FULL interrupts but keep EMPTY interrupts
>>> disabled until a message in transmitted on the mailbox, at which point
>>> we enable the EMPTY interrupt. I suppose at that point we should also
>>> disable the FULL interrupt, given the above discussion.
>>>
>>>> And lastly, if we want to support only Xavier and later, perhaps we should
>>>> be more clear in the bindings? There are no mailbox-specific interrupt
>>>> enable registers available on Parker and your design relies on them.
>>> That was certainly not the intention. I thought I had seen the per-
>>> mailbox interrupt enable registers also in Tegra186 documentation, but
>>> after double-checking they're indeed not there. I don't think the driver
>>> currently "relies" on them because it uses them in addition to the
>>> HSP_IE registers. I suppose that accessing them might cause aborts on
>>> Tegra186 if they don't exist, though.
>>>
>>> I'm not entirely clear on what the advantages are of using the per-
>>> mailbox registers, or how they are supposed to be used. The existing
>>> documentation doesn't really explain how these are supposed to be used
>>> either, so I was mostly just going by trial and error.
>>>
>>> Do you know anything more on how to use these registers? I can easily
>>> make them Tegra194 specific in the code, but if they're aren't any clear
>>> advantages, it might just be easier to stick with HSP_IE programming
>>> only.
>>>
>>> Thierry

Re: [PATCH 7/9] serial: Add Tegra Combined UART driver

[Greg Kroah-Hartman]

On Fri, Oct 26, 2018 at 01:16:36PM +0200, Thierry Reding wrote:
> From: Thierry Reding <treding@nvidia.com>
> 
> The Tegra Combined UART (TCU) is a mailbox-based mechanism that allows
> multiplexing multiple "virtual UARTs" into a single hardware serial
> port. The TCU is the primary serial port on Tegra194 devices.
> 
> Add a TCU driver utilizing the mailbox framework, as the used mailboxes
> are part of Tegra HSP blocks that are already controlled by the Tegra
> HSP mailbox driver.
> 
> Based on work by  Mikko Perttunen <mperttunen@nvidia.com>.
> 
> Signed-off-by: Thierry Reding <treding@nvidia.com>

Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Re: [PATCH 7/9] serial: Add Tegra Combined UART driver

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 2010 bytes --]

On Fri, Nov 09, 2018 at 09:05:26AM -0800, Greg Kroah-Hartman wrote:
> On Fri, Oct 26, 2018 at 01:16:36PM +0200, Thierry Reding wrote:
> > From: Thierry Reding <treding@nvidia.com>
> > 
> > The Tegra Combined UART (TCU) is a mailbox-based mechanism that allows
> > multiplexing multiple "virtual UARTs" into a single hardware serial
> > port. The TCU is the primary serial port on Tegra194 devices.
> > 
> > Add a TCU driver utilizing the mailbox framework, as the used mailboxes
> > are part of Tegra HSP blocks that are already controlled by the Tegra
> > HSP mailbox driver.
> > 
> > Based on work by  Mikko Perttunen <mperttunen@nvidia.com>.
> > 
> > Signed-off-by: Thierry Reding <treding@nvidia.com>
> 
> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Thanks. I just sent out v2 addressing Pekka's comments on the series.
But I'm slightly confused now. Are you expecting anyone else to pick
this up? There are technically no build time dependencies between any
of these patches, so it should be fine to pick them all into the
corresponding subsystem trees.

Perhaps the only thing to note is that there is a runtime dependency
from this patch on patches 1 and 2 in the series. Patch 1 is required to
make blocking a mailbox possible in interrupt context, which we need in
the TCU driver because the TTY and console paths do end up calling the
mbox_send_message() from interrupt context. Patch 2 is only required at
runtime if the two mailboxes are provided by different instances of the
HSP block (which they are for TCU on Tegra194).

Would you prefer for Jassi to pick up the TCU patch along with the
mailbox core and driver changes so that we deal with the runtime
dependencies that way?

Alternatively, if Jassi is okay with the mailbox changes, I can pick up
all of the v2 series of the patches into stable branches for v4.21, deal
with the dependencies there and send out pull requests for everyone to
merge into their subsystem trees.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 7/9] serial: Add Tegra Combined UART driver

[Greg Kroah-Hartman]

On Mon, Nov 12, 2018 at 04:30:14PM +0100, Thierry Reding wrote:
> On Fri, Nov 09, 2018 at 09:05:26AM -0800, Greg Kroah-Hartman wrote:
> > On Fri, Oct 26, 2018 at 01:16:36PM +0200, Thierry Reding wrote:
> > > From: Thierry Reding <treding@nvidia.com>
> > > 
> > > The Tegra Combined UART (TCU) is a mailbox-based mechanism that allows
> > > multiplexing multiple "virtual UARTs" into a single hardware serial
> > > port. The TCU is the primary serial port on Tegra194 devices.
> > > 
> > > Add a TCU driver utilizing the mailbox framework, as the used mailboxes
> > > are part of Tegra HSP blocks that are already controlled by the Tegra
> > > HSP mailbox driver.
> > > 
> > > Based on work by  Mikko Perttunen <mperttunen@nvidia.com>.
> > > 
> > > Signed-off-by: Thierry Reding <treding@nvidia.com>
> > 
> > Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> 
> Thanks. I just sent out v2 addressing Pekka's comments on the series.
> But I'm slightly confused now. Are you expecting anyone else to pick
> this up? There are technically no build time dependencies between any
> of these patches, so it should be fine to pick them all into the
> corresponding subsystem trees.

Ah, I thought there was a built-time dependancy, that's why I gave my
ack.

> Perhaps the only thing to note is that there is a runtime dependency
> from this patch on patches 1 and 2 in the series. Patch 1 is required to
> make blocking a mailbox possible in interrupt context, which we need in
> the TCU driver because the TTY and console paths do end up calling the
> mbox_send_message() from interrupt context. Patch 2 is only required at
> runtime if the two mailboxes are provided by different instances of the
> HSP block (which they are for TCU on Tegra194).
> 
> Would you prefer for Jassi to pick up the TCU patch along with the
> mailbox core and driver changes so that we deal with the runtime
> dependencies that way?
> 
> Alternatively, if Jassi is okay with the mailbox changes, I can pick up
> all of the v2 series of the patches into stable branches for v4.21, deal
> with the dependencies there and send out pull requests for everyone to
> merge into their subsystem trees.

I don't really care either way, whatever is easiest for you all is fine
with me.

thanks,

greg k-h