[PATCH 00/10] usb: host: xhci-tegra: Implement basic ELPG support

[PATCH 00/10] usb: host: xhci-tegra: Implement basic ELPG support

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

Hi,

this series of patches is an attempt to split up JC's large patch adding
ELPG support into more manageable chunks. The original patch can be
found here:

	http://patchwork.ozlabs.org/patch/1115804/

That patch was part of a larger series that also added a couple more
features to more aggressively save power. This series is more limited in
scope and at the same time tries to address most of the comments that
were voiced during review of JC's original posting.

Note that this series is primarily targetted at getting system suspend
and resume to work properly on Jetson TX2 and later, now that we can
actually resume from system sleep on these systems. The code should also
work on Tegra210 and earlier, though Tegra210 is still missing a bit of
the very low-level plumbing to make the system resume properly from
sleep.

I've verified that after this I can successfully use the XHCI on Jetson
TX2 after a system suspend/resume cycle.

Note that I've claimed ownership of these patches because they differ
from JC's patches significantly in many areas and are also a bit
crippled compared to JC's original in terms of features. I've credited
JC in the commit message, but didn't want him to get any blame from me
potentially messing this up.

JC, if you think this is still in line with what you had in mind, let me
know and I'll restore your authorship.

Thierry

Thierry Reding (10):
  usb: host: xhci-tegra: Fix "tega" -> "tegra" typo
  usb: host: xhci-tegra: Separate firmware request and load
  usb: host: xhci-tegra: Avoid a fixed duration sleep
  usb: host: xhci-tegra: Use CNR as firmware ready indicator
  usb: host: xhci-tegra: Extract firmware enable helper
  usb: host: xhci-tegra: Reuse stored register base address
  usb: host: xhci-tegra: Enable runtime PM as late as possible
  usb: host: xhci-tegra: Add support for XUSB context save/restore
  usb: host: xhci-tegra: Add XUSB controller context
  usb: host: xhci-tegra: Implement basic ELPG support

 drivers/usb/host/xhci-tegra.c | 453 +++++++++++++++++++++++++++++-----
 1 file changed, 394 insertions(+), 59 deletions(-)

-- 
2.23.0

[PATCH 01/10] usb: host: xhci-tegra: Fix "tega" -> "tegra" typo

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

The tegra_xusb_mbox_regs structure was misspelled tega_xusb_mbox_regs.
Fortunately this was done consistently so it didn't cause any issues.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 drivers/usb/host/xhci-tegra.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
index bf9065438320..aa1c4e5fd750 100644
--- a/drivers/usb/host/xhci-tegra.c
+++ b/drivers/usb/host/xhci-tegra.c
@@ -145,7 +145,7 @@ struct tegra_xusb_phy_type {
 	unsigned int num;
 };
 
-struct tega_xusb_mbox_regs {
+struct tegra_xusb_mbox_regs {
 	u16 cmd;
 	u16 data_in;
 	u16 data_out;
@@ -166,7 +166,7 @@ struct tegra_xusb_soc {
 		} usb2, ulpi, hsic, usb3;
 	} ports;
 
-	struct tega_xusb_mbox_regs mbox;
+	struct tegra_xusb_mbox_regs mbox;
 
 	bool scale_ss_clock;
 	bool has_ipfs;
-- 
2.23.0

Re: [PATCH 01/10] usb: host: xhci-tegra: Fix "tega" -> "tegra" typo

[JC Kuo]

Reviewed-by: JC Kuo <jckuo@nvidia.com>

On 11/25/19 8:32 PM, Thierry Reding wrote:
> From: Thierry Reding <treding@nvidia.com>
> 
> The tegra_xusb_mbox_regs structure was misspelled tega_xusb_mbox_regs.
> Fortunately this was done consistently so it didn't cause any issues.
> 
> Signed-off-by: Thierry Reding <treding@nvidia.com>
> ---
>  drivers/usb/host/xhci-tegra.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
> index bf9065438320..aa1c4e5fd750 100644
> --- a/drivers/usb/host/xhci-tegra.c
> +++ b/drivers/usb/host/xhci-tegra.c
> @@ -145,7 +145,7 @@ struct tegra_xusb_phy_type {
>  	unsigned int num;
>  };
>  
> -struct tega_xusb_mbox_regs {
> +struct tegra_xusb_mbox_regs {
>  	u16 cmd;
>  	u16 data_in;
>  	u16 data_out;
> @@ -166,7 +166,7 @@ struct tegra_xusb_soc {
>  		} usb2, ulpi, hsic, usb3;
>  	} ports;
>  
> -	struct tega_xusb_mbox_regs mbox;
> +	struct tegra_xusb_mbox_regs mbox;
>  
>  	bool scale_ss_clock;
>  	bool has_ipfs;
> 

[PATCH 02/10] usb: host: xhci-tegra: Separate firmware request and load

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

Subsequent patches for system suspend/resume support will need to reload
the firmware on resume. Since the firmware remains in system memory, the
driver doesn't need to reload it from the filesystem. However, the XUSB
controller will be reset across suspend/resume, so it needs to load the
firmware into its microcontroller on resume.

Split the firmware request and the firmware load code into two separate
functions so that the driver can reuse the firmware in system memory to
reload the microcontroller on resume.

Based on work by JC Kuo <jckuo@nvidia.com>.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 drivers/usb/host/xhci-tegra.c | 40 ++++++++++++++++++++++++++---------
 1 file changed, 30 insertions(+), 10 deletions(-)

diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
index aa1c4e5fd750..5cfd54862670 100644
--- a/drivers/usb/host/xhci-tegra.c
+++ b/drivers/usb/host/xhci-tegra.c
@@ -793,17 +793,10 @@ static int tegra_xusb_runtime_resume(struct device *dev)
 	return err;
 }
 
-static int tegra_xusb_load_firmware(struct tegra_xusb *tegra)
+static int tegra_xusb_request_firmware(struct tegra_xusb *tegra)
 {
-	unsigned int code_tag_blocks, code_size_blocks, code_blocks;
 	struct tegra_xusb_fw_header *header;
-	struct device *dev = tegra->dev;
 	const struct firmware *fw;
-	unsigned long timeout;
-	time64_t timestamp;
-	struct tm time;
-	u64 address;
-	u32 value;
 	int err;
 
 	err = request_firmware(&fw, tegra->soc->firmware, tegra->dev);
@@ -828,6 +821,24 @@ static int tegra_xusb_load_firmware(struct tegra_xusb *tegra)
 	memcpy(tegra->fw.virt, fw->data, tegra->fw.size);
 	release_firmware(fw);
 
+	return 0;
+}
+
+static int tegra_xusb_load_firmware(struct tegra_xusb *tegra)
+{
+	unsigned int code_tag_blocks, code_size_blocks, code_blocks;
+	struct tegra_xusb_fw_header *header;
+	struct xhci_cap_regs __iomem *cap;
+	struct xhci_op_regs __iomem *op;
+	struct device *dev = tegra->dev;
+	unsigned long timeout;
+	time64_t timestamp;
+	struct tm time;
+	u64 address;
+	u32 value;
+
+	header = (struct tegra_xusb_fw_header *)tegra->fw.virt;
+
 	if (csb_readl(tegra, XUSB_CSB_MP_ILOAD_BASE_LO) != 0) {
 		dev_info(dev, "Firmware already loaded, Falcon state %#x\n",
 			 csb_readl(tegra, XUSB_FALC_CPUCTL));
@@ -1208,10 +1219,16 @@ static int tegra_xusb_probe(struct platform_device *pdev)
 		goto put_rpm;
 	}
 
+	err = tegra_xusb_request_firmware(tegra);
+	if (err < 0) {
+		dev_err(&pdev->dev, "failed to request firmware: %d\n", err);
+		goto disable_phy;
+	}
+
 	err = tegra_xusb_load_firmware(tegra);
 	if (err < 0) {
 		dev_err(&pdev->dev, "failed to load firmware: %d\n", err);
-		goto put_rpm;
+		goto free_firmware;
 	}
 
 	tegra->hcd->regs = tegra->regs;
@@ -1221,7 +1238,7 @@ static int tegra_xusb_probe(struct platform_device *pdev)
 	err = usb_add_hcd(tegra->hcd, tegra->xhci_irq, IRQF_SHARED);
 	if (err < 0) {
 		dev_err(&pdev->dev, "failed to add USB HCD: %d\n", err);
-		goto put_rpm;
+		goto free_firmware;
 	}
 
 	device_wakeup_enable(tegra->hcd->self.controller);
@@ -1281,6 +1298,9 @@ static int tegra_xusb_probe(struct platform_device *pdev)
 		tegra_xusb_runtime_suspend(&pdev->dev);
 put_hcd:
 	usb_put_hcd(tegra->hcd);
+free_firmware:
+	dma_free_coherent(&pdev->dev, tegra->fw.size, tegra->fw.virt,
+			  tegra->fw.phys);
 disable_phy:
 	tegra_xusb_phy_disable(tegra);
 	pm_runtime_disable(&pdev->dev);
-- 
2.23.0

[PATCH 03/10] usb: host: xhci-tegra: Avoid a fixed duration sleep

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

Do not use a fixed duration sleep to wait for the DMA controller to
become ready. Instead, poll the L2IMEMOP_RESULT register for the VLD
flag to determine when the XUSB controller's DMA master is ready.

Based on work by JC Kuo <jckuo@nvidia.com>.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 drivers/usb/host/xhci-tegra.c | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
index 5cfd54862670..d15fd16168ae 100644
--- a/drivers/usb/host/xhci-tegra.c
+++ b/drivers/usb/host/xhci-tegra.c
@@ -101,6 +101,8 @@
 #define  L2IMEMOP_ACTION_SHIFT			24
 #define  L2IMEMOP_INVALIDATE_ALL		(0x40 << L2IMEMOP_ACTION_SHIFT)
 #define  L2IMEMOP_LOAD_LOCKED_RESULT		(0x11 << L2IMEMOP_ACTION_SHIFT)
+#define XUSB_CSB_MEMPOOL_L2IMEMOP_RESULT	0x101a18
+#define  L2IMEMOP_RESULT_VLD			BIT(31)
 #define XUSB_CSB_MP_APMAP			0x10181c
 #define  APMAP_BOOTPATH				BIT(31)
 
@@ -893,7 +895,22 @@ static int tegra_xusb_load_firmware(struct tegra_xusb *tegra)
 
 	csb_writel(tegra, 0, XUSB_FALC_DMACTL);
 
-	msleep(50);
+	/* wait for RESULT_VLD to get set */
+	timeout = jiffies + msecs_to_jiffies(10);
+
+	do {
+		value = csb_readl(tegra, XUSB_CSB_MEMPOOL_L2IMEMOP_RESULT);
+		if (value & L2IMEMOP_RESULT_VLD)
+			break;
+
+		usleep_range(50, 100);
+	} while (time_is_after_jiffies(timeout));
+
+	value = csb_readl(tegra, XUSB_CSB_MEMPOOL_L2IMEMOP_RESULT);
+	if ((value & L2IMEMOP_RESULT_VLD) == 0) {
+		dev_err(dev, "DMA controller not ready %#010x\n", value);
+		return -EBUSY;
+	}
 
 	csb_writel(tegra, le32_to_cpu(header->boot_codetag),
 		   XUSB_FALC_BOOTVEC);
-- 
2.23.0

Re: [PATCH 03/10] usb: host: xhci-tegra: Avoid a fixed duration sleep

[Mikko Perttunen]

On 25.11.2019 14.32, Thierry Reding wrote:
> From: Thierry Reding <treding@nvidia.com>
> 
> Do not use a fixed duration sleep to wait for the DMA controller to
> become ready. Instead, poll the L2IMEMOP_RESULT register for the VLD
> flag to determine when the XUSB controller's DMA master is ready.
> 
> Based on work by JC Kuo <jckuo@nvidia.com>.
> 
> Signed-off-by: Thierry Reding <treding@nvidia.com>
> ---
>   drivers/usb/host/xhci-tegra.c | 19 ++++++++++++++++++-
>   1 file changed, 18 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
> index 5cfd54862670..d15fd16168ae 100644
> --- a/drivers/usb/host/xhci-tegra.c
> +++ b/drivers/usb/host/xhci-tegra.c
> @@ -101,6 +101,8 @@
>   #define  L2IMEMOP_ACTION_SHIFT			24
>   #define  L2IMEMOP_INVALIDATE_ALL		(0x40 << L2IMEMOP_ACTION_SHIFT)
>   #define  L2IMEMOP_LOAD_LOCKED_RESULT		(0x11 << L2IMEMOP_ACTION_SHIFT)
> +#define XUSB_CSB_MEMPOOL_L2IMEMOP_RESULT	0x101a18
> +#define  L2IMEMOP_RESULT_VLD			BIT(31)
>   #define XUSB_CSB_MP_APMAP			0x10181c
>   #define  APMAP_BOOTPATH				BIT(31)
>   
> @@ -893,7 +895,22 @@ static int tegra_xusb_load_firmware(struct tegra_xusb *tegra)
>   
>   	csb_writel(tegra, 0, XUSB_FALC_DMACTL);
>   
> -	msleep(50);
> +	/* wait for RESULT_VLD to get set */
> +	timeout = jiffies + msecs_to_jiffies(10);
> +
> +	do {
> +		value = csb_readl(tegra, XUSB_CSB_MEMPOOL_L2IMEMOP_RESULT);
> +		if (value & L2IMEMOP_RESULT_VLD)
> +			break;
> +
> +		usleep_range(50, 100);
> +	} while (time_is_after_jiffies(timeout));
> +
> +	value = csb_readl(tegra, XUSB_CSB_MEMPOOL_L2IMEMOP_RESULT);
> +	if ((value & L2IMEMOP_RESULT_VLD) == 0) {
> +		dev_err(dev, "DMA controller not ready %#010x\n", value);
> +		return -EBUSY;
> +	}
>   
>   	csb_writel(tegra, le32_to_cpu(header->boot_codetag),
>   		   XUSB_FALC_BOOTVEC);
> 

readx_poll_timeout? I think you can define a local macro to "curry" the 
'tegra' argument to csb_readl..

#define tegra_csb_readl(x) csb_readl(tegra, x)
readx_poll_timeout(tegra_csb_readl, ...)

Cheers,
Mikko

[PATCH 04/10] usb: host: xhci-tegra: Use CNR as firmware ready indicator

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

The Falcon CPU state is a suboptimal indicator for firmware readiness,
since the Falcon can be in a running state if the firmware is handling
port state changes or running other tasks. Instead, the driver should
check the STS_CNR bit to determine whether or not the firmware has been
successfully loaded and is ready for XHCI operation.

Based on work by JC Kuo <jckuo@nvidia.com>.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 drivers/usb/host/xhci-tegra.c | 27 +++++++++++++++------------
 1 file changed, 15 insertions(+), 12 deletions(-)

diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
index d15fd16168ae..85642806fa2a 100644
--- a/drivers/usb/host/xhci-tegra.c
+++ b/drivers/usb/host/xhci-tegra.c
@@ -829,10 +829,10 @@ static int tegra_xusb_request_firmware(struct tegra_xusb *tegra)
 static int tegra_xusb_load_firmware(struct tegra_xusb *tegra)
 {
 	unsigned int code_tag_blocks, code_size_blocks, code_blocks;
+	struct xhci_cap_regs __iomem *cap = tegra->regs;
 	struct tegra_xusb_fw_header *header;
-	struct xhci_cap_regs __iomem *cap;
-	struct xhci_op_regs __iomem *op;
 	struct device *dev = tegra->dev;
+	struct xhci_op_regs __iomem *op;
 	unsigned long timeout;
 	time64_t timestamp;
 	struct tm time;
@@ -840,6 +840,7 @@ static int tegra_xusb_load_firmware(struct tegra_xusb *tegra)
 	u32 value;
 
 	header = (struct tegra_xusb_fw_header *)tegra->fw.virt;
+	op = tegra->regs + HC_LENGTH(readl(&cap->hc_capbase));
 
 	if (csb_readl(tegra, XUSB_CSB_MP_ILOAD_BASE_LO) != 0) {
 		dev_info(dev, "Firmware already loaded, Falcon state %#x\n",
@@ -915,21 +916,23 @@ static int tegra_xusb_load_firmware(struct tegra_xusb *tegra)
 	csb_writel(tegra, le32_to_cpu(header->boot_codetag),
 		   XUSB_FALC_BOOTVEC);
 
-	/* Boot Falcon CPU and wait for it to enter the STOPPED (idle) state. */
-	timeout = jiffies + msecs_to_jiffies(5);
-
+	/* Boot Falcon CPU and wait for USBSTS_CNR to get cleared. */
 	csb_writel(tegra, CPUCTL_STARTCPU, XUSB_FALC_CPUCTL);
 
-	while (time_before(jiffies, timeout)) {
-		if (csb_readl(tegra, XUSB_FALC_CPUCTL) == CPUCTL_STATE_STOPPED)
+	timeout = jiffies + msecs_to_jiffies(200);
+
+	do {
+		value = readl(&op->status);
+		if ((value & STS_CNR) == 0)
 			break;
 
-		usleep_range(100, 200);
-	}
+		usleep_range(1000, 2000);
+	} while (time_is_after_jiffies(timeout));
 
-	if (csb_readl(tegra, XUSB_FALC_CPUCTL) != CPUCTL_STATE_STOPPED) {
-		dev_err(dev, "Falcon failed to start, state: %#x\n",
-			csb_readl(tegra, XUSB_FALC_CPUCTL));
+	value = readl(&op->status);
+	if (value & STS_CNR) {
+		value = csb_readl(tegra, XUSB_FALC_CPUCTL);
+		dev_err(dev, "XHCI controller not read: %#010x\n", value);
 		return -EIO;
 	}
 
-- 
2.23.0

[PATCH 05/10] usb: host: xhci-tegra: Extract firmware enable helper

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

Extract a helper that enables message generation from the firmware. This
removes clutter from tegra_xusb_probe() and will also come in useful for
subsequent patches that introduce suspend/resume support.

Based on work by JC Kuo <jckuo@nvidia.com>.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 drivers/usb/host/xhci-tegra.c | 41 +++++++++++++++++++++++++----------
 1 file changed, 29 insertions(+), 12 deletions(-)

diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
index 85642806fa2a..4244367d3573 100644
--- a/drivers/usb/host/xhci-tegra.c
+++ b/drivers/usb/host/xhci-tegra.c
@@ -997,11 +997,37 @@ static int tegra_xusb_powerdomain_init(struct device *dev,
 	return 0;
 }
 
-static int tegra_xusb_probe(struct platform_device *pdev)
+static int __tegra_xusb_enable_firmware_messages(struct tegra_xusb *tegra)
 {
 	struct tegra_xusb_mbox_msg msg;
-	struct resource *regs;
+	int err;
+
+	/* Enable firmware messages from controller. */
+	msg.cmd = MBOX_CMD_MSG_ENABLED;
+	msg.data = 0;
+
+	err = tegra_xusb_mbox_send(tegra, &msg);
+	if (err < 0)
+		dev_err(tegra->dev, "failed to enable messages: %d\n", err);
+
+	return err;
+}
+
+static int tegra_xusb_enable_firmware_messages(struct tegra_xusb *tegra)
+{
+	int err;
+
+	mutex_lock(&tegra->lock);
+	err = __tegra_xusb_enable_firmware_messages(tegra);
+	mutex_unlock(&tegra->lock);
+
+	return err;
+}
+
+static int tegra_xusb_probe(struct platform_device *pdev)
+{
 	struct tegra_xusb *tegra;
+	struct resource *regs;
 	struct xhci_hcd *xhci;
 	unsigned int i, j, k;
 	struct phy *phy;
@@ -1281,21 +1307,12 @@ static int tegra_xusb_probe(struct platform_device *pdev)
 		goto put_usb3;
 	}
 
-	mutex_lock(&tegra->lock);
-
-	/* Enable firmware messages from controller. */
-	msg.cmd = MBOX_CMD_MSG_ENABLED;
-	msg.data = 0;
-
-	err = tegra_xusb_mbox_send(tegra, &msg);
+	err = tegra_xusb_enable_firmware_messages(tegra);
 	if (err < 0) {
 		dev_err(&pdev->dev, "failed to enable messages: %d\n", err);
-		mutex_unlock(&tegra->lock);
 		goto remove_usb3;
 	}
 
-	mutex_unlock(&tegra->lock);
-
 	err = devm_request_threaded_irq(&pdev->dev, tegra->mbox_irq,
 					tegra_xusb_mbox_irq,
 					tegra_xusb_mbox_thread, 0,
-- 
2.23.0

[PATCH 06/10] usb: host: xhci-tegra: Reuse stored register base address

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

The base address of the XUSB controller's registers is already stored in
the HCD. Move assignment to the HCD fields to an earlier point so that
they can be reused in the tegra_xusb_config() function. This avoids the
need to pass the base address as an extra parameter, which in turn comes
in handy in subsequent patches that need to call this function from the
suspend/resume paths where these values are no longer readily available.

Based on work by JC Kuo <jckuo@nvidia.com>.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 drivers/usb/host/xhci-tegra.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
index 4244367d3573..5eca3ea0e8b2 100644
--- a/drivers/usb/host/xhci-tegra.c
+++ b/drivers/usb/host/xhci-tegra.c
@@ -625,9 +625,9 @@ static irqreturn_t tegra_xusb_mbox_thread(int irq, void *data)
 	return IRQ_HANDLED;
 }
 
-static void tegra_xusb_config(struct tegra_xusb *tegra,
-			      struct resource *regs)
+static void tegra_xusb_config(struct tegra_xusb *tegra)
 {
+	u32 regs = tegra->hcd->rsrc_start;
 	u32 value;
 
 	if (tegra->soc->has_ipfs) {
@@ -641,7 +641,7 @@ static void tegra_xusb_config(struct tegra_xusb *tegra,
 	/* Program BAR0 space */
 	value = fpci_readl(tegra, XUSB_CFG_4);
 	value &= ~(XUSB_BASE_ADDR_MASK << XUSB_BASE_ADDR_SHIFT);
-	value |= regs->start & (XUSB_BASE_ADDR_MASK << XUSB_BASE_ADDR_SHIFT);
+	value |= regs & (XUSB_BASE_ADDR_MASK << XUSB_BASE_ADDR_SHIFT);
 	fpci_writel(tegra, value, XUSB_CFG_4);
 
 	usleep_range(100, 200);
@@ -1230,6 +1230,10 @@ static int tegra_xusb_probe(struct platform_device *pdev)
 		goto put_powerdomains;
 	}
 
+	tegra->hcd->regs = tegra->regs;
+	tegra->hcd->rsrc_start = regs->start;
+	tegra->hcd->rsrc_len = resource_size(regs);
+
 	/*
 	 * This must happen after usb_create_hcd(), because usb_create_hcd()
 	 * will overwrite the drvdata of the device with the hcd it creates.
@@ -1253,7 +1257,7 @@ static int tegra_xusb_probe(struct platform_device *pdev)
 		goto disable_phy;
 	}
 
-	tegra_xusb_config(tegra, regs);
+	tegra_xusb_config(tegra);
 
 	/*
 	 * The XUSB Falcon microcontroller can only address 40 bits, so set
@@ -1277,10 +1281,6 @@ static int tegra_xusb_probe(struct platform_device *pdev)
 		goto free_firmware;
 	}
 
-	tegra->hcd->regs = tegra->regs;
-	tegra->hcd->rsrc_start = regs->start;
-	tegra->hcd->rsrc_len = resource_size(regs);
-
 	err = usb_add_hcd(tegra->hcd, tegra->xhci_irq, IRQF_SHARED);
 	if (err < 0) {
 		dev_err(&pdev->dev, "failed to add USB HCD: %d\n", err);
-- 
2.23.0

[PATCH 07/10] usb: host: xhci-tegra: Enable runtime PM as late as possible

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

A number of things can currently go wrong after the XUSB controller has
been enabled, which means that it might need to be disabled again before
it has ever been used.

Avoid this by delaying runtime PM enablement until it's really required
right before registers are accessed for the first time.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 drivers/usb/host/xhci-tegra.c | 33 +++++++++++++++++----------------
 1 file changed, 17 insertions(+), 16 deletions(-)

diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
index 5eca3ea0e8b2..f043aab7bf53 100644
--- a/drivers/usb/host/xhci-tegra.c
+++ b/drivers/usb/host/xhci-tegra.c
@@ -1246,19 +1246,6 @@ static int tegra_xusb_probe(struct platform_device *pdev)
 		goto put_hcd;
 	}
 
-	pm_runtime_enable(&pdev->dev);
-	if (pm_runtime_enabled(&pdev->dev))
-		err = pm_runtime_get_sync(&pdev->dev);
-	else
-		err = tegra_xusb_runtime_resume(&pdev->dev);
-
-	if (err < 0) {
-		dev_err(&pdev->dev, "failed to enable device: %d\n", err);
-		goto disable_phy;
-	}
-
-	tegra_xusb_config(tegra);
-
 	/*
 	 * The XUSB Falcon microcontroller can only address 40 bits, so set
 	 * the DMA mask accordingly.
@@ -1266,7 +1253,7 @@ static int tegra_xusb_probe(struct platform_device *pdev)
 	err = dma_set_mask_and_coherent(tegra->dev, DMA_BIT_MASK(40));
 	if (err < 0) {
 		dev_err(&pdev->dev, "failed to set DMA mask: %d\n", err);
-		goto put_rpm;
+		goto disable_phy;
 	}
 
 	err = tegra_xusb_request_firmware(tegra);
@@ -1275,16 +1262,30 @@ static int tegra_xusb_probe(struct platform_device *pdev)
 		goto disable_phy;
 	}
 
+	pm_runtime_enable(&pdev->dev);
+
+	if (!pm_runtime_enabled(&pdev->dev))
+		err = tegra_xusb_runtime_resume(&pdev->dev);
+	else
+		err = pm_runtime_get_sync(&pdev->dev);
+
+	if (err < 0) {
+		dev_err(&pdev->dev, "failed to enable device: %d\n", err);
+		goto free_firmware;
+	}
+
+	tegra_xusb_config(tegra);
+
 	err = tegra_xusb_load_firmware(tegra);
 	if (err < 0) {
 		dev_err(&pdev->dev, "failed to load firmware: %d\n", err);
-		goto free_firmware;
+		goto put_rpm;
 	}
 
 	err = usb_add_hcd(tegra->hcd, tegra->xhci_irq, IRQF_SHARED);
 	if (err < 0) {
 		dev_err(&pdev->dev, "failed to add USB HCD: %d\n", err);
-		goto free_firmware;
+		goto put_rpm;
 	}
 
 	device_wakeup_enable(tegra->hcd->self.controller);
-- 
2.23.0

[PATCH 08/10] usb: host: xhci-tegra: Add support for XUSB context save/restore

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

The XUSB controller contains registers that need to be saved on suspend
and restored on resume in addition to the XHCI specific registers. Add
support for saving and restoring the XUSB specific context.

Based on work by JC Kuo <jckuo@nvidia.com>.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 drivers/usb/host/xhci-tegra.c | 102 +++++++++++++++++++++++++++++++++-
 1 file changed, 100 insertions(+), 2 deletions(-)

diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
index f043aab7bf53..be1b47fadb3b 100644
--- a/drivers/usb/host/xhci-tegra.c
+++ b/drivers/usb/host/xhci-tegra.c
@@ -154,12 +154,25 @@ struct tegra_xusb_mbox_regs {
 	u16 owner;
 };
 
+struct tegra_xusb_context_soc {
+	struct {
+		const unsigned int *offsets;
+		unsigned int num_offsets;
+	} ipfs;
+
+	struct {
+		const unsigned int *offsets;
+		unsigned int num_offsets;
+	} fpci;
+};
+
 struct tegra_xusb_soc {
 	const char *firmware;
 	const char * const *supply_names;
 	unsigned int num_supplies;
 	const struct tegra_xusb_phy_type *phy_types;
 	unsigned int num_types;
+	const struct tegra_xusb_context_soc *context;
 
 	struct {
 		struct {
@@ -174,6 +187,11 @@ struct tegra_xusb_soc {
 	bool has_ipfs;
 };
 
+struct tegra_xusb_context {
+	u32 *ipfs;
+	u32 *fpci;
+};
+
 struct tegra_xusb {
 	struct device *dev;
 	void __iomem *regs;
@@ -220,6 +238,8 @@ struct tegra_xusb {
 		void *virt;
 		dma_addr_t phys;
 	} fw;
+
+	struct tegra_xusb_context context;
 };
 
 static struct hc_driver __read_mostly tegra_xhci_hc_driver;
@@ -795,6 +815,37 @@ static int tegra_xusb_runtime_resume(struct device *dev)
 	return err;
 }
 
+#ifdef CONFIG_PM_SLEEP
+static int tegra_xusb_init_context(struct tegra_xusb *tegra)
+{
+	const struct tegra_xusb_context_soc *soc = tegra->soc->context;
+
+	/*
+	 * Skip support for context save/restore if the SoC doesn't have any
+	 * XUSB specific context that needs to be saved/restored.
+	 */
+	if (!soc)
+		return 0;
+
+	tegra->context.ipfs = devm_kcalloc(tegra->dev, soc->ipfs.num_offsets,
+					   sizeof(u32), GFP_KERNEL);
+	if (!tegra->context.ipfs)
+		return -ENOMEM;
+
+	tegra->context.fpci = devm_kcalloc(tegra->dev, soc->ipfs.num_offsets,
+					   sizeof(u32), GFP_KERNEL);
+	if (!tegra->context.fpci)
+		return -ENOMEM;
+
+	return 0;
+}
+#else
+static inline int tegra_xusb_init_context(struct tegra_xusb *tegra)
+{
+	return 0;
+}
+#endif
+
 static int tegra_xusb_request_firmware(struct tegra_xusb *tegra)
 {
 	struct tegra_xusb_fw_header *header;
@@ -1043,6 +1094,10 @@ static int tegra_xusb_probe(struct platform_device *pdev)
 	mutex_init(&tegra->lock);
 	tegra->dev = &pdev->dev;
 
+	err = tegra_xusb_init_context(tegra);
+	if (err < 0)
+		return err;
+
 	regs = platform_get_resource(pdev, IORESOURCE_MEM, 0);
 	tegra->regs = devm_ioremap_resource(&pdev->dev, regs);
 	if (IS_ERR(tegra->regs))
@@ -1386,14 +1441,55 @@ static int tegra_xusb_remove(struct platform_device *pdev)
 }
 
 #ifdef CONFIG_PM_SLEEP
+static void tegra_xusb_save_context(struct tegra_xusb *tegra)
+{
+	const struct tegra_xusb_context_soc *soc = tegra->soc->context;
+	struct tegra_xusb_context *ctx = &tegra->context;
+	unsigned int i;
+
+	if (soc && soc->ipfs.num_offsets > 0) {
+		for (i = 0; i < soc->ipfs.num_offsets; i++)
+			ctx->ipfs[i] = ipfs_readl(tegra, soc->ipfs.offsets[i]);
+	}
+
+	if (soc && soc->fpci.num_offsets > 0) {
+		for (i = 0; i < soc->fpci.num_offsets; i++)
+			ctx->fpci[i] = fpci_readl(tegra, soc->fpci.offsets[i]);
+	}
+}
+
+static void tegra_xusb_restore_context(struct tegra_xusb *tegra)
+{
+	const struct tegra_xusb_context_soc *soc = tegra->soc->context;
+	struct tegra_xusb_context *ctx = &tegra->context;
+	unsigned int i;
+
+	if (soc && soc->fpci.num_offsets > 0) {
+		for (i = 0; i < soc->fpci.num_offsets; i++)
+			fpci_writel(tegra, ctx->fpci[i], soc->fpci.offsets[i]);
+	}
+
+	if (soc && soc->ipfs.num_offsets > 0) {
+		for (i = 0; i < soc->ipfs.num_offsets; i++)
+			ipfs_writel(tegra, ctx->ipfs[i], soc->ipfs.offsets[i]);
+	}
+}
+
 static int tegra_xusb_suspend(struct device *dev)
 {
 	struct tegra_xusb *tegra = dev_get_drvdata(dev);
 	struct xhci_hcd *xhci = hcd_to_xhci(tegra->hcd);
 	bool wakeup = device_may_wakeup(dev);
+	int err;
 
 	/* TODO: Powergate controller across suspend/resume. */
-	return xhci_suspend(xhci, wakeup);
+	err = xhci_suspend(xhci, wakeup);
+	if (err < 0)
+		return err;
+
+	tegra_xusb_save_context(tegra);
+
+	return 0;
 }
 
 static int tegra_xusb_resume(struct device *dev)
@@ -1401,7 +1497,9 @@ static int tegra_xusb_resume(struct device *dev)
 	struct tegra_xusb *tegra = dev_get_drvdata(dev);
 	struct xhci_hcd *xhci = hcd_to_xhci(tegra->hcd);
 
-	return xhci_resume(xhci, 0);
+	tegra_xusb_restore_context(tegra);
+
+	return xhci_resume(xhci, false);
 }
 #endif
 
-- 
2.23.0

[PATCH 09/10] usb: host: xhci-tegra: Add XUSB controller context

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

Define the offsets of the registers that need to be saved on suspend and
restored on resume for the various NVIDIA Tegra generations supported by
the XUSB driver.

Based on work by JC Kuo <jckuo@nvidia.com>.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 drivers/usb/host/xhci-tegra.c | 80 ++++++++++++++++++++++++++++++-----
 1 file changed, 69 insertions(+), 11 deletions(-)

diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
index be1b47fadb3b..cd3afec408ea 100644
--- a/drivers/usb/host/xhci-tegra.c
+++ b/drivers/usb/host/xhci-tegra.c
@@ -38,7 +38,15 @@
 #define XUSB_CFG_4				0x010
 #define  XUSB_BASE_ADDR_SHIFT			15
 #define  XUSB_BASE_ADDR_MASK			0x1ffff
+#define XUSB_CFG_16				0x040
+#define XUSB_CFG_24				0x060
+#define XUSB_CFG_AXI_CFG			0x0f8
 #define XUSB_CFG_ARU_C11_CSBRANGE		0x41c
+#define XUSB_CFG_ARU_CONTEXT			0x43c
+#define XUSB_CFG_ARU_CONTEXT_HS_PLS		0x478
+#define XUSB_CFG_ARU_CONTEXT_FS_PLS		0x47c
+#define XUSB_CFG_ARU_CONTEXT_HSFS_SPEED		0x480
+#define XUSB_CFG_ARU_CONTEXT_HSFS_PP		0x484
 #define XUSB_CFG_CSB_BASE_ADDR			0x800
 
 /* FPCI mailbox registers */
@@ -62,11 +70,20 @@
 #define  MBOX_SMI_INTR_EN			BIT(3)
 
 /* IPFS registers */
+#define IPFS_XUSB_HOST_MSI_BAR_SZ_0		0x0c0
+#define IPFS_XUSB_HOST_MSI_AXI_BAR_ST_0		0x0c4
+#define IPFS_XUSB_HOST_MSI_FPCI_BAR_ST_0	0x0c8
+#define IPFS_XUSB_HOST_MSI_VEC0_0		0x100
+#define IPFS_XUSB_HOST_MSI_EN_VEC0_0		0x140
 #define IPFS_XUSB_HOST_CONFIGURATION_0		0x180
 #define  IPFS_EN_FPCI				BIT(0)
+#define IPFS_XUSB_HOST_FPCI_ERROR_MASKS_0	0x184
 #define IPFS_XUSB_HOST_INTR_MASK_0		0x188
 #define  IPFS_IP_INT_MASK			BIT(16)
+#define IPFS_XUSB_HOST_INTR_ENABLE_0		0x198
+#define IPFS_XUSB_HOST_UFPCI_CONFIG_0		0x19c
 #define IPFS_XUSB_HOST_CLKGATE_HYSTERESIS_0	0x1bc
+#define IPFS_XUSB_HOST_MCCIF_FIFOCTRL_0		0x1dc
 
 #define CSB_PAGE_SELECT_MASK			0x7fffff
 #define CSB_PAGE_SELECT_SHIFT			9
@@ -820,13 +837,6 @@ static int tegra_xusb_init_context(struct tegra_xusb *tegra)
 {
 	const struct tegra_xusb_context_soc *soc = tegra->soc->context;
 
-	/*
-	 * Skip support for context save/restore if the SoC doesn't have any
-	 * XUSB specific context that needs to be saved/restored.
-	 */
-	if (!soc)
-		return 0;
-
 	tegra->context.ipfs = devm_kcalloc(tegra->dev, soc->ipfs.num_offsets,
 					   sizeof(u32), GFP_KERNEL);
 	if (!tegra->context.ipfs)
@@ -1447,12 +1457,12 @@ static void tegra_xusb_save_context(struct tegra_xusb *tegra)
 	struct tegra_xusb_context *ctx = &tegra->context;
 	unsigned int i;
 
-	if (soc && soc->ipfs.num_offsets > 0) {
+	if (soc->ipfs.num_offsets > 0) {
 		for (i = 0; i < soc->ipfs.num_offsets; i++)
 			ctx->ipfs[i] = ipfs_readl(tegra, soc->ipfs.offsets[i]);
 	}
 
-	if (soc && soc->fpci.num_offsets > 0) {
+	if (soc->fpci.num_offsets > 0) {
 		for (i = 0; i < soc->fpci.num_offsets; i++)
 			ctx->fpci[i] = fpci_readl(tegra, soc->fpci.offsets[i]);
 	}
@@ -1464,12 +1474,12 @@ static void tegra_xusb_restore_context(struct tegra_xusb *tegra)
 	struct tegra_xusb_context *ctx = &tegra->context;
 	unsigned int i;
 
-	if (soc && soc->fpci.num_offsets > 0) {
+	if (soc->fpci.num_offsets > 0) {
 		for (i = 0; i < soc->fpci.num_offsets; i++)
 			fpci_writel(tegra, ctx->fpci[i], soc->fpci.offsets[i]);
 	}
 
-	if (soc && soc->ipfs.num_offsets > 0) {
+	if (soc->ipfs.num_offsets > 0) {
 		for (i = 0; i < soc->ipfs.num_offsets; i++)
 			ipfs_writel(tegra, ctx->ipfs[i], soc->ipfs.offsets[i]);
 	}
@@ -1526,12 +1536,50 @@ static const struct tegra_xusb_phy_type tegra124_phy_types[] = {
 	{ .name = "hsic", .num = 2, },
 };
 
+static const unsigned int tegra124_xusb_context_ipfs[] = {
+	IPFS_XUSB_HOST_MSI_BAR_SZ_0,
+	IPFS_XUSB_HOST_MSI_BAR_SZ_0,
+	IPFS_XUSB_HOST_MSI_AXI_BAR_ST_0,
+	IPFS_XUSB_HOST_MSI_FPCI_BAR_ST_0,
+	IPFS_XUSB_HOST_MSI_VEC0_0,
+	IPFS_XUSB_HOST_MSI_EN_VEC0_0,
+	IPFS_XUSB_HOST_FPCI_ERROR_MASKS_0,
+	IPFS_XUSB_HOST_INTR_MASK_0,
+	IPFS_XUSB_HOST_INTR_ENABLE_0,
+	IPFS_XUSB_HOST_UFPCI_CONFIG_0,
+	IPFS_XUSB_HOST_CLKGATE_HYSTERESIS_0,
+	IPFS_XUSB_HOST_MCCIF_FIFOCTRL_0,
+};
+
+static const unsigned int tegra124_xusb_context_fpci[] = {
+	XUSB_CFG_ARU_CONTEXT_HS_PLS,
+	XUSB_CFG_ARU_CONTEXT_FS_PLS,
+	XUSB_CFG_ARU_CONTEXT_HSFS_SPEED,
+	XUSB_CFG_ARU_CONTEXT_HSFS_PP,
+	XUSB_CFG_ARU_CONTEXT,
+	XUSB_CFG_AXI_CFG,
+	XUSB_CFG_24,
+	XUSB_CFG_16,
+};
+
+static const struct tegra_xusb_context_soc tegra124_xusb_context = {
+	.ipfs = {
+		.num_offsets = ARRAY_SIZE(tegra124_xusb_context_ipfs),
+		.offsets = tegra124_xusb_context_ipfs,
+	},
+	.fpci = {
+		.num_offsets = ARRAY_SIZE(tegra124_xusb_context_fpci),
+		.offsets = tegra124_xusb_context_fpci,
+	},
+};
+
 static const struct tegra_xusb_soc tegra124_soc = {
 	.firmware = "nvidia/tegra124/xusb.bin",
 	.supply_names = tegra124_supply_names,
 	.num_supplies = ARRAY_SIZE(tegra124_supply_names),
 	.phy_types = tegra124_phy_types,
 	.num_types = ARRAY_SIZE(tegra124_phy_types),
+	.context = &tegra124_xusb_context,
 	.ports = {
 		.usb2 = { .offset = 4, .count = 4, },
 		.hsic = { .offset = 6, .count = 2, },
@@ -1570,6 +1618,7 @@ static const struct tegra_xusb_soc tegra210_soc = {
 	.num_supplies = ARRAY_SIZE(tegra210_supply_names),
 	.phy_types = tegra210_phy_types,
 	.num_types = ARRAY_SIZE(tegra210_phy_types),
+	.context = &tegra124_xusb_context,
 	.ports = {
 		.usb2 = { .offset = 4, .count = 4, },
 		.hsic = { .offset = 8, .count = 1, },
@@ -1595,12 +1644,20 @@ static const struct tegra_xusb_phy_type tegra186_phy_types[] = {
 	{ .name = "hsic", .num = 1, },
 };
 
+static const struct tegra_xusb_context_soc tegra186_xusb_context = {
+	.fpci = {
+		.num_offsets = ARRAY_SIZE(tegra124_xusb_context_fpci),
+		.offsets = tegra124_xusb_context_fpci,
+	},
+};
+
 static const struct tegra_xusb_soc tegra186_soc = {
 	.firmware = "nvidia/tegra186/xusb.bin",
 	.supply_names = tegra186_supply_names,
 	.num_supplies = ARRAY_SIZE(tegra186_supply_names),
 	.phy_types = tegra186_phy_types,
 	.num_types = ARRAY_SIZE(tegra186_phy_types),
+	.context = &tegra186_xusb_context,
 	.ports = {
 		.usb3 = { .offset = 0, .count = 3, },
 		.usb2 = { .offset = 3, .count = 3, },
@@ -1630,6 +1687,7 @@ static const struct tegra_xusb_soc tegra194_soc = {
 	.num_supplies = ARRAY_SIZE(tegra194_supply_names),
 	.phy_types = tegra194_phy_types,
 	.num_types = ARRAY_SIZE(tegra194_phy_types),
+	.context = &tegra186_xusb_context,
 	.ports = {
 		.usb3 = { .offset = 0, .count = 4, },
 		.usb2 = { .offset = 4, .count = 4, },
-- 
2.23.0

[PATCH 10/10] usb: host: xhci-tegra: Implement basic ELPG support

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

This implements basic engine-level powergate support which allows the
XUSB controller to be put into a low power mode on system sleep and get
it out of that low power mode again on resume.

Based on work by JC Kuo <jckuo@nvidia.com>.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 drivers/usb/host/xhci-tegra.c | 137 ++++++++++++++++++++++++++++++++--
 1 file changed, 129 insertions(+), 8 deletions(-)

diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
index cd3afec408ea..d0e30927a73f 100644
--- a/drivers/usb/host/xhci-tegra.c
+++ b/drivers/usb/host/xhci-tegra.c
@@ -1451,6 +1451,45 @@ static int tegra_xusb_remove(struct platform_device *pdev)
 }
 
 #ifdef CONFIG_PM_SLEEP
+static bool xhci_hub_ports_suspended(struct xhci_hub *hub)
+{
+	struct device *dev = hub->hcd->self.controller;
+	bool status = true;
+	unsigned int i;
+	u32 value;
+
+	for (i = 0; i < hub->num_ports; i++) {
+		value = readl(hub->ports[i]->addr);
+		if ((value & PORT_PE) == 0)
+			continue;
+
+		if ((value & PORT_PLS_MASK) != XDEV_U3) {
+			dev_info(dev, "%u-%u isn't suspended: %#010x\n",
+				 hub->hcd->self.busnum, i + 1, value);
+			status = false;
+		}
+	}
+
+	return status;
+}
+
+static int tegra_xusb_check_ports(struct tegra_xusb *tegra)
+{
+	struct xhci_hcd *xhci = hcd_to_xhci(tegra->hcd);
+	unsigned long flags;
+	int err = 0;
+
+	spin_lock_irqsave(&xhci->lock, flags);
+
+	if (!xhci_hub_ports_suspended(&xhci->usb2_rhub) ||
+	    !xhci_hub_ports_suspended(&xhci->usb3_rhub))
+		err = -EBUSY;
+
+	spin_unlock_irqrestore(&xhci->lock, flags);
+
+	return err;
+}
+
 static void tegra_xusb_save_context(struct tegra_xusb *tegra)
 {
 	const struct tegra_xusb_context_soc *soc = tegra->soc->context;
@@ -1485,31 +1524,113 @@ static void tegra_xusb_restore_context(struct tegra_xusb *tegra)
 	}
 }
 
-static int tegra_xusb_suspend(struct device *dev)
+static int tegra_xusb_enter_elpg(struct tegra_xusb *tegra, bool wakeup)
 {
-	struct tegra_xusb *tegra = dev_get_drvdata(dev);
 	struct xhci_hcd *xhci = hcd_to_xhci(tegra->hcd);
-	bool wakeup = device_may_wakeup(dev);
+	u32 value;
 	int err;
 
-	/* TODO: Powergate controller across suspend/resume. */
+	err = tegra_xusb_check_ports(tegra);
+	if (err < 0) {
+		dev_err(tegra->dev, "not all ports suspended: %d\n", err);
+		return err;
+	}
+
 	err = xhci_suspend(xhci, wakeup);
-	if (err < 0)
+	if (err < 0) {
+		dev_err(tegra->dev, "failed to suspend XHCI: %d\n", err);
 		return err;
+	}
 
 	tegra_xusb_save_context(tegra);
+	tegra_xusb_phy_disable(tegra);
+	tegra_xusb_clk_disable(tegra);
 
 	return 0;
 }
 
-static int tegra_xusb_resume(struct device *dev)
+static int tegra_xusb_exit_elpg(struct tegra_xusb *tegra, bool wakeup)
 {
-	struct tegra_xusb *tegra = dev_get_drvdata(dev);
 	struct xhci_hcd *xhci = hcd_to_xhci(tegra->hcd);
+	u32 value;
+	int err;
 
+	err = tegra_xusb_clk_enable(tegra);
+	if (err < 0) {
+		dev_err(tegra->dev, "failed to enable clocks: %d\n", err);
+		return err;
+	}
+
+	err = tegra_xusb_phy_enable(tegra);
+	if (err < 0) {
+		dev_err(tegra->dev, "failed to enable PHYs: %d\n", err);
+		goto disable_clk;
+	}
+
+	tegra_xusb_config(tegra);
 	tegra_xusb_restore_context(tegra);
 
-	return xhci_resume(xhci, false);
+	err = tegra_xusb_load_firmware(tegra);
+	if (err < 0) {
+		dev_err(tegra->dev, "failed to load firmware: %d\n", err);
+		goto disable_phy;
+	}
+
+	err = __tegra_xusb_enable_firmware_messages(tegra);
+	if (err < 0) {
+		dev_err(tegra->dev, "failed to enable messages: %d\n", err);
+		goto disable_phy;
+	}
+
+	err = xhci_resume(xhci, true);
+	if (err < 0) {
+		dev_err(tegra->dev, "failed to resume XHCI: %d\n", err);
+		goto disable_phy;
+	}
+
+	return 0;
+
+disable_phy:
+	tegra_xusb_phy_disable(tegra);
+disable_clk:
+	tegra_xusb_clk_disable(tegra);
+	return err;
+}
+
+static int tegra_xusb_suspend(struct device *dev)
+{
+	struct tegra_xusb *tegra = dev_get_drvdata(dev);
+	bool wakeup = device_may_wakeup(dev);
+	int err;
+
+	synchronize_irq(tegra->mbox_irq);
+
+	mutex_lock(&tegra->lock);
+
+	err = tegra_xusb_enter_elpg(tegra, wakeup);
+	if (err < 0)
+		goto unlock;
+
+unlock:
+	mutex_unlock(&tegra->lock);
+	return err;
+}
+
+static int tegra_xusb_resume(struct device *dev)
+{
+	struct tegra_xusb *tegra = dev_get_drvdata(dev);
+	bool wakeup = device_may_wakeup(dev);
+	int err;
+
+	mutex_lock(&tegra->lock);
+
+	err = tegra_xusb_exit_elpg(tegra, wakeup);
+	if (err < 0)
+		goto unlock;
+
+unlock:
+	mutex_unlock(&tegra->lock);
+	return err;
 }
 #endif
 
-- 
2.23.0

Re: [PATCH 10/10] usb: host: xhci-tegra: Implement basic ELPG support

[Mathias Nyman]

On 25.11.2019 14.32, Thierry Reding wrote:
> From: Thierry Reding <treding@nvidia.com>
> 
> This implements basic engine-level powergate support which allows the
> XUSB controller to be put into a low power mode on system sleep and get
> it out of that low power mode again on resume.
> 
> Based on work by JC Kuo <jckuo@nvidia.com>.
> 
> Signed-off-by: Thierry Reding <treding@nvidia.com>
> ---
>   drivers/usb/host/xhci-tegra.c | 137 ++++++++++++++++++++++++++++++++--
>   1 file changed, 129 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
> index cd3afec408ea..d0e30927a73f 100644
> --- a/drivers/usb/host/xhci-tegra.c
> +++ b/drivers/usb/host/xhci-tegra.c
> @@ -1451,6 +1451,45 @@ static int tegra_xusb_remove(struct platform_device *pdev)
>   }
>   
>   #ifdef CONFIG_PM_SLEEP
> +static bool xhci_hub_ports_suspended(struct xhci_hub *hub)
> +{
> +	struct device *dev = hub->hcd->self.controller;
> +	bool status = true;
> +	unsigned int i;
> +	u32 value;
> +
> +	for (i = 0; i < hub->num_ports; i++) {
> +		value = readl(hub->ports[i]->addr);
> +		if ((value & PORT_PE) == 0)
> +			continue;
> +
> +		if ((value & PORT_PLS_MASK) != XDEV_U3) {
> +			dev_info(dev, "%u-%u isn't suspended: %#010x\n",
> +				 hub->hcd->self.busnum, i + 1, value);
> +			status = false;
> +		}
> +	}
> +
> +	return status;
> +}
> +
> +static int tegra_xusb_check_ports(struct tegra_xusb *tegra)
> +{
> +	struct xhci_hcd *xhci = hcd_to_xhci(tegra->hcd);
> +	unsigned long flags;
> +	int err = 0;
> +
> +	spin_lock_irqsave(&xhci->lock, flags);
> +
> +	if (!xhci_hub_ports_suspended(&xhci->usb2_rhub) ||
> +	    !xhci_hub_ports_suspended(&xhci->usb3_rhub))
> +		err = -EBUSY;
> +
> +	spin_unlock_irqrestore(&xhci->lock, flags);
> +
> +	return err;
> +}
> +
>   static void tegra_xusb_save_context(struct tegra_xusb *tegra)
>   {
>   	const struct tegra_xusb_context_soc *soc = tegra->soc->context;
> @@ -1485,31 +1524,113 @@ static void tegra_xusb_restore_context(struct tegra_xusb *tegra)
>   	}
>   }
>   
> -static int tegra_xusb_suspend(struct device *dev)
> +static int tegra_xusb_enter_elpg(struct tegra_xusb *tegra, bool wakeup)
>   {
> -	struct tegra_xusb *tegra = dev_get_drvdata(dev);
>   	struct xhci_hcd *xhci = hcd_to_xhci(tegra->hcd);
> -	bool wakeup = device_may_wakeup(dev);
> +	u32 value;
>   	int err;
>   
> -	/* TODO: Powergate controller across suspend/resume. */
> +	err = tegra_xusb_check_ports(tegra);
> +	if (err < 0) {
> +		dev_err(tegra->dev, "not all ports suspended: %d\n", err);
> +		return err;
> +	}
> +
>   	err = xhci_suspend(xhci, wakeup);
> -	if (err < 0)
> +	if (err < 0) {
> +		dev_err(tegra->dev, "failed to suspend XHCI: %d\n", err);
>   		return err;
> +	}
>   
>   	tegra_xusb_save_context(tegra);
> +	tegra_xusb_phy_disable(tegra);
> +	tegra_xusb_clk_disable(tegra);
>   
>   	return 0;
>   }
>   
> -static int tegra_xusb_resume(struct device *dev)
> +static int tegra_xusb_exit_elpg(struct tegra_xusb *tegra, bool wakeup)
>   {
> -	struct tegra_xusb *tegra = dev_get_drvdata(dev);
>   	struct xhci_hcd *xhci = hcd_to_xhci(tegra->hcd);
> +	u32 value;
> +	int err;
>   
> +	err = tegra_xusb_clk_enable(tegra);
> +	if (err < 0) {
> +		dev_err(tegra->dev, "failed to enable clocks: %d\n", err);
> +		return err;
> +	}
> +
> +	err = tegra_xusb_phy_enable(tegra);
> +	if (err < 0) {
> +		dev_err(tegra->dev, "failed to enable PHYs: %d\n", err);
> +		goto disable_clk;
> +	}
> +
> +	tegra_xusb_config(tegra);
>   	tegra_xusb_restore_context(tegra);
>   
> -	return xhci_resume(xhci, false);
> +	err = tegra_xusb_load_firmware(tegra);
> +	if (err < 0) {
> +		dev_err(tegra->dev, "failed to load firmware: %d\n", err);
> +		goto disable_phy;
> +	}
> +
> +	err = __tegra_xusb_enable_firmware_messages(tegra);
> +	if (err < 0) {
> +		dev_err(tegra->dev, "failed to enable messages: %d\n", err);
> +		goto disable_phy;
> +	}
> +
> +	err = xhci_resume(xhci, true);
> +	if (err < 0) {
> +		dev_err(tegra->dev, "failed to resume XHCI: %d\n", err);
> +		goto disable_phy;
> +	}
> +
> +	return 0;
> +
> +disable_phy:
> +	tegra_xusb_phy_disable(tegra);
> +disable_clk:
> +	tegra_xusb_clk_disable(tegra);
> +	return err;
> +}
> +
> +static int tegra_xusb_suspend(struct device *dev)
> +{
> +	struct tegra_xusb *tegra = dev_get_drvdata(dev);
> +	bool wakeup = device_may_wakeup(dev);
> +	int err;
> +
> +	synchronize_irq(tegra->mbox_irq);
> +
> +	mutex_lock(&tegra->lock);
> +
> +	err = tegra_xusb_enter_elpg(tegra, wakeup);
> +	if (err < 0)
> +		goto unlock;

Is there some code missing here, or just preparing for some future feature?

> +
> +unlock:
> +	mutex_unlock(&tegra->lock);
> +	return err;
> +}
> +
> +static int tegra_xusb_resume(struct device *dev)
> +{
> +	struct tegra_xusb *tegra = dev_get_drvdata(dev);
> +	bool wakeup = device_may_wakeup(dev);
> +	int err;
> +
> +	mutex_lock(&tegra->lock);
> +
> +	err = tegra_xusb_exit_elpg(tegra, wakeup);
> +	if (err < 0)
> +		goto unlock;
> +
> +unlock:
> +	mutex_unlock(&tegra->lock);
> +	return err;
>   }
>   #endif
>   

Whole series looks good to me otherwise.

Let me know if you want me to take this as is, or if you are planning on making a second version

Thanks
Mathias

Re: [PATCH 10/10] usb: host: xhci-tegra: Implement basic ELPG support

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 5924 bytes --]

On Tue, Nov 26, 2019 at 05:43:15PM +0200, Mathias Nyman wrote:
> On 25.11.2019 14.32, Thierry Reding wrote:
> > From: Thierry Reding <treding@nvidia.com>
> > 
> > This implements basic engine-level powergate support which allows the
> > XUSB controller to be put into a low power mode on system sleep and get
> > it out of that low power mode again on resume.
> > 
> > Based on work by JC Kuo <jckuo@nvidia.com>.
> > 
> > Signed-off-by: Thierry Reding <treding@nvidia.com>
> > ---
> >   drivers/usb/host/xhci-tegra.c | 137 ++++++++++++++++++++++++++++++++--
> >   1 file changed, 129 insertions(+), 8 deletions(-)
> > 
> > diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
> > index cd3afec408ea..d0e30927a73f 100644
> > --- a/drivers/usb/host/xhci-tegra.c
> > +++ b/drivers/usb/host/xhci-tegra.c
> > @@ -1451,6 +1451,45 @@ static int tegra_xusb_remove(struct platform_device *pdev)
> >   }
> >   #ifdef CONFIG_PM_SLEEP
> > +static bool xhci_hub_ports_suspended(struct xhci_hub *hub)
> > +{
> > +	struct device *dev = hub->hcd->self.controller;
> > +	bool status = true;
> > +	unsigned int i;
> > +	u32 value;
> > +
> > +	for (i = 0; i < hub->num_ports; i++) {
> > +		value = readl(hub->ports[i]->addr);
> > +		if ((value & PORT_PE) == 0)
> > +			continue;
> > +
> > +		if ((value & PORT_PLS_MASK) != XDEV_U3) {
> > +			dev_info(dev, "%u-%u isn't suspended: %#010x\n",
> > +				 hub->hcd->self.busnum, i + 1, value);
> > +			status = false;
> > +		}
> > +	}
> > +
> > +	return status;
> > +}
> > +
> > +static int tegra_xusb_check_ports(struct tegra_xusb *tegra)
> > +{
> > +	struct xhci_hcd *xhci = hcd_to_xhci(tegra->hcd);
> > +	unsigned long flags;
> > +	int err = 0;
> > +
> > +	spin_lock_irqsave(&xhci->lock, flags);
> > +
> > +	if (!xhci_hub_ports_suspended(&xhci->usb2_rhub) ||
> > +	    !xhci_hub_ports_suspended(&xhci->usb3_rhub))
> > +		err = -EBUSY;
> > +
> > +	spin_unlock_irqrestore(&xhci->lock, flags);
> > +
> > +	return err;
> > +}
> > +
> >   static void tegra_xusb_save_context(struct tegra_xusb *tegra)
> >   {
> >   	const struct tegra_xusb_context_soc *soc = tegra->soc->context;
> > @@ -1485,31 +1524,113 @@ static void tegra_xusb_restore_context(struct tegra_xusb *tegra)
> >   	}
> >   }
> > -static int tegra_xusb_suspend(struct device *dev)
> > +static int tegra_xusb_enter_elpg(struct tegra_xusb *tegra, bool wakeup)
> >   {
> > -	struct tegra_xusb *tegra = dev_get_drvdata(dev);
> >   	struct xhci_hcd *xhci = hcd_to_xhci(tegra->hcd);
> > -	bool wakeup = device_may_wakeup(dev);
> > +	u32 value;
> >   	int err;
> > -	/* TODO: Powergate controller across suspend/resume. */
> > +	err = tegra_xusb_check_ports(tegra);
> > +	if (err < 0) {
> > +		dev_err(tegra->dev, "not all ports suspended: %d\n", err);
> > +		return err;
> > +	}
> > +
> >   	err = xhci_suspend(xhci, wakeup);
> > -	if (err < 0)
> > +	if (err < 0) {
> > +		dev_err(tegra->dev, "failed to suspend XHCI: %d\n", err);
> >   		return err;
> > +	}
> >   	tegra_xusb_save_context(tegra);
> > +	tegra_xusb_phy_disable(tegra);
> > +	tegra_xusb_clk_disable(tegra);
> >   	return 0;
> >   }
> > -static int tegra_xusb_resume(struct device *dev)
> > +static int tegra_xusb_exit_elpg(struct tegra_xusb *tegra, bool wakeup)
> >   {
> > -	struct tegra_xusb *tegra = dev_get_drvdata(dev);
> >   	struct xhci_hcd *xhci = hcd_to_xhci(tegra->hcd);
> > +	u32 value;
> > +	int err;
> > +	err = tegra_xusb_clk_enable(tegra);
> > +	if (err < 0) {
> > +		dev_err(tegra->dev, "failed to enable clocks: %d\n", err);
> > +		return err;
> > +	}
> > +
> > +	err = tegra_xusb_phy_enable(tegra);
> > +	if (err < 0) {
> > +		dev_err(tegra->dev, "failed to enable PHYs: %d\n", err);
> > +		goto disable_clk;
> > +	}
> > +
> > +	tegra_xusb_config(tegra);
> >   	tegra_xusb_restore_context(tegra);
> > -	return xhci_resume(xhci, false);
> > +	err = tegra_xusb_load_firmware(tegra);
> > +	if (err < 0) {
> > +		dev_err(tegra->dev, "failed to load firmware: %d\n", err);
> > +		goto disable_phy;
> > +	}
> > +
> > +	err = __tegra_xusb_enable_firmware_messages(tegra);
> > +	if (err < 0) {
> > +		dev_err(tegra->dev, "failed to enable messages: %d\n", err);
> > +		goto disable_phy;
> > +	}
> > +
> > +	err = xhci_resume(xhci, true);
> > +	if (err < 0) {
> > +		dev_err(tegra->dev, "failed to resume XHCI: %d\n", err);
> > +		goto disable_phy;
> > +	}
> > +
> > +	return 0;
> > +
> > +disable_phy:
> > +	tegra_xusb_phy_disable(tegra);
> > +disable_clk:
> > +	tegra_xusb_clk_disable(tegra);
> > +	return err;
> > +}
> > +
> > +static int tegra_xusb_suspend(struct device *dev)
> > +{
> > +	struct tegra_xusb *tegra = dev_get_drvdata(dev);
> > +	bool wakeup = device_may_wakeup(dev);
> > +	int err;
> > +
> > +	synchronize_irq(tegra->mbox_irq);
> > +
> > +	mutex_lock(&tegra->lock);
> > +
> > +	err = tegra_xusb_enter_elpg(tegra, wakeup);
> > +	if (err < 0)
> > +		goto unlock;
> 
> Is there some code missing here, or just preparing for some future feature?

This is just leftover I forgot to cleanup after I removed debug printk's
here. There's likely going to be some changes around this for better
runtime PM support, but that will all live in tegra_xusb_enter_elpg(),
so I think we can skip this. I've sent out a v2 with this...

> 
> > +
> > +unlock:
> > +	mutex_unlock(&tegra->lock);
> > +	return err;
> > +}
> > +
> > +static int tegra_xusb_resume(struct device *dev)
> > +{
> > +	struct tegra_xusb *tegra = dev_get_drvdata(dev);
> > +	bool wakeup = device_may_wakeup(dev);
> > +	int err;
> > +
> > +	mutex_lock(&tegra->lock);
> > +
> > +	err = tegra_xusb_exit_elpg(tegra, wakeup);
> > +	if (err < 0)
> > +		goto unlock;
> > +
> > +unlock:
> > +	mutex_unlock(&tegra->lock);
> > +	return err;
> >   }
> >   #endif

... and this fixed.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]