[PATCH v2 0/2 2nd RESEND] Fix USB3.0 DRD PHY calibration issues (DWC3/XHCI) on Exynos542x SoCs

[PATCH v2 0/2 2nd RESEND] Fix USB3.0 DRD PHY calibration issues (DWC3/XHCI) on Exynos542x SoCs

[Marek Szyprowski]

Dear All,

Commit d8c80bb3b55b ("phy: exynos5-usbdrd: Calibrate LOS levels for
exynos5420/5800") added support for Exynos5 USB3.0 DRD PHY calibration,
what enabled proper Super-Speed enumeration of USB3.0 devices connected
to various Exynos5 SoCs. After some time it turned out that the mentioned
patch worked a bit by pure luck and covered only one use case: fresh
boot with all drivers compiled into the kernel.

If drivers were compiled as modules, due to timing issues, it worked only
if XHCI-plat driver was loaded before the DWC3 driver:
https://patchwork.kernel.org/patch/10773947/

Also during the system suspend/resume cycle the calibration was not
performed at the proper time, what resulted in switching USB 3.0 devices to
USB 2.0 high-speed compatibility mode.

This patch addresses all those issues. Exynos5 USB3.0 DRD PHY calibration
is moved to the generic USB HCD PHY handling code, which takes care of
proper PHY calibration after HCD (XHCI) core reset. This fixes all known
use cases (XHCI driver compiled as module and loaded on demand as well as
during system suspend/resume cycle).

The main change comparing to v1 is huge simplification of the code:
generic PHYs are already handled by HCD core code, so the calibration is
added there. No Exynos-specific XHCI driver variant is needed anymore.
There is also no need to change the way the DWC3 driver is instantiated,
what wasn't done right in v1 too (the code oopsed on module remove).

Here are the logs taken on Exynos5422-based Odroid HC1 board (with USB3.0
RTL8153 LAN and USB3.0 JMicron SATA-USB bridge):

Vanilla Linux next-20190716:
----->8-----------------------------------------------------------------
root@target:~# lsusb -t
/:  Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=uas, 5000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=exynos-ohci/3p, 12M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=exynos-ehci/3p, 480M
root@target:~# time rtcwake -s 10 -m mem
rtcwake: wakeup from "mem" using /dev/rtc0 at Fri Jul 19 07:08:29 2019
[   43.641914] PM: suspend entry (deep)
[   43.647758] Filesystems sync: 0.003 seconds
[   43.663038] Freezing user space processes ... (elapsed 0.006 seconds) done.
[   43.674858] OOM killer disabled.
[   43.677824] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
[   43.685644] printk: Suspending console(s) (use no_console_suspend to debug)
[   43.754198] sd 0:0:0:0: [sda] Synchronizing SCSI cache
[   43.831613] wake enabled for irq 145
[   43.994550] samsung-pinctrl 13400000.pinctrl: Setting external wakeup interrupt mask: 0xffffffef
[   44.004378] Disabling non-boot CPUs ...
[   44.014851] IRQ 51: no longer affine to CPU1
[   44.023293] IRQ 52: no longer affine to CPU2
[   44.028975] IRQ 53: no longer affine to CPU3
[   44.031818] IRQ 54: no longer affine to CPU4
[   44.034229] IRQ 55: no longer affine to CPU5
[   44.036648] IRQ 56: no longer affine to CPU6
[   44.040546] IRQ 57: no longer affine to CPU7
[   44.048237] Enabling non-boot CPUs ...
[   44.053004] CPU1 is up
[   44.056036] CPU2 is up
[   44.058860] CPU3 is up
[   44.059552] CPU4: detected I-Cache line size mismatch, workaround enabled
[   44.063502] CPU4 is up
[   44.064097] CPU5: detected I-Cache line size mismatch, workaround enabled
[   44.065997] CPU5 is up
[   44.066611] CPU6: detected I-Cache line size mismatch, workaround enabled
[   44.068640] CPU6 is up
[   44.069259] CPU7: detected I-Cache line size mismatch, workaround enabled
[   44.071689] CPU7 is up
[   44.096037] s3c2410-wdt 101d0000.watchdog: watchdog disabled
[   44.176142] wake disabled for irq 145
[   44.184616] usb usb3: root hub lost power or was reset
[   44.184705] usb usb4: root hub lost power or was reset
[   44.184804] s3c-rtc 101e0000.rtc: rtc disabled, re-enabling
[   44.184877] usb usb5: root hub lost power or was reset
[   44.184894] usb usb6: root hub lost power or was reset
[   48.467048] OOM killer enabled.
[   48.470075] Restarting tasks ...
[   48.471490] usb 4-1: USB disconnect, device number 2
[   48.473789] usb 6-1: USB disconnect, device number 2
[   48.474380] done.
[   48.487766] PM: suspend exit

real    0m15.098s
user    0m0.000s
sys     0m0.358s
[   48.519357] mmc_host mmc0: Bus speed (slot 0) = 50000000Hz (slot req 400000Hz, actual 396825HZ div = 63)
root@target:~# [   48.540888] sd 0:0:0:0: [sda] Synchronizing SCSI cache
[   48.624651] mmc_host mmc0: Bus speed (slot 0) = 200000000Hz (slot req 200000000Hz, actual 200000000HZ div = 0)
[   48.874422] usb 5-1: new high-speed USB device number 3 using xhci-hcd
[   49.154586] sd 0:0:0:0: [sda] Synchronize Cache(10) failed: Result: hostbyte=0x07 driverbyte=0x00
[   49.355211] usb 5-1: New USB device found, idVendor=0bda, idProduct=8153, bcdDevice=30.00
[   49.361906] usb 5-1: New USB device strings: Mfr=1, Product=2, SerialNumber=6
[   49.369288] usb 5-1: Product: USB 10/100/1000 LAN
[   49.373740] usb 5-1: Manufacturer: Realtek
[   49.377760] usb 5-1: SerialNumber: 000001000000
[   49.619366] usb 5-1: reset high-speed USB device number 3 using xhci-hcd
[   49.643116] usb usb3-port1: Cannot enable. Maybe the USB cable is bad?
[   49.903126] r8152 5-1:1.0 eth0: v1.09.10
[   50.673383] usb 3-1: new high-speed USB device number 3 using xhci-hcd
[   50.864851] usb 3-1: New USB device found, idVendor=152d, idProduct=0578, bcdDevice= 1.05
[   50.871612] usb 3-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[   50.878953] usb 3-1: Product: USB to SATA bridge
[   50.883457] usb 3-1: Manufacturer: JMicron
[   50.887376] usb 3-1: SerialNumber: DB00000000013B
[   50.921771] scsi host0: uas
[   50.927573] scsi 0:0:0:0: Direct-Access     JMicron                   0105 PQ: 0 ANSI: 6
[   50.943500] sd 0:0:0:0: Attached scsi generic sg0 type 0
[   50.948997] sd 0:0:0:0: [sda] 117229295 512-byte logical blocks: (60.0 GB/55.9 GiB)
[   50.955537] sd 0:0:0:0: [sda] 4096-byte physical blocks
[   50.962062] sd 0:0:0:0: [sda] Write Protect is off
[   50.969062] sd 0:0:0:0: [sda] Disabling FUA
[   50.971787] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   50.984021] sd 0:0:0:0: [sda] Optimal transfer size 33553920 bytes not a multiple of physical block size (4096 bytes)
[   51.015373]  sda: sda1 sda2 sda3 sda4 < sda5 sda6 >
[   51.041092] sd 0:0:0:0: [sda] Attached SCSI disk
[   53.223864] usb usb6-port1: Cannot enable. Maybe the USB cable is bad?
[   57.544062] usb usb6-port1: Cannot enable. Maybe the USB cable is bad?
root@target:~# lsusb -t
/:  Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
    |__ Port 1: Dev 3, If 0, Class=Vendor Specific Class, Driver=r8152, 480M
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
    |__ Port 1: Dev 3, If 0, Class=Mass Storage, Driver=uas, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=exynos-ohci/3p, 12M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=exynos-ehci/3p, 480M
root@target:~#
----->8-----------------------------------------------------------------


Linux next-20190716 with this patchset applied:
----->8-----------------------------------------------------------------
root@target:~# lsusb -t
/:  Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=uas, 5000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=exynos-ohci/3p, 12M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=exynos-ehci/3p, 480M
root@target:~# time rtcwake -s 10 -m mem
[  475.834797] Filesystems sync: 0.005 seconds
[  475.851002] Freezing user space processes ... (elapsed 0.001 seconds) done.
[  475.858621] OOM killer disabled.
[  475.861856] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
[  475.869783] printk: Suspending console(s) (use no_console_suspend to debug)
[  475.959486] sd 0:0:0:0: [sda] Synchronizing SCSI cache
[  476.036891] wake enabled for irq 145
[  476.189298] samsung-pinctrl 13400000.pinctrl: Setting external wakeup interrupt mask: 0xffffffef
[  476.198502] Disabling non-boot CPUs ...
[  476.205227] IRQ 51: no longer affine to CPU1
[  476.211776] IRQ 52: no longer affine to CPU2
[  476.215564] IRQ 53: no longer affine to CPU3
[  476.218025] IRQ 54: no longer affine to CPU4
[  476.220913] IRQ 55: no longer affine to CPU5
[  476.223242] IRQ 56: no longer affine to CPU6
[  476.226789] IRQ 57: no longer affine to CPU7
[  476.254284] Enabling non-boot CPUs ...
[  476.258860] CPU1 is up
[  476.261658] CPU2 is up
[  476.264471] CPU3 is up
[  476.265161] CPU4: detected I-Cache line size mismatch, workaround enabled
[  476.269026] CPU4 is up
[  476.269640] CPU5: detected I-Cache line size mismatch, workaround enabled
[  476.271481] CPU5 is up
[  476.272094] CPU6: detected I-Cache line size mismatch, workaround enabled
[  476.274056] CPU6 is up
[  476.274668] CPU7: detected I-Cache line size mismatch, workaround enabled
[  476.276976] CPU7 is up
[  476.302530] s3c2410-wdt 101d0000.watchdog: watchdog disabled
[  476.302801] usb usb1: root hub lost power or was reset
[  476.372443] usb usb2: root hub lost power or was reset
[  476.381291] wake disabled for irq 145
[  476.401548] usb usb3: root hub lost power or was reset
[  476.401636] usb usb4: root hub lost power or was reset
[  476.401728] s3c-rtc 101e0000.rtc: rtc disabled, re-enabling
[  476.410472] usb usb5: root hub lost power or was reset
[  476.410495] usb usb6: root hub lost power or was reset
[  478.920232] usb 4-1: reset SuperSpeed Gen 1 USB device number 2 using xhci-hcd
[  480.685206] OOM killer enabled.
[  480.688272] Restarting tasks ... done.
[  480.699751] usb 6-1: USB disconnect, device number 2
[  480.706698] PM: suspend exit

real    0m16.018s
user    0m0.000s
sys     0m0.342s
[  480.734080] mmc_host mmc0: Bus speed (slot 0) = 50000000Hz (slot req 400000Hz, actual 396825HZ div = 63)
root@target:~# [  480.843283] mmc_host mmc0: Bus speed (slot 0) = 200000000Hz (slot req 200000000Hz, actual 200000000HZ div = 0)
[  481.679454] usb 6-1: new SuperSpeed Gen 1 USB device number 3 using xhci-hcd
[  481.716309] usb 6-1: New USB device found, idVendor=0bda, idProduct=8153, bcdDevice=30.00
[  481.723359] usb 6-1: New USB device strings: Mfr=1, Product=2, SerialNumber=6
[  481.731394] usb 6-1: Product: USB 10/100/1000 LAN
[  481.734881] usb 6-1: Manufacturer: Realtek
[  481.739196] usb 6-1: SerialNumber: 000001000000
[  482.002327] usb 6-1: reset SuperSpeed Gen 1 USB device number 3 using xhci-hcd
[  482.141639] r8152 6-1:1.0 eth0: v1.09.10

root@target:~# lsusb -t
/:  Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
    |__ Port 1: Dev 3, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=uas, 5000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=exynos-ohci/3p, 12M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=exynos-ehci/3p, 480M
root@target:~#
----->8-----------------------------------------------------------------


Best regards
Marek Szyprowski
Samsung R&D Institute Poland


Changelog:
v2 2nd resend:
- added acked-by tags

v2 resend: https://lkml.org/lkml/2019/8/8/258
- added tested-by tags

v2: https://lkml.org/lkml/2019/7/19/605
- found that generic phys are already handled by usb hcd core code, so
  phy_calibration has been moved to drivers/usb/core/{hcd,phy}.c
- dropped custom xhci-exynos driver variant, no longer needed
- dropped passing custom properties from dwc3-exynos to generic dwc3
  core driver, no longer needed and mixing custom driver properties
  with DT-properties didn't work well (generic cleanup path caused
  oops in such case)

v1: https://lkml.org/lkml/2019/6/27/123
 - initial version


Patch summary:

Marek Szyprowski (2):
  usb: core: phy: add support for PHY calibration
  usb: dwc3: remove generic PHY calibrate() calls

 drivers/usb/core/hcd.c  |  7 +++++++
 drivers/usb/core/phy.c  | 21 +++++++++++++++++++++
 drivers/usb/core/phy.h  |  1 +
 drivers/usb/dwc3/core.c |  2 --
 4 files changed, 29 insertions(+), 2 deletions(-)

-- 
2.17.1

[PATCH v2 1/2 RESEND] usb: core: phy: add support for PHY calibration

[Marek Szyprowski]

Some PHYs (for example Exynos5 USB3.0 DRD PHY) require calibration to be
done after every USB HCD reset. Generic PHY framework has been already
extended with phy_calibrate() function in commit 36914111e682 ("drivers:
phy: add calibrate method"). This patch adds support for it to generic
PHY handling code in USB HCD core.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Tested-by: Anand Moon <linux.amoon@gmail.com>
Tested-by: Jochen Sprickerhof <jochen@sprickerhof.de>
---
 drivers/usb/core/hcd.c |  7 +++++++
 drivers/usb/core/phy.c | 21 +++++++++++++++++++++
 drivers/usb/core/phy.h |  1 +
 3 files changed, 29 insertions(+)

diff --git a/drivers/usb/core/hcd.c b/drivers/usb/core/hcd.c
index 88533938ce19..b89936c1df23 100644
--- a/drivers/usb/core/hcd.c
+++ b/drivers/usb/core/hcd.c
@@ -2291,6 +2291,9 @@ int hcd_bus_resume(struct usb_device *rhdev, pm_message_t msg)
 	hcd->state = HC_STATE_RESUMING;
 	status = hcd->driver->bus_resume(hcd);
 	clear_bit(HCD_FLAG_WAKEUP_PENDING, &hcd->flags);
+	if (status == 0)
+		status = usb_phy_roothub_calibrate(hcd->phy_roothub);
+
 	if (status == 0) {
 		struct usb_device *udev;
 		int port1;
@@ -2864,6 +2867,10 @@ int usb_add_hcd(struct usb_hcd *hcd,
 	}
 	hcd->rh_pollable = 1;
 
+	retval = usb_phy_roothub_calibrate(hcd->phy_roothub);
+	if (retval)
+		goto err_hcd_driver_setup;
+
 	/* NOTE: root hub and controller capabilities may not be the same */
 	if (device_can_wakeup(hcd->self.controller)
 			&& device_can_wakeup(&hcd->self.root_hub->dev))
diff --git a/drivers/usb/core/phy.c b/drivers/usb/core/phy.c
index 7580493b867a..fb1588e7c282 100644
--- a/drivers/usb/core/phy.c
+++ b/drivers/usb/core/phy.c
@@ -151,6 +151,27 @@ int usb_phy_roothub_set_mode(struct usb_phy_roothub *phy_roothub,
 }
 EXPORT_SYMBOL_GPL(usb_phy_roothub_set_mode);
 
+int usb_phy_roothub_calibrate(struct usb_phy_roothub *phy_roothub)
+{
+	struct usb_phy_roothub *roothub_entry;
+	struct list_head *head;
+	int err;
+
+	if (!phy_roothub)
+		return 0;
+
+	head = &phy_roothub->list;
+
+	list_for_each_entry(roothub_entry, head, list) {
+		err = phy_calibrate(roothub_entry->phy);
+		if (err)
+			return err;
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(usb_phy_roothub_calibrate);
+
 int usb_phy_roothub_power_on(struct usb_phy_roothub *phy_roothub)
 {
 	struct usb_phy_roothub *roothub_entry;
diff --git a/drivers/usb/core/phy.h b/drivers/usb/core/phy.h
index dad564e2d2d4..20a267cd986b 100644
--- a/drivers/usb/core/phy.h
+++ b/drivers/usb/core/phy.h
@@ -18,6 +18,7 @@ int usb_phy_roothub_exit(struct usb_phy_roothub *phy_roothub);
 
 int usb_phy_roothub_set_mode(struct usb_phy_roothub *phy_roothub,
 			     enum phy_mode mode);
+int usb_phy_roothub_calibrate(struct usb_phy_roothub *phy_roothub);
 int usb_phy_roothub_power_on(struct usb_phy_roothub *phy_roothub);
 void usb_phy_roothub_power_off(struct usb_phy_roothub *phy_roothub);
 
-- 
2.17.1

[PATCH v2 2/2 RESEND] usb: dwc3: remove generic PHY calibrate() calls

[Marek Szyprowski]

Calls to USB2 generic PHY calibrate() method has been moved to HCD core,
which now successfully handles generic PHYs and their calibration after
every HCD reset. This fixes all the timing issues related to PHY
calibration done directly from DWC3 driver: incorrect operation after
system suspend/resume or USB3.0 detection failure when XHCI-plat driver
compiled as separate module.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Tested-by: Anand Moon <linux.amoon@gmail.com>
Tested-by: Jochen Sprickerhof <jochen@sprickerhof.de>
Acked-by: Felipe Balbi <felipe.balbi@linux.intel.com>
---
 drivers/usb/dwc3/core.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
index c9bb93a2c81e..7dd6d419254d 100644
--- a/drivers/usb/dwc3/core.c
+++ b/drivers/usb/dwc3/core.c
@@ -168,7 +168,6 @@ static void __dwc3_set_mode(struct work_struct *work)
 				otg_set_vbus(dwc->usb2_phy->otg, true);
 			phy_set_mode(dwc->usb2_generic_phy, PHY_MODE_USB_HOST);
 			phy_set_mode(dwc->usb3_generic_phy, PHY_MODE_USB_HOST);
-			phy_calibrate(dwc->usb2_generic_phy);
 		}
 		break;
 	case DWC3_GCTL_PRTCAP_DEVICE:
@@ -1166,7 +1165,6 @@ static int dwc3_core_init_mode(struct dwc3 *dwc)
 				dev_err(dev, "failed to initialize host\n");
 			return ret;
 		}
-		phy_calibrate(dwc->usb2_generic_phy);
 		break;
 	case USB_DR_MODE_OTG:
 		INIT_WORK(&dwc->drd_work, __dwc3_set_mode);
-- 
2.17.1