* msm8996: qcom-qmp: apq8096-db820c fails to boot, reset back to fastboot and locks up @ 2019-06-10 13:44 Paolo Pisati 2019-06-11 17:12 ` Niklas Cassel 2019-06-13 9:19 ` Marc Gonzalez 0 siblings, 2 replies; 7+ messages in thread From: Paolo Pisati @ 2019-06-10 13:44 UTC (permalink / raw) To: linux-arm-msm, linux-kernel From time to time, my apq8096-db820c fails to boot to userspace, reset back to fastboot and locks up: to easily reproduce the issue, i'm boot looping using a cron job with a 1 min reboot entry on the board while leaving a "while 1; do fastboot boot boot.img; done" on the host pc. The issue is present in mainline up to 5.2-rc4, using defconfig and: CONFIG_SCSI_UFS_QCOM=y CONFIG_PHY_QCOM_QMP=y CONFIG_PHY_QCOM_UFS=y but was present in previous releases too (e.g. 4.14., 4.19, etc qcom-lt or mainline), where it's even easier to reproduce (e.g. takes way less reboots to trigger it). These are the last lines printed out: ... [ 7.407209] qcom-qmp-phy 34000.phy: Registered Qcom-QMP phy [ 7.448058] qcom-qmp-phy 7410000.phy: Registered Qcom-QMP phy [ 7.461859] ufs_qcom_phy_qmp_14nm 627000.phy: invalid resource [ 7.535434] qcom-qmp-phy 34000.phy: phy common block init timed-out [ 7.538596] phy phy-34000.phy.0: phy init failed --> -110 [ 7.550891] qcom-pcie: probe of 600000.pcie failed with error -110 [ 7.619008] qcom-pcie 608000.pcie: 608000.pcie supply vddpe-3v3 not found, using dummy regulator Log Type: B - Since Boot(Power On Reset), D - Delta, S - Statistic S - QC_IMAGE_VERSION_STRING=BOOT.XF.1.0-00301 S - IMAGE_VARIANT_STRING=M8996LAB S - OEM_IMAGE_VERSION_STRING=crm-ubuntu68 S - Boot Interface: UFS S - Secure Boot: Off ... Full boot here: https://pastebin.ubuntu.com/p/rtjVrD3yzk/ Any idea what is going on? Am i doing something wrong? -- bye, p. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: msm8996: qcom-qmp: apq8096-db820c fails to boot, reset back to fastboot and locks up 2019-06-10 13:44 msm8996: qcom-qmp: apq8096-db820c fails to boot, reset back to fastboot and locks up Paolo Pisati @ 2019-06-11 17:12 ` Niklas Cassel 2019-06-12 13:17 ` Niklas Cassel 2019-06-13 9:19 ` Marc Gonzalez 1 sibling, 1 reply; 7+ messages in thread From: Niklas Cassel @ 2019-06-11 17:12 UTC (permalink / raw) To: Paolo Pisati; +Cc: linux-arm-msm, linux-kernel On Mon, Jun 10, 2019 at 03:44:01PM +0200, Paolo Pisati wrote: > From time to time, my apq8096-db820c fails to boot to userspace, reset back to > fastboot and locks up: to easily reproduce the issue, i'm boot looping using a > cron job with a 1 min reboot entry on the board while leaving a "while 1; do > fastboot boot boot.img; done" on the host pc. > > The issue is present in mainline up to 5.2-rc4, using defconfig and: > > CONFIG_SCSI_UFS_QCOM=y > CONFIG_PHY_QCOM_QMP=y > CONFIG_PHY_QCOM_UFS=y > > but was present in previous releases too (e.g. 4.14., 4.19, etc qcom-lt or > mainline), where it's even easier to reproduce (e.g. takes way less reboots to > trigger it). Hello Paolo, I have a guess of what is going on. db820c has 3 PCIe controllers, that shares a singe QMP block (that has clocks, regulators, and resets). The QMP block has 3 PCIe PHYs, that have their own clocks and resets. > > These are the last lines printed out: > ... > [ 7.407209] qcom-qmp-phy 34000.phy: Registered Qcom-QMP phy > [ 7.448058] qcom-qmp-phy 7410000.phy: Registered Qcom-QMP phy > [ 7.461859] ufs_qcom_phy_qmp_14nm 627000.phy: invalid resource > [ 7.535434] qcom-qmp-phy 34000.phy: phy common block init timed-out ^^ here the phy_init() called from pcie-qcom.c which ends up to a call to qcom_qmp_phy_enable() which has this code: ret = qcom_qmp_phy_com_init(qphy); if (ret) return ret; qcom_qmp_phy_com_init() has this code: if (qmp->init_count++) { mutex_unlock(&qmp->phy_mutex); return 0; } qcom_qmp_phy_com_init() later fails, since the common block init time out, so the qmp driver disables clocks, asserts reset, and disables regulators > [ 7.538596] phy phy-34000.phy.0: phy init failed --> -110 > [ 7.550891] qcom-pcie: probe of 600000.pcie failed with error -110 ^^ here the first PCIe controller instance fails to probe > [ 7.619008] qcom-pcie 608000.pcie: 608000.pcie supply vddpe-3v3 not found, > using dummy regulator ^^ here the second PCIe controller is probed. it will call phy_init() which will again call qcom_qmp_phy_enable() which will call qcom_qmp_phy_com_init() where this code: if (qmp->init_count++) { mutex_unlock(&qmp->phy_mutex); return 0; } now will return 0, so clocks will never be enabled, resets never deasserted, regulators never enabled. since qcom_qmp_phy_com_init() returns success in this case, qcom_qmp_phy_enable() will try to continue with the init, and writes to disabled hardware is usually not a good idea. I think the proper fix for this is: diff --git a/drivers/phy/qualcomm/phy-qcom-qmp.c b/drivers/phy/qualcomm/phy-qcom-qmp.c index cd91b4179b10..22352e3b0ec5 100644 --- a/drivers/phy/qualcomm/phy-qcom-qmp.c +++ b/drivers/phy/qualcomm/phy-qcom-qmp.c @@ -1490,7 +1490,7 @@ static int qcom_qmp_phy_enable(struct phy *phy) ret = qcom_qmp_phy_com_init(qphy); if (ret) - return ret; + goto err_lane_rst; if (cfg->has_lane_rst) { ret = reset_control_deassert(qphy->lane_rst); Kind regards, Niklas > > Log Type: B - Since Boot(Power On Reset), D - Delta, S - Statistic > S - QC_IMAGE_VERSION_STRING=BOOT.XF.1.0-00301 > S - IMAGE_VARIANT_STRING=M8996LAB > S - OEM_IMAGE_VERSION_STRING=crm-ubuntu68 > S - Boot Interface: UFS > S - Secure Boot: Off > ... > > Full boot here: https://pastebin.ubuntu.com/p/rtjVrD3yzk/ > > Any idea what is going on? Am i doing something wrong? > -- > bye, > p. ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: msm8996: qcom-qmp: apq8096-db820c fails to boot, reset back to fastboot and locks up 2019-06-11 17:12 ` Niklas Cassel @ 2019-06-12 13:17 ` Niklas Cassel 2019-06-12 14:09 ` Paolo Pisati 0 siblings, 1 reply; 7+ messages in thread From: Niklas Cassel @ 2019-06-12 13:17 UTC (permalink / raw) To: Paolo Pisati, Vivek Gautam; +Cc: linux-arm-msm, linux-kernel On Tue, Jun 11, 2019 at 07:12:25PM +0200, Niklas Cassel wrote: > On Mon, Jun 10, 2019 at 03:44:01PM +0200, Paolo Pisati wrote: > > From time to time, my apq8096-db820c fails to boot to userspace, reset back to > > fastboot and locks up: to easily reproduce the issue, i'm boot looping using a > > cron job with a 1 min reboot entry on the board while leaving a "while 1; do > > fastboot boot boot.img; done" on the host pc. > > > > The issue is present in mainline up to 5.2-rc4, using defconfig and: > > > > CONFIG_SCSI_UFS_QCOM=y > > CONFIG_PHY_QCOM_QMP=y > > CONFIG_PHY_QCOM_UFS=y > > > > but was present in previous releases too (e.g. 4.14., 4.19, etc qcom-lt or > > mainline), where it's even easier to reproduce (e.g. takes way less reboots to > > trigger it). > > Hello Paolo, > > I have a guess of what is going on. > db820c has 3 PCIe controllers, > that shares a singe QMP block (that has clocks, regulators, and resets). > The QMP block has 3 PCIe PHYs, that have their own clocks and resets. > > > > > These are the last lines printed out: > > ... > > [ 7.407209] qcom-qmp-phy 34000.phy: Registered Qcom-QMP phy > > [ 7.448058] qcom-qmp-phy 7410000.phy: Registered Qcom-QMP phy > > [ 7.461859] ufs_qcom_phy_qmp_14nm 627000.phy: invalid resource > > [ 7.535434] qcom-qmp-phy 34000.phy: phy common block init timed-out > > ^^ here the phy_init() called from pcie-qcom.c > which ends up to a call to qcom_qmp_phy_enable() > > which has this code: > > ret = qcom_qmp_phy_com_init(qphy); > if (ret) > return ret; > > qcom_qmp_phy_com_init() has this code: > > if (qmp->init_count++) { > mutex_unlock(&qmp->phy_mutex); > return 0; > } > > qcom_qmp_phy_com_init() later fails, > since the common block init time out, so the qmp driver > disables clocks, asserts reset, and disables regulators > > > > [ 7.538596] phy phy-34000.phy.0: phy init failed --> -110 > > [ 7.550891] qcom-pcie: probe of 600000.pcie failed with error -110 > > ^^ here the first PCIe controller instance fails to probe > > > [ 7.619008] qcom-pcie 608000.pcie: 608000.pcie supply vddpe-3v3 not found, > > using dummy regulator > > ^^ here the second PCIe controller is probed. > > it will call phy_init() > > which will again call qcom_qmp_phy_enable() which will call > qcom_qmp_phy_com_init() > > where this code: > > if (qmp->init_count++) { > mutex_unlock(&qmp->phy_mutex); > return 0; > } > > now will return 0, > > so clocks will never be enabled, resets never deasserted, regulators > never enabled. > > since qcom_qmp_phy_com_init() returns success in this case, > qcom_qmp_phy_enable() will try to continue with the init, > and writes to disabled hardware is usually not a good idea. > > I think the proper fix for this is: > > diff --git a/drivers/phy/qualcomm/phy-qcom-qmp.c b/drivers/phy/qualcomm/phy-qcom-qmp.c > index cd91b4179b10..22352e3b0ec5 100644 > --- a/drivers/phy/qualcomm/phy-qcom-qmp.c > +++ b/drivers/phy/qualcomm/phy-qcom-qmp.c > @@ -1490,7 +1490,7 @@ static int qcom_qmp_phy_enable(struct phy *phy) > > ret = qcom_qmp_phy_com_init(qphy); > if (ret) > - return ret; > + goto err_lane_rst; > > if (cfg->has_lane_rst) { > ret = reset_control_deassert(qphy->lane_rst); > > > > Kind regards, > Niklas > > > > > Log Type: B - Since Boot(Power On Reset), D - Delta, S - Statistic > > S - QC_IMAGE_VERSION_STRING=BOOT.XF.1.0-00301 > > S - IMAGE_VARIANT_STRING=M8996LAB > > S - OEM_IMAGE_VERSION_STRING=crm-ubuntu68 > > S - Boot Interface: UFS > > S - Secure Boot: Off > > ... > > > > Full boot here: https://pastebin.ubuntu.com/p/rtjVrD3yzk/ > > > > Any idea what is going on? Am i doing something wrong? > > -- > > bye, > > p. Adding Vivek. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: msm8996: qcom-qmp: apq8096-db820c fails to boot, reset back to fastboot and locks up 2019-06-12 13:17 ` Niklas Cassel @ 2019-06-12 14:09 ` Paolo Pisati 2019-06-12 16:20 ` Niklas Cassel 0 siblings, 1 reply; 7+ messages in thread From: Paolo Pisati @ 2019-06-12 14:09 UTC (permalink / raw) To: Niklas Cassel; +Cc: Paolo Pisati, Vivek Gautam, linux-arm-msm, linux-kernel On Wed, Jun 12, 2019 at 03:17:35PM +0200, Niklas Cassel wrote: > > diff --git a/drivers/phy/qualcomm/phy-qcom-qmp.c b/drivers/phy/qualcomm/phy-qcom-qmp.c > > index cd91b4179b10..22352e3b0ec5 100644 > > --- a/drivers/phy/qualcomm/phy-qcom-qmp.c > > +++ b/drivers/phy/qualcomm/phy-qcom-qmp.c > > @@ -1490,7 +1490,7 @@ static int qcom_qmp_phy_enable(struct phy *phy) > > > > ret = qcom_qmp_phy_com_init(qphy); > > if (ret) > > - return ret; > > + goto err_lane_rst; > > > > if (cfg->has_lane_rst) { > > ret = reset_control_deassert(qphy->lane_rst); Hi Niklas, unfortunately, it didn't help - i added a printk, to highlight when it failed: --- a/drivers/phy/qualcomm/phy-qcom-qmp.c +++ b/drivers/phy/qualcomm/phy-qcom-qmp.c @@ -1489,8 +1489,10 @@ static int qcom_qmp_phy_enable(struct phy *phy) } ret = qcom_qmp_phy_com_init(qphy); - if (ret) - return ret; + if (ret) { + dev_err(qmp->dev, "qphy initialization failed\n"); + goto err_lane_rst; + } if (cfg->has_lane_rst) { ret = reset_control_deassert(qphy->lane_rst); After several reboots i was able to trigger the phy init failure again: ... [ 2.223999] qcom-qmp-phy 34000.phy: Registered Qcom-QMP phy [ 2.224956] qcom-qmp-phy 7410000.phy: Registered Qcom-QMP phy [ 2.228798] ufs_qcom_phy_qmp_14nm 627000.phy: invalid resource [ 2.237271] qcom-qmp-phy 34000.phy: phy common block init timed-out [ 2.240315] qcom-qmp-phy 34000.phy: qphy initialization failed ... these are the last lines printed, before rebooting in fastboot and locking up there (as before[*]). So, as far as i understand there are two distinct problems: 1) sometimes, qcom-qmp-phy fails to initialize 2) and when that happens, the failure is fatal and it led to a reboot & lockup in fastboot 1: https://pastebin.ubuntu.com/p/rtjVrD3yzk/ -- bye, p. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: msm8996: qcom-qmp: apq8096-db820c fails to boot, reset back to fastboot and locks up 2019-06-12 14:09 ` Paolo Pisati @ 2019-06-12 16:20 ` Niklas Cassel 2019-06-13 8:57 ` Paolo Pisati 0 siblings, 1 reply; 7+ messages in thread From: Niklas Cassel @ 2019-06-12 16:20 UTC (permalink / raw) To: Paolo Pisati; +Cc: Vivek Gautam, linux-arm-msm, linux-kernel On Wed, Jun 12, 2019 at 04:09:11PM +0200, Paolo Pisati wrote: > On Wed, Jun 12, 2019 at 03:17:35PM +0200, Niklas Cassel wrote: > > > diff --git a/drivers/phy/qualcomm/phy-qcom-qmp.c b/drivers/phy/qualcomm/phy-qcom-qmp.c > > > index cd91b4179b10..22352e3b0ec5 100644 > > > --- a/drivers/phy/qualcomm/phy-qcom-qmp.c > > > +++ b/drivers/phy/qualcomm/phy-qcom-qmp.c > > > @@ -1490,7 +1490,7 @@ static int qcom_qmp_phy_enable(struct phy *phy) > > > > > > ret = qcom_qmp_phy_com_init(qphy); > > > if (ret) > > > - return ret; > > > + goto err_lane_rst; > > > > > > if (cfg->has_lane_rst) { > > > ret = reset_control_deassert(qphy->lane_rst); > > Hi Niklas, > unfortunately, it didn't help - i added a printk, to highlight when it failed: > > --- a/drivers/phy/qualcomm/phy-qcom-qmp.c > +++ b/drivers/phy/qualcomm/phy-qcom-qmp.c > @@ -1489,8 +1489,10 @@ static int qcom_qmp_phy_enable(struct phy *phy) > } > > ret = qcom_qmp_phy_com_init(qphy); > - if (ret) > - return ret; > + if (ret) { > + dev_err(qmp->dev, "qphy initialization failed\n"); > + goto err_lane_rst; > + } > > if (cfg->has_lane_rst) { > ret = reset_control_deassert(qphy->lane_rst); > > After several reboots i was able to trigger the phy init failure again: > > ... > [ 2.223999] qcom-qmp-phy 34000.phy: Registered Qcom-QMP phy > [ 2.224956] qcom-qmp-phy 7410000.phy: Registered Qcom-QMP phy > [ 2.228798] ufs_qcom_phy_qmp_14nm 627000.phy: invalid resource > [ 2.237271] qcom-qmp-phy 34000.phy: phy common block init timed-out > [ 2.240315] qcom-qmp-phy 34000.phy: qphy initialization failed > ... I still think that the above patch is correct, even though it didn't fix your problem. If you try to disable the two other PCIe controllers: diff --git a/arch/arm64/boot/dts/qcom/apq8096-db820c.dtsi b/arch/arm64/boot/dts/qcom/apq8096-db820c.dtsi index 943f69912074..95900fe99f89 100644 --- a/arch/arm64/boot/dts/qcom/apq8096-db820c.dtsi +++ b/arch/arm64/boot/dts/qcom/apq8096-db820c.dtsi @@ -455,12 +455,12 @@ }; pcie@608000 { - status = "okay"; + status = "disabled"; perst-gpio = <&msmgpio 130 GPIO_ACTIVE_LOW>; }; pcie@610000 { - status = "okay"; + status = "disabled"; perst-gpio = <&msmgpio 114 GPIO_ACTIVE_LOW>; }; }; Can you still reproduce the reboot? Kind regards, Niklas > > these are the last lines printed, before rebooting in fastboot and > locking up there (as before[*]). > > So, as far as i understand there are two distinct problems: > > 1) sometimes, qcom-qmp-phy fails to initialize > > 2) and when that happens, the failure is fatal and it led to a reboot & lockup > in fastboot > > 1: https://pastebin.ubuntu.com/p/rtjVrD3yzk/ > -- > bye, > p. ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: msm8996: qcom-qmp: apq8096-db820c fails to boot, reset back to fastboot and locks up 2019-06-12 16:20 ` Niklas Cassel @ 2019-06-13 8:57 ` Paolo Pisati 0 siblings, 0 replies; 7+ messages in thread From: Paolo Pisati @ 2019-06-13 8:57 UTC (permalink / raw) To: Niklas Cassel; +Cc: Paolo Pisati, Vivek Gautam, linux-arm-msm, linux-kernel On Wed, Jun 12, 2019 at 06:20:48PM +0200, Niklas Cassel wrote: > > Can you still reproduce the reboot? --- a/arch/arm64/boot/dts/qcom/apq8096-db820c.dtsi +++ b/arch/arm64/boot/dts/qcom/apq8096-db820c.dtsi @@ -447,12 +447,12 @@ }; pcie@608000 { - status = "okay"; + status = "disabled"; perst-gpio = <&msmgpio 130 GPIO_ACTIVE_LOW>; }; pcie@610000 { - status = "okay"; + status = "disabled"; perst-gpio = <&msmgpio 114 GPIO_ACTIVE_LOW>; }; }; disabling the two other pci controllers (and leaving your previous patch applied), made the issue more frequent: i was never able to boot to user space three times in a row without experiencing a crash, previously it was more sporadic. On the other hand, now i can reproduce the issue more easily this way. -- bye, p. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: msm8996: qcom-qmp: apq8096-db820c fails to boot, reset back to fastboot and locks up 2019-06-10 13:44 msm8996: qcom-qmp: apq8096-db820c fails to boot, reset back to fastboot and locks up Paolo Pisati 2019-06-11 17:12 ` Niklas Cassel @ 2019-06-13 9:19 ` Marc Gonzalez 1 sibling, 0 replies; 7+ messages in thread From: Marc Gonzalez @ 2019-06-13 9:19 UTC (permalink / raw) To: Paolo Pisati Cc: MSM, LKML, Bjorn Andersson, Niklas Cassel, Stanimir Varbanov, Vivek Gautam, Srinivas Kandagatla On 10/06/2019 15:44, Paolo Pisati wrote: > From time to time, my apq8096-db820c fails to boot to userspace, reset back to > fastboot and locks up: to easily reproduce the issue, i'm boot looping using a > cron job with a 1 min reboot entry on the board while leaving a "while 1; do > fastboot boot boot.img; done" on the host pc. > > The issue is present in mainline up to 5.2-rc4, using defconfig and: > > CONFIG_SCSI_UFS_QCOM=y > CONFIG_PHY_QCOM_QMP=y > CONFIG_PHY_QCOM_UFS=y > > but was present in previous releases too (e.g. 4.14., 4.19, etc qcom-lt or > mainline), where it's even easier to reproduce (e.g. takes way less reboots to > trigger it). > > These are the last lines printed out: > ... > [ 7.407209] qcom-qmp-phy 34000.phy: Registered Qcom-QMP phy > [ 7.448058] qcom-qmp-phy 7410000.phy: Registered Qcom-QMP phy > [ 7.461859] ufs_qcom_phy_qmp_14nm 627000.phy: invalid resource > [ 7.535434] qcom-qmp-phy 34000.phy: phy common block init timed-out > [ 7.538596] phy phy-34000.phy.0: phy init failed --> -110 > [ 7.550891] qcom-pcie: probe of 600000.pcie failed with error -110 > [ 7.619008] qcom-pcie 608000.pcie: 608000.pcie supply vddpe-3v3 not found, > using dummy regulator > > Log Type: B - Since Boot(Power On Reset), D - Delta, S - Statistic > S - QC_IMAGE_VERSION_STRING=BOOT.XF.1.0-00301 > S - IMAGE_VARIANT_STRING=M8996LAB > S - OEM_IMAGE_VERSION_STRING=crm-ubuntu68 > S - Boot Interface: UFS > S - Secure Boot: Off > ... > > Full boot here: https://pastebin.ubuntu.com/p/rtjVrD3yzk/ > > Any idea what is going on? Am i doing something wrong? Hmmm... this might be related to: https://patchwork.kernel.org/patch/10976217/ Regards. ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-06-13 15:53 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-06-10 13:44 msm8996: qcom-qmp: apq8096-db820c fails to boot, reset back to fastboot and locks up Paolo Pisati 2019-06-11 17:12 ` Niklas Cassel 2019-06-12 13:17 ` Niklas Cassel 2019-06-12 14:09 ` Paolo Pisati 2019-06-12 16:20 ` Niklas Cassel 2019-06-13 8:57 ` Paolo Pisati 2019-06-13 9:19 ` Marc Gonzalez
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).