* Sierra Wireless EM9191 integration issues in mhi+wwan
@ 2021-10-07 13:04 Aleksander Morgado
2021-10-09 10:51 ` Manivannan Sadhasivam
2021-10-11 14:44 ` Thomas Perrot
0 siblings, 2 replies; 33+ messages in thread
From: Aleksander Morgado @ 2021-10-07 13:04 UTC (permalink / raw)
To: Manivannan Sadhasivam, Loic Poulain, Thomas Perrot, hemantk,
Thomas Petazzoni, linux-arm-msm
Hey all,
I'm working on a setup using a RPi CM4 based board and a Sierra
Wireless EM9191 module, running OpenWRT 21.02 and backported mhi+wwan
drivers (from latest 5.15 rc). The kernel also has the
mhi_pci_dev_info entry written by Thomas, as per
https://forum.sierrawireless.com/t/sierra-wireless-airprime-em919x-pcie-support/24927
The EM9191 is now running 02.08.01.00 (latest firmware from Sierra),
upgraded in several steps back from the original 01.04.01.02 the
module came with. The firmware upgrades were done with
qmi-firmware-update and the module in USB mode in a desktop PC.
Most of the system boots end up with the mhi driver reporting that the
module firmware crashed in different ways:
[ 7.060730] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
[ 7.067906] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 7.076455] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
[ 7.083277] mhi-pci-generic 0000:01:00.0: using shared MSI
[ 7.089508] mhi mhi0: Requested to power ON
[ 7.094080] mhi mhi0: Attempting power on with EE: PASS THROUGH,
state: SYS ERROR
[ 7.180371] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
PASS THROUGH state: SYS ERROR
[ 7.187146] mhi mhi0: Power on setup success
[ 7.187219] mhi mhi0: Handling state transition: PBL
[ 7.189165] mhi mhi0: System error detected
[ 7.189178] mhi mhi0: Device MHI is not in valid state
[ 7.189189] mhi-pci-generic 0000:01:00.0: firmware crashed (7)
[ 7.213682] mhi mhi0: Handling state transition: SYS ERROR
[ 7.219183] mhi mhi0: Transitioning from PM state: Linkdown or
Error Fatal Detect to: SYS ERROR Process
[ 7.228590] mhi-pci-generic 0000:01:00.0: firmware crashed (6)
[ 7.234429] mhi mhi0: Failed to transition from PM state: Linkdown
or Error Fatal Detect to: SYS ERROR Process
[ 7.244433] mhi mhi0: Exiting with PM state: Linkdown or Error
Fatal Detect, MHI state: RESET
[ 7.252963] mhi mhi0: Handling state transition: DISABLE
[ 7.258278] mhi mhi0: Processing disable transition with PM state:
Linkdown or Error Fatal Detect
[ 7.267155] mhi mhi0: Waiting for all pending event ring processing
to complete
[ 7.274480] mhi mhi0: Waiting for all pending threads to complete
[ 7.280576] mhi mhi0: Reset all active channels and remove MHI devices
[ 7.287110] mhi mhi0: Resetting EV CTXT and CMD CTXT
[ 7.292077] mhi mhi0: Exiting with PM state: DISABLE, MHI state: RESET
[ 7.298683] mhi-pci-generic 0000:01:00.0: failed to power up MHI controller
[ 7.306184] mhi-pci-generic: probe of 0000:01:00.0 failed with error -110
Some of the boots successfully finish and I can talk to both the QMI
and MBIM ports exposed by the WWAN subsystem, but the success rate is
extremely low.
Thomas, are you seeing similar issues in your setup?
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-07 13:04 Sierra Wireless EM9191 integration issues in mhi+wwan Aleksander Morgado
@ 2021-10-09 10:51 ` Manivannan Sadhasivam
2021-10-12 19:38 ` Aleksander Morgado
2021-10-11 14:44 ` Thomas Perrot
1 sibling, 1 reply; 33+ messages in thread
From: Manivannan Sadhasivam @ 2021-10-09 10:51 UTC (permalink / raw)
To: Aleksander Morgado
Cc: Loic Poulain, Thomas Perrot, hemantk, Thomas Petazzoni,
linux-arm-msm, bbhatt
+ Bhaumik
On Thu, Oct 07, 2021 at 03:04:44PM +0200, Aleksander Morgado wrote:
> Hey all,
>
> I'm working on a setup using a RPi CM4 based board and a Sierra
> Wireless EM9191 module, running OpenWRT 21.02 and backported mhi+wwan
> drivers (from latest 5.15 rc). The kernel also has the
> mhi_pci_dev_info entry written by Thomas, as per
> https://forum.sierrawireless.com/t/sierra-wireless-airprime-em919x-pcie-support/24927
>
> The EM9191 is now running 02.08.01.00 (latest firmware from Sierra),
> upgraded in several steps back from the original 01.04.01.02 the
> module came with. The firmware upgrades were done with
> qmi-firmware-update and the module in USB mode in a desktop PC.
>
> Most of the system boots end up with the mhi driver reporting that the
> module firmware crashed in different ways:
>
> [ 7.060730] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
> [ 7.067906] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
> 0x600000000-0x600000fff 64bit]
> [ 7.076455] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
> [ 7.083277] mhi-pci-generic 0000:01:00.0: using shared MSI
> [ 7.089508] mhi mhi0: Requested to power ON
> [ 7.094080] mhi mhi0: Attempting power on with EE: PASS THROUGH,
> state: SYS ERROR
> [ 7.180371] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
> PASS THROUGH state: SYS ERROR
> [ 7.187146] mhi mhi0: Power on setup success
> [ 7.187219] mhi mhi0: Handling state transition: PBL
> [ 7.189165] mhi mhi0: System error detected
> [ 7.189178] mhi mhi0: Device MHI is not in valid state
> [ 7.189189] mhi-pci-generic 0000:01:00.0: firmware crashed (7)
> [ 7.213682] mhi mhi0: Handling state transition: SYS ERROR
> [ 7.219183] mhi mhi0: Transitioning from PM state: Linkdown or
> Error Fatal Detect to: SYS ERROR Process
> [ 7.228590] mhi-pci-generic 0000:01:00.0: firmware crashed (6)
> [ 7.234429] mhi mhi0: Failed to transition from PM state: Linkdown
> or Error Fatal Detect to: SYS ERROR Process
> [ 7.244433] mhi mhi0: Exiting with PM state: Linkdown or Error
> Fatal Detect, MHI state: RESET
> [ 7.252963] mhi mhi0: Handling state transition: DISABLE
> [ 7.258278] mhi mhi0: Processing disable transition with PM state:
> Linkdown or Error Fatal Detect
> [ 7.267155] mhi mhi0: Waiting for all pending event ring processing
> to complete
> [ 7.274480] mhi mhi0: Waiting for all pending threads to complete
> [ 7.280576] mhi mhi0: Reset all active channels and remove MHI devices
> [ 7.287110] mhi mhi0: Resetting EV CTXT and CMD CTXT
> [ 7.292077] mhi mhi0: Exiting with PM state: DISABLE, MHI state: RESET
> [ 7.298683] mhi-pci-generic 0000:01:00.0: failed to power up MHI controller
> [ 7.306184] mhi-pci-generic: probe of 0000:01:00.0 failed with error -110
>
> Some of the boots successfully finish and I can talk to both the QMI
> and MBIM ports exposed by the WWAN subsystem, but the success rate is
> extremely low.
>
Can you share the success log also?
Thanks,
Mani
> Thomas, are you seeing similar issues in your setup?
>
> --
> Aleksander
> https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-07 13:04 Sierra Wireless EM9191 integration issues in mhi+wwan Aleksander Morgado
2021-10-09 10:51 ` Manivannan Sadhasivam
@ 2021-10-11 14:44 ` Thomas Perrot
2021-10-12 19:44 ` Aleksander Morgado
2021-11-08 15:11 ` Aleksander Morgado
1 sibling, 2 replies; 33+ messages in thread
From: Thomas Perrot @ 2021-10-11 14:44 UTC (permalink / raw)
To: Aleksander Morgado, Manivannan Sadhasivam, Loic Poulain, hemantk,
Thomas Petazzoni, linux-arm-msm
[-- Attachment #1: Type: text/plain, Size: 3967 bytes --]
Hi Aleksander,
On Thu, 2021-10-07 at 15:04 +0200, Aleksander Morgado wrote:
> Hey all,
>
> I'm working on a setup using a RPi CM4 based board and a Sierra
> Wireless EM9191 module, running OpenWRT 21.02 and backported mhi+wwan
> drivers (from latest 5.15 rc). The kernel also has the
> mhi_pci_dev_info entry written by Thomas, as per
> https://forum.sierrawireless.com/t/sierra-wireless-airprime-em919x-pcie-support/24927
>
> The EM9191 is now running 02.08.01.00 (latest firmware from Sierra),
> upgraded in several steps back from the original 01.04.01.02 the
> module came with. The firmware upgrades were done with
> qmi-firmware-update and the module in USB mode in a desktop PC.
>
> Most of the system boots end up with the mhi driver reporting that the
> module firmware crashed in different ways:
>
> [ 7.060730] mhi-pci-generic 0000:01:00.0: MHI PCI device found:
> sierra-em919x
> [ 7.067906] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
> 0x600000000-0x600000fff 64bit]
> [ 7.076455] mhi-pci-generic 0000:01:00.0: enabling device (0000 ->
> 0002)
> [ 7.083277] mhi-pci-generic 0000:01:00.0: using shared MSI
> [ 7.089508] mhi mhi0: Requested to power ON
> [ 7.094080] mhi mhi0: Attempting power on with EE: PASS THROUGH,
> state: SYS ERROR
> [ 7.180371] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
> PASS THROUGH state: SYS ERROR
> [ 7.187146] mhi mhi0: Power on setup success
> [ 7.187219] mhi mhi0: Handling state transition: PBL
> [ 7.189165] mhi mhi0: System error detected
> [ 7.189178] mhi mhi0: Device MHI is not in valid state
> [ 7.189189] mhi-pci-generic 0000:01:00.0: firmware crashed (7)
> [ 7.213682] mhi mhi0: Handling state transition: SYS ERROR
> [ 7.219183] mhi mhi0: Transitioning from PM state: Linkdown or
> Error Fatal Detect to: SYS ERROR Process
> [ 7.228590] mhi-pci-generic 0000:01:00.0: firmware crashed (6)
> [ 7.234429] mhi mhi0: Failed to transition from PM state: Linkdown
> or Error Fatal Detect to: SYS ERROR Process
> [ 7.244433] mhi mhi0: Exiting with PM state: Linkdown or Error
> Fatal Detect, MHI state: RESET
> [ 7.252963] mhi mhi0: Handling state transition: DISABLE
> [ 7.258278] mhi mhi0: Processing disable transition with PM state:
> Linkdown or Error Fatal Detect
> [ 7.267155] mhi mhi0: Waiting for all pending event ring processing
> to complete
> [ 7.274480] mhi mhi0: Waiting for all pending threads to complete
> [ 7.280576] mhi mhi0: Reset all active channels and remove MHI
> devices
> [ 7.287110] mhi mhi0: Resetting EV CTXT and CMD CTXT
> [ 7.292077] mhi mhi0: Exiting with PM state: DISABLE, MHI state:
> RESET
> [ 7.298683] mhi-pci-generic 0000:01:00.0: failed to power up MHI
> controller
> [ 7.306184] mhi-pci-generic: probe of 0000:01:00.0 failed with error
> -110
>
> Some of the boots successfully finish and I can talk to both the QMI
> and MBIM ports exposed by the WWAN subsystem, but the success rate is
> extremely low.
>
> Thomas, are you seeing similar issues in your setup?
On our setup, using i.MX6DL based board and a PCIe Sierra Wireless
EM9190 module, running Yocto and Linux 5.13, we don't have much success
for the moment, qmi and mbim commands very often end in timeout.
Otherwise, when responses are received, we also can observe strange
things: unexpected messages, response to previous commands or queue
buffer issue.
Moreover, AT commands and the firmware update seem to work fine.
I updated the topic opened on the Sierra Wireless forum, with our
latest progress and as well as additional information.
In addition, we observed some strange behavior of the EM919x after warm
reboots.
Best regards,
Thomas
>
--
Thomas Perrot, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-09 10:51 ` Manivannan Sadhasivam
@ 2021-10-12 19:38 ` Aleksander Morgado
2021-10-22 4:42 ` Manivannan Sadhasivam
0 siblings, 1 reply; 33+ messages in thread
From: Aleksander Morgado @ 2021-10-12 19:38 UTC (permalink / raw)
To: Manivannan Sadhasivam
Cc: Loic Poulain, Thomas Perrot, hemantk, Thomas Petazzoni,
linux-arm-msm, Bhaumik Bhatt
Hey
> > I'm working on a setup using a RPi CM4 based board and a Sierra
> > Wireless EM9191 module, running OpenWRT 21.02 and backported mhi+wwan
> > drivers (from latest 5.15 rc). The kernel also has the
> > mhi_pci_dev_info entry written by Thomas, as per
> > https://forum.sierrawireless.com/t/sierra-wireless-airprime-em919x-pcie-support/24927
> >
> > The EM9191 is now running 02.08.01.00 (latest firmware from Sierra),
> > upgraded in several steps back from the original 01.04.01.02 the
> > module came with. The firmware upgrades were done with
> > qmi-firmware-update and the module in USB mode in a desktop PC.
> >
> > Most of the system boots end up with the mhi driver reporting that the
> > module firmware crashed in different ways:
> >
> > [ 7.060730] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
> > [ 7.067906] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
> > 0x600000000-0x600000fff 64bit]
> > [ 7.076455] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
> > [ 7.083277] mhi-pci-generic 0000:01:00.0: using shared MSI
> > [ 7.089508] mhi mhi0: Requested to power ON
> > [ 7.094080] mhi mhi0: Attempting power on with EE: PASS THROUGH,
> > state: SYS ERROR
> > [ 7.180371] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
> > PASS THROUGH state: SYS ERROR
> > [ 7.187146] mhi mhi0: Power on setup success
> > [ 7.187219] mhi mhi0: Handling state transition: PBL
> > [ 7.189165] mhi mhi0: System error detected
> > [ 7.189178] mhi mhi0: Device MHI is not in valid state
> > [ 7.189189] mhi-pci-generic 0000:01:00.0: firmware crashed (7)
> > [ 7.213682] mhi mhi0: Handling state transition: SYS ERROR
> > [ 7.219183] mhi mhi0: Transitioning from PM state: Linkdown or
> > Error Fatal Detect to: SYS ERROR Process
> > [ 7.228590] mhi-pci-generic 0000:01:00.0: firmware crashed (6)
> > [ 7.234429] mhi mhi0: Failed to transition from PM state: Linkdown
> > or Error Fatal Detect to: SYS ERROR Process
> > [ 7.244433] mhi mhi0: Exiting with PM state: Linkdown or Error
> > Fatal Detect, MHI state: RESET
> > [ 7.252963] mhi mhi0: Handling state transition: DISABLE
> > [ 7.258278] mhi mhi0: Processing disable transition with PM state:
> > Linkdown or Error Fatal Detect
> > [ 7.267155] mhi mhi0: Waiting for all pending event ring processing
> > to complete
> > [ 7.274480] mhi mhi0: Waiting for all pending threads to complete
> > [ 7.280576] mhi mhi0: Reset all active channels and remove MHI devices
> > [ 7.287110] mhi mhi0: Resetting EV CTXT and CMD CTXT
> > [ 7.292077] mhi mhi0: Exiting with PM state: DISABLE, MHI state: RESET
> > [ 7.298683] mhi-pci-generic 0000:01:00.0: failed to power up MHI controller
> > [ 7.306184] mhi-pci-generic: probe of 0000:01:00.0 failed with error -110
> >
> > Some of the boots successfully finish and I can talk to both the QMI
> > and MBIM ports exposed by the WWAN subsystem, but the success rate is
> > extremely low.
> >
>
> Can you share the success log also?
>
The successful boots seem to happen always on cold boots, and the
success rate is low (30% or so) on some manual testing here. I haven't
seen one single successful boot on system restarts, they all fail like
in the previous email.
When the boot is successful it looks like this:
[ 0.099938] brcm-pcie fd500000.pcie: host bridge /scb/pcie@7d500000 ranges:
[ 0.099961] brcm-pcie fd500000.pcie: No bus range found for
/scb/pcie@7d500000, using [bus 00-ff]
[ 0.099999] brcm-pcie fd500000.pcie: MEM
0x0600000000..0x0603ffffff -> 0x00f8000000
[ 0.100032] brcm-pcie fd500000.pcie: IB MEM
0x0000000000..0x00bfffffff -> 0x0000000000
[ 0.149955] brcm-pcie fd500000.pcie: link up, 5 GT/s x1 (SSC)
[ 0.150114] brcm-pcie fd500000.pcie: PCI host bridge to bus 0000:00
[ 0.150133] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.150152] pci_bus 0000:00: root bus resource [mem
0x600000000-0x603ffffff] (bus address [0xf8000000-0xfbffffff])
[ 0.150193] pci 0000:00:00.0: [14e4:2711] type 01 class 0x060400
[ 0.150282] pci 0000:00:00.0: PME# supported from D0 D3hot
[ 0.153464] pci 0000:00:00.0: bridge configuration invalid ([bus
ff-ff]), reconfiguring
[ 0.153589] pci 0000:01:00.0: [17cb:0306] type 00 class 0xff0000
[ 0.153650] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00000fff 64bit]
[ 0.153680] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit]
[ 0.153838] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[ 0.153889] pci 0000:01:00.0: 4.000 Gb/s available PCIe bandwidth,
limited by 5 GT/s x1 link at 0000:00:00.0 (capable of 31.506 Gb/s with
16 GT/s x2 link)
[ 0.156944] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[ 0.156983] pci 0000:00:00.0: BAR 8: assigned [mem 0x600000000-0x6000fffff]
[ 0.157004] pci 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 0.157035] pci 0000:01:00.0: BAR 2: assigned [mem
0x600001000-0x600001fff 64bit]
[ 0.157064] pci 0000:00:00.0: PCI bridge to [bus 01]
[ 0.157079] pci 0000:00:00.0: bridge window [mem 0x600000000-0x6000fffff]
[ 0.157222] pcieport 0000:00:00.0: enabling device (0000 -> 0002)
[ 0.157340] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
[ 0.157567] pcieport 0000:00:00.0: AER: enabled with IRQ 38
[ 7.052080] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
[ 7.059284] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 7.067839] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
[ 7.074676] mhi-pci-generic 0000:01:00.0: using shared MSI
[ 7.080858] mhi mhi0: Requested to power ON
[ 7.085357] mhi mhi0: Attempting power on with EE: PASS THROUGH,
state: SYS ERROR
[ 7.431361] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
PASS THROUGH state: RESET
[ 7.432247] mhi mhi0: Power on setup success
[ 7.440015] mhi mhi0: Handling state transition: PBL
[ 8.387907] mhi mhi0: Device in READY State
[ 8.392128] mhi mhi0: Initializing MHI registers
[ 8.396815] mhi mhi0: Wait for device to enter SBL or Mission mode
[ 14.148059] mhi mhi0: State change event to state: M0
[ 14.148076] mhi mhi0: local ee: PASS THROUGH state: READY device
ee: MISSION MODE state: M0
[ 14.161518] mhi mhi0: Received EE event: MISSION MODE
[ 14.161522] mhi mhi0: local ee: PASS THROUGH state: M0 device ee:
MISSION MODE state: M0
[ 14.174684] mhi mhi0: Handling state transition: MISSION MODE
[ 14.180447] mhi mhi0: Processing Mission Mode transition
[ 14.186199] mhi_net mhi0_IP_HW0: 100: Updating channel state to: START
[ 14.361842] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 14.363918] mhi_net mhi0_IP_HW0: 100: Channel state change to START
successful
[ 14.377324] mhi_net mhi0_IP_HW0: 101: Updating channel state to: START
[ 14.490249] mhi_net mhi0_IP_HW0: 101: Channel state change to START
successful
[ 14.490255] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 16.835372] mhi mhi0: Allowing M3 transition
[ 16.839695] mhi mhi0: Waiting for M3 completion
[ 16.966594] mhi mhi0: State change event to state: M3
[ 16.966615] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M3
[ 20.460243] mhi_wwan_ctrl mhi0_DUN: 32: Updating channel state to: START
[ 20.484117] mhi mhi0: Entered with PM state: M3, MHI state: M3
[ 20.499999] mhi mhi0: State change event to state: M0
[ 20.500021] mhi mhi0: local ee: MISSION MODE state: M3 device ee:
MISSION MODE state: M0
[ 20.517791] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
....
When in this state, ModemManager/qmicli/mbimcli can pick up the device
right away, without any issues observed there. Haven't tested data
connection yet.
root@OpenWrt:~# mmcli -m 0
-----------------------------
General | path: /org/freedesktop/ModemManager1/Modem/0
| device id: 58a66d754412bf0a8ec2a840f111856d85a4b053
-----------------------------
Hardware | manufacturer: Sierra Wireless, Incorporated
| model: EM9191
| firmware revision: SWIX55C_02.08.01.00 58f60e jenkins
2021/03/12 18:05:48
| carrier config: default
| h/w revision: 0.2
| supported: gsm-umts, lte, 5gnr
| current: gsm-umts, lte, 5gnr
| equipment id: XXXXXXXXXXXXXX
-----------------------------
System | device:
/sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0
| drivers: mhi-pci-generic, mhi_net
| plugin: generic
| primary port: wwan0qmi0
| ports: mhi_hwip0 (net), wwan0at0 (at),
wwan0mbim0 (mbim),
| wwan0qcdm0 (qcdm), wwan0qmi0 (qmi)
-----------------------------
Status | state: failed
| failed reason: sim-missing
| power state: on
| signal quality: 0% (cached)
-----------------------------
Modes | supported: allowed: 3g; preferred: none
| allowed: 4g; preferred: none
| allowed: 3g, 4g; preferred: 4g
| allowed: 3g, 4g; preferred: 3g
| allowed: 5g; preferred: none
| allowed: 3g, 5g; preferred: 5g
| allowed: 3g, 5g; preferred: 3g
| allowed: 4g, 5g; preferred: 5g
| allowed: 4g, 5g; preferred: 4g
| allowed: 3g, 4g, 5g; preferred: 5g
| allowed: 3g, 4g, 5g; preferred: 4g
| allowed: 3g, 4g, 5g; preferred: 3g
| current: allowed: any; preferred: none
-----------------------------
Bands | supported: utran-1, utran-3, utran-4, utran-6,
utran-5, utran-8,
| utran-9, utran-2, eutran-1, eutran-2,
eutran-3, eutran-4, eutran-5,
| eutran-7, eutran-8, eutran-12,
eutran-13, eutran-14, eutran-17,
| eutran-18, eutran-19, eutran-20,
eutran-25, eutran-26, eutran-28,
| eutran-29, eutran-30, eutran-32,
eutran-34, eutran-38, eutran-39,
| eutran-40, eutran-41, eutran-42,
eutran-46, eutran-48, eutran-66,
| eutran-71, utran-19
-----------------------------
IP | supported: ipv4, ipv6, ipv4v6
-----------------------------
SIM | sim slot paths: slot 1: none (active)
root@OpenWrt:~# qmicli -d /dev/wwan0qmi0 -p --dms-get-capabilities
[/dev/wwan0qmi0] Device capabilities retrieved:
Max TX channel rate: '50000000'
Max RX channel rate: '100000000'
Data Service: 'non-simultaneous-cs-ps'
SIM: 'supported'
Networks: 'umts, lte, 5gnr'
root@OpenWrt:~# mbimcli -d /dev/wwan0mbim0 -p --query-device-caps
[/dev/wwan0mbim0] Device capabilities retrieved:
Device type: 'embedded'
Cellular class: 'gsm'
Voice class: 'no-voice'
SIM class: 'removable'
Data class: 'umts, hsdpa, hsupa, lte, custom'
SMS caps: 'pdu-receive, pdu-send'
Ctrl caps: 'reg-manual'
Max sessions: '15'
Custom data class: '5G/TDS'
Device ID: 'XXXXXXXXXXX'
Firmware info: '02.08.01.00_GENERI_020.007_001'
Hardware info: 'EM9191'
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-11 14:44 ` Thomas Perrot
@ 2021-10-12 19:44 ` Aleksander Morgado
2021-10-14 9:51 ` Thomas Perrot
2021-11-08 15:11 ` Aleksander Morgado
1 sibling, 1 reply; 33+ messages in thread
From: Aleksander Morgado @ 2021-10-12 19:44 UTC (permalink / raw)
To: Thomas Perrot
Cc: Manivannan Sadhasivam, Loic Poulain, hemantk, Thomas Petazzoni,
linux-arm-msm
> > I'm working on a setup using a RPi CM4 based board and a Sierra
> > Wireless EM9191 module, running OpenWRT 21.02 and backported mhi+wwan
> > drivers (from latest 5.15 rc). The kernel also has the
> > mhi_pci_dev_info entry written by Thomas, as per
> > https://forum.sierrawireless.com/t/sierra-wireless-airprime-em919x-pcie-support/24927
> >
> > The EM9191 is now running 02.08.01.00 (latest firmware from Sierra),
> > upgraded in several steps back from the original 01.04.01.02 the
> > module came with. The firmware upgrades were done with
> > qmi-firmware-update and the module in USB mode in a desktop PC.
> >
> > Most of the system boots end up with the mhi driver reporting that the
> > module firmware crashed in different ways:
> >
> > [ 7.060730] mhi-pci-generic 0000:01:00.0: MHI PCI device found:
> > sierra-em919x
> > [ 7.067906] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
> > 0x600000000-0x600000fff 64bit]
> > [ 7.076455] mhi-pci-generic 0000:01:00.0: enabling device (0000 ->
> > 0002)
> > [ 7.083277] mhi-pci-generic 0000:01:00.0: using shared MSI
> > [ 7.089508] mhi mhi0: Requested to power ON
> > [ 7.094080] mhi mhi0: Attempting power on with EE: PASS THROUGH,
> > state: SYS ERROR
> > [ 7.180371] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
> > PASS THROUGH state: SYS ERROR
> > [ 7.187146] mhi mhi0: Power on setup success
> > [ 7.187219] mhi mhi0: Handling state transition: PBL
> > [ 7.189165] mhi mhi0: System error detected
> > [ 7.189178] mhi mhi0: Device MHI is not in valid state
> > [ 7.189189] mhi-pci-generic 0000:01:00.0: firmware crashed (7)
> > [ 7.213682] mhi mhi0: Handling state transition: SYS ERROR
> > [ 7.219183] mhi mhi0: Transitioning from PM state: Linkdown or
> > Error Fatal Detect to: SYS ERROR Process
> > [ 7.228590] mhi-pci-generic 0000:01:00.0: firmware crashed (6)
> > [ 7.234429] mhi mhi0: Failed to transition from PM state: Linkdown
> > or Error Fatal Detect to: SYS ERROR Process
> > [ 7.244433] mhi mhi0: Exiting with PM state: Linkdown or Error
> > Fatal Detect, MHI state: RESET
> > [ 7.252963] mhi mhi0: Handling state transition: DISABLE
> > [ 7.258278] mhi mhi0: Processing disable transition with PM state:
> > Linkdown or Error Fatal Detect
> > [ 7.267155] mhi mhi0: Waiting for all pending event ring processing
> > to complete
> > [ 7.274480] mhi mhi0: Waiting for all pending threads to complete
> > [ 7.280576] mhi mhi0: Reset all active channels and remove MHI
> > devices
> > [ 7.287110] mhi mhi0: Resetting EV CTXT and CMD CTXT
> > [ 7.292077] mhi mhi0: Exiting with PM state: DISABLE, MHI state:
> > RESET
> > [ 7.298683] mhi-pci-generic 0000:01:00.0: failed to power up MHI
> > controller
> > [ 7.306184] mhi-pci-generic: probe of 0000:01:00.0 failed with error
> > -110
> >
> > Some of the boots successfully finish and I can talk to both the QMI
> > and MBIM ports exposed by the WWAN subsystem, but the success rate is
> > extremely low.
> >
> > Thomas, are you seeing similar issues in your setup?
>
> On our setup, using i.MX6DL based board and a PCIe Sierra Wireless
> EM9190 module, running Yocto and Linux 5.13, we don't have much success
> for the moment, qmi and mbim commands very often end in timeout.
>
> Otherwise, when responses are received, we also can observe strange
> things: unexpected messages, response to previous commands or queue
> buffer issue.
>
Are you testing this with qmicli + mbimcli + ModemManager? And if so,
are you running the qmicli/mbimcli commands with the "-p" option in
order to always use the intermediate qmi-proxy/mbim-proxy? I'd suggest
to always do that to avoid having multiple processes talking to the
ports at the same time.
> I updated the topic opened on the Sierra Wireless forum, with our
> latest progress and as well as additional information.
>
> In addition, we observed some strange behavior of the EM919x after warm
> reboots.
>
Is the log after the warm reboots similar to the one I showed in my
first email, i.e. with the 2 reported "firmware crashes"? Or does it
look totally different?
And, do you always see the module booting properly on cold boots? Or
do you see failed boots like the one i showed in my first email?
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-12 19:44 ` Aleksander Morgado
@ 2021-10-14 9:51 ` Thomas Perrot
2021-10-14 10:04 ` Aleksander Morgado
0 siblings, 1 reply; 33+ messages in thread
From: Thomas Perrot @ 2021-10-14 9:51 UTC (permalink / raw)
To: Aleksander Morgado
Cc: Manivannan Sadhasivam, Loic Poulain, hemantk, Thomas Petazzoni,
linux-arm-msm
[-- Attachment #1: Type: text/plain, Size: 5344 bytes --]
Hello Aleksander,
On Tue, 2021-10-12 at 21:44 +0200, Aleksander Morgado wrote:
> > > I'm working on a setup using a RPi CM4 based board and a Sierra
> > > Wireless EM9191 module, running OpenWRT 21.02 and backported
> > > mhi+wwan
> > > drivers (from latest 5.15 rc). The kernel also has the
> > > mhi_pci_dev_info entry written by Thomas, as per
> > > https://forum.sierrawireless.com/t/sierra-wireless-airprime-em919x-pcie-support/24927
> > >
> > > The EM9191 is now running 02.08.01.00 (latest firmware from
> > > Sierra),
> > > upgraded in several steps back from the original 01.04.01.02 the
> > > module came with. The firmware upgrades were done with
> > > qmi-firmware-update and the module in USB mode in a desktop PC.
> > >
> > > Most of the system boots end up with the mhi driver reporting
> > > that the
> > > module firmware crashed in different ways:
> > >
> > > [ 7.060730] mhi-pci-generic 0000:01:00.0: MHI PCI device
> > > found:
> > > sierra-em919x
> > > [ 7.067906] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
> > > 0x600000000-0x600000fff 64bit]
> > > [ 7.076455] mhi-pci-generic 0000:01:00.0: enabling device
> > > (0000 ->
> > > 0002)
> > > [ 7.083277] mhi-pci-generic 0000:01:00.0: using shared MSI
> > > [ 7.089508] mhi mhi0: Requested to power ON
> > > [ 7.094080] mhi mhi0: Attempting power on with EE: PASS
> > > THROUGH,
> > > state: SYS ERROR
> > > [ 7.180371] mhi mhi0: local ee: INVALID_EE state: RESET device
> > > ee:
> > > PASS THROUGH state: SYS ERROR
> > > [ 7.187146] mhi mhi0: Power on setup success
> > > [ 7.187219] mhi mhi0: Handling state transition: PBL
> > > [ 7.189165] mhi mhi0: System error detected
> > > [ 7.189178] mhi mhi0: Device MHI is not in valid state
> > > [ 7.189189] mhi-pci-generic 0000:01:00.0: firmware crashed (7)
> > > [ 7.213682] mhi mhi0: Handling state transition: SYS ERROR
> > > [ 7.219183] mhi mhi0: Transitioning from PM state: Linkdown or
> > > Error Fatal Detect to: SYS ERROR Process
> > > [ 7.228590] mhi-pci-generic 0000:01:00.0: firmware crashed (6)
> > > [ 7.234429] mhi mhi0: Failed to transition from PM state:
> > > Linkdown
> > > or Error Fatal Detect to: SYS ERROR Process
> > > [ 7.244433] mhi mhi0: Exiting with PM state: Linkdown or Error
> > > Fatal Detect, MHI state: RESET
> > > [ 7.252963] mhi mhi0: Handling state transition: DISABLE
> > > [ 7.258278] mhi mhi0: Processing disable transition with PM
> > > state:
> > > Linkdown or Error Fatal Detect
> > > [ 7.267155] mhi mhi0: Waiting for all pending event ring
> > > processing
> > > to complete
> > > [ 7.274480] mhi mhi0: Waiting for all pending threads to
> > > complete
> > > [ 7.280576] mhi mhi0: Reset all active channels and remove MHI
> > > devices
> > > [ 7.287110] mhi mhi0: Resetting EV CTXT and CMD CTXT
> > > [ 7.292077] mhi mhi0: Exiting with PM state: DISABLE, MHI
> > > state:
> > > RESET
> > > [ 7.298683] mhi-pci-generic 0000:01:00.0: failed to power up
> > > MHI
> > > controller
> > > [ 7.306184] mhi-pci-generic: probe of 0000:01:00.0 failed with
> > > error
> > > -110
> > >
> > > Some of the boots successfully finish and I can talk to both the
> > > QMI
> > > and MBIM ports exposed by the WWAN subsystem, but the success
> > > rate is
> > > extremely low.
> > >
> > > Thomas, are you seeing similar issues in your setup?
> >
> > On our setup, using i.MX6DL based board and a PCIe Sierra Wireless
> > EM9190 module, running Yocto and Linux 5.13, we don't have much
> > success
> > for the moment, qmi and mbim commands very often end in timeout.
> >
> > Otherwise, when responses are received, we also can observe strange
> > things: unexpected messages, response to previous commands or queue
> > buffer issue.
> >
>
> Are you testing this with qmicli + mbimcli + ModemManager? And if so,
> are you running the qmicli/mbimcli commands with the "-p" option in
> order to always use the intermediate qmi-proxy/mbim-proxy? I'd
> suggest
> to always do that to avoid having multiple processes talking to the
> ports at the same time.
>
In first, I'm testing with qmicli/mbimcli commands with the "-p" option
in order to always use the intermediate qmi-proxy/mbim-proxy that that
I run in debug mode beforehand.
>
> > I updated the topic opened on the Sierra Wireless forum, with our
> > latest progress and as well as additional information.
> >
> > In addition, we observed some strange behavior of the EM919x after
> > warm
> > reboots.
> >
>
> Is the log after the warm reboots similar to the one I showed in my
> first email, i.e. with the 2 reported "firmware crashes"? Or does it
> look totally different?
After warm reboots, we observe no explain message indicating an error,
but we use an old firmware version.
> And, do you always see the module booting properly on cold boots? Or
> do you see failed boots like the one i showed in my first email?
The module doesn't always booting properly, you see failed boots like
the one you showed.
Best regards,
Thomas
>
--
Thomas Perrot, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-14 9:51 ` Thomas Perrot
@ 2021-10-14 10:04 ` Aleksander Morgado
2021-10-14 17:28 ` Loic Poulain
0 siblings, 1 reply; 33+ messages in thread
From: Aleksander Morgado @ 2021-10-14 10:04 UTC (permalink / raw)
To: Thomas Perrot
Cc: Manivannan Sadhasivam, Loic Poulain, hemantk, Thomas Petazzoni,
linux-arm-msm
Hey Thomas,
> > > Otherwise, when responses are received, we also can observe strange
> > > things: unexpected messages, response to previous commands or queue
> > > buffer issue.
> > >
> >
> > Are you testing this with qmicli + mbimcli + ModemManager? And if so,
> > are you running the qmicli/mbimcli commands with the "-p" option in
> > order to always use the intermediate qmi-proxy/mbim-proxy? I'd
> > suggest
> > to always do that to avoid having multiple processes talking to the
> > ports at the same time.
> >
>
> In first, I'm testing with qmicli/mbimcli commands with the "-p" option
> in order to always use the intermediate qmi-proxy/mbim-proxy that that
> I run in debug mode beforehand.
>
Ah, nice, that helps to clarify. When using the proxies, there should
be always one single process accessing the ports.
> >
> > > I updated the topic opened on the Sierra Wireless forum, with our
> > > latest progress and as well as additional information.
> > >
> > > In addition, we observed some strange behavior of the EM919x after
> > > warm
> > > reboots.
> > >
> >
> > Is the log after the warm reboots similar to the one I showed in my
> > first email, i.e. with the 2 reported "firmware crashes"? Or does it
> > look totally different?
>
> After warm reboots, we observe no explain message indicating an error,
> but we use an old firmware version.
>
Ok.
> > And, do you always see the module booting properly on cold boots? Or
> > do you see failed boots like the one i showed in my first email?
>
> The module doesn't always booting properly, you see failed boots like
> the one you showed.
This is good, because it confirms that our fully different platforms
running the same kernel driver show the same symptoms. So it shouldn't
be an issue of the platform, it's likely either the driver or the
module firmware.
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-14 10:04 ` Aleksander Morgado
@ 2021-10-14 17:28 ` Loic Poulain
2021-10-14 20:25 ` Aleksander Morgado
0 siblings, 1 reply; 33+ messages in thread
From: Loic Poulain @ 2021-10-14 17:28 UTC (permalink / raw)
To: Aleksander Morgado
Cc: Thomas Perrot, Manivannan Sadhasivam, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm
HI Aleksander, Thomas,
On Thu, 14 Oct 2021 at 12:04, Aleksander Morgado
<aleksander@aleksander.es> wrote:
>
> Hey Thomas,
>
> > > > Otherwise, when responses are received, we also can observe strange
> > > > things: unexpected messages, response to previous commands or queue
> > > > buffer issue.
> > > >
> > >
> > > Are you testing this with qmicli + mbimcli + ModemManager? And if so,
> > > are you running the qmicli/mbimcli commands with the "-p" option in
> > > order to always use the intermediate qmi-proxy/mbim-proxy? I'd
> > > suggest
> > > to always do that to avoid having multiple processes talking to the
> > > ports at the same time.
> > >
> >
> > In first, I'm testing with qmicli/mbimcli commands with the "-p" option
> > in order to always use the intermediate qmi-proxy/mbim-proxy that that
> > I run in debug mode beforehand.
> >
>
> Ah, nice, that helps to clarify. When using the proxies, there should
> be always one single process accessing the ports.
>
> > >
> > > > I updated the topic opened on the Sierra Wireless forum, with our
> > > > latest progress and as well as additional information.
> > > >
> > > > In addition, we observed some strange behavior of the EM919x after
> > > > warm
> > > > reboots.
> > > >
> > >
> > > Is the log after the warm reboots similar to the one I showed in my
> > > first email, i.e. with the 2 reported "firmware crashes"? Or does it
> > > look totally different?
> >
> > After warm reboots, we observe no explain message indicating an error,
> > but we use an old firmware version.
> >
>
> Ok.
>
> > > And, do you always see the module booting properly on cold boots? Or
> > > do you see failed boots like the one i showed in my first email?
> >
> > The module doesn't always booting properly, you see failed boots like
> > the one you showed.
>
> This is good, because it confirms that our fully different platforms
> running the same kernel driver show the same symptoms. So it shouldn't
> be an issue of the platform, it's likely either the driver or the
> module firmware.
So it looks like the device is not in the state expected by MHI core,
not sure however if it's a bad interpretation of MHI implementation or
a specific issue in that firmware. Maybe one thing you could try is to
call mhi_soc_reset(); msleep(1000); just after
mhi_register_controller() in pci_generic probe() function... it is
supposed to hard reset the modem CPU, and maybe getting it in better
shape?
Regards,
Loic
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-14 17:28 ` Loic Poulain
@ 2021-10-14 20:25 ` Aleksander Morgado
2021-10-18 9:14 ` Aleksander Morgado
0 siblings, 1 reply; 33+ messages in thread
From: Aleksander Morgado @ 2021-10-14 20:25 UTC (permalink / raw)
To: Loic Poulain
Cc: Thomas Perrot, Manivannan Sadhasivam, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm
Hey Loic,
> > > > > Otherwise, when responses are received, we also can observe strange
> > > > > things: unexpected messages, response to previous commands or queue
> > > > > buffer issue.
> > > > >
> > > >
> > > > Are you testing this with qmicli + mbimcli + ModemManager? And if so,
> > > > are you running the qmicli/mbimcli commands with the "-p" option in
> > > > order to always use the intermediate qmi-proxy/mbim-proxy? I'd
> > > > suggest
> > > > to always do that to avoid having multiple processes talking to the
> > > > ports at the same time.
> > > >
> > >
> > > In first, I'm testing with qmicli/mbimcli commands with the "-p" option
> > > in order to always use the intermediate qmi-proxy/mbim-proxy that that
> > > I run in debug mode beforehand.
> > >
> >
> > Ah, nice, that helps to clarify. When using the proxies, there should
> > be always one single process accessing the ports.
> >
> > > >
> > > > > I updated the topic opened on the Sierra Wireless forum, with our
> > > > > latest progress and as well as additional information.
> > > > >
> > > > > In addition, we observed some strange behavior of the EM919x after
> > > > > warm
> > > > > reboots.
> > > > >
> > > >
> > > > Is the log after the warm reboots similar to the one I showed in my
> > > > first email, i.e. with the 2 reported "firmware crashes"? Or does it
> > > > look totally different?
> > >
> > > After warm reboots, we observe no explain message indicating an error,
> > > but we use an old firmware version.
> > >
> >
> > Ok.
> >
> > > > And, do you always see the module booting properly on cold boots? Or
> > > > do you see failed boots like the one i showed in my first email?
> > >
> > > The module doesn't always booting properly, you see failed boots like
> > > the one you showed.
> >
> > This is good, because it confirms that our fully different platforms
> > running the same kernel driver show the same symptoms. So it shouldn't
> > be an issue of the platform, it's likely either the driver or the
> > module firmware.
>
> So it looks like the device is not in the state expected by MHI core,
> not sure however if it's a bad interpretation of MHI implementation or
> a specific issue in that firmware. Maybe one thing you could try is to
> call mhi_soc_reset(); msleep(1000); just after
> mhi_register_controller() in pci_generic probe() function... it is
> supposed to hard reset the modem CPU, and maybe getting it in better
> shape?
>
I've tried your suggestion:
diff --git a/drivers/bus/mhi/pci_generic.c b/drivers/bus/mhi/pci_generic.c
index 1499226c6cef..eaf5243d44f0 100644
--- a/drivers/bus/mhi/pci_generic.c
+++ b/drivers/bus/mhi/pci_generic.c
@@ -783,6 +783,10 @@ static int mhi_pci_probe(struct pci_dev *pdev,
const struct pci_device_id *id)
if (err)
goto err_disable_reporting;
+ /* Hard reset modem CPU */
+ mhi_soc_reset(mhi_cntrl);
+ msleep(1000);
+
/* MHI bus does not power up the controller by default */
err = mhi_prepare_for_power_up(mhi_cntrl);
if (err) {
--
2.33.0
But didn't help :/ The logs look quite similar:
[ 7.056113] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
[ 7.063298] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 7.071846] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
[ 7.078671] mhi-pci-generic 0000:01:00.0: using shared MSI
[ 8.100563] mhi mhi0: Requested to power ON
[ 8.104971] mhi mhi0: Attempting power on with EE: PASS THROUGH,
state: SYS ERROR
[ 10.979537] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
PASS THROUGH state: SYS ERROR
[ 10.988331] mhi mhi0: System error detected
[ 10.992553] mhi-pci-generic 0000:01:00.0: firmware crashed (7)
[ 10.998399] mhi mhi0: Power on setup success
[ 11.002710] mhi mhi0: Handling state transition: SYS ERROR
[ 11.008198] mhi mhi0: Transitioning from PM state: Linkdown or
Error Fatal Detect to: SYS ERROR Process
[ 11.017597] mhi-pci-generic 0000:01:00.0: firmware crashed (6)
[ 11.023430] mhi mhi0: Failed to transition from PM state: Linkdown
or Error Fatal Detect to: SYS ERROR Process
[ 11.033433] mhi mhi0: Exiting with PM state: Linkdown or Error
Fatal Detect, MHI state: RESET
[ 11.041958] mhi mhi0: Handling state transition: PBL
[ 11.046922] mhi mhi0: Device MHI is not in valid state
[ 11.052060] mhi mhi0: Handling state transition: DISABLE
[ 11.057370] mhi mhi0: Processing disable transition with PM state:
Linkdown or Error Fatal Detect
[ 11.066243] mhi mhi0: Waiting for all pending event ring processing
to complete
[ 11.073561] mhi mhi0: Waiting for all pending threads to complete
[ 11.079653] mhi mhi0: Reset all active channels and remove MHI devices
[ 11.086181] mhi mhi0: Resetting EV CTXT and CMD CTXT
[ 11.091146] mhi mhi0: Exiting with PM state: DISABLE, MHI state: RESET
[ 11.097734] mhi-pci-generic 0000:01:00.0: failed to power up MHI controller
[ 11.104937] mhi-pci-generic: probe of 0000:01:00.0 failed with error -110
This is with Sierra's latest firmware for the device, BTW, version 03.04.03.00
--
Aleksander
https://aleksander.es
^ permalink raw reply related [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-14 20:25 ` Aleksander Morgado
@ 2021-10-18 9:14 ` Aleksander Morgado
2021-10-18 9:59 ` Loic Poulain
0 siblings, 1 reply; 33+ messages in thread
From: Aleksander Morgado @ 2021-10-18 9:14 UTC (permalink / raw)
To: Loic Poulain
Cc: Thomas Perrot, Manivannan Sadhasivam, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm
Hey all,
> [ 7.056113] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
> [ 7.063298] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
> 0x600000000-0x600000fff 64bit]
> [ 7.071846] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
> [ 7.078671] mhi-pci-generic 0000:01:00.0: using shared MSI
In this specific setup we request 4 MSI vectors through
pci_alloc_irq_vectors(), but only end up allocating a single one (i.e.
mhi_cntrl->nr_irqs = 1). Could that be related to the problem somehow?
> [ 8.100563] mhi mhi0: Requested to power ON
> [ 8.104971] mhi mhi0: Attempting power on with EE: PASS THROUGH,
> state: SYS ERROR
> [ 10.979537] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
> PASS THROUGH state: SYS ERROR
> [ 10.988331] mhi mhi0: System error detected
> [ 10.992553] mhi-pci-generic 0000:01:00.0: firmware crashed (7)
> [ 10.998399] mhi mhi0: Power on setup success
> [ 11.002710] mhi mhi0: Handling state transition: SYS ERROR
> [ 11.008198] mhi mhi0: Transitioning from PM state: Linkdown or
> Error Fatal Detect to: SYS ERROR Process
> [ 11.017597] mhi-pci-generic 0000:01:00.0: firmware crashed (6)
> [ 11.023430] mhi mhi0: Failed to transition from PM state: Linkdown
> or Error Fatal Detect to: SYS ERROR Process
> [ 11.033433] mhi mhi0: Exiting with PM state: Linkdown or Error
> Fatal Detect, MHI state: RESET
> [ 11.041958] mhi mhi0: Handling state transition: PBL
> [ 11.046922] mhi mhi0: Device MHI is not in valid state
> [ 11.052060] mhi mhi0: Handling state transition: DISABLE
> [ 11.057370] mhi mhi0: Processing disable transition with PM state:
> Linkdown or Error Fatal Detect
> [ 11.066243] mhi mhi0: Waiting for all pending event ring processing
> to complete
> [ 11.073561] mhi mhi0: Waiting for all pending threads to complete
> [ 11.079653] mhi mhi0: Reset all active channels and remove MHI devices
> [ 11.086181] mhi mhi0: Resetting EV CTXT and CMD CTXT
> [ 11.091146] mhi mhi0: Exiting with PM state: DISABLE, MHI state: RESET
> [ 11.097734] mhi-pci-generic 0000:01:00.0: failed to power up MHI controller
> [ 11.104937] mhi-pci-generic: probe of 0000:01:00.0 failed with error -110
>
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-18 9:14 ` Aleksander Morgado
@ 2021-10-18 9:59 ` Loic Poulain
2021-10-18 11:26 ` Thomas Perrot
0 siblings, 1 reply; 33+ messages in thread
From: Loic Poulain @ 2021-10-18 9:59 UTC (permalink / raw)
To: Aleksander Morgado
Cc: Thomas Perrot, Manivannan Sadhasivam, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm
Hi Aleksander,
On Mon, 18 Oct 2021 at 11:14, Aleksander Morgado
<aleksander@aleksander.es> wrote:
>
> Hey all,
>
> > [ 7.056113] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
> > [ 7.063298] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
> > 0x600000000-0x600000fff 64bit]
> > [ 7.071846] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
> > [ 7.078671] mhi-pci-generic 0000:01:00.0: using shared MSI
>
> In this specific setup we request 4 MSI vectors through
> pci_alloc_irq_vectors(), but only end up allocating a single one (i.e.
> mhi_cntrl->nr_irqs = 1). Could that be related to the problem somehow?
It shouldn't, we have the 'shared IRQ' fallback which is used when we
can not setup multiple MSI, and this works with other SDX55 based
modems.
Regards,
Loic
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-18 9:59 ` Loic Poulain
@ 2021-10-18 11:26 ` Thomas Perrot
2021-10-18 12:46 ` Loic Poulain
0 siblings, 1 reply; 33+ messages in thread
From: Thomas Perrot @ 2021-10-18 11:26 UTC (permalink / raw)
To: Loic Poulain, Aleksander Morgado
Cc: Manivannan Sadhasivam, Hemant Kumar, Thomas Petazzoni, linux-arm-msm
[-- Attachment #1: Type: text/plain, Size: 1670 bytes --]
Hi Loic,
On Mon, 2021-10-18 at 11:59 +0200, Loic Poulain wrote:
> Hi Aleksander,
>
> On Mon, 18 Oct 2021 at 11:14, Aleksander Morgado
> <aleksander@aleksander.es> wrote:
> >
> > Hey all,
> >
> > > [ 7.056113] mhi-pci-generic 0000:01:00.0: MHI PCI device found:
> > > sierra-em919x
> > > [ 7.063298] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
> > > 0x600000000-0x600000fff 64bit]
> > > [ 7.071846] mhi-pci-generic 0000:01:00.0: enabling device (0000
> > > -> 0002)
> > > [ 7.078671] mhi-pci-generic 0000:01:00.0: using shared MSI
> >
> > In this specific setup we request 4 MSI vectors through
> > pci_alloc_irq_vectors(), but only end up allocating a single one
> > (i.e.
> > mhi_cntrl->nr_irqs = 1). Could that be related to the problem
> > somehow?
>
> It shouldn't, we have the 'shared IRQ' fallback which is used when we
> can not setup multiple MSI, and this works with other SDX55 based
> modems.
>
Compared to other SDX55 based modems, EM919x uses the same event ring
for the control, the data and the diag, and we use the macro
MHI_EVENT_CONFIG_CTRL to configure it.
- Perhaps this macro is not suitable in this case?
- Could this be explaining, what are we observing?
Moreover, we have voluntarily reduced the number of shared MSI vectors
to one, on a platform able to provide enough, then we observe the same
kind of issues, as on i.MX6DL which end up allocating a single one.
However, we carried out this test only with the vendor driver.
Best regards,
Thomas
> Regards,
> Loic
--
Thomas Perrot, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-18 11:26 ` Thomas Perrot
@ 2021-10-18 12:46 ` Loic Poulain
2021-10-18 14:07 ` Thomas Perrot
2021-10-22 14:33 ` Aleksander Morgado
0 siblings, 2 replies; 33+ messages in thread
From: Loic Poulain @ 2021-10-18 12:46 UTC (permalink / raw)
To: Thomas Perrot
Cc: Aleksander Morgado, Manivannan Sadhasivam, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm
On Mon, 18 Oct 2021 at 13:26, Thomas Perrot <thomas.perrot@bootlin.com> wrote:
>
> Hi Loic,
>
> On Mon, 2021-10-18 at 11:59 +0200, Loic Poulain wrote:
> > Hi Aleksander,
> >
> > On Mon, 18 Oct 2021 at 11:14, Aleksander Morgado
> > <aleksander@aleksander.es> wrote:
> > >
> > > Hey all,
> > >
> > > > [ 7.056113] mhi-pci-generic 0000:01:00.0: MHI PCI device found:
> > > > sierra-em919x
> > > > [ 7.063298] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
> > > > 0x600000000-0x600000fff 64bit]
> > > > [ 7.071846] mhi-pci-generic 0000:01:00.0: enabling device (0000
> > > > -> 0002)
> > > > [ 7.078671] mhi-pci-generic 0000:01:00.0: using shared MSI
> > >
> > > In this specific setup we request 4 MSI vectors through
> > > pci_alloc_irq_vectors(), but only end up allocating a single one
> > > (i.e.
> > > mhi_cntrl->nr_irqs = 1). Could that be related to the problem
> > > somehow?
> >
> > It shouldn't, we have the 'shared IRQ' fallback which is used when we
> > can not setup multiple MSI, and this works with other SDX55 based
> > modems.
> >
>
> Compared to other SDX55 based modems, EM919x uses the same event ring
> for the control, the data and the diag, and we use the macro
> MHI_EVENT_CONFIG_CTRL to configure it.
> - Perhaps this macro is not suitable in this case?
Well it should work, but it's usually better to have a dedicated event
ring for non-control stuff.
The number of event ring is normally driven by the host, is it a
limitation with EM919X?
What is done in the downstream driver?
> - Could this be explaining, what are we observing?
Hmm, as I said device should follow what the host is configuring in
terms of event rings, but maybe in your case a specific configuration
is expected, so it would be nice to double check with what is done in
the downstream driver. As well, do you have any way to access the
serial/debug console of the EM919X?
> Moreover, we have voluntarily reduced the number of shared MSI vectors
> to one, on a platform able to provide enough, then we observe the same
> kind of issues, as on i.MX6DL which end up allocating a single one.
> However, we carried out this test only with the vendor driver.
You mean the same initialization issue?
Regards,
Loic
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-18 12:46 ` Loic Poulain
@ 2021-10-18 14:07 ` Thomas Perrot
2021-10-18 14:16 ` Thomas Perrot
2021-10-19 8:38 ` Aleksander Morgado
2021-10-22 14:33 ` Aleksander Morgado
1 sibling, 2 replies; 33+ messages in thread
From: Thomas Perrot @ 2021-10-18 14:07 UTC (permalink / raw)
To: Loic Poulain
Cc: Aleksander Morgado, Manivannan Sadhasivam, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm
[-- Attachment #1: Type: text/plain, Size: 3037 bytes --]
Hi,
On Mon, 2021-10-18 at 14:46 +0200, Loic Poulain wrote:
> On Mon, 18 Oct 2021 at 13:26, Thomas Perrot <
> thomas.perrot@bootlin.com> wrote:
> >
> > Hi Loic,
> >
> > On Mon, 2021-10-18 at 11:59 +0200, Loic Poulain wrote:
> > > Hi Aleksander,
> > >
> > > On Mon, 18 Oct 2021 at 11:14, Aleksander Morgado
> > > <aleksander@aleksander.es> wrote:
> > > >
> > > > Hey all,
> > > >
> > > > > [ 7.056113] mhi-pci-generic 0000:01:00.0: MHI PCI device
> > > > > found:
> > > > > sierra-em919x
> > > > > [ 7.063298] mhi-pci-generic 0000:01:00.0: BAR 0: assigned
> > > > > [mem
> > > > > 0x600000000-0x600000fff 64bit]
> > > > > [ 7.071846] mhi-pci-generic 0000:01:00.0: enabling device
> > > > > (0000
> > > > > -> 0002)
> > > > > [ 7.078671] mhi-pci-generic 0000:01:00.0: using shared MSI
> > > >
> > > > In this specific setup we request 4 MSI vectors through
> > > > pci_alloc_irq_vectors(), but only end up allocating a single
> > > > one
> > > > (i.e.
> > > > mhi_cntrl->nr_irqs = 1). Could that be related to the problem
> > > > somehow?
> > >
> > > It shouldn't, we have the 'shared IRQ' fallback which is used
> > > when we
> > > can not setup multiple MSI, and this works with other SDX55 based
> > > modems.
> > >
> >
> > Compared to other SDX55 based modems, EM919x uses the same event
> > ring
> > for the control, the data and the diag, and we use the macro
> > MHI_EVENT_CONFIG_CTRL to configure it.
> > - Perhaps this macro is not suitable in this case?
>
> Well it should work, but it's usually better to have a dedicated
> event
> ring for non-control stuff.
> The number of event ring is normally driven by the host, is it a
> limitation with EM919X?
I asked the question to our Sierra distributor, because it isn't
indicated in the technical documentation that I have, I'm still waiting
for the answer.
> What is done in the downstream driver?
As we encountered issues with the generic event ring configuration, I
tried with a configuration equivalent to that of the vendor driver,
that uses the same ring for data, control and diag stuff.
Best regards,
Thomas
>
> > - Could this be explaining, what are we observing?
>
> Hmm, as I said device should follow what the host is configuring in
> terms of event rings, but maybe in your case a specific configuration
> is expected, so it would be nice to double check with what is done in
> the downstream driver. As well, do you have any way to access the
> serial/debug console of the EM919X?
>
> > Moreover, we have voluntarily reduced the number of shared MSI
> > vectors
> > to one, on a platform able to provide enough, then we observe the
> > same
> > kind of issues, as on i.MX6DL which end up allocating a single one.
> > However, we carried out this test only with the vendor driver.
>
> You mean the same initialization issue?
>
> Regards,
> Loic
--
Thomas Perrot, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-18 14:07 ` Thomas Perrot
@ 2021-10-18 14:16 ` Thomas Perrot
2021-10-19 8:38 ` Aleksander Morgado
1 sibling, 0 replies; 33+ messages in thread
From: Thomas Perrot @ 2021-10-18 14:16 UTC (permalink / raw)
To: Loic Poulain
Cc: Aleksander Morgado, Manivannan Sadhasivam, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm
[-- Attachment #1: Type: text/plain, Size: 3757 bytes --]
Hi Loic,
Sorry, I forgot to answer your other questions.
On Mon, 2021-10-18 at 16:07 +0200, Thomas Perrot wrote:
> Hi,
>
> On Mon, 2021-10-18 at 14:46 +0200, Loic Poulain wrote:
> > On Mon, 18 Oct 2021 at 13:26, Thomas Perrot <
> > thomas.perrot@bootlin.com> wrote:
> > >
> > > Hi Loic,
> > >
> > > On Mon, 2021-10-18 at 11:59 +0200, Loic Poulain wrote:
> > > > Hi Aleksander,
> > > >
> > > > On Mon, 18 Oct 2021 at 11:14, Aleksander Morgado
> > > > <aleksander@aleksander.es> wrote:
> > > > >
> > > > > Hey all,
> > > > >
> > > > > > [ 7.056113] mhi-pci-generic 0000:01:00.0: MHI PCI device
> > > > > > found:
> > > > > > sierra-em919x
> > > > > > [ 7.063298] mhi-pci-generic 0000:01:00.0: BAR 0: assigned
> > > > > > [mem
> > > > > > 0x600000000-0x600000fff 64bit]
> > > > > > [ 7.071846] mhi-pci-generic 0000:01:00.0: enabling device
> > > > > > (0000
> > > > > > -> 0002)
> > > > > > [ 7.078671] mhi-pci-generic 0000:01:00.0: using shared MSI
> > > > >
> > > > > In this specific setup we request 4 MSI vectors through
> > > > > pci_alloc_irq_vectors(), but only end up allocating a single
> > > > > one
> > > > > (i.e.
> > > > > mhi_cntrl->nr_irqs = 1). Could that be related to the problem
> > > > > somehow?
> > > >
> > > > It shouldn't, we have the 'shared IRQ' fallback which is used
> > > > when we
> > > > can not setup multiple MSI, and this works with other SDX55 based
> > > > modems.
> > > >
> > >
> > > Compared to other SDX55 based modems, EM919x uses the same event
> > > ring
> > > for the control, the data and the diag, and we use the macro
> > > MHI_EVENT_CONFIG_CTRL to configure it.
> > > - Perhaps this macro is not suitable in this case?
> >
> > Well it should work, but it's usually better to have a dedicated
> > event
> > ring for non-control stuff.
> > The number of event ring is normally driven by the host, is it a
> > limitation with EM919X?
>
> I asked the question to our Sierra distributor, because it isn't
> indicated in the technical documentation that I have, I'm still waiting
> for the answer.
>
> > What is done in the downstream driver?
>
> As we encountered issues with the generic event ring configuration, I
> tried with a configuration equivalent to that of the vendor driver,
> that uses the same ring for data, control and diag stuff.
>
> Best regards,
> Thomas
>
> >
> > > - Could this be explaining, what are we observing?
> >
We observing following things:
- Either the kernel spam in loop this error: “mhi_wwan_ctrl mhi0_QMI:
"Failed to queue buffer”
- Either some command succeed, then timeout,
- Received unexpected response or the response to a previous command,
- All AT commands seem succeed,
- And the firmware is well updated.
> > Hmm, as I said device should follow what the host is configuring in
> > terms of event rings, but maybe in your case a specific
> > configuration
> > is expected, so it would be nice to double check with what is done
> > in
> > the downstream driver. As well, do you have any way to access the
> > serial/debug console of the EM919X?
> >
> > > Moreover, we have voluntarily reduced the number of shared MSI
> > > vectors
> > > to one, on a platform able to provide enough, then we observe the
> > > same
> > > kind of issues, as on i.MX6DL which end up allocating a single
> > > one.
> > > However, we carried out this test only with the vendor driver.
> >
> > You mean the same initialization issue?
> >
Yes, we are also seeing the same initialization issue.
Best regards,
Thomas
> > Regards,
> > Loic
>
--
Thomas Perrot, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-18 14:07 ` Thomas Perrot
2021-10-18 14:16 ` Thomas Perrot
@ 2021-10-19 8:38 ` Aleksander Morgado
2021-10-20 8:43 ` Aleksander Morgado
1 sibling, 1 reply; 33+ messages in thread
From: Aleksander Morgado @ 2021-10-19 8:38 UTC (permalink / raw)
To: Thomas Perrot
Cc: Loic Poulain, Manivannan Sadhasivam, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm
Hey,
> > What is done in the downstream driver?
>
> As we encountered issues with the generic event ring configuration, I
> tried with a configuration equivalent to that of the vendor driver,
> that uses the same ring for data, control and diag stuff.
>
This is the event ring configuration in the MBPL driver from Sierra,
with NUM_MHI_EVT_RING_ELEMENTS=2048:
static struct mhi_event_properties event_config[] = {
/* num intmod msi chan pri brs type
hw_ev c_m off */
{ NUM_MHI_EVT_RING_ELEMENTS, 1, 1, 0, 1, 2, 1,
0, 0, 0},
{ NUM_MHI_EVT_RING_ELEMENTS, 1, 2, 100, 1, 3, 0,
1, 0, 0},
{ NUM_MHI_EVT_RING_ELEMENTS, 1, 3, 101, 1, 3, 0,
1, 0, 0},
// { 240, 1, 3, 101, 1, 2, 0, 1, 0, 0},
// { 240, 1, 4, 0, 1, 2, 1, 0, 0, 0},
};
And this is the event ring configuration we're using in the upstream
integration:
static struct mhi_event_config modem_sierra_em919x_mhi_events[] = {
/* first ring is control+data and DIAG ring */
MHI_EVENT_CONFIG_CTRL(0, 2048),
/* Hardware channels request dedicated hardware event rings */
MHI_EVENT_CONFIG_HW_DATA(1, 2048, 100),
MHI_EVENT_CONFIG_HW_DATA(2, 2048, 101)
};
I have tried to add a non-ctrl ring copying the setup of the Foxconn
SDX55, but did not help in my case:
MHI_EVENT_CONFIG_CTRL(0, 128),
MHI_EVENT_CONFIG_DATA(1, 128),
MHI_EVENT_CONFIG_HW_DATA(2, 1024, 100),
MHI_EVENT_CONFIG_HW_DATA(3, 1024, 101)
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-19 8:38 ` Aleksander Morgado
@ 2021-10-20 8:43 ` Aleksander Morgado
0 siblings, 0 replies; 33+ messages in thread
From: Aleksander Morgado @ 2021-10-20 8:43 UTC (permalink / raw)
To: Thomas Perrot
Cc: Loic Poulain, Manivannan Sadhasivam, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm
Hey,
> This is the event ring configuration in the MBPL driver from Sierra,
> with NUM_MHI_EVT_RING_ELEMENTS=2048:
>
> static struct mhi_event_properties event_config[] = {
> /* num intmod msi chan pri brs type
> hw_ev c_m off */
> { NUM_MHI_EVT_RING_ELEMENTS, 1, 1, 0, 1, 2, 1,
> 0, 0, 0},
> { NUM_MHI_EVT_RING_ELEMENTS, 1, 2, 100, 1, 3, 0,
> 1, 0, 0},
> { NUM_MHI_EVT_RING_ELEMENTS, 1, 3, 101, 1, 3, 0,
> 1, 0, 0},
> // { 240, 1, 3, 101, 1, 2, 0, 1, 0, 0},
> // { 240, 1, 4, 0, 1, 2, 1, 0, 0, 0},
> };
>
Looking in detail at the table above, I can see at least 2 differences
between the Sierra MBPL driver and the upstream one:
* In the upstream driver, MHI_EVENT_CONFIG_CTRL() sets
irq_moderation_ms to 0, while in the Sierra MBPL driver the table
above (intmod column) we can see it's set to 1 in channel 0.
* In the upstream driver, MHI_EVENT_CONFIG_HW_DATA() sets mode to
MHI_DB_BRST_DISABLE(2), while in the Sierra MBPL driver the table
above (brs column) we can see it's set to MHI_DB_BRST_ENABLE (3) in
channels 100 and 101.
But changing those in my setup to be in line with the MBPL driver
didn't make any difference :/
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-12 19:38 ` Aleksander Morgado
@ 2021-10-22 4:42 ` Manivannan Sadhasivam
2021-10-22 9:20 ` Aleksander Morgado
0 siblings, 1 reply; 33+ messages in thread
From: Manivannan Sadhasivam @ 2021-10-22 4:42 UTC (permalink / raw)
To: Aleksander Morgado
Cc: Loic Poulain, Thomas Perrot, hemantk, Thomas Petazzoni,
linux-arm-msm, Bhaumik Bhatt
Hi Aleksander,
On Tue, Oct 12, 2021 at 09:38:41PM +0200, Aleksander Morgado wrote:
> Hey
>
[...]
> The successful boots seem to happen always on cold boots, and the
> success rate is low (30% or so) on some manual testing here. I haven't
> seen one single successful boot on system restarts, they all fail like
> in the previous email.
>
> When the boot is successful it looks like this:
>
This looks to be a firmware issue. The device is in SYS_ERR state during
boot and that's expected. But what is strange is that the device stays
in SYS_ERR even after host issues RESET.
Can you try the below diff and see if it does any good?
diff --git a/drivers/bus/mhi/core/pm.c b/drivers/bus/mhi/core/pm.c
index fb99e3727155..a43c3ed77fb1 100644
--- a/drivers/bus/mhi/core/pm.c
+++ b/drivers/bus/mhi/core/pm.c
@@ -104,7 +104,8 @@ static struct mhi_pm_transitions const dev_state_transitions[] = {
/* L3 States */
{
MHI_PM_LD_ERR_FATAL_DETECT,
- MHI_PM_LD_ERR_FATAL_DETECT | MHI_PM_DISABLE
+ MHI_PM_LD_ERR_FATAL_DETECT | MHI_PM_DISABLE |
+ MHI_PM_SYS_ERR_PROCESS
},
};
Thanks,
Mani
> [ 0.099938] brcm-pcie fd500000.pcie: host bridge /scb/pcie@7d500000 ranges:
> [ 0.099961] brcm-pcie fd500000.pcie: No bus range found for
> /scb/pcie@7d500000, using [bus 00-ff]
> [ 0.099999] brcm-pcie fd500000.pcie: MEM
> 0x0600000000..0x0603ffffff -> 0x00f8000000
> [ 0.100032] brcm-pcie fd500000.pcie: IB MEM
> 0x0000000000..0x00bfffffff -> 0x0000000000
> [ 0.149955] brcm-pcie fd500000.pcie: link up, 5 GT/s x1 (SSC)
> [ 0.150114] brcm-pcie fd500000.pcie: PCI host bridge to bus 0000:00
> [ 0.150133] pci_bus 0000:00: root bus resource [bus 00-ff]
> [ 0.150152] pci_bus 0000:00: root bus resource [mem
> 0x600000000-0x603ffffff] (bus address [0xf8000000-0xfbffffff])
> [ 0.150193] pci 0000:00:00.0: [14e4:2711] type 01 class 0x060400
> [ 0.150282] pci 0000:00:00.0: PME# supported from D0 D3hot
> [ 0.153464] pci 0000:00:00.0: bridge configuration invalid ([bus
> ff-ff]), reconfiguring
> [ 0.153589] pci 0000:01:00.0: [17cb:0306] type 00 class 0xff0000
> [ 0.153650] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00000fff 64bit]
> [ 0.153680] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit]
> [ 0.153838] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
> [ 0.153889] pci 0000:01:00.0: 4.000 Gb/s available PCIe bandwidth,
> limited by 5 GT/s x1 link at 0000:00:00.0 (capable of 31.506 Gb/s with
> 16 GT/s x2 link)
> [ 0.156944] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
> [ 0.156983] pci 0000:00:00.0: BAR 8: assigned [mem 0x600000000-0x6000fffff]
> [ 0.157004] pci 0000:01:00.0: BAR 0: assigned [mem
> 0x600000000-0x600000fff 64bit]
> [ 0.157035] pci 0000:01:00.0: BAR 2: assigned [mem
> 0x600001000-0x600001fff 64bit]
> [ 0.157064] pci 0000:00:00.0: PCI bridge to [bus 01]
> [ 0.157079] pci 0000:00:00.0: bridge window [mem 0x600000000-0x6000fffff]
> [ 0.157222] pcieport 0000:00:00.0: enabling device (0000 -> 0002)
> [ 0.157340] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
> [ 0.157567] pcieport 0000:00:00.0: AER: enabled with IRQ 38
> [ 7.052080] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
> [ 7.059284] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
> 0x600000000-0x600000fff 64bit]
> [ 7.067839] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
> [ 7.074676] mhi-pci-generic 0000:01:00.0: using shared MSI
> [ 7.080858] mhi mhi0: Requested to power ON
> [ 7.085357] mhi mhi0: Attempting power on with EE: PASS THROUGH,
> state: SYS ERROR
> [ 7.431361] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
> PASS THROUGH state: RESET
> [ 7.432247] mhi mhi0: Power on setup success
> [ 7.440015] mhi mhi0: Handling state transition: PBL
> [ 8.387907] mhi mhi0: Device in READY State
> [ 8.392128] mhi mhi0: Initializing MHI registers
> [ 8.396815] mhi mhi0: Wait for device to enter SBL or Mission mode
> [ 14.148059] mhi mhi0: State change event to state: M0
> [ 14.148076] mhi mhi0: local ee: PASS THROUGH state: READY device
> ee: MISSION MODE state: M0
> [ 14.161518] mhi mhi0: Received EE event: MISSION MODE
> [ 14.161522] mhi mhi0: local ee: PASS THROUGH state: M0 device ee:
> MISSION MODE state: M0
> [ 14.174684] mhi mhi0: Handling state transition: MISSION MODE
> [ 14.180447] mhi mhi0: Processing Mission Mode transition
> [ 14.186199] mhi_net mhi0_IP_HW0: 100: Updating channel state to: START
> [ 14.361842] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
> MISSION MODE state: M0
> [ 14.363918] mhi_net mhi0_IP_HW0: 100: Channel state change to START
> successful
> [ 14.377324] mhi_net mhi0_IP_HW0: 101: Updating channel state to: START
> [ 14.490249] mhi_net mhi0_IP_HW0: 101: Channel state change to START
> successful
> [ 14.490255] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
> MISSION MODE state: M0
> [ 16.835372] mhi mhi0: Allowing M3 transition
> [ 16.839695] mhi mhi0: Waiting for M3 completion
> [ 16.966594] mhi mhi0: State change event to state: M3
> [ 16.966615] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
> MISSION MODE state: M3
> [ 20.460243] mhi_wwan_ctrl mhi0_DUN: 32: Updating channel state to: START
> [ 20.484117] mhi mhi0: Entered with PM state: M3, MHI state: M3
> [ 20.499999] mhi mhi0: State change event to state: M0
> [ 20.500021] mhi mhi0: local ee: MISSION MODE state: M3 device ee:
> MISSION MODE state: M0
> [ 20.517791] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
> MISSION MODE state: M0
> ....
>
> When in this state, ModemManager/qmicli/mbimcli can pick up the device
> right away, without any issues observed there. Haven't tested data
> connection yet.
>
> root@OpenWrt:~# mmcli -m 0
> -----------------------------
> General | path: /org/freedesktop/ModemManager1/Modem/0
> | device id: 58a66d754412bf0a8ec2a840f111856d85a4b053
> -----------------------------
> Hardware | manufacturer: Sierra Wireless, Incorporated
> | model: EM9191
> | firmware revision: SWIX55C_02.08.01.00 58f60e jenkins
> 2021/03/12 18:05:48
> | carrier config: default
> | h/w revision: 0.2
> | supported: gsm-umts, lte, 5gnr
> | current: gsm-umts, lte, 5gnr
> | equipment id: XXXXXXXXXXXXXX
> -----------------------------
> System | device:
> /sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0
> | drivers: mhi-pci-generic, mhi_net
> | plugin: generic
> | primary port: wwan0qmi0
> | ports: mhi_hwip0 (net), wwan0at0 (at),
> wwan0mbim0 (mbim),
> | wwan0qcdm0 (qcdm), wwan0qmi0 (qmi)
> -----------------------------
> Status | state: failed
> | failed reason: sim-missing
> | power state: on
> | signal quality: 0% (cached)
> -----------------------------
> Modes | supported: allowed: 3g; preferred: none
> | allowed: 4g; preferred: none
> | allowed: 3g, 4g; preferred: 4g
> | allowed: 3g, 4g; preferred: 3g
> | allowed: 5g; preferred: none
> | allowed: 3g, 5g; preferred: 5g
> | allowed: 3g, 5g; preferred: 3g
> | allowed: 4g, 5g; preferred: 5g
> | allowed: 4g, 5g; preferred: 4g
> | allowed: 3g, 4g, 5g; preferred: 5g
> | allowed: 3g, 4g, 5g; preferred: 4g
> | allowed: 3g, 4g, 5g; preferred: 3g
> | current: allowed: any; preferred: none
> -----------------------------
> Bands | supported: utran-1, utran-3, utran-4, utran-6,
> utran-5, utran-8,
> | utran-9, utran-2, eutran-1, eutran-2,
> eutran-3, eutran-4, eutran-5,
> | eutran-7, eutran-8, eutran-12,
> eutran-13, eutran-14, eutran-17,
> | eutran-18, eutran-19, eutran-20,
> eutran-25, eutran-26, eutran-28,
> | eutran-29, eutran-30, eutran-32,
> eutran-34, eutran-38, eutran-39,
> | eutran-40, eutran-41, eutran-42,
> eutran-46, eutran-48, eutran-66,
> | eutran-71, utran-19
> -----------------------------
> IP | supported: ipv4, ipv6, ipv4v6
> -----------------------------
> SIM | sim slot paths: slot 1: none (active)
>
> root@OpenWrt:~# qmicli -d /dev/wwan0qmi0 -p --dms-get-capabilities
> [/dev/wwan0qmi0] Device capabilities retrieved:
> Max TX channel rate: '50000000'
> Max RX channel rate: '100000000'
> Data Service: 'non-simultaneous-cs-ps'
> SIM: 'supported'
> Networks: 'umts, lte, 5gnr'
>
> root@OpenWrt:~# mbimcli -d /dev/wwan0mbim0 -p --query-device-caps
> [/dev/wwan0mbim0] Device capabilities retrieved:
> Device type: 'embedded'
> Cellular class: 'gsm'
> Voice class: 'no-voice'
> SIM class: 'removable'
> Data class: 'umts, hsdpa, hsupa, lte, custom'
> SMS caps: 'pdu-receive, pdu-send'
> Ctrl caps: 'reg-manual'
> Max sessions: '15'
> Custom data class: '5G/TDS'
> Device ID: 'XXXXXXXXXXX'
> Firmware info: '02.08.01.00_GENERI_020.007_001'
> Hardware info: 'EM9191'
>
> --
> Aleksander
> https://aleksander.es
^ permalink raw reply related [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-22 4:42 ` Manivannan Sadhasivam
@ 2021-10-22 9:20 ` Aleksander Morgado
2021-10-22 14:40 ` Manivannan Sadhasivam
0 siblings, 1 reply; 33+ messages in thread
From: Aleksander Morgado @ 2021-10-22 9:20 UTC (permalink / raw)
To: Manivannan Sadhasivam
Cc: Loic Poulain, Thomas Perrot, Hemant Kumar, Thomas Petazzoni,
linux-arm-msm, Bhaumik Bhatt
Hey,
> > The successful boots seem to happen always on cold boots, and the
> > success rate is low (30% or so) on some manual testing here. I haven't
> > seen one single successful boot on system restarts, they all fail like
> > in the previous email.
> >
> > When the boot is successful it looks like this:
> >
>
> This looks to be a firmware issue. The device is in SYS_ERR state during
> boot and that's expected. But what is strange is that the device stays
> in SYS_ERR even after host issues RESET.
>
> Can you try the below diff and see if it does any good?
>
> diff --git a/drivers/bus/mhi/core/pm.c b/drivers/bus/mhi/core/pm.c
> index fb99e3727155..a43c3ed77fb1 100644
> --- a/drivers/bus/mhi/core/pm.c
> +++ b/drivers/bus/mhi/core/pm.c
> @@ -104,7 +104,8 @@ static struct mhi_pm_transitions const dev_state_transitions[] = {
> /* L3 States */
> {
> MHI_PM_LD_ERR_FATAL_DETECT,
> - MHI_PM_LD_ERR_FATAL_DETECT | MHI_PM_DISABLE
> + MHI_PM_LD_ERR_FATAL_DETECT | MHI_PM_DISABLE |
> + MHI_PM_SYS_ERR_PROCESS
> },
> };
Tested again in the RPi CM4 based setup, but didn't help, it's failing
in the same way, still says PASS THROUGH state: SYS ERROR:
[ 7.032037] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
[ 7.039213] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 7.047759] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
[ 7.054573] mhi-pci-generic 0000:01:00.0: using shared MSI
[ 7.060848] mhi mhi0: Requested to power ON
[ 7.065277] mhi mhi0: Attempting power on with EE: PASS THROUGH,
state: SYS ERROR
[ 7.072799] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
PASS THROUGH state: SYS ERROR
[ 7.081589] mhi mhi0: System error detected
[ 7.085867] mhi-pci-generic 0000:01:00.0: firmware crashed (7)
[ 7.091886] mhi mhi0: Handling state transition: SYS ERROR
[ 7.097399] mhi mhi0: Transitioning from PM state: SYS ERROR Detect
to: SYS ERROR Process
[ 7.105588] mhi-pci-generic 0000:01:00.0: firmware crashed (6)
I've tested the same patches in my desktop PC (based on 5.13.1, and
even without this last addition) and the boot process is much more
stable and I cannot see the "firmware crashed" errors reported. My
assumption right now is that the pci_generic.c entries we're adding
are correct, but there's some limitation in this system that is making
the EM9191 boot fail, but I still don't know which limitation it is.
The memory addresses in the "BAR 0: assigned" log are definitely
different in the RPi CM4, and also the shared MSI limitation. I recall
Thomas saying that he also tested on a desktop PC forcing the shared
MSI limitation and he had the same kind of firmware errors reported;
I'll also try to test that.
Here are the logs in my desktop pc for reference:
oct 21 09:24:06 ares kernel: mhi-pci-generic 0000:17:00.0: MHI PCI
device found: sierra-em919x
oct 21 09:24:06 ares kernel: mhi-pci-generic 0000:17:00.0: BAR 0:
assigned [mem 0xb5e01000-0xb5e01fff 64bit]
oct 21 09:24:06 ares kernel: mhi mhi0: Requested to power ON
oct 21 09:24:06 ares kernel: mhi mhi0: Power on setup success
oct 21 09:24:06 ares kernel: mhi mhi0: Handling state transition: READY
oct 21 09:24:06 ares kernel: mhi mhi0: Device in READY State
oct 21 09:24:06 ares kernel: mhi mhi0: Initializing MHI registers
oct 21 09:24:06 ares kernel: mhi mhi0: State change event to state: M0
oct 21 09:24:06 ares kernel: mhi mhi0: Received EE event: MISSION MODE
oct 21 09:24:06 ares kernel: mhi mhi0: Handling state transition: MISSION MODE
oct 21 09:24:06 ares kernel: mhi mhi0: Processing Mission Mode transition
oct 21 09:24:06 ares kernel: mhi_net mhi0_IP_HW0: 100: Updating
channel state to: START
oct 21 09:24:06 ares kernel: mhi_net mhi0_IP_HW0: 100: Channel state
change to START successful
oct 21 09:24:06 ares kernel: mhi_net mhi0_IP_HW0: 101: Updating
channel state to: START
oct 21 09:24:06 ares kernel: mhi_net mhi0_IP_HW0: 101: Channel state
change to START successful
oct 21 09:24:08 ares kernel: mhi mhi0: Allowing M3 transition
oct 21 09:24:08 ares kernel: mhi mhi0: Waiting for M3 completion
oct 21 09:24:08 ares kernel: mhi mhi0: State change event to state: M3
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-18 12:46 ` Loic Poulain
2021-10-18 14:07 ` Thomas Perrot
@ 2021-10-22 14:33 ` Aleksander Morgado
1 sibling, 0 replies; 33+ messages in thread
From: Aleksander Morgado @ 2021-10-22 14:33 UTC (permalink / raw)
To: Loic Poulain
Cc: Thomas Perrot, Manivannan Sadhasivam, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm
Hey,
> What is done in the downstream driver?
>
For reference, the same kind of instabilities can also be observed
when using Sierra Wireless' MBPL driver (R20).
The failed boots look like:
[ 7.048072] [D][mhi_pci_probe] enter
[ 7.051852] mhictrl 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 7.059720] mhictrl 0000:01:00.0: enabling device (0000 -> 0002)
[ 12.582341] [D][mhi_pci_probe] failed!
[ 12.586130] mhictrl: probe of 0000:01:00.0 failed with error -5
[ 12.593561] [D][mhi_netdev_init] enter
[ 12.598049] [D][mhitty_init] Enter
[ 12.601493] [D][mhitty_init] mhi_driver_register 0x0
[ 12.606975] [D][mhi_uci_init] enter
The successful boots look like:
[ 7.009119] [D][mhi_pci_probe] enter
[ 7.012995] mhictrl 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 7.020877] mhictrl 0000:01:00.0: enabling device (0000 -> 0002)
[ 7.027542] [D][mhi_async_power_up] current EE PASS THRU
[ 7.027572] [D][mhi_pci_probe] Return successful
[ 7.037764] [D][mhi_pm_st_worker] current EE AMSS
[ 7.037773] [D][mhi_intvec_threaded_handlr] device ee:AMSS dev_state:READY
[ 7.049532] [D][mhi_netdev_init] enter
[ 7.054771] [D][mhitty_init] Enter
[ 7.058247] [D][mhitty_init] mhi_driver_register 0x0
[ 7.063982] [D][mhi_uci_init] enter
[ 15.432947] [D][mhi_process_ctrl_ev_ring] MHI state change event to state:M0
[ 15.440028] [D][mhi_pm_m0_transition] Entered With State:READY PM_STATE:POR
[ 15.447060] [D][mhi_process_ctrl_ev_ring] MHI EE received event:AMSS
[ 15.453447] [D][mhi_pm_mission_mode_transition] current EE AMSS
[ 15.468941] [D][mhi_process_ctrl_ev_ring] MHI state change event to state:M3
[ 15.481902] [D][mhi_pm_m3_transition] Entered mhi_state:M3 pm_state:M3
[ 15.488438] [D][mhi_pm_resume] Entered with pm_state:M3 dev_state:M3
[ 15.507320] [D][mhi_process_ctrl_ev_ring] MHI state change event to state:M0
[ 15.514363] [D][mhi_pm_m0_transition] Entered With State:M3 PM_STATE:M3->M0
[ 15.521437] [D][mhi_create_time_sync_dev] device add 0306_00.01.00_TIME_SYNC
[ 15.521553] [D][mhi_create_devices] chan 5 device add 0306_00.01.00_DIAG
[ 15.528676] [D][mhi_tty_probe] enter
[ 15.539024] [D][mhi_tty_probe] probe chan DIAG successful!
[ 15.544673] [D][mhi_create_devices] chan 13 device add 0306_00.01.00_MBIM
[ 15.544735] [D][mhi_uci_probe] enter
[ 15.555217] [D][mhi_uci_probe] channel:MBIM successfully probed
[ 15.561312] [D][mhi_create_devices] chan 15 device add 0306_00.01.00_QMI0
[ 15.561377] [D][mhi_uci_probe] enter
[ 15.571824] [D][mhi_uci_probe] channel:QMI0 successfully probed
[ 15.577897] [D][mhi_create_devices] chan 33 device add 0306_00.01.00_DUN
[ 15.577960] [D][mhi_tty_probe] enter
[ 15.588328] [D][mhi_tty_probe] probe chan DUN successful!
[ 15.593893] [D][mhi_create_devices] chan 101 device add 0306_00.01.00_IP_HW0
[ 15.593953] [D][mhi_netdev_probe] enter
[ 15.826612] [D][mhi_netdev_probe] success
[ 15.832353] [D][mhi_pm_mission_mode_transition] current EE AMSS
[ 15.969077] [D][mhi_process_ctrl_ev_ring] MHI state change event to state:M3
[ 15.982047] [D][mhi_pm_m3_transition] Entered mhi_state:M3 pm_state:M3
[ 15.988602] [D][mhi_pm_resume] Entered with pm_state:M3 dev_state:M3
[ 16.071030] [D][mhi_process_ctrl_ev_ring] MHI state change event to state:M0
[ 16.078074] [D][mhi_pm_m0_transition] Entered With State:M3 PM_STATE:M3->M0
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-22 9:20 ` Aleksander Morgado
@ 2021-10-22 14:40 ` Manivannan Sadhasivam
2021-10-25 8:10 ` Aleksander Morgado
0 siblings, 1 reply; 33+ messages in thread
From: Manivannan Sadhasivam @ 2021-10-22 14:40 UTC (permalink / raw)
To: Aleksander Morgado
Cc: Loic Poulain, Thomas Perrot, Hemant Kumar, Thomas Petazzoni,
linux-arm-msm, Bhaumik Bhatt
On Fri, Oct 22, 2021 at 11:20:00AM +0200, Aleksander Morgado wrote:
> Hey,
>
> > > The successful boots seem to happen always on cold boots, and the
> > > success rate is low (30% or so) on some manual testing here. I haven't
> > > seen one single successful boot on system restarts, they all fail like
> > > in the previous email.
> > >
> > > When the boot is successful it looks like this:
> > >
> >
> > This looks to be a firmware issue. The device is in SYS_ERR state during
> > boot and that's expected. But what is strange is that the device stays
> > in SYS_ERR even after host issues RESET.
> >
> > Can you try the below diff and see if it does any good?
> >
> > diff --git a/drivers/bus/mhi/core/pm.c b/drivers/bus/mhi/core/pm.c
> > index fb99e3727155..a43c3ed77fb1 100644
> > --- a/drivers/bus/mhi/core/pm.c
> > +++ b/drivers/bus/mhi/core/pm.c
> > @@ -104,7 +104,8 @@ static struct mhi_pm_transitions const dev_state_transitions[] = {
> > /* L3 States */
> > {
> > MHI_PM_LD_ERR_FATAL_DETECT,
> > - MHI_PM_LD_ERR_FATAL_DETECT | MHI_PM_DISABLE
> > + MHI_PM_LD_ERR_FATAL_DETECT | MHI_PM_DISABLE |
> > + MHI_PM_SYS_ERR_PROCESS
> > },
> > };
>
> Tested again in the RPi CM4 based setup, but didn't help, it's failing
> in the same way, still says PASS THROUGH state: SYS ERROR:
>
Yes, that's expected. As I said, the device is going to a bad state and from the
host side, we could only try to recover it.
> [ 7.032037] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
> [ 7.039213] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
> 0x600000000-0x600000fff 64bit]
> [ 7.047759] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
> [ 7.054573] mhi-pci-generic 0000:01:00.0: using shared MSI
> [ 7.060848] mhi mhi0: Requested to power ON
> [ 7.065277] mhi mhi0: Attempting power on with EE: PASS THROUGH,
> state: SYS ERROR
> [ 7.072799] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
> PASS THROUGH state: SYS ERROR
> [ 7.081589] mhi mhi0: System error detected
> [ 7.085867] mhi-pci-generic 0000:01:00.0: firmware crashed (7)
> [ 7.091886] mhi mhi0: Handling state transition: SYS ERROR
> [ 7.097399] mhi mhi0: Transitioning from PM state: SYS ERROR Detect
> to: SYS ERROR Process
> [ 7.105588] mhi-pci-generic 0000:01:00.0: firmware crashed (6)
>
What happened after this point? Can you share the complete log?
> I've tested the same patches in my desktop PC (based on 5.13.1, and
> even without this last addition) and the boot process is much more
> stable and I cannot see the "firmware crashed" errors reported. My
> assumption right now is that the pci_generic.c entries we're adding
> are correct, but there's some limitation in this system that is making
> the EM9191 boot fail, but I still don't know which limitation it is.
> The memory addresses in the "BAR 0: assigned" log are definitely
> different in the RPi CM4, and also the shared MSI limitation. I recall
> Thomas saying that he also tested on a desktop PC forcing the shared
> MSI limitation and he had the same kind of firmware errors reported;
> I'll also try to test that.
>
I think the PCI behaviour could be the issue between these 2 setups. But for
knowing exactly what's happening we need to get the log of the modem (I don't
think you can get that though).
Thanks,
Mani
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-22 14:40 ` Manivannan Sadhasivam
@ 2021-10-25 8:10 ` Aleksander Morgado
2021-11-02 10:50 ` Manivannan Sadhasivam
0 siblings, 1 reply; 33+ messages in thread
From: Aleksander Morgado @ 2021-10-25 8:10 UTC (permalink / raw)
To: Manivannan Sadhasivam
Cc: Loic Poulain, Thomas Perrot, Hemant Kumar, Thomas Petazzoni,
linux-arm-msm, Bhaumik Bhatt
Hey Mani,
> > > > The successful boots seem to happen always on cold boots, and the
> > > > success rate is low (30% or so) on some manual testing here. I haven't
> > > > seen one single successful boot on system restarts, they all fail like
> > > > in the previous email.
> > > >
> > > > When the boot is successful it looks like this:
> > > >
> > >
> > > This looks to be a firmware issue. The device is in SYS_ERR state during
> > > boot and that's expected. But what is strange is that the device stays
> > > in SYS_ERR even after host issues RESET.
> > >
> > > Can you try the below diff and see if it does any good?
> > >
> > > diff --git a/drivers/bus/mhi/core/pm.c b/drivers/bus/mhi/core/pm.c
> > > index fb99e3727155..a43c3ed77fb1 100644
> > > --- a/drivers/bus/mhi/core/pm.c
> > > +++ b/drivers/bus/mhi/core/pm.c
> > > @@ -104,7 +104,8 @@ static struct mhi_pm_transitions const dev_state_transitions[] = {
> > > /* L3 States */
> > > {
> > > MHI_PM_LD_ERR_FATAL_DETECT,
> > > - MHI_PM_LD_ERR_FATAL_DETECT | MHI_PM_DISABLE
> > > + MHI_PM_LD_ERR_FATAL_DETECT | MHI_PM_DISABLE |
> > > + MHI_PM_SYS_ERR_PROCESS
> > > },
> > > };
> >
> > Tested again in the RPi CM4 based setup, but didn't help, it's failing
> > in the same way, still says PASS THROUGH state: SYS ERROR:
> >
>
> Yes, that's expected. As I said, the device is going to a bad state and from the
> host side, we could only try to recover it.
>
> > [ 7.032037] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
> > [ 7.039213] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
> > 0x600000000-0x600000fff 64bit]
> > [ 7.047759] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
> > [ 7.054573] mhi-pci-generic 0000:01:00.0: using shared MSI
> > [ 7.060848] mhi mhi0: Requested to power ON
> > [ 7.065277] mhi mhi0: Attempting power on with EE: PASS THROUGH,
> > state: SYS ERROR
> > [ 7.072799] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
> > PASS THROUGH state: SYS ERROR
> > [ 7.081589] mhi mhi0: System error detected
> > [ 7.085867] mhi-pci-generic 0000:01:00.0: firmware crashed (7)
> > [ 7.091886] mhi mhi0: Handling state transition: SYS ERROR
> > [ 7.097399] mhi mhi0: Transitioning from PM state: SYS ERROR Detect
> > to: SYS ERROR Process
> > [ 7.105588] mhi-pci-generic 0000:01:00.0: firmware crashed (6)
> >
>
> What happened after this point? Can you share the complete log?
>
Here it is, including all pci and mhi related logs:
root@OpenWrt:~# dmesg | grep -E "mhi|pci"
[ 0.100084] brcm-pcie fd500000.pcie: host bridge /scb/pcie@7d500000 ranges:
[ 0.100108] brcm-pcie fd500000.pcie: No bus range found for
/scb/pcie@7d500000, using [bus 00-ff]
[ 0.100146] brcm-pcie fd500000.pcie: MEM
0x0600000000..0x0603ffffff -> 0x00f8000000
[ 0.100179] brcm-pcie fd500000.pcie: IB MEM
0x0000000000..0x00bfffffff -> 0x0000000000
[ 0.149979] brcm-pcie fd500000.pcie: link up, 5 GT/s x1 (SSC)
[ 0.150139] brcm-pcie fd500000.pcie: PCI host bridge to bus 0000:00
[ 0.150158] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.150177] pci_bus 0000:00: root bus resource [mem
0x600000000-0x603ffffff] (bus address [0xf8000000-0xfbffffff])
[ 0.150218] pci 0000:00:00.0: [14e4:2711] type 01 class 0x060400
[ 0.150310] pci 0000:00:00.0: PME# supported from D0 D3hot
[ 0.153509] pci 0000:00:00.0: bridge configuration invalid ([bus
ff-ff]), reconfiguring
[ 0.153633] pci 0000:01:00.0: [17cb:0306] type 00 class 0xff0000
[ 0.153695] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00000fff 64bit]
[ 0.153724] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit]
[ 0.153882] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[ 0.153933] pci 0000:01:00.0: 4.000 Gb/s available PCIe bandwidth,
limited by 5 GT/s x1 link at 0000:00:00.0 (capable of 31.506 Gb/s with
16 GT/s x2 link)
[ 0.156989] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[ 0.157028] pci 0000:00:00.0: BAR 8: assigned [mem 0x600000000-0x6000fffff]
[ 0.157049] pci 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 0.157081] pci 0000:01:00.0: BAR 2: assigned [mem
0x600001000-0x600001fff 64bit]
[ 0.157109] pci 0000:00:00.0: PCI bridge to [bus 01]
[ 0.157125] pci 0000:00:00.0: bridge window [mem 0x600000000-0x6000fffff]
[ 0.157269] pcieport 0000:00:00.0: enabling device (0000 -> 0002)
[ 0.157388] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
[ 0.157615] pcieport 0000:00:00.0: AER: enabled with IRQ 38
[ 7.078915] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
[ 7.086096] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 7.094681] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
[ 7.101507] mhi-pci-generic 0000:01:00.0: using shared MSI
[ 7.107746] mhi mhi0: Requested to power ON
[ 7.112093] mhi mhi0: Attempting power on with EE: PASS THROUGH,
state: SYS ERROR
[ 7.160844] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
PASS THROUGH state: SYS ERROR
[ 7.169642] mhi mhi0: System error detected
[ 7.173895] mhi-pci-generic 0000:01:00.0: firmware crashed (7)
[ 7.179762] mhi mhi0: Power on setup success
[ 7.179833] mhi mhi0: Handling state transition: SYS ERROR
[ 7.189547] mhi mhi0: Transitioning from PM state: Linkdown or
Error Fatal Detect to: SYS ERROR Process
[ 7.198951] mhi-pci-generic 0000:01:00.0: firmware crashed (6)
[ 7.204792] mhi mhi0: Waiting for all pending event ring processing
to complete
[ 7.212102] mhi mhi0: Waiting for all pending threads to complete
[ 7.218195] mhi mhi0: Reset all active channels and remove MHI devices
[ 7.224726] mhi mhi0: Resetting EV CTXT and CMD CTXT
[ 7.229702] mhi mhi0: Exiting with PM state: SYS ERROR Process, MHI
state: RESET
[ 7.237103] mhi mhi0: Handling state transition: PBL
[ 7.242071] mhi mhi0: Device MHI is not in valid state
[ 7.247229] mhi mhi0: Handling state transition: DISABLE
[ 7.252560] mhi mhi0: Processing disable transition with PM state:
SYS ERROR Process
[ 7.260333] mhi mhi0: Triggering MHI Reset in device
[ 14.716598] mhi mhi0: local ee: MISSION MODE state: RESET device
ee: MISSION MODE state: READY
[ 14.739911] mhi mhi0: Waiting for all pending event ring processing
to complete
[ 14.747239] mhi mhi0: Waiting for all pending threads to complete
[ 14.753336] mhi mhi0: Reset all active channels and remove MHI devices
[ 14.759867] mhi mhi0: Resetting EV CTXT and CMD CTXT
[ 14.764834] mhi mhi0: Error moving from PM state: SYS ERROR Process
to: DISABLE
[ 14.772146] mhi mhi0: Exiting with PM state: SYS ERROR Process, MHI
state: RESET
[ 14.779545] mhi mhi0: Handling state transition: READY
[ 14.784685] mhi mhi0: Device in READY State
[ 14.788869] mhi mhi0: Initializing MHI registers
[ 14.793593] mhi-pci-generic 0000:01:00.0: failed to power up MHI controller
[ 14.800842] mhi-pci-generic: probe of 0000:01:00.0 failed with error -110
> > I've tested the same patches in my desktop PC (based on 5.13.1, and
> > even without this last addition) and the boot process is much more
> > stable and I cannot see the "firmware crashed" errors reported. My
> > assumption right now is that the pci_generic.c entries we're adding
> > are correct, but there's some limitation in this system that is making
> > the EM9191 boot fail, but I still don't know which limitation it is.
> > The memory addresses in the "BAR 0: assigned" log are definitely
> > different in the RPi CM4, and also the shared MSI limitation. I recall
> > Thomas saying that he also tested on a desktop PC forcing the shared
> > MSI limitation and he had the same kind of firmware errors reported;
> > I'll also try to test that.
> >
>
> I think the PCI behaviour could be the issue between these 2 setups. But for
> knowing exactly what's happening we need to get the log of the modem (I don't
> think you can get that though).
>
Nope, I can't :/
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-25 8:10 ` Aleksander Morgado
@ 2021-11-02 10:50 ` Manivannan Sadhasivam
2021-11-02 16:13 ` Aleksander Morgado
0 siblings, 1 reply; 33+ messages in thread
From: Manivannan Sadhasivam @ 2021-11-02 10:50 UTC (permalink / raw)
To: Aleksander Morgado
Cc: Manivannan Sadhasivam, Loic Poulain, Thomas Perrot, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm, Bhaumik Bhatt
Hey,
On Mon, Oct 25, 2021 at 10:10:48AM +0200, Aleksander Morgado wrote:
> Hey Mani,
>
> [ 7.189547] mhi mhi0: Transitioning from PM state: Linkdown or
> Error Fatal Detect to: SYS ERROR Process
Hmm, I think the use of sync_power_up might be causing the issue here as it
forces the MHI state to fatal error.
Ignore the previous diff and try the below one:
diff --git a/drivers/bus/mhi/pci_generic.c b/drivers/bus/mhi/pci_generic.c
index 59a4896a8030..b1e8c7de4e54 100644
--- a/drivers/bus/mhi/pci_generic.c
+++ b/drivers/bus/mhi/pci_generic.c
@@ -637,7 +637,7 @@ static void mhi_pci_recovery_work(struct work_struct *work)
if (err)
goto err_try_reset;
- err = mhi_sync_power_up(mhi_cntrl);
+ err = mhi_async_power_up(mhi_cntrl);
if (err)
goto err_unprepare;
Thanks,
Mani
^ permalink raw reply related [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-11-02 10:50 ` Manivannan Sadhasivam
@ 2021-11-02 16:13 ` Aleksander Morgado
2021-11-02 16:22 ` Manivannan Sadhasivam
0 siblings, 1 reply; 33+ messages in thread
From: Aleksander Morgado @ 2021-11-02 16:13 UTC (permalink / raw)
To: Manivannan Sadhasivam
Cc: Manivannan Sadhasivam, Loic Poulain, Thomas Perrot, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm, Bhaumik Bhatt
Hey Mani,
> > [ 7.189547] mhi mhi0: Transitioning from PM state: Linkdown or
> > Error Fatal Detect to: SYS ERROR Process
>
> Hmm, I think the use of sync_power_up might be causing the issue here as it
> forces the MHI state to fatal error.
>
> Ignore the previous diff and try the below one:
>
> diff --git a/drivers/bus/mhi/pci_generic.c b/drivers/bus/mhi/pci_generic.c
> index 59a4896a8030..b1e8c7de4e54 100644
> --- a/drivers/bus/mhi/pci_generic.c
> +++ b/drivers/bus/mhi/pci_generic.c
> @@ -637,7 +637,7 @@ static void mhi_pci_recovery_work(struct work_struct *work)
> if (err)
> goto err_try_reset;
>
> - err = mhi_sync_power_up(mhi_cntrl);
> + err = mhi_async_power_up(mhi_cntrl);
> if (err)
> goto err_unprepare;
>
Same thing I think, see the logs below:
root@OpenWrt:~# dmesg | grep -E "mhi|pci"
[ 0.099139] brcm-pcie fd500000.pcie: host bridge /scb/pcie@7d500000 ranges:
[ 0.099163] brcm-pcie fd500000.pcie: No bus range found for
/scb/pcie@7d500000, using [bus 00-ff]
[ 0.099200] brcm-pcie fd500000.pcie: MEM
0x0600000000..0x0603ffffff -> 0x00f8000000
[ 0.099234] brcm-pcie fd500000.pcie: IB MEM
0x0000000000..0x00bfffffff -> 0x0000000000
[ 0.145977] brcm-pcie fd500000.pcie: link up, 5 GT/s x1 (SSC)
[ 0.146136] brcm-pcie fd500000.pcie: PCI host bridge to bus 0000:00
[ 0.146155] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.146173] pci_bus 0000:00: root bus resource [mem
0x600000000-0x603ffffff] (bus address [0xf8000000-0xfbffffff])
[ 0.146214] pci 0000:00:00.0: [14e4:2711] type 01 class 0x060400
[ 0.146305] pci 0000:00:00.0: PME# supported from D0 D3hot
[ 0.149482] pci 0000:00:00.0: bridge configuration invalid ([bus
ff-ff]), reconfiguring
[ 0.149605] pci 0000:01:00.0: [17cb:0306] type 00 class 0xff0000
[ 0.149667] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00000fff 64bit]
[ 0.149698] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit]
[ 0.149856] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[ 0.149906] pci 0000:01:00.0: 4.000 Gb/s available PCIe bandwidth,
limited by 5 GT/s x1 link at 0000:00:00.0 (capable of 31.506 Gb/s with
16 GT/s x2 link)
[ 0.152957] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[ 0.152995] pci 0000:00:00.0: BAR 8: assigned [mem 0x600000000-0x6000fffff]
[ 0.153017] pci 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 0.153047] pci 0000:01:00.0: BAR 2: assigned [mem
0x600001000-0x600001fff 64bit]
[ 0.153077] pci 0000:00:00.0: PCI bridge to [bus 01]
[ 0.153092] pci 0000:00:00.0: bridge window [mem 0x600000000-0x6000fffff]
[ 0.153237] pcieport 0000:00:00.0: enabling device (0000 -> 0002)
[ 0.153354] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
[ 0.153579] pcieport 0000:00:00.0: AER: enabled with IRQ 38
[ 7.068355] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
[ 7.075535] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 7.084077] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
[ 7.090865] mhi-pci-generic 0000:01:00.0: using shared MSI
[ 7.096975] mhi mhi0: Requested to power ON
[ 7.101275] mhi mhi0: Attempting power on with EE: PASS THROUGH,
state: SYS ERROR
[ 7.143799] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
PASS THROUGH state: SYS ERROR
[ 7.152590] mhi mhi0: System error detected
[ 7.156796] mhi-pci-generic 0000:01:00.0: firmware crashed (7)
[ 7.162671] mhi mhi0: Power on setup success
[ 7.166957] mhi mhi0: Handling state transition: SYS ERROR
[ 7.172442] mhi mhi0: Transitioning from PM state: Linkdown or
Error Fatal Detect to: SYS ERROR Process
[ 7.181832] mhi-pci-generic 0000:01:00.0: firmware crashed (6)
[ 7.187661] mhi mhi0: Failed to transition from PM state: Linkdown
or Error Fatal Detect to: SYS ERROR Process
[ 7.197659] mhi mhi0: Exiting with PM state: Linkdown or Error
Fatal Detect, MHI state: RESET
[ 7.206180] mhi mhi0: Handling state transition: PBL
[ 7.211140] mhi mhi0: Device MHI is not in valid state
[ 7.216273] mhi mhi0: Handling state transition: DISABLE
[ 7.221580] mhi mhi0: Processing disable transition with PM state:
Linkdown or Error Fatal Detect
[ 7.230449] mhi mhi0: Waiting for all pending event ring processing
to complete
[ 7.237762] mhi mhi0: Waiting for all pending threads to complete
[ 7.243851] mhi mhi0: Reset all active channels and remove MHI devices
[ 7.250374] mhi mhi0: Resetting EV CTXT and CMD CTXT
[ 7.255335] mhi mhi0: Exiting with PM state: DISABLE, MHI state: RESET
[ 7.261895] mhi-pci-generic 0000:01:00.0: failed to power up MHI controller
[ 7.269057] mhi-pci-generic: probe of 0000:01:00.0 failed with error -110
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-11-02 16:13 ` Aleksander Morgado
@ 2021-11-02 16:22 ` Manivannan Sadhasivam
2021-11-02 16:55 ` Aleksander Morgado
0 siblings, 1 reply; 33+ messages in thread
From: Manivannan Sadhasivam @ 2021-11-02 16:22 UTC (permalink / raw)
To: Aleksander Morgado
Cc: Manivannan Sadhasivam, Loic Poulain, Thomas Perrot, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm, Bhaumik Bhatt
On Tue, Nov 02, 2021 at 05:13:58PM +0100, Aleksander Morgado wrote:
> Hey Mani,
>
> > > [ 7.189547] mhi mhi0: Transitioning from PM state: Linkdown or
> > > Error Fatal Detect to: SYS ERROR Process
> >
> > Hmm, I think the use of sync_power_up might be causing the issue here as it
> > forces the MHI state to fatal error.
> >
> > Ignore the previous diff and try the below one:
> >
> > diff --git a/drivers/bus/mhi/pci_generic.c b/drivers/bus/mhi/pci_generic.c
> > index 59a4896a8030..b1e8c7de4e54 100644
> > --- a/drivers/bus/mhi/pci_generic.c
> > +++ b/drivers/bus/mhi/pci_generic.c
> > @@ -637,7 +637,7 @@ static void mhi_pci_recovery_work(struct work_struct *work)
> > if (err)
> > goto err_try_reset;
> >
> > - err = mhi_sync_power_up(mhi_cntrl);
> > + err = mhi_async_power_up(mhi_cntrl);
Doh! Sorry, I modified the wrong function. Here is the correct one:
diff --git a/drivers/bus/mhi/pci_generic.c b/drivers/bus/mhi/pci_generic.c
index 59a4896a8030..1e3c74bfbe34 100644
--- a/drivers/bus/mhi/pci_generic.c
+++ b/drivers/bus/mhi/pci_generic.c
@@ -743,7 +743,7 @@ static int mhi_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
goto err_unregister;
}
- err = mhi_sync_power_up(mhi_cntrl);
+ err = mhi_async_power_up(mhi_cntrl);
if (err) {
dev_err(&pdev->dev, "failed to power up MHI controller\n");
goto err_unprepare;
Let's see how it goes :)
Thanks,
Mani
^ permalink raw reply related [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-11-02 16:22 ` Manivannan Sadhasivam
@ 2021-11-02 16:55 ` Aleksander Morgado
2021-11-02 18:09 ` Manivannan Sadhasivam
0 siblings, 1 reply; 33+ messages in thread
From: Aleksander Morgado @ 2021-11-02 16:55 UTC (permalink / raw)
To: Manivannan Sadhasivam
Cc: Manivannan Sadhasivam, Loic Poulain, Thomas Perrot, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm, Bhaumik Bhatt
> > > > [ 7.189547] mhi mhi0: Transitioning from PM state: Linkdown or
> > > > Error Fatal Detect to: SYS ERROR Process
> > >
> > > Hmm, I think the use of sync_power_up might be causing the issue here as it
> > > forces the MHI state to fatal error.
> > >
> > > Ignore the previous diff and try the below one:
> > >
> > > diff --git a/drivers/bus/mhi/pci_generic.c b/drivers/bus/mhi/pci_generic.c
> > > index 59a4896a8030..b1e8c7de4e54 100644
> > > --- a/drivers/bus/mhi/pci_generic.c
> > > +++ b/drivers/bus/mhi/pci_generic.c
> > > @@ -637,7 +637,7 @@ static void mhi_pci_recovery_work(struct work_struct *work)
> > > if (err)
> > > goto err_try_reset;
> > >
> > > - err = mhi_sync_power_up(mhi_cntrl);
> > > + err = mhi_async_power_up(mhi_cntrl);
>
> Doh! Sorry, I modified the wrong function. Here is the correct one:
>
> diff --git a/drivers/bus/mhi/pci_generic.c b/drivers/bus/mhi/pci_generic.c
> index 59a4896a8030..1e3c74bfbe34 100644
> --- a/drivers/bus/mhi/pci_generic.c
> +++ b/drivers/bus/mhi/pci_generic.c
> @@ -743,7 +743,7 @@ static int mhi_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> goto err_unregister;
> }
>
> - err = mhi_sync_power_up(mhi_cntrl);
> + err = mhi_async_power_up(mhi_cntrl);
> if (err) {
> dev_err(&pdev->dev, "failed to power up MHI controller\n");
> goto err_unprepare;
>
> Let's see how it goes :)
>
Oh, wow, what a difference a single extra byte in the correct place makes! :D
This looks waaaaay better; I've rebooted the board at least 10 times
now and all of them worked nicely, at least just the process to probe
the device and get the control and net ports exposed looks very
reliable now. I'll test the setup with ModemManager in the next days
and see how everything goes. Thank you very much!
[ 0.099778] brcm-pcie fd500000.pcie: host bridge /scb/pcie@7d500000 ranges:
[ 0.099801] brcm-pcie fd500000.pcie: No bus range found for
/scb/pcie@7d500000, using [bus 00-ff]
[ 0.099839] brcm-pcie fd500000.pcie: MEM
0x0600000000..0x0603ffffff -> 0x00f8000000
[ 0.099891] brcm-pcie fd500000.pcie: IB MEM
0x0000000000..0x00bfffffff -> 0x0000000000
[ 0.149936] brcm-pcie fd500000.pcie: link up, 5 GT/s x1 (SSC)
[ 0.150094] brcm-pcie fd500000.pcie: PCI host bridge to bus 0000:00
[ 0.150113] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.150131] pci_bus 0000:00: root bus resource [mem
0x600000000-0x603ffffff] (bus address [0xf8000000-0xfbffffff])
[ 0.150172] pci 0000:00:00.0: [14e4:2711] type 01 class 0x060400
[ 0.150263] pci 0000:00:00.0: PME# supported from D0 D3hot
[ 0.153442] pci 0000:00:00.0: bridge configuration invalid ([bus
ff-ff]), reconfiguring
[ 0.153565] pci 0000:01:00.0: [17cb:0306] type 00 class 0xff0000
[ 0.153627] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00000fff 64bit]
[ 0.153657] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit]
[ 0.153815] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[ 0.153867] pci 0000:01:00.0: 4.000 Gb/s available PCIe bandwidth,
limited by 5 GT/s x1 link at 0000:00:00.0 (capable of 31.506 Gb/s with
16 GT/s x2 link)
[ 0.156925] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[ 0.156963] pci 0000:00:00.0: BAR 8: assigned [mem 0x600000000-0x6000fffff]
[ 0.156985] pci 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 0.157016] pci 0000:01:00.0: BAR 2: assigned [mem
0x600001000-0x600001fff 64bit]
[ 0.157045] pci 0000:00:00.0: PCI bridge to [bus 01]
[ 0.157060] pci 0000:00:00.0: bridge window [mem 0x600000000-0x6000fffff]
[ 0.157204] pcieport 0000:00:00.0: enabling device (0000 -> 0002)
[ 0.157322] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
[ 0.157550] pcieport 0000:00:00.0: AER: enabled with IRQ 38
[ 7.054135] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
[ 7.061303] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 7.069852] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
[ 7.076642] mhi-pci-generic 0000:01:00.0: using shared MSI
[ 7.082916] mhi mhi0: Requested to power ON
[ 7.087228] mhi mhi0: Attempting power on with EE: PASS THROUGH,
state: SYS ERROR
[ 7.094732] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
PASS THROUGH state: SYS ERROR
[ 7.103513] mhi mhi0: System error detected
[ 7.107695] mhi-pci-generic 0000:01:00.0: firmware crashed (7)
[ 7.113584] mhi mhi0: Handling state transition: SYS ERROR
[ 7.119072] mhi mhi0: Transitioning from PM state: SYS ERROR Detect
to: SYS ERROR Process
[ 7.127249] mhi-pci-generic 0000:01:00.0: firmware crashed (6)
[ 14.646025] mhi mhi0: local ee: PASS THROUGH state: RESET device
ee: MISSION MODE state: READY
[ 14.654700] mhi mhi0: Power on setup success
[ 14.654712] mhi mhi0: Triggering MHI Reset in device
[ 39.907896] mhi mhi0: Waiting for all pending event ring processing
to complete
[ 39.915245] mhi mhi0: Waiting for all pending threads to complete
[ 39.921363] mhi mhi0: Reset all active channels and remove MHI devices
[ 39.927909] mhi mhi0: Resetting EV CTXT and CMD CTXT
[ 39.932898] mhi mhi0: Exiting with PM state: SYS ERROR Process, MHI
state: RESET
[ 39.940493] mhi mhi0: Handling state transition: PBL
[ 39.945480] mhi mhi0: Device MHI is not in valid state
[ 39.950633] mhi mhi0: Handling state transition: READY
[ 39.955789] mhi mhi0: Device in READY State
[ 39.959985] mhi mhi0: Initializing MHI registers
[ 40.072692] mhi mhi0: State change event to state: M0
[ 40.072714] mhi mhi0: local ee: MISSION MODE state: READY device
ee: MISSION MODE state: M0
[ 40.086132] mhi mhi0: Received EE event: MISSION MODE
[ 40.086135] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 40.099321] mhi mhi0: Handling state transition: MISSION MODE
[ 40.105093] mhi mhi0: Processing Mission Mode transition
[ 40.111360] mhi-pci-generic 0000:01:00.0: failed to suspend device: -16
[ 40.118190] mhi_net mhi0_IP_HW0: 100: Updating channel state to: START
[ 40.142376] mhi_net mhi0_IP_HW0: 100: Channel state change to START
successful
[ 40.142379] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 40.157833] mhi_net mhi0_IP_HW0: 101: Updating channel state to: START
[ 40.176197] mhi_net mhi0_IP_HW0: 101: Channel state change to START
successful
[ 40.176200] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.595902] mhi mhi0: Allowing M3 transition
[ 42.600217] mhi mhi0: Waiting for M3 completion
[ 42.615163] mhi mhi0: State change event to state: M3
[ 42.615184] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M3
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-11-02 16:55 ` Aleksander Morgado
@ 2021-11-02 18:09 ` Manivannan Sadhasivam
[not found] ` <CAMZdPi9+zrsDy9WTipamRWBXMOxUX1tfsk2W52b9wG-4q21fWA@mail.gmail.com>
2021-11-08 7:40 ` Manivannan Sadhasivam
0 siblings, 2 replies; 33+ messages in thread
From: Manivannan Sadhasivam @ 2021-11-02 18:09 UTC (permalink / raw)
To: Aleksander Morgado
Cc: Manivannan Sadhasivam, Loic Poulain, Thomas Perrot, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm, Bhaumik Bhatt
On Tue, Nov 02, 2021 at 05:55:34PM +0100, Aleksander Morgado wrote:
> > > > > [ 7.189547] mhi mhi0: Transitioning from PM state: Linkdown or
> > > > > Error Fatal Detect to: SYS ERROR Process
> > > >
> > > > Hmm, I think the use of sync_power_up might be causing the issue here as it
> > > > forces the MHI state to fatal error.
> > > >
> > > > Ignore the previous diff and try the below one:
> > > >
> > > > diff --git a/drivers/bus/mhi/pci_generic.c b/drivers/bus/mhi/pci_generic.c
> > > > index 59a4896a8030..b1e8c7de4e54 100644
> > > > --- a/drivers/bus/mhi/pci_generic.c
> > > > +++ b/drivers/bus/mhi/pci_generic.c
> > > > @@ -637,7 +637,7 @@ static void mhi_pci_recovery_work(struct work_struct *work)
> > > > if (err)
> > > > goto err_try_reset;
> > > >
> > > > - err = mhi_sync_power_up(mhi_cntrl);
> > > > + err = mhi_async_power_up(mhi_cntrl);
> >
> > Doh! Sorry, I modified the wrong function. Here is the correct one:
> >
> > diff --git a/drivers/bus/mhi/pci_generic.c b/drivers/bus/mhi/pci_generic.c
> > index 59a4896a8030..1e3c74bfbe34 100644
> > --- a/drivers/bus/mhi/pci_generic.c
> > +++ b/drivers/bus/mhi/pci_generic.c
> > @@ -743,7 +743,7 @@ static int mhi_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> > goto err_unregister;
> > }
> >
> > - err = mhi_sync_power_up(mhi_cntrl);
> > + err = mhi_async_power_up(mhi_cntrl);
> > if (err) {
> > dev_err(&pdev->dev, "failed to power up MHI controller\n");
> > goto err_unprepare;
> >
> > Let's see how it goes :)
> >
>
> Oh, wow, what a difference a single extra byte in the correct place makes! :D
>
> This looks waaaaay better; I've rebooted the board at least 10 times
> now and all of them worked nicely, at least just the process to probe
> the device and get the control and net ports exposed looks very
> reliable now. I'll test the setup with ModemManager in the next days
> and see how everything goes. Thank you very much!
>
That's great to hear! Let me know how your testing goes with ModemManager. I'll
then submit a patch to fix the pci-generic driver.
Thanks,
Mani
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
[not found] ` <CAMZdPi9+zrsDy9WTipamRWBXMOxUX1tfsk2W52b9wG-4q21fWA@mail.gmail.com>
@ 2021-11-04 22:50 ` Bhaumik Bhatt
0 siblings, 0 replies; 33+ messages in thread
From: Bhaumik Bhatt @ 2021-11-04 22:50 UTC (permalink / raw)
To: Loic Poulain
Cc: Manivannan Sadhasivam, Aleksander Morgado, Hemant Kumar,
Manivannan Sadhasivam, Thomas Perrot, Thomas Petazzoni,
linux-arm-msm
Hi Loic,
On 2021-11-02 01:47 PM, Loic Poulain wrote:
> Hi Mani,
>
> Le mar. 2 nov. 2021 à 19:09, Manivannan Sadhasivam
> <manivannan.sadhasivam@linaro.org> a écrit :
>
>> On Tue, Nov 02, 2021 at 05:55:34PM +0100, Aleksander Morgado wrote:
>>>> > > > [ 7.189547] mhi mhi0: Transitioning from PM state:
>> Linkdown or
>>>> > > > Error Fatal Detect to: SYS ERROR Process
>>>> > >
>>>> > > Hmm, I think the use of sync_power_up might be causing the
>> issue here as it
>>>> > > forces the MHI state to fatal error.
>>>> > >
>>>> > > Ignore the previous diff and try the below one:
>>>> > >
>>>> > > diff --git a/drivers/bus/mhi/pci_generic.c
>> b/drivers/bus/mhi/pci_generic.c
>>>> > > index 59a4896a8030..b1e8c7de4e54 100644
>>>> > > --- a/drivers/bus/mhi/pci_generic.c
>>>> > > +++ b/drivers/bus/mhi/pci_generic.c
>>>> > > @@ -637,7 +637,7 @@ static void mhi_pci_recovery_work(struct
>> work_struct *work)
>>>> > > if (err)
>>>> > > goto err_try_reset;
>>>> > >
>>>> > > - err = mhi_sync_power_up(mhi_cntrl);
>>>> > > + err = mhi_async_power_up(mhi_cntrl);
>>>>
>>>> Doh! Sorry, I modified the wrong function. Here is the correct
>> one:
>>>>
>>>> diff --git a/drivers/bus/mhi/pci_generic.c
>> b/drivers/bus/mhi/pci_generic.c
>>>> index 59a4896a8030..1e3c74bfbe34 100644
>>>> --- a/drivers/bus/mhi/pci_generic.c
>>>> +++ b/drivers/bus/mhi/pci_generic.c
>>>> @@ -743,7 +743,7 @@ static int mhi_pci_probe(struct pci_dev
>> *pdev, const struct pci_device_id *id)
>>>> goto err_unregister;
>>>> }
>>>>
>>>> - err = mhi_sync_power_up(mhi_cntrl);
>>>> + err = mhi_async_power_up(mhi_cntrl);
>>>> if (err) {
>>>> dev_err(&pdev->dev, "failed to power up MHI
>> controller\n");
>>>> goto err_unprepare;
>>>>
>>>> Let's see how it goes :)
>>>>
>>>
>>> Oh, wow, what a difference a single extra byte in the correct
>> place makes! :D
>>>
>>> This looks waaaaay better; I've rebooted the board at least 10
>> times
>>> now and all of them worked nicely, at least just the process to
>> probe
>>> the device and get the control and net ports exposed looks very
>>> reliable now. I'll test the setup with ModemManager in the next
>> days
>>> and see how everything goes. Thank you very much!
>>>
>>
>> That's great to hear! Let me know how your testing goes with
>> ModemManager. I'll
>> then submit a patch to fix the pci-generic driver.
>
> I’ve not followed entirely what is going wrong, so I may miss the
> point here, but the sync and async function should behave the same
> regarding device initialization, wouldn’t make sense to fix the core
> instead? Is the _sync variant broken?
>
> Regards,
> Loic
Yes both variants should behave the same way. Mani, Hemant and I spoke
about this
and saw there could be improvements in core mhi_async_power_up().
I believe Mani will come up with a patch soon.
Thanks,
Bhaumik
---
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora
Forum,
a Linux Foundation Collaborative Project
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-11-02 18:09 ` Manivannan Sadhasivam
[not found] ` <CAMZdPi9+zrsDy9WTipamRWBXMOxUX1tfsk2W52b9wG-4q21fWA@mail.gmail.com>
@ 2021-11-08 7:40 ` Manivannan Sadhasivam
2021-11-08 13:38 ` Aleksander Morgado
1 sibling, 1 reply; 33+ messages in thread
From: Manivannan Sadhasivam @ 2021-11-08 7:40 UTC (permalink / raw)
To: Aleksander Morgado
Cc: Manivannan Sadhasivam, Loic Poulain, Thomas Perrot, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm, Bhaumik Bhatt
[-- Attachment #1: Type: text/plain, Size: 2619 bytes --]
Hey,
On Tue, Nov 02, 2021 at 11:39:12PM +0530, Manivannan Sadhasivam wrote:
> On Tue, Nov 02, 2021 at 05:55:34PM +0100, Aleksander Morgado wrote:
> > > > > > [ 7.189547] mhi mhi0: Transitioning from PM state: Linkdown or
> > > > > > Error Fatal Detect to: SYS ERROR Process
> > > > >
> > > > > Hmm, I think the use of sync_power_up might be causing the issue here as it
> > > > > forces the MHI state to fatal error.
> > > > >
> > > > > Ignore the previous diff and try the below one:
> > > > >
> > > > > diff --git a/drivers/bus/mhi/pci_generic.c b/drivers/bus/mhi/pci_generic.c
> > > > > index 59a4896a8030..b1e8c7de4e54 100644
> > > > > --- a/drivers/bus/mhi/pci_generic.c
> > > > > +++ b/drivers/bus/mhi/pci_generic.c
> > > > > @@ -637,7 +637,7 @@ static void mhi_pci_recovery_work(struct work_struct *work)
> > > > > if (err)
> > > > > goto err_try_reset;
> > > > >
> > > > > - err = mhi_sync_power_up(mhi_cntrl);
> > > > > + err = mhi_async_power_up(mhi_cntrl);
> > >
> > > Doh! Sorry, I modified the wrong function. Here is the correct one:
> > >
> > > diff --git a/drivers/bus/mhi/pci_generic.c b/drivers/bus/mhi/pci_generic.c
> > > index 59a4896a8030..1e3c74bfbe34 100644
> > > --- a/drivers/bus/mhi/pci_generic.c
> > > +++ b/drivers/bus/mhi/pci_generic.c
> > > @@ -743,7 +743,7 @@ static int mhi_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> > > goto err_unregister;
> > > }
> > >
> > > - err = mhi_sync_power_up(mhi_cntrl);
> > > + err = mhi_async_power_up(mhi_cntrl);
> > > if (err) {
> > > dev_err(&pdev->dev, "failed to power up MHI controller\n");
> > > goto err_unprepare;
> > >
> > > Let's see how it goes :)
> > >
> >
> > Oh, wow, what a difference a single extra byte in the correct place makes! :D
> >
> > This looks waaaaay better; I've rebooted the board at least 10 times
> > now and all of them worked nicely, at least just the process to probe
> > the device and get the control and net ports exposed looks very
> > reliable now. I'll test the setup with ModemManager in the next days
> > and see how everything goes. Thank you very much!
> >
>
> That's great to hear! Let me know how your testing goes with ModemManager. I'll
> then submit a patch to fix the pci-generic driver.
>
Can you please test the attached patch? We had an internal discussion and came
to a conclusion that the device is not handling the SYS_ERR case during host
bootup. So we need to adjust the SYS_ERR logic handling in host to workaround
it.
Thanks,
Mani
> Thanks,
> Mani
[-- Attachment #2: 0001-bus-mhi-Register-IRQ-handler-after-SYS_ERR-check-dur.patch --]
[-- Type: text/x-diff, Size: 2983 bytes --]
From f2fc172e2fcaadb07e2d22bb0cc5642df7cd6495 Mon Sep 17 00:00:00 2001
From: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Date: Mon, 8 Nov 2021 13:05:50 +0530
Subject: [PATCH] bus: mhi: Register IRQ handler after SYS_ERR check during
power up
Some devices tend to issue SYS_ERR interrupt while the host handling
the SYS_ERR state of the device during power up. This creates a race
and causes failure in booting up the device.
Hence, register the irq handler only after handling the SYS_ERR check
to avoid getting spurious IRQs from the device.
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
drivers/bus/mhi/core/pm.c | 26 +++++++++++---------------
1 file changed, 11 insertions(+), 15 deletions(-)
diff --git a/drivers/bus/mhi/core/pm.c b/drivers/bus/mhi/core/pm.c
index fb99e3727155..ec5f11166820 100644
--- a/drivers/bus/mhi/core/pm.c
+++ b/drivers/bus/mhi/core/pm.c
@@ -1055,10 +1055,6 @@ int mhi_async_power_up(struct mhi_controller *mhi_cntrl)
mutex_lock(&mhi_cntrl->pm_mutex);
mhi_cntrl->pm_state = MHI_PM_DISABLE;
- ret = mhi_init_irq_setup(mhi_cntrl);
- if (ret)
- goto error_setup_irq;
-
/* Setup BHI INTVEC */
write_lock_irq(&mhi_cntrl->pm_lock);
mhi_write_reg(mhi_cntrl, mhi_cntrl->bhi, BHI_INTVEC, 0);
@@ -1072,7 +1068,7 @@ int mhi_async_power_up(struct mhi_controller *mhi_cntrl)
dev_err(dev, "%s is not a valid EE for power on\n",
TO_MHI_EXEC_STR(current_ee));
ret = -EIO;
- goto error_async_power_up;
+ goto error_setup_irq;
}
state = mhi_get_mhi_state(mhi_cntrl);
@@ -1082,19 +1078,18 @@ int mhi_async_power_up(struct mhi_controller *mhi_cntrl)
if (state == MHI_STATE_SYS_ERR) {
mhi_set_mhi_state(mhi_cntrl, MHI_STATE_RESET);
ret = wait_event_timeout(mhi_cntrl->state_event,
- MHI_PM_IN_FATAL_STATE(mhi_cntrl->pm_state) ||
- mhi_read_reg_field(mhi_cntrl,
- mhi_cntrl->regs,
- MHICTRL,
- MHICTRL_RESET_MASK,
- MHICTRL_RESET_SHIFT,
+ mhi_read_reg_field(mhi_cntrl,
+ mhi_cntrl->regs,
+ MHICTRL,
+ MHICTRL_RESET_MASK,
+ MHICTRL_RESET_SHIFT,
&val) ||
!val,
msecs_to_jiffies(mhi_cntrl->timeout_ms));
if (!ret) {
ret = -EIO;
dev_info(dev, "Failed to reset MHI due to syserr state\n");
- goto error_async_power_up;
+ goto error_setup_irq;
}
/*
@@ -1104,6 +1099,10 @@ int mhi_async_power_up(struct mhi_controller *mhi_cntrl)
mhi_write_reg(mhi_cntrl, mhi_cntrl->bhi, BHI_INTVEC, 0);
}
+ ret = mhi_init_irq_setup(mhi_cntrl);
+ if (ret)
+ goto error_setup_irq;
+
/* Transition to next state */
next_state = MHI_IN_PBL(current_ee) ?
DEV_ST_TRANSITION_PBL : DEV_ST_TRANSITION_READY;
@@ -1116,9 +1115,6 @@ int mhi_async_power_up(struct mhi_controller *mhi_cntrl)
return 0;
-error_async_power_up:
- mhi_deinit_free_irq(mhi_cntrl);
-
error_setup_irq:
mhi_cntrl->pm_state = MHI_PM_DISABLE;
mutex_unlock(&mhi_cntrl->pm_mutex);
--
2.25.1
^ permalink raw reply related [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-11-08 7:40 ` Manivannan Sadhasivam
@ 2021-11-08 13:38 ` Aleksander Morgado
0 siblings, 0 replies; 33+ messages in thread
From: Aleksander Morgado @ 2021-11-08 13:38 UTC (permalink / raw)
To: Manivannan Sadhasivam
Cc: Manivannan Sadhasivam, Loic Poulain, Thomas Perrot, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm, Bhaumik Bhatt
Hey Mani,
> On Tue, Nov 02, 2021 at 11:39:12PM +0530, Manivannan Sadhasivam wrote:
> > On Tue, Nov 02, 2021 at 05:55:34PM +0100, Aleksander Morgado wrote:
> > > > > > > [ 7.189547] mhi mhi0: Transitioning from PM state: Linkdown or
> > > > > > > Error Fatal Detect to: SYS ERROR Process
> > > > > >
> > > > > > Hmm, I think the use of sync_power_up might be causing the issue here as it
> > > > > > forces the MHI state to fatal error.
> > > > > >
> > > > > > Ignore the previous diff and try the below one:
> > > > > >
> > > > > > diff --git a/drivers/bus/mhi/pci_generic.c b/drivers/bus/mhi/pci_generic.c
> > > > > > index 59a4896a8030..b1e8c7de4e54 100644
> > > > > > --- a/drivers/bus/mhi/pci_generic.c
> > > > > > +++ b/drivers/bus/mhi/pci_generic.c
> > > > > > @@ -637,7 +637,7 @@ static void mhi_pci_recovery_work(struct work_struct *work)
> > > > > > if (err)
> > > > > > goto err_try_reset;
> > > > > >
> > > > > > - err = mhi_sync_power_up(mhi_cntrl);
> > > > > > + err = mhi_async_power_up(mhi_cntrl);
> > > >
> > > > Doh! Sorry, I modified the wrong function. Here is the correct one:
> > > >
> > > > diff --git a/drivers/bus/mhi/pci_generic.c b/drivers/bus/mhi/pci_generic.c
> > > > index 59a4896a8030..1e3c74bfbe34 100644
> > > > --- a/drivers/bus/mhi/pci_generic.c
> > > > +++ b/drivers/bus/mhi/pci_generic.c
> > > > @@ -743,7 +743,7 @@ static int mhi_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> > > > goto err_unregister;
> > > > }
> > > >
> > > > - err = mhi_sync_power_up(mhi_cntrl);
> > > > + err = mhi_async_power_up(mhi_cntrl);
> > > > if (err) {
> > > > dev_err(&pdev->dev, "failed to power up MHI controller\n");
> > > > goto err_unprepare;
> > > >
> > > > Let's see how it goes :)
> > > >
> > >
> > > Oh, wow, what a difference a single extra byte in the correct place makes! :D
> > >
> > > This looks waaaaay better; I've rebooted the board at least 10 times
> > > now and all of them worked nicely, at least just the process to probe
> > > the device and get the control and net ports exposed looks very
> > > reliable now. I'll test the setup with ModemManager in the next days
> > > and see how everything goes. Thank you very much!
> > >
> >
> > That's great to hear! Let me know how your testing goes with ModemManager. I'll
> > then submit a patch to fix the pci-generic driver.
> >
>
> Can you please test the attached patch? We had an internal discussion and came
> to a conclusion that the device is not handling the SYS_ERR case during host
> bootup. So we need to adjust the SYS_ERR logic handling in host to workaround
> it.
>
I reverted the previous patch and added this one; seems to be working
well. See debug log:
[ 0.099473] brcm-pcie fd500000.pcie: host bridge /scb/pcie@7d500000 ranges:
[ 0.099497] brcm-pcie fd500000.pcie: No bus range found for
/scb/pcie@7d500000, using [bus 00-ff]
[ 0.099534] brcm-pcie fd500000.pcie: MEM
0x0600000000..0x0603ffffff -> 0x00f8000000
[ 0.099568] brcm-pcie fd500000.pcie: IB MEM
0x0000000000..0x00bfffffff -> 0x0000000000
[ 0.149884] brcm-pcie fd500000.pcie: link up, 5 GT/s x1 (SSC)
[ 0.150042] brcm-pcie fd500000.pcie: PCI host bridge to bus 0000:00
[ 0.150061] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.150080] pci_bus 0000:00: root bus resource [mem
0x600000000-0x603ffffff] (bus address [0xf8000000-0xfbffffff])
[ 0.150121] pci 0000:00:00.0: [14e4:2711] type 01 class 0x060400
[ 0.150212] pci 0000:00:00.0: PME# supported from D0 D3hot
[ 0.153399] pci 0000:00:00.0: bridge configuration invalid ([bus
ff-ff]), reconfiguring
[ 0.153521] pci 0000:01:00.0: [17cb:0306] type 00 class 0xff0000
[ 0.153583] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00000fff 64bit]
[ 0.153613] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit]
[ 0.153771] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[ 0.153822] pci 0000:01:00.0: 4.000 Gb/s available PCIe bandwidth,
limited by 5 GT/s x1 link at 0000:00:00.0 (capable of 31.506 Gb/s with
16 GT/s x2 link)
[ 0.156875] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[ 0.156914] pci 0000:00:00.0: BAR 8: assigned [mem 0x600000000-0x6000fffff]
[ 0.156935] pci 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 0.156967] pci 0000:01:00.0: BAR 2: assigned [mem
0x600001000-0x600001fff 64bit]
[ 0.156996] pci 0000:00:00.0: PCI bridge to [bus 01]
[ 0.157011] pci 0000:00:00.0: bridge window [mem 0x600000000-0x6000fffff]
[ 0.157156] pcieport 0000:00:00.0: enabling device (0000 -> 0002)
[ 0.157275] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
[ 0.157498] pcieport 0000:00:00.0: AER: enabled with IRQ 38
[ 7.029598] mhi-pci-generic 0000:01:00.0: MHI PCI device found: sierra-em919x
[ 7.036761] mhi-pci-generic 0000:01:00.0: BAR 0: assigned [mem
0x600000000-0x600000fff 64bit]
[ 7.045318] mhi-pci-generic 0000:01:00.0: enabling device (0000 -> 0002)
[ 7.052156] mhi-pci-generic 0000:01:00.0: using shared MSI
[ 7.058407] mhi mhi0: Requested to power ON
[ 7.062644] mhi mhi0: Attempting power on with EE: PASS THROUGH,
state: SYS ERROR
[ 31.712022] mhi mhi0: Power on setup success
[ 31.712113] mhi mhi0: Handling state transition: PBL
[ 31.716353] mhi mhi0: local ee: INVALID_EE state: RESET device ee:
MISSION MODE state: READY
[ 31.729790] mhi mhi0: Device in READY State
[ 31.733990] mhi mhi0: Initializing MHI registers
[ 31.738704] mhi mhi0: Wait for device to enter SBL or Mission mode
[ 31.815817] mhi mhi0: State change event to state: M0
[ 31.820899] mhi mhi0: Received EE event: MISSION MODE
[ 31.825955] mhi mhi0: Handling state transition: MISSION MODE
[ 31.825964] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 31.831706] mhi mhi0: Processing Mission Mode transition
[ 31.845374] mhi_net mhi0_IP_HW0: 100: Updating channel state to: START
[ 31.865484] mhi_net mhi0_IP_HW0: 100: Channel state change to START
successful
[ 31.865486] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 31.885952] mhi_net mhi0_IP_HW0: 101: Updating channel state to: START
[ 31.900721] mhi_net mhi0_IP_HW0: 101: Channel state change to START
successful
[ 31.900724] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 34.355880] mhi mhi0: Allowing M3 transition
[ 34.360195] mhi mhi0: Waiting for M3 completion
[ 34.375625] mhi mhi0: State change event to state: M3
[ 38.168330] mhi_wwan_ctrl mhi0_DUN: 32: Updating channel state to: START
[ 38.192041] mhi mhi0: Entered with PM state: M3, MHI state: M3
[ 38.227956] mhi mhi0: State change event to state: M0
[ 38.233062] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 38.244311] mhi_wwan_ctrl mhi0_DUN: 32: Channel state change to
START successful
[ 38.244313] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 38.259829] mhi_wwan_ctrl mhi0_DUN: 33: Updating channel state to: START
[ 38.269556] mhi_wwan_ctrl mhi0_DUN: 33: Channel state change to
START successful
[ 38.269559] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 38.303920] mhi_wwan_ctrl mhi0_DIAG: 4: Updating channel state to: START
[ 38.312881] mhi_wwan_ctrl mhi0_DIAG: 4: Channel state change to
START successful
[ 38.312887] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 38.328385] mhi_wwan_ctrl mhi0_DIAG: 5: Updating channel state to: START
[ 38.342113] mhi_wwan_ctrl mhi0_DIAG: 5: Channel state change to
START successful
[ 38.342115] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 38.380484] mhi_wwan_ctrl mhi0_DIAG: mhi_ul_xfer_cb: status: 0 xfer_len: 5
[ 38.387387] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: 0
receive_len: 58
[ 38.394623] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 38.402731] mhi_wwan_ctrl mhi0_DIAG: 5: Updating channel state to: RESET
[ 38.412993] mhi_wwan_ctrl mhi0_DIAG: 5: Channel state change to
RESET successful
[ 38.412995] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 38.428487] mhi mhi0: Marking all events for chan: 5 as stale
[ 38.434231] mhi mhi0: Finished marking events as stale events
[ 38.439975] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.447373] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.454773] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.462172] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.469571] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.476967] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.484364] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.491761] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.499158] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.506555] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.513951] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.521349] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.526634] mhi_wwan_ctrl mhi0_DUN: mhi_ul_xfer_cb: status: 0 xfer_len: 4
[ 38.528746] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.535527] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 3
[ 38.542914] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.549951] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 6
[ 38.557342] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.571770] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.579167] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.586564] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.593961] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.601357] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.608754] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.616151] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.623548] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.630945] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.638341] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.645737] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.653134] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.660531] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.667927] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.675325] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.682721] mhi_wwan_ctrl mhi0_DIAG: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.690120] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 38.690139] mhi_wwan_ctrl mhi0_DIAG: 5: successfully reset
[ 38.703696] mhi_wwan_ctrl mhi0_DIAG: 4: Updating channel state to: RESET
[ 38.714240] mhi_wwan_ctrl mhi0_DIAG: 4: Channel state change to
RESET successful
[ 38.714242] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 38.729761] mhi mhi0: Marking all events for chan: 4 as stale
[ 38.735523] mhi mhi0: Finished marking events as stale events
[ 38.741315] mhi_wwan_ctrl mhi0_DIAG: 4: successfully reset
[ 38.749124] mhi_wwan_ctrl mhi0_QMI: 14: Updating channel state to: START
[ 38.751835] mhi_wwan_ctrl mhi0_DUN: mhi_ul_xfer_cb: status: 0 xfer_len: 9
[ 38.762634] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 8
[ 38.769692] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 39
[ 38.776840] mhi_wwan_ctrl mhi0_QMI: 14: Channel state change to
START successful
[ 38.776845] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 38.777142] mhi_wwan_ctrl mhi0_DUN: 33: Updating channel state to: RESET
[ 38.784281] mhi_wwan_ctrl mhi0_QMI: 15: Updating channel state to: START
[ 38.794145] mhi_wwan_ctrl mhi0_DUN: 33: Channel state change to
RESET successful
[ 38.794147] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 38.802739] mhi_wwan_ctrl mhi0_QMI: 15: Channel state change to
START successful
[ 38.802740] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 38.805745] mhi mhi0: Marking all events for chan: 33 as stale
[ 38.805748] mhi mhi0: Finished marking events as stale events
[ 38.848247] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.855561] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.862872] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.870185] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.877496] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.884807] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.892117] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.899428] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.906738] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.914049] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.921359] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.928669] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.935980] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.943290] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.950601] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.957911] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.965222] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.972532] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.979842] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.987152] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 38.994462] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 39.001772] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 39.009083] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 39.016393] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 39.023703] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 39.031013] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 39.038323] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 39.045634] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 39.052944] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 39.060255] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 39.067565] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 39.074910] mhi_wwan_ctrl mhi0_DUN: 33: successfully reset
[ 39.080401] mhi_wwan_ctrl mhi0_DUN: 32: Updating channel state to: RESET
[ 39.092830] mhi_wwan_ctrl mhi0_DUN: 32: Channel state change to
RESET successful
[ 39.092833] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 39.108353] mhi mhi0: Marking all events for chan: 32 as stale
[ 39.114190] mhi mhi0: Finished marking events as stale events
[ 39.119965] mhi_wwan_ctrl mhi0_DUN: 32: successfully reset
[ 39.910315] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 12
[ 39.917246] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 12
[ 39.924405] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 223
[ 39.931651] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 39.944797] mhi_wwan_ctrl mhi0_QMI: 15: Updating channel state to: RESET
[ 39.954196] mhi_wwan_ctrl mhi0_QMI: 15: Channel state change to
RESET successful
[ 39.954198] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 39.969693] mhi mhi0: Marking all events for chan: 15 as stale
[ 39.975525] mhi mhi0: Finished marking events as stale events
[ 39.981273] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 39.988587] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 39.995898] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.003210] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.010521] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.017832] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.025143] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.032453] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.039764] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.047076] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.054386] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.061697] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.069007] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.076318] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.083628] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.090939] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.098249] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.105560] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.112871] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.120181] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.127492] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.134803] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.142113] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.149423] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.156734] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.164045] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.171356] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.178667] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.185977] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.193287] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.200598] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: -107
receive_len: 0
[ 40.207934] mhi_wwan_ctrl mhi0_QMI: 15: successfully reset
[ 40.213422] mhi_wwan_ctrl mhi0_QMI: 14: Updating channel state to: RESET
[ 40.225553] mhi_wwan_ctrl mhi0_QMI: 14: Channel state change to
RESET successful
[ 40.225555] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 40.241050] mhi mhi0: Marking all events for chan: 14 as stale
[ 40.246884] mhi mhi0: Finished marking events as stale events
[ 40.252647] mhi_wwan_ctrl mhi0_QMI: 14: successfully reset
[ 40.259068] mhi_wwan_ctrl mhi0_QMI: 14: Updating channel state to: START
[ 40.268178] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 40.271816] mhi_wwan_ctrl mhi0_QMI: 14: Channel state change to
START successful
[ 40.283691] mhi_wwan_ctrl mhi0_QMI: 15: Updating channel state to: START
[ 40.293126] mhi_wwan_ctrl mhi0_QMI: 15: Channel state change to
START successful
[ 40.293129] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.388917] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 12
[ 41.395829] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 12
[ 41.402981] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 223
[ 41.410234] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.438041] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 16
[ 41.444932] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 24
[ 41.452076] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.463093] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 41.469971] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 77
[ 41.477111] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.497724] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 16
[ 41.504615] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 17
[ 41.511503] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 24
[ 41.518647] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 24
[ 41.525820] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.553652] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 16
[ 41.560554] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 24
[ 41.567707] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.587928] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 16
[ 41.594830] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 24
[ 41.601982] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.613012] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 16
[ 41.619896] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 24
[ 41.627031] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 27
[ 41.634168] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 16
[ 41.641048] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 24
[ 41.648195] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.659376] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 16
[ 41.666256] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 24
[ 41.673398] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.684442] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 16
[ 41.691329] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 24
[ 41.698471] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.709331] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 16
[ 41.716212] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 24
[ 41.723352] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.723731] mhi_wwan_ctrl mhi0_DUN: 32: Updating channel state to: START
[ 41.741600] mhi_wwan_ctrl mhi0_DUN: 32: Channel state change to
START successful
[ 41.741602] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.757124] mhi_wwan_ctrl mhi0_DUN: 33: Updating channel state to: START
[ 41.766019] mhi_wwan_ctrl mhi0_DUN: 33: Channel state change to
START successful
[ 41.766023] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.785341] mhi_wwan_ctrl mhi0_DUN: mhi_ul_xfer_cb: status: 0 xfer_len: 6
[ 41.792144] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 41.799024] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 294
[ 41.806249] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 5
[ 41.813302] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 6
[ 41.820349] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 41.827225] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 26
[ 41.834358] mhi_wwan_ctrl mhi0_DUN: mhi_ul_xfer_cb: status: 0 xfer_len: 6
[ 41.841145] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 6
[ 41.848188] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 41.855067] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 197
[ 41.862287] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 6
[ 41.869334] mhi_wwan_ctrl mhi0_DUN: mhi_ul_xfer_cb: status: 0 xfer_len: 11
[ 41.876206] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 41.883079] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 52
[ 41.890210] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 6
[ 41.897256] mhi_wwan_ctrl mhi0_DUN: mhi_ul_xfer_cb: status: 0 xfer_len: 6
[ 41.904041] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 41.910913] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 29
[ 41.918045] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 6
[ 41.925090] mhi_wwan_ctrl mhi0_DUN: mhi_ul_xfer_cb: status: 0 xfer_len: 7
[ 41.931874] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 41.938746] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 77
[ 41.945888] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.958858] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 27
[ 41.965739] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 20
[ 41.972871] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 29
[ 41.980011] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 41.990655] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 41.997534] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 26
[ 42.004674] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.016836] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.023719] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 42
[ 42.030857] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.041906] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.048787] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 145
[ 42.056018] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.068150] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.075056] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 24
[ 42.082221] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.094405] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 20
[ 42.101285] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 27
[ 42.108424] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.120823] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.127729] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 148
[ 42.134982] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.147132] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.154012] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 148
[ 42.161242] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.173040] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.179927] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 20
[ 42.187068] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.198434] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.205317] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 152
[ 42.212544] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.223609] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.230493] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 152
[ 42.237722] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.250242] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.257126] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 152
[ 42.264356] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.276996] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 35
[ 42.283883] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 59
[ 42.291028] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.302840] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.309720] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 28
[ 42.316863] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.329098] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.335985] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 28
[ 42.343122] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 12
[ 42.350258] mhi_wwan_ctrl mhi0_DUN: mhi_ul_xfer_cb: status: 0 xfer_len: 26
[ 42.357131] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 25
[ 42.364273] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.375434] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.382315] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 46
[ 42.389454] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.401267] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.408174] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 23
[ 42.415338] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.428215] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.435097] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 294
[ 42.442327] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.454144] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.461025] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 294
[ 42.468256] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.479429] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 20
[ 42.486314] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 24
[ 42.493454] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.504504] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.511385] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 152
[ 42.518612] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.529675] mhi_wwan_ctrl mhi0_DUN: mhi_ul_xfer_cb: status: 0 xfer_len: 12
[ 42.536560] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 20
[ 42.543702] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.554858] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.561738] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 26
[ 42.568878] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.581755] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 18
[ 42.588637] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 736
[ 42.595869] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.608518] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 17
[ 42.615403] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 46
[ 42.622543] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.633622] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 18
[ 42.640509] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 736
[ 42.647741] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.660219] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 18
[ 42.667125] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 739
[ 42.674378] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.685479] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 24
[ 42.692373] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 20
[ 42.699507] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 50
[ 42.706646] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.717880] mhi_wwan_ctrl mhi0_QMI: mhi_ul_xfer_cb: status: 0 xfer_len: 13
[ 42.724764] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 20
[ 42.731898] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0
receive_len: 156
[ 42.739125] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.749313] mhi_wwan_ctrl mhi0_DUN: mhi_ul_xfer_cb: status: 0 xfer_len: 10
[ 42.756197] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 49
[ 42.763336] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.774589] mhi_wwan_ctrl mhi0_DUN: mhi_ul_xfer_cb: status: 0 xfer_len: 11
[ 42.781473] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 6
[ 42.788527] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 42.800278] mhi_wwan_ctrl mhi0_DUN: mhi_ul_xfer_cb: status: 0 xfer_len: 10
[ 42.807183] mhi_wwan_ctrl mhi0_DUN: mhi_dl_xfer_cb: status: 0 receive_len: 36
[ 42.814347] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 44.343867] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 12
[ 44.351051] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 46.858056] mhi mhi0: Allowing M3 transition
[ 46.862369] mhi mhi0: Waiting for M3 completion
[ 46.903504] mhi mhi0: State change event to state: M3
[ 48.979995] mhi mhi0: Entered with PM state: M3, MHI state: M3
[ 48.997968] mhi mhi0: State change event to state: M0
[ 49.003080] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 49.107630] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 12
[ 49.114796] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 51.189638] mhi mhi0: Allowing M3 transition
[ 51.193961] mhi mhi0: Waiting for M3 completion
[ 51.214179] mhi mhi0: State change event to state: M3
[ 57.364001] mhi mhi0: Entered with PM state: M3, MHI state: M3
[ 57.396293] mhi mhi0: State change event to state: M0
[ 57.401406] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 57.438409] mhi_wwan_ctrl mhi0_QMI: mhi_dl_xfer_cb: status: 0 receive_len: 12
[ 57.445550] mhi mhi0: local ee: MISSION MODE state: M0 device ee:
MISSION MODE state: M0
[ 59.869534] mhi mhi0: Allowing M3 transition
[ 59.873851] mhi mhi0: Waiting for M3 completion
[ 59.893867] mhi mhi0: State change event to state: M3
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-10-11 14:44 ` Thomas Perrot
2021-10-12 19:44 ` Aleksander Morgado
@ 2021-11-08 15:11 ` Aleksander Morgado
2021-11-08 16:31 ` Thomas Perrot
1 sibling, 1 reply; 33+ messages in thread
From: Aleksander Morgado @ 2021-11-08 15:11 UTC (permalink / raw)
To: Thomas Perrot
Cc: Manivannan Sadhasivam, Loic Poulain, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm
Hey Thomas,
Reviving an old email :)
> On our setup, using i.MX6DL based board and a PCIe Sierra Wireless
> EM9190 module, running Yocto and Linux 5.13, we don't have much success
> for the moment, qmi and mbim commands very often end in timeout.
>
> Otherwise, when responses are received, we also can observe strange
> things: unexpected messages, response to previous commands or queue
> buffer issue.
>
Once all my boot reliability issues seem solved, I've also started to
notice what you mean here. If I run a normal ModemManager build with
both QMI and MBIM enabled, MM will try to probe both the QMI and MBIM
ports. When that happens, I have no idea why, the modem gets in some
weird state with commands timing out and what not. Maybe it's because
we're using both ports at the same time, maybe it's because we run QMI
on both the QMI and MBIM ports, no idea, the only thing I know is that
if you choose to use either one or the other, the whole setup is fully
stable.
E.g. I'm right now testing my build after compiling ModemManager using
--without-mbim (so QMI only), and I have absolutely no error. Another
option if you don't want to rebuild MM is to flag the MBIM or the QMI
port as ID_MM_PORT_IGNORE with udev rules, which is very likely what
I'll end up doing in upstream ModemManager to have a proper default.
I was thinking in preparing and sending for review the EM91xx entry
for drivers/bus/mhi/pci_generic.c, but it's mostly based on what you
suggested in the Sierra Wireless forum, so not sure if you'd like to
send it yourself here? The only changes I did w.r.t. what you
suggested are setting sideband_wake to false, and listing the
PCI_DEVICE_SUB() before the more generic PCI_DEVICE() one.
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-11-08 15:11 ` Aleksander Morgado
@ 2021-11-08 16:31 ` Thomas Perrot
2021-11-08 20:16 ` Aleksander Morgado
0 siblings, 1 reply; 33+ messages in thread
From: Thomas Perrot @ 2021-11-08 16:31 UTC (permalink / raw)
To: Aleksander Morgado
Cc: Manivannan Sadhasivam, Loic Poulain, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm
[-- Attachment #1: Type: text/plain, Size: 2158 bytes --]
Hi Aleksander,
On Mon, 2021-11-08 at 16:11 +0100, Aleksander Morgado wrote:
> Hey Thomas,
>
> Reviving an old email :)
>
> > On our setup, using i.MX6DL based board and a PCIe Sierra Wireless
> > EM9190 module, running Yocto and Linux 5.13, we don't have much
> > success
> > for the moment, qmi and mbim commands very often end in timeout.
> >
> > Otherwise, when responses are received, we also can observe strange
> > things: unexpected messages, response to previous commands or queue
> > buffer issue.
> >
>
> Once all my boot reliability issues seem solved, I've also started to
> notice what you mean here. If I run a normal ModemManager build with
> both QMI and MBIM enabled, MM will try to probe both the QMI and MBIM
> ports. When that happens, I have no idea why, the modem gets in some
> weird state with commands timing out and what not. Maybe it's because
> we're using both ports at the same time, maybe it's because we run
> QMI
> on both the QMI and MBIM ports, no idea, the only thing I know is
> that
> if you choose to use either one or the other, the whole setup is
> fully
> stable.
>
> E.g. I'm right now testing my build after compiling ModemManager
> using
> --without-mbim (so QMI only), and I have absolutely no error.
That's very good news!
> Another
> option if you don't want to rebuild MM is to flag the MBIM or the QMI
> port as ID_MM_PORT_IGNORE with udev rules, which is very likely what
> I'll end up doing in upstream ModemManager to have a proper default.
>
> I was thinking in preparing and sending for review the EM91xx entry
> for drivers/bus/mhi/pci_generic.c, but it's mostly based on what you
> suggested in the Sierra Wireless forum, so not sure if you'd like to
> send it yourself here?
I can send it myself here, and I add you as developer if you'd like ?
Best regards,
Thomas
> The only changes I did w.r.t. what you
> suggested are setting sideband_wake to false, and listing the
> PCI_DEVICE_SUB() before the more generic PCI_DEVICE() one.
>
--
Thomas Perrot, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Sierra Wireless EM9191 integration issues in mhi+wwan
2021-11-08 16:31 ` Thomas Perrot
@ 2021-11-08 20:16 ` Aleksander Morgado
0 siblings, 0 replies; 33+ messages in thread
From: Aleksander Morgado @ 2021-11-08 20:16 UTC (permalink / raw)
To: Thomas Perrot
Cc: Manivannan Sadhasivam, Loic Poulain, Hemant Kumar,
Thomas Petazzoni, linux-arm-msm
Hey!
> > > On our setup, using i.MX6DL based board and a PCIe Sierra Wireless
> > > EM9190 module, running Yocto and Linux 5.13, we don't have much
> > > success
> > > for the moment, qmi and mbim commands very often end in timeout.
> > >
> > > Otherwise, when responses are received, we also can observe strange
> > > things: unexpected messages, response to previous commands or queue
> > > buffer issue.
> > >
> >
> > Once all my boot reliability issues seem solved, I've also started to
> > notice what you mean here. If I run a normal ModemManager build with
> > both QMI and MBIM enabled, MM will try to probe both the QMI and MBIM
> > ports. When that happens, I have no idea why, the modem gets in some
> > weird state with commands timing out and what not. Maybe it's because
> > we're using both ports at the same time, maybe it's because we run
> > QMI
> > on both the QMI and MBIM ports, no idea, the only thing I know is
> > that
> > if you choose to use either one or the other, the whole setup is
> > fully
> > stable.
> >
> > E.g. I'm right now testing my build after compiling ModemManager
> > using
> > --without-mbim (so QMI only), and I have absolutely no error.
>
> That's very good news!
>
> > Another
> > option if you don't want to rebuild MM is to flag the MBIM or the QMI
> > port as ID_MM_PORT_IGNORE with udev rules, which is very likely what
> > I'll end up doing in upstream ModemManager to have a proper default.
> >
> > I was thinking in preparing and sending for review the EM91xx entry
> > for drivers/bus/mhi/pci_generic.c, but it's mostly based on what you
> > suggested in the Sierra Wireless forum, so not sure if you'd like to
> > send it yourself here?
>
> I can send it myself here, and I add you as developer if you'd like ?
>
Then please send it yourself, I just suggested a couple minor changes
on the work you did :)
Thanks!
--
Aleksander
https://aleksander.es
^ permalink raw reply [flat|nested] 33+ messages in thread
end of thread, other threads:[~2021-11-08 20:17 UTC | newest]
Thread overview: 33+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-07 13:04 Sierra Wireless EM9191 integration issues in mhi+wwan Aleksander Morgado
2021-10-09 10:51 ` Manivannan Sadhasivam
2021-10-12 19:38 ` Aleksander Morgado
2021-10-22 4:42 ` Manivannan Sadhasivam
2021-10-22 9:20 ` Aleksander Morgado
2021-10-22 14:40 ` Manivannan Sadhasivam
2021-10-25 8:10 ` Aleksander Morgado
2021-11-02 10:50 ` Manivannan Sadhasivam
2021-11-02 16:13 ` Aleksander Morgado
2021-11-02 16:22 ` Manivannan Sadhasivam
2021-11-02 16:55 ` Aleksander Morgado
2021-11-02 18:09 ` Manivannan Sadhasivam
[not found] ` <CAMZdPi9+zrsDy9WTipamRWBXMOxUX1tfsk2W52b9wG-4q21fWA@mail.gmail.com>
2021-11-04 22:50 ` Bhaumik Bhatt
2021-11-08 7:40 ` Manivannan Sadhasivam
2021-11-08 13:38 ` Aleksander Morgado
2021-10-11 14:44 ` Thomas Perrot
2021-10-12 19:44 ` Aleksander Morgado
2021-10-14 9:51 ` Thomas Perrot
2021-10-14 10:04 ` Aleksander Morgado
2021-10-14 17:28 ` Loic Poulain
2021-10-14 20:25 ` Aleksander Morgado
2021-10-18 9:14 ` Aleksander Morgado
2021-10-18 9:59 ` Loic Poulain
2021-10-18 11:26 ` Thomas Perrot
2021-10-18 12:46 ` Loic Poulain
2021-10-18 14:07 ` Thomas Perrot
2021-10-18 14:16 ` Thomas Perrot
2021-10-19 8:38 ` Aleksander Morgado
2021-10-20 8:43 ` Aleksander Morgado
2021-10-22 14:33 ` Aleksander Morgado
2021-11-08 15:11 ` Aleksander Morgado
2021-11-08 16:31 ` Thomas Perrot
2021-11-08 20:16 ` Aleksander Morgado
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.