Re: [PATCH v1 0/5] Enable fw_devlink=on by default

[Saravana Kannan <saravanak@google.com>]

On Wed, Jan 13, 2021 at 3:11 AM Marc Zyngier <maz@kernel.org> wrote:
>
> On 2021-01-07 20:05, Greg Kroah-Hartman wrote:
> > On Thu, Dec 17, 2020 at 07:16:58PM -0800, Saravana Kannan wrote:
> >> As discussed in LPC 2020, cyclic dependencies in firmware that
> >> couldn't
> >> be broken using logic was one of the last remaining reasons
> >> fw_devlink=on couldn't be set by default.
> >>
> >> This series changes fw_devlink so that when a cyclic dependency is
> >> found
> >> in firmware, the links between those devices fallback to permissive
> >> mode
> >> behavior. This way, the rest of the system still benefits from
> >> fw_devlink, but the ambiguous cases fallback to permissive mode.
> >>
> >> Setting fw_devlink=on by default brings a bunch of benefits
> >> (currently,
> >> only for systems with device tree firmware):
> >> * Significantly cuts down deferred probes.
> >> * Device probe is effectively attempted in graph order.
> >> * Makes it much easier to load drivers as modules without having to
> >>   worry about functional dependencies between modules (depmod is still
> >>   needed for symbol dependencies).
> >>
> >> Greg/Rafael,
> >>
> >> Can we get this pulled into 5.11-rc1 or -rc2 soon please? I expect to
> >> see some issues due to device drivers that aren't following best
> >> practices (they don't expose the device to driver core). Want to
> >> identify those early on and try to have them fixed before 5.11
> >> release.
> >> See [1] for an example of such a case.
> >
> > Now queued up in my tree, will show up in linux-next in a few days,
> > let's see what breaks!  :)
> >
> > And it is scheduled for 5.12-rc1, not 5.11, sorry.
>
> For the record, this breaks my rk3399 board, (NanoPC-T4) as no mass
> storage can be discovered (it lives on PCIe):
>
> (initramfs) find /sys -name 'waiting_for_supplier'| xargs grep .| egrep
> -v ':0$'
> /sys/devices/platform/ff3d0000.i2c/i2c-4/4-0022/waiting_for_supplier:1
> /sys/devices/platform/f8000000.pcie/waiting_for_supplier:1
> /sys/devices/platform/fe320000.mmc/waiting_for_supplier:1
> /sys/devices/platform/sdio-pwrseq/waiting_for_supplier:1
> /sys/devices/platform/ff3c0000.i2c/i2c-0/0-001b/waiting_for_supplier:1
>
> Enabling the debug prints in device_links_check_suppliers(), I end up
> with
> the dump below (apologies for the size).
>
> This seems to all hang on the GPIO banks, but it is pretty unclear what
> is wrong with them.
>
> Happy to test things further.

Thanks for the logs Marc. Looks like a majority/all of the issue is
due to gpio device nodes [1] being "probed" without creating a proper
struct device for it.

You can see here [2] how the driver for the parent device just loops
through the child DT nodes and initializes them. This would be okay if
the DT nodes didn't have a "compatible" property for the gpio device
[1]. So to fix this, the driver[2] needs to be updated to properly
populate the child devices and then probe them using a proper driver
for "rockchip,gpio-bank". And most of the gpio init code into this new
driver.

The DT implementation of fw_devlink has the expectation that device
tree nodes with "compatible" properties will have struct devices
created for them. Without this expectation, it has no way to know how
far up the ancestor chain fw_devlink needs to walk up before it can
expect a supplier device to create a device link to.

Heiko,

Could you please refactor drivers/pinctrl/pinctrl-rockchip.c to create
and probe struct devices for "rockchip,gpio-bank" nodes? This allows
fw_devlink to work for these devices and makes sure devices probe in
the right order, suspend/resume in the right order, etc.

-Saravana

[1] - https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/rockchip/rk3399.dtsi#n1956
[2] - https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pinctrl/pinctrl-rockchip.c#n3566

>
>          M.
>
>   platform vcc3v3-sys: probe deferral - supplier vcc12v0-sys not ready
>   platform vcc5v0-sys: probe deferral - supplier vcc12v0-sys not ready
>   platform vcc1v8-s3: probe deferral - wait for supplier pmic@1b
>   platform vcc3v0-sd: probe deferral - supplier vcc3v3-sys not ready
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vbus-typec: probe deferral - supplier vcc5v0-sys not ready
>   platform vcc5v0-host0: probe deferral - supplier vcc5v0-sys not ready
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - wait for supplier pmic@1b
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - wait for supplier pmic@1b
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - wait for supplier pmic@1b
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - wait for supplier pmic@1b
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - wait for supplier pmic@1b
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - wait for supplier pmic@1b
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform fe320000.mmc: probe deferral - wait for supplier pmic@1b
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - wait for supplier pmic@1b
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform fe320000.mmc: probe deferral - wait for supplier pmic@1b
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - supplier 0-001b not ready
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   platform fe320000.mmc: probe deferral - wait for supplier
> gpio0@ff720000
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - supplier 0-001b not ready
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   platform fe320000.mmc: probe deferral - wait for supplier
> gpio0@ff720000
>   platform fe300000.ethernet: probe deferral - supplier 0-001b not ready
>   platform fe380000.usb: probe deferral - supplier
> ff770000.syscon:usb2-phy@e450 not ready
>   platform fe3c0000.usb: probe deferral - supplier
> ff770000.syscon:usb2-phy@e460 not ready
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - supplier 0-001b not ready
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   platform fe3a0000.usb: probe deferral - supplier
> ff770000.syscon:usb2-phy@e450 not ready
>   platform fe3e0000.usb: probe deferral - supplier
> ff770000.syscon:usb2-phy@e460 not ready
>   platform fe320000.mmc: probe deferral - wait for supplier
> gpio0@ff720000
>   platform fe300000.ethernet: probe deferral - supplier 0-001b not ready
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - supplier 0-001b not ready
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   i2c 0-001b: probe deferral - wait for supplier gpio1@ff730000
>   platform fe320000.mmc: probe deferral - wait for supplier
> gpio0@ff720000
>   platform fe300000.ethernet: probe deferral - supplier 0-001b not ready
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - supplier 0-001b not ready
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   i2c 0-001b: probe deferral - wait for supplier gpio1@ff730000
>   platform fe320000.mmc: probe deferral - wait for supplier
> gpio0@ff720000
>   platform fe300000.ethernet: probe deferral - supplier 0-001b not ready
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - supplier 0-001b not ready
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   i2c 0-001b: probe deferral - wait for supplier gpio1@ff730000
>   platform fe320000.mmc: probe deferral - wait for supplier
> gpio0@ff720000
>   platform fe300000.ethernet: probe deferral - supplier 0-001b not ready
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - supplier 0-001b not ready
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   i2c 0-001b: probe deferral - wait for supplier gpio1@ff730000
>   platform fe320000.mmc: probe deferral - wait for supplier
> gpio0@ff720000
>   platform fe300000.ethernet: probe deferral - supplier 0-001b not ready
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>   platform vcc1v8-s3: probe deferral - supplier 0-001b not ready
>   platform vcca0v9-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform vcca1v8-s3: probe deferral - supplier vcc1v8-s3 not ready
>   platform f8000000.pcie: probe deferral - wait for supplier
> gpio2@ff780000
>   i2c 0-001b: probe deferral - wait for supplier gpio1@ff730000
>   platform fe320000.mmc: probe deferral - wait for supplier
> gpio0@ff720000
>   platform fe300000.ethernet: probe deferral - supplier 0-001b not ready
>   platform sdio-pwrseq: probe deferral - wait for supplier gpio0@ff720000
>
> --
> Jazz is not dead. It just smells funny...