Am 2021-01-18 22:01, schrieb Saravana Kannan: > On Sun, Jan 17, 2021 at 3:01 PM Michael Walle wrote: >> > Cyclic dependencies in some firmware was one of the last remaining >> > reasons fw_devlink=on couldn't be set by default. Now that cyclic >> > dependencies don't block probing, set fw_devlink=on by default. >> > >> > Setting fw_devlink=on by default brings a bunch of benefits (currently, >> > only for systems with device tree firmware): >> > * Significantly cuts down deferred probes. >> > * Device probe is effectively attempted in graph order. >> > * Makes it much easier to load drivers as modules without having to >> > worry about functional dependencies between modules (depmod is still >> > needed for symbol dependencies). >> > >> > If this patch prevents some devices from probing, it's very likely due >> > to the system having one or more device drivers that "probe"/set up a >> > device (DT node with compatible property) without creating a struct >> > device for it. If we hit such cases, the device drivers need to be >> > fixed so that they populate struct devices and probe them like normal >> > device drivers so that the driver core is aware of the devices and their >> > status. See [1] for an example of such a case. >> > >> > [1] - https://lore.kernel.org/lkml/CAGETcx9PiX==mLxB9PO8Myyk6u2vhPVwTMsA5NkD-ywH5xhusw@mail.gmail.com/ >> > Signed-off-by: Saravana Kannan >> >> This breaks (at least) probing of the PCIe controllers of my board. >> The >> driver in question is >> drivers/pci/controller/dwc/pci-layerscape.c >> I've also put the maintainers of this driver on CC. Looks like it uses >> a >> proper struct device. But it uses builtin_platform_driver_probe() and >> apparently it waits for the iommu which uses module_platform_driver(). >> Dunno if that will work together. > > Yeah, the builtin vs module doesn't matter. I've had fw_devlink work > multiple times with the consumer driver being built in and the > supplier actually loaded as a module. Making that work is one of the > goals of fw_devlink. Ok. >> The board device tree can be found here: >> arch/arm64/boot/dts/freescale/fsl-ls1028a-kontron-sl28-var3-ads2.dts >> >> Attached is the log with enabled "probe deferral" messages enabled. > > I took a look at the logs. As you said, pci seems to be waiting on > iommu, but it's not clear why the iommu didn't probe by then. Can you > add initcall_debug=1 and enable the logs in device_link_add()? Btw, I > realize one compromise on the logs is to send them as an attachment > instead of inline. That way, it's still archived in the list, but I > don't have to deal with log lines getting wrapped, etc. > > Thanks for reporting the issues. Also, could you try picking up all of > these changes and giving it a shot. It's unlikely to help, but I want > to rule out issues related to fixes in progress. > > https://lore.kernel.org/lkml/20210116011412.3211292-1-saravanak@google.com/ > https://lore.kernel.org/lkml/20210115210159.3090203-1-saravanak@google.com/ > https://lore.kernel.org/lkml/20201218210750.3455872-1-saravanak@google.com/ Did pick them up, the last one had a conflict due some superfluous lines. Maybe they got reordered in that arrray. Issue still persist. I've enabled the debug in device_link_add(), in device_links_check_suppliers() and booted with initcall_debug. Please see attached log. Lets see how that goes ;) [ 0.132687] calling ls_pcie_driver_init+0x0/0x38 @ 1 [ 0.132762] platform 3400000.pcie: probe deferral - supplier 5000000.iommu not ready [ 0.132777] platform 3500000.pcie: probe deferral - supplier 5000000.iommu not ready [ 0.132818] initcall ls_pcie_driver_init+0x0/0x38 returned -19 after 119 usecs After that, ls_pcie_driver_init() is never called again. -michael