Raspberry Pi 4B: Failed to create device link with soc:firmware:gpio

Raspberry Pi 4B: Failed to create device link with soc:firmware:gpio

[Florian Fainelli]

Hi Saravana,

Using v6.2-10217-ga93e884edf61v my Raspberry Pi 4B issues the following 
for the "extended GPIO" provider:

[    5.969855] uart-pl011 fe201000.serial: Failed to create device link 
with soc:firmware:gpio

The kernel configuration I am using can be found here:

https://gist.github.com/ffainelli/4eb83740c25b10f75b54560f8c8febb1

And the DTS is arch/arm*/boot/dts/bcm2711-rpi-4-b.dts

Does this ring any bell?

Thanks!
-- 
Florian

Re: Raspberry Pi 4B: Failed to create device link with soc:firmware:gpio

[Saravana Kannan]

On Sat, Feb 25, 2023 at 7:38 AM Florian Fainelli <f.fainelli@gmail.com> wrote:
>
> Hi Saravana,
>
> Using v6.2-10217-ga93e884edf61v my Raspberry Pi 4B issues the following
> for the "extended GPIO" provider:
>
> [    5.969855] uart-pl011 fe201000.serial: Failed to create device link
> with soc:firmware:gpio

Outside of this error, is it actually breaking anything? Also, can you
pull in this patch and tell me what it says? I want to know what the
flags are.
https://lore.kernel.org/lkml/20230225064148.274376-1-saravanak@google.com/

Can you also change every pr_debug and dev_dbg in drivers/base/core.c
to an info and then give me the logs as an attachment?

> The kernel configuration I am using can be found here:
>
> https://gist.github.com/ffainelli/4eb83740c25b10f75b54560f8c8febb1
>
> And the DTS is arch/arm*/boot/dts/bcm2711-rpi-4-b.dts

Ah, is this yet another case of a DTS that's not upstream? Don't
worry, I'll still look at it as it might point to some existing
upstream issue too.

Thanks,
Saravana

Re: Raspberry Pi 4B: Failed to create device link with soc:firmware:gpio

[Florian Fainelli]

On 2/25/2023 4:01 PM, Saravana Kannan wrote:
> On Sat, Feb 25, 2023 at 7:38 AM Florian Fainelli <f.fainelli@gmail.com> wrote:
>>
>> Hi Saravana,
>>
>> Using v6.2-10217-ga93e884edf61v my Raspberry Pi 4B issues the following
>> for the "extended GPIO" provider:
>>
>> [    5.969855] uart-pl011 fe201000.serial: Failed to create device link
>> with soc:firmware:gpio
> 
> Outside of this error, is it actually breaking anything? 

There is just this warning, there does not appear to be a functional issue.

> Also, can you
> pull in this patch and tell me what it says? I want to know what the
> flags are.
> https://lore.kernel.org/lkml/20230225064148.274376-1-saravanak@google.com/
> 
> Can you also change every pr_debug and dev_dbg in drivers/base/core.c
> to an info and then give me the logs as an attachment?

Sure will do.

> 
>> The kernel configuration I am using can be found here:
>>
>> https://gist.github.com/ffainelli/4eb83740c25b10f75b54560f8c8febb1
>>
>> And the DTS is arch/arm*/boot/dts/bcm2711-rpi-4-b.dts
> 
> Ah, is this yet another case of a DTS that's not upstream? Don't
> worry, I'll still look at it as it might point to some existing
> upstream issue too.

This is with the upstream DTS file under that directory, I used * to 
denote that this was used equally for both arm and arm64.

Thanks!
-- 
Florian

Re: Raspberry Pi 4B: Failed to create device link with soc:firmware:gpio

[Florian Fainelli]

On 2/25/2023 5:58 PM, Florian Fainelli wrote:
> 
> 
> On 2/25/2023 4:01 PM, Saravana Kannan wrote:
>> On Sat, Feb 25, 2023 at 7:38 AM Florian Fainelli 
>> <f.fainelli@gmail.com> wrote:
>>>
>>> Hi Saravana,
>>>
>>> Using v6.2-10217-ga93e884edf61v my Raspberry Pi 4B issues the following
>>> for the "extended GPIO" provider:
>>>
>>> [    5.969855] uart-pl011 fe201000.serial: Failed to create device link
>>> with soc:firmware:gpio
>>
>> Outside of this error, is it actually breaking anything? 
> 
> There is just this warning, there does not appear to be a functional issue.
> 
>> Also, can you
>> pull in this patch and tell me what it says? I want to know what the
>> flags are.
>> https://lore.kernel.org/lkml/20230225064148.274376-1-saravanak@google.com/

Pulling in this patch results in the following being printed:

[   14.866835] uart-pl011 fe201000.serial: Failed to create device link 
(0x180) with soc:firmware:gpio

>>
>> Can you also change every pr_debug and dev_dbg in drivers/base/core.c
>> to an info and then give me the logs as an attachment?
> 
> Sure will do.

Please find the log here:

https://gist.github.com/4e7e5e09ebfb3994505c917a8c0892d6

Thanks
-- 
Florian

Re: Raspberry Pi 4B: Failed to create device link with soc:firmware:gpio

[Saravana Kannan]

On Sun, Feb 26, 2023 at 11:14 AM Florian Fainelli <f.fainelli@gmail.com> wrote:
>
>
>
> On 2/25/2023 5:58 PM, Florian Fainelli wrote:
> >
> >
> > On 2/25/2023 4:01 PM, Saravana Kannan wrote:
> >> On Sat, Feb 25, 2023 at 7:38 AM Florian Fainelli
> >> <f.fainelli@gmail.com> wrote:
> >>>
> >>> Hi Saravana,
> >>>
> >>> Using v6.2-10217-ga93e884edf61v my Raspberry Pi 4B issues the following
> >>> for the "extended GPIO" provider:
> >>>
> >>> [    5.969855] uart-pl011 fe201000.serial: Failed to create device link
> >>> with soc:firmware:gpio
> >>
> >> Outside of this error, is it actually breaking anything?
> >
> > There is just this warning, there does not appear to be a functional issue.
> >
> >> Also, can you
> >> pull in this patch and tell me what it says? I want to know what the
> >> flags are.
> >> https://lore.kernel.org/lkml/20230225064148.274376-1-saravanak@google.com/
>
> Pulling in this patch results in the following being printed:
>
> [   14.866835] uart-pl011 fe201000.serial: Failed to create device link
> (0x180) with soc:firmware:gpio

I spent at least 2 hours looking at the logs and the DT files and I'm
still kinda lost.

The 0x180 means it's a DL_FLAG_INFERRED | DL_FLAG_SYNC_STATE_ONLY.
That's just fw_devlink trying to create a "proxy" link where an
ancestor of a consumer (can be several levels above consumer) creates
a SYNC_STATE_ONLY link to the supplier while we wait for the consumer
device to get added. This prevents sync_state() from being called too
early on the supplier.

There are so many includes in the dts/dtsi files that my head is
spinning. I finally found out where the soc:firmware:gpio device was
coming from (after confusing myself with gpio@7e200000 for a bit) and
where fe201000.serial was coming from. I still couldn't figure out how
the address got mapped to fe201000 in fe201000.serial -- that
generally means the parent has some address offset, but I don't see
that in DT (but it is not important for this discussion, so we can
ignore that).

Anyway, I see no supplier-consumer link between serial@7e201000 (or
any of its zero descendants) and soc:firmware:gpio (which is the node
expgpio:). So I'm very confused why we might even try to create this
sync state only device link between these two.

There are actually two times where we try to create such a link:

First attempt that actually succeeds -- but I have no idea why we even do this:
[    0.100047] device:
'platform:soc:firmware:gpio--amba:fe201000.serial': device_add
[    0.100232] amba fe201000.serial: Linked as a sync state only
consumer to soc:firmware:gpio
the "amba" prefix tells us a driver hasn't been bound to fe201000.serial yet.

Second attempt is the one that fails.
[   15.516166] uart-pl011 fe201000.serial: Failed to create device
link (0x180) with soc:firmware:gpio
The uart-pl011 tells us that the driver has bound to fe201000.serial.

And it fails because of this sensible check I had put up a while ago
inside device_link_add():
        /*
         * SYNC_STATE_ONLY links are useless once a consumer device
has probed.
         * So, only create it if the consumer hasn't probed yet.
         */
        if (flags & DL_FLAG_SYNC_STATE_ONLY &&
            consumer->links.status != DL_DEV_NO_DRIVER &&
            consumer->links.status != DL_DEV_PROBING) {
                link = NULL;
                goto out;
        }

So the real question is still to figure out why fw_devlink is trying
to create this device link.

So to debug this further the following would help a lot:
1. Pull the dtb from the device and then decompile it and provide me
the result. This way, I can be sure I'm not missing something and
don't have to dig through all the includes -- I forgot the exact
commands for it, but it's been shared in LKML before. Let me know if
you need me to dig this up.

2. I think a stack backtrace when these two device link attempts are
made might help me figure out why they are being created. To get these
backtraces, can you do the following please?
a. Put a WARN_ON() inside device_add() for when the device name matches:
platform:soc:firmware:gpio--amba:fe201000.serial
b. Put a WARN_ON(1) where we print "Failed to create device link..."

Feel free to dig more into 2a and 2b if you find that the stack trace
doesn't tell much -- one problem with the stack trace is that it
doesn't show the params being passed to
__fw_devlink_link_to_consumers() and __fw_devlink_link_to_suppliers()

Thanks,
Saravana

Re: Raspberry Pi 4B: Failed to create device link with soc:firmware:gpio

[Stefan Wahren]

Hi Saravana,

Am 01.03.23 um 08:49 schrieb Saravana Kannan:
> On Sun, Feb 26, 2023 at 11:14 AM Florian Fainelli <f.fainelli@gmail.com> wrote:
>>
>>
>> On 2/25/2023 5:58 PM, Florian Fainelli wrote:
>>>
>>> On 2/25/2023 4:01 PM, Saravana Kannan wrote:
>>>> On Sat, Feb 25, 2023 at 7:38 AM Florian Fainelli
>>>> <f.fainelli@gmail.com> wrote:
>>>>> Hi Saravana,
>>>>>
>>>>> Using v6.2-10217-ga93e884edf61v my Raspberry Pi 4B issues the following
>>>>> for the "extended GPIO" provider:
>>>>>
>>>>> [    5.969855] uart-pl011 fe201000.serial: Failed to create device link
>>>>> with soc:firmware:gpio
>>>> Outside of this error, is it actually breaking anything?
>>> There is just this warning, there does not appear to be a functional issue.
>>>
>>>> Also, can you
>>>> pull in this patch and tell me what it says? I want to know what the
>>>> flags are.
>>>> https://lore.kernel.org/lkml/20230225064148.274376-1-saravanak@google.com/
>> Pulling in this patch results in the following being printed:
>>
>> [   14.866835] uart-pl011 fe201000.serial: Failed to create device link
>> (0x180) with soc:firmware:gpio
> I spent at least 2 hours looking at the logs and the DT files and I'm
> still kinda lost.
>
> The 0x180 means it's a DL_FLAG_INFERRED | DL_FLAG_SYNC_STATE_ONLY.
> That's just fw_devlink trying to create a "proxy" link where an
> ancestor of a consumer (can be several levels above consumer) creates
> a SYNC_STATE_ONLY link to the supplier while we wait for the consumer
> device to get added. This prevents sync_state() from being called too
> early on the supplier.
>
> There are so many includes in the dts/dtsi files that my head is
> spinning. I finally found out where the soc:firmware:gpio device was
> coming from (after confusing myself with gpio@7e200000 for a bit) and
> where fe201000.serial was coming from. I still couldn't figure out how
> the address got mapped to fe201000 in fe201000.serial -- that
> generally means the parent has some address offset, but I don't see
> that in DT (but it is not important for this discussion, so we can
> ignore that).
This is uart0 which is at first defined in bcm283x.dtsi. On the 
Raspberry Pi 4 this UART is connected to the Bluetooth IC. On Linux 
probing of the serial communication via DT is done via serial device bus 
(see bcm283x-rpi-wifi-bt.dtsi).
> Anyway, I see no supplier-consumer link between serial@7e201000 (or
> any of its zero descendants) and soc:firmware:gpio (which is the node
> expgpio:). So I'm very confused why we might even try to create this
> sync state only device link between these two.
>
> There are actually two times where we try to create such a link:
>
> First attempt that actually succeeds -- but I have no idea why we even do this:
> [    0.100047] device:
> 'platform:soc:firmware:gpio--amba:fe201000.serial': device_add
> [    0.100232] amba fe201000.serial: Linked as a sync state only
> consumer to soc:firmware:gpio

I assume the link is established by raspberry,firmware-gpio which 
provides the necessary BT shutdown-gpios defined in bcm2711-rpi-4-b.dts. 
Seems to me that the problem is, that necessary underlying firmware 
driver is probed "too late":

[ 15.456506] raspberrypi-firmware soc:firmware: Attached to firmware 
from 2020-02-12T12:36:21

Hope this helps a little bit

> the "amba" prefix tells us a driver hasn't been bound to fe201000.serial yet.
>
> Second attempt is the one that fails.
> [   15.516166] uart-pl011 fe201000.serial: Failed to create device
> link (0x180) with soc:firmware:gpio
> The uart-pl011 tells us that the driver has bound to fe201000.serial.
>
> And it fails because of this sensible check I had put up a while ago
> inside device_link_add():
>          /*
>           * SYNC_STATE_ONLY links are useless once a consumer device
> has probed.
>           * So, only create it if the consumer hasn't probed yet.
>           */
>          if (flags & DL_FLAG_SYNC_STATE_ONLY &&
>              consumer->links.status != DL_DEV_NO_DRIVER &&
>              consumer->links.status != DL_DEV_PROBING) {
>                  link = NULL;
>                  goto out;
>          }
>
> So the real question is still to figure out why fw_devlink is trying
> to create this device link.
>
> So to debug this further the following would help a lot:
> 1. Pull the dtb from the device and then decompile it and provide me
> the result. This way, I can be sure I'm not missing something and
> don't have to dig through all the includes -- I forgot the exact
> commands for it, but it's been shared in LKML before. Let me know if
> you need me to dig this up.
>
> 2. I think a stack backtrace when these two device link attempts are
> made might help me figure out why they are being created. To get these
> backtraces, can you do the following please?
> a. Put a WARN_ON() inside device_add() for when the device name matches:
> platform:soc:firmware:gpio--amba:fe201000.serial
> b. Put a WARN_ON(1) where we print "Failed to create device link..."
>
> Feel free to dig more into 2a and 2b if you find that the stack trace
> doesn't tell much -- one problem with the stack trace is that it
> doesn't show the params being passed to
> __fw_devlink_link_to_consumers() and __fw_devlink_link_to_suppliers()
>
> Thanks,
> Saravana
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Raspberry Pi 4B: Failed to create device link with soc:firmware:gpio

[Saravana Kannan]

On Wed, Mar 1, 2023 at 8:57 AM Stefan Wahren <stefan.wahren@i2se.com> wrote:
>
> Hi Saravana,
>
> Am 01.03.23 um 08:49 schrieb Saravana Kannan:
> > On Sun, Feb 26, 2023 at 11:14 AM Florian Fainelli <f.fainelli@gmail.com> wrote:
> >>
> >>
> >> On 2/25/2023 5:58 PM, Florian Fainelli wrote:
> >>>
> >>> On 2/25/2023 4:01 PM, Saravana Kannan wrote:
> >>>> On Sat, Feb 25, 2023 at 7:38 AM Florian Fainelli
> >>>> <f.fainelli@gmail.com> wrote:
> >>>>> Hi Saravana,
> >>>>>
> >>>>> Using v6.2-10217-ga93e884edf61v my Raspberry Pi 4B issues the following
> >>>>> for the "extended GPIO" provider:
> >>>>>
> >>>>> [    5.969855] uart-pl011 fe201000.serial: Failed to create device link
> >>>>> with soc:firmware:gpio
> >>>> Outside of this error, is it actually breaking anything?
> >>> There is just this warning, there does not appear to be a functional issue.
> >>>
> >>>> Also, can you
> >>>> pull in this patch and tell me what it says? I want to know what the
> >>>> flags are.
> >>>> https://lore.kernel.org/lkml/20230225064148.274376-1-saravanak@google.com/
> >> Pulling in this patch results in the following being printed:
> >>
> >> [   14.866835] uart-pl011 fe201000.serial: Failed to create device link
> >> (0x180) with soc:firmware:gpio
> > I spent at least 2 hours looking at the logs and the DT files and I'm
> > still kinda lost.
> >
> > The 0x180 means it's a DL_FLAG_INFERRED | DL_FLAG_SYNC_STATE_ONLY.
> > That's just fw_devlink trying to create a "proxy" link where an
> > ancestor of a consumer (can be several levels above consumer) creates
> > a SYNC_STATE_ONLY link to the supplier while we wait for the consumer
> > device to get added. This prevents sync_state() from being called too
> > early on the supplier.
> >
> > There are so many includes in the dts/dtsi files that my head is
> > spinning. I finally found out where the soc:firmware:gpio device was
> > coming from (after confusing myself with gpio@7e200000 for a bit) and
> > where fe201000.serial was coming from. I still couldn't figure out how
> > the address got mapped to fe201000 in fe201000.serial -- that
> > generally means the parent has some address offset, but I don't see
> > that in DT (but it is not important for this discussion, so we can
> > ignore that).
> This is uart0 which is at first defined in bcm283x.dtsi. On the
> Raspberry Pi 4 this UART is connected to the Bluetooth IC. On Linux
> probing of the serial communication via DT is done via serial device bus
> (see bcm283x-rpi-wifi-bt.dtsi).

Sigh... this is the connection I needed.

> > Anyway, I see no supplier-consumer link between serial@7e201000 (or
> > any of its zero descendants) and soc:firmware:gpio (which is the node
> > expgpio:). So I'm very confused why we might even try to create this
> > sync state only device link between these two.
> >
> > There are actually two times where we try to create such a link:
> >
> > First attempt that actually succeeds -- but I have no idea why we even do this:
> > [    0.100047] device:
> > 'platform:soc:firmware:gpio--amba:fe201000.serial': device_add
> > [    0.100232] amba fe201000.serial: Linked as a sync state only
> > consumer to soc:firmware:gpio
>
> I assume the link is established by raspberry,firmware-gpio which
> provides the necessary BT shutdown-gpios defined in bcm2711-rpi-4-b.dts.
> Seems to me that the problem is, that necessary underlying firmware
> driver is probed "too late":
>
> [ 15.456506] raspberrypi-firmware soc:firmware: Attached to firmware
> from 2020-02-12T12:36:21
>
> Hope this helps a little bit

Definitely! Thanks!

Florian, don't bother with the rest of my debug request. I think I
know what's going on. I'll come back if I need more help.

-Saravana

>
> > the "amba" prefix tells us a driver hasn't been bound to fe201000.serial yet.
> >
> > Second attempt is the one that fails.
> > [   15.516166] uart-pl011 fe201000.serial: Failed to create device
> > link (0x180) with soc:firmware:gpio
> > The uart-pl011 tells us that the driver has bound to fe201000.serial.
> >
> > And it fails because of this sensible check I had put up a while ago
> > inside device_link_add():
> >          /*
> >           * SYNC_STATE_ONLY links are useless once a consumer device
> > has probed.
> >           * So, only create it if the consumer hasn't probed yet.
> >           */
> >          if (flags & DL_FLAG_SYNC_STATE_ONLY &&
> >              consumer->links.status != DL_DEV_NO_DRIVER &&
> >              consumer->links.status != DL_DEV_PROBING) {
> >                  link = NULL;
> >                  goto out;
> >          }
> >
> > So the real question is still to figure out why fw_devlink is trying
> > to create this device link.
> >
> > So to debug this further the following would help a lot:
> > 1. Pull the dtb from the device and then decompile it and provide me
> > the result. This way, I can be sure I'm not missing something and
> > don't have to dig through all the includes -- I forgot the exact
> > commands for it, but it's been shared in LKML before. Let me know if
> > you need me to dig this up.
> >
> > 2. I think a stack backtrace when these two device link attempts are
> > made might help me figure out why they are being created. To get these
> > backtraces, can you do the following please?
> > a. Put a WARN_ON() inside device_add() for when the device name matches:
> > platform:soc:firmware:gpio--amba:fe201000.serial
> > b. Put a WARN_ON(1) where we print "Failed to create device link..."
> >
> > Feel free to dig more into 2a and 2b if you find that the stack trace
> > doesn't tell much -- one problem with the stack trace is that it
> > doesn't show the params being passed to
> > __fw_devlink_link_to_consumers() and __fw_devlink_link_to_suppliers()
> >
> > Thanks,
> > Saravana
> >
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel