Re: [PATCH v3] driver core: Fix SYNC_STATE_ONLY device link implementation

[Saravana Kannan <saravanak@google.com>]

On Mon, May 25, 2020 at 4:31 AM Michael Walle <michael@walle.cc> wrote:
>
> Am 2020-05-23 00:47, schrieb Michael Walle:
> > Am 2020-05-23 00:21, schrieb Saravana Kannan:
> >> On Fri, May 22, 2020 at 11:41 AM Michael Walle <michael@walle.cc>
> >> wrote:
> >>>
> >>> Am Mon, 18 May 2020 23:30:00 -0700
> >>> schrieb Saravana Kannan <saravanak@google.com>:
> >>>
> >>> > When SYNC_STATE_ONLY support was added in commit 05ef983e0d65 ("driver
> >>> > core: Add device link support for SYNC_STATE_ONLY flag"),
> >>> > device_link_add() incorrectly skipped adding the new SYNC_STATE_ONLY
> >>> > device link to the supplier's and consumer's "device link" list.
> >>> >
> >>> > This causes multiple issues:
> >>> > - The device link is lost forever from driver core if the caller
> >>> >   didn't keep track of it (caller typically isn't expected to). This
> >>> > is a memory leak.
> >>> > - The device link is also never visible to any other code path after
> >>> >   device_link_add() returns.
> >>> >
> >>> > If we fix the "device link" list handling, that exposes a bunch of
> >>> > issues.
> >>> >
> >>> > 1. The device link "status" state management code rightfully doesn't
> >>> > handle the case where a DL_FLAG_MANAGED device link exists between a
> >>> > supplier and consumer, but the consumer manages to probe successfully
> >>> > before the supplier. The addition of DL_FLAG_SYNC_STATE_ONLY links
> >>> > break this assumption. This causes device_links_driver_bound() to
> >>> > throw a warning when this happens.
> >>> >
> >>> > Since DL_FLAG_SYNC_STATE_ONLY device links are mainly used for
> >>> > creating proxy device links for child device dependencies and aren't
> >>> > useful once the consumer device probes successfully, this patch just
> >>> > deletes DL_FLAG_SYNC_STATE_ONLY device links once its consumer device
> >>> > probes. This way, we avoid the warning, free up some memory and avoid
> >>> > complicating the device links "status" state management code.
> >>> >
> >>> > 2. Creating a DL_FLAG_STATELESS device link between two devices that
> >>> > already have a DL_FLAG_SYNC_STATE_ONLY device link will result in the
> >>> > DL_FLAG_STATELESS flag not getting set correctly. This patch also
> >>> > fixes this.
> >>> >
> >>> > Lastly, this patch also fixes minor whitespace issues.
> >>>
> >>> My board triggers the
> >>>   WARN_ON(link->status != DL_STATE_CONSUMER_PROBE);
> >>>
> >>> Full bootlog:
> > [..]
> >
> >> Thanks for the log and report. I haven't spent too much time thinking
> >> about this, but can you give this a shot?
> >> https://lore.kernel.org/lkml/20200520043626.181820-1-saravanak@google.com/
> >
> > I've already tried that, as this is already in linux-next. Doesn't fix
> > it,
> > though.
>
> btw. this only happens on linux-next (tested with next-20200522), not on
> 5.7-rc7 (which has the same two patches of yours)

I wouldn't be surprised if the difference is due to
fw_devlink_pause/resume() calls in driver/of/property.c. It chops off
~1s in boot time by changing the order in which device links are
created from DT. So, I think it's just masking the issue.

On linux-next where you see the issue, can you get the logs with this change:
+++ b/drivers/base/core.c
@@ -907,7 +907,10 @@ void device_links_driver_bound(struct device *dev)
                         */
                        device_link_drop_managed(link);
                } else {
-                       WARN_ON(link->status != DL_STATE_CONSUMER_PROBE);
+                       WARN(link->status != DL_STATE_CONSUMER_PROBE,
+                               "sup:%s - con:%s f:%d s:%d\n",
+                               dev_name(supplier), dev_name(link->consumer),
+                               link->flags, link->status);
                        WRITE_ONCE(link->status, DL_STATE_ACTIVE);
                }

My goal is to figure out the order in which the device links between
the supplier and consumers devices are created and how that's changing
the flag and status. Then I can come up with a fix.

Thanks,
Saravana