From: Saravana Kannan <saravanak@google.com>
To: Abel Vesa <abel.vesa@linaro.org>
Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>,
"Rafael J. Wysocki" <rafael@kernel.org>,
Len Brown <lenb@kernel.org>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Ard Biesheuvel <ardb@kernel.org>,
Rob Herring <robh+dt@kernel.org>,
Frank Rowand <frowand.list@gmail.com>,
Marc Zyngier <maz@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
Tomi Valkeinen <tomi.valkeinen@ti.com>,
Laurent Pinchart <laurent.pinchart@ideasonboard.com>,
Grygorii Strashko <grygorii.strashko@ti.com>,
kernel-team@android.com, linux-acpi@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org,
devicetree@vger.kernel.org,
Bjorn Andersson <bjorn.andersson@linaro.org>,
Dmitry Baryshkov <dmitry.baryshkov@linaro.org>,
linux-arm-msm@vger.kernel.org
Subject: Re: [PATCH v2 13/17] driver core: Use device's fwnode to check if it is waiting for suppliers
Date: Tue, 28 Jun 2022 11:09:17 -0700 [thread overview]
Message-ID: <CAGETcx8F0wP+RA0KpjOJeZfc=DVG-MbM_=SkRHD4UhD2ReL7Kw@mail.gmail.com> (raw)
In-Reply-To: <YrskVLshWeps+NXw@linaro.org>
On Tue, Jun 28, 2022 at 8:55 AM Abel Vesa <abel.vesa@linaro.org> wrote:
>
> On 22-06-27 15:30:25, Saravana Kannan wrote:
> > On Mon, Jun 27, 2022 at 4:42 AM Abel Vesa <abel.vesa@linaro.org> wrote:
> > >
> > > On 20-11-20 18:02:28, Saravana Kannan wrote:
> > > > To check if a device is still waiting for its supplier devices to be
> > > > added, we used to check if the devices is in a global
> > > > waiting_for_suppliers list. Since the global list will be deleted in
> > > > subsequent patches, this patch stops using this check.
> > > >
> > > > Instead, this patch uses a more device specific check. It checks if the
> > > > device's fwnode has any fwnode links that haven't been converted to
> > > > device links yet.
> > > >
> > > > Signed-off-by: Saravana Kannan <saravanak@google.com>
> > > > ---
> > > > drivers/base/core.c | 18 ++++++++----------
> > > > 1 file changed, 8 insertions(+), 10 deletions(-)
> > > >
> > > > diff --git a/drivers/base/core.c b/drivers/base/core.c
> > > > index 395dece1c83a..1873cecb0cc4 100644
> > > > --- a/drivers/base/core.c
> > > > +++ b/drivers/base/core.c
> > > > @@ -51,6 +51,7 @@ static DEFINE_MUTEX(wfs_lock);
> > > > static LIST_HEAD(deferred_sync);
> > > > static unsigned int defer_sync_state_count = 1;
> > > > static DEFINE_MUTEX(fwnode_link_lock);
> > > > +static bool fw_devlink_is_permissive(void);
> > > >
> > > > /**
> > > > * fwnode_link_add - Create a link between two fwnode_handles.
> > > > @@ -995,13 +996,13 @@ int device_links_check_suppliers(struct device *dev)
> > > > * Device waiting for supplier to become available is not allowed to
> > > > * probe.
> > > > */
> > > > - mutex_lock(&wfs_lock);
> > > > - if (!list_empty(&dev->links.needs_suppliers) &&
> > > > - dev->links.need_for_probe) {
> > > > - mutex_unlock(&wfs_lock);
> > > > + mutex_lock(&fwnode_link_lock);
> > > > + if (dev->fwnode && !list_empty(&dev->fwnode->suppliers) &&
> > > > + !fw_devlink_is_permissive()) {
> > > > + mutex_unlock(&fwnode_link_lock);
> > >
> > > Hi Saravana,
> > >
> > > First of, sorry for going back to this.
> >
> > No worries at all. If there's an issue with fw_devlink, I want to have it fixed.
> >
> > > There is a scenario where this check will not work and probably should
> > > work. It goes like this:
> > >
> > > A clock controller is not allowed to probe because it uses a clock from a child device of a
> > > consumer, like so:
> > >
> > > dispcc: clock-controller@af00000 {
> > > clocks = <&dsi0_phy 0>;
> > > };
> > >
> > > mdss: mdss@ae00000 {
> > > clocks = <&dispcc DISP_CC_MDSS_MDP_CLK>;
> > >
> > > dsi0_phy: dsi-phy@ae94400 {
> > > clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
> > > };
> > > };
> > >
> > > This is a real scenario actually, but I stripped it down to the essentials.
> >
> > I'm well aware of this scenario and explicitly wrote code to address this :)
> >
>
> Actually, the problem seems to be when you have two dsi phys.
> Like so:
>
> dispcc: clock-controller@af00000 {
> clocks = <&dsi0_phy 0>;
> clocks = <&dsi1_phy 0>;
> };
>
> mdss: mdss@ae00000 {
> clocks = <&dispcc DISP_CC_MDSS_MDP_CLK>;
>
> dsi0_phy: dsi-phy@ae94400 {
> clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
> };
>
> dsi1_phy: dsi-phy@ae64400 {
> clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
> };
> };
>
> And from what I've seen happening so far is that the device_is_dependent
> check for the parent of the supplier (if it also a consumer) seems to return
> false on second pass of the same link due to the DL_FLAG_SYNC_STATE_ONLY
> being set this time around.
>
> > See this comment in fw_devlink_create_devlink()
> >
> > /*
> > * If we can't find the supplier device from its fwnode, it might be
> > * due to a cyclic dependency between fwnodes. Some of these cycles can
> > * be broken by applying logic. Check for these types of cycles and
> > * break them so that devices in the cycle probe properly.
> > *
> > * If the supplier's parent is dependent on the consumer, then the
> > * consumer and supplier have a cyclic dependency. Since fw_devlink
> > * can't tell which of the inferred dependencies are incorrect, don't
> > * enforce probe ordering between any of the devices in this cyclic
> > * dependency. Do this by relaxing all the fw_devlink device links in
> > * this cycle and by treating the fwnode link between the consumer and
> > * the supplier as an invalid dependency.
> > */
> >
>
> So when this thing you mentioned above is happening for the second dsi
> phy (order doesn't matter), since the dsi phy itself cannot be found,
> the device_is_dependent is run for the same link: dispcc -> mdss
> (supplier -> consumer), but again, since it has the
> DL_FLAG_SYNC_STATE_ONLY this time around, it will skip that specific
> link.
Ugh... I knew there was this gap, but didn't expect it to be a real world issue.
There are different ways of addressing this and they all fall
somewhere within a spectrum of:
"stop enforcing very specific edges of the dependency graph when you
find a cycles"
To
"just don't enforce any dependency for devices in a cycle and let the
drivers figure out when to -EPROBE_DEFER".
And each of those are of varying complexity. Ideally I'd prefer to
relax specific edges, but I need to balance it out with the code
complexity. Let me soak this for a few weeks to decide on what option
to take.
Thanks for the report.
-Saravana
>
> > Applying this comment to your example, dispcc is the "consumer",
> > dsi0_phy is the "supplier" and mdss is the "supplier's parent".
> >
> > And because we can't guarantee the order of addition of these top
> > level devices is why I also have this piece of recursive call inside
> > __fw_devlink_link_to_suppliers():
> >
> > /*
> > * If a device link was successfully created to a supplier, we
> > * now need to try and link the supplier to all its suppliers.
> > *
> > * This is needed to detect and delete false dependencies in
> > * fwnode links that haven't been converted to a device link
> > * yet. See comments in fw_devlink_create_devlink() for more
> > * details on the false dependency.
> > *
> > * Without deleting these false dependencies, some devices will
> > * never probe because they'll keep waiting for their false
> > * dependency fwnode links to be converted to device links.
> > */
> > sup_dev = get_dev_from_fwnode(sup);
> > __fw_devlink_link_to_suppliers(sup_dev, sup_dev->fwnode);
> > put_device(sup_dev);
> >
> > So when mdss gets added, we'll link it to dispcc and then check if
> > dispcc has any suppliers it needs to link to. And that's when the
> > logic will catch the cycle and fix it.
> >
> > Can you tell me why this wouldn't unblock the probing of dispcc? Are
> > you actually hitting this on a device? If so, can you please check why
> > this logic isn't sufficient to catch and undo the cycle?
> >
>
> This is happening on Qualcomm SDM845 with Linus's tree.
>
> > Thanks,
> > Saravana
> >
> > > So, the dsi0_phy will be "device_add'ed" (through of_platform_populate) by the mdss probe.
> > > The mdss will probe defer waiting for the DISP_CC_MDSS_MDP_CLK, while
> > > the dispcc will probe defer waiting for the dsi0_phy (supplier).
> > >
> > > Basically, this 'supplier availability check' does not work when a supplier might
> > > be populated by a consumer of the device that is currently trying to probe.
> > >
> > >
> > > Abel
> > >
> > >
> > > > return -EPROBE_DEFER;
> > > > }
> > > > - mutex_unlock(&wfs_lock);
> > > > + mutex_unlock(&fwnode_link_lock);
> > > >
> > > > device_links_write_lock();
> > > >
> > > > @@ -1167,10 +1168,7 @@ static ssize_t waiting_for_supplier_show(struct device *dev,
> > > > bool val;
> > > >
> > > > device_lock(dev);
> > > > - mutex_lock(&wfs_lock);
> > > > - val = !list_empty(&dev->links.needs_suppliers)
> > > > - && dev->links.need_for_probe;
> > > > - mutex_unlock(&wfs_lock);
> > > > + val = !list_empty(&dev->fwnode->suppliers);
> > > > device_unlock(dev);
> > > > return sysfs_emit(buf, "%u\n", val);
> > > > }
> > > > @@ -2202,7 +2200,7 @@ static int device_add_attrs(struct device *dev)
> > > > goto err_remove_dev_groups;
> > > > }
> > > >
> > > > - if (fw_devlink_flags && !fw_devlink_is_permissive()) {
> > > > + if (fw_devlink_flags && !fw_devlink_is_permissive() && dev->fwnode) {
> > > > error = device_create_file(dev, &dev_attr_waiting_for_supplier);
> > > > if (error)
> > > > goto err_remove_dev_online;
> > > > --
> > > > 2.29.2.454.gaff20da3a2-goog
> > > >
> > > >
> >
next prev parent reply other threads:[~2022-06-28 18:09 UTC|newest]
Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-21 2:02 [PATCH v2 00/17] Refactor fw_devlink to significantly improve boot time Saravana Kannan
2020-11-21 2:02 ` [PATCH v2 01/17] Revert "driver core: Avoid deferred probe due to fw_devlink_pause/resume()" Saravana Kannan
2020-11-21 2:02 ` [PATCH v2 02/17] Revert "driver core: Rename dev_links_info.defer_sync to defer_hook" Saravana Kannan
2020-11-21 2:02 ` [PATCH v2 03/17] Revert "driver core: Don't do deferred probe in parallel with kernel_init thread" Saravana Kannan
2020-11-21 2:02 ` [PATCH v2 04/17] Revert "driver core: Remove check in driver_deferred_probe_force_trigger()" Saravana Kannan
2020-11-21 2:02 ` [PATCH v2 05/17] Revert "of: platform: Batch fwnode parsing when adding all top level devices" Saravana Kannan
2020-12-07 22:18 ` Rob Herring
2020-11-21 2:02 ` [PATCH v2 06/17] Revert "driver core: fw_devlink: Add support for batching fwnode parsing" Saravana Kannan
2020-11-21 2:02 ` [PATCH v2 07/17] driver core: Add fwnode_init() Saravana Kannan
2020-12-06 7:26 ` Leon Romanovsky
2020-12-07 19:25 ` Saravana Kannan
2020-12-07 19:53 ` Leon Romanovsky
2020-12-07 20:36 ` Saravana Kannan
2020-12-08 6:34 ` Leon Romanovsky
2020-12-07 22:20 ` Rob Herring
2020-11-21 2:02 ` [PATCH v2 08/17] driver core: Add fwnode link support Saravana Kannan
2020-12-06 7:48 ` Leon Romanovsky
2020-12-07 19:25 ` Saravana Kannan
2020-12-07 19:56 ` Leon Romanovsky
2020-12-07 22:21 ` Rob Herring
2020-11-21 2:02 ` [PATCH v2 09/17] driver core: Allow only unprobed consumers for SYNC_STATE_ONLY device links Saravana Kannan
2020-11-21 2:02 ` [PATCH v2 10/17] device property: Add fwnode_is_ancestor_of() and fwnode_get_next_parent_dev() Saravana Kannan
2020-11-21 2:02 ` [PATCH v2 11/17] driver core: Redefine the meaning of fwnode_operations.add_links() Saravana Kannan
2020-11-21 2:02 ` [PATCH v2 12/17] driver core: Add fw_devlink_parse_fwtree() Saravana Kannan
2020-11-21 2:02 ` [PATCH v2 13/17] driver core: Use device's fwnode to check if it is waiting for suppliers Saravana Kannan
2022-06-27 11:42 ` Abel Vesa
2022-06-27 22:30 ` Saravana Kannan
2022-06-28 15:24 ` Abel Vesa
2022-06-28 15:44 ` Abel Vesa
2022-06-28 15:55 ` Abel Vesa
2022-06-28 18:09 ` Saravana Kannan [this message]
2023-01-05 14:47 ` Dmitry Baryshkov
2020-11-21 2:02 ` [PATCH v2 14/17] of: property: Update implementation of add_links() to create fwnode links Saravana Kannan
2020-12-07 22:37 ` Rob Herring
2020-11-21 2:02 ` [PATCH v2 15/17] efi: " Saravana Kannan
2020-11-21 2:02 ` [PATCH v2 16/17] driver core: Refactor fw_devlink feature Saravana Kannan
2020-11-23 16:00 ` [driver core] 95f755a4ef: will-it-scale.per_process_ops 2.2% improvement kernel test robot
2020-12-11 14:11 ` [PATCH v2 16/17] driver core: Refactor fw_devlink feature Qian Cai
2020-12-11 16:34 ` Robin Murphy
2020-12-11 17:51 ` Saravana Kannan
2020-12-11 18:03 ` Marc Zyngier
2020-12-11 18:20 ` Saravana Kannan
2020-12-11 19:07 ` Marc Zyngier
2020-12-11 22:29 ` Saravana Kannan
2020-12-29 3:34 ` Michael Walle
2021-01-05 19:00 ` Saravana Kannan
2021-01-05 21:03 ` Michael Walle
2021-01-06 23:29 ` Saravana Kannan
2020-11-21 2:02 ` [PATCH v2 17/17] driver core: Delete pointless parameter in fwnode_operations.add_links Saravana Kannan
2020-12-07 22:38 ` Rob Herring
2020-11-24 8:29 ` [PATCH v2 00/17] Refactor fw_devlink to significantly improve boot time Tomi Valkeinen
2020-11-24 17:25 ` Saravana Kannan
2020-12-03 19:05 ` Saravana Kannan
2020-12-09 18:16 ` Greg Kroah-Hartman
2020-12-09 20:24 ` Saravana Kannan
2020-12-10 9:26 ` Greg Kroah-Hartman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAGETcx8F0wP+RA0KpjOJeZfc=DVG-MbM_=SkRHD4UhD2ReL7Kw@mail.gmail.com' \
--to=saravanak@google.com \
--cc=abel.vesa@linaro.org \
--cc=ardb@kernel.org \
--cc=bjorn.andersson@linaro.org \
--cc=devicetree@vger.kernel.org \
--cc=dmitry.baryshkov@linaro.org \
--cc=frowand.list@gmail.com \
--cc=gregkh@linuxfoundation.org \
--cc=grygorii.strashko@ti.com \
--cc=kernel-team@android.com \
--cc=laurent.pinchart@ideasonboard.com \
--cc=lenb@kernel.org \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-efi@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=maz@kernel.org \
--cc=rafael@kernel.org \
--cc=rjw@rjwysocki.net \
--cc=robh+dt@kernel.org \
--cc=tglx@linutronix.de \
--cc=tomi.valkeinen@ti.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).