From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8816AC43334 for ; Tue, 28 Jun 2022 15:55:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348055AbiF1PzK (ORCPT ); Tue, 28 Jun 2022 11:55:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348027AbiF1PzH (ORCPT ); Tue, 28 Jun 2022 11:55:07 -0400 Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD7B42EA08 for ; Tue, 28 Jun 2022 08:55:04 -0700 (PDT) Received: by mail-ed1-x52f.google.com with SMTP id ej4so18218527edb.7 for ; Tue, 28 Jun 2022 08:55:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=DL6UdLR7FEJazkCj2FBmbmojXNOORr6ZP3bxU8f4s34=; b=l0dA5NX94rb9ZtgdVBBElJ7NSfh1y1w9K5EBLTCi6RyVbe81X5TmiiqC/bsBYdeHsA VKjaosDs3VClNI0MJyP4z+CekQuEBc7r1lgfi17LJUxp8uxAVRpsPNfhD0jkS2wRPhFD g70V1tChV3KlHnnkaVw13E/AzJpuuwwrcY4PoY+s+cGG+9M3lh9BSTUGVxX++ofdUUab QmLAnTgAGAjzn6/yiCkJUnxWgTYdg45D2Uh7N+TzXuGTGqhwbGOKE5GnhXSJJ/42TTsK dzSkP9Z6qw2a+vvUPHveUIfSQl3kJWIfYvKHGSmqsVHF3Tqg3qgH6AghHVOmWXscDGcF n5ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=DL6UdLR7FEJazkCj2FBmbmojXNOORr6ZP3bxU8f4s34=; b=AVFhSp3EN5YzYaofv6k5TaUd3oUC/LfP5lNV3uEmWFnuEcHRHLR2lPpVsPsyG3z2m2 APs1QG8iIdtxg0zLFJlq49Sn7fvbSCDJFMjfOSG1jCnwUlLA4GwCDAs1nTXVdjn1T1+/ cYfTpTUSmoB3F2Ron7cME3SaYwjSRL1lIiNsziTf9wtkfJ5xocxX1bHuOi4SWYM3V39o Lz5LtTWILXWBorOdGiiD4/qa6AHYwrzP0guNnGm+obr/McmfsC+GgMZNIfuwNXSGwMn4 sSqZMB010Dnif0xsWyk/zQxYq7s47q4HrBrWHTV9URcKb4KqdpNoY3TiK9WFQP7bzprU v7TQ== X-Gm-Message-State: AJIora/P4h3uIfn7BgJf8CCiuunUCxNaaReU3TsC9SIuqn+l6tS4Bw2j r3LQKCTEWQgmQZ6HDxiSKHuC6TGyMw2I8tMy X-Google-Smtp-Source: AGRyM1u9rro4xRv9FJA6pS1u5agZm7Yo8S41wqUnC+qxUtvszIakDl3IlWWUuftH/pjc1KRB2WmOhg== X-Received: by 2002:a05:6402:328f:b0:435:7f7a:7666 with SMTP id f15-20020a056402328f00b004357f7a7666mr23627336eda.168.1656431703419; Tue, 28 Jun 2022 08:55:03 -0700 (PDT) Received: from linaro.org ([94.52.112.99]) by smtp.gmail.com with ESMTPSA id y1-20020aa7c241000000b004355dc75066sm9726089edo.86.2022.06.28.08.55.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jun 2022 08:55:02 -0700 (PDT) Date: Tue, 28 Jun 2022 18:55:00 +0300 From: Abel Vesa To: Saravana Kannan Cc: "Rafael J. Wysocki" , "Rafael J. Wysocki" , Len Brown , Greg Kroah-Hartman , Ard Biesheuvel , Rob Herring , Frank Rowand , Marc Zyngier , Thomas Gleixner , Tomi Valkeinen , Laurent Pinchart , Grygorii Strashko , kernel-team@android.com, linux-acpi@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, devicetree@vger.kernel.org, Bjorn Andersson , Dmitry Baryshkov , linux-arm-msm@vger.kernel.org Subject: Re: [PATCH v2 13/17] driver core: Use device's fwnode to check if it is waiting for suppliers Message-ID: References: <20201121020232.908850-1-saravanak@google.com> <20201121020232.908850-14-saravanak@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 22-06-27 15:30:25, Saravana Kannan wrote: > On Mon, Jun 27, 2022 at 4:42 AM Abel Vesa wrote: > > > > On 20-11-20 18:02:28, Saravana Kannan wrote: > > > To check if a device is still waiting for its supplier devices to be > > > added, we used to check if the devices is in a global > > > waiting_for_suppliers list. Since the global list will be deleted in > > > subsequent patches, this patch stops using this check. > > > > > > Instead, this patch uses a more device specific check. It checks if the > > > device's fwnode has any fwnode links that haven't been converted to > > > device links yet. > > > > > > Signed-off-by: Saravana Kannan > > > --- > > > drivers/base/core.c | 18 ++++++++---------- > > > 1 file changed, 8 insertions(+), 10 deletions(-) > > > > > > diff --git a/drivers/base/core.c b/drivers/base/core.c > > > index 395dece1c83a..1873cecb0cc4 100644 > > > --- a/drivers/base/core.c > > > +++ b/drivers/base/core.c > > > @@ -51,6 +51,7 @@ static DEFINE_MUTEX(wfs_lock); > > > static LIST_HEAD(deferred_sync); > > > static unsigned int defer_sync_state_count = 1; > > > static DEFINE_MUTEX(fwnode_link_lock); > > > +static bool fw_devlink_is_permissive(void); > > > > > > /** > > > * fwnode_link_add - Create a link between two fwnode_handles. > > > @@ -995,13 +996,13 @@ int device_links_check_suppliers(struct device *dev) > > > * Device waiting for supplier to become available is not allowed to > > > * probe. > > > */ > > > - mutex_lock(&wfs_lock); > > > - if (!list_empty(&dev->links.needs_suppliers) && > > > - dev->links.need_for_probe) { > > > - mutex_unlock(&wfs_lock); > > > + mutex_lock(&fwnode_link_lock); > > > + if (dev->fwnode && !list_empty(&dev->fwnode->suppliers) && > > > + !fw_devlink_is_permissive()) { > > > + mutex_unlock(&fwnode_link_lock); > > > > Hi Saravana, > > > > First of, sorry for going back to this. > > No worries at all. If there's an issue with fw_devlink, I want to have it fixed. > > > There is a scenario where this check will not work and probably should > > work. It goes like this: > > > > A clock controller is not allowed to probe because it uses a clock from a child device of a > > consumer, like so: > > > > dispcc: clock-controller@af00000 { > > clocks = <&dsi0_phy 0>; > > }; > > > > mdss: mdss@ae00000 { > > clocks = <&dispcc DISP_CC_MDSS_MDP_CLK>; > > > > dsi0_phy: dsi-phy@ae94400 { > > clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>, > > }; > > }; > > > > This is a real scenario actually, but I stripped it down to the essentials. > > I'm well aware of this scenario and explicitly wrote code to address this :) > Actually, the problem seems to be when you have two dsi phys. Like so: dispcc: clock-controller@af00000 { clocks = <&dsi0_phy 0>; clocks = <&dsi1_phy 0>; }; mdss: mdss@ae00000 { clocks = <&dispcc DISP_CC_MDSS_MDP_CLK>; dsi0_phy: dsi-phy@ae94400 { clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>, }; dsi1_phy: dsi-phy@ae64400 { clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>, }; }; And from what I've seen happening so far is that the device_is_dependent check for the parent of the supplier (if it also a consumer) seems to return false on second pass of the same link due to the DL_FLAG_SYNC_STATE_ONLY being set this time around. > See this comment in fw_devlink_create_devlink() > > /* > * If we can't find the supplier device from its fwnode, it might be > * due to a cyclic dependency between fwnodes. Some of these cycles can > * be broken by applying logic. Check for these types of cycles and > * break them so that devices in the cycle probe properly. > * > * If the supplier's parent is dependent on the consumer, then the > * consumer and supplier have a cyclic dependency. Since fw_devlink > * can't tell which of the inferred dependencies are incorrect, don't > * enforce probe ordering between any of the devices in this cyclic > * dependency. Do this by relaxing all the fw_devlink device links in > * this cycle and by treating the fwnode link between the consumer and > * the supplier as an invalid dependency. > */ > So when this thing you mentioned above is happening for the second dsi phy (order doesn't matter), since the dsi phy itself cannot be found, the device_is_dependent is run for the same link: dispcc -> mdss (supplier -> consumer), but again, since it has the DL_FLAG_SYNC_STATE_ONLY this time around, it will skip that specific link. > Applying this comment to your example, dispcc is the "consumer", > dsi0_phy is the "supplier" and mdss is the "supplier's parent". > > And because we can't guarantee the order of addition of these top > level devices is why I also have this piece of recursive call inside > __fw_devlink_link_to_suppliers(): > > /* > * If a device link was successfully created to a supplier, we > * now need to try and link the supplier to all its suppliers. > * > * This is needed to detect and delete false dependencies in > * fwnode links that haven't been converted to a device link > * yet. See comments in fw_devlink_create_devlink() for more > * details on the false dependency. > * > * Without deleting these false dependencies, some devices will > * never probe because they'll keep waiting for their false > * dependency fwnode links to be converted to device links. > */ > sup_dev = get_dev_from_fwnode(sup); > __fw_devlink_link_to_suppliers(sup_dev, sup_dev->fwnode); > put_device(sup_dev); > > So when mdss gets added, we'll link it to dispcc and then check if > dispcc has any suppliers it needs to link to. And that's when the > logic will catch the cycle and fix it. > > Can you tell me why this wouldn't unblock the probing of dispcc? Are > you actually hitting this on a device? If so, can you please check why > this logic isn't sufficient to catch and undo the cycle? > This is happening on Qualcomm SDM845 with Linus's tree. > Thanks, > Saravana > > > So, the dsi0_phy will be "device_add'ed" (through of_platform_populate) by the mdss probe. > > The mdss will probe defer waiting for the DISP_CC_MDSS_MDP_CLK, while > > the dispcc will probe defer waiting for the dsi0_phy (supplier). > > > > Basically, this 'supplier availability check' does not work when a supplier might > > be populated by a consumer of the device that is currently trying to probe. > > > > > > Abel > > > > > > > return -EPROBE_DEFER; > > > } > > > - mutex_unlock(&wfs_lock); > > > + mutex_unlock(&fwnode_link_lock); > > > > > > device_links_write_lock(); > > > > > > @@ -1167,10 +1168,7 @@ static ssize_t waiting_for_supplier_show(struct device *dev, > > > bool val; > > > > > > device_lock(dev); > > > - mutex_lock(&wfs_lock); > > > - val = !list_empty(&dev->links.needs_suppliers) > > > - && dev->links.need_for_probe; > > > - mutex_unlock(&wfs_lock); > > > + val = !list_empty(&dev->fwnode->suppliers); > > > device_unlock(dev); > > > return sysfs_emit(buf, "%u\n", val); > > > } > > > @@ -2202,7 +2200,7 @@ static int device_add_attrs(struct device *dev) > > > goto err_remove_dev_groups; > > > } > > > > > > - if (fw_devlink_flags && !fw_devlink_is_permissive()) { > > > + if (fw_devlink_flags && !fw_devlink_is_permissive() && dev->fwnode) { > > > error = device_create_file(dev, &dev_attr_waiting_for_supplier); > > > if (error) > > > goto err_remove_dev_online; > > > -- > > > 2.29.2.454.gaff20da3a2-goog > > > > > > >