[RFD] drivers: base: A driver's ->sync_state() callback don't get called

[RFD] drivers: base: A driver's ->sync_state() callback don't get called

[Ulf Hansson]

Hi Saravana, Rafael, Stephen,

I have observed an issue with a driver's  ->sync_state() callback that
doesn't get called, even if consumers/supplier device links conditions
should have been satisfied. I have narrowed down the problem, but I am
not sure what is the best solution to fix it, hence I am turning to
you for some discussion/advice.

I have two test platform drivers, one that matches on the
"test,pm-domains-test" compatible string (this driver also has the
->sync_state() callback assigned) and another driver that matches on
"test,runtime-pm-test".

This is the relevant part in my DTS file:

       pm_domain_test {
                compatible = "test,pm-domains-test";

                pdParent: power-domain-parent {
                        #power-domain-cells = <0>;
                };

                pdChild: power-domain-child {
                        #power-domain-cells = <0>;
                        power-domains = <&pdParent>;
                };
        };

        soctest: soctest {
                compatible = "simple-bus";

                rpmtest0 {
                        compatible = "test,runtime-pm-test";
                        power-domains = <&pdParent>;
                };
        };

During boot the fw_devlinks are being created and their corresponding
links. With some debug enabled this shows some of the interesting
parts that are being printed to the log:

[    0.041539] device: 'pm_domain_test': device_add
[    0.041629] OF: Not linking pm_domain_test to pm_domain_test - is descendant
[    0.041718] device: 'soctest': device_add
[    0.041803] OF: Linking rpmtest0 (consumer) to pm_domain_test (supplier)
[    0.041829] device: 'platform:pm_domain_test--platform:soctest': device_add
[    0.041892] platform soctest: Linked as a sync state only consumer
to pm_domain_test
[    0.041957] OF:    create child: /soctest/rpmtest0
[    0.041995] device: 'soctest:rpmtest0': device_add
[    0.042072] device:
'platform:pm_domain_test--platform:soctest:rpmtest0': device_add
[    0.042132] devices_kset: Moving soctest:rpmtest0 to end of list
[    0.042141] platform soctest:rpmtest0: Linked as a consumer to pm_domain_test

The interesting thing here is the "sync state only" link that gets
created. I assume there are good reasons for creating this link, even
if I fail to understand exactly why.

In any case, the sync state only link never gets dropped, which I
assume is because there is no driver getting bound for the "soctest"
device (it has only the "simple-bus" compatible).

In other words, it doesn't matter that both the rpmtest0 and the
pm_domain_test devices are probed, thus satisfying the
supplier/consumer conditions, the ->sync_state() callback doesn't get
called anyway.

Can you perhaps help to point me in a direction of how to best fix this problem?

Kind regards
Ulf Hansson

Re: [RFD] drivers: base: A driver's ->sync_state() callback don't get called

[Saravana Kannan]

On Tue, Aug 31, 2021 at 6:56 AM Ulf Hansson <ulf.hansson@linaro.org> wrote:
>
> Hi Saravana, Rafael, Stephen,
>
> I have observed an issue with a driver's  ->sync_state() callback that
> doesn't get called, even if consumers/supplier device links conditions
> should have been satisfied. I have narrowed down the problem, but I am
> not sure what is the best solution to fix it, hence I am turning to
> you for some discussion/advice.
>
> I have two test platform drivers, one that matches on the
> "test,pm-domains-test" compatible string (this driver also has the
> ->sync_state() callback assigned) and another driver that matches on
> "test,runtime-pm-test".
>
> This is the relevant part in my DTS file:
>
>        pm_domain_test {
>                 compatible = "test,pm-domains-test";
>
>                 pdParent: power-domain-parent {
>                         #power-domain-cells = <0>;
>                 };
>
>                 pdChild: power-domain-child {
>                         #power-domain-cells = <0>;
>                         power-domains = <&pdParent>;
>                 };
>         };
>
>         soctest: soctest {
>                 compatible = "simple-bus";
>
>                 rpmtest0 {
>                         compatible = "test,runtime-pm-test";
>                         power-domains = <&pdParent>;
>                 };
>         };
>
> During boot the fw_devlinks are being created and their corresponding
> links. With some debug enabled this shows some of the interesting
> parts that are being printed to the log:
>
> [    0.041539] device: 'pm_domain_test': device_add
> [    0.041629] OF: Not linking pm_domain_test to pm_domain_test - is descendant
> [    0.041718] device: 'soctest': device_add
> [    0.041803] OF: Linking rpmtest0 (consumer) to pm_domain_test (supplier)
> [    0.041829] device: 'platform:pm_domain_test--platform:soctest': device_add
> [    0.041892] platform soctest: Linked as a sync state only consumer
> to pm_domain_test
> [    0.041957] OF:    create child: /soctest/rpmtest0
> [    0.041995] device: 'soctest:rpmtest0': device_add
> [    0.042072] device:
> 'platform:pm_domain_test--platform:soctest:rpmtest0': device_add
> [    0.042132] devices_kset: Moving soctest:rpmtest0 to end of list
> [    0.042141] platform soctest:rpmtest0: Linked as a consumer to pm_domain_test
>
> The interesting thing here is the "sync state only" link that gets
> created. I assume there are good reasons for creating this link, even
> if I fail to understand exactly why.

In general there's a good reason for creating these links from parent
devices of the consumer to the supplier. It is documented in the code
under __fw_devlink_link_to_consumers().

/*
 * If consumer device is not available yet, make a "proxy"
 * SYNC_STATE_ONLY link from the consumer's parent device to
 * the supplier device. This is necessary to make sure the
 * supplier doesn't get a sync_state() callback before the real
 * consumer can create a device link to the supplier.
 *
 * This proxy link step is needed to handle the case where the
 * consumer's parent device is added before the supplier.
 */

and under __fw_devlink_link_to_suppliers().

/*
 * Make "proxy" SYNC_STATE_ONLY device links to represent the needs of
 * all the descendants. This proxy link step is needed to handle the
 * case where the supplier is added before the consumer's parent device
 * (@dev).
 */


>
> In any case, the sync state only link never gets dropped, which I
> assume is because there is no driver getting bound for the "soctest"
> device (it has only the "simple-bus" compatible).

Yeah, you've identified the problem correctly. I've been thinking
about this possibility (and all the side effects a fix might have). I
can send out a fix for this soon (within a week or so).

> In other words, it doesn't matter that both the rpmtest0 and the
> pm_domain_test devices are probed, thus satisfying the
> supplier/consumer conditions, the ->sync_state() callback doesn't get
> called anyway.
>
> Can you perhaps help to point me in a direction of how to best fix this problem?

I hope you are okay with me sending a fix.


-Saravana

Re: [RFD] drivers: base: A driver's ->sync_state() callback don't get called

[Saravana Kannan]

On Tue, Aug 31, 2021 at 9:59 AM Saravana Kannan <saravanak@google.com> wrote:
>
> On Tue, Aug 31, 2021 at 6:56 AM Ulf Hansson <ulf.hansson@linaro.org> wrote:
> >
> > Hi Saravana, Rafael, Stephen,
> >
> > I have observed an issue with a driver's  ->sync_state() callback that
> > doesn't get called, even if consumers/supplier device links conditions
> > should have been satisfied. I have narrowed down the problem, but I am
> > not sure what is the best solution to fix it, hence I am turning to
> > you for some discussion/advice.
> >
> > I have two test platform drivers, one that matches on the
> > "test,pm-domains-test" compatible string (this driver also has the
> > ->sync_state() callback assigned) and another driver that matches on
> > "test,runtime-pm-test".
> >
> > This is the relevant part in my DTS file:
> >
> >        pm_domain_test {
> >                 compatible = "test,pm-domains-test";
> >
> >                 pdParent: power-domain-parent {
> >                         #power-domain-cells = <0>;
> >                 };
> >
> >                 pdChild: power-domain-child {
> >                         #power-domain-cells = <0>;
> >                         power-domains = <&pdParent>;
> >                 };
> >         };
> >
> >         soctest: soctest {
> >                 compatible = "simple-bus";
> >
> >                 rpmtest0 {
> >                         compatible = "test,runtime-pm-test";
> >                         power-domains = <&pdParent>;
> >                 };
> >         };
> >
> > During boot the fw_devlinks are being created and their corresponding
> > links. With some debug enabled this shows some of the interesting
> > parts that are being printed to the log:
> >
> > [    0.041539] device: 'pm_domain_test': device_add
> > [    0.041629] OF: Not linking pm_domain_test to pm_domain_test - is descendant
> > [    0.041718] device: 'soctest': device_add
> > [    0.041803] OF: Linking rpmtest0 (consumer) to pm_domain_test (supplier)
> > [    0.041829] device: 'platform:pm_domain_test--platform:soctest': device_add
> > [    0.041892] platform soctest: Linked as a sync state only consumer
> > to pm_domain_test
> > [    0.041957] OF:    create child: /soctest/rpmtest0
> > [    0.041995] device: 'soctest:rpmtest0': device_add
> > [    0.042072] device:
> > 'platform:pm_domain_test--platform:soctest:rpmtest0': device_add
> > [    0.042132] devices_kset: Moving soctest:rpmtest0 to end of list
> > [    0.042141] platform soctest:rpmtest0: Linked as a consumer to pm_domain_test
> >
> > The interesting thing here is the "sync state only" link that gets
> > created. I assume there are good reasons for creating this link, even
> > if I fail to understand exactly why.
>
> In general there's a good reason for creating these links from parent
> devices of the consumer to the supplier. It is documented in the code
> under __fw_devlink_link_to_consumers().
>
> /*
>  * If consumer device is not available yet, make a "proxy"
>  * SYNC_STATE_ONLY link from the consumer's parent device to
>  * the supplier device. This is necessary to make sure the
>  * supplier doesn't get a sync_state() callback before the real
>  * consumer can create a device link to the supplier.
>  *
>  * This proxy link step is needed to handle the case where the
>  * consumer's parent device is added before the supplier.
>  */
>
> and under __fw_devlink_link_to_suppliers().
>
> /*
>  * Make "proxy" SYNC_STATE_ONLY device links to represent the needs of
>  * all the descendants. This proxy link step is needed to handle the
>  * case where the supplier is added before the consumer's parent device
>  * (@dev).
>  */
>
>
> >
> > In any case, the sync state only link never gets dropped, which I
> > assume is because there is no driver getting bound for the "soctest"
> > device (it has only the "simple-bus" compatible).
>
> Yeah, you've identified the problem correctly. I've been thinking
> about this possibility (and all the side effects a fix might have). I
> can send out a fix for this soon (within a week or so).
>
> > In other words, it doesn't matter that both the rpmtest0 and the
> > pm_domain_test devices are probed, thus satisfying the
> > supplier/consumer conditions, the ->sync_state() callback doesn't get
> > called anyway.
> >
> > Can you perhaps help to point me in a direction of how to best fix this problem?
>
> I hope you are okay with me sending a fix.

Ulf,

Could you give this a shot please?

https://lore.kernel.org/lkml/20210831224510.703253-1-saravanak@google.com/T/#u

-Saravana