[PATCH] dd: Invoke one probe retry cycle after every initcall level

[PATCH] dd: Invoke one probe retry cycle after every initcall level

[Rishabh Bhatnagar]

Drivers that are registered at an initcall level may have to
wait until late_init before the probe deferral mechanism can
retry their probe functions. It is possible that their
dependencies were resolved much earlier, in some cases even
before the next initcall level. Invoke one probe retry cycle
at every _sync initcall level, allowing these drivers to be
probed earlier.

Signed-off-by: Vikram Mulukutla <markivx@codeaurora.org>
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
---
 drivers/base/dd.c | 33 +++++++++++++++++++++++++++------
 1 file changed, 27 insertions(+), 6 deletions(-)

diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index 1435d72..e6a6821 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -224,23 +224,44 @@ void device_unblock_probing(void)
 	driver_deferred_probe_trigger();
 }
 
+static void enable_trigger_defer_cycle(void)
+{
+	driver_deferred_probe_enable = true;
+	driver_deferred_probe_trigger();
+	/*
+	 * Sort as many dependencies as possible before the next initcall
+	 * level
+	 */
+	flush_work(&deferred_probe_work);
+}
+
 /**
  * deferred_probe_initcall() - Enable probing of deferred devices
  *
  * We don't want to get in the way when the bulk of drivers are getting probed.
  * Instead, this initcall makes sure that deferred probing is delayed until
- * late_initcall time.
+ * all the registered initcall functions at a particular level are completed.
+ * This function is invoked at every *_initcall_sync level.
  */
 static int deferred_probe_initcall(void)
 {
-	driver_deferred_probe_enable = true;
-	driver_deferred_probe_trigger();
-	/* Sort as many dependencies as possible before exiting initcalls */
-	flush_work(&deferred_probe_work);
+	enable_trigger_defer_cycle();
+	driver_deferred_probe_enable = false;
+	return 0;
+}
+arch_initcall_sync(deferred_probe_initcall);
+subsys_initcall_sync(deferred_probe_initcall);
+fs_initcall_sync(deferred_probe_initcall);
+device_initcall_sync(deferred_probe_initcall);
+
+static int deferred_probe_enable_fn(void)
+{
+	/* Enable deferred probing for all time */
+	enable_trigger_defer_cycle();
 	initcalls_done = true;
 	return 0;
 }
-late_initcall(deferred_probe_initcall);
+late_initcall(deferred_probe_enable_fn);
 
 /**
  * device_is_bound() - Check if device is bound to a driver
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [PATCH] dd: Invoke one probe retry cycle after every initcall level

[Rafael J. Wysocki]

On Thu, Jul 19, 2018 at 11:24 PM, Rishabh Bhatnagar
<rishabhb@codeaurora.org> wrote:
> Drivers that are registered at an initcall level may have to
> wait until late_init before the probe deferral mechanism can
> retry their probe functions. It is possible that their
> dependencies were resolved much earlier, in some cases even
> before the next initcall level. Invoke one probe retry cycle
> at every _sync initcall level, allowing these drivers to be
> probed earlier.

Can you please say something about the actual use case this is
expected to address?

> Signed-off-by: Vikram Mulukutla <markivx@codeaurora.org>
> Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
> ---
>  drivers/base/dd.c | 33 +++++++++++++++++++++++++++------
>  1 file changed, 27 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/base/dd.c b/drivers/base/dd.c
> index 1435d72..e6a6821 100644
> --- a/drivers/base/dd.c
> +++ b/drivers/base/dd.c
> @@ -224,23 +224,44 @@ void device_unblock_probing(void)
>         driver_deferred_probe_trigger();
>  }
>
> +static void enable_trigger_defer_cycle(void)
> +{
> +       driver_deferred_probe_enable = true;
> +       driver_deferred_probe_trigger();
> +       /*
> +        * Sort as many dependencies as possible before the next initcall
> +        * level
> +        */
> +       flush_work(&deferred_probe_work);
> +}
> +
>  /**
>   * deferred_probe_initcall() - Enable probing of deferred devices
>   *
>   * We don't want to get in the way when the bulk of drivers are getting probed.
>   * Instead, this initcall makes sure that deferred probing is delayed until
> - * late_initcall time.
> + * all the registered initcall functions at a particular level are completed.
> + * This function is invoked at every *_initcall_sync level.
>   */
>  static int deferred_probe_initcall(void)
>  {
> -       driver_deferred_probe_enable = true;
> -       driver_deferred_probe_trigger();
> -       /* Sort as many dependencies as possible before exiting initcalls */
> -       flush_work(&deferred_probe_work);
> +       enable_trigger_defer_cycle();
> +       driver_deferred_probe_enable = false;
> +       return 0;
> +}
> +arch_initcall_sync(deferred_probe_initcall);
> +subsys_initcall_sync(deferred_probe_initcall);
> +fs_initcall_sync(deferred_probe_initcall);
> +device_initcall_sync(deferred_probe_initcall);
> +
> +static int deferred_probe_enable_fn(void)
> +{
> +       /* Enable deferred probing for all time */
> +       enable_trigger_defer_cycle();
>         initcalls_done = true;
>         return 0;
>  }
> -late_initcall(deferred_probe_initcall);
> +late_initcall(deferred_probe_enable_fn);
>
>  /**
>   * device_is_bound() - Check if device is bound to a driver
> --
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project

Re: [PATCH] dd: Invoke one probe retry cycle after every initcall level

[rishabhb]

On 2018-07-23 04:17, Rafael J. Wysocki wrote:
> On Thu, Jul 19, 2018 at 11:24 PM, Rishabh Bhatnagar
> <rishabhb@codeaurora.org> wrote:
>> Drivers that are registered at an initcall level may have to
>> wait until late_init before the probe deferral mechanism can
>> retry their probe functions. It is possible that their
>> dependencies were resolved much earlier, in some cases even
>> before the next initcall level. Invoke one probe retry cycle
>> at every _sync initcall level, allowing these drivers to be
>> probed earlier.
> 
> Can you please say something about the actual use case this is
> expected to address?
We have a display driver that depends 3 other devices to be
probed so that it can bring-up the display. Because of dependencies
not being met the deferral mechanism defers the probes for a later time,
even though the dependencies might be met earlier. With this change
display can be brought up much earlier.
> 
>> Signed-off-by: Vikram Mulukutla <markivx@codeaurora.org>
>> Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
>> ---
>>  drivers/base/dd.c | 33 +++++++++++++++++++++++++++------
>>  1 file changed, 27 insertions(+), 6 deletions(-)
>> 
>> diff --git a/drivers/base/dd.c b/drivers/base/dd.c
>> index 1435d72..e6a6821 100644
>> --- a/drivers/base/dd.c
>> +++ b/drivers/base/dd.c
>> @@ -224,23 +224,44 @@ void device_unblock_probing(void)
>>         driver_deferred_probe_trigger();
>>  }
>> 
>> +static void enable_trigger_defer_cycle(void)
>> +{
>> +       driver_deferred_probe_enable = true;
>> +       driver_deferred_probe_trigger();
>> +       /*
>> +        * Sort as many dependencies as possible before the next 
>> initcall
>> +        * level
>> +        */
>> +       flush_work(&deferred_probe_work);
>> +}
>> +
>>  /**
>>   * deferred_probe_initcall() - Enable probing of deferred devices
>>   *
>>   * We don't want to get in the way when the bulk of drivers are 
>> getting probed.
>>   * Instead, this initcall makes sure that deferred probing is delayed 
>> until
>> - * late_initcall time.
>> + * all the registered initcall functions at a particular level are 
>> completed.
>> + * This function is invoked at every *_initcall_sync level.
>>   */
>>  static int deferred_probe_initcall(void)
>>  {
>> -       driver_deferred_probe_enable = true;
>> -       driver_deferred_probe_trigger();
>> -       /* Sort as many dependencies as possible before exiting 
>> initcalls */
>> -       flush_work(&deferred_probe_work);
>> +       enable_trigger_defer_cycle();
>> +       driver_deferred_probe_enable = false;
>> +       return 0;
>> +}
>> +arch_initcall_sync(deferred_probe_initcall);
>> +subsys_initcall_sync(deferred_probe_initcall);
>> +fs_initcall_sync(deferred_probe_initcall);
>> +device_initcall_sync(deferred_probe_initcall);
>> +
>> +static int deferred_probe_enable_fn(void)
>> +{
>> +       /* Enable deferred probing for all time */
>> +       enable_trigger_defer_cycle();
>>         initcalls_done = true;
>>         return 0;
>>  }
>> -late_initcall(deferred_probe_initcall);
>> +late_initcall(deferred_probe_enable_fn);
>> 
>>  /**
>>   * device_is_bound() - Check if device is bound to a driver
>> --
>> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora 
>> Forum,
>> a Linux Foundation Collaborative Project

Re: [PATCH] dd: Invoke one probe retry cycle after every initcall level

[Rafael J. Wysocki]

On Mon, Jul 23, 2018 at 10:22 PM,  <rishabhb@codeaurora.org> wrote:
> On 2018-07-23 04:17, Rafael J. Wysocki wrote:
>>
>> On Thu, Jul 19, 2018 at 11:24 PM, Rishabh Bhatnagar
>> <rishabhb@codeaurora.org> wrote:
>>>
>>> Drivers that are registered at an initcall level may have to
>>> wait until late_init before the probe deferral mechanism can
>>> retry their probe functions. It is possible that their
>>> dependencies were resolved much earlier, in some cases even
>>> before the next initcall level. Invoke one probe retry cycle
>>> at every _sync initcall level, allowing these drivers to be
>>> probed earlier.
>>
>>
>> Can you please say something about the actual use case this is
>> expected to address?
>
> We have a display driver that depends 3 other devices to be
> probed so that it can bring-up the display. Because of dependencies
> not being met the deferral mechanism defers the probes for a later time,
> even though the dependencies might be met earlier. With this change
> display can be brought up much earlier.

OK

What runlevel brings up the display after the change?

Thanks,
Rafael

Re: [PATCH] dd: Invoke one probe retry cycle after every initcall level

[rishabhb]

On 2018-07-24 01:24, Rafael J. Wysocki wrote:
> On Mon, Jul 23, 2018 at 10:22 PM,  <rishabhb@codeaurora.org> wrote:
>> On 2018-07-23 04:17, Rafael J. Wysocki wrote:
>>> 
>>> On Thu, Jul 19, 2018 at 11:24 PM, Rishabh Bhatnagar
>>> <rishabhb@codeaurora.org> wrote:
>>>> 
>>>> Drivers that are registered at an initcall level may have to
>>>> wait until late_init before the probe deferral mechanism can
>>>> retry their probe functions. It is possible that their
>>>> dependencies were resolved much earlier, in some cases even
>>>> before the next initcall level. Invoke one probe retry cycle
>>>> at every _sync initcall level, allowing these drivers to be
>>>> probed earlier.
>>> 
>>> 
>>> Can you please say something about the actual use case this is
>>> expected to address?
>> 
>> We have a display driver that depends 3 other devices to be
>> probed so that it can bring-up the display. Because of dependencies
>> not being met the deferral mechanism defers the probes for a later 
>> time,
>> even though the dependencies might be met earlier. With this change
>> display can be brought up much earlier.
> 
> OK
> 
> What runlevel brings up the display after the change?
> 
> Thanks,
> Rafael
After the change the display can come up after device_initcall level
itself.
The above mentioned 3 devices are probed at 0.503253, 0.505210 and 
0.523264 secs.
Only the first device is probed successfully. With the current
deferral mechanism the devices get probed again after late_initcall
at 9.19 and 9.35 secs. So display can only come up after 9.35 secs.
With this change the devices are re-probed successfully at 0.60 and
0.613 secs. Therefore display can come just after 0.613 secs.
This change helps in overall android bootup as well.

-Rishabh

Re: [PATCH] dd: Invoke one probe retry cycle after every initcall level

[Rafael J. Wysocki]

On Wed, Aug 1, 2018 at 11:18 PM,  <rishabhb@codeaurora.org> wrote:
> On 2018-07-24 01:24, Rafael J. Wysocki wrote:
>>
>> On Mon, Jul 23, 2018 at 10:22 PM,  <rishabhb@codeaurora.org> wrote:
>>>
>>> On 2018-07-23 04:17, Rafael J. Wysocki wrote:
>>>>
>>>>
>>>> On Thu, Jul 19, 2018 at 11:24 PM, Rishabh Bhatnagar
>>>> <rishabhb@codeaurora.org> wrote:
>>>>>
>>>>>
>>>>> Drivers that are registered at an initcall level may have to
>>>>> wait until late_init before the probe deferral mechanism can
>>>>> retry their probe functions. It is possible that their
>>>>> dependencies were resolved much earlier, in some cases even
>>>>> before the next initcall level. Invoke one probe retry cycle
>>>>> at every _sync initcall level, allowing these drivers to be
>>>>> probed earlier.
>>>>
>>>>
>>>>
>>>> Can you please say something about the actual use case this is
>>>> expected to address?
>>>
>>>
>>> We have a display driver that depends 3 other devices to be
>>> probed so that it can bring-up the display. Because of dependencies
>>> not being met the deferral mechanism defers the probes for a later time,
>>> even though the dependencies might be met earlier. With this change
>>> display can be brought up much earlier.
>>
>>
>> OK
>>
>> What runlevel brings up the display after the change?
>>
>> Thanks,
>> Rafael
>
> After the change the display can come up after device_initcall level
> itself.
> The above mentioned 3 devices are probed at 0.503253, 0.505210 and 0.523264
> secs.
> Only the first device is probed successfully. With the current
> deferral mechanism the devices get probed again after late_initcall
> at 9.19 and 9.35 secs. So display can only come up after 9.35 secs.
> With this change the devices are re-probed successfully at 0.60 and
> 0.613 secs. Therefore display can come just after 0.613 secs.

OK, so why do you touch the initcall levels earlier than device_?

> This change helps in overall android bootup as well.

How exactly?

Re: [PATCH] dd: Invoke one probe retry cycle after every initcall level

[Sodagudi Prasad]

> From: RAFAEL J. WYSOCKI <rafael@kernel.org>
> Date: Wed, Aug 1, 2018 at 2:21 PM
> Subject: Re: [PATCH] dd: Invoke one probe retry cycle after every
> initcall level
> To: Rishabh Bhatnagar <rishabhb@codeaurora.org>
> Cc: "Rafael J. Wysocki" <rafael@kernel.org>, Greg Kroah-Hartman
> <gregkh@linuxfoundation.org>, Linux Kernel Mailing List
> <linux-kernel@vger.kernel.org>, ckadabi@codeaurora.org,
> tsoni@codeaurora.org, Vikram Mulukutla <markivx@codeaurora.org>
> 
> On Wed, Aug 1, 2018 at 11:18 PM,  <rishabhb@codeaurora.org> wrote:
>> On 2018-07-24 01:24, Rafael J. Wysocki wrote:
>>> 
>>> On Mon, Jul 23, 2018 at 10:22 PM,  <rishabhb@codeaurora.org> wrote:
>>>> 
>>>> On 2018-07-23 04:17, Rafael J. Wysocki wrote:
>>>>> 
>>>>> 
>>>>> On Thu, Jul 19, 2018 at 11:24 PM, Rishabh Bhatnagar
>>>>> <rishabhb@codeaurora.org> wrote:
>>>>>> 
>>>>>> 
>>>>>> Drivers that are registered at an initcall level may have to
>>>>>> wait until late_init before the probe deferral mechanism can
>>>>>> retry their probe functions. It is possible that their
>>>>>> dependencies were resolved much earlier, in some cases even
>>>>>> before the next initcall level. Invoke one probe retry cycle
>>>>>> at every _sync initcall level, allowing these drivers to be
>>>>>> probed earlier.
>>>>> 
>>>>> 
>>>>> 
>>>>> Can you please say something about the actual use case this is
>>>>> expected to address?
>>>> 
>>>> 
>>>> We have a display driver that depends 3 other devices to be
>>>> probed so that it can bring-up the display. Because of
> dependencies
>>>> not being met the deferral mechanism defers the probes for a later
> time,
>>>> even though the dependencies might be met earlier. With this
> change
>>>> display can be brought up much earlier.
>>> 
>>> 
>>> OK
>>> 
>>> What runlevel brings up the display after the change?
>>> 
>>> Thanks,
>>> Rafael
>> After the change the display can come up after device_initcall level
>> itself.
>> The above mentioned 3 devices are probed at 0.503253, 0.505210 and
> 0.523264
>> secs.
>> Only the first device is probed successfully. With the current
>> deferral mechanism the devices get probed again after late_initcall
>> at 9.19 and 9.35 secs. So display can only come up after 9.35 secs.
>> With this change the devices are re-probed successfully at 0.60 and
>> 0.613 secs. Therefore display can come just after 0.613 secs.
> 
> OK, so why do you touch the initcall levels earlier than device_?
1)	re-probe probing devices in the active list on every level help to 
avoid circular dependency pending list.
2)	There are so many devices which gets deferred in earlier init call 
levels, so we wanted to reprobe them at every successive init call 
level.

> 
>> This change helps in overall android bootup as well.
> 
> How exactly?
We have seen less no of re-probes at late_init and most of the driver's 
dependency met earlier than late_init call level. It helped display and 
couple of other drivers by executing the re probe work at every init 
level.

-thanks, Prasad
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora 
Forum,
Linux Foundation Collaborative Project

Re: [PATCH] dd: Invoke one probe retry cycle after every initcall level

[Rafael J. Wysocki]

On Fri, Aug 3, 2018 at 12:20 AM, Sodagudi Prasad
<psodagud@codeaurora.org> wrote:
>> From: RAFAEL J. WYSOCKI <rafael@kernel.org>
>> Date: Wed, Aug 1, 2018 at 2:21 PM
>> Subject: Re: [PATCH] dd: Invoke one probe retry cycle after every
>> initcall level
>> To: Rishabh Bhatnagar <rishabhb@codeaurora.org>
>> Cc: "Rafael J. Wysocki" <rafael@kernel.org>, Greg Kroah-Hartman
>> <gregkh@linuxfoundation.org>, Linux Kernel Mailing List
>> <linux-kernel@vger.kernel.org>, ckadabi@codeaurora.org,
>> tsoni@codeaurora.org, Vikram Mulukutla <markivx@codeaurora.org>
>>
>>
>> On Wed, Aug 1, 2018 at 11:18 PM,  <rishabhb@codeaurora.org> wrote:
>>>
>>> On 2018-07-24 01:24, Rafael J. Wysocki wrote:
>>>>
>>>>
>>>> On Mon, Jul 23, 2018 at 10:22 PM,  <rishabhb@codeaurora.org> wrote:
>>>>>
>>>>>
>>>>> On 2018-07-23 04:17, Rafael J. Wysocki wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jul 19, 2018 at 11:24 PM, Rishabh Bhatnagar
>>>>>> <rishabhb@codeaurora.org> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Drivers that are registered at an initcall level may have to
>>>>>>> wait until late_init before the probe deferral mechanism can
>>>>>>> retry their probe functions. It is possible that their
>>>>>>> dependencies were resolved much earlier, in some cases even
>>>>>>> before the next initcall level. Invoke one probe retry cycle
>>>>>>> at every _sync initcall level, allowing these drivers to be
>>>>>>> probed earlier.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Can you please say something about the actual use case this is
>>>>>> expected to address?
>>>>>
>>>>>
>>>>>
>>>>> We have a display driver that depends 3 other devices to be
>>>>> probed so that it can bring-up the display. Because of
>>
>> dependencies
>>>>>
>>>>> not being met the deferral mechanism defers the probes for a later
>>
>> time,
>>>>>
>>>>> even though the dependencies might be met earlier. With this
>>
>> change
>>>>>
>>>>> display can be brought up much earlier.
>>>>
>>>>
>>>>
>>>> OK
>>>>
>>>> What runlevel brings up the display after the change?
>>>>
>>>> Thanks,
>>>> Rafael
>>>
>>> After the change the display can come up after device_initcall level
>>> itself.
>>> The above mentioned 3 devices are probed at 0.503253, 0.505210 and
>>
>> 0.523264
>>>
>>> secs.
>>> Only the first device is probed successfully. With the current
>>> deferral mechanism the devices get probed again after late_initcall
>>> at 9.19 and 9.35 secs. So display can only come up after 9.35 secs.
>>> With this change the devices are re-probed successfully at 0.60 and
>>> 0.613 secs. Therefore display can come just after 0.613 secs.
>>
>>
>> OK, so why do you touch the initcall levels earlier than device_?
>
> 1)      re-probe probing devices in the active list on every level help to
> avoid circular dependency pending list.
> 2)      There are so many devices which gets deferred in earlier init call
> levels, so we wanted to reprobe them at every successive init call level.

Do you have specific examples of devices for which that helps?

>>
>>> This change helps in overall android bootup as well.
>>
>>
>> How exactly?
>
> We have seen less no of re-probes at late_init and most of the driver's
> dependency met earlier than late_init call level. It helped display and
> couple of other drivers by executing the re probe work at every init level.

So I can believe that walking the deferred list on device_initcall and
maybe on device_initcall_sync may help, but I'm not quite convinced
that it matters for the earlier initcall levels.

Re: [PATCH] dd: Invoke one probe retry cycle after every initcall level

[rishabhb]

On 2018-08-06 01:53, Rafael J. Wysocki wrote:
> On Fri, Aug 3, 2018 at 12:20 AM, Sodagudi Prasad
> <psodagud@codeaurora.org> wrote:
>>> From: RAFAEL J. WYSOCKI <rafael@kernel.org>
>>> Date: Wed, Aug 1, 2018 at 2:21 PM
>>> Subject: Re: [PATCH] dd: Invoke one probe retry cycle after every
>>> initcall level
>>> To: Rishabh Bhatnagar <rishabhb@codeaurora.org>
>>> Cc: "Rafael J. Wysocki" <rafael@kernel.org>, Greg Kroah-Hartman
>>> <gregkh@linuxfoundation.org>, Linux Kernel Mailing List
>>> <linux-kernel@vger.kernel.org>, ckadabi@codeaurora.org,
>>> tsoni@codeaurora.org, Vikram Mulukutla <markivx@codeaurora.org>
>>> 
>>> 
>>> On Wed, Aug 1, 2018 at 11:18 PM,  <rishabhb@codeaurora.org> wrote:
>>>> 
>>>> On 2018-07-24 01:24, Rafael J. Wysocki wrote:
>>>>> 
>>>>> 
>>>>> On Mon, Jul 23, 2018 at 10:22 PM,  <rishabhb@codeaurora.org> wrote:
>>>>>> 
>>>>>> 
>>>>>> On 2018-07-23 04:17, Rafael J. Wysocki wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On Thu, Jul 19, 2018 at 11:24 PM, Rishabh Bhatnagar
>>>>>>> <rishabhb@codeaurora.org> wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Drivers that are registered at an initcall level may have to
>>>>>>>> wait until late_init before the probe deferral mechanism can
>>>>>>>> retry their probe functions. It is possible that their
>>>>>>>> dependencies were resolved much earlier, in some cases even
>>>>>>>> before the next initcall level. Invoke one probe retry cycle
>>>>>>>> at every _sync initcall level, allowing these drivers to be
>>>>>>>> probed earlier.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Can you please say something about the actual use case this is
>>>>>>> expected to address?
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> We have a display driver that depends 3 other devices to be
>>>>>> probed so that it can bring-up the display. Because of
>>> 
>>> dependencies
>>>>>> 
>>>>>> not being met the deferral mechanism defers the probes for a later
>>> 
>>> time,
>>>>>> 
>>>>>> even though the dependencies might be met earlier. With this
>>> 
>>> change
>>>>>> 
>>>>>> display can be brought up much earlier.
>>>>> 
>>>>> 
>>>>> 
>>>>> OK
>>>>> 
>>>>> What runlevel brings up the display after the change?
>>>>> 
>>>>> Thanks,
>>>>> Rafael
>>>> 
>>>> After the change the display can come up after device_initcall level
>>>> itself.
>>>> The above mentioned 3 devices are probed at 0.503253, 0.505210 and
>>> 
>>> 0.523264
>>>> 
>>>> secs.
>>>> Only the first device is probed successfully. With the current
>>>> deferral mechanism the devices get probed again after late_initcall
>>>> at 9.19 and 9.35 secs. So display can only come up after 9.35 secs.
>>>> With this change the devices are re-probed successfully at 0.60 and
>>>> 0.613 secs. Therefore display can come just after 0.613 secs.
>>> 
>>> 
>>> OK, so why do you touch the initcall levels earlier than device_?
>> 
>> 1)      re-probe probing devices in the active list on every level 
>> help to
>> avoid circular dependency pending list.
>> 2)      There are so many devices which gets deferred in earlier init 
>> call
>> levels, so we wanted to reprobe them at every successive init call 
>> level.
> 
> Do you have specific examples of devices for which that helps?
> 
>>> 
>>>> This change helps in overall android bootup as well.
>>> 
>>> 
>>> How exactly?
>> 
>> We have seen less no of re-probes at late_init and most of the 
>> driver's
>> dependency met earlier than late_init call level. It helped display 
>> and
>> couple of other drivers by executing the re probe work at every init 
>> level.
> 
> So I can believe that walking the deferred list on device_initcall and
> maybe on device_initcall_sync may help, but I'm not quite convinced
> that it matters for the earlier initcall levels.

Many of our drivers are dependent on the regulator and bus driver.
Both the regulator and bus driver are probed in the subsys_initcall 
level.
Now the probe of bus driver requires regulator to be working. If the 
probe of
bus driver happens before regulator, then bus driver's probe will be
deferred and all other device's probes which depend on bus driver will 
also be deferred.
The impact of this problem is reduced if we have this patch.

Re: [PATCH] dd: Invoke one probe retry cycle after every initcall level

[Rafael J. Wysocki]

On Fri, Aug 10, 2018 at 12:30 AM,  <rishabhb@codeaurora.org> wrote:
> On 2018-08-06 01:53, Rafael J. Wysocki wrote:
>>
>> On Fri, Aug 3, 2018 at 12:20 AM, Sodagudi Prasad
>> <psodagud@codeaurora.org> wrote:
>>>>
>>>> From: RAFAEL J. WYSOCKI <rafael@kernel.org>
>>>> Date: Wed, Aug 1, 2018 at 2:21 PM
>>>> Subject: Re: [PATCH] dd: Invoke one probe retry cycle after every
>>>> initcall level
>>>> To: Rishabh Bhatnagar <rishabhb@codeaurora.org>
>>>> Cc: "Rafael J. Wysocki" <rafael@kernel.org>, Greg Kroah-Hartman
>>>> <gregkh@linuxfoundation.org>, Linux Kernel Mailing List
>>>> <linux-kernel@vger.kernel.org>, ckadabi@codeaurora.org,
>>>> tsoni@codeaurora.org, Vikram Mulukutla <markivx@codeaurora.org>
>>>>
>>>>
>>>> On Wed, Aug 1, 2018 at 11:18 PM,  <rishabhb@codeaurora.org> wrote:
>>>>>
>>>>>
>>>>> On 2018-07-24 01:24, Rafael J. Wysocki wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Jul 23, 2018 at 10:22 PM,  <rishabhb@codeaurora.org> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 2018-07-23 04:17, Rafael J. Wysocki wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jul 19, 2018 at 11:24 PM, Rishabh Bhatnagar
>>>>>>>> <rishabhb@codeaurora.org> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Drivers that are registered at an initcall level may have to
>>>>>>>>> wait until late_init before the probe deferral mechanism can
>>>>>>>>> retry their probe functions. It is possible that their
>>>>>>>>> dependencies were resolved much earlier, in some cases even
>>>>>>>>> before the next initcall level. Invoke one probe retry cycle
>>>>>>>>> at every _sync initcall level, allowing these drivers to be
>>>>>>>>> probed earlier.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Can you please say something about the actual use case this is
>>>>>>>> expected to address?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> We have a display driver that depends 3 other devices to be
>>>>>>> probed so that it can bring-up the display. Because of
>>>>
>>>>
>>>> dependencies
>>>>>>>
>>>>>>>
>>>>>>> not being met the deferral mechanism defers the probes for a later
>>>>
>>>>
>>>> time,
>>>>>>>
>>>>>>>
>>>>>>> even though the dependencies might be met earlier. With this
>>>>
>>>>
>>>> change
>>>>>>>
>>>>>>>
>>>>>>> display can be brought up much earlier.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> OK
>>>>>>
>>>>>> What runlevel brings up the display after the change?
>>>>>>
>>>>>> Thanks,
>>>>>> Rafael
>>>>>
>>>>>
>>>>> After the change the display can come up after device_initcall level
>>>>> itself.
>>>>> The above mentioned 3 devices are probed at 0.503253, 0.505210 and
>>>>
>>>>
>>>> 0.523264
>>>>>
>>>>>
>>>>> secs.
>>>>> Only the first device is probed successfully. With the current
>>>>> deferral mechanism the devices get probed again after late_initcall
>>>>> at 9.19 and 9.35 secs. So display can only come up after 9.35 secs.
>>>>> With this change the devices are re-probed successfully at 0.60 and
>>>>> 0.613 secs. Therefore display can come just after 0.613 secs.
>>>>
>>>>
>>>>
>>>> OK, so why do you touch the initcall levels earlier than device_?
>>>
>>>
>>> 1)      re-probe probing devices in the active list on every level help
>>> to
>>> avoid circular dependency pending list.
>>> 2)      There are so many devices which gets deferred in earlier init
>>> call
>>> levels, so we wanted to reprobe them at every successive init call level.
>>
>>
>> Do you have specific examples of devices for which that helps?
>>
>>>>
>>>>> This change helps in overall android bootup as well.
>>>>
>>>>
>>>>
>>>> How exactly?
>>>
>>>
>>> We have seen less no of re-probes at late_init and most of the driver's
>>> dependency met earlier than late_init call level. It helped display and
>>> couple of other drivers by executing the re probe work at every init
>>> level.
>>
>>
>> So I can believe that walking the deferred list on device_initcall and
>> maybe on device_initcall_sync may help, but I'm not quite convinced
>> that it matters for the earlier initcall levels.
>
>
> Many of our drivers are dependent on the regulator and bus driver.
> Both the regulator and bus driver are probed in the subsys_initcall level.
> Now the probe of bus driver requires regulator to be working. If the probe
> of
> bus driver happens before regulator, then bus driver's probe will be
> deferred and all other device's probes which depend on bus driver will also
> be deferred.
> The impact of this problem is reduced if we have this patch.

Fair enough, but this information should be there in the changelog of
your patch.

And why do you do that for arch_initcall_sync()?

Re: [PATCH] dd: Invoke one probe retry cycle after every initcall level

[Sodagudi Prasad]

On 2018-08-10 00:10, Rafael J. Wysocki wrote:
> On Fri, Aug 10, 2018 at 12:30 AM,  <rishabhb@codeaurora.org> wrote:
>> On 2018-08-06 01:53, Rafael J. Wysocki wrote:
>>> 
>>> On Fri, Aug 3, 2018 at 12:20 AM, Sodagudi Prasad
>>> <psodagud@codeaurora.org> wrote:
>>>>> 
>>>>> From: RAFAEL J. WYSOCKI <rafael@kernel.org>
>>>>> Date: Wed, Aug 1, 2018 at 2:21 PM
>>>>> Subject: Re: [PATCH] dd: Invoke one probe retry cycle after every
>>>>> initcall level
>>>>> To: Rishabh Bhatnagar <rishabhb@codeaurora.org>
>>>>> Cc: "Rafael J. Wysocki" <rafael@kernel.org>, Greg Kroah-Hartman
>>>>> <gregkh@linuxfoundation.org>, Linux Kernel Mailing List
>>>>> <linux-kernel@vger.kernel.org>, ckadabi@codeaurora.org,
>>>>> tsoni@codeaurora.org, Vikram Mulukutla <markivx@codeaurora.org>
>>>>> 
>>>>> 
>>>>> On Wed, Aug 1, 2018 at 11:18 PM,  <rishabhb@codeaurora.org> wrote:
>>>>>> 
>>>>>> 
>>>>>> On 2018-07-24 01:24, Rafael J. Wysocki wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On Mon, Jul 23, 2018 at 10:22 PM,  <rishabhb@codeaurora.org> 
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 2018-07-23 04:17, Rafael J. Wysocki wrote:
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Thu, Jul 19, 2018 at 11:24 PM, Rishabh Bhatnagar
>>>>>>>>> <rishabhb@codeaurora.org> wrote:
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Drivers that are registered at an initcall level may have to
>>>>>>>>>> wait until late_init before the probe deferral mechanism can
>>>>>>>>>> retry their probe functions. It is possible that their
>>>>>>>>>> dependencies were resolved much earlier, in some cases even
>>>>>>>>>> before the next initcall level. Invoke one probe retry cycle
>>>>>>>>>> at every _sync initcall level, allowing these drivers to be
>>>>>>>>>> probed earlier.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Can you please say something about the actual use case this is
>>>>>>>>> expected to address?
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> We have a display driver that depends 3 other devices to be
>>>>>>>> probed so that it can bring-up the display. Because of
>>>>> 
>>>>> 
>>>>> dependencies
>>>>>>>> 
>>>>>>>> 
>>>>>>>> not being met the deferral mechanism defers the probes for a 
>>>>>>>> later
>>>>> 
>>>>> 
>>>>> time,
>>>>>>>> 
>>>>>>>> 
>>>>>>>> even though the dependencies might be met earlier. With this
>>>>> 
>>>>> 
>>>>> change
>>>>>>>> 
>>>>>>>> 
>>>>>>>> display can be brought up much earlier.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> OK
>>>>>>> 
>>>>>>> What runlevel brings up the display after the change?
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Rafael
>>>>>> 
>>>>>> 
>>>>>> After the change the display can come up after device_initcall 
>>>>>> level
>>>>>> itself.
>>>>>> The above mentioned 3 devices are probed at 0.503253, 0.505210 and
>>>>> 
>>>>> 
>>>>> 0.523264
>>>>>> 
>>>>>> 
>>>>>> secs.
>>>>>> Only the first device is probed successfully. With the current
>>>>>> deferral mechanism the devices get probed again after 
>>>>>> late_initcall
>>>>>> at 9.19 and 9.35 secs. So display can only come up after 9.35 
>>>>>> secs.
>>>>>> With this change the devices are re-probed successfully at 0.60 
>>>>>> and
>>>>>> 0.613 secs. Therefore display can come just after 0.613 secs.
>>>>> 
>>>>> 
>>>>> 
>>>>> OK, so why do you touch the initcall levels earlier than device_?
>>>> 
>>>> 
>>>> 1)      re-probe probing devices in the active list on every level 
>>>> help
>>>> to
>>>> avoid circular dependency pending list.
>>>> 2)      There are so many devices which gets deferred in earlier 
>>>> init
>>>> call
>>>> levels, so we wanted to reprobe them at every successive init call 
>>>> level.
>>> 
>>> 
>>> Do you have specific examples of devices for which that helps?
>>> 
>>>>> 
>>>>>> This change helps in overall android bootup as well.
>>>>> 
>>>>> 
>>>>> 
>>>>> How exactly?
>>>> 
>>>> 
>>>> We have seen less no of re-probes at late_init and most of the 
>>>> driver's
>>>> dependency met earlier than late_init call level. It helped display 
>>>> and
>>>> couple of other drivers by executing the re probe work at every init
>>>> level.
>>> 
>>> 
>>> So I can believe that walking the deferred list on device_initcall 
>>> and
>>> maybe on device_initcall_sync may help, but I'm not quite convinced
>>> that it matters for the earlier initcall levels.
>> 
>> 
>> Many of our drivers are dependent on the regulator and bus driver.
>> Both the regulator and bus driver are probed in the subsys_initcall 
>> level.
>> Now the probe of bus driver requires regulator to be working. If the 
>> probe
>> of
>> bus driver happens before regulator, then bus driver's probe will be
>> deferred and all other device's probes which depend on bus driver will 
>> also
>> be deferred.
>> The impact of this problem is reduced if we have this patch.
> 
> Fair enough, but this information should be there in the changelog of
> your patch.
> 
> And why do you do that for arch_initcall_sync()?
<Prasad> You are right and we can remove arch_initcall_sync().

Added some logging and observed that, none of the devices are re-probed 
in the arch_initcall_sync level.

-Thanks, Prasad
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora 
Forum,
Linux Foundation Collaborative Project