Query on device links

Query on device links

[John Garry]

Hi guys,

According to "Limitations" section @ 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/driver-api/device_link.rst#n110, 
for a managed link, lack of the supplier driver may cause indefinite 
delay in probing of the consumer. Is there any way around this?

So I just want the probe order attempt of the supplier and consumer to 
be guaranteed, but the supplier probe may not be successful, i.e. does 
not actually bind.

In my case, I would like to use device_link_add(supplier, consumer, 
DL_FLAG_AUTOPROBE_CONSUMER), but I find the supplier probe may fail (and 
not due to -EPROBE_DEFER), and my consumer remains in limbo.

You may ask my I want this ordering at all - it is because in 
really_probe(), we do the device DMA configure before the actual device 
driver probe, and I just need that ordering to be ensured between devices.

Thanks,
John

Re: Query on device links

[Saravana Kannan]

On Thu, Feb 6, 2020 at 2:32 AM John Garry <john.garry@huawei.com> wrote:
>
> Hi guys,

Sorry it took a while to get back.

>
> According to "Limitations" section @
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/driver-api/device_link.rst#n110,
> for a managed link, lack of the supplier driver may cause indefinite
> delay in probing of the consumer. Is there any way around this?

Currently, no. There's no way to guarantee ordering AND ignore
supplier failures.

> So I just want the probe order attempt of the supplier and consumer to
> be guaranteed, but the supplier probe may not be successful, i.e. does
> not actually bind.
>
> In my case, I would like to use device_link_add(supplier, consumer,
> DL_FLAG_AUTOPROBE_CONSUMER), but I find the supplier probe may fail (and
> not due to -EPROBE_DEFER), and my consumer remains in limbo.

The requirements seem to contradict each other. If you depend on the
supplier, how can you probe the consumer if the supplier fails?

> You may ask my I want this ordering at all - it is because in
> really_probe(), we do the device DMA configure before the actual device
> driver probe, and I just need that ordering to be ensured between devices.

I'm assuming the supplier in your case is the "dma device" (is it an
iommu?)? So if it fails, how is your consumer probing without the
supplier? I'd think something like a DMA would be fundamental?

Why can't this logic be handled in your consumer driver instead of
using device links? Why can't your consumer driver return
-EPROBE_DEFER if the dma ops are not set up correctly until some point
after which (after late_initcall?) the consumer will continue probing
without returning -EPROBE_DEFER even if the supplier isn't there/dma
ops aren't set up?

Can you give a more concrete example of your devices? Or why the
suggestion above might not work?

-Saravana

Re: Query on device links

[John Garry]

On 22/02/2020 06:55, Saravana Kannan wrote:
> On Thu, Feb 6, 2020 at 2:32 AM John Garry <john.garry@huawei.com> wrote:
>>
>> Hi guys,
> 
> Sorry it took a while to get back.

no worries, it may have seemed a daft question

> 
>>
>> According to "Limitations" section @
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/driver-api/device_link.rst#n110,
>> for a managed link, lack of the supplier driver may cause indefinite
>> delay in probing of the consumer. Is there any way around this?
> 
> Currently, no. There's no way to guarantee ordering AND ignore
> supplier failures.
> 
>> So I just want the probe order attempt of the supplier and consumer to
>> be guaranteed, but the supplier probe may not be successful, i.e. does
>> not actually bind.
>>
>> In my case, I would like to use device_link_add(supplier, consumer,
>> DL_FLAG_AUTOPROBE_CONSUMER), but I find the supplier probe may fail (and
>> not due to -EPROBE_DEFER), and my consumer remains in limbo.
> 
> The requirements seem to contradict each other. If you depend on the
> supplier, how can you probe the consumer if the supplier fails?

In fact, I now consider what I was attempting to be a hack and not the 
proper solution to the problem.

So we assign an IOMMU group to a device in the really_probe() dma 
configure, which I required to be in order.

However the proper approach may now be to take the device iommu group 
assignment outside the device driver probe path, and not consider device 
links as the solution or any such device driver probe ordering.

> 
>> You may ask my I want this ordering at all - it is because in
>> really_probe(), we do the device DMA configure before the actual device
>> driver probe, and I just need that ordering to be ensured between devices.
> 
> I'm assuming the supplier in your case is the "dma device" (is it an
> iommu?)?

No, just 2 PCI devices, the port and an end device.

  So if it fails, how is your consumer probing without the
> supplier? I'd think something like a DMA would be fundamental?
> 
> Why can't this logic be handled in your consumer driver instead of
> using device links? 

Yes, I did try something like this, but, again, it just looks like the 
IOMMU group device assignment needs rework.

Why can't your consumer driver return
> -EPROBE_DEFER if the dma ops are not set up correctly until some point
> after which (after late_initcall?) the consumer will continue probing
> without returning -EPROBE_DEFER even if the supplier isn't there/dma
> ops aren't set up? >
> Can you give a more concrete example of your devices? Or why the
> suggestion above might not work?

As above.

Please note that we still may want to use device links for ensuring 
probe ordering of the DMA device and the IOMMU, but this is not a 
specifically related issue.

Much appreciated,
john