Qualcomm DT bindings and DTS cleanups - tracking community wide

Qualcomm DT bindings and DTS cleanups - tracking community wide

[Krzysztof Kozlowski]

Hi everyone,

Quite a lot of people are working on Qualcomm DT bindings conversion
(TXT->YAML) and fixups to Qualcomm DTS. We track a bit of this effort
internally in Linaro, but that has many shortcomings and we would like
to track it rather community-wide with the support and contributions
from the community.

What to track:
1. Which bindings to convert to YAML,
2. Missing compatibles (either entirely or because of missing conversion),
3. `dt_binding_check` warnings (usually connected with 1-2),
4. `dtbs_check` warnings.

Rob's bot gives us daily output for 1-4, but how can we track current
efforts to avoid duplication of work? Also it would allow people to find
tasks for them to get contributions to Linux kernel :). Is anyone in
community interested in tracking it together, in a public way?

If so, where?
A. elinux.org (needs some formatting when pasting the output from tools)
B. gitlab pages/wiki (maybe scripts could parse tools and create the page?)
C. gitlab dedicated repo - some text file
D. Linux kernel TODO file (might be difficult to keep updated)
E. kernel.org wiki (requires LF accounts, AFAIK, a bit pain to edit; I
have it for Exynos but I don't find it usable -
https://exynos.wiki.kernel.org/todo_tasks)

I am leaning towards Gitlab pages because they could be quite automated
- with CI or with scripts.

The point would be to list all of tasks (1-4 from the first list), keep
it updated with new results, pick/assign tasks and mark as done.

References:
https://gitlab.com/robherring/linux-dt/-/jobs/3066878011
https://gitlab.com/robherring/linux-dt/-/jobs/3066878006
https://gitlab.com/robherring/linux-dt/-/jobs/3066877999

Best regards,
Krzysztof

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Krzysztof Kozlowski]

On 22/09/2022 16:32, Krzysztof Kozlowski wrote:
> Hi everyone,
> 
> Quite a lot of people are working on Qualcomm DT bindings conversion
> (TXT->YAML) and fixups to Qualcomm DTS. We track a bit of this effort
> internally in Linaro, but that has many shortcomings and we would like
> to track it rather community-wide with the support and contributions
> from the community.
> 
> What to track:
> 1. Which bindings to convert to YAML,
> 2. Missing compatibles (either entirely or because of missing conversion),
> 3. `dt_binding_check` warnings (usually connected with 1-2),
> 4. `dtbs_check` warnings.
> 
> Rob's bot gives us daily output for 1-4, but how can we track current
> efforts to avoid duplication of work? Also it would allow people to find
> tasks for them to get contributions to Linux kernel :). Is anyone in
> community interested in tracking it together, in a public way?
> 
> If so, where?
> A. elinux.org (needs some formatting when pasting the output from tools)
> B. gitlab pages/wiki (maybe scripts could parse tools and create the page?)
> C. gitlab dedicated repo - some text file
> D. Linux kernel TODO file (might be difficult to keep updated)
> E. kernel.org wiki (requires LF accounts, AFAIK, a bit pain to edit; I
> have it for Exynos but I don't find it usable -
> https://exynos.wiki.kernel.org/todo_tasks)


Hi All,

Any thoughts on this? So far I did not receive any responses, so
probably this could mean that there is little interest in this?

Best regards,
Krzysztof

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Trilok Soni]

On 10/4/2022 7:50 AM, Krzysztof Kozlowski wrote:
> On 22/09/2022 16:32, Krzysztof Kozlowski wrote:
>> Hi everyone,
>>
>> Quite a lot of people are working on Qualcomm DT bindings conversion
>> (TXT->YAML) and fixups to Qualcomm DTS. We track a bit of this effort
>> internally in Linaro, but that has many shortcomings and we would like
>> to track it rather community-wide with the support and contributions
>> from the community.
>>
>> What to track:
>> 1. Which bindings to convert to YAML,
>> 2. Missing compatibles (either entirely or because of missing conversion),
>> 3. `dt_binding_check` warnings (usually connected with 1-2),
>> 4. `dtbs_check` warnings.
>>
>> Rob's bot gives us daily output for 1-4, but how can we track current
>> efforts to avoid duplication of work? Also it would allow people to find
>> tasks for them to get contributions to Linux kernel :). Is anyone in
>> community interested in tracking it together, in a public way?
>>
>> If so, where?
>> A. elinux.org (needs some formatting when pasting the output from tools)
>> B. gitlab pages/wiki (maybe scripts could parse tools and create the page?)
>> C. gitlab dedicated repo - some text file
>> D. Linux kernel TODO file (might be difficult to keep updated)
>> E. kernel.org wiki (requires LF accounts, AFAIK, a bit pain to edit; I
>> have it for Exynos but I don't find it usable -
>> https://exynos.wiki.kernel.org/todo_tasks)
> 
> 
> Hi All,
> 
> Any thoughts on this? So far I did not receive any responses, so
> probably this could mean that there is little interest in this?

My preference for tracking is gitlab. B or C. Everyone will have login 
and understands the workflow.

---Trilok Soni

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Krzysztof Kozlowski]

On 04/10/2022 18:37, Trilok Soni wrote:
>>> If so, where?
>>> A. elinux.org (needs some formatting when pasting the output from tools)
>>> B. gitlab pages/wiki (maybe scripts could parse tools and create the page?)
>>> C. gitlab dedicated repo - some text file
>>> D. Linux kernel TODO file (might be difficult to keep updated)
>>> E. kernel.org wiki (requires LF accounts, AFAIK, a bit pain to edit; I
>>> have it for Exynos but I don't find it usable -
>>> https://exynos.wiki.kernel.org/todo_tasks)
>>
>>
>> Hi All,
>>
>> Any thoughts on this? So far I did not receive any responses, so
>> probably this could mean that there is little interest in this?
> 
> My preference for tracking is gitlab. B or C. Everyone will have login 
> and understands the workflow.

Thanks Trilok for the input.

Best regards,
Krzysztof

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Luca Weiss]

Hi Krzysztof,

On Tue Oct 4, 2022 at 4:50 PM CEST, Krzysztof Kozlowski wrote:
> On 22/09/2022 16:32, Krzysztof Kozlowski wrote:
> > Hi everyone,
> > 
> > Quite a lot of people are working on Qualcomm DT bindings conversion
> > (TXT->YAML) and fixups to Qualcomm DTS. We track a bit of this effort
> > internally in Linaro, but that has many shortcomings and we would like
> > to track it rather community-wide with the support and contributions
> > from the community.
> > 
> > What to track:
> > 1. Which bindings to convert to YAML,
> > 2. Missing compatibles (either entirely or because of missing conversion),
> > 3. `dt_binding_check` warnings (usually connected with 1-2),
> > 4. `dtbs_check` warnings.
> > 
> > Rob's bot gives us daily output for 1-4, but how can we track current
> > efforts to avoid duplication of work? Also it would allow people to find
> > tasks for them to get contributions to Linux kernel :). Is anyone in
> > community interested in tracking it together, in a public way?
> > 
> > If so, where?
> > A. elinux.org (needs some formatting when pasting the output from tools)
> > B. gitlab pages/wiki (maybe scripts could parse tools and create the page?)
> > C. gitlab dedicated repo - some text file
> > D. Linux kernel TODO file (might be difficult to keep updated)
> > E. kernel.org wiki (requires LF accounts, AFAIK, a bit pain to edit; I
> > have it for Exynos but I don't find it usable -
> > https://exynos.wiki.kernel.org/todo_tasks)
>
>
> Hi All,
>
> Any thoughts on this? So far I did not receive any responses, so
> probably this could mean that there is little interest in this?

I'd also appreciate having something there. Similar to the count of
similar warnings that Rob is sometimes posting, I personally don't
see those apart from checking my boards (msm8226, msm8974, msm8953,
sm6350), where I recently did a cleanup spree for 8974 for low-hanging
fruit. Of course given that not every device uses all the functionality
some things that are disabled on my fairphone-fp2 device I won't see,
but only when checking other devices e.g. lg-hammerhead.

So some gitlab project with issues for each thing would be pretty nice I
believe. While I probably won't tackle big topics like mdss+mdp5 because
it's just very complex, I'm happy to pick up some small tasks that are
(comparatively) quick to fix.

Regards
Luca

>
> Best regards,
> Krzysztof

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Krzysztof Kozlowski]

On 10/10/2022 06:32, Luca Weiss wrote:
> Hi Krzysztof,
> 
> On Tue Oct 4, 2022 at 4:50 PM CEST, Krzysztof Kozlowski wrote:
>> On 22/09/2022 16:32, Krzysztof Kozlowski wrote:
>>> Hi everyone,
>>>
>>> Quite a lot of people are working on Qualcomm DT bindings conversion
>>> (TXT->YAML) and fixups to Qualcomm DTS. We track a bit of this effort
>>> internally in Linaro, but that has many shortcomings and we would like
>>> to track it rather community-wide with the support and contributions
>>> from the community.
>>>
>>> What to track:
>>> 1. Which bindings to convert to YAML,
>>> 2. Missing compatibles (either entirely or because of missing conversion),
>>> 3. `dt_binding_check` warnings (usually connected with 1-2),
>>> 4. `dtbs_check` warnings.
>>>
>>> Rob's bot gives us daily output for 1-4, but how can we track current
>>> efforts to avoid duplication of work? Also it would allow people to find
>>> tasks for them to get contributions to Linux kernel :). Is anyone in
>>> community interested in tracking it together, in a public way?
>>>
>>> If so, where?
>>> A. elinux.org (needs some formatting when pasting the output from tools)
>>> B. gitlab pages/wiki (maybe scripts could parse tools and create the page?)
>>> C. gitlab dedicated repo - some text file
>>> D. Linux kernel TODO file (might be difficult to keep updated)
>>> E. kernel.org wiki (requires LF accounts, AFAIK, a bit pain to edit; I
>>> have it for Exynos but I don't find it usable -
>>> https://exynos.wiki.kernel.org/todo_tasks)
>>
>>
>> Hi All,
>>
>> Any thoughts on this? So far I did not receive any responses, so
>> probably this could mean that there is little interest in this?
> 
> I'd also appreciate having something there. Similar to the count of
> similar warnings that Rob is sometimes posting, I personally don't
> see those apart from checking my boards (msm8226, msm8974, msm8953,
> sm6350), where I recently did a cleanup spree for 8974 for low-hanging
> fruit. Of course given that not every device uses all the functionality
> some things that are disabled on my fairphone-fp2 device I won't see,
> but only when checking other devices e.g. lg-hammerhead.
> 
> So some gitlab project with issues for each thing would be pretty nice I
> believe. While I probably won't tackle big topics like mdss+mdp5 because
> it's just very complex, I'm happy to pick up some small tasks that are
> (comparatively) quick to fix.
> 

Thanks Lucas. I am not sure how easy is to create automatically a set of
gitlab issues based on some file with warnings, thus probably in the
beginning this might be just a TXT file or set of files.

I don't want to put too much effort on the mechanism of tracking, rather
have something working, editable by many (e.g. restricted only to a
Gitlab account) and with some ways of automation.

Caleb mentioned splitting results per board, which could be done easily
with some scripts.

Best regards,
Krzysztof

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Neil Armstrong]

Hi,

On 10/10/2022 17:10, Krzysztof Kozlowski wrote:
> On 10/10/2022 06:32, Luca Weiss wrote:
>> Hi Krzysztof,
>>
>> On Tue Oct 4, 2022 at 4:50 PM CEST, Krzysztof Kozlowski wrote:
>>> On 22/09/2022 16:32, Krzysztof Kozlowski wrote:
>>>> Hi everyone,
>>>>
>>>> Quite a lot of people are working on Qualcomm DT bindings conversion
>>>> (TXT->YAML) and fixups to Qualcomm DTS. We track a bit of this effort
>>>> internally in Linaro, but that has many shortcomings and we would like
>>>> to track it rather community-wide with the support and contributions
>>>> from the community.
>>>>
>>>> What to track:
>>>> 1. Which bindings to convert to YAML,
>>>> 2. Missing compatibles (either entirely or because of missing conversion),
>>>> 3. `dt_binding_check` warnings (usually connected with 1-2),
>>>> 4. `dtbs_check` warnings.
>>>>
>>>> Rob's bot gives us daily output for 1-4, but how can we track current
>>>> efforts to avoid duplication of work? Also it would allow people to find
>>>> tasks for them to get contributions to Linux kernel :). Is anyone in
>>>> community interested in tracking it together, in a public way?
>>>>
>>>> If so, where?
>>>> A. elinux.org (needs some formatting when pasting the output from tools)
>>>> B. gitlab pages/wiki (maybe scripts could parse tools and create the page?)
>>>> C. gitlab dedicated repo - some text file
>>>> D. Linux kernel TODO file (might be difficult to keep updated)
>>>> E. kernel.org wiki (requires LF accounts, AFAIK, a bit pain to edit; I
>>>> have it for Exynos but I don't find it usable -
>>>> https://exynos.wiki.kernel.org/todo_tasks)
>>>
>>>
>>> Hi All,
>>>
>>> Any thoughts on this? So far I did not receive any responses, so
>>> probably this could mean that there is little interest in this?
>>
>> I'd also appreciate having something there. Similar to the count of
>> similar warnings that Rob is sometimes posting, I personally don't
>> see those apart from checking my boards (msm8226, msm8974, msm8953,
>> sm6350), where I recently did a cleanup spree for 8974 for low-hanging
>> fruit. Of course given that not every device uses all the functionality
>> some things that are disabled on my fairphone-fp2 device I won't see,
>> but only when checking other devices e.g. lg-hammerhead.
>>
>> So some gitlab project with issues for each thing would be pretty nice I
>> believe. While I probably won't tackle big topics like mdss+mdp5 because
>> it's just very complex, I'm happy to pick up some small tasks that are
>> (comparatively) quick to fix.
>>
> 
> Thanks Lucas. I am not sure how easy is to create automatically a set of
> gitlab issues based on some file with warnings, thus probably in the
> beginning this might be just a TXT file or set of files.
> 
> I don't want to put too much effort on the mechanism of tracking, rather
> have something working, editable by many (e.g. restricted only to a
> Gitlab account) and with some ways of automation.
> 
> Caleb mentioned splitting results per board, which could be done easily
> with some scripts.

I'm new (and old) to msm upstreaming, but having a status automagically updated
from master, -next and even maybe from lists would be neat.

If you need some help to set up the gitlab stuff, I can try to be helpful.

Thanks,
Neil

> 
> Best regards,
> Krzysztof
> 

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Caleb Connolly]

On 04/10/2022 15:50, Krzysztof Kozlowski wrote:
> On 22/09/2022 16:32, Krzysztof Kozlowski wrote:
>> Hi everyone,
>>
>> Quite a lot of people are working on Qualcomm DT bindings conversion
>> (TXT->YAML) and fixups to Qualcomm DTS. We track a bit of this effort
>> internally in Linaro, but that has many shortcomings and we would like
>> to track it rather community-wide with the support and contributions
>> from the community.
>>
>> What to track:
>> 1. Which bindings to convert to YAML,
>> 2. Missing compatibles (either entirely or because of missing conversion),
>> 3. `dt_binding_check` warnings (usually connected with 1-2),
>> 4. `dtbs_check` warnings.
>>
>> Rob's bot gives us daily output for 1-4, but how can we track current
>> efforts to avoid duplication of work? Also it would allow people to find
>> tasks for them to get contributions to Linux kernel :). Is anyone in
>> community interested in tracking it together, in a public way?
>>
>> If so, where?
>> A. elinux.org (needs some formatting when pasting the output from tools)
>> B. gitlab pages/wiki (maybe scripts could parse tools and create the page?)
>> C. gitlab dedicated repo - some text file
>> D. Linux kernel TODO file (might be difficult to keep updated)
>> E. kernel.org wiki (requires LF accounts, AFAIK, a bit pain to edit; I
>> have it for Exynos but I don't find it usable -
>> https://exynos.wiki.kernel.org/todo_tasks)
>
>
> Hi All,
>
> Any thoughts on this? So far I did not receive any responses, so
> probably this could mean that there is little interest in this?

Hi Krzysztof,

I think a GitLab repo with some automatically generated issues and some sort of
overall coverage map might be a good way to track this. Especially if it can be
shown on a per-device basis as well.


>
> Best regards,
> Krzysztof
>

--
Kind Regards,
Caleb

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Krzysztof Kozlowski]

On 10/10/2022 07:34, Caleb Connolly wrote:
> 
> 
> On 04/10/2022 15:50, Krzysztof Kozlowski wrote:
>> On 22/09/2022 16:32, Krzysztof Kozlowski wrote:
>>> Hi everyone,
>>>
>>> Quite a lot of people are working on Qualcomm DT bindings conversion
>>> (TXT->YAML) and fixups to Qualcomm DTS. We track a bit of this effort
>>> internally in Linaro, but that has many shortcomings and we would like
>>> to track it rather community-wide with the support and contributions
>>> from the community.
>>>
>>> What to track:
>>> 1. Which bindings to convert to YAML,
>>> 2. Missing compatibles (either entirely or because of missing conversion),
>>> 3. `dt_binding_check` warnings (usually connected with 1-2),
>>> 4. `dtbs_check` warnings.
>>>
>>> Rob's bot gives us daily output for 1-4, but how can we track current
>>> efforts to avoid duplication of work? Also it would allow people to find
>>> tasks for them to get contributions to Linux kernel :). Is anyone in
>>> community interested in tracking it together, in a public way?
>>>
>>> If so, where?
>>> A. elinux.org (needs some formatting when pasting the output from tools)
>>> B. gitlab pages/wiki (maybe scripts could parse tools and create the page?)
>>> C. gitlab dedicated repo - some text file
>>> D. Linux kernel TODO file (might be difficult to keep updated)
>>> E. kernel.org wiki (requires LF accounts, AFAIK, a bit pain to edit; I
>>> have it for Exynos but I don't find it usable -
>>> https://exynos.wiki.kernel.org/todo_tasks)
>>
>>
>> Hi All,
>>
>> Any thoughts on this? So far I did not receive any responses, so
>> probably this could mean that there is little interest in this?
> 
> Hi Krzysztof,
> 
> I think a GitLab repo with some automatically generated issues and some sort of
> overall coverage map might be a good way to track this. Especially if it can be
> shown on a per-device basis as well.

Thanks for the feedback!

Best regards,
Krzysztof

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Johan Hovold]

On Thu, Sep 22, 2022 at 04:32:00PM +0200, Krzysztof Kozlowski wrote:
> Hi everyone,
> 
> Quite a lot of people are working on Qualcomm DT bindings conversion
> (TXT->YAML) and fixups to Qualcomm DTS. We track a bit of this effort
> internally in Linaro, but that has many shortcomings and we would like
> to track it rather community-wide with the support and contributions
> from the community.
> 
> What to track:
> 1. Which bindings to convert to YAML,
> 2. Missing compatibles (either entirely or because of missing conversion),
> 3. `dt_binding_check` warnings (usually connected with 1-2),
> 4. `dtbs_check` warnings.
> 
> Rob's bot gives us daily output for 1-4, but how can we track current
> efforts to avoid duplication of work? Also it would allow people to find
> tasks for them to get contributions to Linux kernel :). Is anyone in
> community interested in tracking it together, in a public way?

Is this a real problem that needs fixing? I mean how often does it
happen that people submit the same YAML conversion for example? Since it
doesn't take that long to do a conversion, I'm not sure what tracking
this on some webpage buys us. It's better to just search lore before
starting a new conversion. Or search the linux-next tree to see what's
still pending.

Similarly for the other points above, as it doesn't take very long to
add a missing compatible or fix a warning it seems a bit excessive to
try to track this manually.

Perhaps a list of pending conversions or missing compatibles could be
useful for someone who's short on work, but it's bound to get outdated
pretty quickly.

> If so, where?
> A. elinux.org (needs some formatting when pasting the output from tools)
> B. gitlab pages/wiki (maybe scripts could parse tools and create the page?)
> C. gitlab dedicated repo - some text file
> D. Linux kernel TODO file (might be difficult to keep updated)
> E. kernel.org wiki (requires LF accounts, AFAIK, a bit pain to edit; I
> have it for Exynos but I don't find it usable -
> https://exynos.wiki.kernel.org/todo_tasks)
> 
> I am leaning towards Gitlab pages because they could be quite automated
> - with CI or with scripts.
> 
> The point would be to list all of tasks (1-4 from the first list), keep
> it updated with new results, pick/assign tasks and mark as done.

I don't really see the need for more process here, sorry.

If I'm working on support for a new platform and the DT checker warnings
gets too noisy I may pick some of the low hanging fruit. In the odd
chance that someone beats me to it, it's not the end of the world.

Johan

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Krzysztof Kozlowski]

On 06/10/2022 10:21, Johan Hovold wrote:
>> What to track:
>> 1. Which bindings to convert to YAML,
>> 2. Missing compatibles (either entirely or because of missing conversion),
>> 3. `dt_binding_check` warnings (usually connected with 1-2),
>> 4. `dtbs_check` warnings.
>>
>> Rob's bot gives us daily output for 1-4, but how can we track current
>> efforts to avoid duplication of work? Also it would allow people to find
>> tasks for them to get contributions to Linux kernel :). Is anyone in
>> community interested in tracking it together, in a public way?
> 
> Is this a real problem that needs fixing? I mean how often does it
> happen that people submit the same YAML conversion for example? Since it
> doesn't take that long to do a conversion, I'm not sure what tracking
> this on some webpage buys us. It's better to just search lore before
> starting a new conversion. Or search the linux-next tree to see what's
> still pending.

In terms of DT bindings conversion to DT schema:
If I were not checking for ongoing work, I would duplicate effort like
~10 times. Few other folks hit it few times, at least. Several bindings
are being converted for ~1 year!

In terms of DTS warnings - it's difficult even to check/search. For what
do you search? Warnings? Pretty often they are not part of commit msg.
By file? Then you might have many, many unrelated search results.

> 
> Similarly for the other points above, as it doesn't take very long to
> add a missing compatible or fix a warning it seems a bit excessive to
> try to track this manually.

True, some are trivial. Some however need fixing the binding which takes
time.

> Perhaps a list of pending conversions or missing compatibles could be
> useful for someone who's short on work, but it's bound to get outdated
> pretty quickly.

Another point is to have the visibility on the amount of work to be
done. But I understand that's maybe topic just for few, e.g. me, so I
can just track stuff for myself.

> 
>> If so, where?
>> A. elinux.org (needs some formatting when pasting the output from tools)
>> B. gitlab pages/wiki (maybe scripts could parse tools and create the page?)
>> C. gitlab dedicated repo - some text file
>> D. Linux kernel TODO file (might be difficult to keep updated)
>> E. kernel.org wiki (requires LF accounts, AFAIK, a bit pain to edit; I
>> have it for Exynos but I don't find it usable -
>> https://exynos.wiki.kernel.org/todo_tasks)
>>
>> I am leaning towards Gitlab pages because they could be quite automated
>> - with CI or with scripts.
>>
>> The point would be to list all of tasks (1-4 from the first list), keep
>> it updated with new results, pick/assign tasks and mark as done.
> 
> I don't really see the need for more process here, sorry.
> 
> If I'm working on support for a new platform and the DT checker warnings
> gets too noisy I may pick some of the low hanging fruit. In the odd
> chance that someone beats me to it, it's not the end of the world.

Thanks for the input!

Best regards,
Krzysztof

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Johan Hovold]

On Thu, Oct 06, 2022 at 10:39:22AM +0200, Krzysztof Kozlowski wrote:
> On 06/10/2022 10:21, Johan Hovold wrote:
> >> What to track:
> >> 1. Which bindings to convert to YAML,
> >> 2. Missing compatibles (either entirely or because of missing conversion),
> >> 3. `dt_binding_check` warnings (usually connected with 1-2),
> >> 4. `dtbs_check` warnings.
> >>
> >> Rob's bot gives us daily output for 1-4, but how can we track current
> >> efforts to avoid duplication of work? Also it would allow people to find
> >> tasks for them to get contributions to Linux kernel :). Is anyone in
> >> community interested in tracking it together, in a public way?
> > 
> > Is this a real problem that needs fixing? I mean how often does it
> > happen that people submit the same YAML conversion for example? Since it
> > doesn't take that long to do a conversion, I'm not sure what tracking
> > this on some webpage buys us. It's better to just search lore before
> > starting a new conversion. Or search the linux-next tree to see what's
> > still pending.
> 
> In terms of DT bindings conversion to DT schema:
> If I were not checking for ongoing work, I would duplicate effort like
> ~10 times. Few other folks hit it few times, at least. Several bindings
> are being converted for ~1 year!

Ok, but the conversion itself doesn't take that long even if getting it
merged and fixing up new warnings may take some time.

And after the initial posting, a quick lore search allows you to find
any on-going conversion efforts.

Perhaps that can just be mentioned in a wiki-page of sorts that lists
remaining conversions with some suggestions for how best to go about
things.

> In terms of DTS warnings - it's difficult even to check/search. For what
> do you search? Warnings? Pretty often they are not part of commit msg.
> By file? Then you might have many, many unrelated search results.

I wasn't suggesting to use lore for warnings, but searching the lists
for changes to a particular dts before embarking on a clean up doesn't
seem unreasonable.

> > Similarly for the other points above, as it doesn't take very long to
> > add a missing compatible or fix a warning it seems a bit excessive to
> > try to track this manually.
> 
> True, some are trivial. Some however need fixing the binding which takes
> time.

Right.

> > Perhaps a list of pending conversions or missing compatibles could be
> > useful for someone who's short on work, but it's bound to get outdated
> > pretty quickly.
> 
> Another point is to have the visibility on the amount of work to be
> done. But I understand that's maybe topic just for few, e.g. me, so I
> can just track stuff for myself.

Yeah, I don't think that any such extra process should be needed outside
a small group that may potentially be working on cleaning up bindings
and dts in bulk.

If you were to maintain such a list of pending and on-going conversions
for yourself, perhaps making that public is all that's needed here?

Johan

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Dmitry Baryshkov]

Hi,

On Thu, 6 Oct 2022 at 11:21, Johan Hovold <johan@kernel.org> wrote:
>
> On Thu, Sep 22, 2022 at 04:32:00PM +0200, Krzysztof Kozlowski wrote:
> > Hi everyone,
> >
> > Quite a lot of people are working on Qualcomm DT bindings conversion
> > (TXT->YAML) and fixups to Qualcomm DTS. We track a bit of this effort
> > internally in Linaro, but that has many shortcomings and we would like
> > to track it rather community-wide with the support and contributions
> > from the community.
> >
> > What to track:
> > 1. Which bindings to convert to YAML,
> > 2. Missing compatibles (either entirely or because of missing conversion),
> > 3. `dt_binding_check` warnings (usually connected with 1-2),
> > 4. `dtbs_check` warnings.
> >
> > Rob's bot gives us daily output for 1-4, but how can we track current
> > efforts to avoid duplication of work? Also it would allow people to find
> > tasks for them to get contributions to Linux kernel :). Is anyone in
> > community interested in tracking it together, in a public way?
>
> Is this a real problem that needs fixing? I mean how often does it
> happen that people submit the same YAML conversion for example? Since it
> doesn't take that long to do a conversion, I'm not sure what tracking
> this on some webpage buys us. It's better to just search lore before
> starting a new conversion. Or search the linux-next tree to see what's
> still pending.

As Krzysztof wrote, fixing a warning/adding a new platform is usually
not a big deal. However converting old txt bindings usually results in
a significant amount of work. Fixing YAML and dtsi at the same time
can take a long time, especially for obscure cases like apq8084 or old
ipq boards.

>
> Similarly for the other points above, as it doesn't take very long to
> add a missing compatible or fix a warning it seems a bit excessive to
> try to track this manually.
>
> Perhaps a list of pending conversions or missing compatibles could be
> useful for someone who's short on work, but it's bound to get outdated
> pretty quickly.

I'd suggest having a list of `qcom-legacy' tasks: outdated DT bindings
(both txt and yaml), drivers using system clock list (rather than DT
bindings). DT files pending conversion to labels. msm8974 interconnect
driver. hsuart aliases. You name it. This can be as simple as several
gitlab issues, one per each topic.
At the very least this would allow us to assess legacy bits and pieces.

Note: for me this is close to another topic that was discussed several
times, but for which we never reached a conclusion. A matrix of
supported per-SoC features.

>
> > If so, where?
> > A. elinux.org (needs some formatting when pasting the output from tools)
> > B. gitlab pages/wiki (maybe scripts could parse tools and create the page?)
> > C. gitlab dedicated repo - some text file
> > D. Linux kernel TODO file (might be difficult to keep updated)
> > E. kernel.org wiki (requires LF accounts, AFAIK, a bit pain to edit; I
> > have it for Exynos but I don't find it usable -
> > https://exynos.wiki.kernel.org/todo_tasks)
> >
> > I am leaning towards Gitlab pages because they could be quite automated
> > - with CI or with scripts.
> >
> > The point would be to list all of tasks (1-4 from the first list), keep
> > it updated with new results, pick/assign tasks and mark as done.
>
> I don't really see the need for more process here, sorry.
>
> If I'm working on support for a new platform and the DT checker warnings
> gets too noisy I may pick some of the low hanging fruit. In the odd
> chance that someone beats me to it, it's not the end of the world.

I hope that we can finally land patches to support per-file check.
Then fixing warnings for a new platform would be very easy. Even today
you can run `make something.dtb CHECK_DTBS=y` and get all your
warnings (if the processed schema is up to date). However Krzysztof's
point is about old platforms, rather than new ones.


-- 
With best wishes
Dmitry

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Johan Hovold]

On Thu, Oct 06, 2022 at 12:44:09PM +0300, Dmitry Baryshkov wrote:

> On Thu, 6 Oct 2022 at 11:21, Johan Hovold <johan@kernel.org> wrote:

> > Is this a real problem that needs fixing? I mean how often does it
> > happen that people submit the same YAML conversion for example? Since it
> > doesn't take that long to do a conversion, I'm not sure what tracking
> > this on some webpage buys us. It's better to just search lore before
> > starting a new conversion. Or search the linux-next tree to see what's
> > still pending.
> 
> As Krzysztof wrote, fixing a warning/adding a new platform is usually
> not a big deal. However converting old txt bindings usually results in
> a significant amount of work. Fixing YAML and dtsi at the same time
> can take a long time, especially for obscure cases like apq8084 or old
> ipq boards.

Yeah, if there are more than one person working on larger efforts like
that, then that may require some coordination. But you can't expect
drive-by developers to login to some web service before fixing dts
checker warnings that are bugging them.

Johan

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Krzysztof Kozlowski]

On 22/09/2022 10:32, Krzysztof Kozlowski wrote:
> Hi everyone,
> 
> Quite a lot of people are working on Qualcomm DT bindings conversion
> (TXT->YAML) and fixups to Qualcomm DTS. We track a bit of this effort
> internally in Linaro, but that has many shortcomings and we would like
> to track it rather community-wide with the support and contributions
> from the community.
> 
> What to track:
> 1. Which bindings to convert to YAML,
> 2. Missing compatibles (either entirely or because of missing conversion),
> 3. `dt_binding_check` warnings (usually connected with 1-2),
> 4. `dtbs_check` warnings.
> 
> Rob's bot gives us daily output for 1-4, but how can we track current
> efforts to avoid duplication of work? Also it would allow people to find
> tasks for them to get contributions to Linux kernel :). Is anyone in
> community interested in tracking it together, in a public way?
> 
> If so, where?
> A. elinux.org (needs some formatting when pasting the output from tools)
> B. gitlab pages/wiki (maybe scripts could parse tools and create the page?)
> C. gitlab dedicated repo - some text file
> D. Linux kernel TODO file (might be difficult to keep updated)
> E. kernel.org wiki (requires LF accounts, AFAIK, a bit pain to edit; I
> have it for Exynos but I don't find it usable -
> https://exynos.wiki.kernel.org/todo_tasks)
> 
> I am leaning towards Gitlab pages because they could be quite automated
> - with CI or with scripts.

This could be looking like that - the list of missing compatibles from
Rob's tasks:
https://gitlab.com/krzkoz/linux-dt-todo/-/blob/main/todo-compatibles.rst

There is a script which will automatically add new entries to the list
(above RST file), once fed with Rob's job output. Further this could be
probably pipelined with Rob's jobs.

List anyway has to be manually updated with work in progress.

This is for the compatibles. Missing part is doing something similar for
the dtbs_check warnings.

In replies Luca, Caleb and Neil mentioned GitLab issues. That could be
useful, so if someone would like to hook into GitLab API - feel free to
work on that (either in that repo or in separate).

Best regards,
Krzysztof

Re: Qualcomm DT bindings and DTS cleanups - tracking community wide

[Krzysztof Kozlowski]

On 11/10/2022 09:57, Krzysztof Kozlowski wrote:
> On 22/09/2022 10:32, Krzysztof Kozlowski wrote:
>> Hi everyone,
>>
>> Quite a lot of people are working on Qualcomm DT bindings conversion
>> (TXT->YAML) and fixups to Qualcomm DTS. We track a bit of this effort
>> internally in Linaro, but that has many shortcomings and we would like
>> to track it rather community-wide with the support and contributions
>> from the community.
>>
>> What to track:
>> 1. Which bindings to convert to YAML,
>> 2. Missing compatibles (either entirely or because of missing conversion),
>> 3. `dt_binding_check` warnings (usually connected with 1-2),
>> 4. `dtbs_check` warnings.
>>
>> Rob's bot gives us daily output for 1-4, but how can we track current
>> efforts to avoid duplication of work? Also it would allow people to find
>> tasks for them to get contributions to Linux kernel :). Is anyone in
>> community interested in tracking it together, in a public way?
>>
>> If so, where?
>> A. elinux.org (needs some formatting when pasting the output from tools)
>> B. gitlab pages/wiki (maybe scripts could parse tools and create the page?)
>> C. gitlab dedicated repo - some text file
>> D. Linux kernel TODO file (might be difficult to keep updated)
>> E. kernel.org wiki (requires LF accounts, AFAIK, a bit pain to edit; I
>> have it for Exynos but I don't find it usable -
>> https://exynos.wiki.kernel.org/todo_tasks)
>>
>> I am leaning towards Gitlab pages because they could be quite automated
>> - with CI or with scripts.
> 
> This could be looking like that - the list of missing compatibles from
> Rob's tasks:
> https://gitlab.com/krzkoz/linux-dt-todo/-/blob/main/todo-compatibles.rst
> 
> There is a script which will automatically add new entries to the list
> (above RST file), once fed with Rob's job output. Further this could be
> probably pipelined with Rob's jobs.
> 
> List anyway has to be manually updated with work in progress.
> 
> This is for the compatibles. Missing part is doing something similar for
> the dtbs_check warnings.
> 
> In replies Luca, Caleb and Neil mentioned GitLab issues. That could be
> useful, so if someone would like to hook into GitLab API - feel free to
> work on that (either in that repo or in separate).

Just to be clear - if the approach is interesting to anyone, just join
the project / ask for access and make edits/changes etc.

To do this, you can visit https://gitlab.com/krzkoz/linux-dt-todo/ and
next to repository name (top/middle part of page) there should be
"Request Access" link.

Best regards,
Krzysztof