[Buildroot] More maintainers

[Buildroot] More maintainers

[Adam Duskett]

Hello;

Angelo and I were discussing the state of patches today in regards to Buildroot.
It seems like the project could benefit from a few more maintainers of
the project.
We both understand that this is a volunteer project and that we are
all adults and have a life outside of the project, and as such, we
can't expect instant reviews on every submitted patch. However, as the
project continues to grow in popularity, more patches will continue to
be offered at an ever-increasing rate. This popularity increase is
excellent news for the project, but also comes at the cost of more
time and effort in regards to the number of patches and fixes in the
queue to review and test.

As such, we would both like to throw our hat in the ring to help
maintain the project. We both don't have all the time in the world,
but any amount of help on the project can help alleviate the primary
maintainers.

Let me know what you think.

Adam

[Buildroot] More maintainers

[Angelo Compagnucci]

Adam, all

Il giorno gio 27 ago 2020 alle ore 19:34 Adam Duskett
<aduskett@gmail.com> ha scritto:
>
> Hello;
>
> Angelo and I were discussing the state of patches today in regards to Buildroot.
> It seems like the project could benefit from a few more maintainers of
> the project.
> We both understand that this is a volunteer project and that we are
> all adults and have a life outside of the project, and as such, we
> can't expect instant reviews on every submitted patch. However, as the
> project continues to grow in popularity, more patches will continue to
> be offered at an ever-increasing rate. This popularity increase is
> excellent news for the project, but also comes at the cost of more
> time and effort in regards to the number of patches and fixes in the
> queue to review and test.
>
> As such, we would both like to throw our hat in the ring to help
> maintain the project. We both don't have all the time in the world,
> but any amount of help on the project can help alleviate the primary
> maintainers.

Adam, thanks for the trust even if I don't think I'm ready for such a big leap!

By the way, I would really like to see some more maintainers and Adam
is of course well suited for the role.
I'm sure there are other people either, having more maintainers will
benefit greatly the project.

>
> Let me know what you think.
>
> Adam
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] More maintainers

[Thomas Petazzoni]

Hello Adam,

On Thu, 27 Aug 2020 10:34:31 -0700
Adam Duskett <aduskett@gmail.com> wrote:

> Angelo and I were discussing the state of patches today in regards to Buildroot.
> It seems like the project could benefit from a few more maintainers of
> the project.
> We both understand that this is a volunteer project and that we are
> all adults and have a life outside of the project, and as such, we
> can't expect instant reviews on every submitted patch. However, as the
> project continues to grow in popularity, more patches will continue to
> be offered at an ever-increasing rate. This popularity increase is
> excellent news for the project, but also comes at the cost of more
> time and effort in regards to the number of patches and fixes in the
> queue to review and test.
> 
> As such, we would both like to throw our hat in the ring to help
> maintain the project. We both don't have all the time in the world,
> but any amount of help on the project can help alleviate the primary
> maintainers.

I agree that we probably need to extend or change the current
maintainers team.

However, what is also true is that very little patches that are posted
get review from other contributors. And this is actually what is the
most important. When I see a patch series that has been looked at by 1
or 2 other people that I trust and who gave their Reviewed-by, I will
apply such a patch series very easily, without almost looking at it.

You can go have a look at
https://patchwork.ozlabs.org/project/buildroot/list/ and see how many
patches in the list have a Reviewed-by tag. Very few. This is
definitely something where everybody can help, and it is by doing such
reviews that the trust will increase with the current maintainers,
which is key for becoming a maintainer IMO.

What do you think ?

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] More maintainers

[Yann E. MORIN]

Thomas, Adam. Angelo, All,

On 2020-08-27 22:39 +0200, Thomas Petazzoni spake thusly:
> On Thu, 27 Aug 2020 10:34:31 -0700
> Adam Duskett <aduskett@gmail.com> wrote:
> > Angelo and I were discussing the state of patches today in regards to Buildroot.
> > It seems like the project could benefit from a few more maintainers of
> > the project.
> > We both understand that this is a volunteer project and that we are
> > all adults and have a life outside of the project, and as such, we
> > can't expect instant reviews on every submitted patch. However, as the
> > project continues to grow in popularity, more patches will continue to
> > be offered at an ever-increasing rate. This popularity increase is
> > excellent news for the project, but also comes at the cost of more
> > time and effort in regards to the number of patches and fixes in the
> > queue to review and test.
> > 
> > As such, we would both like to throw our hat in the ring to help
> > maintain the project. We both don't have all the time in the world,
> > but any amount of help on the project can help alleviate the primary
> > maintainers.
> 
> I agree that we probably need to extend or change the current
> maintainers team.
> 
> However, what is also true is that very little patches that are posted
> get review from other contributors. And this is actually what is the
> most important. When I see a patch series that has been looked at by 1
> or 2 other people that I trust and who gave their Reviewed-by, I will
> apply such a patch series very easily, without almost looking at it.
> 
> You can go have a look at
> https://patchwork.ozlabs.org/project/buildroot/list/ and see how many
> patches in the list have a Reviewed-by tag. Very few. This is
> definitely something where everybody can help, and it is by doing such
> reviews that the trust will increase with the current maintainers,
> which is key for becoming a maintainer IMO.
> 
> What do you think ?

I do agree with all of this, both the remarks from Adam and Angelo, and
the feedback from Thomas.

The burden is not in applying patches; this is the easy part.

What really takes time is actually reviewing patches: understanding the
problem, understanding how it is fixed, why it is fixed in such a way,
and seeing how it fits in the overall big-picture across the whole
project.

Then, requesting changes also takes time: explaining what has to be
changed, why it has to be changed, and how it has to be changed also
takes time.

So, I would also like to emphasize that contributing is not only about
sending patches; it is also about reviewing and testing the patches of
others, and such contributions are dearly needed.

When a patch is technically ready, the review can range from the laconic
"reviewed-by", which means just that, up to complementing the commit log
with additional details and explanations, which means that the problem
and solution have been fully assessed.

Now for a personal comment: my pet-peeve are commit logs that just
describe the patch; I usually don't need that, except in very complex
patches. What I really need is a commit log that explains the patch,
that tells me the why, not the how.

Of course, finding the right balance between "not enough details" and
"too many details" is not trivial, and is an exercise in its own right.

But again, I do appreciate that people are concerned enough about the
project, and call on the maintainers.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] More maintainers

[Adam Duskett]

All;

On Sat, Aug 29, 2020 at 2:50 AM Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>
> Thomas, Adam. Angelo, All,
>
> On 2020-08-27 22:39 +0200, Thomas Petazzoni spake thusly:
> > On Thu, 27 Aug 2020 10:34:31 -0700
> > Adam Duskett <aduskett@gmail.com> wrote:
> > > Angelo and I were discussing the state of patches today in regards to Buildroot.
> > > It seems like the project could benefit from a few more maintainers of
> > > the project.
> > > We both understand that this is a volunteer project and that we are
> > > all adults and have a life outside of the project, and as such, we
> > > can't expect instant reviews on every submitted patch. However, as the
> > > project continues to grow in popularity, more patches will continue to
> > > be offered at an ever-increasing rate. This popularity increase is
> > > excellent news for the project, but also comes at the cost of more
> > > time and effort in regards to the number of patches and fixes in the
> > > queue to review and test.
> > >
> > > As such, we would both like to throw our hat in the ring to help
> > > maintain the project. We both don't have all the time in the world,
> > > but any amount of help on the project can help alleviate the primary
> > > maintainers.
> >
> > I agree that we probably need to extend or change the current
> > maintainers team.
> >
> > However, what is also true is that very little patches that are posted
> > get review from other contributors. And this is actually what is the
> > most important. When I see a patch series that has been looked at by 1
> > or 2 other people that I trust and who gave their Reviewed-by, I will
> > apply such a patch series very easily, without almost looking at it.
> >
> > You can go have a look at
> > https://patchwork.ozlabs.org/project/buildroot/list/ and see how many
> > patches in the list have a Reviewed-by tag. Very few. This is
> > definitely something where everybody can help, and it is by doing such
> > reviews that the trust will increase with the current maintainers,
> > which is key for becoming a maintainer IMO.
> >
> > What do you think ?
>
> I do agree with all of this, both the remarks from Adam and Angelo, and
> the feedback from Thomas.
>
> The burden is not in applying patches; this is the easy part.
>
> What really takes time is actually reviewing patches: understanding the
> problem, understanding how it is fixed, why it is fixed in such a way,
> and seeing how it fits in the overall big-picture across the whole
> project.
I have seen many patches reviewed by not only myself but other's for months.
If the current maintainers want to have the last word, then there is
no incentive for
non-maintainers to review patches, as it is seen as a waste of time.

I apologize for wasting your time.

Adam
>
> Then, requesting changes also takes time: explaining what has to be
> changed, why it has to be changed, and how it has to be changed also
> takes time.
>
> So, I would also like to emphasize that contributing is not only about
> sending patches; it is also about reviewing and testing the patches of
> others, and such contributions are dearly needed.
>
> When a patch is technically ready, the review can range from the laconic
> "reviewed-by", which means just that, up to complementing the commit log
> with additional details and explanations, which means that the problem
> and solution have been fully assessed.
>
> Now for a personal comment: my pet-peeve are commit logs that just
> describe the patch; I usually don't need that, except in very complex
> patches. What I really need is a commit log that explains the patch,
> that tells me the why, not the how.
>
> Of course, finding the right balance between "not enough details" and
> "too many details" is not trivial, and is an exercise in its own right.
>
> But again, I do appreciate that people are concerned enough about the
> project, and call on the maintainers.
>
> Regards,
> Yann E. MORIN.
>
> --
> .-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
> | +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
> '------------------------------^-------^------------------^--------------------'

[Buildroot] More maintainers

[Arnout Vandecappelle]

On 02/09/2020 19:53, Adam Duskett wrote:
> I have seen many patches reviewed by not only myself but other's for months.

 True. However, please consider that maybe the maintainers don't intentionally
ignore those patches. Even though in absolute terms maybe a lot of patches have
been reviewed, in relative terms it's not exactly great. Currently, for example,
15 out of 400 patches in patchwork have a Reviewed-by tag. They simply slipped
through the cracks, because we tend to look at most recent patches first.

 Regards,
 Arnout

> If the current maintainers want to have the last word, then there is
> no incentive for
> non-maintainers to review patches, as it is seen as a waste of time.

[Buildroot] More maintainers

[Adam Duskett]

Arnout;

On Wed, Sep 2, 2020 at 1:08 PM Arnout Vandecappelle <arnout@mind.be> wrote:
>
>
>
> On 02/09/2020 19:53, Adam Duskett wrote:
> > I have seen many patches reviewed by not only myself but other's for months.
>
>  True. However, please consider that maybe the maintainers don't intentionally
> ignore those patches. Even though in absolute terms maybe a lot of patches have
> been reviewed, in relative terms it's not exactly great. Currently, for example,
> 15 out of 400 patches in patchwork have a Reviewed-by tag. They simply slipped
> through the cracks, because we tend to look at most recent patches first.
>
This ignores my other point which still stands. The maintainers
historically always want to also
review the patches individually, and thus demand the last word on all
submitted patches.

This process makes it pointless for anybody but maintainers to review
patches because the
reviewers feel outright ignored.


>  Regards,
>  Arnout
>
> > If the current maintainers want to have the last word, then there is
> > no incentive for
> > non-maintainers to review patches, as it is seen as a waste of time.

[Buildroot] More maintainers

[Christian Stewart]

Hi Adam,

On Wed, Sep 2, 2020 at 1:11 PM Adam Duskett <aduskett@gmail.com> wrote:
> >  True. However, please consider that maybe the maintainers don't intentionally
> > ignore those patches. Even though in absolute terms maybe a lot of patches have
> > been reviewed, in relative terms it's not exactly great. Currently, for example,
> > 15 out of 400 patches in patchwork have a Reviewed-by tag. They simply slipped
> > through the cracks, because we tend to look at most recent patches first.
> >
> This ignores my other point which still stands. The maintainers
> historically always want to also
> review the patches individually, and thus demand the last word on all
> submitted patches.
>
> This process makes it pointless for anybody but maintainers to review
> patches because the
> reviewers feel outright ignored.

I understand how you feel here, however, in my experience patch
reviews from non-maintainers have often lead to me marking the series
as "Changes Requested" myself, and submit an additional revision later
on without any maintainers needing to look at it. It's always possible
for just about anyone to point out a problem with a patch, and for the
submitter to validate that that indeed is a problem. So there's
definitely value to the Reviewed-By. My understanding of what the
maintainers are trying to say here, is that they're looking to
prioritize which series they review based on how many people have
backed it with a Reviewed-By tag.

From watching the project and others, I've personally seen that it's
very valuable to have a core team of maintainers with a very
opinionated approach and process structure, because it builds trust
with the downstream userbase. And because Git is decentralized, we can
always run our own third-party forks with all sorts of experimental
patches applied. The primary Buildroot repository is the "gold
standard" in that sense.

Discoverability is a problem right now, there's too many patches and
no way to subscribe to a given series for further updates (similar to
Pull Requests on GitHub), unless the submitter happens to add you to
the CC list - which, by the way, usually happens if you add a review.

Today, probably the best approach is to hunt through the backlog
(sorted by reviewed-by priority order) and cherry-pick some of these
series into downstream, more "experimental" repositories. Then, once
they've been tested there, you can submit a Reviewed-By back upstream
to indicate that it was useful and worth merging more urgently.

> > > If the current maintainers want to have the last word, then there is
> > > no incentive for
> > > non-maintainers to review patches, as it is seen as a waste of time.

If a non-maintainer reviews a patch -> code quality increases ->
reviewed-by count increases -> priority increases on patch ->
maintainers see it sooner -> it gets merged sooner. "Last say" or not,
I don't really see a problem with maintainers having the authority to
impose their opinionated approach to patches & rewrite things as they
see fit.

I'm interested to hear everyone's thoughts on this: particularly, that
this is a decentralized process and we can have multiple downstreams
with testing patches applied & a wider userbase than a single
developer.

Best regards,
Christian Stewart

[Buildroot] More maintainers

[Avraham Shukron]

On Thu, Sep 3, 2020, 01:20 Christian Stewart <christian@paral.in> wrote:

> Discoverability is a problem right now, there's too many patches and
> no way to subscribe to a given series for further updates (similar to
> Pull Requests on GitHub), unless the submitter happens to add you to
> the CC list - which, by the way, usually happens if you add a review.
>
> Today, probably the best approach is to hunt through the backlog
> (sorted by reviewed-by priority order) and cherry-pick some of these
> series into downstream, more "experimental" repositories. Then, once
> they've been tested there, you can submit a Reviewed-By back upstream
> to indicate that it was useful and worth merging more urgently.
>

For what it's worth I think that the mail-based contribution process is
part of the problem.
With a decent Git server like GitHub/GitLab patches could be reviewed more
easily.
A CI pipeline could run tests that will give immediate feedback for any
pull request.
More importantly, it'll dramatically reduce the barrier for new and young
contributors.

Buildroot is all about using simple and familiar tools like make and
Kconfig, and I personally think that this principle should also be applied
to the contribution process and right now buildroot is one of the last
active open source projects using the mailing list approach.

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200903/c57da135/attachment.html>

[Buildroot] More maintainers

[Thomas Petazzoni]

Hello Avraham,

On Thu, 3 Sep 2020 18:44:27 +0300
Avraham Shukron <avraham.shukron@gmail.com> wrote:

> For what it's worth I think that the mail-based contribution process is
> part of the problem.
> With a decent Git server like GitHub/GitLab patches could be reviewed more
> easily.
> A CI pipeline could run tests that will give immediate feedback for any
> pull request.
> More importantly, it'll dramatically reduce the barrier for new and young
> contributors.

This has been discussed multiple times in the past in Buildroot, and in
other open-source projects. There is even as we speak some pretty
intense debate in the Linux kernel community about this.

As we've seen from the discussion here, the Buildroot issue is not a
lack of contribution, but a lack of review from trusted and experienced
reviewers, and a lack of maintainers time. So while I'm all for new
contributors and contributions, I don't think reducing the barrier is
really key here. Also, I've always been skeptical about this statement
that using Github reduces the barrier to entry. When you contribute to
a project, is really sending a patch over e-mail the difficult part
compared to understanding the code base, debugging the issue you've
found or implementing the feature you wanted ? Really installing "git
send-email" is a no-brainer straightforward process that is
ridiculously easy even compared writing any single change in the code
base with a decent commit message. Aren't we using this "reducing the
barrier" argument a bit inappropriately here ?

I believe I can say that all four Buildroot maintainers have a very
strong preference for and a very optimized workflow to work with e-mail
based patch submission and review.

Somewhat related, recently a patch series I submitted last year to
OpenWrt (which wasn't merged) got picked up by someone else, and
re-submitted with new updates and fixes. Due to being the original
author, I was in copy of all the Github discussion that took place. And
I found it absolutely impossible and awful to follow the different
revisions of the patch series, to which version of the patch series the
comments were being made, etc.

Perhaps for some people the Github pull request workflow makes sense,
but I believe it's important to recognize and realize that there are
also people for which this workflow doesn't make sense.

> Buildroot is all about using simple and familiar tools like make and
> Kconfig, and I personally think that this principle should also be applied
> to the contribution process and right now buildroot is one of the last
> active open source projects using the mailing list approach.

This is a very biased statement: there are still plenty of open-source
projects that use e-mail based contribution workflow. I don't think we
can call Linux or U-Boot "inactive" projects. Can we? :-)

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] More maintainers

[Angelo Compagnucci]

Il giorno gio 3 set 2020 alle ore 18:24 Thomas Petazzoni
<thomas.petazzoni@bootlin.com> ha scritto:
>
> Hello Avraham,
>
> On Thu, 3 Sep 2020 18:44:27 +0300
> Avraham Shukron <avraham.shukron@gmail.com> wrote:
>
> > For what it's worth I think that the mail-based contribution process is
> > part of the problem.
> > With a decent Git server like GitHub/GitLab patches could be reviewed more
> > easily.
> > A CI pipeline could run tests that will give immediate feedback for any
> > pull request.
> > More importantly, it'll dramatically reduce the barrier for new and young
> > contributors.
>
> This has been discussed multiple times in the past in Buildroot, and in
> other open-source projects. There is even as we speak some pretty
> intense debate in the Linux kernel community about this.
>
> As we've seen from the discussion here, the Buildroot issue is not a
> lack of contribution, but a lack of review from trusted and experienced
> reviewers, and a lack of maintainers time. So while I'm all for new
> contributors and contributions, I don't think reducing the barrier is
> really key here. Also, I've always been skeptical about this statement
> that using Github reduces the barrier to entry. When you contribute to
> a project, is really sending a patch over e-mail the difficult part
> compared to understanding the code base, debugging the issue you've
> found or implementing the feature you wanted ? Really installing "git
> send-email" is a no-brainer straightforward process that is
> ridiculously easy even compared writing any single change in the code
> base with a decent commit message. Aren't we using this "reducing the
> barrier" argument a bit inappropriately here ?
>
> I believe I can say that all four Buildroot maintainers have a very
> strong preference for and a very optimized workflow to work with e-mail
> based patch submission and review.
>
> Somewhat related, recently a patch series I submitted last year to
> OpenWrt (which wasn't merged) got picked up by someone else, and
> re-submitted with new updates and fixes. Due to being the original
> author, I was in copy of all the Github discussion that took place. And
> I found it absolutely impossible and awful to follow the different
> revisions of the patch series, to which version of the patch series the
> comments were being made, etc.
>
> Perhaps for some people the Github pull request workflow makes sense,
> but I believe it's important to recognize and realize that there are
> also people for which this workflow doesn't make sense.

I strongly disagree here.
Having to write a patch changelog or download a series through the
awful patchwork website doesn't make any sense in 2020 (or using any
other commandline tool btw).

But it's not the point here. 419 patches are waiting to be reviewed,
action must be taken.

If you move to github/gitlab you can test each one of the patches even
before starting a review and mark as change requested without human
intervention.
That's what we want. With the ML this is simply impossible.
We should aim to review patches that are already checked, that we know
are compiling for all the architectures, that respect some rules in
the commit message.
Many commit messages have syntax and grammar errors just because we
don't run languagetool on them.
Doing so, the quality of the review would only rise because most of
the work is already done by the system.
Moreover, doing a review on github/gitlab on a sane web interface with
all the bell and whistles is way easier for the most of the normal
people out there who want to contribute.

> > Buildroot is all about using simple and familiar tools like make and
> > Kconfig, and I personally think that this principle should also be applied
> > to the contribution process and right now buildroot is one of the last
> > active open source projects using the mailing list approach.
>
> This is a very biased statement: there are still plenty of open-source
> projects that use e-mail based contribution workflow. I don't think we
> can call Linux or U-Boot "inactive" projects. Can we? :-)

We cannot, but if you look closer, they are ones of the few last
projects to be based on ML. Everyone else moved years ago.

>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] More maintainers

[Adam Duskett]

Ciao;

On Thu, Sep 3, 2020 at 9:45 AM Angelo Compagnucci
<angelo.compagnucci@gmail.com> wrote:
>
> Il giorno gio 3 set 2020 alle ore 18:24 Thomas Petazzoni
> <thomas.petazzoni@bootlin.com> ha scritto:
> >
> > Hello Avraham,
> >
> > On Thu, 3 Sep 2020 18:44:27 +0300
> > Avraham Shukron <avraham.shukron@gmail.com> wrote:
> >
> > > For what it's worth I think that the mail-based contribution process is
> > > part of the problem.
> > > With a decent Git server like GitHub/GitLab patches could be reviewed more
> > > easily.
> > > A CI pipeline could run tests that will give immediate feedback for any
> > > pull request.
> > > More importantly, it'll dramatically reduce the barrier for new and young
> > > contributors.
> >
> > This has been discussed multiple times in the past in Buildroot, and in
> > other open-source projects. There is even as we speak some pretty
> > intense debate in the Linux kernel community about this.
> >
> > As we've seen from the discussion here, the Buildroot issue is not a
> > lack of contribution, but a lack of review from trusted and experienced
> > reviewers, and a lack of maintainers time. So while I'm all for new
> > contributors and contributions, I don't think reducing the barrier is
> > really key here. Also, I've always been skeptical about this statement
> > that using Github reduces the barrier to entry. When you contribute to
> > a project, is really sending a patch over e-mail the difficult part
> > compared to understanding the code base, debugging the issue you've
> > found or implementing the feature you wanted ? Really installing "git
> > send-email" is a no-brainer straightforward process that is
> > ridiculously easy even compared writing any single change in the code
> > base with a decent commit message. Aren't we using this "reducing the
> > barrier" argument a bit inappropriately here ?
> >
> > I believe I can say that all four Buildroot maintainers have a very
> > strong preference for and a very optimized workflow to work with e-mail
> > based patch submission and review.
> >
> > Somewhat related, recently a patch series I submitted last year to
> > OpenWrt (which wasn't merged) got picked up by someone else, and
> > re-submitted with new updates and fixes. Due to being the original
> > author, I was in copy of all the Github discussion that took place. And
> > I found it absolutely impossible and awful to follow the different
> > revisions of the patch series, to which version of the patch series the
> > comments were being made, etc.
> >
> > Perhaps for some people the Github pull request workflow makes sense,
> > but I believe it's important to recognize and realize that there are
> > also people for which this workflow doesn't make sense.
>
> I strongly disagree here.
Same.

> Having to write a patch changelog or download a series through the
> awful patchwork website doesn't make any sense in 2020 (or using any
> other commandline tool btw).
>
> But it's not the point here. 419 patches are waiting to be reviewed,
> action must be taken.
>
Also agreed.

> If you move to github/gitlab you can test each one of the patches even
> before starting a review and mark as change requested without human
> intervention.

This would reduce change requests by at least 50, if not 90%.

> That's what we want. With the ML this is simply impossible.
> We should aim to review patches that are already checked, that we know
> are compiling for all the architectures, that respect some rules in
> the commit message.

And having a simple CI/CD pipeline for these tasks would also reduce
the amount of time
AND the number of arbitrary opinions on styling and testing, which
would mean less
work for the few maintainers there are.

> Many commit messages have syntax and grammar errors just because we
> don't run languagetool on them.
> Doing so, the quality of the review would only rise because most of
> the work is already done by the system.
> Moreover, doing a review on github/gitlab on a sane web interface with
> all the bell and whistles is way easier for the most of the normal
> people out there who want to contribute.

I also agree with this.

>
> > > Buildroot is all about using simple and familiar tools like make and
> > > Kconfig, and I personally think that this principle should also be applied
> > > to the contribution process and right now buildroot is one of the last
> > > active open source projects using the mailing list approach.
> >
> > This is a very biased statement: there are still plenty of open-source
> > projects that use e-mail based contribution workflow. I don't think we
> > can call Linux or U-Boot "inactive" projects. Can we? :-)

The LKML is there because having a web interface for thousands of
contributions per week would
simply not be possible. Buildroot does not generate that kind of traffic.

Adam
>
> We cannot, but if you look closer, they are ones of the few last
> projects to be based on ML. Everyone else moved years ago.
>
> >
> > Thomas
> > --
> > Thomas Petazzoni, CTO, Bootlin
> > Embedded Linux and Kernel engineering
> > https://bootlin.com
> > _______________________________________________
> > buildroot mailing list
> > buildroot at busybox.net
> > http://lists.busybox.net/mailman/listinfo/buildroot
>
>
>
> --
> Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] More maintainers

[Nicolas Cavallari]

On 03/09/2020 19:29, Adam Duskett wrote:
>> That's what we want. With the ML this is simply impossible.
>> We should aim to review patches that are already checked, that we know
>> are compiling for all the architectures, that respect some rules in
>> the commit message.
> 
> And having a simple CI/CD pipeline for these tasks would also reduce
> the amount of time
> AND the number of arbitrary opinions on styling and testing, which
> would mean less
> work for the few maintainers there are.

Except this wouldn't change anything. checkpackage doesn't check much (judging
by the amount of changes the maintainers do to 80% of my patches),
and test-pkg cannot be run automatically, because it expects a config
with the package and its dependencies.

Plus, to be really useful, test-pkg must be run multiple times with various
dependencies and/or reverse dependencies, and at that point, this simply
reinvents the autobuilders.

Improving checkpackage and test-pkg would already benefit email-based workflows,
and it is perfectly possible to have a bot reply to a patch series if it doesn't
compile. This already happens in Linux.

And about the initial question: if buildroot need more maintainers or changes
its workflow, maybe it should look at how Linux distributions do things, rather
than how big projects do things.

[Buildroot] More maintainers

[Peter Korsgaard]

>>>>> "Angelo" == Angelo Compagnucci <angelo.compagnucci@gmail.com> writes:

Hi,

 >> Perhaps for some people the Github pull request workflow makes sense,
 >> but I believe it's important to recognize and realize that there are
 >> also people for which this workflow doesn't make sense.

 > I strongly disagree here.
 > Having to write a patch changelog or download a series through the
 > awful patchwork website doesn't make any sense in 2020 (or using any
 > other commandline tool btw).

Everyone is free to have their opinions, but arguing against command
line tools when Buildroot is exactly built on a bunch of command line
tools (make, shell, git, wget, ..) seems a bit counter productive.


 > But it's not the point here. 419 patches are waiting to be reviewed,
 > action must be taken.

Agreed, please help review patches.


 > If you move to github/gitlab you can test each one of the patches even
 > before starting a review and mark as change requested without human
 > intervention.

And why wouldn't you be able to do the same with the email based setup
today? With tools like snowpatch (https://github.com/ruscur/snowpatch) or
some basic scripting around the patchwork REST API nicely allows you to
run (command line :P) tools whenever new patches are posted.

The big problem is coming up with good automation and feedback that
helps. check-package, test-pkg and our runtime tests are quite basic,
but already a good start. Please help improving them.


 >> > Buildroot is all about using simple and familiar tools like make and
 >> > Kconfig, and I personally think that this principle should also be applied
 >> > to the contribution process and right now buildroot is one of the last
 >> > active open source projects using the mailing list approach.
 >> 
 >> This is a very biased statement: there are still plenty of open-source
 >> projects that use e-mail based contribution workflow. I don't think we
 >> can call Linux or U-Boot "inactive" projects. Can we? :-)

 > We cannot, but if you look closer, they are ones of the few last
 > projects to be based on ML. Everyone else moved years ago.

Blanket statements like "Everone" aren't very helpful. Furthermore,
projects are not created equal. In the majority of the projects I have
worked on, I had to make U-Boot or Linux kernel modifications and
contributions, but rarely on some random github project.

Anyway, lets keep it constructive here and improve the tooling we have
around the existing workflow rather some wild ideas of changing
everything which is not going to fly.

-- 
Bye, Peter Korsgaard

[Buildroot] More maintainers

[Michael Nazzareno Trimarchi]

Hi all

On Fri, Sep 4, 2020 at 7:39 PM Peter Korsgaard <peter@korsgaard.com> wrote:

> >>>>> "Angelo" == Angelo Compagnucci <angelo.compagnucci@gmail.com>
> writes:
>
> Hi,
>
>  >> Perhaps for some people the Github pull request workflow makes sense,
>  >> but I believe it's important to recognize and realize that there are
>  >> also people for which this workflow doesn't make sense.
>
>  > I strongly disagree here.
>  > Having to write a patch changelog or download a series through the
>  > awful patchwork website doesn't make any sense in 2020 (or using any
>  > other commandline tool btw).
>
> Everyone is free to have their opinions, but arguing against command
> line tools when Buildroot is exactly built on a bunch of command line
> tools (make, shell, git, wget, ..) seems a bit counter productive.
>
>
>
Agree on this but this is my personal opinion ;)

 > But it's not the point here. 419 patches are waiting to be reviewed,
>  > action must be taken.
>
> Agreed, please help review patches.
>
>
>  > If you move to github/gitlab you can test each one of the patches even
>  > before starting a review and mark as change requested without human
>  > intervention.
>
> And why wouldn't you be able to do the same with the email based setup
> today? With tools like snowpatch (https://github.com/ruscur/snowpatch) or
> some basic scripting around the patchwork REST API nicely allows you to
> run (command line :P) tools whenever new patches are posted.
>

 I think that we should explore using snowpatch. Problem is to have a setup
and some machines that can run the CI. Is there a cost to have this running?


> The big problem is coming up with good automation and feedback that
> helps. check-package, test-pkg and our runtime tests are quite basic,
> but already a good start. Please help improving them.
>
>
>  >> > Buildroot is all about using simple and familiar tools like make and
>  >> > Kconfig, and I personally think that this principle should also be
> applied
>  >> > to the contribution process and right now buildroot is one of the
> last
>  >> > active open source projects using the mailing list approach.
>  >>
>  >> This is a very biased statement: there are still plenty of open-source
>  >> projects that use e-mail based contribution workflow. I don't think we
>  >> can call Linux or U-Boot "inactive" projects. Can we? :-)
>
>  > We cannot, but if you look closer, they are ones of the few last
>  > projects to be based on ML. Everyone else moved years ago.
>
> Blanket statements like "Everone" aren't very helpful. Furthermore,
> projects are not created equal. In the majority of the projects I have
> worked on, I had to make U-Boot or Linux kernel modifications and
> contributions, but rarely on some random github project.
>
> Anyway, lets keep it constructive here and improve the tooling we have
> around the existing workflow rather some wild ideas of changing
> everything which is not going to fly.
>

Thank you
Peter


>
> --
> Bye, Peter Korsgaard
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
>


-- 
Michael Nazzareno Trimarchi
Amarula Solutions BV
COO Co-Founder
Cruquiuskade 47 Amsterdam 1018 AM NL
T. +31(0)851119172
M. +39(0)3479132170
[`as] https://www.amarulasolutions.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200904/1bbd0744/attachment.html>

[Buildroot] More maintainers

[Peter Korsgaard]

>>>>> "Michael" == Michael Nazzareno Trimarchi <michael@amarulasolutions.com> writes:

Hi,

 >> And why wouldn't you be able to do the same with the email based setup
 >> today? With tools like snowpatch (https://github.com/ruscur/snowpatch) or
 >> some basic scripting around the patchwork REST API nicely allows you to
 >> run (command line :P) tools whenever new patches are posted.
 >> 

 >  I think that we should explore using snowpatch. Problem is to have a setup
 > and some machines that can run the CI. Is there a cost to have this running?

Yes, it would need to run somewhere. Given the machine ressources spent
on the autobuilders and defconfigs/runtime tests already, we can probably
find ressources for that as well. Perhaps osuosl can help?

But before we can do that we would need to figure out exactly what tests
to do and what to report back to the submitter, E.G. running
check-package is trivial, but figuring out what build tests or what
runtime tests to run less so.

-- 
Bye, Peter Korsgaard

[Buildroot] More maintainers

[Angelo Compagnucci]

Il giorno ven 4 set 2020 alle ore 19:39 Peter Korsgaard
<peter@korsgaard.com> ha scritto:
>
> >>>>> "Angelo" == Angelo Compagnucci <angelo.compagnucci@gmail.com> writes:
>
> Hi,
>
>  >> Perhaps for some people the Github pull request workflow makes sense,
>  >> but I believe it's important to recognize and realize that there are
>  >> also people for which this workflow doesn't make sense.
>
>  > I strongly disagree here.
>  > Having to write a patch changelog or download a series through the
>  > awful patchwork website doesn't make any sense in 2020 (or using any
>  > other commandline tool btw).
>
> Everyone is free to have their opinions, but arguing against command
> line tools when Buildroot is exactly built on a bunch of command line
> tools (make, shell, git, wget, ..) seems a bit counter productive.

I think it was clear I was referring to the way we manage patches via
ML/patchwork.
Doing a "git push" or "repo upload" to open a review request is
perfectly fine on my side.

>  > But it's not the point here. 419 patches are waiting to be reviewed,
>  > action must be taken.
>
> Agreed, please help review patches.

I do what I can, you do what you can, others do what they can, we
anyway have this big number of patches piling up.
I still think we should improve the workflow or we can continue to not
seeing the problem and hope someone will hop in sooner or later.
But this is only my pov.

I think that having an extensive CI that runs on each commit would
only benefit the work of the reviewers and the project as a whole.

>  > If you move to github/gitlab you can test each one of the patches even
>  > before starting a review and mark as change requested without human
>  > intervention.
>
> And why wouldn't you be able to do the same with the email based setup
> today? With tools like snowpatch (https://github.com/ruscur/snowpatch) or
> some basic scripting around the patchwork REST API nicely allows you to
> run (command line :P) tools whenever new patches are posted.

Honestly, we can do that, but you really want to setup and maintain a
complex infrastructure like that when we have gitlab with all the
features we need?
Do you really think we have the workforce to setup/hack/maintain that
custom infrastructure?
Buildroot is my OSS preferred project, but I don't think we have as
many people as the Linux kernel to implement a custom workflow.

Probably finding some sponsorship to hack the system, but again, why
disregard the sponsorship of gitlab that offers the full package for
free to OSS projects?

> The big problem is coming up with good automation and feedback that
> helps. check-package, test-pkg and our runtime tests are quite basic,
> but already a good start. Please help improving them.

They are not good because we don't care about continuous integration
on each review request.
If we do, it will be natural to invest time to setup all the best
checking rules we came up with, due to the fact that rules run on each
review request and we know the value of that.

>  >> > Buildroot is all about using simple and familiar tools like make and
>  >> > Kconfig, and I personally think that this principle should also be applied
>  >> > to the contribution process and right now buildroot is one of the last
>  >> > active open source projects using the mailing list approach.
>  >>
>  >> This is a very biased statement: there are still plenty of open-source
>  >> projects that use e-mail based contribution workflow. I don't think we
>  >> can call Linux or U-Boot "inactive" projects. Can we? :-)
>
>  > We cannot, but if you look closer, they are ones of the few last
>  > projects to be based on ML. Everyone else moved years ago.
>
> Blanket statements like "Everone" aren't very helpful. Furthermore,
> projects are not created equal. In the majority of the projects I have
> worked on, I had to make U-Boot or Linux kernel modifications and
> contributions, but rarely on some random github project.

Probably is my impression only, but conservatively 80% of the commits
I do on oss projects are on gerrit, github or gitlab.

> Anyway, lets keep it constructive here and improve the tooling we have
> around the existing workflow rather some wild ideas of changing
> everything which is not going to fly.

Yeah, I know and it's sad it's not going to fly. We don't have the
workforce to do reviews, let's hope someone improve the tooling we
have.

I'm still of the idea that moving to a modern ready-made and
hassle-free CI/CD environment would only benefit the project as a
whole, spare some time on the maintainers and rise the contribution
quality.

I do really love Buildroot, trust me, really. I would only see it
growing and striking at a fast pace and for this reason, I think the
tooling is a very very big part of its success.

>
> --
> Bye, Peter Korsgaard



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] More maintainers

[Peter Korsgaard]

>>>>> "Angelo" == Angelo Compagnucci <angelo.compagnucci@gmail.com> writes:

Hi,

 > I think it was clear I was referring to the way we manage patches via
 > ML/patchwork.
 > Doing a "git push" or "repo upload" to open a review request is
 > perfectly fine on my side.

Ehh, ok. I still don't see the big advantage of git push vs git
send-email, but ok.


 >> > But it's not the point here. 419 patches are waiting to be reviewed,
 >> > action must be taken.
 >> 
 >> Agreed, please help review patches.

 > I do what I can, you do what you can, others do what they can, we
 > anyway have this big number of patches piling up.
 > I still think we should improve the workflow or we can continue to not
 > seeing the problem and hope someone will hop in sooner or later.
 > But this is only my pov.

I think it is unrealistic to think that review bandwidth won't always be
an issue. Given the meta nature (a build system of buildsystems) of
Buildroot, no automation will ever be a substitute for real human
review.



 > I think that having an extensive CI that runs on each commit would
 > only benefit the work of the reviewers and the project as a whole.

 >> > If you move to github/gitlab you can test each one of the patches even
 >> > before starting a review and mark as change requested without human
 >> > intervention.
 >> 
 >> And why wouldn't you be able to do the same with the email based setup
 >> today? With tools like snowpatch (https://github.com/ruscur/snowpatch) or
 >> some basic scripting around the patchwork REST API nicely allows you to
 >> run (command line :P) tools whenever new patches are posted.

 > Honestly, we can do that, but you really want to setup and maintain a
 > complex infrastructure like that when we have gitlab with all the
 > features we need?

The complicated part is the actual CI logic. The interaction with
patchwork and email seems to me to be quite small effort (and largely
done already with snowpatch), and for running it, the difference between
gitlab with gitlab runners that we maintain ourselves or something like
this seems pretty small.


 > Probably finding some sponsorship to hack the system, but again, why
 > disregard the sponsorship of gitlab that offers the full package for
 > free to OSS projects?

I am not sure what you are getting at. We already use gitlab (+ our
runners) for defconfig and runtime tests. The only thing we are not
using is the pull request webui.


 >> The big problem is coming up with good automation and feedback that
 >> helps. check-package, test-pkg and our runtime tests are quite basic,
 >> but already a good start. Please help improving them.

 > They are not good because we don't care about continuous integration
 > on each review request.
 > If we do, it will be natural to invest time to setup all the best
 > checking rules we came up with, due to the fact that rules run on each
 > review request and we know the value of that.

check-package runs every time a maintainer applies a patch, and is often
used by contributors, so I doubt that is really the reason - I rather
think it is because it is quite a difficult problem.

This is also why we build random configurations in the autobuilder
rather than E.G. running specific tests for each package as the number
of variables are simply too big.


 >> Anyway, lets keep it constructive here and improve the tooling we have
 >> around the existing workflow rather some wild ideas of changing
 >> everything which is not going to fly.

 > Yeah, I know and it's sad it's not going to fly. We don't have the
 > workforce to do reviews, let's hope someone improve the tooling we
 > have.

 > I'm still of the idea that moving to a modern ready-made and
 > hassle-free CI/CD environment would only benefit the project as a
 > whole, spare some time on the maintainers and rise the contribution
 > quality.

Yes, and I would like a pony ;)

How would such CI setup work exactly. Lets take a random patch from
patchwork, E.G.:

https://patchwork.ozlabs.org/project/buildroot/patch/20200825000246.4035-1-joseph.kogut at gmail.com/

What should the CI exactly do to make human review unneeded? Sure, we
can come up with some more style checks like in check-package, but those
issues are NOT the reason why we have a bunch of pending patches.


So yes, lets please discuss concrete improvements to the automation
checks we already have or ways to get more people to help review rather
than yet another discussion about the merits of pull requests versus
emails.

-- 
Bye, Peter Korsgaard

[Buildroot] More maintainers

[Christian Stewart]

Hi Peter,


On Fri, Sep 4, 2020 at 2:10 PM Peter Korsgaard <peter@korsgaard.com> wrote:
> I think it is unrealistic to think that review bandwidth won't always be
> an issue. Given the meta nature (a build system of buildsystems) of
> Buildroot, no automation will ever be a substitute for real human
> review.

"No robotic pilot can ever beat a human!"

I think you're right about architectural changes, there's probably
some maintenance tasks that could be fully automated (at least for
approval) with some rules. The most simple example would be upgrades:
auto bump version + hash -> e2e test in Docker container -> manual
review period of 2-3 days -> merge. If any issue in this process, open
a bug report w/ the results.

This is how tools like "Dependabot" and "Greenkeeper" work today for
other projects. We could opt-in packages for automated bump testing -
for docker-engine it would be fine for example.

> So yes, lets please discuss concrete improvements to the automation
> checks we already have or ways to get more people to help review rather
> than yet another discussion about the merits of pull requests versus
> emails.

One thing I'm working on right now is automated testing on
cross-compile targets - like Odroid, Raspberry Pi, etc, with network
boot + watchdog of course.

Probably with a very extensive test matrix, it could be possible to
allow very small patches to be merged automatically to "next" with a
qualifying # of reviewed-by?

Best regards,
Christian

[Buildroot] More maintainers

[Peter Korsgaard]

>>>>> "Christian" == Christian Stewart <christian@paral.in> writes:

 > Hi Peter,
 > On Fri, Sep 4, 2020 at 2:10 PM Peter Korsgaard <peter@korsgaard.com> wrote:
 >> I think it is unrealistic to think that review bandwidth won't always be
 >> an issue. Given the meta nature (a build system of buildsystems) of
 >> Buildroot, no automation will ever be a substitute for real human
 >> review.

 > "No robotic pilot can ever beat a human!"

 > I think you're right about architectural changes, there's probably
 > some maintenance tasks that could be fully automated (at least for
 > approval) with some rules. The most simple example would be upgrades:
 > auto bump version + hash -> e2e test in Docker container -> manual
 > review period of 2-3 days -> merge. If any issue in this process, open
 > a bug report w/ the results.

Possibly, but what would that solve? We are not really lacking in
version bumps, are we? And trivial version bumps are typically applied
very fast.

The issue for any such testing is the number of combinations to get good
test coverage is very big, which is why we have the autobuilders to
catch things like "bumping foo breaks building bar when package x is
enabled and building for architecture y with toolchain version z" (an
extreme example, but it does happen).

I don't think adding more automation for these trivial version bumps
will make any significant change to our backlog.

But sure, we could do the basics, E.G. check-package and perhaps even
building a basic config. That could cover basic issues and would have
the advantage that the contributor gets fast feedback. It naturally also
means quite a bit more mails on the list (E.G. to send a reviewed-by or
any issues found).


 >> So yes, lets please discuss concrete improvements to the automation
 >> checks we already have or ways to get more people to help review rather
 >> than yet another discussion about the merits of pull requests versus
 >> emails.

 > One thing I'm working on right now is automated testing on
 > cross-compile targets - like Odroid, Raspberry Pi, etc, with network
 > boot + watchdog of course.

 > Probably with a very extensive test matrix, it could be possible to
 > allow very small patches to be merged automatically to "next" with a
 > qualifying # of reviewed-by?

More testing is certainly good, but aren't those "very small patches"
already getting applied to next pretty fast? Actually applying a patch
is a single git command so that doesn't take a long time.

-- 
Bye, Peter Korsgaard

[Buildroot] More maintainers

[Adam Duskett]

On Fri, Sep 4, 2020 at 11:44 PM Peter Korsgaard <peter@korsgaard.com> wrote:
>
> >>>>> "Christian" == Christian Stewart <christian@paral.in> writes:
>
>  > Hi Peter,
>  > On Fri, Sep 4, 2020 at 2:10 PM Peter Korsgaard <peter@korsgaard.com> wrote:
>  >> I think it is unrealistic to think that review bandwidth won't always be
>  >> an issue. Given the meta nature (a build system of buildsystems) of
>  >> Buildroot, no automation will ever be a substitute for real human
>  >> review.
>
>  > "No robotic pilot can ever beat a human!"
>
>  > I think you're right about architectural changes, there's probably
>  > some maintenance tasks that could be fully automated (at least for
>  > approval) with some rules. The most simple example would be upgrades:
>  > auto bump version + hash -> e2e test in Docker container -> manual
>  > review period of 2-3 days -> merge. If any issue in this process, open
>  > a bug report w/ the results.
>
> Possibly, but what would that solve? We are not really lacking in
> version bumps, are we? And trivial version bumps are typically applied
> very fast.
>
> The issue for any such testing is the number of combinations to get good
> test coverage is very big, which is why we have the autobuilders to
> catch things like "bumping foo breaks building bar when package x is
> enabled and building for architecture y with toolchain version z" (an
> extreme example, but it does happen).
>
> I don't think adding more automation for these trivial version bumps
> will make any significant change to our backlog.
>
> But sure, we could do the basics, E.G. check-package and perhaps even
> building a basic config. That could cover basic issues and would have
> the advantage that the contributor gets fast feedback. It naturally also
> means quite a bit more mails on the list (E.G. to send a reviewed-by or
> any issues found).
>
>
>  >> So yes, lets please discuss concrete improvements to the automation
>  >> checks we already have or ways to get more people to help review rather
>  >> than yet another discussion about the merits of pull requests versus
>  >> emails.
>

Commit 9faba29108e74eb4acab21f5919dfab0288b23ac broke systemd with
busybox in the stock Buildroot configuration.

An incredibly simple test:
Download the tarball
select systemd
make

Not only was this incredibly embarrassing yesterday when I was showing a
client how to update Buildroot, but it could have been prevented with a basic
CI/CD test. I don't have to mention how much guff I would have received from
the maintainers if I had submitted a patch like that without a simple test.

>  > One thing I'm working on right now is automated testing on
>  > cross-compile targets - like Odroid, Raspberry Pi, etc, with network
>  > boot + watchdog of course.
>
>  > Probably with a very extensive test matrix, it could be possible to
>  > allow very small patches to be merged automatically to "next" with a
>  > qualifying # of reviewed-by?
>
> More testing is certainly good, but aren't those "very small patches"
> already getting applied to next pretty fast? Actually applying a patch
> is a single git command so that doesn't take a long time.
>
I have had several small patches in the past sit for months if not almost years
before being applied. Including an OpenJDK bump that took over 3 months.

In fact, just looking at the list right now, a simple patch that has
been reviewed
AND tested is still sitting from 2019:
https://patchwork.ozlabs.org/project/buildroot/patch/20191029095736.16586-1-fontaine.fabrice at gmail.com/

Here's another:
https://patchwork.ozlabs.org/project/buildroot/patch/20191109185512.24139-2-jeremy.rosen at smile.fr/

Here's a simple test:
https://patchwork.ozlabs.org/project/buildroot/patch/20191112162325.529637-2-m.niestroj at grinn-global.com/

So this is demonstrably false, which is why a lot of us are growing
more frustrated.

Adam
> --
> Bye, Peter Korsgaard

[Buildroot] More maintainers

[Peter Korsgaard]

>>>>> "Adam" == Adam Duskett <aduskett@gmail.com> writes:

Hi,

 >> >> So yes, lets please discuss concrete improvements to the automation
 >> >> checks we already have or ways to get more people to help review rather
 >> >> than yet another discussion about the merits of pull requests versus
 >> >> emails.
 >> 

 > Commit 9faba29108e74eb4acab21f5919dfab0288b23ac broke systemd with
 > busybox in the stock Buildroot configuration.

Indeed, like discussed last week:

http://lists.busybox.net/pipermail/buildroot/2020-August/290576.html

And announced as a reply to the 2020.02.5 announcment:

http://lists.busybox.net/pipermail/buildroot/2020-August/290584.html

 > An incredibly simple test:
 > Download the tarball
 > select systemd
 > make

 > Not only was this incredibly embarrassing yesterday when I was showing a
 > client how to update Buildroot, but it could have been prevented with a basic
 > CI/CD test.

Improvements to our test automation are always welcome. This has still
not triggered on the autobuilders:

http://autobuild.buildroot.net/?branch=2020.02.x&reason=busybox-%


 > I don't have to mention how much guff I would have received from
 > the maintainers if I had submitted a patch like that without a simple test.

Accidents happen, certainly with such hidden dependencies - Which is
part of the reason we have the autobuilders. For all the ~40 LTS
releases I have done, this is afaik the first time we had such an issue.


 >> > Probably with a very extensive test matrix, it could be possible to
 >> > allow very small patches to be merged automatically to "next" with a
 >> > qualifying # of reviewed-by?
 >> 
 >> More testing is certainly good, but aren't those "very small patches"
 >> already getting applied to next pretty fast? Actually applying a patch
 >> is a single git command so that doesn't take a long time.
 >> 
 > I have had several small patches in the past sit for months if not almost years
 > before being applied. Including an OpenJDK bump that took over 3 months.

 > In fact, just looking at the list right now, a simple patch that has
 > been reviewed
 > AND tested is still sitting from 2019:
 > https://patchwork.ozlabs.org/project/buildroot/patch/20191029095736.16586-1-fontaine.fabrice at gmail.com/

 > Here's another:
 > https://patchwork.ozlabs.org/project/buildroot/patch/20191109185512.24139-2-jeremy.rosen at smile.fr/

That patch series was discussed in detail in Lyon last year, and should
probably just be marked as rejected.


 > Here's a simple test:
 > https://patchwork.ozlabs.org/project/buildroot/patch/20191112162325.529637-2-m.niestroj at grinn-global.com/

 > So this is demonstrably false, which is why a lot of us are growing
 > more frustrated.

Nobody is claiming that we are perfect or that there isn't a patch
backlog, but given that we merge ~500 patches/month I still would say
that it IS generally true.

Anyway, Thomas has sent out a mail for a live discussion for next week,
so lets take the discussion there.

-- 
Bye, Peter Korsgaard

[Buildroot] More maintainers

[Thomas Petazzoni]

Hello Adam,

Thanks for your participation in this discussion.

On Sat, 5 Sep 2020 09:20:30 -0700
Adam Duskett <aduskett@gmail.com> wrote:

> >  >> So yes, lets please discuss concrete improvements to the automation
> >  >> checks we already have or ways to get more people to help review rather
> >  >> than yet another discussion about the merits of pull requests versus
> >  >> emails.  
> >  
> 
> Commit 9faba29108e74eb4acab21f5919dfab0288b23ac broke systemd with
> busybox in the stock Buildroot configuration.
> 
> An incredibly simple test:
> Download the tarball
> select systemd
> make
> 
> Not only was this incredibly embarrassing yesterday when I was showing a
> client how to update Buildroot, but it could have been prevented with a basic
> CI/CD test.

It is indeed embarrassing, but your statement that it "would have been
prevented with basic CI/CD test" is not completely correct.

Indeed:

 (1) We are already doing CI/CD: all runtime tests and all defconfigs
     are built/executed regularly on Gitlab CI. We currently do that
     once a week, and it also happens whenever a tag is pushed. For
     example, the 2020.05.2 tag was tested by Gitlab CI, and we have a
     number of failures:

     https://gitlab.com/buildroot.org/buildroot/-/pipelines/183457594/failures

     So: CI is *already* happening.

 (2) Unfortunately, the CPU effort needed to build all our defconfigs
     and run all our runtime tests is very significant, and therefore,
     it is not possible to run all this CI testing on each and every
     commit, and even less so on each and every patch series/pull
     request that is being submitted.

Several of you are calling for more CI, and that's obviously very good.
But what would be even better is to define more precisely which
additional CI you're talking about that: (a) we're not already doing
and (b) is reasonably possible to do.

Could you describe that ?

> I don't have to mention how much guff I would have received from
> the maintainers if I had submitted a patch like that without a simple
> test.

I agree that you've received some pretty strong feedback on some of
your patches. But I don't think we should confuse two different
situations.

Commit 9faba29108e74eb4acab21f5919dfab0288b23ac in the 2020.02.x branch
is a backport of c2caf816e9296c4812f83f0a5d16e0e33a264b72 in master.
The commit in master works fine, and what was missed is that when
backporting, the KCONFIG_ENABLE_OPT calls should have been adjusted.
That is indeed a mistake. However, Peter is the only person backporting
*all* patches to stable/LTS branches, so he is obviously doing that as
best as he can, and relies on autobuilder testing.

To me, this is a bit different than a new package being submitted,
which in its basic configuration, doesn't even build with our default C
library. There is an obvious expectation from us as maintainers, that
experienced contributors will submit new packages that have been tested
with our default C library.

Your recent libblockdev submission had this issue, for example. And
even though it was not great, I hope you noticed that I fixed the
issues myself, in order to make progress with the series and not
require another iteration.

> I have had several small patches in the past sit for months if not
> almost years before being applied. Including an OpenJDK bump that
> took over 3 months.

If there are obviously simple patch that are ready to be applied,
please let us know. I've very often applied patches within minutes
after someone points me on IRC or by e-mail to a particular patch.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] More maintainers

[Angelo Compagnucci]

Il giorno sab 5 set 2020 alle ore 21:16 Thomas Petazzoni
<thomas.petazzoni@bootlin.com> ha scritto:
>
> Hello Adam,
>
> Thanks for your participation in this discussion.
>
> On Sat, 5 Sep 2020 09:20:30 -0700
> Adam Duskett <aduskett@gmail.com> wrote:
>
> > >  >> So yes, lets please discuss concrete improvements to the automation
> > >  >> checks we already have or ways to get more people to help review rather
> > >  >> than yet another discussion about the merits of pull requests versus
> > >  >> emails.
> > >
> >
> > Commit 9faba29108e74eb4acab21f5919dfab0288b23ac broke systemd with
> > busybox in the stock Buildroot configuration.
> >
> > An incredibly simple test:
> > Download the tarball
> > select systemd
> > make
> >
> > Not only was this incredibly embarrassing yesterday when I was showing a
> > client how to update Buildroot, but it could have been prevented with a basic
> > CI/CD test.
>
> It is indeed embarrassing, but your statement that it "would have been
> prevented with basic CI/CD test" is not completely correct.
>
> Indeed:
>
>  (1) We are already doing CI/CD: all runtime tests and all defconfigs
>      are built/executed regularly on Gitlab CI. We currently do that
>      once a week, and it also happens whenever a tag is pushed. For
>      example, the 2020.05.2 tag was tested by Gitlab CI, and we have a
>      number of failures:
>
>      https://gitlab.com/buildroot.org/buildroot/-/pipelines/183457594/failures
>
>      So: CI is *already* happening.
>
>  (2) Unfortunately, the CPU effort needed to build all our defconfigs
>      and run all our runtime tests is very significant, and therefore,
>      it is not possible to run all this CI testing on each and every
>      commit, and even less so on each and every patch series/pull
>      request that is being submitted.
>
> Several of you are calling for more CI, and that's obviously very good.
> But what would be even better is to define more precisely which
> additional CI you're talking about that: (a) we're not already doing
> and (b) is reasonably possible to do.

Honestly, I can't see a CI here or we have a big misunderstanding: CI
runs on review requests, not on the committed code.

> Could you describe that ?

Simple:

* A Developer opens a review request: the author only is notified, the
CI starts running.
* Commit has style problems, the author only is notified, red flag
* Commit message has grammar/syntax errors, the author only is
notified, red flag
* Commit doesn't have a minimal defconfig to build on all
architectures where it is supported: the author only is notified, red
flag
* Commit compiles on some architectures and not on others: the author
only is notified, red flag
* Commit doesn't have a runtime test attached: the author only is
notified, red flag
* Commit has testing but testing fails, the author only is notified, red flag
* Commit pass al the requirements: ML is notified, the commit is now
ready for a review, green flag.

Imho, this is how a CI should work.

That way, the autobuild could be even superfluous because each commit
was already tested by CI. Yes, autobuild could catch some more
intricate problems, but it can run less frequently.

>
> > I don't have to mention how much guff I would have received from
> > the maintainers if I had submitted a patch like that without a simple
> > test.
>
> I agree that you've received some pretty strong feedback on some of
> your patches. But I don't think we should confuse two different
> situations.
>
> Commit 9faba29108e74eb4acab21f5919dfab0288b23ac in the 2020.02.x branch
> is a backport of c2caf816e9296c4812f83f0a5d16e0e33a264b72 in master.
> The commit in master works fine, and what was missed is that when
> backporting, the KCONFIG_ENABLE_OPT calls should have been adjusted.
> That is indeed a mistake. However, Peter is the only person backporting
> *all* patches to stable/LTS branches, so he is obviously doing that as
> best as he can, and relies on autobuilder testing.
>
> To me, this is a bit different than a new package being submitted,
> which in its basic configuration, doesn't even build with our default C
> library. There is an obvious expectation from us as maintainers, that
> experienced contributors will submit new packages that have been tested
> with our default C library.
>
> Your recent libblockdev submission had this issue, for example. And
> even though it was not great, I hope you noticed that I fixed the
> issues myself, in order to make progress with the series and not
> require another iteration.
>
> > I have had several small patches in the past sit for months if not
> > almost years before being applied. Including an OpenJDK bump that
> > took over 3 months.
>
> If there are obviously simple patch that are ready to be applied,
> please let us know. I've very often applied patches within minutes
> after someone points me on IRC or by e-mail to a particular patch.
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] More maintainers

[Thomas Petazzoni]

Hello Angelo,

On Sat, 5 Sep 2020 22:10:51 +0200
Angelo Compagnucci <angelo.compagnucci@gmail.com> wrote:

> Honestly, I can't see a CI here or we have a big misunderstanding: CI
> runs on review requests, not on the committed code.

As I've explained, we simply can't do that in Buildroot: running just
our current CI tests requires several hours of CPU time.

> Simple:
> 
> * A Developer opens a review request: the author only is notified, the
> CI starts running.
> * Commit has style problems, the author only is notified, red flag
> * Commit message has grammar/syntax errors, the author only is
> notified, red flag
> * Commit doesn't have a minimal defconfig to build on all
> architectures where it is supported: the author only is notified, red
> flag

I'm not sure what you mean here. You would require all commits to come
with a snippet of configuration that allows the commit to be build
tested ?

> * Commit compiles on some architectures and not on others: the author
> only is notified, red flag

There is no such thing as "Commit compiles" in the context of
Buildroot. Packages have optional dependencies, we have 3 C libraries,
we have 15 CPU architectures. How are you going to test all of those
combinations, for all patches that are submitted ?

If you have a magic solution for this, I'm interested!

> * Commit doesn't have a runtime test attached: the author only is
> notified, red flag

How would a runtime test be "attached" to a commit?

> * Commit has testing but testing fails, the author only is notified, red flag

What does "Commit has testing" means ?

> Imho, this is how a CI should work.

See above all questions that are raised.

In all what you have described, the difficult point is not e-mail vs.
Github/Gitlab, it is to define and implement the CI jobs/tests that you
are describing.

> That way, the autobuild could be even superfluous because each commit
> was already tested by CI. Yes, autobuild could catch some more
> intricate problems, but it can run less frequently.

See above: "commit compiles" is not something that can be answered
within a bounded time in the context of Buildroot.

I think you are confusing the type of CI that can be done on an
application or library, where indeed it is possible for every commit to
run the build on all supported platforms, run the unit tests, and have
good confidence that all combinations have been tested.

In the context of Buildroot, saying "a commit is good" requires
building thousands of different configurations: number of CPU
architectures * number of GCC versions * number of binutils versions *
number of C libraries * number of optional dependencies of the package
* number of optional dependencies of all dependencies of the package.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] More maintainers

[Angelo Compagnucci]

Il giorno sab 5 set 2020 alle ore 22:38 Thomas Petazzoni
<thomas.petazzoni@bootlin.com> ha scritto:
>
> Hello Angelo,
>
> On Sat, 5 Sep 2020 22:10:51 +0200
> Angelo Compagnucci <angelo.compagnucci@gmail.com> wrote:
>
> > Honestly, I can't see a CI here or we have a big misunderstanding: CI
> > runs on review requests, not on the committed code.
>
> As I've explained, we simply can't do that in Buildroot: running just
> our current CI tests requires several hours of CPU time.
>
> > Simple:
> >
> > * A Developer opens a review request: the author only is notified, the
> > CI starts running.
> > * Commit has style problems, the author only is notified, red flag
> > * Commit message has grammar/syntax errors, the author only is
> > notified, red flag
> > * Commit doesn't have a minimal defconfig to build on all
> > architectures where it is supported: the author only is notified, red
> > flag
>
> I'm not sure what you mean here. You would require all commits to come
> with a snippet of configuration that allows the commit to be build
> tested ?

Yes, exactly.
>
> > * Commit compiles on some architectures and not on others: the author
> > only is notified, red flag
>
> There is no such thing as "Commit compiles" in the context of
> Buildroot. Packages have optional dependencies, we have 3 C libraries,
> we have 15 CPU architectures. How are you going to test all of those
> combinations, for all patches that are submitted ?
>
> If you have a magic solution for this, I'm interested!

Well, I'm completely sure of that.
Anyway, in the context of doing buildable and testable review
requests, the developer should attach to the commit a minimal
defconfig and minimal testing.
That minimal defconfig and that testing are chosen by the developer to
enhance confidence that the request is working.

> > * Commit doesn't have a runtime test attached: the author only is
> > notified, red flag
>
> How would a runtime test be "attached" to a commit?
>
> > * Commit has testing but testing fails, the author only is notified, red flag
>
> What does "Commit has testing" means ?
>
> > Imho, this is how a CI should work.
>
> See above all questions that are raised.
>
> In all what you have described, the difficult point is not e-mail vs.
> Github/Gitlab, it is to define and implement the CI jobs/tests that you
> are describing.
>
> > That way, the autobuild could be even superfluous because each commit
> > was already tested by CI. Yes, autobuild could catch some more
> > intricate problems, but it can run less frequently.
>
> See above: "commit compiles" is not something that can be answered
> within a bounded time in the context of Buildroot.
>
> I think you are confusing the type of CI that can be done on an
> application or library, where indeed it is possible for every commit to
> run the build on all supported platforms, run the unit tests, and have
> good confidence that all combinations have been tested.
>
> In the context of Buildroot, saying "a commit is good" requires
> building thousands of different configurations: number of CPU
> architectures * number of GCC versions * number of binutils versions *
> number of C libraries * number of optional dependencies of the package
> * number of optional dependencies of all dependencies of the package.

I'm not confusing, really, and from what I learned on Computer
Science, that type of testing you're saying here is not even possile.

What I'm saying here is that we should and must enforce a minimal "it
compiles" verification and "it runs" verification.

Recent history says that we had packages with a service never started
because the init script was wrong, we should avoid that.

Doing postmortem autopsy with autobuild is not going to raise the
review requests quality or lower the pressure on the maintainers.

>
> Best regards,
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] More maintainers

[Thomas Petazzoni]

Hello,

On Sat, 5 Sep 2020 22:55:11 +0200
Angelo Compagnucci <angelo.compagnucci@gmail.com> wrote:

> > I'm not sure what you mean here. You would require all commits to come
> > with a snippet of configuration that allows the commit to be build
> > tested ?  
> 
> Yes, exactly.

This actually raises the barrier to entry even more than an e-mail
based workflow! Newcomers have to understand this practice, what a
configuration snippet is, etc. It's not super complex, but it's not
trivial either.

Also, it's easy to get this snippet wrong, and have the automated
testing not actually test anything.

Also, the amount of CPU time that will be needed remains quite
significant. Another thing we could do is require contributors to
attach the output of a test-pkg run.

But overall, the buildability of something is not really the primary
reason for the review taking time. Indeed, I typically never build test
version bumps, or even simple Python packages, I let the autobuilders
do this job. For new packages, I tend to have a different approach
depending on who the submitter is: experienced contributors are more
likely to know how to properly test their submissions, while newcomers
are more likely to make mistakes (which is normal!), so I tend to test
a bit more contributions from newcomers.

> > If you have a magic solution for this, I'm interested!  
> 
> Well, I'm completely sure of that.
> Anyway, in the context of doing buildable and testable review
> requests, the developer should attach to the commit a minimal
> defconfig and minimal testing.
> That minimal defconfig and that testing are chosen by the developer to
> enhance confidence that the request is working.

Well, it is generally the case that the developer has tested... but
that doesn't mean that this testing was good.

Take the example of libblockdev, recently submitted by Adam. Adam
probably built it with a glibc toolchain, and it worked all fine for
him. And when I tested his libblockdev package, I used what is the
default C library in Buildroot, i.e uClibc, and it failed to build;

As you can see, a minimal defconfig + testing chosen by the developer
is no silver bullet.

> > In the context of Buildroot, saying "a commit is good" requires
> > building thousands of different configurations: number of CPU
> > architectures * number of GCC versions * number of binutils versions *
> > number of C libraries * number of optional dependencies of the package
> > * number of optional dependencies of all dependencies of the package.  
> 
> I'm not confusing, really, and from what I learned on Computer
> Science, that type of testing you're saying here is not even possile.

Glad we agree that it is not possible to do such testing :-)

> What I'm saying here is that we should and must enforce a minimal "it
> compiles" verification and "it runs" verification.

But this is already done by the autobuilders, and works very well: we
as maintainers rely on the autobuilders a lot to tell us if something
is wrong.

Let's take an example: I just pushed the commit adding the
"makedumpfile" package. It has been submitted by a newcomer, so I did
some local build testing, but with only one architecture, one
toolchain, one configuration. It built in this configuration locally,
so I pushed. All the rest of the testing will be done by the
autobuilders.

I just pushed the python-crayons package. I have done zero build
testing. The only thing I've checked is verifying the legal information
in the package, and this cannot be automated.

> Recent history says that we had packages with a service never started
> because the init script was wrong, we should avoid that.

We already have the infrastructure for that: runtime tests. They exist,
they are run by Gitlab CI.

Please submit more runtime tests :-)

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] More maintainers

[Angelo Compagnucci]

Il giorno sab 5 set 2020 alle ore 23:04 Thomas Petazzoni
<thomas.petazzoni@bootlin.com> ha scritto:
>
> Hello,
>
> On Sat, 5 Sep 2020 22:55:11 +0200
> Angelo Compagnucci <angelo.compagnucci@gmail.com> wrote:
>
> > > I'm not sure what you mean here. You would require all commits to come
> > > with a snippet of configuration that allows the commit to be build
> > > tested ?
> >
> > Yes, exactly.
>
> This actually raises the barrier to entry even more than an e-mail
> based workflow! Newcomers have to understand this practice, what a
> configuration snippet is, etc. It's not super complex, but it's not
> trivial either.

Probably, less contribution of more quality, aren't we talking about
lowering the pending patches on patchwork ;) ?

> Also, it's easy to get this snippet wrong, and have the automated
> testing not actually test anything.

Mmm, not if the checking system is configured properly.

>
> Also, the amount of CPU time that will be needed remains quite
> significant. Another thing we could do is require contributors to
> attach the output of a test-pkg run.
>
> But overall, the buildability of something is not really the primary
> reason for the review taking time. Indeed, I typically never build test
> version bumps, or even simple Python packages, I let the autobuilders
> do this job. For new packages, I tend to have a different approach
> depending on who the submitter is: experienced contributors are more
> likely to know how to properly test their submissions, while newcomers
> are more likely to make mistakes (which is normal!), so I tend to test
> a bit more contributions from newcomers.

Stealing time for applying patches, doing other reviews or live your life.
Why do you have to test something if the system have already tested for you?

>
> > > If you have a magic solution for this, I'm interested!
> >
> > Well, I'm completely sure of that.
> > Anyway, in the context of doing buildable and testable review
> > requests, the developer should attach to the commit a minimal
> > defconfig and minimal testing.
> > That minimal defconfig and that testing are chosen by the developer to
> > enhance confidence that the request is working.
>
> Well, it is generally the case that the developer has tested... but
> that doesn't mean that this testing was good.

Not if that minimal defconfig is tested against some/all architectures
by the system.
Or alternatively, because that defconfig is pushed with the commit
somewhere, we can update that defconfig to reflect the knowledge we
gain when something breaks,

> Take the example of libblockdev, recently submitted by Adam. Adam
> probably built it with a glibc toolchain, and it worked all fine for
> him. And when I tested his libblockdev package, I used what is the
> default C library in Buildroot, i.e uClibc, and it failed to build;
>
> As you can see, a minimal defconfig + testing chosen by the developer
> is no silver bullet.

No, but you can ask to add a snippet more or another test, and move
back the patch is on the shoulders of the developer and the review
system.

In the end, when every check passes, you will only have to scan the
patch, give you suggestions, ask to change here and there while being
sure that even the new patchset will be built and tested by the system
for you.

How about a resubmitted patchset that introduced a bug went unnoticed?
The could be caught.

> > > In the context of Buildroot, saying "a commit is good" requires
> > > building thousands of different configurations: number of CPU
> > > architectures * number of GCC versions * number of binutils versions *
> > > number of C libraries * number of optional dependencies of the package
> > > * number of optional dependencies of all dependencies of the package.
> >
> > I'm not confusing, really, and from what I learned on Computer
> > Science, that type of testing you're saying here is not even possile.
>
> Glad we agree that it is not possible to do such testing :-)
>
> > What I'm saying here is that we should and must enforce a minimal "it
> > compiles" verification and "it runs" verification.
>
> But this is already done by the autobuilders, and works very well: we
> as maintainers rely on the autobuilders a lot to tell us if something
> is wrong.
>
> Let's take an example: I just pushed the commit adding the
> "makedumpfile" package. It has been submitted by a newcomer, so I did
> some local build testing, but with only one architecture, one
> toolchain, one configuration. It built in this configuration locally,
> so I pushed. All the rest of the testing will be done by the
> autobuilders.

This is exactly a postmortem autopsy.
If the testing was run before merging automatically for you by the
system, of course on a some more architectures and a couple of c
libraries for example, you could have noticed some problems before
merging.

> I just pushed the python-crayons package. I have done zero build
> testing. The only thing I've checked is verifying the legal information
> in the package, and this cannot be automated.
>
> > Recent history says that we had packages with a service never started
> > because the init script was wrong, we should avoid that.
>
> We already have the infrastructure for that: runtime tests. They exist,
> they are run by Gitlab CI.

Sorry for repeat myself: we are using GitLab CI but not doing CI. CI
is the process of testing the patches before merging them.

> Please submit more runtime tests :-)

Nobody will add testing if not enforced, really, it's how is going ...

Nowadays good development practices enforce testing.

> Best regards,
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] More maintainers

[Adam Duskett]

Basic things that could be done with CI/CD that don't take much CPU power and
would reduce the time spent on each patch:

 - Running check-pkg automatically
 - Running basic grammar and spelling checks
 - Checking if all recursive dependencies of new packages have been added in
   new Config.in files.

If you want to get complicated a reduction of CPU time/resources could
also be done with:
  - Having prebuilt docker images or tarballs that have a partially
complete build
  - Requiring all new packages to have at a minimum a simple test if applicable

On Sat, Sep 5, 2020 at 2:35 PM Angelo Compagnucci
<angelo.compagnucci@gmail.com> wrote:
>
> Il giorno sab 5 set 2020 alle ore 23:04 Thomas Petazzoni
> <thomas.petazzoni@bootlin.com> ha scritto:
> >
> > Hello,
> >
> > On Sat, 5 Sep 2020 22:55:11 +0200
> > Angelo Compagnucci <angelo.compagnucci@gmail.com> wrote:
> >
> > > > I'm not sure what you mean here. You would require all commits to come
> > > > with a snippet of configuration that allows the commit to be build
> > > > tested ?
> > >
> > > Yes, exactly.
> >
> > This actually raises the barrier to entry even more than an e-mail
> > based workflow! Newcomers have to understand this practice, what a
> > configuration snippet is, etc. It's not super complex, but it's not
> > trivial either.
>
> Probably, less contribution of more quality, aren't we talking about
> lowering the pending patches on patchwork ;) ?
>
> > Also, it's easy to get this snippet wrong, and have the automated
> > testing not actually test anything.
>
> Mmm, not if the checking system is configured properly.
>
> >
> > Also, the amount of CPU time that will be needed remains quite
> > significant. Another thing we could do is require contributors to
> > attach the output of a test-pkg run.
> >
> > But overall, the buildability of something is not really the primary
> > reason for the review taking time. Indeed, I typically never build test
> > version bumps, or even simple Python packages, I let the autobuilders
> > do this job. For new packages, I tend to have a different approach
> > depending on who the submitter is: experienced contributors are more
> > likely to know how to properly test their submissions, while newcomers
> > are more likely to make mistakes (which is normal!), so I tend to test
> > a bit more contributions from newcomers.
>
> Stealing time for applying patches, doing other reviews or live your life.
> Why do you have to test something if the system have already tested for you?
>
> >
> > > > If you have a magic solution for this, I'm interested!
> > >
> > > Well, I'm completely sure of that.
> > > Anyway, in the context of doing buildable and testable review
> > > requests, the developer should attach to the commit a minimal
> > > defconfig and minimal testing.
> > > That minimal defconfig and that testing are chosen by the developer to
> > > enhance confidence that the request is working.
> >
> > Well, it is generally the case that the developer has tested... but
> > that doesn't mean that this testing was good.
>
> Not if that minimal defconfig is tested against some/all architectures
> by the system.
> Or alternatively, because that defconfig is pushed with the commit
> somewhere, we can update that defconfig to reflect the knowledge we
> gain when something breaks,
>
> > Take the example of libblockdev, recently submitted by Adam. Adam
> > probably built it with a glibc toolchain, and it worked all fine for
> > him. And when I tested his libblockdev package, I used what is the
> > default C library in Buildroot, i.e uClibc, and it failed to build;
> >
> > As you can see, a minimal defconfig + testing chosen by the developer
> > is no silver bullet.
>
> No, but you can ask to add a snippet more or another test, and move
> back the patch is on the shoulders of the developer and the review
> system.
>
> In the end, when every check passes, you will only have to scan the
> patch, give you suggestions, ask to change here and there while being
> sure that even the new patchset will be built and tested by the system
> for you.
>
> How about a resubmitted patchset that introduced a bug went unnoticed?
> The could be caught.
>
> > > > In the context of Buildroot, saying "a commit is good" requires
> > > > building thousands of different configurations: number of CPU
> > > > architectures * number of GCC versions * number of binutils versions *
> > > > number of C libraries * number of optional dependencies of the package
> > > > * number of optional dependencies of all dependencies of the package.
> > >
> > > I'm not confusing, really, and from what I learned on Computer
> > > Science, that type of testing you're saying here is not even possile.
> >
> > Glad we agree that it is not possible to do such testing :-)
> >
> > > What I'm saying here is that we should and must enforce a minimal "it
> > > compiles" verification and "it runs" verification.
> >
> > But this is already done by the autobuilders, and works very well: we
> > as maintainers rely on the autobuilders a lot to tell us if something
> > is wrong.
> >
> > Let's take an example: I just pushed the commit adding the
> > "makedumpfile" package. It has been submitted by a newcomer, so I did
> > some local build testing, but with only one architecture, one
> > toolchain, one configuration. It built in this configuration locally,
> > so I pushed. All the rest of the testing will be done by the
> > autobuilders.
>
> This is exactly a postmortem autopsy.
> If the testing was run before merging automatically for you by the
> system, of course on a some more architectures and a couple of c
> libraries for example, you could have noticed some problems before
> merging.
>
> > I just pushed the python-crayons package. I have done zero build
> > testing. The only thing I've checked is verifying the legal information
> > in the package, and this cannot be automated.
> >
> > > Recent history says that we had packages with a service never started
> > > because the init script was wrong, we should avoid that.
> >
> > We already have the infrastructure for that: runtime tests. They exist,
> > they are run by Gitlab CI.
>
> Sorry for repeat myself: we are using GitLab CI but not doing CI. CI
> is the process of testing the patches before merging them.
>
> > Please submit more runtime tests :-)
>
> Nobody will add testing if not enforced, really, it's how is going ...
>
> Nowadays good development practices enforce testing.
>
> > Best regards,
> >
> > Thomas
> > --
> > Thomas Petazzoni, CTO, Bootlin
> > Embedded Linux and Kernel engineering
> > https://bootlin.com
>
>
>
> --
> Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] More maintainers

[Peter Korsgaard]

>>>>> "Adam" == Adam Duskett <aduskett@gmail.com> writes:

Hi,

Thanks for the constructive input!

 > Basic things that could be done with CI/CD that don't take much CPU power and
 > would reduce the time spent on each patch:

 >  - Running check-pkg automatically

I guess you mean check-package?


 >  - Running basic grammar and spelling checks

I don't know how easy that is to do, given that commit messages often
quote code and/or command output. We can certainly extend check-package
to run the commit message through aspell. What open source utilities are
available for grammar checks?


 >  - Checking if all recursive dependencies of new packages have been added in
 >    new Config.in files.

That IMHO very fast becomes complicated with optional dependencies and
patches to existing packages rather than a completely new package, but
if you have a good idea?


 > If you want to get complicated a reduction of CPU time/resources could
 > also be done with:
 >   - Having prebuilt docker images or tarballs that have a partially
 > complete build

I am not sure how realistic that is given that master (or next) is a
moving target and we would want to test against the current situation?

-- 
Bye, Peter Korsgaard

[Buildroot] More maintainers

[Yegor Yefremov]

Hi,

On Sun, Sep 6, 2020 at 9:43 AM Peter Korsgaard <peter@korsgaard.com> wrote:
>
> >>>>> "Adam" == Adam Duskett <aduskett@gmail.com> writes:
>
> Hi,
>
> Thanks for the constructive input!
>
>  > Basic things that could be done with CI/CD that don't take much CPU power and
>  > would reduce the time spent on each patch:
>
>  >  - Running check-pkg automatically
>
> I guess you mean check-package?
>
>
>  >  - Running basic grammar and spelling checks
>
> I don't know how easy that is to do, given that commit messages often
> quote code and/or command output. We can certainly extend check-package
> to run the commit message through aspell. What open source utilities are
> available for grammar checks?

I am using codespell: https://github.com/codespell-project/codespell

>
>  >  - Checking if all recursive dependencies of new packages have been added in
>  >    new Config.in files.
>
> That IMHO very fast becomes complicated with optional dependencies and
> patches to existing packages rather than a completely new package, but
> if you have a good idea?
>
>
>  > If you want to get complicated a reduction of CPU time/resources could
>  > also be done with:
>  >   - Having prebuilt docker images or tarballs that have a partially
>  > complete build
>
> I am not sure how realistic that is given that master (or next) is a
> moving target and we would want to test against the current situation?

Regards,
Yegor

[Buildroot] More maintainers

[Heiko Thiery]

Hi all,

regarding to this discussion I also have two infos I would like to
share with you:

[1] "As GitLab has grown over the years, we?re incredibly humbled by
the number of users who choose to use our free platform. As the
adoption of our free product has scaled, so have the underlying costs
to support it. As we are committed to continue to support the free
product for the larger community, we need to implement further CI/CD
limits on GitLab.com. This change is only applicable to free
GitLab.com users?current GitLab.com paid tiers and all self-managed
GitLab tiers will remain unchanged."

Does the buildroot gitlab is affected by this change? Or is buildroot
a member of the GitLab for Open Source program [2]?

The other thing is a talk about "New Tools Improve Patch Submission,
Review, and Other Processes" by Frank Rowand, Sony on the upcoming
Open Source Summit:

"The Linux kernel patch submission, review, and acceptance process has
long been email based. There have been both benefits and problems
resulting from being email based. Some new tools appear to reduce or
remove some of the problems. This presentation will describe the new
tools, how to use them, and how they solve problems for submitters,
reviewers, and maintainers. Any open source project whose contribution
process uses email may be interested in considering these new tools."

Maybe there are interesting new tools available that can be
used/adopted to buildroot as well.

-- 
Heiko

[1] https://about.gitlab.com/pricing/faq-consumption-cicd/
[2] https://about.gitlab.com/solutions/open-source/

[Buildroot] More maintainers

[Thomas Petazzoni]

On Mon, 7 Sep 2020 17:02:20 +0200
Heiko Thiery <heiko.thiery@gmail.com> wrote:

> regarding to this discussion I also have two infos I would like to
> share with you:
> 
> [1] "As GitLab has grown over the years, we?re incredibly humbled by
> the number of users who choose to use our free platform. As the
> adoption of our free product has scaled, so have the underlying costs
> to support it. As we are committed to continue to support the free
> product for the larger community, we need to implement further CI/CD
> limits on GitLab.com. This change is only applicable to free
> GitLab.com users?current GitLab.com paid tiers and all self-managed
> GitLab tiers will remain unchanged."
> 
> Does the buildroot gitlab is affected by this change? Or is buildroot
> a member of the GitLab for Open Source program [2]?

For gitlab.com/buildroot/buildroot, we are using our own Gitlab CI
runners, and not the free runners, so my understanding is that we are
not affected by this change.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] More maintainers

[Heiko Thiery]

Hi Thomas,

Thomas Petazzoni <thomas.petazzoni@bootlin.com> schrieb am Mo., 7. Sept.
2020, 17:20:

> On Mon, 7 Sep 2020 17:02:20 +0200
> Heiko Thiery <heiko.thiery@gmail.com> wrote:
>
> > regarding to this discussion I also have two infos I would like to
> > share with you:
> >
> > [1] "As GitLab has grown over the years, we?re incredibly humbled by
> > the number of users who choose to use our free platform. As the
> > adoption of our free product has scaled, so have the underlying costs
> > to support it. As we are committed to continue to support the free
> > product for the larger community, we need to implement further CI/CD
> > limits on GitLab.com. This change is only applicable to free
> > GitLab.com users?current GitLab.com paid tiers and all self-managed
> > GitLab tiers will remain unchanged."
> >
> > Does the buildroot gitlab is affected by this change? Or is buildroot
> > a member of the GitLab for Open Source program [2]?
>
> For gitlab.com/buildroot/buildroot, we are using our own Gitlab CI
> runners, and not the free runners, so my understanding is that we are
> not affected by this change.
>


Ah ok.

-- 
Heiko
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200907/017c45f8/attachment.html>

[Buildroot] More maintainers

[Christian Stewart]

Hi all,

Thank you all for your time on the phone discussion today. I really
appreciated hearing everyone's viewpoints.

My goal here is /not/ to fragment the project, but rather find a way
to allow everyone's desired workflows to come together.

In terms of the social issue with getting more quality Reviewed-by,
that is not something I know any solution to.

As for tooling;

I don't see any incompatibility between emails and GitLab - if we have
a bot that formats the discussion on GitLab into the exact same format
we expect in emails, and vise-versa.

Is there really a direct issue with having the same information in two places?

So: to be most useful, this is what I will try to start work on this
week towards this topic:

 - Bot prototype for syncing between emails and GitLab PRs
 - Automated CI w/ Buildroot in Initramfs against "real hardware"
(odroid, pi, intel mostly)
 - Contributing more GitLab runners for the CI using spare compute resources.
 - Automated PR + test (and then my own manual review) of the patch backlog.


On Mon, Sep 7, 2020 at 8:21 AM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> On Mon, 7 Sep 2020 17:02:20 +0200
> Heiko Thiery <heiko.thiery@gmail.com> wrote:
>
> > regarding to this discussion I also have two infos I would like to
> > share with you:
> >
> > [1] "As GitLab has grown over the years, we?re incredibly humbled by
> > the number of users who choose to use our free platform. As the
> > adoption of our free product has scaled, so have the underlying costs
> > to support it. As we are committed to continue to support the free
> > product for the larger community, we need to implement further CI/CD
> > limits on GitLab.com. This change is only applicable to free
> > GitLab.com users?current GitLab.com paid tiers and all self-managed
> > GitLab tiers will remain unchanged."
> >
> > Does the buildroot gitlab is affected by this change? Or is buildroot
> > a member of the GitLab for Open Source program [2]?
>
> For gitlab.com/buildroot/buildroot, we are using our own Gitlab CI
> runners, and not the free runners, so my understanding is that we are
> not affected by this change.

Would be interested to hear what everyone thinks of the concept of
keeping both e-mail and GitLab together.

Best regards,
Christian Stewart

[Buildroot] More maintainers

[Yann E. MORIN]

Adam, All,

On 2020-09-05 09:20 -0700, Adam Duskett spake thusly:
[--SNIP--]
> In fact, just looking at the list right now, a simple patch that has
> been reviewed
> AND tested is still sitting from 2019:
> https://patchwork.ozlabs.org/project/buildroot/patch/20191029095736.16586-1-fontaine.fabrice at gmail.com/

Thanks for pointing out to that seemingly trivial patch; I now have
taken time to do a proper review of it; please see my reply (I did add
you in Cc).

> Here's another:
> https://patchwork.ozlabs.org/project/buildroot/patch/20191109185512.24139-2-jeremy.rosen at smile.fr/

As already mentioned by Thomas, this was discussed with J?r?my when we
were in Lyon, last October, and the conclusion we came to was that we
needed a generic way to provide this feature. There was even some
explanations posted to the list (unfortunately not as a reply to that
series), which outlined the evetual goal, but no one has ever sent a
patchset toward that goal so far.

I'll send a proper rejection notice now.

> Here's a simple test:
> https://patchwork.ozlabs.org/project/buildroot/patch/20191112162325.529637-2-m.niestroj at grinn-global.com/

Thanks, I will have a look at it.

> So this is demonstrably false, which is why a lot of us are growing
> more frustrated.

I can certainly understand the frustration, having been in that same
situation in the past, and still being in that situation with some of my
own patches...

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] More maintainers

[Yann E. MORIN]

Adam, All,

On 2020-09-05 22:25 +0200, Yann E. MORIN spake thusly:
> On 2020-09-05 09:20 -0700, Adam Duskett spake thusly:
> > Here's a simple test:
> > https://patchwork.ozlabs.org/project/buildroot/patch/20191112162325.529637-2-m.niestroj at grinn-global.com/

So, I have now had a look at that pending patch.

I tried to apply it to master, but it failed: there is a conflict.

And indeed, we've had that test since 2020-03-24, with commit 803fc6ad51
(support/testing: add netdata test).

The patch still pending is v3 of the patchset, while there has been a
v4, a v5, and eventually a v6, which was applied.

Thus the pending patch is just some missed-to-be-updated patch in
patchwork; I've now marked it as superseded.

Thanks, I like it when the backlog decreases. ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] More maintainers

[Alexander Dahl]

Hei hei,

On Sat, Sep 05, 2020 at 08:44:08AM +0200, Peter Korsgaard wrote:
> But sure, we could do the basics, E.G. check-package and perhaps even
> building a basic config. That could cover basic issues and would have
> the advantage that the contributor gets fast feedback. It naturally also
> means quite a bit more mails on the list (E.G. to send a reviewed-by or
> any issues found).

The number of mail could be kept low, if only problems or failures are
sent by mail.  Reporting each success would be too much, wouldn't it?
I think that's how most bots on the Linux kernel lists currently
behave, screaming when things fail, but stay silent if everything is
okay?

Greets
Alex

-- 
/"\ ASCII RIBBON | ?With the first link, the chain is forged. The first
\ / CAMPAIGN     | speech censured, the first thought forbidden, the
 X  AGAINST      | first freedom denied, chains us all irrevocably.?
/ \ HTML MAIL    | (Jean-Luc Picard, quoting Judge Aaron Satie)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200905/a3cdec77/attachment.asc>

[Buildroot] More maintainers

[Peter Korsgaard]

>>>>> "Alexander" == Alexander Dahl <post@lespocky.de> writes:

 > Hei hei,
 > On Sat, Sep 05, 2020 at 08:44:08AM +0200, Peter Korsgaard wrote:
 >> But sure, we could do the basics, E.G. check-package and perhaps even
 >> building a basic config. That could cover basic issues and would have
 >> the advantage that the contributor gets fast feedback. It naturally also
 >> means quite a bit more mails on the list (E.G. to send a reviewed-by or
 >> any issues found).

 > The number of mail could be kept low, if only problems or failures are
 > sent by mail.  Reporting each success would be too much, wouldn't it?
 > I think that's how most bots on the Linux kernel lists currently
 > behave, screaming when things fail, but stay silent if everything is
 > okay?

Yes, that is certainly possible. This would mean that the bot would be
used to give early feedback to contributors whenever a (style) mistake
was made, rather than only when a maintainer would look at the patch and
try to apply it - But it wouldn't really help getting patches applied
faster.

But even so it might be useful to do.

-- 
Bye, Peter Korsgaard

[Buildroot] More maintainers

[Christian Stewart]

All,

On Sat, Sep 5, 2020 at 12:18 PM Peter Korsgaard <peter@korsgaard.com> wrote:
>
> >>>>> "Alexander" == Alexander Dahl <post@lespocky.de> writes:
>
>  > Hei hei,
>  > On Sat, Sep 05, 2020 at 08:44:08AM +0200, Peter Korsgaard wrote:
>  >> But sure, we could do the basics, E.G. check-package and perhaps even
>  >> building a basic config. That could cover basic issues and would have
>  >> the advantage that the contributor gets fast feedback. It naturally also
>  >> means quite a bit more mails on the list (E.G. to send a reviewed-by or
>  >> any issues found).
>
>  > The number of mail could be kept low, if only problems or failures are
>  > sent by mail.  Reporting each success would be too much, wouldn't it?
>  > I think that's how most bots on the Linux kernel lists currently
>  > behave, screaming when things fail, but stay silent if everything is
>  > okay?
>
> Yes, that is certainly possible. This would mean that the bot would be
> used to give early feedback to contributors whenever a (style) mistake
> was made, rather than only when a maintainer would look at the patch and
> try to apply it - But it wouldn't really help getting patches applied
> faster.
>
> But even so it might be useful to do.

I'm working on this right now:

 - Submit PRs to a "experimental" branch
 - When auto-tests pass, merge.
 - Run runtime tests on actual hardware (odroid, etc)
 - If everything looks good, forward the patches to the list

Key goals here: upon submission to list, a series should have
automated Tested-by, and several Reviewed-by based on PR approvals (or
whatever). Then, the maintainers can waste less time catching small
mistakes (like the package not building against the default C library,
or not running correctly on a real device) and be more efficient about
crunching through the backlog, especially if you're prioritizing based
on Reviewed-by.

Best regards,
Christian Stewart

[Buildroot] More maintainers

[James Hilliard]

On Thu, Sep 3, 2020 at 10:24 AM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> Hello Avraham,
>
> On Thu, 3 Sep 2020 18:44:27 +0300
> Avraham Shukron <avraham.shukron@gmail.com> wrote:
>
> > For what it's worth I think that the mail-based contribution process is
> > part of the problem.
> > With a decent Git server like GitHub/GitLab patches could be reviewed more
> > easily.
> > A CI pipeline could run tests that will give immediate feedback for any
> > pull request.
> > More importantly, it'll dramatically reduce the barrier for new and young
> > contributors.
>
> This has been discussed multiple times in the past in Buildroot, and in
> other open-source projects. There is even as we speak some pretty
> intense debate in the Linux kernel community about this.
Having CI run automatically when pull requests are made is extremely
useful for reducing reviewer workload in my experience, Linux has somewhat
unique requirements due to having many subsystem staging trees, this doesn't
apply to buildroot however so I see little reason that we need to follow the
Linux workflows closely.

My internal company workflows for buildroot based projects use buildbot
integrated with gitlab for running builds on each commit automatically.
I use buildbot workers so that I can use a cheap VPS for hosting the buildbot
master interface while running the actual builds on powerful bare metal
systems. It may also make sense to entirely replace the current autobuilder
setup with something buildbot based(buildbot is extremely flexible/scriptable).

Personally I would like to see buildroot move to something along the lines
of the freedesktop workflows using gitlab or similar where there is integrated
CI on each pull request.
>
>
> As we've seen from the discussion here, the Buildroot issue is not a
> lack of contribution, but a lack of review from trusted and experienced
> reviewers, and a lack of maintainers time. So while I'm all for new
> contributors and contributions, I don't think reducing the barrier is
> really key here. Also, I've always been skeptical about this statement
> that using Github reduces the barrier to entry. When you contribute to
> a project, is really sending a patch over e-mail the difficult part
> compared to understanding the code base, debugging the issue you've
> found or implementing the feature you wanted ? Really installing "git
> send-email" is a no-brainer straightforward process that is
> ridiculously easy even compared writing any single change in the code
> base with a decent commit message. Aren't we using this "reducing the
> barrier" argument a bit inappropriately here ?
I disagree that installing "git send-email" is straight forward, it's something
that's highly specific to ones mail setup and there are often significant
roadblocks, see https://twitter.com/mjg59/status/1299473271721664512
as an example. Personally I don't like git send-email since it requires
bypassing various 2FA security features and generally encourages bad
security practices.
>
> I believe I can say that all four Buildroot maintainers have a very
> strong preference for and a very optimized workflow to work with e-mail
> based patch submission and review.
>
> Somewhat related, recently a patch series I submitted last year to
> OpenWrt (which wasn't merged) got picked up by someone else, and
> re-submitted with new updates and fixes. Due to being the original
> author, I was in copy of all the Github discussion that took place. And
> I found it absolutely impossible and awful to follow the different
> revisions of the patch series, to which version of the patch series the
> comments were being made, etc.
>
> Perhaps for some people the Github pull request workflow makes sense,
> but I believe it's important to recognize and realize that there are
> also people for which this workflow doesn't make sense.
For a project like buildroot where there is only one mailing list I don't see
much advantage to the patch based workflow. From my understanding
the main advantage to using email patch based workflows is that you can
CC multiple subsystem mailing lists. Also replying to threads is non-trivial
as well as you have to do some crazy mbox importing to even respond to
an existing mailing list post if you are not already subscribed to that list.
>
> > Buildroot is all about using simple and familiar tools like make and
> > Kconfig, and I personally think that this principle should also be applied
> > to the contribution process and right now buildroot is one of the last
> > active open source projects using the mailing list approach.
>
> This is a very biased statement: there are still plenty of open-source
> projects that use e-mail based contribution workflow. I don't think we
> can call Linux or U-Boot "inactive" projects. Can we? :-)
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] More maintainers

[Angelo Compagnucci]

Il giorno gio 3 set 2020 alle ore 17:44 Avraham Shukron
<avraham.shukron@gmail.com> ha scritto:
>
> On Thu, Sep 3, 2020, 01:20 Christian Stewart <christian@paral.in> wrote:
>>
>> Discoverability is a problem right now, there's too many patches and
>> no way to subscribe to a given series for further updates (similar to
>> Pull Requests on GitHub), unless the submitter happens to add you to
>> the CC list - which, by the way, usually happens if you add a review.
>>
>> Today, probably the best approach is to hunt through the backlog
>> (sorted by reviewed-by priority order) and cherry-pick some of these
>> series into downstream, more "experimental" repositories. Then, once
>> they've been tested there, you can submit a Reviewed-By back upstream
>> to indicate that it was useful and worth merging more urgently.
>
>
> For what it's worth I think that the mail-based contribution process is part of the problem.
> With a decent Git server like GitHub/GitLab patches could be reviewed more easily.
> A CI pipeline could run tests that will give immediate feedback for any pull request.
> More importantly, it'll dramatically reduce the barrier for new and young contributors.

100% agree here, I think that Buildroot development should definitely
move in 2020.
Not only CI, but we can do  syntax checking, commit message checking,
run check-package and tons of automation more before even moving the
PR to a reviewable state.

>
> Buildroot is all about using simple and familiar tools like make and Kconfig, and I personally think that this principle should also be applied to the contribution process and right now buildroot is one of the last active open source projects using the mailing list approach.
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] More maintainers

[Peter Korsgaard]

>>>>> "Angelo" == Angelo Compagnucci <angelo.compagnucci@gmail.com> writes:

Hi,

 > Not only CI, but we can do  syntax checking, commit message checking,
 > run check-package and tons of automation more before even moving the
 > PR to a reviewable state.

Such improvements to check-package are naturally gladly accepted.

-- 
Bye, Peter Korsgaard

[Buildroot] More maintainers

[Christian Stewart]

Hi all,

I'd first like to clarify that my personal opinion is to maintain the
current email patch and review system for Buildroot proper. It's
certainly true that submitting patches is not anywhere near as
difficult as understanding what's going on Buildroot.

However, there's room for more experimental and fast-paced work in
downstream repositories, which then filter the stuff that works (with
the "lessons learned" applied) back up to the "gold standard"
Buildroot mainline repository. (Similar to how Linus Torvalds has the
"gold standard" Linux repo, but there are also downstream forks).

It really boils down to what the users are trying to do. If you want
to use Buildroot as it is today you need to know a lot about Linux,
that's just it. There's no focus on ease of use, and /there shouldn't
be/ since that's not the point of the project.

The key to bringing Buildroot to a wider audience would be to make it
as easy to build and install to a target platform as any of the other
distributions, if not easier. Most of the time, the issue reports and
bugfixes for this will be around configuration tweaks, additional
hardware support configs and packages (which are eventually
upstreamed), experimental Buildroot features, and bug reports + CI/CD
+ fixes (which are eventually upstreamed).

In the interest of increasing my own patch submission quality, I have
a CI/CD set up on my Buildroot repository (GitHub Actions) which
contains check-package etc, as well as a test build. There are also
Git hooks which run check-package on pre-commit. This has been
immensely helpful for me and has caught many bugs before pushing.

In the interest of implementing an easy to use and install variant of
Buildroot, which might one day be as simple as a point and click
operation to deploy to your laptop or desktop or Raspberry Pi or what
have you, I've been working on SkiffOS
https://github.com/skiffos/skiffos - which just passed its 1000th
commit. Skiff is built around the concept of booting Buildroot as an
initramfs for containers, however it can build just about any
Buildroot workflow. The tx2 config uses a traditional ext4 root
filesystem for example (temporarily).

On that repository I've also set up GitHub Actions with CI/CD, PR
reviews, automated testing of changes with check-package. I've built a
clean architecture for separating out board support and experimental
features into "layers" which can contain configuration fragments or
buildroot extension packages and be merged together into a final board
config on-the-fly, creating a reproducible output across devices. But
most importantly, I'm being careful to submit everything that might be
useful or a bugfix back up to the Buildroot mainline project whenever
it reaches a certain level of testing and maturity.

Perhaps Skiff could be a good community project to test out some of
these more "2020" approaches. I'm not saying my project is the only
one or even the right one for this, but I'm happy to put in the time
to make this happen.

The process would then be (for those of us that want to work quickly):

 - Add a new board, experimental feature, or fix to Skiff
 - Test it out through the "next" and "master" and "stable" release channels
 - Once a fix is known to work (or is urgent enough), submit it to
upstream with multiple reviewed-by already attached to the series upon
submission to the mailing list.
 - For those that don't want to or don't know how to use email
patches, the Skiff maintainers can do this work.

I think this & other downstream projects following a similar approach
would be the best way to scale this up and finally crunch through this
patch backlog. Especially when some of the patches in the backlog are
adding new packages, which could be brought in as external packages
during the "experimental" phase. I followed this process recently with
Apple Macbook and NVIDIA TX2 support.

Replies below:

On Thu, Sep 3, 2020 at 8:44 AM Avraham Shukron
<avraham.shukron@gmail.com> wrote:
>
> On Thu, Sep 3, 2020, 01:20 Christian Stewart <christian@paral.in> wrote:
>>
>> Discoverability is a problem right now, there's too many patches and
>> no way to subscribe to a given series for further updates (similar to
>> Pull Requests on GitHub), unless the submitter happens to add you to
>> the CC list - which, by the way, usually happens if you add a review.
>>
>> Today, probably the best approach is to hunt through the backlog
>> (sorted by reviewed-by priority order) and cherry-pick some of these
>> series into downstream, more "experimental" repositories. Then, once
>> they've been tested there, you can submit a Reviewed-By back upstream
>> to indicate that it was useful and worth merging more urgently.
>
>
> For what it's worth I think that the mail-based contribution process is part of the problem.
> With a decent Git server like GitHub/GitLab patches could be reviewed more easily.
> A CI pipeline could run tests that will give immediate feedback for any pull request.
> More importantly, it'll dramatically reduce the barrier for new and young contributors.

Have a look at:

 - https://github.com/skiffos/skiffos
 - https://github.com/skiffos/buildroot

Let me know what you think. Every commit is tested and has a build
artifact attached (docker image currently).

> Buildroot is all about using simple and familiar tools like make and Kconfig, and I personally think that this principle should also be applied to the contribution process and right now buildroot is one of the last active open source projects using the mailing list approach.

At the moment these tools you are proposing are, to be fair, not
simple or familiar at all to the industry. They are actually the
opposite, brand new (scary), complex, often operated by third party
companies who often change things without notice or way to revert.

This is fine for those of us that don't have major companies and
potentially safety-critical embedded projects in the downstream at
this time.

On Thu, Sep 3, 2020 at 9:24 AM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
> This has been discussed multiple times in the past in Buildroot, and in
> other open-source projects. There is even as we speak some pretty
> intense debate in the Linux kernel community about this.
>
> As we've seen from the discussion here, the Buildroot issue is not a
> lack of contribution, but a lack of review from trusted and experienced
> reviewers, and a lack of maintainers time. So while I'm all for new
> contributors and contributions, I don't think reducing the barrier is
> really key here. Also, I've always been skeptical about this statement
> that using Github reduces the barrier to entry. When you contribute to
> a project, is really sending a patch over e-mail the difficult part
> compared to understanding the code base, debugging the issue you've
> found or implementing the feature you wanted ? Really installing "git
> send-email" is a no-brainer straightforward process that is
> ridiculously easy even compared writing any single change in the code
> base with a decent commit message. Aren't we using this "reducing the
> barrier" argument a bit inappropriately here ?

I agree wholeheartedly with this, for the purposes of what Buildroot
is and is trying to be. For the purposes of distributing Buildroot as
an easy to use downstream distribution, and for testing out patches in
more experimental environments before promoting them to Buildroot
mainline, I disagree. The other tools will lower the barrier to entry
of people submitting patches and increase traffic, but also lowers the
barrier of entry to the maintainers merging things in specifically
defined experimental contexts or branches.

>
> I believe I can say that all four Buildroot maintainers have a very
> strong preference for and a very optimized workflow to work with e-mail
> based patch submission and review.

...I've come to respect this workflow over time and try to emulate it
as much as possible in my other projects.

> Somewhat related, recently a patch series I submitted last year to
> OpenWrt (which wasn't merged) got picked up by someone else, and
> re-submitted with new updates and fixes. Due to being the original
> author, I was in copy of all the Github discussion that took place. And
> I found it absolutely impossible and awful to follow the different
> revisions of the patch series, to which version of the patch series the
> comments were being made, etc.

To be fair: this has been recently improved and the story is quite a
bit better for PR reviews today. I'm no fan of using a closed source
Git frontend but due to the nature of Git and the ephemeral nature of
these pull requests, it's not really an issue for me on a downstream
project. The advantage is the discoverability of having it on that
platform.

> Perhaps for some people the Github pull request workflow makes sense,
> but I believe it's important to recognize and realize that there are
> also people for which this workflow doesn't make sense.

Absolutely.

Best regards,
Christian Stewart

[Buildroot] More maintainers

[Michael Nosthoff]

Hi All,

I want to add the "Buildroot User" perspective onto the whole Mailinglist centered approach.
I subscribed to the Mailinglist and started to contribute patches pretty recently.
But I have been an active BR User for a few years now. So I happened to file some bug reports
before. It happens there is still a report open from roughly a year ago:

https://bugs.busybox.net/show_bug.cgi?id=12301

What I want to point out at this example: Thomas added a link to the Mailinglist which
I followed when it was added. But afterwards lost track of it because the ticket didn't get any
updates (and I could mitigate the issue so it priority was low for me). I see now there 
was indeed some discussion which didn't lead to a fix because (I guess) time/priorities.

So as a User I've got pretty low interest to subscribe to a ML which has a pretty high volume with stuff
that doesn't bother me. I can't 'subscribe' to certain threads in a sensible way other than getting
people to put me in cc for mails. Further: when I see that an issue that I file via the bugtracker is 
discussed on the ML I have no clear way of responding to the discussion (when I'm not interested in
finding out how to get a Mailclient to send a mail to the list with a specific reply-to header, which
is after I found out how to send properly formatted text only mails).
(And I still don't know how I could pick up the ML discussion from a year ago...)

Which leads me to another point: When I want to see what the known issues with the current BR Version
are I have a pretty long way to go. Because the bugtracker itself is a silo which feeds into the ML, but
the ML doesn't feed back. So if I have a problem with a certain package I might find something in the
tracker, but the information might also be somewhere in the ML. Which leads me to another point:
Searchability of the ML. The kernel has this lore system which offers a text search. From what I know
BR doesn't offer that. So what is the normal way to search the ML archive? Google Foo?

So from a user standpoint I also see some room of improvement for tooling around bug reporting/tracking.

I totally see how it is an advantage of having all the projects Metadata in mbox files which gives you total
control. But I fear it requires a lot of hard work and tools to mirror a user experience with a platform
approach like github/gitlab.

Btw. I love that there is a discussion around adding some automated Patch checking. I recently sent a
patch to BlueZ and got pretty instantly a bot response which told me that my reference to a commit
was ill formatted. That's pretty nice. Hope something like this is possible for BR too.

Regards,
Michael
 
On Thursday, September 03, 2020 17:44 CEST, Avraham Shukron <avraham.shukron@gmail.com> wrote: 
 
> On Thu, Sep 3, 2020, 01:20 Christian Stewart <christian@paral.in> wrote:
> 
> > Discoverability is a problem right now, there's too many patches and
> > no way to subscribe to a given series for further updates (similar to
> > Pull Requests on GitHub), unless the submitter happens to add you to
> > the CC list - which, by the way, usually happens if you add a review.
> >
> > Today, probably the best approach is to hunt through the backlog
> > (sorted by reviewed-by priority order) and cherry-pick some of these
> > series into downstream, more "experimental" repositories. Then, once
> > they've been tested there, you can submit a Reviewed-By back upstream
> > to indicate that it was useful and worth merging more urgently.
> >
> 
> For what it's worth I think that the mail-based contribution process is
> part of the problem.
> With a decent Git server like GitHub/GitLab patches could be reviewed more
> easily.
> A CI pipeline could run tests that will give immediate feedback for any
> pull request.
> More importantly, it'll dramatically reduce the barrier for new and young
> contributors.
> 
> Buildroot is all about using simple and familiar tools like make and
> Kconfig, and I personally think that this principle should also be applied
> to the contribution process and right now buildroot is one of the last
> active open source projects using the mailing list approach.
> 
> >

[Buildroot] More maintainers

[Alexander Dahl]

Hei hei,

just want to add a note on the mailing list archive aspect ?

On Mon, Sep 07, 2020 at 05:57:20PM +0200, Michael Nosthoff via buildroot wrote:
> Searchability of the ML. The kernel has this lore system which
> offers a text search. From what I know BR doesn't offer that. So
> what is the normal way to search the ML archive? Google Foo?

The software behind lore.kernel.org is called public-inbox:

https://public-inbox.org/design_notes.html

That's free software. Some weeks ago someone from pengutronix asked
kernel.org if they would archive the mailing lists for barebox and/or
ptxdist, too. The answer was more or less: ?please do it yourself.? So
if buildroot wants to go that way, it probably has to set up
public-inbox by themselves.

Greets
Alex

-- 
/"\ ASCII RIBBON | ?With the first link, the chain is forged. The first
\ / CAMPAIGN     | speech censured, the first thought forbidden, the
 X  AGAINST      | first freedom denied, chains us all irrevocably.?
/ \ HTML MAIL    | (Jean-Luc Picard, quoting Judge Aaron Satie)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200907/3e3979b2/attachment.asc>

[Buildroot] More maintainers

[Peter Korsgaard]

>>>>> "Alexander" == Alexander Dahl <post@lespocky.de> writes:

 > Hei hei,
 > just want to add a note on the mailing list archive aspect ?

 > On Mon, Sep 07, 2020 at 05:57:20PM +0200, Michael Nosthoff via buildroot wrote:
 >> Searchability of the ML. The kernel has this lore system which
 >> offers a text search. From what I know BR doesn't offer that. So
 >> what is the normal way to search the ML archive? Google Foo?

 > The software behind lore.kernel.org is called public-inbox:

 > https://public-inbox.org/design_notes.html

 > That's free software. Some weeks ago someone from pengutronix asked
 > kernel.org if they would archive the mailing lists for barebox and/or
 > ptxdist, too. The answer was more or less: ?please do it yourself.? So
 > if buildroot wants to go that way, it probably has to set up
 > public-inbox by themselves.

Thanks. Maybe this is something we could request to osuosl?

-- 
Bye, Peter Korsgaard

[Buildroot] More maintainers: a live discussion ?

[Thomas Petazzoni]

Hello all,

I've added in Cc I hope all the people who expressed their opinion at
some point in this thread.

Overall my feeling is that we've reached a point where the e-mail
discussion is no longer making much useful progress, so what I would
like to propose is to have a live discussion about the topics that have
been raised: the need for more maintainers, the e-mail vs. pull-request
workflow, CI improvements, etc.

A live discussion would allow everybody to share his point of view, and
hopefully understand better the point of view of others. We could draw
from this discussion a series of conclusions and things to improve in
Buildroot.

It is clear from the discussion that many of us care about the
Buildroot project future, so let's try to work together.

For the sake of defining a suitable date for this live discussion, I've
opened a poll at:

  https://framadate.org/83o5hNwszaihEXOBZHZmJ89A/admin

It would be nice if the interested parties could fill in their
availability.

Thanks in advance for your participation!

Thomas

On Thu, 27 Aug 2020 10:34:31 -0700
Adam Duskett <aduskett@gmail.com> wrote:

> Hello;
> 
> Angelo and I were discussing the state of patches today in regards to Buildroot.
> It seems like the project could benefit from a few more maintainers of
> the project.
> We both understand that this is a volunteer project and that we are
> all adults and have a life outside of the project, and as such, we
> can't expect instant reviews on every submitted patch. However, as the
> project continues to grow in popularity, more patches will continue to
> be offered at an ever-increasing rate. This popularity increase is
> excellent news for the project, but also comes at the cost of more
> time and effort in regards to the number of patches and fixes in the
> queue to review and test.
> 
> As such, we would both like to throw our hat in the ring to help
> maintain the project. We both don't have all the time in the world,
> but any amount of help on the project can help alleviate the primary
> maintainers.
> 
> Let me know what you think.
> 
> Adam
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot



-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] More maintainers: a live discussion ?

[Thomas Petazzoni]

On Sat, 5 Sep 2020 15:06:04 +0200
Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:

> For the sake of defining a suitable date for this live discussion, I've
> opened a poll at:
> 
>   https://framadate.org/83o5hNwszaihEXOBZHZmJ89A/admin

I have sent a separate e-mail, but here is another one for those who
are in Cc. The discussion will take place on September 10th, at 9:00 PM
UTC+2. Use a Chromium web browser, and connect to
https://meet.bootlin.com/buildroot-maintenance.

Thanks for your participation !

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com