Fwd: bitbake style/workflow

Fwd: bitbake style/workflow

[Marius Kriegerowski]

[-- Attachment #1: Type: text/plain, Size: 1509 bytes --]

Hey there,

Is there a plan to switch to some more modern ways to collaborate on
bitbake, you know, like, gitlab/github/gitea with issues, PRs and so forth?
I mean… it’s late 2021 and sending MRs/Issues via email feels reaaaally
outdated, slow, intransparent, ineffective (how to you do reviews, code
comparisons during reviews, insert suggestions, etc.) and indirect (where
is the CI pipeline that I trigger when pushing something somewhere?). I
threw a search engine at ‘yocto bitbake CI’ (and similar) but didn’t find
it. I’m 110 % sure that there is a CI pipeline but it would make
contributions, reviews, etc. so much more efficient if I as the
(hypothetical) contributor would see the outcome of just a linter*. Maybe
it’s somewhere documented where the CI is hosted but then: No other project
I contribute to needs documentation about these things. They are just there
and obvious.

Don’t get me wrong: bitbake is awesome. I’d just love to contribute but the
burden for collaboration here is not really up to date. So, is there a
strategy to improve the on-boarding and workflow?

Greetings
Marius


* side-note on bitbake style/linting: It’s violating PEP8 in a couple of
ways and TBH PEP8 is the bare minimum I would accept these days for
collaboration on projects the dimension of bitbake. So an additional
question: Would you be up for a PEP8 compatibility overhaul of the source
code? I’d volunteer if I don’t have to send a patch via email ;)

[-- Attachment #2: Type: text/html, Size: 6339 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Marius Kriegerowski]

[-- Attachment #1: Type: text/plain, Size: 2368 bytes --]

okay, I don't know why I didn't come here ->
https://github.com/openembedded/bitbake <- directly. Maybe I was misguided
by some remnant documentation roaming on the internet?! I don't know...



Am Mo., 8. Nov. 2021 um 18:37 Uhr schrieb Marius Kriegerowski via
lists.openembedded.org <marius.kriegerowski=gmail.com@lists.openembedded.org
>:

> Hey there,
>
> Is there a plan to switch to some more modern ways to collaborate on
> bitbake, you know, like, gitlab/github/gitea with issues, PRs and so forth?
> I mean… it’s late 2021 and sending MRs/Issues via email feels reaaaally
> outdated, slow, intransparent, ineffective (how to you do reviews, code
> comparisons during reviews, insert suggestions, etc.) and indirect (where
> is the CI pipeline that I trigger when pushing something somewhere?). I
> threw a search engine at ‘yocto bitbake CI’ (and similar) but didn’t find
> it. I’m 110 % sure that there is a CI pipeline but it would make
> contributions, reviews, etc. so much more efficient if I as the
> (hypothetical) contributor would see the outcome of just a linter*. Maybe
> it’s somewhere documented where the CI is hosted but then: No other project
> I contribute to needs documentation about these things. They are just there
> and obvious.
>
> Don’t get me wrong: bitbake is awesome. I’d just love to contribute but
> the burden for collaboration here is not really up to date. So, is there a
> strategy to improve the on-boarding and workflow?
>
> Greetings
> Marius
>
>
> * side-note on bitbake style/linting: It’s violating PEP8 in a couple of
> ways and TBH PEP8 is the bare minimum I would accept these days for
> collaboration on projects the dimension of bitbake. So an additional
> question: Would you be up for a PEP8 compatibility overhaul of the source
> code? I’d volunteer if I don’t have to send a patch via email ;)
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12946):
> https://lists.openembedded.org/g/bitbake-devel/message/12946
> Mute This Topic: https://lists.openembedded.org/mt/86911583/6547911
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> marius.kriegerowski@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 8046 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Marius Kriegerowski]

[-- Attachment #1: Type: text/plain, Size: 12586 bytes --]

On 8. Nov 2021, at 22:47, Richard Purdie <richard.purdie@linuxfoundation.org>
wrote:

On Mon, 2021-11-08 at 18:37 +0100, Marius Kriegerowski wrote:

Is there a plan to switch to some more modern ways to collaborate on
bitbake,
you know, like, gitlab/github/gitea with issues, PRs and so forth? I mean…
it’s
late 2021 and sending MRs/Issues via email feels reaaaally outdated, slow,
intransparent, ineffective (how to you do reviews, code comparisons during
reviews, insert suggestions, etc.) and indirect (where is the CI pipeline
that I
trigger when pushing something somewhere?).


There are pros and cons to switching to something else. Keep in mind that
whilst
the github workflow works well for a single maintainer of a project, it
does not
work well for diverse review.

I strongly disagree with that. Reviews can equally well be done in a MR
thread. That’s what they are there for. You can discuss things there with
as many people as you want and can e.g. define a minimum number of
approvals before a MR can be merged. It also naturally groups discussions.
You can address experts for specific domains which means less noise for the
experts. I really don’t see why mailing list reviews should work any better
here. If google can coordinate 2.1k repos with 1.1k contributors over
GitHub, why should that not work for openembedded??

For wider Openembedded/ Yocto Project, we really
benefit from mailing list code review since we have a wide set of reviewers
and
no single person makes a direct decision on whether code is right or not.

See e.g. option to specify the minimum number of approvals.


The Bitbake case is a little bit different but in general we'd follow what
OE/Yocto Project are doing.

We have discussed changing and it is listed as one topic on:
https://wiki.yoctoproject.org/wiki/Future_Directions
(Code Submission process)

I'd note that it is a huge change to our current developers and most of us
are
very used to the mailing lists so if we did change, it would allow
potential new
collaborators at the risk of losing people with key knowledge/experience.

… which reads to me like: “It has always been that way so we will not
change that" doesn’t it? Always a very poor argument.

You
can do collaboration, code review and development with our current
processes, it
just isn't the github workflow.

Correct. Neither is it gitlab, gitea, gogs, bitbucket, …, you name the
platform that most developers are used to work with these days. These are
tools to enhance collaboration.


We do have CI which is shared with Yocto Project and
OpenEmbedded. You can see it here:

https://autobuilder.yoctoproject.org/typhoon/#/console

and it has it's own manual which was recently added to our documentations
set:

http://docs.yoctoproject.org/test-manual/index.html

… stressing my point: Why does the location of the CI need documentation?
All of the aforementioned platforms smoothly integrate with multiple CI
solutions, drastically reducing the time required for reviews if the
pipeline provides automated feedback to the contributor. It would probably
cost me about one afternoon to setup a decent CI pipeline at least for all
python codes pushing the need to review from manual labor to automated
tools. If open embedded migrates to a proper git platform I happily
volunteer on that end!!


There is work still needed on that but it is a start. One challenge of
bitbake/OE is that the builds take an age and the test matrix is huge so
the CI
process isn't quite as simple as some other simpler projects.

At least for python I bet that most bugs can be caught simply by firing up
`black`, `flake8`, `pylint` (and a hand full of other tools) just to check
the modified files. Regarding meta-layers and recipes out there we are
currently pushing forward with a linter for bitbake recipes (thanks
@priv-kweihmann for this awesome and overdue tool!!!).


As a shortcut through things, the "bitbake-selftest" command within bitbake
may
be one thing you're looking for which does simple tests on bitbake. It isn't
complete coverage and we welcome more being added, it is a start and some
areas
of the code are covered better than others.

I’d love to overhaul the tests e.g. porting them to `pytest` which offers a
much simpler yet much more powerful API/plugin-ecosystem. But won’t do via
email :)


Issues are filed in the project bugzilla and are actively worked on:

https://bugzilla.yoctoproject.org/

Didn’t know that. But why keeping issues separate from MRs? Automatic issue
closing through referencing in MRs is a pretty nice feature… Also the 90s
style is not appealing enough for me to create an account there. (I’m
sensing again that I’m not the only one).


I threw a search engine at ‘yocto bitbake CI’ (and similar) but didn’t find
it. I’m 110 % sure that there is a CI pipeline but it would make
contributions, reviews, etc. so much more efficient if I as the
(hypothetical)
contributor would see the outcome of just a linter*. Maybe it’s somewhere
documented where the CI is hosted but then: No other project I contribute to
needs documentation about these things. They are just there and obvious.

Don’t get me wrong: bitbake is awesome. I’d just love to contribute but the
burden for collaboration here is not really up to date. So, is there a
strategy to improve the on-boarding and workflow?


We've ideas but like the kernel, we do get things from the patch based
workflow
which we'd lose in a single person focused maintainer system like github so
I
doubt we'll change to that.

None of the platforms mentioned is a single person focused maintainer
system.


* side-note on bitbake style/linting: It’s violating PEP8 in a couple of
ways
and TBH PEP8 is the bare minimum I would accept these days for collaboration
on projects the dimension of bitbake. So an additional question: Would you
be
up for a PEP8 compatibility overhaul of the source code? I’d volunteer if I
don’t have to send a patch via email ;)


We're interested in patches to improve the coding style as long as they
don't
break things like readability or performance. Which rules in PEP8 in
particular
are you referring to?


To name a few violations:

 * hanging indents
 * inconsistent indentation
 * 2 separating lines missing in between classes/top level methods
 * unused imported modules
 * missing white-spaces after comma
 * unreachable codes

These are just a couple of examples that struck my PEP8 used eye from
looking at two, three modules.

It’s just like with any code smell: they don’t necessarily break things.
But they make maintenance, code-reviews and quality control harder than
necessary. Code that does not at least follow PEP8 (and I actually strongly
recommend to go the extra mile with `black`) just looks odd and is
unnecessary hard to read. I’m surprised I have to argue for standards.

Besides that, some minor cleanup just to warm up:
https://github.com/openembedded/bitbake/pull/33,
https://github.com/openembedded/bitbake/pull/32/files (it was after these
two that I found the comment in one of those old commits which made me open
https://github.com/openembedded/bitbake/pull/34). At least there should be
an automated reply to each MR that is posted if someone makes the effort to
fix code and open a MRs that this is actually in vein - That’s the mood
that took me here :)


Help appreciated but email patches is how we accept them at the moment.


Not from me then :(


Cheers,

Richard

Bottom line: I know I sounded harsh. Sorry about that! I love bitbake and
use it A LOT! I’d just love even more to get my hands dirty improving a
bunch of things. But only if the tools in place promise an efficient
collaboration reducing overhead as much as possible.

And I’m 100 % there are more developers out there that did not even drop a
message here but instead decided to not even start contribution at all. So,
my suggestion is to think about what are the most significant burdens that
hinder from contributing. My best bet is the email patches :)

Best regards

Marius



Am Mo., 8. Nov. 2021 um 22:47 Uhr schrieb Richard Purdie <
richard.purdie@linuxfoundation.org>:

> On Mon, 2021-11-08 at 18:37 +0100, Marius Kriegerowski wrote:
> > Is there a plan to switch to some more modern ways to collaborate on
> bitbake,
> > you know, like, gitlab/github/gitea with issues, PRs and so forth? I
> mean… it’s
> > late 2021 and sending MRs/Issues via email feels reaaaally outdated,
> slow,
> > intransparent, ineffective (how to you do reviews, code comparisons
> during
> > reviews, insert suggestions, etc.) and indirect (where is the CI
> pipeline that I
> > trigger when pushing something somewhere?).
>
> There are pros and cons to switching to something else. Keep in mind that
> whilst
> the github workflow works well for a single maintainer of a project, it
> does not
> work well for diverse review. For wider Openembedded/ Yocto Project, we
> really
> benefit from mailing list code review since we have a wide set of
> reviewers and
> no single person makes a direct decision on whether code is right or not.
>
> The Bitbake case is a little bit different but in general we'd follow what
> OE/Yocto Project are doing.
>
> We have discussed changing and it is listed as one topic on:
> https://wiki.yoctoproject.org/wiki/Future_Directions
> (Code Submission process)
>
> I'd note that it is a huge change to our current developers and most of us
> are
> very used to the mailing lists so if we did change, it would allow
> potential new
> collaborators at the risk of losing people with key knowledge/experience.
> You
> can do collaboration, code review and development with our current
> processes, it
> just isn't the github workflow.
>
> We do have CI which is shared with Yocto Project and
> OpenEmbedded. You can see it here:
>
> https://autobuilder.yoctoproject.org/typhoon/#/console
>
> and it has it's own manual which was recently added to our documentations
> set:
>
> http://docs.yoctoproject.org/test-manual/index.html
>
> There is work still needed on that but it is a start. One challenge of
> bitbake/OE is that the builds take an age and the test matrix is huge so
> the CI
> process isn't quite as simple as some other simpler projects.
>
> As a shortcut through things, the "bitbake-selftest" command within
> bitbake may
> be one thing you're looking for which does simple tests on bitbake. It
> isn't
> complete coverage and we welcome more being added, it is a start and some
> areas
> of the code are covered better than others.
>
> Issues are filed in the project bugzilla and are actively worked on:
>
> https://bugzilla.yoctoproject.org/
>
> > I threw a search engine at ‘yocto bitbake CI’ (and similar) but didn’t
> find
> > it. I’m 110 % sure that there is a CI pipeline but it would make
> > contributions, reviews, etc. so much more efficient if I as the
> (hypothetical)
> > contributor would see the outcome of just a linter*. Maybe it’s somewhere
> > documented where the CI is hosted but then: No other project I
> contribute to
> > needs documentation about these things. They are just there and obvious.
> >
> > Don’t get me wrong: bitbake is awesome. I’d just love to contribute but
> the
> > burden for collaboration here is not really up to date. So, is there a
> > strategy to improve the on-boarding and workflow?
>
> We've ideas but like the kernel, we do get things from the patch based
> workflow
> which we'd lose in a single person focused maintainer system like github
> so I
> doubt we'll change to that.
>
> > * side-note on bitbake style/linting: It’s violating PEP8 in a couple of
> ways
> > and TBH PEP8 is the bare minimum I would accept these days for
> collaboration
> > on projects the dimension of bitbake. So an additional question: Would
> you be
> > up for a PEP8 compatibility overhaul of the source code? I’d volunteer
> if I
> > don’t have to send a patch via email ;)
>
> We're interested in patches to improve the coding style as long as they
> don't
> break things like readability or performance. Which rules in PEP8 in
> particular
> are you referring to?
>
> Help appreciated but email patches is how we accept them at the moment.
>
> Cheers,
>
> Richard
>
>

[-- Attachment #2: Type: text/html, Size: 26496 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 12287 bytes --]

There's another problem with github/gitlab which I find significant. These
tools do not allow reviewing commit messages, and don't even make it easy
to see them. So the developers using those tools, both submitters and
maintainers, are effectively being conditioned by the tools to see good
commit messages as superfluous and unnecessary. And so you more often than
not end up with poorly written commit history in github projects, or, if
you're a reviewer, have to repeatedly explain to the submitter how to
explain and structure their changes properly.

I'd rather not have a change, than have a change that no one, even the
original author, can understand or remember the reasons for one year down
the line.

Alex


On Tue, 9 Nov 2021 at 00:30, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Mon, 2021-11-08 at 23:44 +0100, Marius Kriegerowski wrote:
> >
> >
> > > On 8. Nov 2021, at 22:47, Richard Purdie
> > > <richard.purdie@linuxfoundation.org> wrote:
> > >
> > > On Mon, 2021-11-08 at 18:37 +0100, Marius Kriegerowski wrote:
> > > > Is there a plan to switch to some more modern ways to collaborate on
> > > > bitbake,
> > > > you know, like, gitlab/github/gitea with issues, PRs and so forth? I
> mean…
> > > > it’s
> > > > late 2021 and sending MRs/Issues via email feels reaaaally outdated,
> slow,
> > > > intransparent, ineffective (how to you do reviews, code comparisons
> during
> > > > reviews, insert suggestions, etc.) and indirect (where is the CI
> pipeline
> > > > that I
> > > > trigger when pushing something somewhere?).
> > >
> > > There are pros and cons to switching to something else. Keep in mind
> that
> > > whilst the github workflow works well for a single maintainer of a
> project,
> > > it does not work well for diverse review.
> >
> > I strongly disagree with that. Reviews can equally well be done in a MR
> > thread. That’s what they are there for. You can discuss things there
> with as
> > many people as you want and can e.g. define a minimum number of approvals
> > before a MR can be merged.
>
> You misunderstand my point. We have a several thousand people on the
> mailing
> lists and they filter things to what interests them. *I* have no people I
> want
> to discuss a given change with, it is a question of whether the people who
> have
> an interest in a given change would see it.
>
> A minimum number of approvals is utterly meaningless for how we work, I
> need the
> interested people to see it and that varies massively depending on what is
> being
> changed. It is this aspect of mailing list change distribution and
> discussion we
> benefit from.
>
> >  It also naturally groups discussions. You can address experts for
> specific
> > domains which means less noise for the experts. I really don’t see why
> mailing
> > list reviews should work any better here.
>
> We all have established mechanisms to "see" the things which interest us as
> maintainers?
>
> Please don't misunderstand me, I do see the attraction of github from your
> perspective. It would just destroy the ecosystem we already have. You are
> basically saying that our current ecosystem is worthless but I would point
> out
> it has resulted in a project which you did say you appreciate so it can't
> be all
> bad.
>
> >  If google can coordinate 2.1k repos with 1.1k contributors over GitHub,
> why
> > should that not work for openembedded??
>
> It isn't about raw numbers, it is about numbers of reviewers for a given
> change.
> We have hundreds of people doing that using the mailing lists.
>
> > > The Bitbake case is a little bit different but in general we'd follow
> what
> > > OE/Yocto Project are doing.
> > >
> > > We have discussed changing and it is listed as one topic on:
> > > https://wiki.yoctoproject.org/wiki/Future_Directions
> > > (Code Submission process)
> > >
> > > I'd note that it is a huge change to our current developers and most
> of us
> > > are very used to the mailing lists so if we did change, it would allow
> > > potential new collaborators at the risk of losing people with key
> > > knowledge/experience.
> > … which reads to me like: “It has always been that way so we will not
> change
> > that" doesn’t it? Always a very poor argument.
>
> No, what I said is "upsetting our existing developers is something which
> we may
> want to potentially avoid".
>
> There are some options out there which have offered mailing list
> integration for
> example and we have tried to explore those. Something which drops the lists
> entirely could in the current view of the TSC result in loss of too many
> of our
> existing maintainers.
>
> >
> > > We do have CI which is shared with Yocto Project and
> > > OpenEmbedded. You can see it here:
> > >
> > > https://autobuilder.yoctoproject.org/typhoon/#/console
> > >
> > > and it has it's own manual which was recently added to our
> documentations
> > > set:
> > >
> > > http://docs.yoctoproject.org/test-manual/index.html
> > … stressing my point: Why does the location of the CI need
> documentation? All
> > of the aforementioned platforms smoothly integrate with multiple CI
> solutions,
> > drastically reducing the time required for reviews if the pipeline
> provides
> > automated feedback to the contributor. It would probably cost me about
> one
> > afternoon to setup a decent CI pipeline at least for all python codes
> pushing
> > the need to review from manual labor to automated tools. If open embedded
> > migrates to a proper git platform I happily volunteer on that end!!
>
> We're not talking about simple python code. We're talking about testing a
> build
> environment which cross compiles multiple operating systems for multiple
> architectures, C libraries, GUI stacks, init systems and so on. Testing the
> python code is one small part of that. I've spent several years trying to
> improve and get our CI to where it is today.
>
> Yes, the python code is tested as part of this as we don't really want to
> have
> multiple test platforms and the real world usage is some of the best
> testing
> changes can have.
>
> > > There is work still needed on that but it is a start. One challenge of
> > > bitbake/OE is that the builds take an age and the test matrix is huge
> so the
> > > CI
> > > process isn't quite as simple as some other simpler projects.
> > At least for python I bet that most bugs can be caught simply by firing
> up
> > `black`, `flake8`, `pylint` (and a hand full of other tools) just to
> check the
> > modified files.
>
> Well, I would bet that most of our bugs are not caused by issues like
> that. I do
> attend the weekly bug triage meetings and have fixed my share of them. You
> could
> have a look at the issues filed in bugzilla over the past few weeks if
> you'd
> like to see the kinds of issues that get filed and are open.
>
> >  Regarding meta-layers and recipes out there we are currently pushing
> forward
> > with a linter for bitbake recipes (thanks @priv-kweihmann for this
> awesome and
> > overdue tool!!!).
>
> Whilst improving recipes in that way does help, it isn't the main source
> of our
> major issues.
>
> > >
> > > As a shortcut through things, the "bitbake-selftest" command within
> bitbake
> > > may
> > > be one thing you're looking for which does simple tests on bitbake. It
> isn't
> > > complete coverage and we welcome more being added, it is a start and
> some
> > > areas
> > > of the code are covered better than others.
> > I’d love to overhaul the tests e.g. porting them to `pytest` which
> offers a
> > much simpler yet much more powerful API/plugin-ecosystem. But won’t do
> via
> > email :)
>
> The tests are written in python's own unittest.
>
> > > Issues are filed in the project bugzilla and are actively worked on:
> > >
> > > https://bugzilla.yoctoproject.org/
> > Didn’t know that. But why keeping issues separate from MRs? Automatic
> issue
> > closing through referencing in MRs is a pretty nice feature…
>
> If I had to list the pain points we have, this is not high on the list.
>
> > > > * side-note on bitbake style/linting: It’s violating PEP8 in a
> couple of
> > > > ways and TBH PEP8 is the bare minimum I would accept these days for
> > > > collaboration on projects the dimension of bitbake. So an additional
> > > > question: Would you be up for a PEP8 compatibility overhaul of the
> source
> > > > code? I’d volunteer if I don’t have to send a patch via email ;)
> > >
> > > We're interested in patches to improve the coding style as long as they
> > > don't
> > > break things like readability or performance. Which rules in PEP8 in
> > > particular
> > > are you referring to?
> >
> > To name a few violations:
> >
> >  * hanging indents
> >  * inconsistent indentation
> >  * 2 separating lines missing in between classes/top level methods
>
> FWIW I find it really hard to get worked up about whitespace issues. I
> appreciate these aren't always great to look at and I appreciate the
> familiarity
> argument and standardising the look but these aren't issues I would
> prioritise.
>
> In particular changing them globally in mass patches tends to break patch
> backporting and taking fixes back to older code which we do benefit from
> being
> able to do so there is a cost to doing it. We are trying to make new code
> more
> consistent and with older code, we do try and fix the worst issues.
>
> >  * unused imported modules
> >  * missing white-spaces after comma
> >  * unreachable codes
>
> These ones we do take patches for and try to fix when/where we see it. For
> example,
> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511
> d830df9cbd4113a4352b2c
> <http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511d830df9cbd4113a4352b2c> contained
> a couple of white-space after comma fixing
> while I was in that area.
>
> > These are just a couple of examples that struck my PEP8 used eye from
> looking
> > at two, three modules.
> >
> > It’s just like with any code smell: they don’t necessarily break things.
> But
> > they make maintenance, code-reviews and quality control harder than
> necessary.
> > Code that does not at least follow PEP8 (and I actually strongly
> recommend to
> > go the extra mile with `black`) just looks odd and is unnecessary hard to
> > read. I’m surprised I have to argue for standards.
>
> I wasn't saying it was a bad thing, I asked which issues you noticed in
> particular since I'm trying to ensure we do at least try and react to
> feedback.
> Some whitespace issues we aren't likely to tackle, some like the others you
> mentioned above we should fix.
> >
> > Besides that, some minor cleanup just to warm
> > up: https://github.com/openembedded/bitbake/pull/33,
> https://github.com/openem
> > bedded/bitbake/pull/32/files
>
> Those two do look like reasonable changes FWIW.
>
> > (it was after these two that I found the comment in one of those old
> commits
> > which made me open https://github.com/openembedded/bitbake/pull/34). At
> least
> > there should be an automated reply to each MR that is posted if someone
> makes
> > the effort to fix code and open a MRs that this is actually in vein -
> That’s
> > the mood that took me here :)
>
> There is a certain irony in that given you patched the section called
> "Contributing" in the README in 32!
>
> > >
> > > Help appreciated but email patches is how we accept them at the moment.
> >
> > Not from me then :(
>
> Sorry to hear that, your changes in 32/33 do look reasonable.
>
> Cheers,
>
> Richard
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12951):
> https://lists.openembedded.org/g/bitbake-devel/message/12951
> Mute This Topic: https://lists.openembedded.org/mt/86911583/1686489
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 14934 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Marius Kriegerowski]

[-- Attachment #1: Type: text/plain, Size: 13048 bytes --]

Good morning,

I don’t think this is correct. You can change the git history and commit
messages of every MR to whatever you like with `git commit —amend`, `git
rebase -I` and `git push -f`. So, you can very well ask contributors to
cleanup their history before merging.

Best
Marius

Am Di., 9. Nov. 2021 um 07:47 Uhr schrieb Alexander Kanavin <
alex.kanavin@gmail.com>:

> There's another problem with github/gitlab which I find significant. These
> tools do not allow reviewing commit messages, and don't even make it easy
> to see them. So the developers using those tools, both submitters and
> maintainers, are effectively being conditioned by the tools to see good
> commit messages as superfluous and unnecessary. And so you more often than
> not end up with poorly written commit history in github projects, or, if
> you're a reviewer, have to repeatedly explain to the submitter how to
> explain and structure their changes properly.
>
> I'd rather not have a change, than have a change that no one, even the
> original author, can understand or remember the reasons for one year down
> the line.
>
> Alex
>
>
> On Tue, 9 Nov 2021 at 00:30, Richard Purdie <
> richard.purdie@linuxfoundation.org> wrote:
>
>> On Mon, 2021-11-08 at 23:44 +0100, Marius Kriegerowski wrote:
>> >
>> >
>> > > On 8. Nov 2021, at 22:47, Richard Purdie
>> > > <richard.purdie@linuxfoundation.org> wrote:
>> > >
>> > > On Mon, 2021-11-08 at 18:37 +0100, Marius Kriegerowski wrote:
>> > > > Is there a plan to switch to some more modern ways to collaborate on
>> > > > bitbake,
>> > > > you know, like, gitlab/github/gitea with issues, PRs and so forth?
>> I mean…
>> > > > it’s
>> > > > late 2021 and sending MRs/Issues via email feels reaaaally
>> outdated, slow,
>> > > > intransparent, ineffective (how to you do reviews, code comparisons
>> during
>> > > > reviews, insert suggestions, etc.) and indirect (where is the CI
>> pipeline
>> > > > that I
>> > > > trigger when pushing something somewhere?).
>> > >
>> > > There are pros and cons to switching to something else. Keep in mind
>> that
>> > > whilst the github workflow works well for a single maintainer of a
>> project,
>> > > it does not work well for diverse review.
>> >
>> > I strongly disagree with that. Reviews can equally well be done in a MR
>> > thread. That’s what they are there for. You can discuss things there
>> with as
>> > many people as you want and can e.g. define a minimum number of
>> approvals
>> > before a MR can be merged.
>>
>> You misunderstand my point. We have a several thousand people on the
>> mailing
>> lists and they filter things to what interests them. *I* have no people I
>> want
>> to discuss a given change with, it is a question of whether the people
>> who have
>> an interest in a given change would see it.
>>
>> A minimum number of approvals is utterly meaningless for how we work, I
>> need the
>> interested people to see it and that varies massively depending on what
>> is being
>> changed. It is this aspect of mailing list change distribution and
>> discussion we
>> benefit from.
>>
>> >  It also naturally groups discussions. You can address experts for
>> specific
>> > domains which means less noise for the experts. I really don’t see why
>> mailing
>> > list reviews should work any better here.
>>
>> We all have established mechanisms to "see" the things which interest us
>> as
>> maintainers?
>>
>> Please don't misunderstand me, I do see the attraction of github from your
>> perspective. It would just destroy the ecosystem we already have. You are
>> basically saying that our current ecosystem is worthless but I would
>> point out
>> it has resulted in a project which you did say you appreciate so it can't
>> be all
>> bad.
>>
>> >  If google can coordinate 2.1k repos with 1.1k contributors over
>> GitHub, why
>> > should that not work for openembedded??
>>
>> It isn't about raw numbers, it is about numbers of reviewers for a given
>> change.
>> We have hundreds of people doing that using the mailing lists.
>>
>> > > The Bitbake case is a little bit different but in general we'd follow
>> what
>> > > OE/Yocto Project are doing.
>> > >
>> > > We have discussed changing and it is listed as one topic on:
>> > > https://wiki.yoctoproject.org/wiki/Future_Directions
>> > > (Code Submission process)
>> > >
>> > > I'd note that it is a huge change to our current developers and most
>> of us
>> > > are very used to the mailing lists so if we did change, it would allow
>> > > potential new collaborators at the risk of losing people with key
>> > > knowledge/experience.
>> > … which reads to me like: “It has always been that way so we will not
>> change
>> > that" doesn’t it? Always a very poor argument.
>>
>> No, what I said is "upsetting our existing developers is something which
>> we may
>> want to potentially avoid".
>>
>> There are some options out there which have offered mailing list
>> integration for
>> example and we have tried to explore those. Something which drops the
>> lists
>> entirely could in the current view of the TSC result in loss of too many
>> of our
>> existing maintainers.
>>
>> >
>> > > We do have CI which is shared with Yocto Project and
>> > > OpenEmbedded. You can see it here:
>> > >
>> > > https://autobuilder.yoctoproject.org/typhoon/#/console
>> > >
>> > > and it has it's own manual which was recently added to our
>> documentations
>> > > set:
>> > >
>> > > http://docs.yoctoproject.org/test-manual/index.html
>> > … stressing my point: Why does the location of the CI need
>> documentation? All
>> > of the aforementioned platforms smoothly integrate with multiple CI
>> solutions,
>> > drastically reducing the time required for reviews if the pipeline
>> provides
>> > automated feedback to the contributor. It would probably cost me about
>> one
>> > afternoon to setup a decent CI pipeline at least for all python codes
>> pushing
>> > the need to review from manual labor to automated tools. If open
>> embedded
>> > migrates to a proper git platform I happily volunteer on that end!!
>>
>> We're not talking about simple python code. We're talking about testing a
>> build
>> environment which cross compiles multiple operating systems for multiple
>> architectures, C libraries, GUI stacks, init systems and so on. Testing
>> the
>> python code is one small part of that. I've spent several years trying to
>> improve and get our CI to where it is today.
>>
>> Yes, the python code is tested as part of this as we don't really want to
>> have
>> multiple test platforms and the real world usage is some of the best
>> testing
>> changes can have.
>>
>> > > There is work still needed on that but it is a start. One challenge of
>> > > bitbake/OE is that the builds take an age and the test matrix is huge
>> so the
>> > > CI
>> > > process isn't quite as simple as some other simpler projects.
>> > At least for python I bet that most bugs can be caught simply by firing
>> up
>> > `black`, `flake8`, `pylint` (and a hand full of other tools) just to
>> check the
>> > modified files.
>>
>> Well, I would bet that most of our bugs are not caused by issues like
>> that. I do
>> attend the weekly bug triage meetings and have fixed my share of them.
>> You could
>> have a look at the issues filed in bugzilla over the past few weeks if
>> you'd
>> like to see the kinds of issues that get filed and are open.
>>
>> >  Regarding meta-layers and recipes out there we are currently pushing
>> forward
>> > with a linter for bitbake recipes (thanks @priv-kweihmann for this
>> awesome and
>> > overdue tool!!!).
>>
>> Whilst improving recipes in that way does help, it isn't the main source
>> of our
>> major issues.
>>
>> > >
>> > > As a shortcut through things, the "bitbake-selftest" command within
>> bitbake
>> > > may
>> > > be one thing you're looking for which does simple tests on bitbake.
>> It isn't
>> > > complete coverage and we welcome more being added, it is a start and
>> some
>> > > areas
>> > > of the code are covered better than others.
>> > I’d love to overhaul the tests e.g. porting them to `pytest` which
>> offers a
>> > much simpler yet much more powerful API/plugin-ecosystem. But won’t do
>> via
>> > email :)
>>
>> The tests are written in python's own unittest.
>>
>> > > Issues are filed in the project bugzilla and are actively worked on:
>> > >
>> > > https://bugzilla.yoctoproject.org/
>> > Didn’t know that. But why keeping issues separate from MRs? Automatic
>> issue
>> > closing through referencing in MRs is a pretty nice feature…
>>
>> If I had to list the pain points we have, this is not high on the list.
>>
>> > > > * side-note on bitbake style/linting: It’s violating PEP8 in a
>> couple of
>> > > > ways and TBH PEP8 is the bare minimum I would accept these days for
>> > > > collaboration on projects the dimension of bitbake. So an additional
>> > > > question: Would you be up for a PEP8 compatibility overhaul of the
>> source
>> > > > code? I’d volunteer if I don’t have to send a patch via email ;)
>> > >
>> > > We're interested in patches to improve the coding style as long as
>> they
>> > > don't
>> > > break things like readability or performance. Which rules in PEP8 in
>> > > particular
>> > > are you referring to?
>> >
>> > To name a few violations:
>> >
>> >  * hanging indents
>> >  * inconsistent indentation
>> >  * 2 separating lines missing in between classes/top level methods
>>
>> FWIW I find it really hard to get worked up about whitespace issues. I
>> appreciate these aren't always great to look at and I appreciate the
>> familiarity
>> argument and standardising the look but these aren't issues I would
>> prioritise.
>>
>> In particular changing them globally in mass patches tends to break patch
>> backporting and taking fixes back to older code which we do benefit from
>> being
>> able to do so there is a cost to doing it. We are trying to make new code
>> more
>> consistent and with older code, we do try and fix the worst issues.
>>
>> >  * unused imported modules
>> >  * missing white-spaces after comma
>> >  * unreachable codes
>>
>> These ones we do take patches for and try to fix when/where we see it. For
>> example,
>> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511
>> d830df9cbd4113a4352b2c
>> <http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511d830df9cbd4113a4352b2c> contained
>> a couple of white-space after comma fixing
>> while I was in that area.
>>
>> > These are just a couple of examples that struck my PEP8 used eye from
>> looking
>> > at two, three modules.
>> >
>> > It’s just like with any code smell: they don’t necessarily break
>> things. But
>> > they make maintenance, code-reviews and quality control harder than
>> necessary.
>> > Code that does not at least follow PEP8 (and I actually strongly
>> recommend to
>> > go the extra mile with `black`) just looks odd and is unnecessary hard
>> to
>> > read. I’m surprised I have to argue for standards.
>>
>> I wasn't saying it was a bad thing, I asked which issues you noticed in
>> particular since I'm trying to ensure we do at least try and react to
>> feedback.
>> Some whitespace issues we aren't likely to tackle, some like the others
>> you
>> mentioned above we should fix.
>> >
>> > Besides that, some minor cleanup just to warm
>> > up: https://github.com/openembedded/bitbake/pull/33,
>> https://github.com/openem
>> > bedded/bitbake/pull/32/files
>>
>> Those two do look like reasonable changes FWIW.
>>
>> > (it was after these two that I found the comment in one of those old
>> commits
>> > which made me open https://github.com/openembedded/bitbake/pull/34).
>> At least
>> > there should be an automated reply to each MR that is posted if someone
>> makes
>> > the effort to fix code and open a MRs that this is actually in vein -
>> That’s
>> > the mood that took me here :)
>>
>> There is a certain irony in that given you patched the section called
>> "Contributing" in the README in 32!
>>
>> > >
>> > > Help appreciated but email patches is how we accept them at the
>> moment.
>> >
>> > Not from me then :(
>>
>> Sorry to hear that, your changes in 32/33 do look reasonable.
>>
>> Cheers,
>>
>> Richard
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#12951):
>> https://lists.openembedded.org/g/bitbake-devel/message/12951
>> Mute This Topic: https://lists.openembedded.org/mt/86911583/1686489
>> Group Owner: bitbake-devel+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
>> alex.kanavin@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>>

[-- Attachment #2: Type: text/html, Size: 17334 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 13765 bytes --]

I think you misunderstood me too. It's about the github UI which does its
best to hide the commit messages, and doesn't allow to comment on their
content. Yes you can contort yourself and still do it, but the tools do not
help.

Alex

On Tue, 9 Nov 2021 at 09:05, Marius Kriegerowski <
marius.kriegerowski@gmail.com> wrote:

> Good morning,
>
> I don’t think this is correct. You can change the git history and commit
> messages of every MR to whatever you like with `git commit —amend`, `git
> rebase -I` and `git push -f`. So, you can very well ask contributors to
> cleanup their history before merging.
>
> Best
> Marius
>
> Am Di., 9. Nov. 2021 um 07:47 Uhr schrieb Alexander Kanavin <
> alex.kanavin@gmail.com>:
>
>> There's another problem with github/gitlab which I find significant.
>> These tools do not allow reviewing commit messages, and don't even make it
>> easy to see them. So the developers using those tools, both submitters and
>> maintainers, are effectively being conditioned by the tools to see good
>> commit messages as superfluous and unnecessary. And so you more often than
>> not end up with poorly written commit history in github projects, or, if
>> you're a reviewer, have to repeatedly explain to the submitter how to
>> explain and structure their changes properly.
>>
>> I'd rather not have a change, than have a change that no one, even the
>> original author, can understand or remember the reasons for one year down
>> the line.
>>
>> Alex
>>
>>
>> On Tue, 9 Nov 2021 at 00:30, Richard Purdie <
>> richard.purdie@linuxfoundation.org> wrote:
>>
>>> On Mon, 2021-11-08 at 23:44 +0100, Marius Kriegerowski wrote:
>>> >
>>> >
>>> > > On 8. Nov 2021, at 22:47, Richard Purdie
>>> > > <richard.purdie@linuxfoundation.org> wrote:
>>> > >
>>> > > On Mon, 2021-11-08 at 18:37 +0100, Marius Kriegerowski wrote:
>>> > > > Is there a plan to switch to some more modern ways to collaborate
>>> on
>>> > > > bitbake,
>>> > > > you know, like, gitlab/github/gitea with issues, PRs and so forth?
>>> I mean…
>>> > > > it’s
>>> > > > late 2021 and sending MRs/Issues via email feels reaaaally
>>> outdated, slow,
>>> > > > intransparent, ineffective (how to you do reviews, code
>>> comparisons during
>>> > > > reviews, insert suggestions, etc.) and indirect (where is the CI
>>> pipeline
>>> > > > that I
>>> > > > trigger when pushing something somewhere?).
>>> > >
>>> > > There are pros and cons to switching to something else. Keep in mind
>>> that
>>> > > whilst the github workflow works well for a single maintainer of a
>>> project,
>>> > > it does not work well for diverse review.
>>> >
>>> > I strongly disagree with that. Reviews can equally well be done in a MR
>>> > thread. That’s what they are there for. You can discuss things there
>>> with as
>>> > many people as you want and can e.g. define a minimum number of
>>> approvals
>>> > before a MR can be merged.
>>>
>>> You misunderstand my point. We have a several thousand people on the
>>> mailing
>>> lists and they filter things to what interests them. *I* have no people
>>> I want
>>> to discuss a given change with, it is a question of whether the people
>>> who have
>>> an interest in a given change would see it.
>>>
>>> A minimum number of approvals is utterly meaningless for how we work, I
>>> need the
>>> interested people to see it and that varies massively depending on what
>>> is being
>>> changed. It is this aspect of mailing list change distribution and
>>> discussion we
>>> benefit from.
>>>
>>> >  It also naturally groups discussions. You can address experts for
>>> specific
>>> > domains which means less noise for the experts. I really don’t see why
>>> mailing
>>> > list reviews should work any better here.
>>>
>>> We all have established mechanisms to "see" the things which interest us
>>> as
>>> maintainers?
>>>
>>> Please don't misunderstand me, I do see the attraction of github from
>>> your
>>> perspective. It would just destroy the ecosystem we already have. You are
>>> basically saying that our current ecosystem is worthless but I would
>>> point out
>>> it has resulted in a project which you did say you appreciate so it
>>> can't be all
>>> bad.
>>>
>>> >  If google can coordinate 2.1k repos with 1.1k contributors over
>>> GitHub, why
>>> > should that not work for openembedded??
>>>
>>> It isn't about raw numbers, it is about numbers of reviewers for a given
>>> change.
>>> We have hundreds of people doing that using the mailing lists.
>>>
>>> > > The Bitbake case is a little bit different but in general we'd
>>> follow what
>>> > > OE/Yocto Project are doing.
>>> > >
>>> > > We have discussed changing and it is listed as one topic on:
>>> > > https://wiki.yoctoproject.org/wiki/Future_Directions
>>> > > (Code Submission process)
>>> > >
>>> > > I'd note that it is a huge change to our current developers and most
>>> of us
>>> > > are very used to the mailing lists so if we did change, it would
>>> allow
>>> > > potential new collaborators at the risk of losing people with key
>>> > > knowledge/experience.
>>> > … which reads to me like: “It has always been that way so we will not
>>> change
>>> > that" doesn’t it? Always a very poor argument.
>>>
>>> No, what I said is "upsetting our existing developers is something which
>>> we may
>>> want to potentially avoid".
>>>
>>> There are some options out there which have offered mailing list
>>> integration for
>>> example and we have tried to explore those. Something which drops the
>>> lists
>>> entirely could in the current view of the TSC result in loss of too many
>>> of our
>>> existing maintainers.
>>>
>>> >
>>> > > We do have CI which is shared with Yocto Project and
>>> > > OpenEmbedded. You can see it here:
>>> > >
>>> > > https://autobuilder.yoctoproject.org/typhoon/#/console
>>> > >
>>> > > and it has it's own manual which was recently added to our
>>> documentations
>>> > > set:
>>> > >
>>> > > http://docs.yoctoproject.org/test-manual/index.html
>>> > … stressing my point: Why does the location of the CI need
>>> documentation? All
>>> > of the aforementioned platforms smoothly integrate with multiple CI
>>> solutions,
>>> > drastically reducing the time required for reviews if the pipeline
>>> provides
>>> > automated feedback to the contributor. It would probably cost me about
>>> one
>>> > afternoon to setup a decent CI pipeline at least for all python codes
>>> pushing
>>> > the need to review from manual labor to automated tools. If open
>>> embedded
>>> > migrates to a proper git platform I happily volunteer on that end!!
>>>
>>> We're not talking about simple python code. We're talking about testing
>>> a build
>>> environment which cross compiles multiple operating systems for multiple
>>> architectures, C libraries, GUI stacks, init systems and so on. Testing
>>> the
>>> python code is one small part of that. I've spent several years trying to
>>> improve and get our CI to where it is today.
>>>
>>> Yes, the python code is tested as part of this as we don't really want
>>> to have
>>> multiple test platforms and the real world usage is some of the best
>>> testing
>>> changes can have.
>>>
>>> > > There is work still needed on that but it is a start. One challenge
>>> of
>>> > > bitbake/OE is that the builds take an age and the test matrix is
>>> huge so the
>>> > > CI
>>> > > process isn't quite as simple as some other simpler projects.
>>> > At least for python I bet that most bugs can be caught simply by
>>> firing up
>>> > `black`, `flake8`, `pylint` (and a hand full of other tools) just to
>>> check the
>>> > modified files.
>>>
>>> Well, I would bet that most of our bugs are not caused by issues like
>>> that. I do
>>> attend the weekly bug triage meetings and have fixed my share of them.
>>> You could
>>> have a look at the issues filed in bugzilla over the past few weeks if
>>> you'd
>>> like to see the kinds of issues that get filed and are open.
>>>
>>> >  Regarding meta-layers and recipes out there we are currently pushing
>>> forward
>>> > with a linter for bitbake recipes (thanks @priv-kweihmann for this
>>> awesome and
>>> > overdue tool!!!).
>>>
>>> Whilst improving recipes in that way does help, it isn't the main source
>>> of our
>>> major issues.
>>>
>>> > >
>>> > > As a shortcut through things, the "bitbake-selftest" command within
>>> bitbake
>>> > > may
>>> > > be one thing you're looking for which does simple tests on bitbake.
>>> It isn't
>>> > > complete coverage and we welcome more being added, it is a start and
>>> some
>>> > > areas
>>> > > of the code are covered better than others.
>>> > I’d love to overhaul the tests e.g. porting them to `pytest` which
>>> offers a
>>> > much simpler yet much more powerful API/plugin-ecosystem. But won’t do
>>> via
>>> > email :)
>>>
>>> The tests are written in python's own unittest.
>>>
>>> > > Issues are filed in the project bugzilla and are actively worked on:
>>> > >
>>> > > https://bugzilla.yoctoproject.org/
>>> > Didn’t know that. But why keeping issues separate from MRs? Automatic
>>> issue
>>> > closing through referencing in MRs is a pretty nice feature…
>>>
>>> If I had to list the pain points we have, this is not high on the list.
>>>
>>> > > > * side-note on bitbake style/linting: It’s violating PEP8 in a
>>> couple of
>>> > > > ways and TBH PEP8 is the bare minimum I would accept these days for
>>> > > > collaboration on projects the dimension of bitbake. So an
>>> additional
>>> > > > question: Would you be up for a PEP8 compatibility overhaul of the
>>> source
>>> > > > code? I’d volunteer if I don’t have to send a patch via email ;)
>>> > >
>>> > > We're interested in patches to improve the coding style as long as
>>> they
>>> > > don't
>>> > > break things like readability or performance. Which rules in PEP8 in
>>> > > particular
>>> > > are you referring to?
>>> >
>>> > To name a few violations:
>>> >
>>> >  * hanging indents
>>> >  * inconsistent indentation
>>> >  * 2 separating lines missing in between classes/top level methods
>>>
>>> FWIW I find it really hard to get worked up about whitespace issues. I
>>> appreciate these aren't always great to look at and I appreciate the
>>> familiarity
>>> argument and standardising the look but these aren't issues I would
>>> prioritise.
>>>
>>> In particular changing them globally in mass patches tends to break patch
>>> backporting and taking fixes back to older code which we do benefit from
>>> being
>>> able to do so there is a cost to doing it. We are trying to make new
>>> code more
>>> consistent and with older code, we do try and fix the worst issues.
>>>
>>> >  * unused imported modules
>>> >  * missing white-spaces after comma
>>> >  * unreachable codes
>>>
>>> These ones we do take patches for and try to fix when/where we see it.
>>> For
>>> example,
>>> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511
>>> d830df9cbd4113a4352b2c
>>> <http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511d830df9cbd4113a4352b2c> contained
>>> a couple of white-space after comma fixing
>>> while I was in that area.
>>>
>>> > These are just a couple of examples that struck my PEP8 used eye from
>>> looking
>>> > at two, three modules.
>>> >
>>> > It’s just like with any code smell: they don’t necessarily break
>>> things. But
>>> > they make maintenance, code-reviews and quality control harder than
>>> necessary.
>>> > Code that does not at least follow PEP8 (and I actually strongly
>>> recommend to
>>> > go the extra mile with `black`) just looks odd and is unnecessary hard
>>> to
>>> > read. I’m surprised I have to argue for standards.
>>>
>>> I wasn't saying it was a bad thing, I asked which issues you noticed in
>>> particular since I'm trying to ensure we do at least try and react to
>>> feedback.
>>> Some whitespace issues we aren't likely to tackle, some like the others
>>> you
>>> mentioned above we should fix.
>>> >
>>> > Besides that, some minor cleanup just to warm
>>> > up: https://github.com/openembedded/bitbake/pull/33,
>>> https://github.com/openem
>>> > bedded/bitbake/pull/32/files
>>>
>>> Those two do look like reasonable changes FWIW.
>>>
>>> > (it was after these two that I found the comment in one of those old
>>> commits
>>> > which made me open https://github.com/openembedded/bitbake/pull/34).
>>> At least
>>> > there should be an automated reply to each MR that is posted if
>>> someone makes
>>> > the effort to fix code and open a MRs that this is actually in vein -
>>> That’s
>>> > the mood that took me here :)
>>>
>>> There is a certain irony in that given you patched the section called
>>> "Contributing" in the README in 32!
>>>
>>> > >
>>> > > Help appreciated but email patches is how we accept them at the
>>> moment.
>>> >
>>> > Not from me then :(
>>>
>>> Sorry to hear that, your changes in 32/33 do look reasonable.
>>>
>>> Cheers,
>>>
>>> Richard
>>>
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#12951):
>>> https://lists.openembedded.org/g/bitbake-devel/message/12951
>>> Mute This Topic: https://lists.openembedded.org/mt/86911583/1686489
>>> Group Owner: bitbake-devel+owner@lists.openembedded.org
>>> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
>>> alex.kanavin@gmail.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>
>>>

[-- Attachment #2: Type: text/html, Size: 17890 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Marius Kriegerowski]

[-- Attachment #1: Type: text/plain, Size: 28728 bytes --]

I see.

On 9. Nov 2021, at 09:19, Alexander Kanavin <alex.kanavin@gmail.com> wrote:

I think you misunderstood me too. It's about the github UI which does its
best to hide the commit messages, and doesn't allow to comment on their
content. Yes you can contort yourself and still do it, but the tools do not
help.


I never felt that the commit messages are really that hidden. I mean… you
find them under `commits`... I find that pretty transparent. You can
comment on the commit message content in the thread of the MR, just as you
would in an email thread, can't you?

Also, the older and closed threads can be very helpful when searching for
answers why something has been changed in which way. I know that I saw the
email threads on patches as well but found them super hard to read (no
syntax highlighting, all black and white, …). Well, probably I could get
used to reading them over time. But then — why should I if there are things
like collapsable threads, syntax highlight, color,…?

Best
Marius


Alex

On Tue, 9 Nov 2021 at 09:05, Marius Kriegerowski <
marius.kriegerowski@gmail.com> wrote:

> Good morning,
>
> I don’t think this is correct. You can change the git history and commit
> messages of every MR to whatever you like with `git commit —amend`, `git
> rebase -I` and `git push -f`. So, you can very well ask contributors to
> cleanup their history before merging.
>
> Best
> Marius
>
> Am Di., 9. Nov. 2021 um 07:47 Uhr schrieb Alexander Kanavin <
> alex.kanavin@gmail.com>:
>
>> There's another problem with github/gitlab which I find significant.
>> These tools do not allow reviewing commit messages, and don't even make it
>> easy to see them. So the developers using those tools, both submitters and
>> maintainers, are effectively being conditioned by the tools to see good
>> commit messages as superfluous and unnecessary. And so you more often than
>> not end up with poorly written commit history in github projects, or, if
>> you're a reviewer, have to repeatedly explain to the submitter how to
>> explain and structure their changes properly.
>>
>> I'd rather not have a change, than have a change that no one, even the
>> original author, can understand or remember the reasons for one year down
>> the line.
>>
>> Alex
>>
>>
>> On Tue, 9 Nov 2021 at 00:30, Richard Purdie <
>> richard.purdie@linuxfoundation.org> wrote:
>>
>>> On Mon, 2021-11-08 at 23:44 +0100, Marius Kriegerowski wrote:
>>> >
>>> >
>>> > > On 8. Nov 2021, at 22:47, Richard Purdie
>>> > > <richard.purdie@linuxfoundation.org> wrote:
>>> > >
>>> > > On Mon, 2021-11-08 at 18:37 +0100, Marius Kriegerowski wrote:
>>> > > > Is there a plan to switch to some more modern ways to collaborate
>>> on
>>> > > > bitbake,
>>> > > > you know, like, gitlab/github/gitea with issues, PRs and so forth?
>>> I mean…
>>> > > > it’s
>>> > > > late 2021 and sending MRs/Issues via email feels reaaaally
>>> outdated, slow,
>>> > > > intransparent, ineffective (how to you do reviews, code
>>> comparisons during
>>> > > > reviews, insert suggestions, etc.) and indirect (where is the CI
>>> pipeline
>>> > > > that I
>>> > > > trigger when pushing something somewhere?).
>>> > >
>>> > > There are pros and cons to switching to something else. Keep in mind
>>> that
>>> > > whilst the github workflow works well for a single maintainer of a
>>> project,
>>> > > it does not work well for diverse review.
>>> >
>>> > I strongly disagree with that. Reviews can equally well be done in a MR
>>> > thread. That’s what they are there for. You can discuss things there
>>> with as
>>> > many people as you want and can e.g. define a minimum number of
>>> approvals
>>> > before a MR can be merged.
>>>
>>> You misunderstand my point. We have a several thousand people on the
>>> mailing
>>> lists and they filter things to what interests them. *I* have no people
>>> I want
>>> to discuss a given change with, it is a question of whether the people
>>> who have
>>> an interest in a given change would see it.
>>>
>>> A minimum number of approvals is utterly meaningless for how we work, I
>>> need the
>>> interested people to see it and that varies massively depending on what
>>> is being
>>> changed. It is this aspect of mailing list change distribution and
>>> discussion we
>>> benefit from.
>>>
>>> >  It also naturally groups discussions. You can address experts for
>>> specific
>>> > domains which means less noise for the experts. I really don’t see why
>>> mailing
>>> > list reviews should work any better here.
>>>
>>> We all have established mechanisms to "see" the things which interest us
>>> as
>>> maintainers?
>>>
>>> Please don't misunderstand me, I do see the attraction of github from
>>> your
>>> perspective. It would just destroy the ecosystem we already have. You are
>>> basically saying that our current ecosystem is worthless but I would
>>> point out
>>> it has resulted in a project which you did say you appreciate so it
>>> can't be all
>>> bad.
>>>
>>> >  If google can coordinate 2.1k repos with 1.1k contributors over
>>> GitHub, why
>>> > should that not work for openembedded??
>>>
>>> It isn't about raw numbers, it is about numbers of reviewers for a given
>>> change.
>>> We have hundreds of people doing that using the mailing lists.
>>>
>>> > > The Bitbake case is a little bit different but in general we'd
>>> follow what
>>> > > OE/Yocto Project are doing.
>>> > >
>>> > > We have discussed changing and it is listed as one topic on:
>>> > > https://wiki.yoctoproject.org/wiki/Future_Directions
>>> > > (Code Submission process)
>>> > >
>>> > > I'd note that it is a huge change to our current developers and most
>>> of us
>>> > > are very used to the mailing lists so if we did change, it would
>>> allow
>>> > > potential new collaborators at the risk of losing people with key
>>> > > knowledge/experience.
>>> > … which reads to me like: “It has always been that way so we will not
>>> change
>>> > that" doesn’t it? Always a very poor argument.
>>>
>>> No, what I said is "upsetting our existing developers is something which
>>> we may
>>> want to potentially avoid".
>>>
>>> There are some options out there which have offered mailing list
>>> integration for
>>> example and we have tried to explore those. Something which drops the
>>> lists
>>> entirely could in the current view of the TSC result in loss of too many
>>> of our
>>> existing maintainers.
>>>
>>> >
>>> > > We do have CI which is shared with Yocto Project and
>>> > > OpenEmbedded. You can see it here:
>>> > >
>>> > > https://autobuilder.yoctoproject.org/typhoon/#/console
>>> > >
>>> > > and it has it's own manual which was recently added to our
>>> documentations
>>> > > set:
>>> > >
>>> > > http://docs.yoctoproject.org/test-manual/index.html
>>> > … stressing my point: Why does the location of the CI need
>>> documentation? All
>>> > of the aforementioned platforms smoothly integrate with multiple CI
>>> solutions,
>>> > drastically reducing the time required for reviews if the pipeline
>>> provides
>>> > automated feedback to the contributor. It would probably cost me about
>>> one
>>> > afternoon to setup a decent CI pipeline at least for all python codes
>>> pushing
>>> > the need to review from manual labor to automated tools. If open
>>> embedded
>>> > migrates to a proper git platform I happily volunteer on that end!!
>>>
>>> We're not talking about simple python code. We're talking about testing
>>> a build
>>> environment which cross compiles multiple operating systems for multiple
>>> architectures, C libraries, GUI stacks, init systems and so on. Testing
>>> the
>>> python code is one small part of that. I've spent several years trying to
>>> improve and get our CI to where it is today.
>>>
>>> Yes, the python code is tested as part of this as we don't really want
>>> to have
>>> multiple test platforms and the real world usage is some of the best
>>> testing
>>> changes can have.
>>>
>>> > > There is work still needed on that but it is a start. One challenge
>>> of
>>> > > bitbake/OE is that the builds take an age and the test matrix is
>>> huge so the
>>> > > CI
>>> > > process isn't quite as simple as some other simpler projects.
>>> > At least for python I bet that most bugs can be caught simply by
>>> firing up
>>> > `black`, `flake8`, `pylint` (and a hand full of other tools) just to
>>> check the
>>> > modified files.
>>>
>>> Well, I would bet that most of our bugs are not caused by issues like
>>> that. I do
>>> attend the weekly bug triage meetings and have fixed my share of them.
>>> You could
>>> have a look at the issues filed in bugzilla over the past few weeks if
>>> you'd
>>> like to see the kinds of issues that get filed and are open.
>>>
>>> >  Regarding meta-layers and recipes out there we are currently pushing
>>> forward
>>> > with a linter for bitbake recipes (thanks @priv-kweihmann for this
>>> awesome and
>>> > overdue tool!!!).
>>>
>>> Whilst improving recipes in that way does help, it isn't the main source
>>> of our
>>> major issues.
>>>
>>> > >
>>> > > As a shortcut through things, the "bitbake-selftest" command within
>>> bitbake
>>> > > may
>>> > > be one thing you're looking for which does simple tests on bitbake.
>>> It isn't
>>> > > complete coverage and we welcome more being added, it is a start and
>>> some
>>> > > areas
>>> > > of the code are covered better than others.
>>> > I’d love to overhaul the tests e.g. porting them to `pytest` which
>>> offers a
>>> > much simpler yet much more powerful API/plugin-ecosystem. But won’t do
>>> via
>>> > email :)
>>>
>>> The tests are written in python's own unittest.
>>>
>>> > > Issues are filed in the project bugzilla and are actively worked on:
>>> > >
>>> > > https://bugzilla.yoctoproject.org/
>>> > Didn’t know that. But why keeping issues separate from MRs? Automatic
>>> issue
>>> > closing through referencing in MRs is a pretty nice feature…
>>>
>>> If I had to list the pain points we have, this is not high on the list.
>>>
>>> > > > * side-note on bitbake style/linting: It’s violating PEP8 in a
>>> couple of
>>> > > > ways and TBH PEP8 is the bare minimum I would accept these days for
>>> > > > collaboration on projects the dimension of bitbake. So an
>>> additional
>>> > > > question: Would you be up for a PEP8 compatibility overhaul of the
>>> source
>>> > > > code? I’d volunteer if I don’t have to send a patch via email ;)
>>> > >
>>> > > We're interested in patches to improve the coding style as long as
>>> they
>>> > > don't
>>> > > break things like readability or performance. Which rules in PEP8 in
>>> > > particular
>>> > > are you referring to?
>>> >
>>> > To name a few violations:
>>> >
>>> >  * hanging indents
>>> >  * inconsistent indentation
>>> >  * 2 separating lines missing in between classes/top level methods
>>>
>>> FWIW I find it really hard to get worked up about whitespace issues. I
>>> appreciate these aren't always great to look at and I appreciate the
>>> familiarity
>>> argument and standardising the look but these aren't issues I would
>>> prioritise.
>>>
>>> In particular changing them globally in mass patches tends to break patch
>>> backporting and taking fixes back to older code which we do benefit from
>>> being
>>> able to do so there is a cost to doing it. We are trying to make new
>>> code more
>>> consistent and with older code, we do try and fix the worst issues.
>>>
>>> >  * unused imported modules
>>> >  * missing white-spaces after comma
>>> >  * unreachable codes
>>>
>>> These ones we do take patches for and try to fix when/where we see it.
>>> For
>>> example,
>>> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511
>>> d830df9cbd4113a4352b2c contained a couple of white-space after comma
>>> fixing
>>> while I was in that area.
>>>
>>> > These are just a couple of examples that struck my PEP8 used eye from
>>> looking
>>> > at two, three modules.
>>> >
>>> > It’s just like with any code smell: they don’t necessarily break
>>> things. But
>>> > they make maintenance, code-reviews and quality control harder than
>>> necessary.
>>> > Code that does not at least follow PEP8 (and I actually strongly
>>> recommend to
>>> > go the extra mile with `black`) just looks odd and is unnecessary hard
>>> to
>>> > read. I’m surprised I have to argue for standards.
>>>
>>> I wasn't saying it was a bad thing, I asked which issues you noticed in
>>> particular since I'm trying to ensure we do at least try and react to
>>> feedback.
>>> Some whitespace issues we aren't likely to tackle, some like the others
>>> you
>>> mentioned above we should fix.
>>> >
>>> > Besides that, some minor cleanup just to warm
>>> > up: https://github.com/openembedded/bitbake/pull/33,
>>> https://github.com/openem
>>> > bedded/bitbake/pull/32/files
>>>
>>> Those two do look like reasonable changes FWIW.
>>>
>>> > (it was after these two that I found the comment in one of those old
>>> commits
>>> > which made me open https://github.com/openembedded/bitbake/pull/34).
>>> At least
>>> > there should be an automated reply to each MR that is posted if
>>> someone makes
>>> > the effort to fix code and open a MRs that this is actually in vein -
>>> That’s
>>> > the mood that took me here :)
>>>
>>> There is a certain irony in that given you patched the section called
>>> "Contributing" in the README in 32!
>>>
>>> > >
>>> > > Help appreciated but email patches is how we accept them at the
>>> moment.
>>> >
>>> > Not from me then :(
>>>
>>> Sorry to hear that, your changes in 32/33 do look reasonable.
>>>
>>> Cheers,
>>>
>>> Richard
>>>
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#12951):
>>> https://lists.openembedded.org/g/bitbake-devel/message/12951
>>> Mute This Topic: https://lists.openembedded.org/mt/86911583/1686489
>>> Group Owner: bitbake-devel+owner@lists.openembedded.org
>>> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
>>> alex.kanavin@gmail.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>
>>>


Am Di., 9. Nov. 2021 um 09:19 Uhr schrieb Alexander Kanavin <
alex.kanavin@gmail.com>:

> I think you misunderstood me too. It's about the github UI which does its
> best to hide the commit messages, and doesn't allow to comment on their
> content. Yes you can contort yourself and still do it, but the tools do not
> help.
>
> Alex
>
> On Tue, 9 Nov 2021 at 09:05, Marius Kriegerowski <
> marius.kriegerowski@gmail.com> wrote:
>
>> Good morning,
>>
>> I don’t think this is correct. You can change the git history and commit
>> messages of every MR to whatever you like with `git commit —amend`, `git
>> rebase -I` and `git push -f`. So, you can very well ask contributors to
>> cleanup their history before merging.
>>
>> Best
>> Marius
>>
>> Am Di., 9. Nov. 2021 um 07:47 Uhr schrieb Alexander Kanavin <
>> alex.kanavin@gmail.com>:
>>
>>> There's another problem with github/gitlab which I find significant.
>>> These tools do not allow reviewing commit messages, and don't even make it
>>> easy to see them. So the developers using those tools, both submitters and
>>> maintainers, are effectively being conditioned by the tools to see good
>>> commit messages as superfluous and unnecessary. And so you more often than
>>> not end up with poorly written commit history in github projects, or, if
>>> you're a reviewer, have to repeatedly explain to the submitter how to
>>> explain and structure their changes properly.
>>>
>>> I'd rather not have a change, than have a change that no one, even the
>>> original author, can understand or remember the reasons for one year down
>>> the line.
>>>
>>> Alex
>>>
>>>
>>> On Tue, 9 Nov 2021 at 00:30, Richard Purdie <
>>> richard.purdie@linuxfoundation.org> wrote:
>>>
>>>> On Mon, 2021-11-08 at 23:44 +0100, Marius Kriegerowski wrote:
>>>> >
>>>> >
>>>> > > On 8. Nov 2021, at 22:47, Richard Purdie
>>>> > > <richard.purdie@linuxfoundation.org> wrote:
>>>> > >
>>>> > > On Mon, 2021-11-08 at 18:37 +0100, Marius Kriegerowski wrote:
>>>> > > > Is there a plan to switch to some more modern ways to collaborate
>>>> on
>>>> > > > bitbake,
>>>> > > > you know, like, gitlab/github/gitea with issues, PRs and so
>>>> forth? I mean…
>>>> > > > it’s
>>>> > > > late 2021 and sending MRs/Issues via email feels reaaaally
>>>> outdated, slow,
>>>> > > > intransparent, ineffective (how to you do reviews, code
>>>> comparisons during
>>>> > > > reviews, insert suggestions, etc.) and indirect (where is the CI
>>>> pipeline
>>>> > > > that I
>>>> > > > trigger when pushing something somewhere?).
>>>> > >
>>>> > > There are pros and cons to switching to something else. Keep in
>>>> mind that
>>>> > > whilst the github workflow works well for a single maintainer of a
>>>> project,
>>>> > > it does not work well for diverse review.
>>>> >
>>>> > I strongly disagree with that. Reviews can equally well be done in a
>>>> MR
>>>> > thread. That’s what they are there for. You can discuss things there
>>>> with as
>>>> > many people as you want and can e.g. define a minimum number of
>>>> approvals
>>>> > before a MR can be merged.
>>>>
>>>> You misunderstand my point. We have a several thousand people on the
>>>> mailing
>>>> lists and they filter things to what interests them. *I* have no people
>>>> I want
>>>> to discuss a given change with, it is a question of whether the people
>>>> who have
>>>> an interest in a given change would see it.
>>>>
>>>> A minimum number of approvals is utterly meaningless for how we work, I
>>>> need the
>>>> interested people to see it and that varies massively depending on what
>>>> is being
>>>> changed. It is this aspect of mailing list change distribution and
>>>> discussion we
>>>> benefit from.
>>>>
>>>> >  It also naturally groups discussions. You can address experts for
>>>> specific
>>>> > domains which means less noise for the experts. I really don’t see
>>>> why mailing
>>>> > list reviews should work any better here.
>>>>
>>>> We all have established mechanisms to "see" the things which interest
>>>> us as
>>>> maintainers?
>>>>
>>>> Please don't misunderstand me, I do see the attraction of github from
>>>> your
>>>> perspective. It would just destroy the ecosystem we already have. You
>>>> are
>>>> basically saying that our current ecosystem is worthless but I would
>>>> point out
>>>> it has resulted in a project which you did say you appreciate so it
>>>> can't be all
>>>> bad.
>>>>
>>>> >  If google can coordinate 2.1k repos with 1.1k contributors over
>>>> GitHub, why
>>>> > should that not work for openembedded??
>>>>
>>>> It isn't about raw numbers, it is about numbers of reviewers for a
>>>> given change.
>>>> We have hundreds of people doing that using the mailing lists.
>>>>
>>>> > > The Bitbake case is a little bit different but in general we'd
>>>> follow what
>>>> > > OE/Yocto Project are doing.
>>>> > >
>>>> > > We have discussed changing and it is listed as one topic on:
>>>> > > https://wiki.yoctoproject.org/wiki/Future_Directions
>>>> > > (Code Submission process)
>>>> > >
>>>> > > I'd note that it is a huge change to our current developers and
>>>> most of us
>>>> > > are very used to the mailing lists so if we did change, it would
>>>> allow
>>>> > > potential new collaborators at the risk of losing people with key
>>>> > > knowledge/experience.
>>>> > … which reads to me like: “It has always been that way so we will not
>>>> change
>>>> > that" doesn’t it? Always a very poor argument.
>>>>
>>>> No, what I said is "upsetting our existing developers is something
>>>> which we may
>>>> want to potentially avoid".
>>>>
>>>> There are some options out there which have offered mailing list
>>>> integration for
>>>> example and we have tried to explore those. Something which drops the
>>>> lists
>>>> entirely could in the current view of the TSC result in loss of too
>>>> many of our
>>>> existing maintainers.
>>>>
>>>> >
>>>> > > We do have CI which is shared with Yocto Project and
>>>> > > OpenEmbedded. You can see it here:
>>>> > >
>>>> > > https://autobuilder.yoctoproject.org/typhoon/#/console
>>>> > >
>>>> > > and it has it's own manual which was recently added to our
>>>> documentations
>>>> > > set:
>>>> > >
>>>> > > http://docs.yoctoproject.org/test-manual/index.html
>>>> > … stressing my point: Why does the location of the CI need
>>>> documentation? All
>>>> > of the aforementioned platforms smoothly integrate with multiple CI
>>>> solutions,
>>>> > drastically reducing the time required for reviews if the pipeline
>>>> provides
>>>> > automated feedback to the contributor. It would probably cost me
>>>> about one
>>>> > afternoon to setup a decent CI pipeline at least for all python codes
>>>> pushing
>>>> > the need to review from manual labor to automated tools. If open
>>>> embedded
>>>> > migrates to a proper git platform I happily volunteer on that end!!
>>>>
>>>> We're not talking about simple python code. We're talking about testing
>>>> a build
>>>> environment which cross compiles multiple operating systems for multiple
>>>> architectures, C libraries, GUI stacks, init systems and so on. Testing
>>>> the
>>>> python code is one small part of that. I've spent several years trying
>>>> to
>>>> improve and get our CI to where it is today.
>>>>
>>>> Yes, the python code is tested as part of this as we don't really want
>>>> to have
>>>> multiple test platforms and the real world usage is some of the best
>>>> testing
>>>> changes can have.
>>>>
>>>> > > There is work still needed on that but it is a start. One challenge
>>>> of
>>>> > > bitbake/OE is that the builds take an age and the test matrix is
>>>> huge so the
>>>> > > CI
>>>> > > process isn't quite as simple as some other simpler projects.
>>>> > At least for python I bet that most bugs can be caught simply by
>>>> firing up
>>>> > `black`, `flake8`, `pylint` (and a hand full of other tools) just to
>>>> check the
>>>> > modified files.
>>>>
>>>> Well, I would bet that most of our bugs are not caused by issues like
>>>> that. I do
>>>> attend the weekly bug triage meetings and have fixed my share of them.
>>>> You could
>>>> have a look at the issues filed in bugzilla over the past few weeks if
>>>> you'd
>>>> like to see the kinds of issues that get filed and are open.
>>>>
>>>> >  Regarding meta-layers and recipes out there we are currently pushing
>>>> forward
>>>> > with a linter for bitbake recipes (thanks @priv-kweihmann for this
>>>> awesome and
>>>> > overdue tool!!!).
>>>>
>>>> Whilst improving recipes in that way does help, it isn't the main
>>>> source of our
>>>> major issues.
>>>>
>>>> > >
>>>> > > As a shortcut through things, the "bitbake-selftest" command within
>>>> bitbake
>>>> > > may
>>>> > > be one thing you're looking for which does simple tests on bitbake.
>>>> It isn't
>>>> > > complete coverage and we welcome more being added, it is a start
>>>> and some
>>>> > > areas
>>>> > > of the code are covered better than others.
>>>> > I’d love to overhaul the tests e.g. porting them to `pytest` which
>>>> offers a
>>>> > much simpler yet much more powerful API/plugin-ecosystem. But won’t
>>>> do via
>>>> > email :)
>>>>
>>>> The tests are written in python's own unittest.
>>>>
>>>> > > Issues are filed in the project bugzilla and are actively worked on:
>>>> > >
>>>> > > https://bugzilla.yoctoproject.org/
>>>> > Didn’t know that. But why keeping issues separate from MRs? Automatic
>>>> issue
>>>> > closing through referencing in MRs is a pretty nice feature…
>>>>
>>>> If I had to list the pain points we have, this is not high on the list.
>>>>
>>>> > > > * side-note on bitbake style/linting: It’s violating PEP8 in a
>>>> couple of
>>>> > > > ways and TBH PEP8 is the bare minimum I would accept these days
>>>> for
>>>> > > > collaboration on projects the dimension of bitbake. So an
>>>> additional
>>>> > > > question: Would you be up for a PEP8 compatibility overhaul of
>>>> the source
>>>> > > > code? I’d volunteer if I don’t have to send a patch via email ;)
>>>> > >
>>>> > > We're interested in patches to improve the coding style as long as
>>>> they
>>>> > > don't
>>>> > > break things like readability or performance. Which rules in PEP8 in
>>>> > > particular
>>>> > > are you referring to?
>>>> >
>>>> > To name a few violations:
>>>> >
>>>> >  * hanging indents
>>>> >  * inconsistent indentation
>>>> >  * 2 separating lines missing in between classes/top level methods
>>>>
>>>> FWIW I find it really hard to get worked up about whitespace issues. I
>>>> appreciate these aren't always great to look at and I appreciate the
>>>> familiarity
>>>> argument and standardising the look but these aren't issues I would
>>>> prioritise.
>>>>
>>>> In particular changing them globally in mass patches tends to break
>>>> patch
>>>> backporting and taking fixes back to older code which we do benefit
>>>> from being
>>>> able to do so there is a cost to doing it. We are trying to make new
>>>> code more
>>>> consistent and with older code, we do try and fix the worst issues.
>>>>
>>>> >  * unused imported modules
>>>> >  * missing white-spaces after comma
>>>> >  * unreachable codes
>>>>
>>>> These ones we do take patches for and try to fix when/where we see it.
>>>> For
>>>> example,
>>>> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511
>>>> d830df9cbd4113a4352b2c
>>>> <http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511d830df9cbd4113a4352b2c> contained
>>>> a couple of white-space after comma fixing
>>>> while I was in that area.
>>>>
>>>> > These are just a couple of examples that struck my PEP8 used eye from
>>>> looking
>>>> > at two, three modules.
>>>> >
>>>> > It’s just like with any code smell: they don’t necessarily break
>>>> things. But
>>>> > they make maintenance, code-reviews and quality control harder than
>>>> necessary.
>>>> > Code that does not at least follow PEP8 (and I actually strongly
>>>> recommend to
>>>> > go the extra mile with `black`) just looks odd and is unnecessary
>>>> hard to
>>>> > read. I’m surprised I have to argue for standards.
>>>>
>>>> I wasn't saying it was a bad thing, I asked which issues you noticed in
>>>> particular since I'm trying to ensure we do at least try and react to
>>>> feedback.
>>>> Some whitespace issues we aren't likely to tackle, some like the others
>>>> you
>>>> mentioned above we should fix.
>>>> >
>>>> > Besides that, some minor cleanup just to warm
>>>> > up: https://github.com/openembedded/bitbake/pull/33,
>>>> https://github.com/openem
>>>> > bedded/bitbake/pull/32/files
>>>>
>>>> Those two do look like reasonable changes FWIW.
>>>>
>>>> > (it was after these two that I found the comment in one of those old
>>>> commits
>>>> > which made me open https://github.com/openembedded/bitbake/pull/34).
>>>> At least
>>>> > there should be an automated reply to each MR that is posted if
>>>> someone makes
>>>> > the effort to fix code and open a MRs that this is actually in vein -
>>>> That’s
>>>> > the mood that took me here :)
>>>>
>>>> There is a certain irony in that given you patched the section called
>>>> "Contributing" in the README in 32!
>>>>
>>>> > >
>>>> > > Help appreciated but email patches is how we accept them at the
>>>> moment.
>>>> >
>>>> > Not from me then :(
>>>>
>>>> Sorry to hear that, your changes in 32/33 do look reasonable.
>>>>
>>>> Cheers,
>>>>
>>>> Richard
>>>>
>>>>
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>> Links: You receive all messages sent to this group.
>>>> View/Reply Online (#12951):
>>>> https://lists.openembedded.org/g/bitbake-devel/message/12951
>>>> Mute This Topic: https://lists.openembedded.org/mt/86911583/1686489
>>>> Group Owner: bitbake-devel+owner@lists.openembedded.org
>>>> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
>>>> alex.kanavin@gmail.com]
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>
>>>>

[-- Attachment #2: Type: text/html, Size: 43330 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 30132 bytes --]

On Tue, 9 Nov 2021 at 09:32, Marius Kriegerowski <
marius.kriegerowski@gmail.com> wrote:

> I see.
>
> On 9. Nov 2021, at 09:19, Alexander Kanavin <alex.kanavin@gmail.com>
> wrote:
>
> I think you misunderstood me too. It's about the github UI which does its
> best to hide the commit messages, and doesn't allow to comment on their
> content. Yes you can contort yourself and still do it, but the tools do not
> help.
>
>
> I never felt that the commit messages are really that hidden. I mean… you
> find them under `commits`... I find that pretty transparent. You can
> comment on the commit message content in the thread of the MR, just as you
> would in an email thread, can't you?
>

No. You need to separately go to the commits tab, open each commit
individually to see both the message and the associated code, then if
there's something you do not like, you need to copy paste the offending bit
and open a MR thread with it. That's not helpful, and it's clear that kind
of thing is not important to the tool providers or users. But it is very
important to me as one of the oe-core maintainers.


> Also, the older and closed threads can be very helpful when searching for
> answers why something has been changed in which way. I know that I saw the
> email threads on patches as well but found them super hard to read (no
> syntax highlighting, all black and white, …). Well, probably I could get
> used to reading them over time. But then — why should I if there are things
> like collapsable threads, syntax highlight, color,…?
>

You can find a better email client perhaps?

Alex


>
> Best
> Marius
>
>
> Alex
>
> On Tue, 9 Nov 2021 at 09:05, Marius Kriegerowski <
> marius.kriegerowski@gmail.com> wrote:
>
>> Good morning,
>>
>> I don’t think this is correct. You can change the git history and commit
>> messages of every MR to whatever you like with `git commit —amend`, `git
>> rebase -I` and `git push -f`. So, you can very well ask contributors to
>> cleanup their history before merging.
>>
>> Best
>> Marius
>>
>> Am Di., 9. Nov. 2021 um 07:47 Uhr schrieb Alexander Kanavin <
>> alex.kanavin@gmail.com>:
>>
>>> There's another problem with github/gitlab which I find significant.
>>> These tools do not allow reviewing commit messages, and don't even make it
>>> easy to see them. So the developers using those tools, both submitters and
>>> maintainers, are effectively being conditioned by the tools to see good
>>> commit messages as superfluous and unnecessary. And so you more often than
>>> not end up with poorly written commit history in github projects, or, if
>>> you're a reviewer, have to repeatedly explain to the submitter how to
>>> explain and structure their changes properly.
>>>
>>> I'd rather not have a change, than have a change that no one, even the
>>> original author, can understand or remember the reasons for one year down
>>> the line.
>>>
>>> Alex
>>>
>>>
>>> On Tue, 9 Nov 2021 at 00:30, Richard Purdie <
>>> richard.purdie@linuxfoundation.org> wrote:
>>>
>>>> On Mon, 2021-11-08 at 23:44 +0100, Marius Kriegerowski wrote:
>>>> >
>>>> >
>>>> > > On 8. Nov 2021, at 22:47, Richard Purdie
>>>> > > <richard.purdie@linuxfoundation.org> wrote:
>>>> > >
>>>> > > On Mon, 2021-11-08 at 18:37 +0100, Marius Kriegerowski wrote:
>>>> > > > Is there a plan to switch to some more modern ways to collaborate
>>>> on
>>>> > > > bitbake,
>>>> > > > you know, like, gitlab/github/gitea with issues, PRs and so
>>>> forth? I mean…
>>>> > > > it’s
>>>> > > > late 2021 and sending MRs/Issues via email feels reaaaally
>>>> outdated, slow,
>>>> > > > intransparent, ineffective (how to you do reviews, code
>>>> comparisons during
>>>> > > > reviews, insert suggestions, etc.) and indirect (where is the CI
>>>> pipeline
>>>> > > > that I
>>>> > > > trigger when pushing something somewhere?).
>>>> > >
>>>> > > There are pros and cons to switching to something else. Keep in
>>>> mind that
>>>> > > whilst the github workflow works well for a single maintainer of a
>>>> project,
>>>> > > it does not work well for diverse review.
>>>> >
>>>> > I strongly disagree with that. Reviews can equally well be done in a
>>>> MR
>>>> > thread. That’s what they are there for. You can discuss things there
>>>> with as
>>>> > many people as you want and can e.g. define a minimum number of
>>>> approvals
>>>> > before a MR can be merged.
>>>>
>>>> You misunderstand my point. We have a several thousand people on the
>>>> mailing
>>>> lists and they filter things to what interests them. *I* have no people
>>>> I want
>>>> to discuss a given change with, it is a question of whether the people
>>>> who have
>>>> an interest in a given change would see it.
>>>>
>>>> A minimum number of approvals is utterly meaningless for how we work, I
>>>> need the
>>>> interested people to see it and that varies massively depending on what
>>>> is being
>>>> changed. It is this aspect of mailing list change distribution and
>>>> discussion we
>>>> benefit from.
>>>>
>>>> >  It also naturally groups discussions. You can address experts for
>>>> specific
>>>> > domains which means less noise for the experts. I really don’t see
>>>> why mailing
>>>> > list reviews should work any better here.
>>>>
>>>> We all have established mechanisms to "see" the things which interest
>>>> us as
>>>> maintainers?
>>>>
>>>> Please don't misunderstand me, I do see the attraction of github from
>>>> your
>>>> perspective. It would just destroy the ecosystem we already have. You
>>>> are
>>>> basically saying that our current ecosystem is worthless but I would
>>>> point out
>>>> it has resulted in a project which you did say you appreciate so it
>>>> can't be all
>>>> bad.
>>>>
>>>> >  If google can coordinate 2.1k repos with 1.1k contributors over
>>>> GitHub, why
>>>> > should that not work for openembedded??
>>>>
>>>> It isn't about raw numbers, it is about numbers of reviewers for a
>>>> given change.
>>>> We have hundreds of people doing that using the mailing lists.
>>>>
>>>> > > The Bitbake case is a little bit different but in general we'd
>>>> follow what
>>>> > > OE/Yocto Project are doing.
>>>> > >
>>>> > > We have discussed changing and it is listed as one topic on:
>>>> > > https://wiki.yoctoproject.org/wiki/Future_Directions
>>>> > > (Code Submission process)
>>>> > >
>>>> > > I'd note that it is a huge change to our current developers and
>>>> most of us
>>>> > > are very used to the mailing lists so if we did change, it would
>>>> allow
>>>> > > potential new collaborators at the risk of losing people with key
>>>> > > knowledge/experience.
>>>> > … which reads to me like: “It has always been that way so we will not
>>>> change
>>>> > that" doesn’t it? Always a very poor argument.
>>>>
>>>> No, what I said is "upsetting our existing developers is something
>>>> which we may
>>>> want to potentially avoid".
>>>>
>>>> There are some options out there which have offered mailing list
>>>> integration for
>>>> example and we have tried to explore those. Something which drops the
>>>> lists
>>>> entirely could in the current view of the TSC result in loss of too
>>>> many of our
>>>> existing maintainers.
>>>>
>>>> >
>>>> > > We do have CI which is shared with Yocto Project and
>>>> > > OpenEmbedded. You can see it here:
>>>> > >
>>>> > > https://autobuilder.yoctoproject.org/typhoon/#/console
>>>> > >
>>>> > > and it has it's own manual which was recently added to our
>>>> documentations
>>>> > > set:
>>>> > >
>>>> > > http://docs.yoctoproject.org/test-manual/index.html
>>>> > … stressing my point: Why does the location of the CI need
>>>> documentation? All
>>>> > of the aforementioned platforms smoothly integrate with multiple CI
>>>> solutions,
>>>> > drastically reducing the time required for reviews if the pipeline
>>>> provides
>>>> > automated feedback to the contributor. It would probably cost me
>>>> about one
>>>> > afternoon to setup a decent CI pipeline at least for all python codes
>>>> pushing
>>>> > the need to review from manual labor to automated tools. If open
>>>> embedded
>>>> > migrates to a proper git platform I happily volunteer on that end!!
>>>>
>>>> We're not talking about simple python code. We're talking about testing
>>>> a build
>>>> environment which cross compiles multiple operating systems for multiple
>>>> architectures, C libraries, GUI stacks, init systems and so on. Testing
>>>> the
>>>> python code is one small part of that. I've spent several years trying
>>>> to
>>>> improve and get our CI to where it is today.
>>>>
>>>> Yes, the python code is tested as part of this as we don't really want
>>>> to have
>>>> multiple test platforms and the real world usage is some of the best
>>>> testing
>>>> changes can have.
>>>>
>>>> > > There is work still needed on that but it is a start. One challenge
>>>> of
>>>> > > bitbake/OE is that the builds take an age and the test matrix is
>>>> huge so the
>>>> > > CI
>>>> > > process isn't quite as simple as some other simpler projects.
>>>> > At least for python I bet that most bugs can be caught simply by
>>>> firing up
>>>> > `black`, `flake8`, `pylint` (and a hand full of other tools) just to
>>>> check the
>>>> > modified files.
>>>>
>>>> Well, I would bet that most of our bugs are not caused by issues like
>>>> that. I do
>>>> attend the weekly bug triage meetings and have fixed my share of them.
>>>> You could
>>>> have a look at the issues filed in bugzilla over the past few weeks if
>>>> you'd
>>>> like to see the kinds of issues that get filed and are open.
>>>>
>>>> >  Regarding meta-layers and recipes out there we are currently pushing
>>>> forward
>>>> > with a linter for bitbake recipes (thanks @priv-kweihmann for this
>>>> awesome and
>>>> > overdue tool!!!).
>>>>
>>>> Whilst improving recipes in that way does help, it isn't the main
>>>> source of our
>>>> major issues.
>>>>
>>>> > >
>>>> > > As a shortcut through things, the "bitbake-selftest" command within
>>>> bitbake
>>>> > > may
>>>> > > be one thing you're looking for which does simple tests on bitbake.
>>>> It isn't
>>>> > > complete coverage and we welcome more being added, it is a start
>>>> and some
>>>> > > areas
>>>> > > of the code are covered better than others.
>>>> > I’d love to overhaul the tests e.g. porting them to `pytest` which
>>>> offers a
>>>> > much simpler yet much more powerful API/plugin-ecosystem. But won’t
>>>> do via
>>>> > email :)
>>>>
>>>> The tests are written in python's own unittest.
>>>>
>>>> > > Issues are filed in the project bugzilla and are actively worked on:
>>>> > >
>>>> > > https://bugzilla.yoctoproject.org/
>>>> > Didn’t know that. But why keeping issues separate from MRs? Automatic
>>>> issue
>>>> > closing through referencing in MRs is a pretty nice feature…
>>>>
>>>> If I had to list the pain points we have, this is not high on the list.
>>>>
>>>> > > > * side-note on bitbake style/linting: It’s violating PEP8 in a
>>>> couple of
>>>> > > > ways and TBH PEP8 is the bare minimum I would accept these days
>>>> for
>>>> > > > collaboration on projects the dimension of bitbake. So an
>>>> additional
>>>> > > > question: Would you be up for a PEP8 compatibility overhaul of
>>>> the source
>>>> > > > code? I’d volunteer if I don’t have to send a patch via email ;)
>>>> > >
>>>> > > We're interested in patches to improve the coding style as long as
>>>> they
>>>> > > don't
>>>> > > break things like readability or performance. Which rules in PEP8 in
>>>> > > particular
>>>> > > are you referring to?
>>>> >
>>>> > To name a few violations:
>>>> >
>>>> >  * hanging indents
>>>> >  * inconsistent indentation
>>>> >  * 2 separating lines missing in between classes/top level methods
>>>>
>>>> FWIW I find it really hard to get worked up about whitespace issues. I
>>>> appreciate these aren't always great to look at and I appreciate the
>>>> familiarity
>>>> argument and standardising the look but these aren't issues I would
>>>> prioritise.
>>>>
>>>> In particular changing them globally in mass patches tends to break
>>>> patch
>>>> backporting and taking fixes back to older code which we do benefit
>>>> from being
>>>> able to do so there is a cost to doing it. We are trying to make new
>>>> code more
>>>> consistent and with older code, we do try and fix the worst issues.
>>>>
>>>> >  * unused imported modules
>>>> >  * missing white-spaces after comma
>>>> >  * unreachable codes
>>>>
>>>> These ones we do take patches for and try to fix when/where we see it.
>>>> For
>>>> example,
>>>> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511
>>>> d830df9cbd4113a4352b2c contained a couple of white-space after comma
>>>> fixing
>>>> while I was in that area.
>>>>
>>>> > These are just a couple of examples that struck my PEP8 used eye from
>>>> looking
>>>> > at two, three modules.
>>>> >
>>>> > It’s just like with any code smell: they don’t necessarily break
>>>> things. But
>>>> > they make maintenance, code-reviews and quality control harder than
>>>> necessary.
>>>> > Code that does not at least follow PEP8 (and I actually strongly
>>>> recommend to
>>>> > go the extra mile with `black`) just looks odd and is unnecessary
>>>> hard to
>>>> > read. I’m surprised I have to argue for standards.
>>>>
>>>> I wasn't saying it was a bad thing, I asked which issues you noticed in
>>>> particular since I'm trying to ensure we do at least try and react to
>>>> feedback.
>>>> Some whitespace issues we aren't likely to tackle, some like the others
>>>> you
>>>> mentioned above we should fix.
>>>> >
>>>> > Besides that, some minor cleanup just to warm
>>>> > up: https://github.com/openembedded/bitbake/pull/33,
>>>> https://github.com/openem
>>>> > bedded/bitbake/pull/32/files
>>>>
>>>> Those two do look like reasonable changes FWIW.
>>>>
>>>> > (it was after these two that I found the comment in one of those old
>>>> commits
>>>> > which made me open https://github.com/openembedded/bitbake/pull/34).
>>>> At least
>>>> > there should be an automated reply to each MR that is posted if
>>>> someone makes
>>>> > the effort to fix code and open a MRs that this is actually in vein -
>>>> That’s
>>>> > the mood that took me here :)
>>>>
>>>> There is a certain irony in that given you patched the section called
>>>> "Contributing" in the README in 32!
>>>>
>>>> > >
>>>> > > Help appreciated but email patches is how we accept them at the
>>>> moment.
>>>> >
>>>> > Not from me then :(
>>>>
>>>> Sorry to hear that, your changes in 32/33 do look reasonable.
>>>>
>>>> Cheers,
>>>>
>>>> Richard
>>>>
>>>>
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>> Links: You receive all messages sent to this group.
>>>> View/Reply Online (#12951):
>>>> https://lists.openembedded.org/g/bitbake-devel/message/12951
>>>> Mute This Topic: https://lists.openembedded.org/mt/86911583/1686489
>>>> Group Owner: bitbake-devel+owner@lists.openembedded.org
>>>> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
>>>> alex.kanavin@gmail.com]
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>
>>>>
>
>
> Am Di., 9. Nov. 2021 um 09:19 Uhr schrieb Alexander Kanavin <
> alex.kanavin@gmail.com>:
>
>> I think you misunderstood me too. It's about the github UI which does its
>> best to hide the commit messages, and doesn't allow to comment on their
>> content. Yes you can contort yourself and still do it, but the tools do not
>> help.
>>
>> Alex
>>
>> On Tue, 9 Nov 2021 at 09:05, Marius Kriegerowski <
>> marius.kriegerowski@gmail.com> wrote:
>>
>>> Good morning,
>>>
>>> I don’t think this is correct. You can change the git history and commit
>>> messages of every MR to whatever you like with `git commit —amend`, `git
>>> rebase -I` and `git push -f`. So, you can very well ask contributors to
>>> cleanup their history before merging.
>>>
>>> Best
>>> Marius
>>>
>>> Am Di., 9. Nov. 2021 um 07:47 Uhr schrieb Alexander Kanavin <
>>> alex.kanavin@gmail.com>:
>>>
>>>> There's another problem with github/gitlab which I find significant.
>>>> These tools do not allow reviewing commit messages, and don't even make it
>>>> easy to see them. So the developers using those tools, both submitters and
>>>> maintainers, are effectively being conditioned by the tools to see good
>>>> commit messages as superfluous and unnecessary. And so you more often than
>>>> not end up with poorly written commit history in github projects, or, if
>>>> you're a reviewer, have to repeatedly explain to the submitter how to
>>>> explain and structure their changes properly.
>>>>
>>>> I'd rather not have a change, than have a change that no one, even the
>>>> original author, can understand or remember the reasons for one year down
>>>> the line.
>>>>
>>>> Alex
>>>>
>>>>
>>>> On Tue, 9 Nov 2021 at 00:30, Richard Purdie <
>>>> richard.purdie@linuxfoundation.org> wrote:
>>>>
>>>>> On Mon, 2021-11-08 at 23:44 +0100, Marius Kriegerowski wrote:
>>>>> >
>>>>> >
>>>>> > > On 8. Nov 2021, at 22:47, Richard Purdie
>>>>> > > <richard.purdie@linuxfoundation.org> wrote:
>>>>> > >
>>>>> > > On Mon, 2021-11-08 at 18:37 +0100, Marius Kriegerowski wrote:
>>>>> > > > Is there a plan to switch to some more modern ways to
>>>>> collaborate on
>>>>> > > > bitbake,
>>>>> > > > you know, like, gitlab/github/gitea with issues, PRs and so
>>>>> forth? I mean…
>>>>> > > > it’s
>>>>> > > > late 2021 and sending MRs/Issues via email feels reaaaally
>>>>> outdated, slow,
>>>>> > > > intransparent, ineffective (how to you do reviews, code
>>>>> comparisons during
>>>>> > > > reviews, insert suggestions, etc.) and indirect (where is the CI
>>>>> pipeline
>>>>> > > > that I
>>>>> > > > trigger when pushing something somewhere?).
>>>>> > >
>>>>> > > There are pros and cons to switching to something else. Keep in
>>>>> mind that
>>>>> > > whilst the github workflow works well for a single maintainer of a
>>>>> project,
>>>>> > > it does not work well for diverse review.
>>>>> >
>>>>> > I strongly disagree with that. Reviews can equally well be done in a
>>>>> MR
>>>>> > thread. That’s what they are there for. You can discuss things there
>>>>> with as
>>>>> > many people as you want and can e.g. define a minimum number of
>>>>> approvals
>>>>> > before a MR can be merged.
>>>>>
>>>>> You misunderstand my point. We have a several thousand people on the
>>>>> mailing
>>>>> lists and they filter things to what interests them. *I* have no
>>>>> people I want
>>>>> to discuss a given change with, it is a question of whether the people
>>>>> who have
>>>>> an interest in a given change would see it.
>>>>>
>>>>> A minimum number of approvals is utterly meaningless for how we work,
>>>>> I need the
>>>>> interested people to see it and that varies massively depending on
>>>>> what is being
>>>>> changed. It is this aspect of mailing list change distribution and
>>>>> discussion we
>>>>> benefit from.
>>>>>
>>>>> >  It also naturally groups discussions. You can address experts for
>>>>> specific
>>>>> > domains which means less noise for the experts. I really don’t see
>>>>> why mailing
>>>>> > list reviews should work any better here.
>>>>>
>>>>> We all have established mechanisms to "see" the things which interest
>>>>> us as
>>>>> maintainers?
>>>>>
>>>>> Please don't misunderstand me, I do see the attraction of github from
>>>>> your
>>>>> perspective. It would just destroy the ecosystem we already have. You
>>>>> are
>>>>> basically saying that our current ecosystem is worthless but I would
>>>>> point out
>>>>> it has resulted in a project which you did say you appreciate so it
>>>>> can't be all
>>>>> bad.
>>>>>
>>>>> >  If google can coordinate 2.1k repos with 1.1k contributors over
>>>>> GitHub, why
>>>>> > should that not work for openembedded??
>>>>>
>>>>> It isn't about raw numbers, it is about numbers of reviewers for a
>>>>> given change.
>>>>> We have hundreds of people doing that using the mailing lists.
>>>>>
>>>>> > > The Bitbake case is a little bit different but in general we'd
>>>>> follow what
>>>>> > > OE/Yocto Project are doing.
>>>>> > >
>>>>> > > We have discussed changing and it is listed as one topic on:
>>>>> > > https://wiki.yoctoproject.org/wiki/Future_Directions
>>>>> > > (Code Submission process)
>>>>> > >
>>>>> > > I'd note that it is a huge change to our current developers and
>>>>> most of us
>>>>> > > are very used to the mailing lists so if we did change, it would
>>>>> allow
>>>>> > > potential new collaborators at the risk of losing people with key
>>>>> > > knowledge/experience.
>>>>> > … which reads to me like: “It has always been that way so we will
>>>>> not change
>>>>> > that" doesn’t it? Always a very poor argument.
>>>>>
>>>>> No, what I said is "upsetting our existing developers is something
>>>>> which we may
>>>>> want to potentially avoid".
>>>>>
>>>>> There are some options out there which have offered mailing list
>>>>> integration for
>>>>> example and we have tried to explore those. Something which drops the
>>>>> lists
>>>>> entirely could in the current view of the TSC result in loss of too
>>>>> many of our
>>>>> existing maintainers.
>>>>>
>>>>> >
>>>>> > > We do have CI which is shared with Yocto Project and
>>>>> > > OpenEmbedded. You can see it here:
>>>>> > >
>>>>> > > https://autobuilder.yoctoproject.org/typhoon/#/console
>>>>> > >
>>>>> > > and it has it's own manual which was recently added to our
>>>>> documentations
>>>>> > > set:
>>>>> > >
>>>>> > > http://docs.yoctoproject.org/test-manual/index.html
>>>>> > … stressing my point: Why does the location of the CI need
>>>>> documentation? All
>>>>> > of the aforementioned platforms smoothly integrate with multiple CI
>>>>> solutions,
>>>>> > drastically reducing the time required for reviews if the pipeline
>>>>> provides
>>>>> > automated feedback to the contributor. It would probably cost me
>>>>> about one
>>>>> > afternoon to setup a decent CI pipeline at least for all python
>>>>> codes pushing
>>>>> > the need to review from manual labor to automated tools. If open
>>>>> embedded
>>>>> > migrates to a proper git platform I happily volunteer on that end!!
>>>>>
>>>>> We're not talking about simple python code. We're talking about
>>>>> testing a build
>>>>> environment which cross compiles multiple operating systems for
>>>>> multiple
>>>>> architectures, C libraries, GUI stacks, init systems and so on.
>>>>> Testing the
>>>>> python code is one small part of that. I've spent several years trying
>>>>> to
>>>>> improve and get our CI to where it is today.
>>>>>
>>>>> Yes, the python code is tested as part of this as we don't really want
>>>>> to have
>>>>> multiple test platforms and the real world usage is some of the best
>>>>> testing
>>>>> changes can have.
>>>>>
>>>>> > > There is work still needed on that but it is a start. One
>>>>> challenge of
>>>>> > > bitbake/OE is that the builds take an age and the test matrix is
>>>>> huge so the
>>>>> > > CI
>>>>> > > process isn't quite as simple as some other simpler projects.
>>>>> > At least for python I bet that most bugs can be caught simply by
>>>>> firing up
>>>>> > `black`, `flake8`, `pylint` (and a hand full of other tools) just to
>>>>> check the
>>>>> > modified files.
>>>>>
>>>>> Well, I would bet that most of our bugs are not caused by issues like
>>>>> that. I do
>>>>> attend the weekly bug triage meetings and have fixed my share of them.
>>>>> You could
>>>>> have a look at the issues filed in bugzilla over the past few weeks if
>>>>> you'd
>>>>> like to see the kinds of issues that get filed and are open.
>>>>>
>>>>> >  Regarding meta-layers and recipes out there we are currently
>>>>> pushing forward
>>>>> > with a linter for bitbake recipes (thanks @priv-kweihmann for this
>>>>> awesome and
>>>>> > overdue tool!!!).
>>>>>
>>>>> Whilst improving recipes in that way does help, it isn't the main
>>>>> source of our
>>>>> major issues.
>>>>>
>>>>> > >
>>>>> > > As a shortcut through things, the "bitbake-selftest" command
>>>>> within bitbake
>>>>> > > may
>>>>> > > be one thing you're looking for which does simple tests on
>>>>> bitbake. It isn't
>>>>> > > complete coverage and we welcome more being added, it is a start
>>>>> and some
>>>>> > > areas
>>>>> > > of the code are covered better than others.
>>>>> > I’d love to overhaul the tests e.g. porting them to `pytest` which
>>>>> offers a
>>>>> > much simpler yet much more powerful API/plugin-ecosystem. But won’t
>>>>> do via
>>>>> > email :)
>>>>>
>>>>> The tests are written in python's own unittest.
>>>>>
>>>>> > > Issues are filed in the project bugzilla and are actively worked
>>>>> on:
>>>>> > >
>>>>> > > https://bugzilla.yoctoproject.org/
>>>>> > Didn’t know that. But why keeping issues separate from MRs?
>>>>> Automatic issue
>>>>> > closing through referencing in MRs is a pretty nice feature…
>>>>>
>>>>> If I had to list the pain points we have, this is not high on the list.
>>>>>
>>>>> > > > * side-note on bitbake style/linting: It’s violating PEP8 in a
>>>>> couple of
>>>>> > > > ways and TBH PEP8 is the bare minimum I would accept these days
>>>>> for
>>>>> > > > collaboration on projects the dimension of bitbake. So an
>>>>> additional
>>>>> > > > question: Would you be up for a PEP8 compatibility overhaul of
>>>>> the source
>>>>> > > > code? I’d volunteer if I don’t have to send a patch via email ;)
>>>>> > >
>>>>> > > We're interested in patches to improve the coding style as long as
>>>>> they
>>>>> > > don't
>>>>> > > break things like readability or performance. Which rules in PEP8
>>>>> in
>>>>> > > particular
>>>>> > > are you referring to?
>>>>> >
>>>>> > To name a few violations:
>>>>> >
>>>>> >  * hanging indents
>>>>> >  * inconsistent indentation
>>>>> >  * 2 separating lines missing in between classes/top level methods
>>>>>
>>>>> FWIW I find it really hard to get worked up about whitespace issues. I
>>>>> appreciate these aren't always great to look at and I appreciate the
>>>>> familiarity
>>>>> argument and standardising the look but these aren't issues I would
>>>>> prioritise.
>>>>>
>>>>> In particular changing them globally in mass patches tends to break
>>>>> patch
>>>>> backporting and taking fixes back to older code which we do benefit
>>>>> from being
>>>>> able to do so there is a cost to doing it. We are trying to make new
>>>>> code more
>>>>> consistent and with older code, we do try and fix the worst issues.
>>>>>
>>>>> >  * unused imported modules
>>>>> >  * missing white-spaces after comma
>>>>> >  * unreachable codes
>>>>>
>>>>> These ones we do take patches for and try to fix when/where we see it.
>>>>> For
>>>>> example,
>>>>> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511
>>>>> d830df9cbd4113a4352b2c
>>>>> <http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511d830df9cbd4113a4352b2c> contained
>>>>> a couple of white-space after comma fixing
>>>>> while I was in that area.
>>>>>
>>>>> > These are just a couple of examples that struck my PEP8 used eye
>>>>> from looking
>>>>> > at two, three modules.
>>>>> >
>>>>> > It’s just like with any code smell: they don’t necessarily break
>>>>> things. But
>>>>> > they make maintenance, code-reviews and quality control harder than
>>>>> necessary.
>>>>> > Code that does not at least follow PEP8 (and I actually strongly
>>>>> recommend to
>>>>> > go the extra mile with `black`) just looks odd and is unnecessary
>>>>> hard to
>>>>> > read. I’m surprised I have to argue for standards.
>>>>>
>>>>> I wasn't saying it was a bad thing, I asked which issues you noticed in
>>>>> particular since I'm trying to ensure we do at least try and react to
>>>>> feedback.
>>>>> Some whitespace issues we aren't likely to tackle, some like the
>>>>> others you
>>>>> mentioned above we should fix.
>>>>> >
>>>>> > Besides that, some minor cleanup just to warm
>>>>> > up: https://github.com/openembedded/bitbake/pull/33,
>>>>> https://github.com/openem
>>>>> > bedded/bitbake/pull/32/files
>>>>>
>>>>> Those two do look like reasonable changes FWIW.
>>>>>
>>>>> > (it was after these two that I found the comment in one of those old
>>>>> commits
>>>>> > which made me open https://github.com/openembedded/bitbake/pull/34).
>>>>> At least
>>>>> > there should be an automated reply to each MR that is posted if
>>>>> someone makes
>>>>> > the effort to fix code and open a MRs that this is actually in vein
>>>>> - That’s
>>>>> > the mood that took me here :)
>>>>>
>>>>> There is a certain irony in that given you patched the section called
>>>>> "Contributing" in the README in 32!
>>>>>
>>>>> > >
>>>>> > > Help appreciated but email patches is how we accept them at the
>>>>> moment.
>>>>> >
>>>>> > Not from me then :(
>>>>>
>>>>> Sorry to hear that, your changes in 32/33 do look reasonable.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Richard
>>>>>
>>>>>
>>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>> Links: You receive all messages sent to this group.
>>>>> View/Reply Online (#12951):
>>>>> https://lists.openembedded.org/g/bitbake-devel/message/12951
>>>>> Mute This Topic: https://lists.openembedded.org/mt/86911583/1686489
>>>>> Group Owner: bitbake-devel+owner@lists.openembedded.org
>>>>> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
>>>>> alex.kanavin@gmail.com]
>>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>>
>>>>>

[-- Attachment #2: Type: text/html, Size: 39727 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Mikko.Rapeli]

Hi,

I'm a heavy user of yocto and bitbake, CI systems and automation, even Gerrit
and GitHub, yet I also perfer they way email allows for a completely distributed
review and possibility to follow and join the discussions when ever a subject
or a change triggers me to respond or hits a nerve. Gerrit, github, gitlab etc
they just don't scale this well. I understand that for drive-by contributions working
with email can be tricky if you are not used it, but same goes for github, gerrit,
gitlab etc.

This community is also very welcoming, well behaving and helping first time
contributors. I do see the problem that some first time contributions may be
see the hurdle of "git send-email" as too hard, especially if corporate email systems
mangle emails. Maybe some tooling could be setup to allow for pull request type
contributions to popup on mailing list if sending emails just doesn't work, but I
don't see the benefits of moving fully to gerrit/github/gitlabl style development.

ps. maybe I'm also part of the old generation which still actually reads all emails.
I don't see any of the chat/collaboration tools really replacing email in global
R&D development world even if chat tools/Slack/MS Teams etc are being proposed
as replacements (they simply don't scale so well in the end).

Cheers,

-Mikko

Re: [bitbake-devel] bitbake style/workflow

[Marius Kriegerowski]

[-- Attachment #1: Type: text/plain, Size: 33091 bytes --]

On 9. Nov 2021, at 09:44, Alexander Kanavin <alex.kanavin@gmail.com> wrote:

On Tue, 9 Nov 2021 at 09:32, Marius Kriegerowski <
marius.kriegerowski@gmail.com> wrote:

> I see.
>
> On 9. Nov 2021, at 09:19, Alexander Kanavin <alex.kanavin@gmail.com>
> wrote:
>
> I think you misunderstood me too. It's about the github UI which does its
> best to hide the commit messages, and doesn't allow to comment on their
> content. Yes you can contort yourself and still do it, but the tools do not
> help.
>
>
> I never felt that the commit messages are really that hidden. I mean… you
> find them under `commits`... I find that pretty transparent. You can
> comment on the commit message content in the thread of the MR, just as you
> would in an email thread, can't you?
>

No. You need to separately go to the commits tab, open each commit
individually to see both the message and the associated code, then if
there's something you do not like, you need to copy paste the offending bit
and open a MR thread with it. That's not helpful, and it's clear that kind
of thing is not important to the tool providers or users. But it is very
important to me as one of the oe-core maintainers.






> Also, the older and closed threads can be very helpful when searching for
> answers why something has been changed in which way. I know that I saw the
> email threads on patches as well but found them super hard to read (no
> syntax highlighting, all black and white, …). Well, probably I could get
> used to reading them over time. But then — why should I if there are things
> like collapsable threads, syntax highlight, color,…?
>

You can find a better email client perhaps?


Not sure I was clear about what I meant: I was speaking about the closed
threads (https://lists.openembedded.org/g/openembedded-core). Is it somehow
sortable by open MRs/closed MRs/pure text/issues?

So, which email client would I have to install to get a better user
experience? What about the other features that make me use the email client
because of which I which Iike to use that client? Do you expect me to dump
them those features and my workflow routines in favour of just
openembedded?! Should I have two email clients then? And then hard-copy the
old mail threads to my HD from how many years?

BTW: I just went for the first time to the online thread:
https://lists.openembedded.org/g/bitbake-devel/topic/86911583
I would probably stop reading that monstrous thread after few seconds.

So, to move forward: Maybe you can improve the online threads so that it’s
easier to navigate and read those pure retro-black-and-white threads.

Marius



PS I’d like to emphasise again:
I know I sound harsh. I’m not blaming here and of course I’m also not
belittling the workflow neither the proficiency of the contributors. I’m
totally aware that most regular contributors here are extremely highly
skilled and work on an incredibly impactful set of software. I’m just
providing a very detailed feedback of what I as a really really really
really hard wanting-to-become future contributor see what really keeps me
from joining.



Alex


>
> Best
> Marius
>
>
> Alex
>
> On Tue, 9 Nov 2021 at 09:05, Marius Kriegerowski <
> marius.kriegerowski@gmail.com> wrote:
>
>> Good morning,
>>
>> I don’t think this is correct. You can change the git history and commit
>> messages of every MR to whatever you like with `git commit —amend`, `git
>> rebase -I` and `git push -f`. So, you can very well ask contributors to
>> cleanup their history before merging.
>>
>> Best
>> Marius
>>
>> Am Di., 9. Nov. 2021 um 07:47 Uhr schrieb Alexander Kanavin <
>> alex.kanavin@gmail.com>:
>>
>>> There's another problem with github/gitlab which I find significant.
>>> These tools do not allow reviewing commit messages, and don't even make it
>>> easy to see them. So the developers using those tools, both submitters and
>>> maintainers, are effectively being conditioned by the tools to see good
>>> commit messages as superfluous and unnecessary. And so you more often than
>>> not end up with poorly written commit history in github projects, or, if
>>> you're a reviewer, have to repeatedly explain to the submitter how to
>>> explain and structure their changes properly.
>>>
>>> I'd rather not have a change, than have a change that no one, even the
>>> original author, can understand or remember the reasons for one year down
>>> the line.
>>>
>>> Alex
>>>
>>>
>>> On Tue, 9 Nov 2021 at 00:30, Richard Purdie <
>>> richard.purdie@linuxfoundation.org> wrote:
>>>
>>>> On Mon, 2021-11-08 at 23:44 +0100, Marius Kriegerowski wrote:
>>>> >
>>>> >
>>>> > > On 8. Nov 2021, at 22:47, Richard Purdie
>>>> > > <richard.purdie@linuxfoundation.org> wrote:
>>>> > >
>>>> > > On Mon, 2021-11-08 at 18:37 +0100, Marius Kriegerowski wrote:
>>>> > > > Is there a plan to switch to some more modern ways to collaborate
>>>> on
>>>> > > > bitbake,
>>>> > > > you know, like, gitlab/github/gitea with issues, PRs and so
>>>> forth? I mean…
>>>> > > > it’s
>>>> > > > late 2021 and sending MRs/Issues via email feels reaaaally
>>>> outdated, slow,
>>>> > > > intransparent, ineffective (how to you do reviews, code
>>>> comparisons during
>>>> > > > reviews, insert suggestions, etc.) and indirect (where is the CI
>>>> pipeline
>>>> > > > that I
>>>> > > > trigger when pushing something somewhere?).
>>>> > >
>>>> > > There are pros and cons to switching to something else. Keep in
>>>> mind that
>>>> > > whilst the github workflow works well for a single maintainer of a
>>>> project,
>>>> > > it does not work well for diverse review.
>>>> >
>>>> > I strongly disagree with that. Reviews can equally well be done in a
>>>> MR
>>>> > thread. That’s what they are there for. You can discuss things there
>>>> with as
>>>> > many people as you want and can e.g. define a minimum number of
>>>> approvals
>>>> > before a MR can be merged.
>>>>
>>>> You misunderstand my point. We have a several thousand people on the
>>>> mailing
>>>> lists and they filter things to what interests them. *I* have no people
>>>> I want
>>>> to discuss a given change with, it is a question of whether the people
>>>> who have
>>>> an interest in a given change would see it.
>>>>
>>>> A minimum number of approvals is utterly meaningless for how we work, I
>>>> need the
>>>> interested people to see it and that varies massively depending on what
>>>> is being
>>>> changed. It is this aspect of mailing list change distribution and
>>>> discussion we
>>>> benefit from.
>>>>
>>>> >  It also naturally groups discussions. You can address experts for
>>>> specific
>>>> > domains which means less noise for the experts. I really don’t see
>>>> why mailing
>>>> > list reviews should work any better here.
>>>>
>>>> We all have established mechanisms to "see" the things which interest
>>>> us as
>>>> maintainers?
>>>>
>>>> Please don't misunderstand me, I do see the attraction of github from
>>>> your
>>>> perspective. It would just destroy the ecosystem we already have. You
>>>> are
>>>> basically saying that our current ecosystem is worthless but I would
>>>> point out
>>>> it has resulted in a project which you did say you appreciate so it
>>>> can't be all
>>>> bad.
>>>>
>>>> >  If google can coordinate 2.1k repos with 1.1k contributors over
>>>> GitHub, why
>>>> > should that not work for openembedded??
>>>>
>>>> It isn't about raw numbers, it is about numbers of reviewers for a
>>>> given change.
>>>> We have hundreds of people doing that using the mailing lists.
>>>>
>>>> > > The Bitbake case is a little bit different but in general we'd
>>>> follow what
>>>> > > OE/Yocto Project are doing.
>>>> > >
>>>> > > We have discussed changing and it is listed as one topic on:
>>>> > > https://wiki.yoctoproject.org/wiki/Future_Directions
>>>> > > (Code Submission process)
>>>> > >
>>>> > > I'd note that it is a huge change to our current developers and
>>>> most of us
>>>> > > are very used to the mailing lists so if we did change, it would
>>>> allow
>>>> > > potential new collaborators at the risk of losing people with key
>>>> > > knowledge/experience.
>>>> > … which reads to me like: “It has always been that way so we will not
>>>> change
>>>> > that" doesn’t it? Always a very poor argument.
>>>>
>>>> No, what I said is "upsetting our existing developers is something
>>>> which we may
>>>> want to potentially avoid".
>>>>
>>>> There are some options out there which have offered mailing list
>>>> integration for
>>>> example and we have tried to explore those. Something which drops the
>>>> lists
>>>> entirely could in the current view of the TSC result in loss of too
>>>> many of our
>>>> existing maintainers.
>>>>
>>>> >
>>>> > > We do have CI which is shared with Yocto Project and
>>>> > > OpenEmbedded. You can see it here:
>>>> > >
>>>> > > https://autobuilder.yoctoproject.org/typhoon/#/console
>>>> > >
>>>> > > and it has it's own manual which was recently added to our
>>>> documentations
>>>> > > set:
>>>> > >
>>>> > > http://docs.yoctoproject.org/test-manual/index.html
>>>> > … stressing my point: Why does the location of the CI need
>>>> documentation? All
>>>> > of the aforementioned platforms smoothly integrate with multiple CI
>>>> solutions,
>>>> > drastically reducing the time required for reviews if the pipeline
>>>> provides
>>>> > automated feedback to the contributor. It would probably cost me
>>>> about one
>>>> > afternoon to setup a decent CI pipeline at least for all python codes
>>>> pushing
>>>> > the need to review from manual labor to automated tools. If open
>>>> embedded
>>>> > migrates to a proper git platform I happily volunteer on that end!!
>>>>
>>>> We're not talking about simple python code. We're talking about testing
>>>> a build
>>>> environment which cross compiles multiple operating systems for multiple
>>>> architectures, C libraries, GUI stacks, init systems and so on. Testing
>>>> the
>>>> python code is one small part of that. I've spent several years trying
>>>> to
>>>> improve and get our CI to where it is today.
>>>>
>>>> Yes, the python code is tested as part of this as we don't really want
>>>> to have
>>>> multiple test platforms and the real world usage is some of the best
>>>> testing
>>>> changes can have.
>>>>
>>>> > > There is work still needed on that but it is a start. One challenge
>>>> of
>>>> > > bitbake/OE is that the builds take an age and the test matrix is
>>>> huge so the
>>>> > > CI
>>>> > > process isn't quite as simple as some other simpler projects.
>>>> > At least for python I bet that most bugs can be caught simply by
>>>> firing up
>>>> > `black`, `flake8`, `pylint` (and a hand full of other tools) just to
>>>> check the
>>>> > modified files.
>>>>
>>>> Well, I would bet that most of our bugs are not caused by issues like
>>>> that. I do
>>>> attend the weekly bug triage meetings and have fixed my share of them.
>>>> You could
>>>> have a look at the issues filed in bugzilla over the past few weeks if
>>>> you'd
>>>> like to see the kinds of issues that get filed and are open.
>>>>
>>>> >  Regarding meta-layers and recipes out there we are currently pushing
>>>> forward
>>>> > with a linter for bitbake recipes (thanks @priv-kweihmann for this
>>>> awesome and
>>>> > overdue tool!!!).
>>>>
>>>> Whilst improving recipes in that way does help, it isn't the main
>>>> source of our
>>>> major issues.
>>>>
>>>> > >
>>>> > > As a shortcut through things, the "bitbake-selftest" command within
>>>> bitbake
>>>> > > may
>>>> > > be one thing you're looking for which does simple tests on bitbake.
>>>> It isn't
>>>> > > complete coverage and we welcome more being added, it is a start
>>>> and some
>>>> > > areas
>>>> > > of the code are covered better than others.
>>>> > I’d love to overhaul the tests e.g. porting them to `pytest` which
>>>> offers a
>>>> > much simpler yet much more powerful API/plugin-ecosystem. But won’t
>>>> do via
>>>> > email :)
>>>>
>>>> The tests are written in python's own unittest.
>>>>
>>>> > > Issues are filed in the project bugzilla and are actively worked on:
>>>> > >
>>>> > > https://bugzilla.yoctoproject.org/
>>>> > Didn’t know that. But why keeping issues separate from MRs? Automatic
>>>> issue
>>>> > closing through referencing in MRs is a pretty nice feature…
>>>>
>>>> If I had to list the pain points we have, this is not high on the list.
>>>>
>>>> > > > * side-note on bitbake style/linting: It’s violating PEP8 in a
>>>> couple of
>>>> > > > ways and TBH PEP8 is the bare minimum I would accept these days
>>>> for
>>>> > > > collaboration on projects the dimension of bitbake. So an
>>>> additional
>>>> > > > question: Would you be up for a PEP8 compatibility overhaul of
>>>> the source
>>>> > > > code? I’d volunteer if I don’t have to send a patch via email ;)
>>>> > >
>>>> > > We're interested in patches to improve the coding style as long as
>>>> they
>>>> > > don't
>>>> > > break things like readability or performance. Which rules in PEP8 in
>>>> > > particular
>>>> > > are you referring to?
>>>> >
>>>> > To name a few violations:
>>>> >
>>>> >  * hanging indents
>>>> >  * inconsistent indentation
>>>> >  * 2 separating lines missing in between classes/top level methods
>>>>
>>>> FWIW I find it really hard to get worked up about whitespace issues. I
>>>> appreciate these aren't always great to look at and I appreciate the
>>>> familiarity
>>>> argument and standardising the look but these aren't issues I would
>>>> prioritise.
>>>>
>>>> In particular changing them globally in mass patches tends to break
>>>> patch
>>>> backporting and taking fixes back to older code which we do benefit
>>>> from being
>>>> able to do so there is a cost to doing it. We are trying to make new
>>>> code more
>>>> consistent and with older code, we do try and fix the worst issues.
>>>>
>>>> >  * unused imported modules
>>>> >  * missing white-spaces after comma
>>>> >  * unreachable codes
>>>>
>>>> These ones we do take patches for and try to fix when/where we see it.
>>>> For
>>>> example,
>>>> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511
>>>> d830df9cbd4113a4352b2c contained a couple of white-space after comma
>>>> fixing
>>>> while I was in that area.
>>>>
>>>> > These are just a couple of examples that struck my PEP8 used eye from
>>>> looking
>>>> > at two, three modules.
>>>> >
>>>> > It’s just like with any code smell: they don’t necessarily break
>>>> things. But
>>>> > they make maintenance, code-reviews and quality control harder than
>>>> necessary.
>>>> > Code that does not at least follow PEP8 (and I actually strongly
>>>> recommend to
>>>> > go the extra mile with `black`) just looks odd and is unnecessary
>>>> hard to
>>>> > read. I’m surprised I have to argue for standards.
>>>>
>>>> I wasn't saying it was a bad thing, I asked which issues you noticed in
>>>> particular since I'm trying to ensure we do at least try and react to
>>>> feedback.
>>>> Some whitespace issues we aren't likely to tackle, some like the others
>>>> you
>>>> mentioned above we should fix.
>>>> >
>>>> > Besides that, some minor cleanup just to warm
>>>> > up: https://github.com/openembedded/bitbake/pull/33,
>>>> https://github.com/openem
>>>> > bedded/bitbake/pull/32/files
>>>>
>>>> Those two do look like reasonable changes FWIW.
>>>>
>>>> > (it was after these two that I found the comment in one of those old
>>>> commits
>>>> > which made me open https://github.com/openembedded/bitbake/pull/34).
>>>> At least
>>>> > there should be an automated reply to each MR that is posted if
>>>> someone makes
>>>> > the effort to fix code and open a MRs that this is actually in vein -
>>>> That’s
>>>> > the mood that took me here :)
>>>>
>>>> There is a certain irony in that given you patched the section called
>>>> "Contributing" in the README in 32!
>>>>
>>>> > >
>>>> > > Help appreciated but email patches is how we accept them at the
>>>> moment.
>>>> >
>>>> > Not from me then :(
>>>>
>>>> Sorry to hear that, your changes in 32/33 do look reasonable.
>>>>
>>>> Cheers,
>>>>
>>>> Richard
>>>>
>>>>
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>> Links: You receive all messages sent to this group.
>>>> View/Reply Online (#12951):
>>>> https://lists.openembedded.org/g/bitbake-devel/message/12951
>>>> Mute This Topic: https://lists.openembedded.org/mt/86911583/1686489
>>>> Group Owner: bitbake-devel+owner@lists.openembedded.org
>>>> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
>>>> alex.kanavin@gmail.com]
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>
>>>>
>
>
> Am Di., 9. Nov. 2021 um 09:19 Uhr schrieb Alexander Kanavin <
> alex.kanavin@gmail.com>:
>
>> I think you misunderstood me too. It's about the github UI which does its
>> best to hide the commit messages, and doesn't allow to comment on their
>> content. Yes you can contort yourself and still do it, but the tools do not
>> help.
>>
>> Alex
>>
>> On Tue, 9 Nov 2021 at 09:05, Marius Kriegerowski <
>> marius.kriegerowski@gmail.com> wrote:
>>
>>> Good morning,
>>>
>>> I don’t think this is correct. You can change the git history and commit
>>> messages of every MR to whatever you like with `git commit —amend`, `git
>>> rebase -I` and `git push -f`. So, you can very well ask contributors to
>>> cleanup their history before merging.
>>>
>>> Best
>>> Marius
>>>
>>> Am Di., 9. Nov. 2021 um 07:47 Uhr schrieb Alexander Kanavin <
>>> alex.kanavin@gmail.com>:
>>>
>>>> There's another problem with github/gitlab which I find significant.
>>>> These tools do not allow reviewing commit messages, and don't even make it
>>>> easy to see them. So the developers using those tools, both submitters and
>>>> maintainers, are effectively being conditioned by the tools to see good
>>>> commit messages as superfluous and unnecessary. And so you more often than
>>>> not end up with poorly written commit history in github projects, or, if
>>>> you're a reviewer, have to repeatedly explain to the submitter how to
>>>> explain and structure their changes properly.
>>>>
>>>> I'd rather not have a change, than have a change that no one, even the
>>>> original author, can understand or remember the reasons for one year down
>>>> the line.
>>>>
>>>> Alex
>>>>
>>>>
>>>> On Tue, 9 Nov 2021 at 00:30, Richard Purdie <
>>>> richard.purdie@linuxfoundation.org> wrote:
>>>>
>>>>> On Mon, 2021-11-08 at 23:44 +0100, Marius Kriegerowski wrote:
>>>>> >
>>>>> >
>>>>> > > On 8. Nov 2021, at 22:47, Richard Purdie
>>>>> > > <richard.purdie@linuxfoundation.org> wrote:
>>>>> > >
>>>>> > > On Mon, 2021-11-08 at 18:37 +0100, Marius Kriegerowski wrote:
>>>>> > > > Is there a plan to switch to some more modern ways to
>>>>> collaborate on
>>>>> > > > bitbake,
>>>>> > > > you know, like, gitlab/github/gitea with issues, PRs and so
>>>>> forth? I mean…
>>>>> > > > it’s
>>>>> > > > late 2021 and sending MRs/Issues via email feels reaaaally
>>>>> outdated, slow,
>>>>> > > > intransparent, ineffective (how to you do reviews, code
>>>>> comparisons during
>>>>> > > > reviews, insert suggestions, etc.) and indirect (where is the CI
>>>>> pipeline
>>>>> > > > that I
>>>>> > > > trigger when pushing something somewhere?).
>>>>> > >
>>>>> > > There are pros and cons to switching to something else. Keep in
>>>>> mind that
>>>>> > > whilst the github workflow works well for a single maintainer of a
>>>>> project,
>>>>> > > it does not work well for diverse review.
>>>>> >
>>>>> > I strongly disagree with that. Reviews can equally well be done in a
>>>>> MR
>>>>> > thread. That’s what they are there for. You can discuss things there
>>>>> with as
>>>>> > many people as you want and can e.g. define a minimum number of
>>>>> approvals
>>>>> > before a MR can be merged.
>>>>>
>>>>> You misunderstand my point. We have a several thousand people on the
>>>>> mailing
>>>>> lists and they filter things to what interests them. *I* have no
>>>>> people I want
>>>>> to discuss a given change with, it is a question of whether the people
>>>>> who have
>>>>> an interest in a given change would see it.
>>>>>
>>>>> A minimum number of approvals is utterly meaningless for how we work,
>>>>> I need the
>>>>> interested people to see it and that varies massively depending on
>>>>> what is being
>>>>> changed. It is this aspect of mailing list change distribution and
>>>>> discussion we
>>>>> benefit from.
>>>>>
>>>>> >  It also naturally groups discussions. You can address experts for
>>>>> specific
>>>>> > domains which means less noise for the experts. I really don’t see
>>>>> why mailing
>>>>> > list reviews should work any better here.
>>>>>
>>>>> We all have established mechanisms to "see" the things which interest
>>>>> us as
>>>>> maintainers?
>>>>>
>>>>> Please don't misunderstand me, I do see the attraction of github from
>>>>> your
>>>>> perspective. It would just destroy the ecosystem we already have. You
>>>>> are
>>>>> basically saying that our current ecosystem is worthless but I would
>>>>> point out
>>>>> it has resulted in a project which you did say you appreciate so it
>>>>> can't be all
>>>>> bad.
>>>>>
>>>>> >  If google can coordinate 2.1k repos with 1.1k contributors over
>>>>> GitHub, why
>>>>> > should that not work for openembedded??
>>>>>
>>>>> It isn't about raw numbers, it is about numbers of reviewers for a
>>>>> given change.
>>>>> We have hundreds of people doing that using the mailing lists.
>>>>>
>>>>> > > The Bitbake case is a little bit different but in general we'd
>>>>> follow what
>>>>> > > OE/Yocto Project are doing.
>>>>> > >
>>>>> > > We have discussed changing and it is listed as one topic on:
>>>>> > > https://wiki.yoctoproject.org/wiki/Future_Directions
>>>>> > > (Code Submission process)
>>>>> > >
>>>>> > > I'd note that it is a huge change to our current developers and
>>>>> most of us
>>>>> > > are very used to the mailing lists so if we did change, it would
>>>>> allow
>>>>> > > potential new collaborators at the risk of losing people with key
>>>>> > > knowledge/experience.
>>>>> > … which reads to me like: “It has always been that way so we will
>>>>> not change
>>>>> > that" doesn’t it? Always a very poor argument.
>>>>>
>>>>> No, what I said is "upsetting our existing developers is something
>>>>> which we may
>>>>> want to potentially avoid".
>>>>>
>>>>> There are some options out there which have offered mailing list
>>>>> integration for
>>>>> example and we have tried to explore those. Something which drops the
>>>>> lists
>>>>> entirely could in the current view of the TSC result in loss of too
>>>>> many of our
>>>>> existing maintainers.
>>>>>
>>>>> >
>>>>> > > We do have CI which is shared with Yocto Project and
>>>>> > > OpenEmbedded. You can see it here:
>>>>> > >
>>>>> > > https://autobuilder.yoctoproject.org/typhoon/#/console
>>>>> > >
>>>>> > > and it has it's own manual which was recently added to our
>>>>> documentations
>>>>> > > set:
>>>>> > >
>>>>> > > http://docs.yoctoproject.org/test-manual/index.html
>>>>> > … stressing my point: Why does the location of the CI need
>>>>> documentation? All
>>>>> > of the aforementioned platforms smoothly integrate with multiple CI
>>>>> solutions,
>>>>> > drastically reducing the time required for reviews if the pipeline
>>>>> provides
>>>>> > automated feedback to the contributor. It would probably cost me
>>>>> about one
>>>>> > afternoon to setup a decent CI pipeline at least for all python
>>>>> codes pushing
>>>>> > the need to review from manual labor to automated tools. If open
>>>>> embedded
>>>>> > migrates to a proper git platform I happily volunteer on that end!!
>>>>>
>>>>> We're not talking about simple python code. We're talking about
>>>>> testing a build
>>>>> environment which cross compiles multiple operating systems for
>>>>> multiple
>>>>> architectures, C libraries, GUI stacks, init systems and so on.
>>>>> Testing the
>>>>> python code is one small part of that. I've spent several years trying
>>>>> to
>>>>> improve and get our CI to where it is today.
>>>>>
>>>>> Yes, the python code is tested as part of this as we don't really want
>>>>> to have
>>>>> multiple test platforms and the real world usage is some of the best
>>>>> testing
>>>>> changes can have.
>>>>>
>>>>> > > There is work still needed on that but it is a start. One
>>>>> challenge of
>>>>> > > bitbake/OE is that the builds take an age and the test matrix is
>>>>> huge so the
>>>>> > > CI
>>>>> > > process isn't quite as simple as some other simpler projects.
>>>>> > At least for python I bet that most bugs can be caught simply by
>>>>> firing up
>>>>> > `black`, `flake8`, `pylint` (and a hand full of other tools) just to
>>>>> check the
>>>>> > modified files.
>>>>>
>>>>> Well, I would bet that most of our bugs are not caused by issues like
>>>>> that. I do
>>>>> attend the weekly bug triage meetings and have fixed my share of them.
>>>>> You could
>>>>> have a look at the issues filed in bugzilla over the past few weeks if
>>>>> you'd
>>>>> like to see the kinds of issues that get filed and are open.
>>>>>
>>>>> >  Regarding meta-layers and recipes out there we are currently
>>>>> pushing forward
>>>>> > with a linter for bitbake recipes (thanks @priv-kweihmann for this
>>>>> awesome and
>>>>> > overdue tool!!!).
>>>>>
>>>>> Whilst improving recipes in that way does help, it isn't the main
>>>>> source of our
>>>>> major issues.
>>>>>
>>>>> > >
>>>>> > > As a shortcut through things, the "bitbake-selftest" command
>>>>> within bitbake
>>>>> > > may
>>>>> > > be one thing you're looking for which does simple tests on
>>>>> bitbake. It isn't
>>>>> > > complete coverage and we welcome more being added, it is a start
>>>>> and some
>>>>> > > areas
>>>>> > > of the code are covered better than others.
>>>>> > I’d love to overhaul the tests e.g. porting them to `pytest` which
>>>>> offers a
>>>>> > much simpler yet much more powerful API/plugin-ecosystem. But won’t
>>>>> do via
>>>>> > email :)
>>>>>
>>>>> The tests are written in python's own unittest.
>>>>>
>>>>> > > Issues are filed in the project bugzilla and are actively worked
>>>>> on:
>>>>> > >
>>>>> > > https://bugzilla.yoctoproject.org/
>>>>> > Didn’t know that. But why keeping issues separate from MRs?
>>>>> Automatic issue
>>>>> > closing through referencing in MRs is a pretty nice feature…
>>>>>
>>>>> If I had to list the pain points we have, this is not high on the list.
>>>>>
>>>>> > > > * side-note on bitbake style/linting: It’s violating PEP8 in a
>>>>> couple of
>>>>> > > > ways and TBH PEP8 is the bare minimum I would accept these days
>>>>> for
>>>>> > > > collaboration on projects the dimension of bitbake. So an
>>>>> additional
>>>>> > > > question: Would you be up for a PEP8 compatibility overhaul of
>>>>> the source
>>>>> > > > code? I’d volunteer if I don’t have to send a patch via email ;)
>>>>> > >
>>>>> > > We're interested in patches to improve the coding style as long as
>>>>> they
>>>>> > > don't
>>>>> > > break things like readability or performance. Which rules in PEP8
>>>>> in
>>>>> > > particular
>>>>> > > are you referring to?
>>>>> >
>>>>> > To name a few violations:
>>>>> >
>>>>> >  * hanging indents
>>>>> >  * inconsistent indentation
>>>>> >  * 2 separating lines missing in between classes/top level methods
>>>>>
>>>>> FWIW I find it really hard to get worked up about whitespace issues. I
>>>>> appreciate these aren't always great to look at and I appreciate the
>>>>> familiarity
>>>>> argument and standardising the look but these aren't issues I would
>>>>> prioritise.
>>>>>
>>>>> In particular changing them globally in mass patches tends to break
>>>>> patch
>>>>> backporting and taking fixes back to older code which we do benefit
>>>>> from being
>>>>> able to do so there is a cost to doing it. We are trying to make new
>>>>> code more
>>>>> consistent and with older code, we do try and fix the worst issues.
>>>>>
>>>>> >  * unused imported modules
>>>>> >  * missing white-spaces after comma
>>>>> >  * unreachable codes
>>>>>
>>>>> These ones we do take patches for and try to fix when/where we see it.
>>>>> For
>>>>> example,
>>>>> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511
>>>>> d830df9cbd4113a4352b2c contained a couple of white-space after comma
>>>>> fixing
>>>>> while I was in that area.
>>>>>
>>>>> > These are just a couple of examples that struck my PEP8 used eye
>>>>> from looking
>>>>> > at two, three modules.
>>>>> >
>>>>> > It’s just like with any code smell: they don’t necessarily break
>>>>> things. But
>>>>> > they make maintenance, code-reviews and quality control harder than
>>>>> necessary.
>>>>> > Code that does not at least follow PEP8 (and I actually strongly
>>>>> recommend to
>>>>> > go the extra mile with `black`) just looks odd and is unnecessary
>>>>> hard to
>>>>> > read. I’m surprised I have to argue for standards.
>>>>>
>>>>> I wasn't saying it was a bad thing, I asked which issues you noticed in
>>>>> particular since I'm trying to ensure we do at least try and react to
>>>>> feedback.
>>>>> Some whitespace issues we aren't likely to tackle, some like the
>>>>> others you
>>>>> mentioned above we should fix.
>>>>> >
>>>>> > Besides that, some minor cleanup just to warm
>>>>> > up: https://github.com/openembedded/bitbake/pull/33,
>>>>> https://github.com/openem
>>>>> > bedded/bitbake/pull/32/files
>>>>>
>>>>> Those two do look like reasonable changes FWIW.
>>>>>
>>>>> > (it was after these two that I found the comment in one of those old
>>>>> commits
>>>>> > which made me open https://github.com/openembedded/bitbake/pull/34).
>>>>> At least
>>>>> > there should be an automated reply to each MR that is posted if
>>>>> someone makes
>>>>> > the effort to fix code and open a MRs that this is actually in vein
>>>>> - That’s
>>>>> > the mood that took me here :)
>>>>>
>>>>> There is a certain irony in that given you patched the section called
>>>>> "Contributing" in the README in 32!
>>>>>
>>>>> > >
>>>>> > > Help appreciated but email patches is how we accept them at the
>>>>> moment.
>>>>> >
>>>>> > Not from me then :(
>>>>>
>>>>> Sorry to hear that, your changes in 32/33 do look reasonable.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Richard
>>>>>
>>>>>
>>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>> Links: You receive all messages sent to this group.
>>>>> View/Reply Online (#12951):
>>>>> https://lists.openembedded.org/g/bitbake-devel/message/12951
>>>>> Mute This Topic: https://lists.openembedded.org/mt/86911583/1686489
>>>>> Group Owner: bitbake-devel+owner@lists.openembedded.org
>>>>> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
>>>>> alex.kanavin@gmail.com]
>>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>>
>>>>>


Am Di., 9. Nov. 2021 um 09:49 Uhr schrieb <Mikko.Rapeli@bmw.de>:

> Hi,
>
> I'm a heavy user of yocto and bitbake, CI systems and automation, even
> Gerrit
> and GitHub, yet I also perfer they way email allows for a completely
> distributed
> review and possibility to follow and join the discussions when ever a
> subject
> or a change triggers me to respond or hits a nerve. Gerrit, github, gitlab
> etc
> they just don't scale this well. I understand that for drive-by
> contributions working
> with email can be tricky if you are not used it, but same goes for github,
> gerrit,
> gitlab etc.
>
> This community is also very welcoming, well behaving and helping first time
> contributors. I do see the problem that some first time contributions may
> be
> see the hurdle of "git send-email" as too hard, especially if corporate
> email systems
> mangle emails. Maybe some tooling could be setup to allow for pull request
> type
> contributions to popup on mailing list if sending emails just doesn't
> work, but I
> don't see the benefits of moving fully to gerrit/github/gitlabl style
> development.
>
> ps. maybe I'm also part of the old generation which still actually reads
> all emails.
> I don't see any of the chat/collaboration tools really replacing email in
> global
> R&D development world even if chat tools/Slack/MS Teams etc are being
> proposed
> as replacements (they simply don't scale so well in the end).
>
> Cheers,
>
> -Mikko

[-- Attachment #2: Type: text/html, Size: 58660 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Marius Kriegerowski]

[-- Attachment #1: Type: text/plain, Size: 19976 bytes --]

So, I'd like to come to a conclusing with this time consuming thread:

I shared my view which I'm sure many early contributors share. There is a
lot of resentment in this developer community against changing the workflow
(which dates back more than a decade) - which I understand. I still think
that the inconvenience when commenting on a commit message (that was the
only drawback I found legit) does not outweigh the benefits of discussing
MRs on one of the platforms mentioned. We disagree on that.

I'm keen on adding contributions not only to `meta-openembedded` (which
uses github - thanks for that) but also to `bitbake`. So, I'll surrender
and send a patch via email. Maybe. One day.

Have a good day
Marius

Am Di., 9. Nov. 2021 um 10:09 Uhr schrieb Marius Kriegerowski via
lists.openembedded.org <marius.kriegerowski=gmail.com@lists.openembedded.org
>:

>
>
> On 9. Nov 2021, at 09:44, Alexander Kanavin <alex.kanavin@gmail.com>
> wrote:
>
> On Tue, 9 Nov 2021 at 09:32, Marius Kriegerowski <
> marius.kriegerowski@gmail.com> wrote:
>
>> I see.
>>
>> On 9. Nov 2021, at 09:19, Alexander Kanavin <alex.kanavin@gmail.com>
>> wrote:
>>
>> I think you misunderstood me too. It's about the github UI which does its
>> best to hide the commit messages, and doesn't allow to comment on their
>> content. Yes you can contort yourself and still do it, but the tools do not
>> help.
>>
>>
>> I never felt that the commit messages are really that hidden. I mean… you
>> find them under `commits`... I find that pretty transparent. You can
>> comment on the commit message content in the thread of the MR, just as you
>> would in an email thread, can't you?
>>
>
> No. You need to separately go to the commits tab, open each commit
> individually to see both the message and the associated code, then if
> there's something you do not like, you need to copy paste the offending bit
> and open a MR thread with it. That's not helpful, and it's clear that kind
> of thing is not important to the tool providers or users. But it is very
> important to me as one of the oe-core maintainers.
>
>
>
>
>
>
>> Also, the older and closed threads can be very helpful when searching for
>> answers why something has been changed in which way. I know that I saw the
>> email threads on patches as well but found them super hard to read (no
>> syntax highlighting, all black and white, …). Well, probably I could get
>> used to reading them over time. But then — why should I if there are things
>> like collapsable threads, syntax highlight, color,…?
>>
>
> You can find a better email client perhaps?
>
>
> Not sure I was clear about what I meant: I was speaking about the closed
> threads (https://lists.openembedded.org/g/openembedded-core). Is it
> somehow sortable by open MRs/closed MRs/pure text/issues?
>
> So, which email client would I have to install to get a better user
> experience? What about the other features that make me use the email client
> because of which I which Iike to use that client? Do you expect me to dump
> them those features and my workflow routines in favour of just
> openembedded?! Should I have two email clients then? And then hard-copy the
> old mail threads to my HD from how many years?
>
> BTW: I just went for the first time to the online thread:
> https://lists.openembedded.org/g/bitbake-devel/topic/86911583
> I would probably stop reading that monstrous thread after few seconds.
>
> So, to move forward: Maybe you can improve the online threads so that it’s
> easier to navigate and read those pure retro-black-and-white threads.
>
> Marius
>
>
>
> PS I’d like to emphasise again:
> I know I sound harsh. I’m not blaming here and of course I’m also not
> belittling the workflow neither the proficiency of the contributors. I’m
> totally aware that most regular contributors here are extremely highly
> skilled and work on an incredibly impactful set of software. I’m just
> providing a very detailed feedback of what I as a really really really
> really hard wanting-to-become future contributor see what really keeps me
> from joining.
>
>
>
> Alex
>
>
>>
>> Best
>> Marius
>>
>>
>> Alex
>>
>> On Tue, 9 Nov 2021 at 09:05, Marius Kriegerowski <
>> marius.kriegerowski@gmail.com> wrote:
>>
>>> Good morning,
>>>
>>> I don’t think this is correct. You can change the git history and commit
>>> messages of every MR to whatever you like with `git commit —amend`, `git
>>> rebase -I` and `git push -f`. So, you can very well ask contributors to
>>> cleanup their history before merging.
>>>
>>> Best
>>> Marius
>>>
>>> Am Di., 9. Nov. 2021 um 07:47 Uhr schrieb Alexander Kanavin <
>>> alex.kanavin@gmail.com>:
>>>
>>>> There's another problem with github/gitlab which I find significant.
>>>> These tools do not allow reviewing commit messages, and don't even make it
>>>> easy to see them. So the developers using those tools, both submitters and
>>>> maintainers, are effectively being conditioned by the tools to see good
>>>> commit messages as superfluous and unnecessary. And so you more often than
>>>> not end up with poorly written commit history in github projects, or, if
>>>> you're a reviewer, have to repeatedly explain to the submitter how to
>>>> explain and structure their changes properly.
>>>>
>>>> I'd rather not have a change, than have a change that no one, even the
>>>> original author, can understand or remember the reasons for one year down
>>>> the line.
>>>>
>>>> Alex
>>>>
>>>>
>>>> On Tue, 9 Nov 2021 at 00:30, Richard Purdie <
>>>> richard.purdie@linuxfoundation.org> wrote:
>>>>
>>>>> On Mon, 2021-11-08 at 23:44 +0100, Marius Kriegerowski wrote:
>>>>> >
>>>>> >
>>>>> > > On 8. Nov 2021, at 22:47, Richard Purdie
>>>>> > > <richard.purdie@linuxfoundation.org> wrote:
>>>>> > >
>>>>> > > On Mon, 2021-11-08 at 18:37 +0100, Marius Kriegerowski wrote:
>>>>> > > > Is there a plan to switch to some more modern ways to
>>>>> collaborate on
>>>>> > > > bitbake,
>>>>> > > > you know, like, gitlab/github/gitea with issues, PRs and so
>>>>> forth? I mean…
>>>>> > > > it’s
>>>>> > > > late 2021 and sending MRs/Issues via email feels reaaaally
>>>>> outdated, slow,
>>>>> > > > intransparent, ineffective (how to you do reviews, code
>>>>> comparisons during
>>>>> > > > reviews, insert suggestions, etc.) and indirect (where is the CI
>>>>> pipeline
>>>>> > > > that I
>>>>> > > > trigger when pushing something somewhere?).
>>>>> > >
>>>>> > > There are pros and cons to switching to something else. Keep in
>>>>> mind that
>>>>> > > whilst the github workflow works well for a single maintainer of a
>>>>> project,
>>>>> > > it does not work well for diverse review.
>>>>> >
>>>>> > I strongly disagree with that. Reviews can equally well be done in a
>>>>> MR
>>>>> > thread. That’s what they are there for. You can discuss things there
>>>>> with as
>>>>> > many people as you want and can e.g. define a minimum number of
>>>>> approvals
>>>>> > before a MR can be merged.
>>>>>
>>>>> You misunderstand my point. We have a several thousand people on the
>>>>> mailing
>>>>> lists and they filter things to what interests them. *I* have no
>>>>> people I want
>>>>> to discuss a given change with, it is a question of whether the people
>>>>> who have
>>>>> an interest in a given change would see it.
>>>>>
>>>>> A minimum number of approvals is utterly meaningless for how we work,
>>>>> I need the
>>>>> interested people to see it and that varies massively depending on
>>>>> what is being
>>>>> changed. It is this aspect of mailing list change distribution and
>>>>> discussion we
>>>>> benefit from.
>>>>>
>>>>> >  It also naturally groups discussions. You can address experts for
>>>>> specific
>>>>> > domains which means less noise for the experts. I really don’t see
>>>>> why mailing
>>>>> > list reviews should work any better here.
>>>>>
>>>>> We all have established mechanisms to "see" the things which interest
>>>>> us as
>>>>> maintainers?
>>>>>
>>>>> Please don't misunderstand me, I do see the attraction of github from
>>>>> your
>>>>> perspective. It would just destroy the ecosystem we already have. You
>>>>> are
>>>>> basically saying that our current ecosystem is worthless but I would
>>>>> point out
>>>>> it has resulted in a project which you did say you appreciate so it
>>>>> can't be all
>>>>> bad.
>>>>>
>>>>> >  If google can coordinate 2.1k repos with 1.1k contributors over
>>>>> GitHub, why
>>>>> > should that not work for openembedded??
>>>>>
>>>>> It isn't about raw numbers, it is about numbers of reviewers for a
>>>>> given change.
>>>>> We have hundreds of people doing that using the mailing lists.
>>>>>
>>>>> > > The Bitbake case is a little bit different but in general we'd
>>>>> follow what
>>>>> > > OE/Yocto Project are doing.
>>>>> > >
>>>>> > > We have discussed changing and it is listed as one topic on:
>>>>> > > https://wiki.yoctoproject.org/wiki/Future_Directions
>>>>> > > (Code Submission process)
>>>>> > >
>>>>> > > I'd note that it is a huge change to our current developers and
>>>>> most of us
>>>>> > > are very used to the mailing lists so if we did change, it would
>>>>> allow
>>>>> > > potential new collaborators at the risk of losing people with key
>>>>> > > knowledge/experience.
>>>>> > … which reads to me like: “It has always been that way so we will
>>>>> not change
>>>>> > that" doesn’t it? Always a very poor argument.
>>>>>
>>>>> No, what I said is "upsetting our existing developers is something
>>>>> which we may
>>>>> want to potentially avoid".
>>>>>
>>>>> There are some options out there which have offered mailing list
>>>>> integration for
>>>>> example and we have tried to explore those. Something which drops the
>>>>> lists
>>>>> entirely could in the current view of the TSC result in loss of too
>>>>> many of our
>>>>> existing maintainers.
>>>>>
>>>>> >
>>>>> > > We do have CI which is shared with Yocto Project and
>>>>> > > OpenEmbedded. You can see it here:
>>>>> > >
>>>>> > > https://autobuilder.yoctoproject.org/typhoon/#/console
>>>>> > >
>>>>> > > and it has it's own manual which was recently added to our
>>>>> documentations
>>>>> > > set:
>>>>> > >
>>>>> > > http://docs.yoctoproject.org/test-manual/index.html
>>>>> > … stressing my point: Why does the location of the CI need
>>>>> documentation? All
>>>>> > of the aforementioned platforms smoothly integrate with multiple CI
>>>>> solutions,
>>>>> > drastically reducing the time required for reviews if the pipeline
>>>>> provides
>>>>> > automated feedback to the contributor. It would probably cost me
>>>>> about one
>>>>> > afternoon to setup a decent CI pipeline at least for all python
>>>>> codes pushing
>>>>> > the need to review from manual labor to automated tools. If open
>>>>> embedded
>>>>> > migrates to a proper git platform I happily volunteer on that end!!
>>>>>
>>>>> We're not talking about simple python code. We're talking about
>>>>> testing a build
>>>>> environment which cross compiles multiple operating systems for
>>>>> multiple
>>>>> architectures, C libraries, GUI stacks, init systems and so on.
>>>>> Testing the
>>>>> python code is one small part of that. I've spent several years trying
>>>>> to
>>>>> improve and get our CI to where it is today.
>>>>>
>>>>> Yes, the python code is tested as part of this as we don't really want
>>>>> to have
>>>>> multiple test platforms and the real world usage is some of the best
>>>>> testing
>>>>> changes can have.
>>>>>
>>>>> > > There is work still needed on that but it is a start. One
>>>>> challenge of
>>>>> > > bitbake/OE is that the builds take an age and the test matrix is
>>>>> huge so the
>>>>> > > CI
>>>>> > > process isn't quite as simple as some other simpler projects.
>>>>> > At least for python I bet that most bugs can be caught simply by
>>>>> firing up
>>>>> > `black`, `flake8`, `pylint` (and a hand full of other tools) just to
>>>>> check the
>>>>> > modified files.
>>>>>
>>>>> Well, I would bet that most of our bugs are not caused by issues like
>>>>> that. I do
>>>>> attend the weekly bug triage meetings and have fixed my share of them.
>>>>> You could
>>>>> have a look at the issues filed in bugzilla over the past few weeks if
>>>>> you'd
>>>>> like to see the kinds of issues that get filed and are open.
>>>>>
>>>>> >  Regarding meta-layers and recipes out there we are currently
>>>>> pushing forward
>>>>> > with a linter for bitbake recipes (thanks @priv-kweihmann for this
>>>>> awesome and
>>>>> > overdue tool!!!).
>>>>>
>>>>> Whilst improving recipes in that way does help, it isn't the main
>>>>> source of our
>>>>> major issues.
>>>>>
>>>>> > >
>>>>> > > As a shortcut through things, the "bitbake-selftest" command
>>>>> within bitbake
>>>>> > > may
>>>>> > > be one thing you're looking for which does simple tests on
>>>>> bitbake. It isn't
>>>>> > > complete coverage and we welcome more being added, it is a start
>>>>> and some
>>>>> > > areas
>>>>> > > of the code are covered better than others.
>>>>> > I’d love to overhaul the tests e.g. porting them to `pytest` which
>>>>> offers a
>>>>> > much simpler yet much more powerful API/plugin-ecosystem. But won’t
>>>>> do via
>>>>> > email :)
>>>>>
>>>>> The tests are written in python's own unittest.
>>>>>
>>>>> > > Issues are filed in the project bugzilla and are actively worked
>>>>> on:
>>>>> > >
>>>>> > > https://bugzilla.yoctoproject.org/
>>>>> > Didn’t know that. But why keeping issues separate from MRs?
>>>>> Automatic issue
>>>>> > closing through referencing in MRs is a pretty nice feature…
>>>>>
>>>>> If I had to list the pain points we have, this is not high on the list.
>>>>>
>>>>> > > > * side-note on bitbake style/linting: It’s violating PEP8 in a
>>>>> couple of
>>>>> > > > ways and TBH PEP8 is the bare minimum I would accept these days
>>>>> for
>>>>> > > > collaboration on projects the dimension of bitbake. So an
>>>>> additional
>>>>> > > > question: Would you be up for a PEP8 compatibility overhaul of
>>>>> the source
>>>>> > > > code? I’d volunteer if I don’t have to send a patch via email ;)
>>>>> > >
>>>>> > > We're interested in patches to improve the coding style as long as
>>>>> they
>>>>> > > don't
>>>>> > > break things like readability or performance. Which rules in PEP8
>>>>> in
>>>>> > > particular
>>>>> > > are you referring to?
>>>>> >
>>>>> > To name a few violations:
>>>>> >
>>>>> >  * hanging indents
>>>>> >  * inconsistent indentation
>>>>> >  * 2 separating lines missing in between classes/top level methods
>>>>>
>>>>> FWIW I find it really hard to get worked up about whitespace issues. I
>>>>> appreciate these aren't always great to look at and I appreciate the
>>>>> familiarity
>>>>> argument and standardising the look but these aren't issues I would
>>>>> prioritise.
>>>>>
>>>>> In particular changing them globally in mass patches tends to break
>>>>> patch
>>>>> backporting and taking fixes back to older code which we do benefit
>>>>> from being
>>>>> able to do so there is a cost to doing it. We are trying to make new
>>>>> code more
>>>>> consistent and with older code, we do try and fix the worst issues.
>>>>>
>>>>> >  * unused imported modules
>>>>> >  * missing white-spaces after comma
>>>>> >  * unreachable codes
>>>>>
>>>>> These ones we do take patches for and try to fix when/where we see it.
>>>>> For
>>>>> example,
>>>>> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511
>>>>> d830df9cbd4113a4352b2c contained a couple of white-space after comma
>>>>> fixing
>>>>> while I was in that area.
>>>>>
>>>>> > These are just a couple of examples that struck my PEP8 used eye
>>>>> from looking
>>>>> > at two, three modules.
>>>>> >
>>>>> > It’s just like with any code smell: they don’t necessarily break
>>>>> things. But
>>>>> > they make maintenance, code-reviews and quality control harder than
>>>>> necessary.
>>>>> > Code that does not at least follow PEP8 (and I actually strongly
>>>>> recommend to
>>>>> > go the extra mile with `black`) just looks odd and is unnecessary
>>>>> hard to
>>>>> > read. I’m surprised I have to argue for standards.
>>>>>
>>>>> I wasn't saying it was a bad thing, I asked which issues you noticed in
>>>>> particular since I'm trying to ensure we do at least try and react to
>>>>> feedback.
>>>>> Some whitespace issues we aren't likely to tackle, some like the
>>>>> others you
>>>>> mentioned above we should fix.
>>>>> >
>>>>> > Besides that, some minor cleanup just to warm
>>>>> > up: https://github.com/openembedded/bitbake/pull/33,
>>>>> https://github.com/openem
>>>>> > bedded/bitbake/pull/32/files
>>>>>
>>>>> Those two do look like reasonable changes FWIW.
>>>>>
>>>>> > (it was after these two that I found the comment in one of those old
>>>>> commits
>>>>> > which made me open https://github.com/openembedded/bitbake/pull/34).
>>>>> At least
>>>>> > there should be an automated reply to each MR that is posted if
>>>>> someone makes
>>>>> > the effort to fix code and open a MRs that this is actually in vein
>>>>> - That’s
>>>>> > the mood that took me here :)
>>>>>
>>>>> There is a certain irony in that given you patched the section called
>>>>> "Contributing" in the README in 32!
>>>>>
>>>>> > >
>>>>> > > Help appreciated but email patches is how we accept them at the
>>>>> moment.
>>>>> >
>>>>> > Not from me then :(
>>>>>
>>>>> Sorry to hear that, your changes in 32/33 do look reasonable.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Richard
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>
>
> Am Di., 9. Nov. 2021 um 09:49 Uhr schrieb <Mikko.Rapeli@bmw.de>:
>
>> Hi,
>>
>> I'm a heavy user of yocto and bitbake, CI systems and automation, even
>> Gerrit
>> and GitHub, yet I also perfer they way email allows for a completely
>> distributed
>> review and possibility to follow and join the discussions when ever a
>> subject
>> or a change triggers me to respond or hits a nerve. Gerrit, github,
>> gitlab etc
>> they just don't scale this well. I understand that for drive-by
>> contributions working
>> with email can be tricky if you are not used it, but same goes for
>> github, gerrit,
>> gitlab etc.
>>
>> This community is also very welcoming, well behaving and helping first
>> time
>> contributors. I do see the problem that some first time contributions may
>> be
>> see the hurdle of "git send-email" as too hard, especially if corporate
>> email systems
>> mangle emails. Maybe some tooling could be setup to allow for pull
>> request type
>> contributions to popup on mailing list if sending emails just doesn't
>> work, but I
>> don't see the benefits of moving fully to gerrit/github/gitlabl style
>> development.
>>
>> ps. maybe I'm also part of the old generation which still actually reads
>> all emails.
>> I don't see any of the chat/collaboration tools really replacing email in
>> global
>> R&D development world even if chat tools/Slack/MS Teams etc are being
>> proposed
>> as replacements (they simply don't scale so well in the end).
>>
>> Cheers,
>>
>> -Mikko
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12959):
> https://lists.openembedded.org/g/bitbake-devel/message/12959
> Mute This Topic: https://lists.openembedded.org/mt/86911583/6547911
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> marius.kriegerowski@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 30910 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Marius Kriegerowski]

[-- Attachment #1.1: Type: text/plain, Size: 22042 bytes --]

Okay, one thing I have to add after killing some time with this: I quickly
scrutinized the git history of `bitbake` and `meta-openembedded`.

Attached are two figures that show the first time contributor count of both
projects over the last ~ten years. While `bitbake` shows a linear growth of
first-time contributors, `meta-openembedded` is exponential. I didn't
analyze the first derivative but to me it looks like there is a kink
following that timestamp indicated by the vertical bar. That line
represents a subtle, yet important change in intonation in the merge
requests. Before that there was always a comment "Please read README and
send patches to openembedded-devel@lists.openembedded.org for review." or
alike. After that the intonation changed to messages "Patches should be
sent to openembedded-devel as described in README." and if I interprete the
following MRs correctly the maintainers (mostly @kraj) were kind enough to
apply the MRs as patches. I'm sure there is a readon not to hit the merge
button but instead doing that manually which I don't understand. But most
important point to mention:

The development of the community around meta-openembedded is what a
community driven project should look like - the evolution of `bitbake` not
so much...

Have a look, reproduce, ask and draw your own conclusions.

Marius

Am Di., 9. Nov. 2021 um 10:28 Uhr schrieb Marius Kriegerowski via
lists.openembedded.org <marius.kriegerowski=gmail.com@lists.openembedded.org
>:

> So, I'd like to come to a conclusing with this time consuming thread:
>
> I shared my view which I'm sure many early contributors share. There is a
> lot of resentment in this developer community against changing the workflow
> (which dates back more than a decade) - which I understand. I still think
> that the inconvenience when commenting on a commit message (that was the
> only drawback I found legit) does not outweigh the benefits of discussing
> MRs on one of the platforms mentioned. We disagree on that.
>
> I'm keen on adding contributions not only to `meta-openembedded` (which
> uses github - thanks for that) but also to `bitbake`. So, I'll surrender
> and send a patch via email. Maybe. One day.
>
> Have a good day
> Marius
>
> Am Di., 9. Nov. 2021 um 10:09 Uhr schrieb Marius Kriegerowski via
> lists.openembedded.org <marius.kriegerowski=
> gmail.com@lists.openembedded.org>:
>
>>
>>
>> On 9. Nov 2021, at 09:44, Alexander Kanavin <alex.kanavin@gmail.com>
>> wrote:
>>
>> On Tue, 9 Nov 2021 at 09:32, Marius Kriegerowski <
>> marius.kriegerowski@gmail.com> wrote:
>>
>>> I see.
>>>
>>> On 9. Nov 2021, at 09:19, Alexander Kanavin <alex.kanavin@gmail.com>
>>> wrote:
>>>
>>> I think you misunderstood me too. It's about the github UI which does
>>> its best to hide the commit messages, and doesn't allow to comment on their
>>> content. Yes you can contort yourself and still do it, but the tools do not
>>> help.
>>>
>>>
>>> I never felt that the commit messages are really that hidden. I mean…
>>> you find them under `commits`... I find that pretty transparent. You can
>>> comment on the commit message content in the thread of the MR, just as you
>>> would in an email thread, can't you?
>>>
>>
>> No. You need to separately go to the commits tab, open each commit
>> individually to see both the message and the associated code, then if
>> there's something you do not like, you need to copy paste the offending bit
>> and open a MR thread with it. That's not helpful, and it's clear that kind
>> of thing is not important to the tool providers or users. But it is very
>> important to me as one of the oe-core maintainers.
>>
>>
>>
>>
>>
>>
>>> Also, the older and closed threads can be very helpful when searching
>>> for answers why something has been changed in which way. I know that I saw
>>> the email threads on patches as well but found them super hard to read (no
>>> syntax highlighting, all black and white, …). Well, probably I could get
>>> used to reading them over time. But then — why should I if there are things
>>> like collapsable threads, syntax highlight, color,…?
>>>
>>
>> You can find a better email client perhaps?
>>
>>
>> Not sure I was clear about what I meant: I was speaking about the closed
>> threads (https://lists.openembedded.org/g/openembedded-core). Is it
>> somehow sortable by open MRs/closed MRs/pure text/issues?
>>
>> So, which email client would I have to install to get a better user
>> experience? What about the other features that make me use the email client
>> because of which I which Iike to use that client? Do you expect me to dump
>> them those features and my workflow routines in favour of just
>> openembedded?! Should I have two email clients then? And then hard-copy the
>> old mail threads to my HD from how many years?
>>
>> BTW: I just went for the first time to the online thread:
>> https://lists.openembedded.org/g/bitbake-devel/topic/86911583
>> I would probably stop reading that monstrous thread after few seconds.
>>
>> So, to move forward: Maybe you can improve the online threads so that
>> it’s easier to navigate and read those pure retro-black-and-white threads.
>>
>> Marius
>>
>>
>>
>> PS I’d like to emphasise again:
>> I know I sound harsh. I’m not blaming here and of course I’m also not
>> belittling the workflow neither the proficiency of the contributors. I’m
>> totally aware that most regular contributors here are extremely highly
>> skilled and work on an incredibly impactful set of software. I’m just
>> providing a very detailed feedback of what I as a really really really
>> really hard wanting-to-become future contributor see what really keeps me
>> from joining.
>>
>>
>>
>> Alex
>>
>>
>>>
>>> Best
>>> Marius
>>>
>>>
>>> Alex
>>>
>>> On Tue, 9 Nov 2021 at 09:05, Marius Kriegerowski <
>>> marius.kriegerowski@gmail.com> wrote:
>>>
>>>> Good morning,
>>>>
>>>> I don’t think this is correct. You can change the git history and
>>>> commit messages of every MR to whatever you like with `git commit —amend`,
>>>> `git rebase -I` and `git push -f`. So, you can very well ask contributors
>>>> to cleanup their history before merging.
>>>>
>>>> Best
>>>> Marius
>>>>
>>>> Am Di., 9. Nov. 2021 um 07:47 Uhr schrieb Alexander Kanavin <
>>>> alex.kanavin@gmail.com>:
>>>>
>>>>> There's another problem with github/gitlab which I find significant.
>>>>> These tools do not allow reviewing commit messages, and don't even make it
>>>>> easy to see them. So the developers using those tools, both submitters and
>>>>> maintainers, are effectively being conditioned by the tools to see good
>>>>> commit messages as superfluous and unnecessary. And so you more often than
>>>>> not end up with poorly written commit history in github projects, or, if
>>>>> you're a reviewer, have to repeatedly explain to the submitter how to
>>>>> explain and structure their changes properly.
>>>>>
>>>>> I'd rather not have a change, than have a change that no one, even the
>>>>> original author, can understand or remember the reasons for one year down
>>>>> the line.
>>>>>
>>>>> Alex
>>>>>
>>>>>
>>>>> On Tue, 9 Nov 2021 at 00:30, Richard Purdie <
>>>>> richard.purdie@linuxfoundation.org> wrote:
>>>>>
>>>>>> On Mon, 2021-11-08 at 23:44 +0100, Marius Kriegerowski wrote:
>>>>>> >
>>>>>> >
>>>>>> > > On 8. Nov 2021, at 22:47, Richard Purdie
>>>>>> > > <richard.purdie@linuxfoundation.org> wrote:
>>>>>> > >
>>>>>> > > On Mon, 2021-11-08 at 18:37 +0100, Marius Kriegerowski wrote:
>>>>>> > > > Is there a plan to switch to some more modern ways to
>>>>>> collaborate on
>>>>>> > > > bitbake,
>>>>>> > > > you know, like, gitlab/github/gitea with issues, PRs and so
>>>>>> forth? I mean…
>>>>>> > > > it’s
>>>>>> > > > late 2021 and sending MRs/Issues via email feels reaaaally
>>>>>> outdated, slow,
>>>>>> > > > intransparent, ineffective (how to you do reviews, code
>>>>>> comparisons during
>>>>>> > > > reviews, insert suggestions, etc.) and indirect (where is the
>>>>>> CI pipeline
>>>>>> > > > that I
>>>>>> > > > trigger when pushing something somewhere?).
>>>>>> > >
>>>>>> > > There are pros and cons to switching to something else. Keep in
>>>>>> mind that
>>>>>> > > whilst the github workflow works well for a single maintainer of
>>>>>> a project,
>>>>>> > > it does not work well for diverse review.
>>>>>> >
>>>>>> > I strongly disagree with that. Reviews can equally well be done in
>>>>>> a MR
>>>>>> > thread. That’s what they are there for. You can discuss things
>>>>>> there with as
>>>>>> > many people as you want and can e.g. define a minimum number of
>>>>>> approvals
>>>>>> > before a MR can be merged.
>>>>>>
>>>>>> You misunderstand my point. We have a several thousand people on the
>>>>>> mailing
>>>>>> lists and they filter things to what interests them. *I* have no
>>>>>> people I want
>>>>>> to discuss a given change with, it is a question of whether the
>>>>>> people who have
>>>>>> an interest in a given change would see it.
>>>>>>
>>>>>> A minimum number of approvals is utterly meaningless for how we work,
>>>>>> I need the
>>>>>> interested people to see it and that varies massively depending on
>>>>>> what is being
>>>>>> changed. It is this aspect of mailing list change distribution and
>>>>>> discussion we
>>>>>> benefit from.
>>>>>>
>>>>>> >  It also naturally groups discussions. You can address experts for
>>>>>> specific
>>>>>> > domains which means less noise for the experts. I really don’t see
>>>>>> why mailing
>>>>>> > list reviews should work any better here.
>>>>>>
>>>>>> We all have established mechanisms to "see" the things which interest
>>>>>> us as
>>>>>> maintainers?
>>>>>>
>>>>>> Please don't misunderstand me, I do see the attraction of github from
>>>>>> your
>>>>>> perspective. It would just destroy the ecosystem we already have. You
>>>>>> are
>>>>>> basically saying that our current ecosystem is worthless but I would
>>>>>> point out
>>>>>> it has resulted in a project which you did say you appreciate so it
>>>>>> can't be all
>>>>>> bad.
>>>>>>
>>>>>> >  If google can coordinate 2.1k repos with 1.1k contributors over
>>>>>> GitHub, why
>>>>>> > should that not work for openembedded??
>>>>>>
>>>>>> It isn't about raw numbers, it is about numbers of reviewers for a
>>>>>> given change.
>>>>>> We have hundreds of people doing that using the mailing lists.
>>>>>>
>>>>>> > > The Bitbake case is a little bit different but in general we'd
>>>>>> follow what
>>>>>> > > OE/Yocto Project are doing.
>>>>>> > >
>>>>>> > > We have discussed changing and it is listed as one topic on:
>>>>>> > > https://wiki.yoctoproject.org/wiki/Future_Directions
>>>>>> > > (Code Submission process)
>>>>>> > >
>>>>>> > > I'd note that it is a huge change to our current developers and
>>>>>> most of us
>>>>>> > > are very used to the mailing lists so if we did change, it would
>>>>>> allow
>>>>>> > > potential new collaborators at the risk of losing people with key
>>>>>> > > knowledge/experience.
>>>>>> > … which reads to me like: “It has always been that way so we will
>>>>>> not change
>>>>>> > that" doesn’t it? Always a very poor argument.
>>>>>>
>>>>>> No, what I said is "upsetting our existing developers is something
>>>>>> which we may
>>>>>> want to potentially avoid".
>>>>>>
>>>>>> There are some options out there which have offered mailing list
>>>>>> integration for
>>>>>> example and we have tried to explore those. Something which drops the
>>>>>> lists
>>>>>> entirely could in the current view of the TSC result in loss of too
>>>>>> many of our
>>>>>> existing maintainers.
>>>>>>
>>>>>> >
>>>>>> > > We do have CI which is shared with Yocto Project and
>>>>>> > > OpenEmbedded. You can see it here:
>>>>>> > >
>>>>>> > > https://autobuilder.yoctoproject.org/typhoon/#/console
>>>>>> > >
>>>>>> > > and it has it's own manual which was recently added to our
>>>>>> documentations
>>>>>> > > set:
>>>>>> > >
>>>>>> > > http://docs.yoctoproject.org/test-manual/index.html
>>>>>> > … stressing my point: Why does the location of the CI need
>>>>>> documentation? All
>>>>>> > of the aforementioned platforms smoothly integrate with multiple CI
>>>>>> solutions,
>>>>>> > drastically reducing the time required for reviews if the pipeline
>>>>>> provides
>>>>>> > automated feedback to the contributor. It would probably cost me
>>>>>> about one
>>>>>> > afternoon to setup a decent CI pipeline at least for all python
>>>>>> codes pushing
>>>>>> > the need to review from manual labor to automated tools. If open
>>>>>> embedded
>>>>>> > migrates to a proper git platform I happily volunteer on that end!!
>>>>>>
>>>>>> We're not talking about simple python code. We're talking about
>>>>>> testing a build
>>>>>> environment which cross compiles multiple operating systems for
>>>>>> multiple
>>>>>> architectures, C libraries, GUI stacks, init systems and so on.
>>>>>> Testing the
>>>>>> python code is one small part of that. I've spent several years
>>>>>> trying to
>>>>>> improve and get our CI to where it is today.
>>>>>>
>>>>>> Yes, the python code is tested as part of this as we don't really
>>>>>> want to have
>>>>>> multiple test platforms and the real world usage is some of the best
>>>>>> testing
>>>>>> changes can have.
>>>>>>
>>>>>> > > There is work still needed on that but it is a start. One
>>>>>> challenge of
>>>>>> > > bitbake/OE is that the builds take an age and the test matrix is
>>>>>> huge so the
>>>>>> > > CI
>>>>>> > > process isn't quite as simple as some other simpler projects.
>>>>>> > At least for python I bet that most bugs can be caught simply by
>>>>>> firing up
>>>>>> > `black`, `flake8`, `pylint` (and a hand full of other tools) just
>>>>>> to check the
>>>>>> > modified files.
>>>>>>
>>>>>> Well, I would bet that most of our bugs are not caused by issues like
>>>>>> that. I do
>>>>>> attend the weekly bug triage meetings and have fixed my share of
>>>>>> them. You could
>>>>>> have a look at the issues filed in bugzilla over the past few weeks
>>>>>> if you'd
>>>>>> like to see the kinds of issues that get filed and are open.
>>>>>>
>>>>>> >  Regarding meta-layers and recipes out there we are currently
>>>>>> pushing forward
>>>>>> > with a linter for bitbake recipes (thanks @priv-kweihmann for this
>>>>>> awesome and
>>>>>> > overdue tool!!!).
>>>>>>
>>>>>> Whilst improving recipes in that way does help, it isn't the main
>>>>>> source of our
>>>>>> major issues.
>>>>>>
>>>>>> > >
>>>>>> > > As a shortcut through things, the "bitbake-selftest" command
>>>>>> within bitbake
>>>>>> > > may
>>>>>> > > be one thing you're looking for which does simple tests on
>>>>>> bitbake. It isn't
>>>>>> > > complete coverage and we welcome more being added, it is a start
>>>>>> and some
>>>>>> > > areas
>>>>>> > > of the code are covered better than others.
>>>>>> > I’d love to overhaul the tests e.g. porting them to `pytest` which
>>>>>> offers a
>>>>>> > much simpler yet much more powerful API/plugin-ecosystem. But won’t
>>>>>> do via
>>>>>> > email :)
>>>>>>
>>>>>> The tests are written in python's own unittest.
>>>>>>
>>>>>> > > Issues are filed in the project bugzilla and are actively worked
>>>>>> on:
>>>>>> > >
>>>>>> > > https://bugzilla.yoctoproject.org/
>>>>>> > Didn’t know that. But why keeping issues separate from MRs?
>>>>>> Automatic issue
>>>>>> > closing through referencing in MRs is a pretty nice feature…
>>>>>>
>>>>>> If I had to list the pain points we have, this is not high on the
>>>>>> list.
>>>>>>
>>>>>> > > > * side-note on bitbake style/linting: It’s violating PEP8 in a
>>>>>> couple of
>>>>>> > > > ways and TBH PEP8 is the bare minimum I would accept these days
>>>>>> for
>>>>>> > > > collaboration on projects the dimension of bitbake. So an
>>>>>> additional
>>>>>> > > > question: Would you be up for a PEP8 compatibility overhaul of
>>>>>> the source
>>>>>> > > > code? I’d volunteer if I don’t have to send a patch via email ;)
>>>>>> > >
>>>>>> > > We're interested in patches to improve the coding style as long
>>>>>> as they
>>>>>> > > don't
>>>>>> > > break things like readability or performance. Which rules in PEP8
>>>>>> in
>>>>>> > > particular
>>>>>> > > are you referring to?
>>>>>> >
>>>>>> > To name a few violations:
>>>>>> >
>>>>>> >  * hanging indents
>>>>>> >  * inconsistent indentation
>>>>>> >  * 2 separating lines missing in between classes/top level methods
>>>>>>
>>>>>> FWIW I find it really hard to get worked up about whitespace issues. I
>>>>>> appreciate these aren't always great to look at and I appreciate the
>>>>>> familiarity
>>>>>> argument and standardising the look but these aren't issues I would
>>>>>> prioritise.
>>>>>>
>>>>>> In particular changing them globally in mass patches tends to break
>>>>>> patch
>>>>>> backporting and taking fixes back to older code which we do benefit
>>>>>> from being
>>>>>> able to do so there is a cost to doing it. We are trying to make new
>>>>>> code more
>>>>>> consistent and with older code, we do try and fix the worst issues.
>>>>>>
>>>>>> >  * unused imported modules
>>>>>> >  * missing white-spaces after comma
>>>>>> >  * unreachable codes
>>>>>>
>>>>>> These ones we do take patches for and try to fix when/where we see
>>>>>> it. For
>>>>>> example,
>>>>>> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511
>>>>>> d830df9cbd4113a4352b2c contained a couple of white-space after comma
>>>>>> fixing
>>>>>> while I was in that area.
>>>>>>
>>>>>> > These are just a couple of examples that struck my PEP8 used eye
>>>>>> from looking
>>>>>> > at two, three modules.
>>>>>> >
>>>>>> > It’s just like with any code smell: they don’t necessarily break
>>>>>> things. But
>>>>>> > they make maintenance, code-reviews and quality control harder than
>>>>>> necessary.
>>>>>> > Code that does not at least follow PEP8 (and I actually strongly
>>>>>> recommend to
>>>>>> > go the extra mile with `black`) just looks odd and is unnecessary
>>>>>> hard to
>>>>>> > read. I’m surprised I have to argue for standards.
>>>>>>
>>>>>> I wasn't saying it was a bad thing, I asked which issues you noticed
>>>>>> in
>>>>>> particular since I'm trying to ensure we do at least try and react to
>>>>>> feedback.
>>>>>> Some whitespace issues we aren't likely to tackle, some like the
>>>>>> others you
>>>>>> mentioned above we should fix.
>>>>>> >
>>>>>> > Besides that, some minor cleanup just to warm
>>>>>> > up: https://github.com/openembedded/bitbake/pull/33,
>>>>>> https://github.com/openem
>>>>>> > bedded/bitbake/pull/32/files
>>>>>>
>>>>>> Those two do look like reasonable changes FWIW.
>>>>>>
>>>>>> > (it was after these two that I found the comment in one of those
>>>>>> old commits
>>>>>> > which made me open https://github.com/openembedded/bitbake/pull/34).
>>>>>> At least
>>>>>> > there should be an automated reply to each MR that is posted if
>>>>>> someone makes
>>>>>> > the effort to fix code and open a MRs that this is actually in vein
>>>>>> - That’s
>>>>>> > the mood that took me here :)
>>>>>>
>>>>>> There is a certain irony in that given you patched the section called
>>>>>> "Contributing" in the README in 32!
>>>>>>
>>>>>> > >
>>>>>> > > Help appreciated but email patches is how we accept them at the
>>>>>> moment.
>>>>>> >
>>>>>> > Not from me then :(
>>>>>>
>>>>>> Sorry to hear that, your changes in 32/33 do look reasonable.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Richard
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>
>>
>> Am Di., 9. Nov. 2021 um 09:49 Uhr schrieb <Mikko.Rapeli@bmw.de>:
>>
>>> Hi,
>>>
>>> I'm a heavy user of yocto and bitbake, CI systems and automation, even
>>> Gerrit
>>> and GitHub, yet I also perfer they way email allows for a completely
>>> distributed
>>> review and possibility to follow and join the discussions when ever a
>>> subject
>>> or a change triggers me to respond or hits a nerve. Gerrit, github,
>>> gitlab etc
>>> they just don't scale this well. I understand that for drive-by
>>> contributions working
>>> with email can be tricky if you are not used it, but same goes for
>>> github, gerrit,
>>> gitlab etc.
>>>
>>> This community is also very welcoming, well behaving and helping first
>>> time
>>> contributors. I do see the problem that some first time contributions
>>> may be
>>> see the hurdle of "git send-email" as too hard, especially if corporate
>>> email systems
>>> mangle emails. Maybe some tooling could be setup to allow for pull
>>> request type
>>> contributions to popup on mailing list if sending emails just doesn't
>>> work, but I
>>> don't see the benefits of moving fully to gerrit/github/gitlabl style
>>> development.
>>>
>>> ps. maybe I'm also part of the old generation which still actually reads
>>> all emails.
>>> I don't see any of the chat/collaboration tools really replacing email
>>> in global
>>> R&D development world even if chat tools/Slack/MS Teams etc are being
>>> proposed
>>> as replacements (they simply don't scale so well in the end).
>>>
>>> Cheers,
>>>
>>> -Mikko
>>
>>
>>
>>
>>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12960):
> https://lists.openembedded.org/g/bitbake-devel/message/12960
> Mute This Topic: https://lists.openembedded.org/mt/86911583/6547911
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> marius.kriegerowski@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #1.2: Type: text/html, Size: 32987 bytes --]

[-- Attachment #2: fist-time-contributors-bitbake.png --]
[-- Type: image/png, Size: 26676 bytes --]

[-- Attachment #3: first-time-contributors-meta-openembedded.png --]
[-- Type: image/png, Size: 27445 bytes --]

[-- Attachment #4: first_contributors_evaluation.py --]
[-- Type: text/x-python-script, Size: 1507 bytes --]

import datetime
import logging
import sys

import matplotlib.pyplot as plt
import numpy
from git import Repo


"""Investigate the development of first contributors of repositories

Significant timestamps and time limits are hard coded which should be adjusted.

Depends on https://gitpython.readthedocs.io/en/stable/
"""


if __name__ == "__main__":

    path = sys.argv[-1]

    logging.basicConfig(level=logging.INFO)
    repo = Repo(path)

    authors = set()
    timestamps = []
    tmin_str = "2012-01-01 00:00:00"
    tmin = datetime.datetime.fromisoformat(tmin_str).timestamp()
    t_shouldbe = datetime.datetime.fromisoformat("2018-02-02 00:00:00").timestamp()

    for commit in repo.iter_commits("master"):

        author = commit.author
        timestamp = commit.committed_datetime.timestamp()

        if author in authors or timestamp <= tmin:
            continue
        else:
            authors.add(author)

        timestamps.append(commit.committed_datetime.timestamp())

    timestamps.reverse()
    data = numpy.array([timestamps, range(len(timestamps))])

    ax = plt.subplot(111)
    ax.set_title(f"{path.split('/')[-1]}\nFirst time contributors since {tmin_str}")
    ax.set_xlabel("timestamp")
    ax.set_ylabel("first time contributor count")
    ax.plot(*data)
    ax.axvline(t_shouldbe, color="grey")
    ax.text(t_shouldbe, 10, " lifting email patch\n restriction intonation")
    ax.spines["right"].set_visible(False)
    ax.spines["top"].set_visible(False)

    plt.show()

Re: [bitbake-devel] bitbake style/workflow

[Marius Kriegerowski]

[-- Attachment #1: Type: text/plain, Size: 23006 bytes --]

BTW: I’m honestly asking for some feedback of this analysis. I’m
considering taking this as a starting point for an article on medium.com as
I think that there might be other projects that may learn something from
analysis their contributors base evolution.

Am Di., 9. Nov. 2021 um 12:56 Uhr schrieb Marius Kriegerowski <
marius.kriegerowski@gmail.com>:

> Okay, one thing I have to add after killing some time with this: I quickly
> scrutinized the git history of `bitbake` and `meta-openembedded`.
>
> Attached are two figures that show the first time contributor count of
> both projects over the last ~ten years. While `bitbake` shows a linear
> growth of first-time contributors, `meta-openembedded` is exponential. I
> didn't analyze the first derivative but to me it looks like there is a kink
> following that timestamp indicated by the vertical bar. That line
> represents a subtle, yet important change in intonation in the merge
> requests. Before that there was always a comment "Please read README and
> send patches to openembedded-devel@lists.openembedded.org for review." or
> alike. After that the intonation changed to messages "Patches should be
> sent to openembedded-devel as described in README." and if I interprete the
> following MRs correctly the maintainers (mostly @kraj) were kind enough to
> apply the MRs as patches. I'm sure there is a readon not to hit the merge
> button but instead doing that manually which I don't understand. But most
> important point to mention:
>
> The development of the community around meta-openembedded is what a
> community driven project should look like - the evolution of `bitbake` not
> so much...
>
> Have a look, reproduce, ask and draw your own conclusions.
>
> Marius
>
> Am Di., 9. Nov. 2021 um 10:28 Uhr schrieb Marius Kriegerowski via
> lists.openembedded.org <marius.kriegerowski=
> gmail.com@lists.openembedded.org>:
>
>> So, I'd like to come to a conclusing with this time consuming thread:
>>
>> I shared my view which I'm sure many early contributors share. There is a
>> lot of resentment in this developer community against changing the workflow
>> (which dates back more than a decade) - which I understand. I still think
>> that the inconvenience when commenting on a commit message (that was the
>> only drawback I found legit) does not outweigh the benefits of discussing
>> MRs on one of the platforms mentioned. We disagree on that.
>>
>> I'm keen on adding contributions not only to `meta-openembedded` (which
>> uses github - thanks for that) but also to `bitbake`. So, I'll surrender
>> and send a patch via email. Maybe. One day.
>>
>> Have a good day
>> Marius
>>
>> Am Di., 9. Nov. 2021 um 10:09 Uhr schrieb Marius Kriegerowski via
>> lists.openembedded.org <marius.kriegerowski=
>> gmail.com@lists.openembedded.org>:
>>
>>>
>>>
>>> On 9. Nov 2021, at 09:44, Alexander Kanavin <alex.kanavin@gmail.com>
>>> wrote:
>>>
>>> On Tue, 9 Nov 2021 at 09:32, Marius Kriegerowski <
>>> marius.kriegerowski@gmail.com> wrote:
>>>
>>>> I see.
>>>>
>>>> On 9. Nov 2021, at 09:19, Alexander Kanavin <alex.kanavin@gmail.com>
>>>> wrote:
>>>>
>>>> I think you misunderstood me too. It's about the github UI which does
>>>> its best to hide the commit messages, and doesn't allow to comment on their
>>>> content. Yes you can contort yourself and still do it, but the tools do not
>>>> help.
>>>>
>>>>
>>>> I never felt that the commit messages are really that hidden. I mean…
>>>> you find them under `commits`... I find that pretty transparent. You can
>>>> comment on the commit message content in the thread of the MR, just as you
>>>> would in an email thread, can't you?
>>>>
>>>
>>> No. You need to separately go to the commits tab, open each commit
>>> individually to see both the message and the associated code, then if
>>> there's something you do not like, you need to copy paste the offending bit
>>> and open a MR thread with it. That's not helpful, and it's clear that kind
>>> of thing is not important to the tool providers or users. But it is very
>>> important to me as one of the oe-core maintainers.
>>>
>>>
>>>
>>>
>>>
>>>
>>>> Also, the older and closed threads can be very helpful when searching
>>>> for answers why something has been changed in which way. I know that I saw
>>>> the email threads on patches as well but found them super hard to read (no
>>>> syntax highlighting, all black and white, …). Well, probably I could get
>>>> used to reading them over time. But then — why should I if there are things
>>>> like collapsable threads, syntax highlight, color,…?
>>>>
>>>
>>> You can find a better email client perhaps?
>>>
>>>
>>> Not sure I was clear about what I meant: I was speaking about the closed
>>> threads (https://lists.openembedded.org/g/openembedded-core). Is it
>>> somehow sortable by open MRs/closed MRs/pure text/issues?
>>>
>>> So, which email client would I have to install to get a better user
>>> experience? What about the other features that make me use the email client
>>> because of which I which Iike to use that client? Do you expect me to dump
>>> them those features and my workflow routines in favour of just
>>> openembedded?! Should I have two email clients then? And then hard-copy the
>>> old mail threads to my HD from how many years?
>>>
>>> BTW: I just went for the first time to the online thread:
>>> https://lists.openembedded.org/g/bitbake-devel/topic/86911583
>>> I would probably stop reading that monstrous thread after few seconds.
>>>
>>> So, to move forward: Maybe you can improve the online threads so that
>>> it’s easier to navigate and read those pure retro-black-and-white threads.
>>>
>>> Marius
>>>
>>>
>>>
>>> PS I’d like to emphasise again:
>>> I know I sound harsh. I’m not blaming here and of course I’m also not
>>> belittling the workflow neither the proficiency of the contributors. I’m
>>> totally aware that most regular contributors here are extremely highly
>>> skilled and work on an incredibly impactful set of software. I’m just
>>> providing a very detailed feedback of what I as a really really really
>>> really hard wanting-to-become future contributor see what really keeps me
>>> from joining.
>>>
>>>
>>>
>>> Alex
>>>
>>>
>>>>
>>>> Best
>>>> Marius
>>>>
>>>>
>>>> Alex
>>>>
>>>> On Tue, 9 Nov 2021 at 09:05, Marius Kriegerowski <
>>>> marius.kriegerowski@gmail.com> wrote:
>>>>
>>>>> Good morning,
>>>>>
>>>>> I don’t think this is correct. You can change the git history and
>>>>> commit messages of every MR to whatever you like with `git commit —amend`,
>>>>> `git rebase -I` and `git push -f`. So, you can very well ask contributors
>>>>> to cleanup their history before merging.
>>>>>
>>>>> Best
>>>>> Marius
>>>>>
>>>>> Am Di., 9. Nov. 2021 um 07:47 Uhr schrieb Alexander Kanavin <
>>>>> alex.kanavin@gmail.com>:
>>>>>
>>>>>> There's another problem with github/gitlab which I find significant.
>>>>>> These tools do not allow reviewing commit messages, and don't even make it
>>>>>> easy to see them. So the developers using those tools, both submitters and
>>>>>> maintainers, are effectively being conditioned by the tools to see good
>>>>>> commit messages as superfluous and unnecessary. And so you more often than
>>>>>> not end up with poorly written commit history in github projects, or, if
>>>>>> you're a reviewer, have to repeatedly explain to the submitter how to
>>>>>> explain and structure their changes properly.
>>>>>>
>>>>>> I'd rather not have a change, than have a change that no one, even
>>>>>> the original author, can understand or remember the reasons for one year
>>>>>> down the line.
>>>>>>
>>>>>> Alex
>>>>>>
>>>>>>
>>>>>> On Tue, 9 Nov 2021 at 00:30, Richard Purdie <
>>>>>> richard.purdie@linuxfoundation.org> wrote:
>>>>>>
>>>>>>> On Mon, 2021-11-08 at 23:44 +0100, Marius Kriegerowski wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > > On 8. Nov 2021, at 22:47, Richard Purdie
>>>>>>> > > <richard.purdie@linuxfoundation.org> wrote:
>>>>>>> > >
>>>>>>> > > On Mon, 2021-11-08 at 18:37 +0100, Marius Kriegerowski wrote:
>>>>>>> > > > Is there a plan to switch to some more modern ways to
>>>>>>> collaborate on
>>>>>>> > > > bitbake,
>>>>>>> > > > you know, like, gitlab/github/gitea with issues, PRs and so
>>>>>>> forth? I mean…
>>>>>>> > > > it’s
>>>>>>> > > > late 2021 and sending MRs/Issues via email feels reaaaally
>>>>>>> outdated, slow,
>>>>>>> > > > intransparent, ineffective (how to you do reviews, code
>>>>>>> comparisons during
>>>>>>> > > > reviews, insert suggestions, etc.) and indirect (where is the
>>>>>>> CI pipeline
>>>>>>> > > > that I
>>>>>>> > > > trigger when pushing something somewhere?).
>>>>>>> > >
>>>>>>> > > There are pros and cons to switching to something else. Keep in
>>>>>>> mind that
>>>>>>> > > whilst the github workflow works well for a single maintainer of
>>>>>>> a project,
>>>>>>> > > it does not work well for diverse review.
>>>>>>> >
>>>>>>> > I strongly disagree with that. Reviews can equally well be done in
>>>>>>> a MR
>>>>>>> > thread. That’s what they are there for. You can discuss things
>>>>>>> there with as
>>>>>>> > many people as you want and can e.g. define a minimum number of
>>>>>>> approvals
>>>>>>> > before a MR can be merged.
>>>>>>>
>>>>>>> You misunderstand my point. We have a several thousand people on the
>>>>>>> mailing
>>>>>>> lists and they filter things to what interests them. *I* have no
>>>>>>> people I want
>>>>>>> to discuss a given change with, it is a question of whether the
>>>>>>> people who have
>>>>>>> an interest in a given change would see it.
>>>>>>>
>>>>>>> A minimum number of approvals is utterly meaningless for how we
>>>>>>> work, I need the
>>>>>>> interested people to see it and that varies massively depending on
>>>>>>> what is being
>>>>>>> changed. It is this aspect of mailing list change distribution and
>>>>>>> discussion we
>>>>>>> benefit from.
>>>>>>>
>>>>>>> >  It also naturally groups discussions. You can address experts for
>>>>>>> specific
>>>>>>> > domains which means less noise for the experts. I really don’t see
>>>>>>> why mailing
>>>>>>> > list reviews should work any better here.
>>>>>>>
>>>>>>> We all have established mechanisms to "see" the things which
>>>>>>> interest us as
>>>>>>> maintainers?
>>>>>>>
>>>>>>> Please don't misunderstand me, I do see the attraction of github
>>>>>>> from your
>>>>>>> perspective. It would just destroy the ecosystem we already have.
>>>>>>> You are
>>>>>>> basically saying that our current ecosystem is worthless but I would
>>>>>>> point out
>>>>>>> it has resulted in a project which you did say you appreciate so it
>>>>>>> can't be all
>>>>>>> bad.
>>>>>>>
>>>>>>> >  If google can coordinate 2.1k repos with 1.1k contributors over
>>>>>>> GitHub, why
>>>>>>> > should that not work for openembedded??
>>>>>>>
>>>>>>> It isn't about raw numbers, it is about numbers of reviewers for a
>>>>>>> given change.
>>>>>>> We have hundreds of people doing that using the mailing lists.
>>>>>>>
>>>>>>> > > The Bitbake case is a little bit different but in general we'd
>>>>>>> follow what
>>>>>>> > > OE/Yocto Project are doing.
>>>>>>> > >
>>>>>>> > > We have discussed changing and it is listed as one topic on:
>>>>>>> > > https://wiki.yoctoproject.org/wiki/Future_Directions
>>>>>>> > > (Code Submission process)
>>>>>>> > >
>>>>>>> > > I'd note that it is a huge change to our current developers and
>>>>>>> most of us
>>>>>>> > > are very used to the mailing lists so if we did change, it would
>>>>>>> allow
>>>>>>> > > potential new collaborators at the risk of losing people with key
>>>>>>> > > knowledge/experience.
>>>>>>> > … which reads to me like: “It has always been that way so we will
>>>>>>> not change
>>>>>>> > that" doesn’t it? Always a very poor argument.
>>>>>>>
>>>>>>> No, what I said is "upsetting our existing developers is something
>>>>>>> which we may
>>>>>>> want to potentially avoid".
>>>>>>>
>>>>>>> There are some options out there which have offered mailing list
>>>>>>> integration for
>>>>>>> example and we have tried to explore those. Something which drops
>>>>>>> the lists
>>>>>>> entirely could in the current view of the TSC result in loss of too
>>>>>>> many of our
>>>>>>> existing maintainers.
>>>>>>>
>>>>>>> >
>>>>>>> > > We do have CI which is shared with Yocto Project and
>>>>>>> > > OpenEmbedded. You can see it here:
>>>>>>> > >
>>>>>>> > > https://autobuilder.yoctoproject.org/typhoon/#/console
>>>>>>> > >
>>>>>>> > > and it has it's own manual which was recently added to our
>>>>>>> documentations
>>>>>>> > > set:
>>>>>>> > >
>>>>>>> > > http://docs.yoctoproject.org/test-manual/index.html
>>>>>>> > … stressing my point: Why does the location of the CI need
>>>>>>> documentation? All
>>>>>>> > of the aforementioned platforms smoothly integrate with multiple
>>>>>>> CI solutions,
>>>>>>> > drastically reducing the time required for reviews if the pipeline
>>>>>>> provides
>>>>>>> > automated feedback to the contributor. It would probably cost me
>>>>>>> about one
>>>>>>> > afternoon to setup a decent CI pipeline at least for all python
>>>>>>> codes pushing
>>>>>>> > the need to review from manual labor to automated tools. If open
>>>>>>> embedded
>>>>>>> > migrates to a proper git platform I happily volunteer on that end!!
>>>>>>>
>>>>>>> We're not talking about simple python code. We're talking about
>>>>>>> testing a build
>>>>>>> environment which cross compiles multiple operating systems for
>>>>>>> multiple
>>>>>>> architectures, C libraries, GUI stacks, init systems and so on.
>>>>>>> Testing the
>>>>>>> python code is one small part of that. I've spent several years
>>>>>>> trying to
>>>>>>> improve and get our CI to where it is today.
>>>>>>>
>>>>>>> Yes, the python code is tested as part of this as we don't really
>>>>>>> want to have
>>>>>>> multiple test platforms and the real world usage is some of the best
>>>>>>> testing
>>>>>>> changes can have.
>>>>>>>
>>>>>>> > > There is work still needed on that but it is a start. One
>>>>>>> challenge of
>>>>>>> > > bitbake/OE is that the builds take an age and the test matrix is
>>>>>>> huge so the
>>>>>>> > > CI
>>>>>>> > > process isn't quite as simple as some other simpler projects.
>>>>>>> > At least for python I bet that most bugs can be caught simply by
>>>>>>> firing up
>>>>>>> > `black`, `flake8`, `pylint` (and a hand full of other tools) just
>>>>>>> to check the
>>>>>>> > modified files.
>>>>>>>
>>>>>>> Well, I would bet that most of our bugs are not caused by issues
>>>>>>> like that. I do
>>>>>>> attend the weekly bug triage meetings and have fixed my share of
>>>>>>> them. You could
>>>>>>> have a look at the issues filed in bugzilla over the past few weeks
>>>>>>> if you'd
>>>>>>> like to see the kinds of issues that get filed and are open.
>>>>>>>
>>>>>>> >  Regarding meta-layers and recipes out there we are currently
>>>>>>> pushing forward
>>>>>>> > with a linter for bitbake recipes (thanks @priv-kweihmann for this
>>>>>>> awesome and
>>>>>>> > overdue tool!!!).
>>>>>>>
>>>>>>> Whilst improving recipes in that way does help, it isn't the main
>>>>>>> source of our
>>>>>>> major issues.
>>>>>>>
>>>>>>> > >
>>>>>>> > > As a shortcut through things, the "bitbake-selftest" command
>>>>>>> within bitbake
>>>>>>> > > may
>>>>>>> > > be one thing you're looking for which does simple tests on
>>>>>>> bitbake. It isn't
>>>>>>> > > complete coverage and we welcome more being added, it is a start
>>>>>>> and some
>>>>>>> > > areas
>>>>>>> > > of the code are covered better than others.
>>>>>>> > I’d love to overhaul the tests e.g. porting them to `pytest` which
>>>>>>> offers a
>>>>>>> > much simpler yet much more powerful API/plugin-ecosystem. But
>>>>>>> won’t do via
>>>>>>> > email :)
>>>>>>>
>>>>>>> The tests are written in python's own unittest.
>>>>>>>
>>>>>>> > > Issues are filed in the project bugzilla and are actively worked
>>>>>>> on:
>>>>>>> > >
>>>>>>> > > https://bugzilla.yoctoproject.org/
>>>>>>> > Didn’t know that. But why keeping issues separate from MRs?
>>>>>>> Automatic issue
>>>>>>> > closing through referencing in MRs is a pretty nice feature…
>>>>>>>
>>>>>>> If I had to list the pain points we have, this is not high on the
>>>>>>> list.
>>>>>>>
>>>>>>> > > > * side-note on bitbake style/linting: It’s violating PEP8 in a
>>>>>>> couple of
>>>>>>> > > > ways and TBH PEP8 is the bare minimum I would accept these
>>>>>>> days for
>>>>>>> > > > collaboration on projects the dimension of bitbake. So an
>>>>>>> additional
>>>>>>> > > > question: Would you be up for a PEP8 compatibility overhaul of
>>>>>>> the source
>>>>>>> > > > code? I’d volunteer if I don’t have to send a patch via email
>>>>>>> ;)
>>>>>>> > >
>>>>>>> > > We're interested in patches to improve the coding style as long
>>>>>>> as they
>>>>>>> > > don't
>>>>>>> > > break things like readability or performance. Which rules in
>>>>>>> PEP8 in
>>>>>>> > > particular
>>>>>>> > > are you referring to?
>>>>>>> >
>>>>>>> > To name a few violations:
>>>>>>> >
>>>>>>> >  * hanging indents
>>>>>>> >  * inconsistent indentation
>>>>>>> >  * 2 separating lines missing in between classes/top level methods
>>>>>>>
>>>>>>> FWIW I find it really hard to get worked up about whitespace issues.
>>>>>>> I
>>>>>>> appreciate these aren't always great to look at and I appreciate the
>>>>>>> familiarity
>>>>>>> argument and standardising the look but these aren't issues I would
>>>>>>> prioritise.
>>>>>>>
>>>>>>> In particular changing them globally in mass patches tends to break
>>>>>>> patch
>>>>>>> backporting and taking fixes back to older code which we do benefit
>>>>>>> from being
>>>>>>> able to do so there is a cost to doing it. We are trying to make new
>>>>>>> code more
>>>>>>> consistent and with older code, we do try and fix the worst issues.
>>>>>>>
>>>>>>> >  * unused imported modules
>>>>>>> >  * missing white-spaces after comma
>>>>>>> >  * unreachable codes
>>>>>>>
>>>>>>> These ones we do take patches for and try to fix when/where we see
>>>>>>> it. For
>>>>>>> example,
>>>>>>> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=47e5c456a20432b511
>>>>>>> d830df9cbd4113a4352b2c contained a couple of white-space after
>>>>>>> comma fixing
>>>>>>> while I was in that area.
>>>>>>>
>>>>>>> > These are just a couple of examples that struck my PEP8 used eye
>>>>>>> from looking
>>>>>>> > at two, three modules.
>>>>>>> >
>>>>>>> > It’s just like with any code smell: they don’t necessarily break
>>>>>>> things. But
>>>>>>> > they make maintenance, code-reviews and quality control harder
>>>>>>> than necessary.
>>>>>>> > Code that does not at least follow PEP8 (and I actually strongly
>>>>>>> recommend to
>>>>>>> > go the extra mile with `black`) just looks odd and is unnecessary
>>>>>>> hard to
>>>>>>> > read. I’m surprised I have to argue for standards.
>>>>>>>
>>>>>>> I wasn't saying it was a bad thing, I asked which issues you noticed
>>>>>>> in
>>>>>>> particular since I'm trying to ensure we do at least try and react
>>>>>>> to feedback.
>>>>>>> Some whitespace issues we aren't likely to tackle, some like the
>>>>>>> others you
>>>>>>> mentioned above we should fix.
>>>>>>> >
>>>>>>> > Besides that, some minor cleanup just to warm
>>>>>>> > up: https://github.com/openembedded/bitbake/pull/33,
>>>>>>> https://github.com/openem
>>>>>>> > bedded/bitbake/pull/32/files
>>>>>>>
>>>>>>> Those two do look like reasonable changes FWIW.
>>>>>>>
>>>>>>> > (it was after these two that I found the comment in one of those
>>>>>>> old commits
>>>>>>> > which made me open https://github.com/openembedded/bitbake/pull/34).
>>>>>>> At least
>>>>>>> > there should be an automated reply to each MR that is posted if
>>>>>>> someone makes
>>>>>>> > the effort to fix code and open a MRs that this is actually in
>>>>>>> vein - That’s
>>>>>>> > the mood that took me here :)
>>>>>>>
>>>>>>> There is a certain irony in that given you patched the section called
>>>>>>> "Contributing" in the README in 32!
>>>>>>>
>>>>>>> > >
>>>>>>> > > Help appreciated but email patches is how we accept them at the
>>>>>>> moment.
>>>>>>> >
>>>>>>> > Not from me then :(
>>>>>>>
>>>>>>> Sorry to hear that, your changes in 32/33 do look reasonable.
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Richard
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>
>>>
>>> Am Di., 9. Nov. 2021 um 09:49 Uhr schrieb <Mikko.Rapeli@bmw.de>:
>>>
>>>> Hi,
>>>>
>>>> I'm a heavy user of yocto and bitbake, CI systems and automation, even
>>>> Gerrit
>>>> and GitHub, yet I also perfer they way email allows for a completely
>>>> distributed
>>>> review and possibility to follow and join the discussions when ever a
>>>> subject
>>>> or a change triggers me to respond or hits a nerve. Gerrit, github,
>>>> gitlab etc
>>>> they just don't scale this well. I understand that for drive-by
>>>> contributions working
>>>> with email can be tricky if you are not used it, but same goes for
>>>> github, gerrit,
>>>> gitlab etc.
>>>>
>>>> This community is also very welcoming, well behaving and helping first
>>>> time
>>>> contributors. I do see the problem that some first time contributions
>>>> may be
>>>> see the hurdle of "git send-email" as too hard, especially if corporate
>>>> email systems
>>>> mangle emails. Maybe some tooling could be setup to allow for pull
>>>> request type
>>>> contributions to popup on mailing list if sending emails just doesn't
>>>> work, but I
>>>> don't see the benefits of moving fully to gerrit/github/gitlabl style
>>>> development.
>>>>
>>>> ps. maybe I'm also part of the old generation which still actually
>>>> reads all emails.
>>>> I don't see any of the chat/collaboration tools really replacing email
>>>> in global
>>>> R&D development world even if chat tools/Slack/MS Teams etc are being
>>>> proposed
>>>> as replacements (they simply don't scale so well in the end).
>>>>
>>>> Cheers,
>>>>
>>>> -Mikko
>>>
>>>
>>>
>>>
>>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#12960):
>> https://lists.openembedded.org/g/bitbake-devel/message/12960
>> Mute This Topic: https://lists.openembedded.org/mt/86911583/6547911
>> Group Owner: bitbake-devel+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
>> marius.kriegerowski@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>>

[-- Attachment #2: Type: text/html, Size: 34638 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Richard Purdie]

On Tue, 2021-11-09 at 12:56 +0100, Marius Kriegerowski wrote:
> Okay, one thing I have to add after killing some time with this: I quickly
> scrutinized the git history of `bitbake` and `meta-openembedded`.
> 
> Attached are two figures that show the first time contributor count of both
> projects over the last ~ten years. While `bitbake` shows a linear growth of
> first-time contributors, `meta-openembedded` is exponential. I didn't analyze
> the first derivative but to me it looks like there is a kink following that
> timestamp indicated by the vertical bar. That line represents a subtle, yet
> important change in intonation in the merge requests. Before that there was
> always a comment "Please read README and send patches to
> openembedded-devel@lists.openembedded.org for review." or alike. After that
> the intonation changed to messages "Patches should be sent to openembedded-
> devel as described in README." and if I interprete the following MRs correctly
> the maintainers (mostly @kraj) were kind enough to apply the MRs as patches.
> I'm sure there is a readon not to hit the merge button but instead doing that
> manually which I don't understand. But most important point to mention:
> 
> The development of the community around meta-openembedded is what a community
> driven project should look like - the evolution of `bitbake` not so much...
> 
> Have a look, reproduce, ask and draw your own conclusions.
> > > > > > > > > > 

Thanks for sharing that, data is interesting and useful. Looking at the data, I
agree one is linear, the other is more like a growth curve however what I don't
see is that it changes from linear to the curve at the point other commits are
taken via MRs. It looks more like the change didn't make much difference as the
curve was already there before that.

The nature of the repositories and code is very different too so I'm cautious
about reading too much into this one way or another but it is interesting.

Cheers,

Richard

Re: [bitbake-devel] bitbake style/workflow

[Richard Purdie]

On Tue, 2021-11-09 at 12:56 +0100, Marius Kriegerowski wrote:
> I'm sure there is a readon not to hit the merge button but instead doing that
> manually which I don't understand.

No, you don't understand and it is actually a *really* important difference.

We are a community project with wide peer review. *Nobody* gets to merge changes
without that peer review. I may be the maintainer for some of it but even my own
patches get peer review. This means there is no single "this patch can merge"
point without it having been on the mailing list for potential discussion.

If a patch gets sent via the github MR, someone needs to take that, put it on
the mailing list and ensure the community gets to review it. Khem has kindly
taken on that role for some patches which is his choice. If there is review
feedback, it puts him in a tricky position since he then has to merge that back
to the github discussion somehow. We do not encourage this and in the case of
the repositories I maintain, I simply do not have the bandwidth to try and do
this for multiple github MRs.

I understand it is easier for people to submit but our processes simply don't
scale to that and people would get *very* upset if changes through github MRs
were bypassing normal peer review. This is why Khem cannot simply hit the merge
button and why our other repos don't support that workflow.

Cheers,

Richard

Re: [bitbake-devel] bitbake style/workflow

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2484 bytes --]

On Tue, Nov 9, 2021 at 12:56 PM Marius Kriegerowski <
marius.kriegerowski@gmail.com> wrote:

> Okay, one thing I have to add after killing some time with this: I quickly
> scrutinized the git history of `bitbake` and `meta-openembedded`.
>
> Attached are two figures that show the first time contributor count of
> both projects over the last ~ten years. While `bitbake` shows a linear
> growth of first-time contributors, `meta-openembedded` is exponential. I
> didn't analyze the first derivative but to me it looks like there is a kink
> following that timestamp indicated by the vertical bar. That line
> represents a subtle, yet important change in intonation in the merge
> requests. Before that there was always a comment "Please read README and
> send patches to openembedded-devel@lists.openembedded.org for review." or
> alike. After that the intonation changed to messages "Patches should be
> sent to openembedded-devel as described in README." and if I interprete the
> following MRs correctly the maintainers (mostly @kraj) were kind enough to
> apply the MRs as patches.
>

and as you can see in the ML, Khem is sending those commits from meta-oe
PRs as e-mails to openembedded-devel ML for others to review, which is very
kind of him, but also causes a lot of overhead (especially when there is a
feedback on ML which he needs to forward back to the discussion on the
github PR and then resend the vN patch to ML and so on).

And it's not only the review process of the new changes, there is a lot of
discussions in various MLs which even occasional contributors should find
useful (especially some global changes or project directions discussed in
openembedded-architecture ML).

I'm sure there is a readon not to hit the merge button but instead doing
> that manually which I don't understand.
>

github.com/openembedded are read-only mirrors of authoritative git server
https://git.openembedded.org/ so just hitting the merge button would break
the synchronization. And the changes are first cherry-picked to various
branches to be at least build tested in some world build, before they are
considered for merge.

PS: I'm retro, I actually prefer black and white plain text e-mails. And
for review in other projects I'm involved with I definitely prefer gerrit
over github/gitlab. Not only it allows correct review of commit messages,
but also is significantly better at comparing various revisions of the same
changes and tracking inline comments there.

Regards,

[-- Attachment #2: Type: text/html, Size: 3322 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Marius Kriegerowski]

[-- Attachment #1: Type: text/plain, Size: 1832 bytes --]

Point is: I actually don't care if you merge or use the MR to create a
patch for peer review as long as I don't have to use email...

If the process that kraj is doing manually can be automated then this would
combine the best of both worlds :thumbsup:

Marius


Am Di., 9. Nov. 2021 um 13:21 Uhr schrieb Richard Purdie <
richard.purdie@linuxfoundation.org>:

> On Tue, 2021-11-09 at 12:56 +0100, Marius Kriegerowski wrote:
> > I'm sure there is a readon not to hit the merge button but instead doing
> that
> > manually which I don't understand.
>
> No, you don't understand and it is actually a *really* important
> difference.
>
> We are a community project with wide peer review. *Nobody* gets to merge
> changes
> without that peer review. I may be the maintainer for some of it but even
> my own
> patches get peer review. This means there is no single "this patch can
> merge"
> point without it having been on the mailing list for potential discussion.
>
> If a patch gets sent via the github MR, someone needs to take that, put it
> on
> the mailing list and ensure the community gets to review it. Khem has
> kindly
> taken on that role for some patches which is his choice. If there is review
> feedback, it puts him in a tricky position since he then has to merge that
> back
> to the github discussion somehow. We do not encourage this and in the case
> of
> the repositories I maintain, I simply do not have the bandwidth to try and
> do
> this for multiple github MRs.
>
> I understand it is easier for people to submit but our processes simply
> don't
> scale to that and people would get *very* upset if changes through github
> MRs
> were bypassing normal peer review. This is why Khem cannot simply hit the
> merge
> button and why our other repos don't support that workflow.
>
> Cheers,
>
> Richard
>
>
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 2322 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Marius Kriegerowski]

[-- Attachment #1: Type: text/plain, Size: 2906 bytes --]

Sorry for double-posting BTW. Hit the resend button after I had an issue
with the local vpn here.

Funny thing is: I only get feedback from people who contributed for 10+
years or they of themselves that they are old. Can the early contributors
that are <35 yrs old and recently started contributing speak up and share
their thoughts? That would be very appreciated.

Anyways, time for lunch. I'll leave it at that I think.

Am Di., 9. Nov. 2021 um 13:26 Uhr schrieb Marius Kriegerowski via
lists.openembedded.org <marius.kriegerowski=gmail.com@lists.openembedded.org
>:

> Point is: I actually don't care if you merge or use the MR to create a
> patch for peer review as long as I don't have to use email...
>
> If the process that kraj is doing manually can be automated then this
> would combine the best of both worlds :thumbsup:
>
> Marius
>
>
> Am Di., 9. Nov. 2021 um 13:21 Uhr schrieb Richard Purdie <
> richard.purdie@linuxfoundation.org>:
>
>> On Tue, 2021-11-09 at 12:56 +0100, Marius Kriegerowski wrote:
>> > I'm sure there is a readon not to hit the merge button but instead
>> doing that
>> > manually which I don't understand.
>>
>> No, you don't understand and it is actually a *really* important
>> difference.
>>
>> We are a community project with wide peer review. *Nobody* gets to merge
>> changes
>> without that peer review. I may be the maintainer for some of it but even
>> my own
>> patches get peer review. This means there is no single "this patch can
>> merge"
>> point without it having been on the mailing list for potential discussion.
>>
>> If a patch gets sent via the github MR, someone needs to take that, put
>> it on
>> the mailing list and ensure the community gets to review it. Khem has
>> kindly
>> taken on that role for some patches which is his choice. If there is
>> review
>> feedback, it puts him in a tricky position since he then has to merge
>> that back
>> to the github discussion somehow. We do not encourage this and in the
>> case of
>> the repositories I maintain, I simply do not have the bandwidth to try
>> and do
>> this for multiple github MRs.
>>
>> I understand it is easier for people to submit but our processes simply
>> don't
>> scale to that and people would get *very* upset if changes through github
>> MRs
>> were bypassing normal peer review. This is why Khem cannot simply hit the
>> merge
>> button and why our other repos don't support that workflow.
>>
>> Cheers,
>>
>> Richard
>>
>>
>>
>>
>>
>>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12968):
> https://lists.openembedded.org/g/bitbake-devel/message/12968
> Mute This Topic: https://lists.openembedded.org/mt/86911583/6547911
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> marius.kriegerowski@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 4229 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Ernst Sjöstrand]

[-- Attachment #1: Type: text/plain, Size: 542 bytes --]

Hi,

jumping into the thread a bit randomly.

I work with Yocto at $DAYJOB and the use of a mailing list for patches has
really hindered me from contributing, with Outlook and DMARC etc.
I'd much prefer something like Gitlab! When the Mesa project switched to
Gitlab I set up an email subscription for all new merge requests
which emulated the old feed of emails pretty well IMHO.

For the code review I've always preferred Gerrit for it's powerful system
of patchsets and ability to review the commit messages very clearly.

Regards
//Ernst

[-- Attachment #2: Type: text/html, Size: 1549 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Peter Hatina]

Hi there,

same thing here.

+1

On Tuesday, November 9, 2021 1:57:15 PM CET Ernst Sjöstrand wrote:
> Hi,
> 
> jumping into the thread a bit randomly.
> 
> I work with Yocto at $DAYJOB and the use of a mailing list for patches has
> really hindered me from contributing, with Outlook and DMARC etc.
> I'd much prefer something like Gitlab! When the Mesa project switched to
> Gitlab I set up an email subscription for all new merge requests
> which emulated the old feed of emails pretty well IMHO.
> 
> For the code review I've always preferred Gerrit for it's powerful system
> of patchsets and ability to review the commit messages very clearly.
> 
> Regards
> //Ernst

-- 
s pozdravom/regards,
Peter Hatina

Re: [bitbake-devel] bitbake style/workflow

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 822 bytes --]

I have to note that being forced to use outlook/exchange at work is not, in
itself, a reason to abandon or dislike email based contribution process.

Alex

On Tue 9. Nov 2021 at 13.57, Ernst Sjöstrand <ernstp@gmail.com> wrote:

> Hi,
>
> jumping into the thread a bit randomly.
>
> I work with Yocto at $DAYJOB and the use of a mailing list for patches has
> really hindered me from contributing, with Outlook and DMARC etc.
> I'd much prefer something like Gitlab! When the Mesa project switched to
> Gitlab I set up an email subscription for all new merge requests
> which emulated the old feed of emails pretty well IMHO.
>
> For the code review I've always preferred Gerrit for it's powerful system
> of patchsets and ability to review the commit messages very clearly.
>
> Regards
> //Ernst
>

[-- Attachment #2: Type: text/html, Size: 2197 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Trevor Gamblin]

[-- Attachment #1: Type: text/plain, Size: 4661 bytes --]


On 2021-11-09 07:31, Marius Kriegerowski wrote:
>
> **[Please note: This e-mail is from an EXTERNAL e-mail address]
>
> Sorry for double-posting BTW. Hit the resend button after I had an 
> issue with the local vpn here.
>
> Funny thing is: I only get feedback from people who contributed for 
> 10+ years or they of themselves that they are old. Can the early 
> contributors that are <35 yrs old and recently started contributing 
> speak up and share their thoughts? That would be very appreciated.

33 year-old here with about 2.5 years time working on YP/OE, including 
maintaining meta-python (which I send PRs for via GitHub). The process 
used for patches was foreign and seemed a bit excessive to me at first, 
but that changed quickly. I have to echo Mikko's response - the mailing 
lists are convenient for me because they consist of mostly text records 
that I can search and filter as needed, and that I can also keep local 
backups of for longer-term referencing. There is situational appeal to 
using newer CI tools and interfaces where possible, but when it comes to 
discussion and code review I really don't want more than basic text to 
have to deal with for a combination of reasons, including but not 
limited to load times/site status/resource usage/over-stimulation/etc. 
that would come with relying on a web UI and cloud services for tracking 
multiple discussions and repositories at once.

For the record, I use Thunderbird as my client. My workflow to create 
PRs for meta-openembedded is also automated using Tekton and a few Bash 
scripts.

- Trevor

>
> Anyways, time for lunch. I'll leave it at that I think.
>
> Am Di., 9. Nov. 2021 um 13:26 Uhr schrieb Marius Kriegerowski via 
> lists.openembedded.org 
> <https://urldefense.com/v3/__http://lists.openembedded.org__;!!AjveYdw8EvQ!IvcyputSb5rOHUrgBdxvE6sfTciN5PFoyCoiylG5E91rcO8XIWZdicYKmRKm-wT1GASefw$> 
> <marius.kriegerowski=gmail.com@lists.openembedded.org>:
>
>     Point is: I actually don't care if you merge or use the MR to
>     create a patch for peer review as long as I don't have to use email...
>
>     If the process that kraj is doing manually can be automated then
>     this would combine the best of both worlds :thumbsup:
>
>     Marius
>
>
>     Am Di., 9. Nov. 2021 um 13:21 Uhr schrieb Richard Purdie
>     <richard.purdie@linuxfoundation.org>:
>
>         On Tue, 2021-11-09 at 12:56 +0100, Marius Kriegerowski wrote:
>         > I'm sure there is a readon not to hit the merge button but
>         instead doing that
>         > manually which I don't understand.
>
>         No, you don't understand and it is actually a *really*
>         important difference.
>
>         We are a community project with wide peer review. *Nobody*
>         gets to merge changes
>         without that peer review. I may be the maintainer for some of
>         it but even my own
>         patches get peer review. This means there is no single "this
>         patch can merge"
>         point without it having been on the mailing list for potential
>         discussion.
>
>         If a patch gets sent via the github MR, someone needs to take
>         that, put it on
>         the mailing list and ensure the community gets to review it.
>         Khem has kindly
>         taken on that role for some patches which is his choice. If
>         there is review
>         feedback, it puts him in a tricky position since he then has
>         to merge that back
>         to the github discussion somehow. We do not encourage this and
>         in the case of
>         the repositories I maintain, I simply do not have the
>         bandwidth to try and do
>         this for multiple github MRs.
>
>         I understand it is easier for people to submit but our
>         processes simply don't
>         scale to that and people would get *very* upset if changes
>         through github MRs
>         were bypassing normal peer review. This is why Khem cannot
>         simply hit the merge
>         button and why our other repos don't support that workflow.
>
>         Cheers,
>
>         Richard
>
>
>
>
>
>
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12969):https://lists.openembedded.org/g/bitbake-devel/message/12969
> Mute This Topic:https://lists.openembedded.org/mt/86911583/3616776
> Group Owner:bitbake-devel+owner@lists.openembedded.org
> Unsubscribe:https://lists.openembedded.org/g/bitbake-devel/unsub  [trevor.gamblin@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

[-- Attachment #2: Type: text/html, Size: 8179 bytes --]

Re: [bitbake-devel] bitbake style/workflow

[Quentin Schulz]

Hi Marius,

Too many mails gone in too many different directions so answering here.

On Mon, Nov 08, 2021 at 06:37:42PM +0100, Marius Kriegerowski wrote:
> Hey there,
> 
> Is there a plan to switch to some more modern ways to collaborate on
> bitbake, you know, like, gitlab/github/gitea with issues, PRs and so forth?
> I mean… it’s late 2021 and sending MRs/Issues via email feels reaaaally
> outdated, slow, intransparent, ineffective (how to you do reviews, code
> comparisons during reviews, insert suggestions, etc.) and indirect (where
> is the CI pipeline that I trigger when pushing something somewhere?). I

Do you have some data to support the claim that MRs via email are slow?

Intransparent? There are mail archives available with all patches
posted and their reviews. Same for issues sent on the mailing list. Though I
have to say that I do miss the "Merged, thanks" mail that Buildroot and most
of the kernel subsystems are sending. Though you can fetch the git repo and
check by yourself. AFAIU, the patchwork should be fixed some time soon
and make this a bit more explicit.

Ineffective? Yocto, Buildroot, Busybox, most kernel subsystems are using
mail-based patch sending/reviews. I would have imagined that if that way
of collaborating was so ineffective we/they would have switched by now.

How do we do reviews? You send a mail whose body is the patch and only
the patch. We receive your patch as a mail in our mailbox in plain text
and we then split the mail where we need to insert a comment. Just like
I did here, you can see that some of your message is above the first
part of my answer and some after this paragraph.

We could probably have the same mechanism that the kernel has with build
bots directly sending a mail as an answer to a patch if it fails.
However that is not the case today, I do not know if it is an
infrastructure limitation or not (builds are very expensive in terms of
CPU resource and time).

> threw a search engine at ‘yocto bitbake CI’ (and similar) but didn’t find
> it. I’m 110 % sure that there is a CI pipeline but it would make
> contributions, reviews, etc. so much more efficient if I as the
> (hypothetical) contributor would see the outcome of just a linter*. Maybe

There seems to be only one linter available and I cannot say how good it
is since I don't use it. I see that you're already a contributor to it
but I'll link here anyway: https://github.com/priv-kweihmann/oelint-adv

> it’s somewhere documented where the CI is hosted but then: No other project
> I contribute to needs documentation about these things. They are just there
> and obvious.
> 
> Don’t get me wrong: bitbake is awesome. I’d just love to contribute but the
> burden for collaboration here is not really up to date. So, is there a
> strategy to improve the on-boarding and workflow?
> 
> Greetings
> Marius
> 
> 
> * side-note on bitbake style/linting: It’s violating PEP8 in a couple of
> ways and TBH PEP8 is the bare minimum I would accept these days for
> collaboration on projects the dimension of bitbake. So an additional
> question: Would you be up for a PEP8 compatibility overhaul of the source
> code? I’d volunteer if I don’t have to send a patch via email ;)

Sorry to hear that sending a mail is that much of a hassle for you that
we won't get patches from you. I hope this changes in the future.

Now, to get to the bottom of the topic. You asked in another mail for
<35yo people. I'm one. I'm working only in "low-level" BSP (understand
kernel, bootloader and basic recipes in Yocto or packages in Buildroot).
All the projects I contribute to are using the mail submission
mechanism. This works great for me. You can have SIEVE filters (or
client side filters) to filter the mails that are touching a specific
area of a project or you being directly cc'ed or to'ed. I receive all my
mails in the same folder right now, meaning I can spot some interesting
patches every now and then in-between internal mails.
Submission of a new version of a patchset is easy to understand and the
status of said patchset isn't going to change overtime. It is a mail, you
don't edit a mail once it's been sent. GitHub and GitLab are updating the
merge requests every time you push/force-push to the branch used in the
merge request. How as I maintainer do I know if the new version of the
branch is ready to be reviewed and that active development/debugging is
finished?

I also don't care about differences between older and newer versions of
a branch, you're not merging a diff between versions but a patch, in its
Nth version, therefore you should take the time to re-read the patch in
its entirety or at least skim through it. I can say it is VERY common
in the kernel community to have reviews on code that wasn't changed,
just because the reviewer had time to think about the patchset between
versions or just misread it the first time they went through the code.

I use GitHub and GitLab for personal projects and Gerrit for some
projects at work. I prefer the mail based review and merging process by
far. Maybe because I'm just use to it. Maybe if you were to try
submitting patches via mail you'd get use to it?

You cannot (as a user) make backups of GitHub and GitLab issues and MRs
to have them accessible in the event that the website would shutdown or
lose data. This has happened before, see:
https://about.gitlab.com/blog/2017/02/10/postmortem-of-database-outage-of-january-31/

Though GitHub is owned by Microsoft which apparently loves Linux, and is
a sponsor of the Linux foundation which sponsors the work on Bitbake/OE
IIRC, their source code is still not publicly available. I already
complained about the use of Zoom for conferences held by the Linux
Foundation, I'm not sure a move to GitHub would be well received by
everyone. GitLab does not have this issue though.

Both GitLab and GitHub require accounts to contribute to projects. This
is a big issue to me. Everyone has a mail account, every one can get
one. You can even use temporary ones if you want to. Just as a reminder,
Github shut down access to Iranian people for some time, see
https://github.blog/2021-01-05-advancing-developer-freedom-github-is-fully-available-in-iran/
and https://github.blog/2019-09-12-global-software-collaboration-in-the-face-of-sanctions/
Also, they both have their Terms of Service you have to comply with.
They can change over time and you can also disagree with the ToS which
would then prevent you from contributing. There are plenty of mail
providers, I suppose one would be able to find one whose ToS would be
acceptable to them.

I don't think this is something we would be okay with. I do not want my
or any other people's contribution to YP/OE/Bitbake depend on what the US
decides for the rest of the world.

Also.. GitHub (and probably GitLab) have some down time, like most
website do every now and then. I've yet to see a global disruption of
the mail servers world wide that would prevent us from reviewing or
contributing. See:
https://hackaday.com/2017/02/02/fotw-gitlab-goes-down/
https://www.theverge.com/2020/6/29/21306674/github-down-errors-outage-june-2020

Also on the Linux-geeky-nerdy side, I don't need to click on links to do
reviews within my CLI-based mail client (neomutt) as opposed to web-based
ones.

GitHub and GitLab or any web-hosted solution require Internet access to
review patches, this is inconvenient at best, or an important limitation
for developing countries. I do not have numbers to back this up. I have
heard a few kernel maintainers doing reviews offline (in the plane to
conferences for example or while working remotely, off-grid), piling up
mails and sending them as batch once they do have access to Internet
again.

We could host our own instance of GitLab but compared to the current
cgit web interface and git servers, this brings additional maintenance
cost and security issues. Adding financial and time cost to a field in
which the project is already struggling to find resources.

In addition, the migration is not an easy process and takes time and
effort from many, during which the development of new features and fixes
will most likely suffer.

Having 2+ ways of contribution is a nightmare and a time-bomb just
waiting to explode. Where are the reviews done, would most reviewers
review on the 2+ media where the MR and issues are reported? I wouldn't
do it for example, not by choice particularly, just that I'll forget
about one.

Bitbake and YP/OE are successful projects. A project does not survive
for more than a decade if it's not any good or useful. It is backed by
the Linux foundation for a few years now at least. Most embedded SoC
vendors are providing some sort of Yocto-based images. Despite this
apparent barrier to contribution, the project is doing ok.

You also cannot compare meta-openembedded with Bitbake for two reasons:
as far as my understanding goes, meta-openembedded is community maintained,
it is not really part of the core of YP/OE. This means that it *could*
(I'm not saying it is) be less reviewed or of worse quality.
Second, meta-openembdded is a layer.. it has recipes, classes,
configuration files. Those are much more accessible than Bitbake. I can
write recipes, classes and configuration files with my current knowledge
as a user of YP/OE. Bitbake is what's making use of the metadata
provided by meta-openembedded, it is python-based and is what makes
YP/OE YP/OE. I for example wouldn't be able to contribute right now to
bitbake because I do not have any knowledge of it nor do I and most
users need to. A kernel analogy would be: kernel drivers vs kernel
subsystems. Yocto layers are receiving much more updates than Bitbake
do, as would be expected. You can check poky/openembedded-core too, for
example Alex Kanavin to name him regularly updates recipes in there. We
see many patches, this would be a fairer comparison.

Now, YP and Bitbake projects are struggling to find regular contributors
that is true, public and known. Lowering the barrier to contribution
should be among our goals. However there's another problem with changing
the contribution process directly... we do have regular contributors and
changing the contribution process is taking a risk that those
contributors stop contributing or contribute less jeopardizing the
project as a whole. Considering that the change also does not
necessarily nor immediately bring new, regular, long-term and
"in-quantity" contributors, it is a big risk for an already established
project.

I can hear your side of the argument, I can agree with some parts but
you also have to understand that this is a risk for the project that is
non-negligible, for just a hope that it'll be a positive change in the
short, mid and long-term.

I have to finish this mail by stating that I'm a mere contributor to
Yocto Project (and not Bitbake) so all of the above are my opinions and
my own only. I do not speak for the Yocto project, OE or Linux
foundation even though I regularly use "we" to define the YP/OE/Bitbake
community of which I am a "member".

I hope you'll review your stance on contributing via mail, we're looking
forward to receiving your patches.

N.B.: We now have a description on https://github.com/openembedded/bitbake
reflecting that no issue or pull request should be opened there.
Hopefully this makes the contributing over Github clearer now :)

On a side note, there was similar discussion about moving our IRC
channel to Matrix. We've decided for now to stay on IRC though a bridge
is provided by some Matrix servers IIRC (we do have some people using
Matrix on our IRC channel). I for example wouldn't move to Matrix
because I had a very bad experience with it for a few months which made
me stop using it entirely (and also it's hard to find the ToS of Matrix
servers and matrix.org's wasn't acceptable to me).

And finally, having a rude or harsh tone is deserving your cause, it
triggers people and does not start the discussion the correct way. As
the saying goes, if you smell your own sweat, it's been long since
others have been bothered by it. Same goes for the tone of your mail,
if you think it is harsh, it probably is more than you think. You
probably would have received better answers were the tone a bit tamer.
We'll never know :)

Have a nice day,
Cheers,
Quentin