RFC: Grub project management

RFC: Grub project management

[Glenn Washburn]

Hi fellow GRUB developers,

I want to start by recognizing that most people who are involved with
the project maintenance are doing so on a voluntary basis (or at least
that's my assumption) and people have busy lives. Its a mostly
thankless job, so thank you guys for all the time spent keeping this
essential project alive. I hope that the following will not be construed
as judging anyone, but merely taking account of where we've been, where
we are, and how to move forward.

TLDR; Check out https://gitlab.com/gnu-grub/grub and do merge requests
with changes rebased on to feature/gitlab-testing against
feature/gitlab-testing to get automatic CI testing.

I believe I speak for more than just myself when I say that the current
development process leaves much to be desired. GRUB has been in a
feature freeze since last March, with the release being pushed
rescheduled several times and now looks like its projected to be in
March (assuming its not pushed back again). I get the impression that
Daniel is overloaded with non-GRUB responsibilities. I also wonder if
part of the issue is concerned about making a release which he believes
hasn't had sufficient testing.

Daniel, could you please clarify what needs to be done to get the
release out the door and what the community can do to help expedite
this process?

In the meantime, patches are being sent to the list which will not be
considered for acceptance until the release is out and feature freeze
lifted. Does anyone believe that someone is going to go back to last
March and check all the submitted patches to see about including them?
I judge the answer to be no. So patches will be continually falling
into a blackhole. What we need is an issue/merge management system so
things don't get lost, or can easily be found.

GRUB currently uses the Savannah GNU project hosting site. And while I
love GNU philosophy and the role that Savannah has played for free
software projects, it is showing its age and not ideal for a modern
development process. My two big pain points is that it does not do
merge requests and the bug tracker is an unpleasant experience when
compared to modern ones. If you've ever tried to search for a bug,
you'll know what I mean. So unfortunately, I think its time to say
goodbye Savannah.

Another issue, as I see it, is the testing. The only testing that I'm
aware of is the travis CI, which is only a build test, and Adrian's
testing for debian, which does do the make check tests but is missing
filesystem testing and qemu tests for most architectures. Ideally, the
make check tests should be done within travis, but we're not there yet.
This process feels closed and non-transparent to me, which I find
undesirable in general, but especially so for a free software project
which is the corner-stone of the linux community. And I'm not saying
that that is intentional on the part of those working on those tests.
So, for one, it would be nice to have publicly accessible the status of
Travis runs. And may be it is, but why isn't it obvious?

Based on what I've gathered from the list and private conversations
(which isn't much), the travis is only really run as a prep for making
a release. If that's true, the process fails to use the main point of a
continuous integration system. I would like GRUB to get to a place
where every commit is run through CI (at least the build test). In the
short term, it would be nice to kick off a CI job every time master
changes.

It would be even more nice if we could integrate the merge tracking
system with the CI system so that merge requests automatically kick off
a CI job to see if the proposed changes pass both the build test and
functional make check tests.

I believe, and my sincere hope, is that by resolving these issues we
can speed up the development cycle. I have a lot of changes that I've
been waiting months to submit to the list. I don't think it should take
a mountain of patience to get changes accepted. The GRUB project and
its users are worse off for it.

It is in consideration of all the above that I've setup a system on
GitLab that addresses all these concerns and more. I also believe that
GitLab was founded as a more ethical company that would be
more compatible with a GNU software project than most other project
hosting sites. I chose to use GitLab's builtin CI because it has what
appears to be unlimited free resources (although resource constrained
to an extent) for open source accounts.

I've setup an unofficial GNU GRUB group and project repository at
https://gitlab.com/gnu-grub/grub. It is currently configured to mirror
the official repository. I have created a few merge requests that are
real examples to show what the process would look like.

The most important merge request is the one adding the GitLab CI
feature (https://gitlab.com/gnu-grub/grub/-/merge_requests/2). This is
a merge request created from a branch in the repo, as opposed to one
from a fork. Because it has the .gitlab-ci.yml file configured to run
the CI on merge requests, a "Detached merge request pipeline" is kicked
off. The results of that can be seen on the merge request page with a
link to the pipeline and jobs so that failing jobs can be quickly found
and debugged. The last job in the pipeline outputs a summary of
architectures for which "make check" tests were run and different kinds
of test failures, for a quick overview. Importantly, the GitLab CI
config runs "make check" tests successfully for 8 targets: arm-efi,
arm64-efi, i386-efi, i386-pc, i386-qemu, powerpc-ieee1275,
sparc64-ieee1275, and x86_64-efi.

Another merge request is titled "Add basic support for xHCI USB
controllers and non root xHCI hubs." As noted in the description, this
is a patch series I took off the list authored by Patrick Rudolph. The
merge request pipeline fails for the x86_64-efi build, but all the
others succeed. I suspect that the gcc being used (10.1.0) is newer than
the one developed on and this gcc may be grumpier. These errors can be
fixed, the pushed to the same branch and another CI job will be started
to test the updated merge request automatically.

Until the GitLab CI feature is merge into master, merge requests should
be rebased onto the feature/gitlab-testing branch and merged requested
against that branch (not master). This is needed so that the merge
request does not show the GitLab CI commits as part of the merge
request. Once the GitLab CI feature has been merged into master, merge
requests can be off master and against master, as one normally would.
One can see that this has been done for all the current merge requests.

For patches that people don't want to disappear on the list, I think a
merge request can help mitigate that. Also since the merge request is
effectively a whole commit tree, instead of some free floating diffs,
we can know precisely which commit its based off of (master? or last
stable?).

This is not intended to change the current patch submission
requirements for the project. Patches will still need to be sent to the
list and will be reviewed on the list. I think for non-trivial patches
it should be required to make a merge request as well so that the CI
passing is a requirement. I hope that by enforcing a passing CI that
maintainer and reviewer time is saved by ensuring that patch series
pass the sniff test.

This is currently not officially part of the GRUB project, so creating
issues, for instance, should still be done on Savannah. I'll be sending
out a patch series to the list soon with the changes used to integrate
GRUB into GitLab's CI.

If you've made it this far, congratulations! I hope we can put this to
good use and help speed up GRUB's development cycle.

Glenn

Re: RFC: Grub project management

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 8641 bytes --]

On 2/11/21 10:53 PM, Glenn Washburn wrote:
> Hi fellow GRUB developers,
> 
> I want to start by recognizing that most people who are involved with
> the project maintenance are doing so on a voluntary basis (or at least
> that's my assumption) and people have busy lives. Its a mostly
> thankless job, so thank you guys for all the time spent keeping this
> essential project alive. I hope that the following will not be construed
> as judging anyone, but merely taking account of where we've been, where
> we are, and how to move forward.
> 
> TLDR; Check out https://gitlab.com/gnu-grub/grub and do merge requests
> with changes rebased on to feature/gitlab-testing against
> feature/gitlab-testing to get automatic CI testing.
> 
> I believe I speak for more than just myself when I say that the current
> development process leaves much to be desired. GRUB has been in a
> feature freeze since last March, with the release being pushed
> rescheduled several times and now looks like its projected to be in
> March (assuming its not pushed back again). I get the impression that
> Daniel is overloaded with non-GRUB responsibilities. I also wonder if
> part of the issue is concerned about making a release which he believes
> hasn't had sufficient testing.
> 
> Daniel, could you please clarify what needs to be done to get the
> release out the door and what the community can do to help expedite
> this process?
> 
> In the meantime, patches are being sent to the list which will not be
> considered for acceptance until the release is out and feature freeze
> lifted. Does anyone believe that someone is going to go back to last
> March and check all the submitted patches to see about including them?
> I judge the answer to be no. So patches will be continually falling
> into a blackhole. What we need is an issue/merge management system so
> things don't get lost, or can easily be found.
> 
> GRUB currently uses the Savannah GNU project hosting site. And while I
> love GNU philosophy and the role that Savannah has played for free
> software projects, it is showing its age and not ideal for a modern
> development process. My two big pain points is that it does not do
> merge requests and the bug tracker is an unpleasant experience when
> compared to modern ones. If you've ever tried to search for a bug,
> you'll know what I mean. So unfortunately, I think its time to say
> goodbye Savannah.
> 
> Another issue, as I see it, is the testing. The only testing that I'm
> aware of is the travis CI, which is only a build test, and Adrian's
> testing for debian, which does do the make check tests but is missing
> filesystem testing and qemu tests for most architectures. Ideally, the
> make check tests should be done within travis, but we're not there yet.
> This process feels closed and non-transparent to me, which I find
> undesirable in general, but especially so for a free software project
> which is the corner-stone of the linux community. And I'm not saying
> that that is intentional on the part of those working on those tests.
> So, for one, it would be nice to have publicly accessible the status of
> Travis runs. And may be it is, but why isn't it obvious?
> 
> Based on what I've gathered from the list and private conversations
> (which isn't much), the travis is only really run as a prep for making
> a release. If that's true, the process fails to use the main point of a
> continuous integration system. I would like GRUB to get to a place
> where every commit is run through CI (at least the build test). In the
> short term, it would be nice to kick off a CI job every time master
> changes.
> 
> It would be even more nice if we could integrate the merge tracking
> system with the CI system so that merge requests automatically kick off
> a CI job to see if the proposed changes pass both the build test and
> functional make check tests.
> 
> I believe, and my sincere hope, is that by resolving these issues we
> can speed up the development cycle. I have a lot of changes that I've
> been waiting months to submit to the list. I don't think it should take
> a mountain of patience to get changes accepted. The GRUB project and
> its users are worse off for it.
> 
> It is in consideration of all the above that I've setup a system on
> GitLab that addresses all these concerns and more. I also believe that
> GitLab was founded as a more ethical company that would be
> more compatible with a GNU software project than most other project
> hosting sites. I chose to use GitLab's builtin CI because it has what
> appears to be unlimited free resources (although resource constrained
> to an extent) for open source accounts.
> 
> I've setup an unofficial GNU GRUB group and project repository at
> https://gitlab.com/gnu-grub/grub. It is currently configured to mirror
> the official repository. I have created a few merge requests that are
> real examples to show what the process would look like.
> 
> The most important merge request is the one adding the GitLab CI
> feature (https://gitlab.com/gnu-grub/grub/-/merge_requests/2). This is
> a merge request created from a branch in the repo, as opposed to one
> from a fork. Because it has the .gitlab-ci.yml file configured to run
> the CI on merge requests, a "Detached merge request pipeline" is kicked
> off. The results of that can be seen on the merge request page with a
> link to the pipeline and jobs so that failing jobs can be quickly found
> and debugged. The last job in the pipeline outputs a summary of
> architectures for which "make check" tests were run and different kinds
> of test failures, for a quick overview. Importantly, the GitLab CI
> config runs "make check" tests successfully for 8 targets: arm-efi,
> arm64-efi, i386-efi, i386-pc, i386-qemu, powerpc-ieee1275,
> sparc64-ieee1275, and x86_64-efi.
> 
> Another merge request is titled "Add basic support for xHCI USB
> controllers and non root xHCI hubs." As noted in the description, this
> is a patch series I took off the list authored by Patrick Rudolph. The
> merge request pipeline fails for the x86_64-efi build, but all the
> others succeed. I suspect that the gcc being used (10.1.0) is newer than
> the one developed on and this gcc may be grumpier. These errors can be
> fixed, the pushed to the same branch and another CI job will be started
> to test the updated merge request automatically.
> 
> Until the GitLab CI feature is merge into master, merge requests should
> be rebased onto the feature/gitlab-testing branch and merged requested
> against that branch (not master). This is needed so that the merge
> request does not show the GitLab CI commits as part of the merge
> request. Once the GitLab CI feature has been merged into master, merge
> requests can be off master and against master, as one normally would.
> One can see that this has been done for all the current merge requests.
> 
> For patches that people don't want to disappear on the list, I think a
> merge request can help mitigate that. Also since the merge request is
> effectively a whole commit tree, instead of some free floating diffs,
> we can know precisely which commit its based off of (master? or last
> stable?).
> 
> This is not intended to change the current patch submission
> requirements for the project. Patches will still need to be sent to the
> list and will be reviewed on the list. I think for non-trivial patches
> it should be required to make a merge request as well so that the CI
> passing is a requirement. I hope that by enforcing a passing CI that
> maintainer and reviewer time is saved by ensuring that patch series
> pass the sniff test.

Sending patches to both the mailing list and a gitlab repo "for CI
purposes" seems confusingly redundant.

It would probably make more sense to use a patchwork configured to run
CI on incoming patches, and maybe even respond to the mailing list with
status reports.

e.g.

https://lists.sr.ht/~sircmpwn/sr.ht-dev/patches/20108

(Using a patchwork also prevents patches from disappearing!)

> This is currently not officially part of the GRUB project, so creating
> issues, for instance, should still be done on Savannah. I'll be sending
> out a patch series to the list soon with the changes used to integrate
> GRUB into GitLab's CI.
> 
> If you've made it this far, congratulations! I hope we can put this to
> good use and help speed up GRUB's development cycle.
> 
> Glenn


-- 
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: RFC: Grub project management

[Glenn Washburn]

On Thu, 11 Feb 2021 23:50:38 -0500
Eli Schwartz <eschwartz@archlinux.org> wrote:

> On 2/11/21 10:53 PM, Glenn Washburn wrote:
> > For patches that people don't want to disappear on the list, I
> > think a merge request can help mitigate that. Also since the merge
> > request is effectively a whole commit tree, instead of some free
> > floating diffs, we can know precisely which commit its based off of
> > (master? or last stable?).
> > 
> > This is not intended to change the current patch submission
> > requirements for the project. Patches will still need to be sent to
> > the list and will be reviewed on the list. I think for non-trivial
> > patches it should be required to make a merge request as well so
> > that the CI passing is a requirement. I hope that by enforcing a
> > passing CI that maintainer and reviewer time is saved by ensuring
> > that patch series pass the sniff test.
> 
> Sending patches to both the mailing list and a gitlab repo "for CI
> purposes" seems confusingly redundant.
> 
> It would probably make more sense to use a patchwork configured to run
> CI on incoming patches, and maybe even respond to the mailing list
> with status reports.
> 
> e.g.
> 
> https://lists.sr.ht/~sircmpwn/sr.ht-dev/patches/20108
> 
> (Using a patchwork also prevents patches from disappearing!)

There are two main parts to this work, the CI and the merge request
automation. As I see it, your issue is not with the CI, but the merge
requests. And you're right the merge request part is not ideal. There
is redundancy, but I don't think its that big of a deal. There does
come more confusion when determining what patch series version the merge
request is currently at and testing. This could be mitigated by having
the title of the merge updated when the merge branch is updated or
editing the merge request to use a new branch of the changes which has
the version number in its name. Sure a few extra steps, but I don't see
it as too much to ask. Plus, we're putting the responsibility on the
merge author and not project maintainers. And yes, someone can in bad
faith abuse the system to waste maintainer time, but they'll quickly be
rooted out.

Also, I was looking for a solution that didn't require me hosting
anything and I don't know of a free hosted patchwork service. It looks
like sourcehut fits the bill. Now I'm curious if there's a reason that
GRUB isn't already using that service, even if unofficially. Perhaps,
I'll look in to switching to that service.

I think a lot of the work done in my GitLab CI changes could be reused
for other CI systems or we can use just the CI part of these changes.
That GitLab repo should be setup already to trigger a CI pipeline
whenever master changes (but only once the .gitlab-ci.yml file is in
master).

Glenn

Re: RFC: Grub project management

[Konrad Rzeszutek Wilk]

On Thu, Feb 11, 2021 at 09:53:37PM -0600, Glenn Washburn wrote:
> Hi fellow GRUB developers,
> 
> I want to start by recognizing that most people who are involved with
> the project maintenance are doing so on a voluntary basis (or at least
> that's my assumption) and people have busy lives. Its a mostly
> thankless job, so thank you guys for all the time spent keeping this
> essential project alive. I hope that the following will not be construed
> as judging anyone, but merely taking account of where we've been, where
> we are, and how to move forward.

So .. May I take your email and break it down?
> 
> TLDR; Check out https://gitlab.com/gnu-grub/grub and do merge requests
> with changes rebased on to feature/gitlab-testing against
> feature/gitlab-testing to get automatic CI testing.
> 
> I believe I speak for more than just myself when I say that the current
> development process leaves much to be desired. GRUB has been in a
> feature freeze since last March, with the release being pushed
> rescheduled several times and now looks like its projected to be in
> March (assuming its not pushed back again). I get the impression that
> Daniel is overloaded with non-GRUB responsibilities. I also wonder if
> part of the issue is concerned about making a release which he believes
> hasn't had sufficient testing.
> 
> Daniel, could you please clarify what needs to be done to get the
> release out the door and what the community can do to help expedite
> this process?

What I am understanding is that Daniel is the bottlenck here. That is
his reviews of patches (and fixes up) are what is holding up the
acceptance?

> 
> In the meantime, patches are being sent to the list which will not be
> considered for acceptance until the release is out and feature freeze
> lifted. Does anyone believe that someone is going to go back to last
> March and check all the submitted patches to see about including them?

Daniel (from the years I have worked with him) is very detailed
person who has a nice workflow to make sure they are there.

> I judge the answer to be no. So patches will be continually falling
> into a blackhole. What we need is an issue/merge management system so

Do you have an example of a particular patchset that went into a
blackhole?

> things don't get lost, or can easily be found.

I would assume that folks who care about their patches would repost once
the release goes out?

Is this why the GitLab came about - as a way to keep track of patches
that need review?

> 
> GRUB currently uses the Savannah GNU project hosting site. And while I
> love GNU philosophy and the role that Savannah has played for free
> software projects, it is showing its age and not ideal for a modern
> development process. My two big pain points is that it does not do
> merge requests and the bug tracker is an unpleasant experience when
> compared to modern ones. If you've ever tried to search for a bug,
> you'll know what I mean. So unfortunately, I think its time to say

It is a feature. We got no bugs :-)

> goodbye Savannah.

But the merge requests are not the pain points. It is reviewing the
patches I would think?
> 
> Another issue, as I see it, is the testing. The only testing that I'm
> aware of is the travis CI, which is only a build test, and Adrian's
> testing for debian, which does do the make check tests but is missing
> filesystem testing and qemu tests for most architectures. Ideally, the
> make check tests should be done within travis, but we're not there yet.
> This process feels closed and non-transparent to me, which I find
> undesirable in general, but especially so for a free software project
> which is the corner-stone of the linux community. And I'm not saying
> that that is intentional on the part of those working on those tests.
> So, for one, it would be nice to have publicly accessible the status of
> Travis runs. And may be it is, but why isn't it obvious?

As you point out the Travis CI is for build tests which is not that
exciting. I think what you are really saying is that you want better
tests?
> 
> Based on what I've gathered from the list and private conversations
> (which isn't much), the travis is only really run as a prep for making
> a release. If that's true, the process fails to use the main point of a
> continuous integration system. I would like GRUB to get to a place
> where every commit is run through CI (at least the build test). In the
> short term, it would be nice to kick off a CI job every time master
> changes.

So I agree it is nice, but I am missing how this is going to help
in getting a release out faster?

> 
> It would be even more nice if we could integrate the merge tracking
> system with the CI system so that merge requests automatically kick off
> a CI job to see if the proposed changes pass both the build test and
> functional make check tests.

Ah, so it fixes the simple compile issues patches may have!
> 
> I believe, and my sincere hope, is that by resolving these issues we
> can speed up the development cycle. I have a lot of changes that I've
> been waiting months to submit to the list. I don't think it should take
> a mountain of patience to get changes accepted. The GRUB project and
> its users are worse off for it.
> 
> It is in consideration of all the above that I've setup a system on
> GitLab that addresses all these concerns and more. I also believe that
> GitLab was founded as a more ethical company that would be
> more compatible with a GNU software project than most other project
> hosting sites. I chose to use GitLab's builtin CI because it has what
> appears to be unlimited free resources (although resource constrained
> to an extent) for open source accounts.
> 
> I've setup an unofficial GNU GRUB group and project repository at
> https://gitlab.com/gnu-grub/grub. It is currently configured to mirror
> the official repository. I have created a few merge requests that are
> real examples to show what the process would look like.
> 
> The most important merge request is the one adding the GitLab CI
> feature (https://gitlab.com/gnu-grub/grub/-/merge_requests/2). This is
> a merge request created from a branch in the repo, as opposed to one
> from a fork. Because it has the .gitlab-ci.yml file configured to run
> the CI on merge requests, a "Detached merge request pipeline" is kicked
> off. The results of that can be seen on the merge request page with a
> link to the pipeline and jobs so that failing jobs can be quickly found
> and debugged. The last job in the pipeline outputs a summary of
> architectures for which "make check" tests were run and different kinds
> of test failures, for a quick overview. Importantly, the GitLab CI
> config runs "make check" tests successfully for 8 targets: arm-efi,
> arm64-efi, i386-efi, i386-pc, i386-qemu, powerpc-ieee1275,
> sparc64-ieee1275, and x86_64-efi.

That is a nice list of targets!
> 
> Another merge request is titled "Add basic support for xHCI USB
> controllers and non root xHCI hubs." As noted in the description, this
> is a patch series I took off the list authored by Patrick Rudolph. The
> merge request pipeline fails for the x86_64-efi build, but all the
> others succeed. I suspect that the gcc being used (10.1.0) is newer than
> the one developed on and this gcc may be grumpier. These errors can be
> fixed, the pushed to the same branch and another CI job will be started
> to test the updated merge request automatically.
> 
> Until the GitLab CI feature is merge into master, merge requests should
> be rebased onto the feature/gitlab-testing branch and merged requested
> against that branch (not master). This is needed so that the merge
> request does not show the GitLab CI commits as part of the merge
> request. Once the GitLab CI feature has been merged into master, merge
> requests can be off master and against master, as one normally would.
> One can see that this has been done for all the current merge requests.
> 
> For patches that people don't want to disappear on the list, I think a
> merge request can help mitigate that. Also since the merge request is
> effectively a whole commit tree, instead of some free floating diffs,
> we can know precisely which commit its based off of (master? or last
> stable?).
> 
> This is not intended to change the current patch submission
> requirements for the project. Patches will still need to be sent to the
> list and will be reviewed on the list. I think for non-trivial patches
> it should be required to make a merge request as well so that the CI
> passing is a requirement. I hope that by enforcing a passing CI that
> maintainer and reviewer time is saved by ensuring that patch series
> pass the sniff test.
> 
> This is currently not officially part of the GRUB project, so creating
> issues, for instance, should still be done on Savannah. I'll be sending
> out a patch series to the list soon with the changes used to integrate
> GRUB into GitLab's CI.

So the intent here is like the 0day bot on LKML? When someone posts a
patch it compiles on different platforms and tells folks what and where
things didn't compile?
> 
> If you've made it this far, congratulations! I hope we can put this to
> good use and help speed up GRUB's development cycle.
> 
> Glenn
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

Re: RFC: Grub project management

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 2977 bytes --]

On 2/12/21 4:58 PM, Glenn Washburn wrote:
> There are two main parts to this work, the CI and the merge request
> automation. As I see it, your issue is not with the CI, but the merge
> requests.

Yes -- CI is indeed wonderful to have and should be implemented if
possible (which it always is, therefore it should always be implemented).

> And you're right the merge request part is not ideal. There
> is redundancy, but I don't think its that big of a deal. There does
> come more confusion when determining what patch series version the merge
> request is currently at and testing. This could be mitigated by having
> the title of the merge updated when the merge branch is updated or
> editing the merge request to use a new branch of the changes which has
> the version number in its name. Sure a few extra steps, but I don't see
> it as too much to ask. Plus, we're putting the responsibility on the
> merge author and not project maintainers. And yes, someone can in bad
> faith abuse the system to waste maintainer time, but they'll quickly be
> rooted out.
> 
> Also, I was looking for a solution that didn't require me hosting
> anything and I don't know of a free hosted patchwork service. It looks
> like sourcehut fits the bill. Now I'm curious if there's a reason that
> GRUB isn't already using that service, even if unofficially. Perhaps,
> I'll look in to switching to that service.

https://libreplanet.org/wiki/FSF_2020_forge_evaluation

There are several interesting options for "we can do better than
savannah these days". Sourcehut is one of the software forges under review.

However as far as I know, there's no unified approach to this, so the
real question reduces back down to "why isn't there something,
regardless of what" -- and the answer is likely something along the
lines of "no one pushed for it, therefore inertia".

Aside: the evaluation is *very* critical of gitlab.com, though it
considers self-hosted gitlab CE should alleviate a bunch of the listed
concerns.

sourcehut and pagure are the likely contenders for a GNU/FSF endorsed
forge, and sourcehut is the only software forge I'm aware of *anywhere*
that considers mailing lists to be a/the core development workflow (and
therefore integrates a patchwork).

> I think a lot of the work done in my GitLab CI changes could be reused
> for other CI systems or we can use just the CI part of these changes.
> That GitLab repo should be setup already to trigger a CI pipeline
> whenever master changes (but only once the .gitlab-ci.yml file is in
> master).

Yeah, having a somewhat independent script to run CI builds is good, it
helps avoid getting locked into specific CI providers. :)

As we've seen from e.g.

https://www.theregister.com/2020/11/02/travis_ci_pricng/
https://news.ycombinator.com/item?id=25338983

Free CI is not guaranteed to stick around...

-- 
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: RFC: Grub project management

[Glenn Washburn]

On Fri, 12 Feb 2021 17:35:05 -0500
Eli Schwartz <eschwartz@archlinux.org> wrote:

> On 2/12/21 4:58 PM, Glenn Washburn wrote:
> > And you're right the merge request part is not ideal. There
> > is redundancy, but I don't think its that big of a deal. There does
> > come more confusion when determining what patch series version the
> > merge request is currently at and testing. This could be mitigated
> > by having the title of the merge updated when the merge branch is
> > updated or editing the merge request to use a new branch of the
> > changes which has the version number in its name. Sure a few extra
> > steps, but I don't see it as too much to ask. Plus, we're putting
> > the responsibility on the merge author and not project maintainers.
> > And yes, someone can in bad faith abuse the system to waste
> > maintainer time, but they'll quickly be rooted out.
> > 
> > Also, I was looking for a solution that didn't require me hosting
> > anything and I don't know of a free hosted patchwork service. It
> > looks like sourcehut fits the bill. Now I'm curious if there's a
> > reason that GRUB isn't already using that service, even if
> > unofficially. Perhaps, I'll look in to switching to that service.
> 
> https://libreplanet.org/wiki/FSF_2020_forge_evaluation
> 
> There are several interesting options for "we can do better than
> savannah these days". Sourcehut is one of the software forges under
> review.
> 
> However as far as I know, there's no unified approach to this, so the
> real question reduces back down to "why isn't there something,
> regardless of what" -- and the answer is likely something along the
> lines of "no one pushed for it, therefore inertia".

I poked around on a sourcehut account a little and its not obvious that
can can integrate with an existing mailing list. It looks like it
expects to host the mailinglist. I'm thinking there might be a way to
configure it act like a peer mailinglist, where it would receive
email from grub-devel and emails it received would be sent to
grub-devel.

Reading more about patchwork, it seems to have its own set of issues,
partly revolving around using a mailing list of development as we do.
see: https://lwn.net/Articles/773456/

> Aside: the evaluation is *very* critical of gitlab.com, though it
> considers self-hosted gitlab CE should alleviate a bunch of the listed
> concerns.

That evaluation wasn't clear on where the captcha was used. I can't
recall ever seeing a captcha on their site, certainly not on a regular
basis. I wonder how hard the GNU position would be against using GitLab
in the manner I suggest (modifying my original comment about requiring
merge requests for CI, but having it optional). Most of their
complaints seem to be around javascript. Would that be alleviated by
using scripts to do things via the API?

> sourcehut and pagure are the likely contenders for a GNU/FSF endorsed
> forge, and sourcehut is the only software forge I'm aware of
> *anywhere* that considers mailing lists to be a/the core development
> workflow (and therefore integrates a patchwork).
>
> > I think a lot of the work done in my GitLab CI changes could be
> > reused for other CI systems or we can use just the CI part of these
> > changes. That GitLab repo should be setup already to trigger a CI
> > pipeline whenever master changes (but only once the .gitlab-ci.yml
> > file is in master).
> 
> Yeah, having a somewhat independent script to run CI builds is good,
> it helps avoid getting locked into specific CI providers. :)
> 
> As we've seen from e.g.
> 
> https://www.theregister.com/2020/11/02/travis_ci_pricng/
> https://news.ycombinator.com/item?id=25338983
> 
> Free CI is not guaranteed to stick around...

Nope, nothing is. But GitLab's may last longer than others that have CI
as a core business.

Re: RFC: Grub project management

[Glenn Washburn]

On Fri, 12 Feb 2021 17:16:39 -0500
Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:

> On Thu, Feb 11, 2021 at 09:53:37PM -0600, Glenn Washburn wrote:
> > I believe I speak for more than just myself when I say that the
> > current development process leaves much to be desired. GRUB has
> > been in a feature freeze since last March, with the release being
> > pushed rescheduled several times and now looks like its projected
> > to be in March (assuming its not pushed back again). I get the
> > impression that Daniel is overloaded with non-GRUB
> > responsibilities. I also wonder if part of the issue is concerned
> > about making a release which he believes hasn't had sufficient
> > testing.
> > 
> > Daniel, could you please clarify what needs to be done to get the
> > release out the door and what the community can do to help expedite
> > this process?
> 
> What I am understanding is that Daniel is the bottlenck here. That is
> his reviews of patches (and fixes up) are what is holding up the
> acceptance?

Its not clear to me exactly what the nature of the bottleneck is. I
would like some clarity on this. I'm frustrated at what appears to me
to be a lack of transparency. So I'm left guessing at what the needs
are.

> > 
> > In the meantime, patches are being sent to the list which will not
> > be considered for acceptance until the release is out and feature
> > freeze lifted. Does anyone believe that someone is going to go back
> > to last March and check all the submitted patches to see about
> > including them?
> 
> Daniel (from the years I have worked with him) is very detailed
> person who has a nice workflow to make sure they are there.

That's great to hear.

> > I judge the answer to be no. So patches will be continually falling
> > into a blackhole. What we need is an issue/merge management system
> > so
> 
> Do you have an example of a particular patchset that went into a
> blackhole?

I can't really say what went into a blackhole because I don't know
what's on Daniel's list of patches to circle back to once the feature
freeze is lifted. I can give a list of patches that I see as being more
at risk (not that I personally care about all of them, but they seem
in concept worthy of inclusion).

* [PATCH 0/4] UEFI IPv6 and DHCPv6 support
  Apparently this was posted back in 2016 and never merged either,
  since the author works for SuSE. He'll probably come back in another
  4 years if its forgotten about

* [PATCH] minix: avoid mistakenly probing ext2 filesystems
  There was no review of this patch. The author appears to be from a
  company, so perhaps he'll keep pushing.

* [PATCH v2 0/7] support >512b sector disks with old/buggy firmware
  Also not reviewed, not super active author.

* search: Support searching for partition UUID with --part-uuid
  Never replied to. Author does not appear to be backed by a company.

* [PATCH] Cryptomount support for key files and detached header
  I reviewed this patch and recommended against it being accepted as is,
  but having something like GitLab's merge requests where a merge
  request was submitted and closed, still keeps track of this for other
  users that would want to use this patch.

* [PATCH] search: Support searching for GPT partition label with
  --part-label
  Not reviewed

* [PATCH v1] fix kernel cmdline corruption
  Will this get dropped like in 2018?

* v7 for detached headers and key files
  I don't think these should be included as is, but these patches have
  been attempted to be merged in one form or another since 2015. Even
  if they weren't accepted, having a closed out merge request allows
  grub users like myself to search for detached header and find the
  closed MR and potentially use it. The barrier to entry for someone
  to search the mailing list, get the patches, and finding an upstream
  commit that they'll apply against is a lot higher.

* [PATCH 0/5] Testing improvements
  These are my patches from almost 2 months ago. They haven't been
  reviewed.

Ok, now there's a list, so these might now avoid the blackhole.

There's lots of things that I'm finding on the list of potentially
useful, unaccepted patches.

> > things don't get lost, or can easily be found.
> 
> I would assume that folks who care about their patches would repost
> once the release goes out?

Depends on what level of care you're talking about. They obviously care
about their patch or they wouldn't have taken the time to create it and
submit it to the list. What I think what you mean is "they don't care
enough to baby-sit the patch for as long as it takes for the GRUB
project to get to the point where non-critical patches are accepted
again." And how do you know that by the time these patches are gotten
back to that the authors will be in a place where they will be willing
or able to pick the patches back up? I'm concerned that I will be one
of those people. So, no I don't think that is a fair assumption.

I'll speak for myself, and say that I don't want to have to continually
check the list to see if my patches have been responded to. I think the
cryptomount detached header patches are a good example of this. I want
to move on and have closure for my patches, ideally getting them into
an acceptable form and getting them merged. It actually add unnecessary
stress into my life.

And yet, I want to contribute to the project. I don't think its good
for the GRUB project to make things harder for those who aren't
employed by Ubuntu, Oracle, SuSE, IBM, or who ever else is actually
getting paid to work on this project. That's not the spirit of Free
Software as I conceive it.

I have a few unsubmitted patches that I've been waiting for the feature
freeze to lift to send. Even if I sent them now and they got reviewed,
there's a decent chance that they'd need to be revise around the time
that they get merged, which I might need to be around for. I think I'll
start sending them and see if they get picked back up. This is a big
reason for GRUB to have a devel branch, so that patches can be merged
even in a feature freeze.

I'm a bit frustrated, because I don't know where I'll be in 3 months in
terms of being able to pick back up patches. Context switching is
costly time-wise. And I've got other non-developer things I want to be
doing.

> Is this why the GitLab came about - as a way to keep track of patches
> that need review?

No, I came up with the idea of using GitLab for its CI. It has a pretty
decent free-tier. Travis seemed a lot more limited. GitLab was a
more featureful solution. I added the automatic CI for merge requests
because it was low hanging fruit, compared to the CI system in general.
I'm not saying that we need to use GitLab for keeping track of patches,
but I do think we need SOMETHING.

> > 
> > GRUB currently uses the Savannah GNU project hosting site. And
> > while I love GNU philosophy and the role that Savannah has played
> > for free software projects, it is showing its age and not ideal for
> > a modern development process. My two big pain points is that it
> > does not do merge requests and the bug tracker is an unpleasant
> > experience when compared to modern ones. If you've ever tried to
> > search for a bug, you'll know what I mean. So unfortunately, I
> > think its time to say
> 
> It is a feature. We got no bugs :-)
> 
> > goodbye Savannah.
> 
> But the merge requests are not the pain points. It is reviewing the
> patches I would think?

Pain for whom, users or developers? A user who wants to take an
unaccepted patch and use it because it fills a need might answer that
question differently than a developer trying to get the patches
submitted. They are both pain points in my opinion. So yes reviewing
does seem to be a big pain point, which is in part why I created the CI
system.

And we need to extend the testing system too, which is why I created the
patch series titled "[CRYPTOMOUNT-TEST 0/7] Add LUKS1/2 tests for
cryptomount". I've reworked them to not depend on the patches series
adding LUKS keyfile support, and will be submitting those soon.

I also think we should be trying to add tests along-side patch
submissions. Tests for bug fixes for regression testings as well as
tests for new features. For instance, the usb xHCI patches should add
some qemu tests (probably not that hard, just modifying the uhci_test).

> > 
> > Another issue, as I see it, is the testing. The only testing that
> > I'm aware of is the travis CI, which is only a build test, and
> > Adrian's testing for debian, which does do the make check tests but
> > is missing filesystem testing and qemu tests for most
> > architectures. Ideally, the make check tests should be done within
> > travis, but we're not there yet. This process feels closed and
> > non-transparent to me, which I find undesirable in general, but
> > especially so for a free software project which is the corner-stone
> > of the linux community. And I'm not saying that that is intentional
> > on the part of those working on those tests. So, for one, it would
> > be nice to have publicly accessible the status of Travis runs. And
> > may be it is, but why isn't it obvious?
> 
> As you point out the Travis CI is for build tests which is not that
> exciting. I think what you are really saying is that you want better
> tests?

The Travis CI runs one "test", which isn't much of a test. It just sees
if mkimage can build an image. We have a boatload of tests that can be
run by "make check". My CI setup does that, and it can be improved. And
yes, I do want better tests, as mentioned above.

> > 
> > Based on what I've gathered from the list and private conversations
> > (which isn't much), the travis is only really run as a prep for
> > making a release. If that's true, the process fails to use the main
> > point of a continuous integration system. I would like GRUB to get
> > to a place where every commit is run through CI (at least the build
> > test). In the short term, it would be nice to kick off a CI job
> > every time master changes.
> 
> So I agree it is nice, but I am missing how this is going to help
> in getting a release out faster?

It might not help in getting this specific release out faster. Because
of the lack of transparency in what actually needs to be done, I don't
know. I do think that it can in general speed up the patch submission
to merge into master and release timelines in general.

> > 
> > It would be even more nice if we could integrate the merge tracking
> > system with the CI system so that merge requests automatically kick
> > off a CI job to see if the proposed changes pass both the build
> > test and functional make check tests.
> 
> Ah, so it fixes the simple compile issues patches may have!

Nope, it doesn't fix anything, though I'm waiting for Google come out
with an AI that can. It'll make our lives much easier.

Joking aside, yes it will detect simple compile issues, just as the
current travis CI will do (though as far as I know its not used for
that), but its way better. What my GitLab CI config does is run the
"make check" functional tests, many of which use qemu for various
architectures to actually test that a built GRUB image can be booted
from. For instance, the sparc64-ieee1275 target builds fine, but is
failing several functional tests, like gzcompress_test. This may be an
actual bug, or a bug in the test. I don't know until we figure out why
its failing.

And of course, we get to find out more immediately when regressions
happen if we're run the CI on every patch series.

> > 
> > I believe, and my sincere hope, is that by resolving these issues we
> > can speed up the development cycle. I have a lot of changes that
> > I've been waiting months to submit to the list. I don't think it
> > should take a mountain of patience to get changes accepted. The
> > GRUB project and its users are worse off for it.
> > 
> > It is in consideration of all the above that I've setup a system on
> > GitLab that addresses all these concerns and more. I also believe
> > that GitLab was founded as a more ethical company that would be
> > more compatible with a GNU software project than most other project
> > hosting sites. I chose to use GitLab's builtin CI because it has
> > what appears to be unlimited free resources (although resource
> > constrained to an extent) for open source accounts.
> > 
> > I've setup an unofficial GNU GRUB group and project repository at
> > https://gitlab.com/gnu-grub/grub. It is currently configured to
> > mirror the official repository. I have created a few merge requests
> > that are real examples to show what the process would look like.
> > 
> > The most important merge request is the one adding the GitLab CI
> > feature (https://gitlab.com/gnu-grub/grub/-/merge_requests/2). This
> > is a merge request created from a branch in the repo, as opposed to
> > one from a fork. Because it has the .gitlab-ci.yml file configured
> > to run the CI on merge requests, a "Detached merge request
> > pipeline" is kicked off. The results of that can be seen on the
> > merge request page with a link to the pipeline and jobs so that
> > failing jobs can be quickly found and debugged. The last job in the
> > pipeline outputs a summary of architectures for which "make check"
> > tests were run and different kinds of test failures, for a quick
> > overview. Importantly, the GitLab CI config runs "make check" tests
> > successfully for 8 targets: arm-efi, arm64-efi, i386-efi, i386-pc,
> > i386-qemu, powerpc-ieee1275, sparc64-ieee1275, and x86_64-efi.
> 
> That is a nice list of targets!

And I'd like to get the mips and risc ones added. I could use some help
in getting past some blocks. Also I should add that by "successfully",
I mean passing tests excluding certain known failing tests. Those are
fairly clearly defined in the CI config file.

> > 
> > Another merge request is titled "Add basic support for xHCI USB
> > controllers and non root xHCI hubs." As noted in the description,
> > this is a patch series I took off the list authored by Patrick
> > Rudolph. The merge request pipeline fails for the x86_64-efi build,
> > but all the others succeed. I suspect that the gcc being used
> > (10.1.0) is newer than the one developed on and this gcc may be
> > grumpier. These errors can be fixed, the pushed to the same branch
> > and another CI job will be started to test the updated merge
> > request automatically.
> > 
> > Until the GitLab CI feature is merge into master, merge requests
> > should be rebased onto the feature/gitlab-testing branch and merged
> > requested against that branch (not master). This is needed so that
> > the merge request does not show the GitLab CI commits as part of
> > the merge request. Once the GitLab CI feature has been merged into
> > master, merge requests can be off master and against master, as one
> > normally would. One can see that this has been done for all the
> > current merge requests.
> > 
> > For patches that people don't want to disappear on the list, I
> > think a merge request can help mitigate that. Also since the merge
> > request is effectively a whole commit tree, instead of some free
> > floating diffs, we can know precisely which commit its based off of
> > (master? or last stable?).
> > 
> > This is not intended to change the current patch submission
> > requirements for the project. Patches will still need to be sent to
> > the list and will be reviewed on the list. I think for non-trivial
> > patches it should be required to make a merge request as well so
> > that the CI passing is a requirement. I hope that by enforcing a
> > passing CI that maintainer and reviewer time is saved by ensuring
> > that patch series pass the sniff test.
> > 
> > This is currently not officially part of the GRUB project, so
> > creating issues, for instance, should still be done on Savannah.
> > I'll be sending out a patch series to the list soon with the
> > changes used to integrate GRUB into GitLab's CI.
> 
> So the intent here is like the 0day bot on LKML? When someone posts a
> patch it compiles on different platforms and tells folks what and
> where things didn't compile?

I'm not super familiar with the LKML or the 0day bot, but as mentioned
earlier, its more than a build test, which makes it much more useful as
a test.

The main intent is making the CI system better and more public. Perhaps
the Travis is public, but I've seen nothing about how to look at run
logs for instance. I think the merge request integrated with the CI
makes the CI even more useful, but its useful without it. Perhaps, this
makes a better case for a devel branch which is pushed to more often. 

Re: RFC: Grub project management

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 4141 bytes --]

On 2/12/21 11:00 PM, Glenn Washburn wrote:
> I poked around on a sourcehut account a little and its not obvious that
> can can integrate with an existing mailing list. It looks like it
> expects to host the mailinglist. I'm thinking there might be a way to
> configure it act like a peer mailinglist, where it would receive
> email from grub-devel and emails it received would be sent to
> grub-devel.

You can import mailing lists via mbox and switch over; I think you can
probably talk to sourcehut to figure out mirroring (I think by
subscribing a sourcehut mailing list to another mailing list)?

> Reading more about patchwork, it seems to have its own set of issues,
> partly revolving around using a mailing list of development as we do.
> see: https://lwn.net/Articles/773456/

No, that's not "partly". That's just an argument against mailing lists.
They note that the ozlabs patchwork project does not know if a patch is
accepted or rejected, which is actually false -- you just need to update
the status separately. The sourcehut lists patchwork-equivalent supports
this, but also supports sending an X-Sourcehut-Patchset-Update header
when replying to the mailing list, to update a patch status (e.g.
"thanks, pulled"), and intends to eventually support automatically
detecting patches applied in git and mark them as applied.

"It is also hard to keep Patchwork in sync with multiple inboxes, so it
is not well suited to group maintainership."

I don't... even understand this sentence, so I couldn't judge its accuracy.

"Because of the mailing-list-based workflow, more spam has to be created
to give that feedback on the build and test status of patches. Patchwork
does show CI status but it is not necessarily obvious since it is also
buried in results for patches that are not relevant for various reasons."

I didn't realize for CI to give feedback via mail counts as "spam".
Aside for that, it's literally repeating the "patchwork bad because no
'patch is accepted' status". Which I've already pointed out is dead
wrong. Patchwork automatically filters out and default-hides patches
marked as applied, so you don't see them. A well-maintained patchwork
only shows pending patches (and their CI status too!) -- you just have
to, well, maintain it.

Overall, the ONLY conclusion I can come up with from your link is
"mailing lists are bad for development because patches via email are
distasteful". Which is a valid opinion for one to hold -- not everyone
likes mailing lists, not everyone *needs* to like mailing lists -- but
it's not really a useful opinion to try convincing people who like
mailing lists, to ditch them.

>> Aside: the evaluation is *very* critical of gitlab.com, though it
>> considers self-hosted gitlab CE should alleviate a bunch of the listed
>> concerns.
> 
> That evaluation wasn't clear on where the captcha was used. I can't
> recall ever seeing a captcha on their site, certainly not on a regular
> basis. I wonder how hard the GNU position would be against using GitLab
> in the manner I suggest (modifying my original comment about requiring
> merge requests for CI, but having it optional). Most of their
> complaints seem to be around javascript. Would that be alleviated by
> using scripts to do things via the API?

I doubt it. https://www.gnu.org/software/repo-criteria.html is rather
clear that requiring javascript for the site to work well is inherently
bad even if it's LibreJS-compliant (but is somewhat neutral on the topic
of whether optional, libre javascript for people who unfathomably *like*
it is evil)... and saying "well, we will never ever visit the site, only
use scripts to interact via the API" is quite missing the point, I'd
say. Do you expect people looking for CI status for patches to not visit
gitlab.com, only run a script to curl CI status over the API?

I'm not personally familiar with the captcha argument, and honestly, my
personal standards on multiple repo-criteria counts are lower than the
GNU or FSF projects. :)

-- 
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: RFC: Grub project management

[Daniel Axtens]

> Reading more about patchwork, it seems to have its own set of issues,
> partly revolving around using a mailing list of development as we do.
> see: https://lwn.net/Articles/773456/

I'm a patchwork maintainer, happy to discuss how Patchwork might be
helpful. It certainly isn't perfect (and alternatives like patchew and
the freedesktop.org fork of patchwork should be seriously considered)
but it certainly could be part of a solution.

It doesn't directly host CI, but provides some API access that can be
used to drive CI. The snowpatch project provides a link to Jenkins,
although Jenkins comes with its own set of problems.

A nice thing about all of upstream patchwork/FDO patchwork/patchew is
that they can all be bolted on to the existing ML and be used as much or
as little as desired.

Another option, and something I have been considering but so far have
lacked the time to do, is to set up something like one of the various
bots that lurks on the linux kernel mailing lists and builds and tests
patches without a web interface - just posting the results back to the
ML as a reply. This could be done without any buy-in from maintainers.

Kind regards,
Daniel

>
>> Aside: the evaluation is *very* critical of gitlab.com, though it
>> considers self-hosted gitlab CE should alleviate a bunch of the listed
>> concerns.
>
> That evaluation wasn't clear on where the captcha was used. I can't
> recall ever seeing a captcha on their site, certainly not on a regular
> basis. I wonder how hard the GNU position would be against using GitLab
> in the manner I suggest (modifying my original comment about requiring
> merge requests for CI, but having it optional). Most of their
> complaints seem to be around javascript. Would that be alleviated by
> using scripts to do things via the API?
>
>> sourcehut and pagure are the likely contenders for a GNU/FSF endorsed
>> forge, and sourcehut is the only software forge I'm aware of
>> *anywhere* that considers mailing lists to be a/the core development
>> workflow (and therefore integrates a patchwork).
>>
>> > I think a lot of the work done in my GitLab CI changes could be
>> > reused for other CI systems or we can use just the CI part of these
>> > changes. That GitLab repo should be setup already to trigger a CI
>> > pipeline whenever master changes (but only once the .gitlab-ci.yml
>> > file is in master).
>> 
>> Yeah, having a somewhat independent script to run CI builds is good,
>> it helps avoid getting locked into specific CI providers. :)
>> 
>> As we've seen from e.g.
>> 
>> https://www.theregister.com/2020/11/02/travis_ci_pricng/
>> https://news.ycombinator.com/item?id=25338983
>> 
>> Free CI is not guaranteed to stick around...
>
> Nope, nothing is. But GitLab's may last longer than others that have CI
> as a core business.
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

Re: RFC: Grub project management

[James Bottomley]

On Thu, 2021-02-11 at 23:50 -0500, Eli Schwartz wrote:
[...]
> It would probably make more sense to use a patchwork configured to
> run CI on incoming patches, and maybe even respond to the mailing
> list with status reports.
> 
> e.g.
> 
> https://lists.sr.ht/~sircmpwn/sr.ht-dev/patches/20108
> 
> (Using a patchwork also prevents patches from disappearing!)

QEMU uses patchew:

https://github.com/patchew-project/patchew

It collects the patch set, merges into current and runs CI testing.  It
can be set up to email the test results as a reply, so it saves a lot
of review time.  It's got quite a nice mailing list interface.  You can
see what it shows with one of my QEMU patch sets:

https://patchew.org/QEMU/20210204193939.16617-1-jejb@linux.ibm.com/

You see the test results and also a diff of what changed from v2 of the
submit.

It seems to work with external mailing lists and any external git repo.
It's not perfect, but it seems to work for a variety of projects as you
can see by www.patchew.org

James

Re: RFC: Grub project management

[Glenn Washburn]

On Sun, 14 Feb 2021 13:58:40 +1100
Daniel Axtens <dja@axtens.net> wrote:

> > Reading more about patchwork, it seems to have its own set of
> > issues, partly revolving around using a mailing list of development
> > as we do. see: https://lwn.net/Articles/773456/
> 
> I'm a patchwork maintainer, happy to discuss how Patchwork might be
> helpful. It certainly isn't perfect (and alternatives like patchew and
> the freedesktop.org fork of patchwork should be seriously considered)
> but it certainly could be part of a solution.

That's great. I'm thinking it would be better to use somebody else's
infrastructure instead of maintaining our own, like maintaining our own
patchwork server. I'm a little surprised that there's not a centralized
multiproject patchwork server with lots of projects that use MLs for
sending patches. Or have I just not seen it?

Although, having a patchwork maintainer in the community might make
maintaining our own infrastructure less costly.

> It doesn't directly host CI, but provides some API access that can be
> used to drive CI. The snowpatch project provides a link to Jenkins,
> although Jenkins comes with its own set of problems.
>
> A nice thing about all of upstream patchwork/FDO patchwork/patchew is
> that they can all be bolted on to the existing ML and be used as much
> or as little as desired.

That's great, but we're still maintaining the infrastructure. But maybe
we've got people in our community that are willing to contribute those
resources.

> Another option, and something I have been considering but so far have
> lacked the time to do, is to set up something like one of the various
> bots that lurks on the linux kernel mailing lists and builds and tests
> patches without a web interface - just posting the results back to the
> ML as a reply. This could be done without any buy-in from maintainers.

I think having automated building/testing of patches would be great.
And the bot could reply to the first email in the patch thread with
success/failure and some links to logs or other artifacts. I don't
really like the custom solution aspect cause that's more overall
maintenance. This is why I like the potential of the sourcehut
suggestion.

Glenn

Re: RFC: Grub project management

[Daniel Axtens]

Glenn Washburn <development@efficientek.com> writes:

> On Sun, 14 Feb 2021 13:58:40 +1100
> Daniel Axtens <dja@axtens.net> wrote:
>
>> > Reading more about patchwork, it seems to have its own set of
>> > issues, partly revolving around using a mailing list of development
>> > as we do. see: https://lwn.net/Articles/773456/
>> 
>> I'm a patchwork maintainer, happy to discuss how Patchwork might be
>> helpful. It certainly isn't perfect (and alternatives like patchew and
>> the freedesktop.org fork of patchwork should be seriously considered)
>> but it certainly could be part of a solution.
>
> That's great. I'm thinking it would be better to use somebody else's
> infrastructure instead of maintaining our own, like maintaining our own
> patchwork server. I'm a little surprised that there's not a centralized
> multiproject patchwork server with lots of projects that use MLs for
> sending patches. Or have I just not seen it?

patchwork.kernel.org
patchwork.ozlabs.org
would be the two big deployments


> Although, having a patchwork maintainer in the community might make
> maintaining our own infrastructure less costly.

Not as much as you might thing, unfortunately.

>> It doesn't directly host CI, but provides some API access that can be
>> used to drive CI. The snowpatch project provides a link to Jenkins,
>> although Jenkins comes with its own set of problems.
>>
>> A nice thing about all of upstream patchwork/FDO patchwork/patchew is
>> that they can all be bolted on to the existing ML and be used as much
>> or as little as desired.
>
> That's great, but we're still maintaining the infrastructure. But maybe
> we've got people in our community that are willing to contribute those
> resources.

Neither of the multiproject patchworks have builtin CI infrastructure,
unfortunately.


>> Another option, and something I have been considering but so far have
>> lacked the time to do, is to set up something like one of the various
>> bots that lurks on the linux kernel mailing lists and builds and tests
>> patches without a web interface - just posting the results back to the
>> ML as a reply. This could be done without any buy-in from maintainers.
>
> I think having automated building/testing of patches would be great.
> And the bot could reply to the first email in the patch thread with
> success/failure and some links to logs or other artifacts. I don't
> really like the custom solution aspect cause that's more overall
> maintenance. This is why I like the potential of the sourcehut
> suggestion.

Fair.

Kind regards,
Daniel

>
> Glenn