Kernel development collaboration platform wish list

Kernel development collaboration platform wish list

[Rafael J. Wysocki]

Hi All,

During the Maintainers Summit session yesterday I started to create a wish list
for the new kernel development collaboration platform to be created (and to
replace the multiple pieces of tooling in use today).  I also asked Bjorn,
Jiri, Greg and Shuah for input and here's the reslut:

1. Compatible with e-mail

  (a) E-mail send to it stored and included automatically; appears as part of
      the normal flow.

  (b) Automatic e-mail responses
      If e-mail is sent to it, the sender will get all responses to it in the
      given thread by e-mail.

2. History tracking

  (a) Should be able to track revisions of a given patch series (or patch) down
      to the initial submission.

3. Integration with git

  (a) Should be able to create git commits from patches (or patch series)
      tracked by it if pointed to a git branch (either locally or remotely).

  (b) Link tags pointing back to it should be added automatically to git
      commits created from patches tracked by it.

4. Distributed

  (a) Support for running offline.

  (b) Support for batch updates.

  (c) CL-frendly.

5. Patchwork-like features

  (a) Delegation support.

  (b) Support for bundles and patch series manipulation.

  (c) Smart mbox (download all selected patches).

6. Easy to set up (especially for local installations)

7. Bug tracking support

I guess there are more items to be added to this list, so please extend it if
you have any ideas and we'll see where this goes. :-)

Cheers,
Rafael

Re: Kernel development collaboration platform wish list

[Jani Nikula]

On Fri, 13 Sep 2019, "Rafael J. Wysocki" <rjw@rjwysocki.net> wrote:
> Hi All,
>
> During the Maintainers Summit session yesterday I started to create a wish list
> for the new kernel development collaboration platform to be created (and to
> replace the multiple pieces of tooling in use today).  I also asked Bjorn,
> Jiri, Greg and Shuah for input and here's the reslut:
>
> 1. Compatible with e-mail
>
>   (a) E-mail send to it stored and included automatically; appears as part of
>       the normal flow.
>
>   (b) Automatic e-mail responses
>       If e-mail is sent to it, the sender will get all responses to it in the
>       given thread by e-mail.
>
> 2. History tracking
>
>   (a) Should be able to track revisions of a given patch series (or patch) down
>       to the initial submission.
>
> 3. Integration with git
>
>   (a) Should be able to create git commits from patches (or patch series)
>       tracked by it if pointed to a git branch (either locally or remotely).

As I wrote in [1], I think git send-email and am are a lossy method of
transmission, and reliably regenerating commits from patches, with the
right baselines, as well as tracking various patch and patch series
versions, is very difficult.

I think we should have a git push based mechanism to contribute, which
would in turn send the patches to the right lists and maintainers for
review. Maintainers could, at their choosing, still use the emailed
patches (which would all be sent using the same pipeline, without the
typical issues) and their exact existing workflows, or switch to using
the commits from a branch directly.

There are more contributors than maintainers by several orders of
magnitude. It would make the maintainers' lives so much easier if the
patches came in the same way, every time. When we have technical issues
with patches, as maintainers, the problems rarely are in the receiving
end. IMO solving *all* the other items in this email would become much
easier if had this first.

BR,
Jani.


[1] http://lore.kernel.org/r/87lfv3w3v6.fsf@intel.com


>
>   (b) Link tags pointing back to it should be added automatically to git
>       commits created from patches tracked by it.
>
> 4. Distributed
>
>   (a) Support for running offline.
>
>   (b) Support for batch updates.
>
>   (c) CL-frendly.
>
> 5. Patchwork-like features
>
>   (a) Delegation support.
>
>   (b) Support for bundles and patch series manipulation.
>
>   (c) Smart mbox (download all selected patches).
>
> 6. Easy to set up (especially for local installations)
>
> 7. Bug tracking support
>
> I guess there are more items to be added to this list, so please extend it if
> you have any ideas and we'll see where this goes. :-)
>
> Cheers,
> Rafael
>
>
>
>

-- 
Jani Nikula, Intel Open Source Graphics Center

Re: Kernel development collaboration platform wish list

[Laurent Pinchart]

Hi Rafael,

On Fri, Sep 13, 2019 at 10:22:20AM +0200, Rafael J. Wysocki wrote:
> Hi All,
> 
> During the Maintainers Summit session yesterday I started to create a wish list
> for the new kernel development collaboration platform to be created (and to
> replace the multiple pieces of tooling in use today).  I also asked Bjorn,
> Jiri, Greg and Shuah for input and here's the reslut:
> 
> 1. Compatible with e-mail
> 
>   (a) E-mail send to it stored and included automatically; appears as part of
>       the normal flow.
> 
>   (b) Automatic e-mail responses
>       If e-mail is sent to it, the sender will get all responses to it in the
>       given thread by e-mail.
> 
> 2. History tracking
> 
>   (a) Should be able to track revisions of a given patch series (or patch) down
>       to the initial submission.
> 
> 3. Integration with git
> 
>   (a) Should be able to create git commits from patches (or patch series)
>       tracked by it if pointed to a git branch (either locally or remotely).
> 
>   (b) Link tags pointing back to it should be added automatically to git
>       commits created from patches tracked by it.

In addition to a link, should the tag message contain a copy of the
cover letter for the patch series ?

> 4. Distributed
> 
>   (a) Support for running offline.
> 
>   (b) Support for batch updates.
> 
>   (c) CL-frendly.
> 
> 5. Patchwork-like features
> 
>   (a) Delegation support.
> 
>   (b) Support for bundles and patch series manipulation.
> 
>   (c) Smart mbox (download all selected patches).
> 
> 6. Easy to set up (especially for local installations)
> 
> 7. Bug tracking support
> 
> I guess there are more items to be added to this list, so please extend it if
> you have any ideas and we'll see where this goes. :-)

-- 
Regards,

Laurent Pinchart

Re: Kernel development collaboration platform wish list

[Geert Uytterhoeven]

Hi Laurent,

On Fri, Sep 13, 2019 at 1:38 PM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
> On Fri, Sep 13, 2019 at 10:22:20AM +0200, Rafael J. Wysocki wrote:
> > 3. Integration with git
> >
> >   (b) Link tags pointing back to it should be added automatically to git
> >       commits created from patches tracked by it.
>
> In addition to a link, should the tag message contain a copy of the
> cover letter for the patch series ?

When the patch has a (lore) link tag pointing to its email submission, the
cover letter is one more click away, isn't it?

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Kernel development collaboration platform wish list

[Laurent Pinchart]

On Fri, Sep 13, 2019 at 02:02:18PM +0200, Geert Uytterhoeven wrote:
> On Fri, Sep 13, 2019 at 1:38 PM Laurent Pinchart wrote:
> > On Fri, Sep 13, 2019 at 10:22:20AM +0200, Rafael J. Wysocki wrote:
> > > 3. Integration with git
> > >
> > >   (b) Link tags pointing back to it should be added automatically to git
> > >       commits created from patches tracked by it.
> >
> > In addition to a link, should the tag message contain a copy of the
> > cover letter for the patch series ?
> 
> When the patch has a (lore) link tag pointing to its email submission, the
> cover letter is one more click away, isn't it?

It is, but it's nice to save that click if it has no cost :-) Storing
cover letters in tags makes it easy to reuse them when submitting new
versions, that's my main personal use case. Being able to read cover
letters offline is another interesting benefit of storing them in tags.

-- 
Regards,

Laurent Pinchart

Re: Kernel development collaboration platform wish list

[Geert Uytterhoeven]

Hi Laurent,

On Fri, Sep 13, 2019 at 2:15 PM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
> On Fri, Sep 13, 2019 at 02:02:18PM +0200, Geert Uytterhoeven wrote:
> > On Fri, Sep 13, 2019 at 1:38 PM Laurent Pinchart wrote:
> > > On Fri, Sep 13, 2019 at 10:22:20AM +0200, Rafael J. Wysocki wrote:
> > > > 3. Integration with git
> > > >
> > > >   (b) Link tags pointing back to it should be added automatically to git
> > > >       commits created from patches tracked by it.
> > >
> > > In addition to a link, should the tag message contain a copy of the
> > > cover letter for the patch series ?
> >
> > When the patch has a (lore) link tag pointing to its email submission, the
> > cover letter is one more click away, isn't it?
>
> It is, but it's nice to save that click if it has no cost :-) Storing
> cover letters in tags makes it easy to reuse them when submitting new
> versions, that's my main personal use case. Being able to read cover
> letters offline is another interesting benefit of storing them in tags.

Oh, you mean git tags?
I think Rafael was talking about "Link" tags at the end of patch/commit
descriptions.

If a series is merged using a pull request, the cover letter could indeed
be stored in the git tag description, and included in the merge commit
description.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Kernel development collaboration platform wish list

[Stefan Schmidt]

Hello.

Here is my Requirement For Comments (RFC).

On 13.09.19 10:22, Rafael J. Wysocki wrote:
> 
> 3. Integration with git
> 
>   (a) Should be able to create git commits from patches (or patch series)
>       tracked by it if pointed to a git branch (either locally or remotely).
> 
>   (b) Link tags pointing back to it should be added automatically to git
>       commits created from patches tracked by it.

(c) Git tag sanity checking
    Some maintainers have scripts for these, but not all. I still see
quite some mails about this coming from linux-next checking.
    1) ensure SOB by the actual author
    2) verify correct SOB/ACK/Reviewed tag usage (funny to see how often
things are manually added and gotten slightly wrong)
    3) verify Fixes: tag usage (min shasum length, one line without
break, etc)

There are likely more.

Where and how this is to be done I would leave open as this is only
requirements collection right now. I just feel that this kind of sanity
checking could be easily automated, hopefully without getting into the
way of maintainers.

regards
Stefan Schmidt

Re: Kernel development collaboration platform wish list

[Rafael J. Wysocki]

On Friday, September 13, 2019 1:13:16 PM CEST Jani Nikula wrote:
> On Fri, 13 Sep 2019, "Rafael J. Wysocki" <rjw@rjwysocki.net> wrote:
> > Hi All,
> >
> > During the Maintainers Summit session yesterday I started to create a wish list
> > for the new kernel development collaboration platform to be created (and to
> > replace the multiple pieces of tooling in use today).  I also asked Bjorn,
> > Jiri, Greg and Shuah for input and here's the reslut:
> >
> > 1. Compatible with e-mail
> >
> >   (a) E-mail send to it stored and included automatically; appears as part of
> >       the normal flow.
> >
> >   (b) Automatic e-mail responses
> >       If e-mail is sent to it, the sender will get all responses to it in the
> >       given thread by e-mail.
> >
> > 2. History tracking
> >
> >   (a) Should be able to track revisions of a given patch series (or patch) down
> >       to the initial submission.
> >
> > 3. Integration with git
> >
> >   (a) Should be able to create git commits from patches (or patch series)
> >       tracked by it if pointed to a git branch (either locally or remotely).
> 
> As I wrote in [1], I think git send-email and am are a lossy method of
> transmission, and reliably regenerating commits from patches, with the
> right baselines, as well as tracking various patch and patch series
> versions, is very difficult.

Assuming that git is used to generate patches in the first place.

Some people use different methods, like quilt, however.

> I think we should have a git push based mechanism to contribute, which
> would in turn send the patches to the right lists and maintainers for
> review.

Alternatively, one could point the change submission mechanism to a
git branch exposed from somewhere and a tree to merge it on top of.

> Maintainers could, at their choosing, still use the emailed
> patches (which would all be sent using the same pipeline, without the
> typical issues) and their exact existing workflows, or switch to using
> the commits from a branch directly.
> 
> There are more contributors than maintainers by several orders of
> magnitude. It would make the maintainers' lives so much easier if the
> patches came in the same way, every time.

I guess you mean a consistent view of all patches regardless of the source.

If so, then I agree, that should be one of the goals.

> When we have technical issues
> with patches, as maintainers, the problems rarely are in the receiving
> end. IMO solving *all* the other items in this email would become much
> easier if had this first.
> 
> BR,
> Jani.
> 
> 
> [1] http://lore.kernel.org/r/87lfv3w3v6.fsf@intel.com
> 
> 
> >
> >   (b) Link tags pointing back to it should be added automatically to git
> >       commits created from patches tracked by it.
> >
> > 4. Distributed
> >
> >   (a) Support for running offline.
> >
> >   (b) Support for batch updates.
> >
> >   (c) CL-frendly.
> >
> > 5. Patchwork-like features
> >
> >   (a) Delegation support.
> >
> >   (b) Support for bundles and patch series manipulation.
> >
> >   (c) Smart mbox (download all selected patches).
> >
> > 6. Easy to set up (especially for local installations)
> >
> > 7. Bug tracking support
> >
> > I guess there are more items to be added to this list, so please extend it if
> > you have any ideas and we'll see where this goes. :-)
> >
> > Cheers,
> > Rafael
> >
> >
> >
> >
> 
> 

Re: Kernel development collaboration platform wish list

[Rafael J. Wysocki]

On Friday, September 13, 2019 2:23:59 PM CEST Geert Uytterhoeven wrote:
> Hi Laurent,
> 
> On Fri, Sep 13, 2019 at 2:15 PM Laurent Pinchart
> <laurent.pinchart@ideasonboard.com> wrote:
> > On Fri, Sep 13, 2019 at 02:02:18PM +0200, Geert Uytterhoeven wrote:
> > > On Fri, Sep 13, 2019 at 1:38 PM Laurent Pinchart wrote:
> > > > On Fri, Sep 13, 2019 at 10:22:20AM +0200, Rafael J. Wysocki wrote:
> > > > > 3. Integration with git
> > > > >
> > > > >   (b) Link tags pointing back to it should be added automatically to git
> > > > >       commits created from patches tracked by it.
> > > >
> > > > In addition to a link, should the tag message contain a copy of the
> > > > cover letter for the patch series ?
> > >
> > > When the patch has a (lore) link tag pointing to its email submission, the
> > > cover letter is one more click away, isn't it?
> >
> > It is, but it's nice to save that click if it has no cost :-) Storing
> > cover letters in tags makes it easy to reuse them when submitting new
> > versions, that's my main personal use case. Being able to read cover
> > letters offline is another interesting benefit of storing them in tags.
> 
> Oh, you mean git tags?
> I think Rafael was talking about "Link" tags at the end of patch/commit
> descriptions.
> 
> If a series is merged using a pull request, the cover letter could indeed
> be stored in the git tag description, and included in the merge commit
> description.

Agreed.

Re: Kernel development collaboration platform wish list

[Rafael J. Wysocki]

On Friday, September 13, 2019 10:22:20 AM CEST Rafael J. Wysocki wrote:
> Hi All,
> 
> During the Maintainers Summit session yesterday I started to create a wish list
> for the new kernel development collaboration platform to be created (and to
> replace the multiple pieces of tooling in use today).  I also asked Bjorn,
> Jiri, Greg and Shuah for input and here's the reslut:
> 
> 1. Compatible with e-mail
> 
>   (a) E-mail send to it stored and included automatically; appears as part of
>       the normal flow.
> 
>   (b) Automatic e-mail responses
>       If e-mail is sent to it, the sender will get all responses to it in the
>       given thread by e-mail.
> 
> 2. History tracking
> 
>   (a) Should be able to track revisions of a given patch series (or patch) down
>       to the initial submission.
> 
> 3. Integration with git
> 
>   (a) Should be able to create git commits from patches (or patch series)
>       tracked by it if pointed to a git branch (either locally or remotely).
> 
>   (b) Link tags pointing back to it should be added automatically to git
>       commits created from patches tracked by it.
> 
> 4. Distributed
> 
>   (a) Support for running offline.
> 
>   (b) Support for batch updates.
> 
>   (c) CL-frendly.
> 
> 5. Patchwork-like features
> 
>   (a) Delegation support.
> 
>   (b) Support for bundles and patch series manipulation.
> 
>   (c) Smart mbox (download all selected patches).

And one more item from myself:

(d) Support for "maintainer views"

    That is, by default subsystem maintainers should see patches, bug reports
    etc against the code maintained by them, with the possibility to extend the
    view to also see the other submissions.

    [That kind of is the case in Patchwork today when patches sent to different
     mailing lists show up under different "projects", but the problem in there
     is that copies of one patch appear under multiple "projects" as different
     entities if sent to multiple lists.]

> 
> 6. Easy to set up (especially for local installations)
> 
> 7. Bug tracking support
> 
> I guess there are more items to be added to this list, so please extend it if
> you have any ideas and we'll see where this goes. :-)

Re: Kernel development collaboration platform wish list

[Jani Nikula]

On Tue, 17 Sep 2019, "Rafael J. Wysocki" <rjw@rjwysocki.net> wrote:
> Assuming that git is used to generate patches in the first place.
>
> Some people use different methods, like quilt, however.

Well, perhaps it should be a requirement to use git to contribute to
upstream kernel.

It doesn't prevent you from managing patches with whatever tools you
want locally, go wild, but for contributing you'd have to shove them in
a git tree.

I fully expect this to be shot down as a too radical idea. For the
kernel. It's pretty normal everywhere else. Even though git was
developed for the kernel.


BR,
Jani.


-- 
Jani Nikula, Intel Open Source Graphics Center

Re: Kernel development collaboration platform wish list

[Steven Rostedt]

On Wed, 18 Sep 2019 11:05:11 +0300
Jani Nikula <jani.nikula@intel.com> wrote:

> On Tue, 17 Sep 2019, "Rafael J. Wysocki" <rjw@rjwysocki.net> wrote:
> > Assuming that git is used to generate patches in the first place.
> >
> > Some people use different methods, like quilt, however.  
> 
> Well, perhaps it should be a requirement to use git to contribute to
> upstream kernel.
> 
> It doesn't prevent you from managing patches with whatever tools you
> want locally, go wild, but for contributing you'd have to shove them in
> a git tree.
> 
> I fully expect this to be shot down as a too radical idea. For the
> kernel. It's pretty normal everywhere else. Even though git was
> developed for the kernel.

I use a combination of quilt and git. But I would suggest that whatever
tool we come up with implements these "extras" via plugins. We should
define an abi such that if another tool is better for a workflow it can
be easily adapted.

-- Steve

Re: Kernel development collaboration platform wish list

[Theodore Y. Ts'o]

One of the things I'd like to add as a strong desire is the ability to
review patches via web if that's what users would like to do.  There
are some real benefits for web-based review.  It means that if you
need to see greater context, it's relatively easy to do this.  It also
is convenient to be able to see the conversation for a particular hunk
of code right alongside the code.

It's clear that whatever we do, it needs to be compatible with e-mail.
That's very clear.  But it would be useful if we can support both the
e-mail and web-based review.  There have some prototypes that have
been floated which shows that it is at least possible; perhaps
imperfectly, but something which provides a bidrectional gateway
between those who perfer to use e-mail and those that prefer to use a
web-based UI would be able to do it.

There will be many potential kernel contributors who will be used to
web-based UI's such as those that are available on github.  So while
remaining e-mail compatible, having some way of allow as many
operations to be done via web interfaces might help us get some newer
developers who are more comfortable to living on the web than some of
us more senior developers who remember when "gopher" was a text-based
search engine, and not a mascot for the Go programming langauge.  :-)

       	       	       	 	- Ted

Re: Kernel development collaboration platform wish list

[Eric Wong]

"Theodore Y. Ts'o" <tytso@mit.edu> wrote:
> One of the things I'd like to add as a strong desire is the ability to
> review patches via web if that's what users would like to do.  There
> are some real benefits for web-based review.  It means that if you
> need to see greater context, it's relatively easy to do this.  It also
> is convenient to be able to see the conversation for a particular hunk
> of code right alongside the code.

I've already started on that with public-inbox; it's not enabled
on lore, yet(*); but public-inbox.org/git can use "git apply"
and dfpre:/dfpost: search prefixes to reconstruct git blobs out
of emailed patches.

For example, a self-rejected patch I posted in 2016:

  https://public-inbox.org/git/20160711210243.GA1604@whir/

If you follow the link at the hunk header offset "+3,7",
it'll bring you to:

  https://public-inbox.org/git/8d27707/s/?b=http-walker.c#n4

  The bottom of that page has a a "debug log:" which tells
  how the blob is reconstructed using the email and pre-existing
  blob.

Since it can recreate blobs using patches, the next step is to
support showing "git diff" against reconstructed blobs with for
arbitrary contexts; but I got side-tracked from that earlier
this year...

(*) lore could configure coderepo associations, but that's a
    bear with the amount of repos kernel.org has...

> It's clear that whatever we do, it needs to be compatible with e-mail.
> That's very clear.  But it would be useful if we can support both the
> e-mail and web-based review.  There have some prototypes that have
> been floated which shows that it is at least possible; perhaps
> imperfectly, but something which provides a bidrectional gateway
> between those who perfer to use e-mail and those that prefer to use a
> web-based UI would be able to do it.

A hacker-oriented a webmail UI could be derived from
public-inbox and completely interoperable with other mail
servers (it would also support SMTP/IMAP, of course).

> There will be many potential kernel contributors who will be used to
> web-based UI's such as those that are available on github.  So while
> remaining e-mail compatible, having some way of allow as many
> operations to be done via web interfaces might help us get some newer
> developers who are more comfortable to living on the web than some of
> us more senior developers who remember when "gopher" was a text-based
> search engine, and not a mascot for the Go programming langauge.  :-)

Unfortunate, but yeah.  Any web-based UIs ought to include some
subtle (or maybe not-so-subtle) hints/pointers towards
cheaper-to-run-and-likely-more-powerful tools and practices.

I can't prove it, but it seems like public-inbox has been
successful at promoting the use of Message-IDs in URLs and
git-send-email :)

Re: Kernel development collaboration platform wish list

[Greg Kroah-Hartman]

On Tue, Sep 24, 2019 at 01:39:20AM +0000, Eric Wong wrote:
> I can't prove it, but it seems like public-inbox has been
> successful at promoting the use of Message-IDs in URLs and
> git-send-email :)

I can prove it, I'm using them in "Link:" tags to point to
lore.kernel.org for all patches I apply to the kernel tree now.  So
that's a few extra hundred links ever release that is happening :)

thanks,

greg k-h

Re: Kernel development collaboration platform wish list

[Paolo Bonzini]

On 17/09/19 20:40, Stefan Schmidt wrote:
> (c) Git tag sanity checking
>     Some maintainers have scripts for these, but not all. I still see
> quite some mails about this coming from linux-next checking.
>     1) ensure SOB by the actual author
>     2) verify correct SOB/ACK/Reviewed tag usage (funny to see how often
> things are manually added and gotten slightly wrong)
>     3) verify Fixes: tag usage (min shasum length, one line without
> break, etc)
> 
> There are likely more.
> 
> Where and how this is to be done I would leave open as this is only
> requirements collection right now. I just feel that this kind of sanity
> checking could be easily automated, hopefully without getting into the
> way of maintainers.

This can be done by a "checkpatch"-like test.  If a full-blown
checkpatch is undesirable (not sure why it would be, though), a special
mode that only checks headers can be added to the script.

Paolo

Re: Kernel development collaboration platform wish list

[Stefan Schmidt]

Hello Paolo.

On 24.09.19 11:43, Paolo Bonzini wrote:
> On 17/09/19 20:40, Stefan Schmidt wrote:
>> (c) Git tag sanity checking
>>      Some maintainers have scripts for these, but not all. I still see
>> quite some mails about this coming from linux-next checking.
>>      1) ensure SOB by the actual author
>>      2) verify correct SOB/ACK/Reviewed tag usage (funny to see how often
>> things are manually added and gotten slightly wrong)
>>      3) verify Fixes: tag usage (min shasum length, one line without
>> break, etc)
>>
>> There are likely more.
>>
>> Where and how this is to be done I would leave open as this is only
>> requirements collection right now. I just feel that this kind of sanity
>> checking could be easily automated, hopefully without getting into the
>> way of maintainers.
> 
> This can be done by a "checkpatch"-like test.  If a full-blown
> checkpatch is undesirable (not sure why it would be, though), a special
> mode that only checks headers can be added to the script.

Sure, I do not really mind where this would sit. It is only a tiny piece 
and also quite easy to do compared to the more complex building blocks 
this will have.

Just brought it up as I have seen it here on there on netdev where a 
missing SOB or a wrongly formatted Fixes tag needed some manual labor to 
explain it.

regards
Stefan Schmidt

Re: Kernel development collaboration platform wish list

[Geert Uytterhoeven]

On Wed, Sep 25, 2019 at 6:19 PM Theodore Y. Ts'o <tytso@mit.edu> wrote:
> One of the things I'd like to add as a strong desire is the ability to
> review patches via web if that's what users would like to do.  There
> are some real benefits for web-based review.  It means that if you
> need to see greater context, it's relatively easy to do this.  It also
> is convenient to be able to see the conversation for a particular hunk
> of code right alongside the code.
>
> It's clear that whatever we do, it needs to be compatible with e-mail.
> That's very clear.  But it would be useful if we can support both the
> e-mail and web-based review.  There have some prototypes that have
> been floated which shows that it is at least possible; perhaps
> imperfectly, but something which provides a bidrectional gateway
> between those who perfer to use e-mail and those that prefer to use a
> web-based UI would be able to do it.
>
> There will be many potential kernel contributors who will be used to
> web-based UI's such as those that are available on github.  So while
> remaining e-mail compatible, having some way of allow as many
> operations to be done via web interfaces might help us get some newer
> developers who are more comfortable to living on the web than some of
> us more senior developers who remember when "gopher" was a text-based
> search engine, and not a mascot for the Go programming langauge.  :-)

And if it's done right, it may solve (or make it harder to show) other
bad review
behavior, like:
  - Top posting review comments,
  - Quoting a 2000-line patch when commenting on a single line deep down.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Kernel development collaboration platform wish list

[Steven Rostedt]

That would be wonderful.

-- Steve

On September 26, 2019 12:18:18 PM GMT+02:00, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>On Wed, Sep 25, 2019 at 6:19 PM Theodore Y. Ts'o <tytso@mit.edu> wrote:
>> One of the things I'd like to add as a strong desire is the ability
>to
>> review patches via web if that's what users would like to do.  There
>> are some real benefits for web-based review.  It means that if you
>> need to see greater context, it's relatively easy to do this.  It
>also
>> is convenient to be able to see the conversation for a particular
>hunk
>> of code right alongside the code.
>>
>> It's clear that whatever we do, it needs to be compatible with
>e-mail.
>> That's very clear.  But it would be useful if we can support both the
>> e-mail and web-based review.  There have some prototypes that have
>> been floated which shows that it is at least possible; perhaps
>> imperfectly, but something which provides a bidrectional gateway
>> between those who perfer to use e-mail and those that prefer to use a
>> web-based UI would be able to do it.
>>
>> There will be many potential kernel contributors who will be used to
>> web-based UI's such as those that are available on github.  So while
>> remaining e-mail compatible, having some way of allow as many
>> operations to be done via web interfaces might help us get some newer
>> developers who are more comfortable to living on the web than some of
>> us more senior developers who remember when "gopher" was a text-based
>> search engine, and not a mascot for the Go programming langauge.  :-)
>
>And if it's done right, it may solve (or make it harder to show) other
>bad review
>behavior, like:
>  - Top posting review comments,
>- Quoting a 2000-line patch when commenting on a single line deep down.
>
>Gr{oetje,eeting}s,
>
>                        Geert

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity and top posting.

Re: Kernel development collaboration platform wish list

[Jani Nikula]

On Thu, 26 Sep 2019, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>   - Quoting a 2000-line patch when commenting on a single line deep down.

I've actually become almost completely immune to that because my mail
client collapses long quotes by default, thus highlighting the replies.

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Graphics Center

Re: Kernel development collaboration platform wish list

[Geert Uytterhoeven]

On Thu, Sep 26, 2019 at 3:02 PM Steven Rostedt <rostedt@goodmis.org> wrote:
> That would be wonderful.

Oh, the irony ;-)

> -- Steve
>
> On September 26, 2019 12:18:18 PM GMT+02:00, Geert Uytterhoeven <geert@linux-m68k.org> wrote:

[...]

> >And if it's done right, it may solve (or make it harder to show) other
> >bad review
> >behavior, like:
> >  - Top posting review comments,
> >- Quoting a 2000-line patch when commenting on a single line deep down.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Kernel development collaboration platform wish list

[Geert Uytterhoeven]

Hi Jani,

On Thu, Sep 26, 2019 at 3:10 PM Jani Nikula <jani.nikula@intel.com> wrote:
> On Thu, 26 Sep 2019, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> >   - Quoting a 2000-line patch when commenting on a single line deep down.
>
> I've actually become almost completely immune to that because my mail
> client collapses long quotes by default, thus highlighting the replies.

The Gmail web interface is indeed good at that... Until you want to rebuke
a review comment, and have to go digging...

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Kernel development collaboration platform wish list

[Jani Nikula]

On Thu, 26 Sep 2019, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> Hi Jani,
>
> On Thu, Sep 26, 2019 at 3:10 PM Jani Nikula <jani.nikula@intel.com> wrote:
>> On Thu, 26 Sep 2019, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>> >   - Quoting a 2000-line patch when commenting on a single line deep down.
>>
>> I've actually become almost completely immune to that because my mail
>> client collapses long quotes by default, thus highlighting the replies.
>
> The Gmail web interface is indeed good at that... Until you want to rebuke
> a review comment, and have to go digging...

Who said anything about gmail? https://notmuchmail.org/ ;)

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Graphics Center

Re: Kernel development collaboration platform wish list

[Steven Rostedt]

On Thu, 26 Sep 2019 15:39:54 +0200
Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> On Thu, Sep 26, 2019 at 3:02 PM Steven Rostedt <rostedt@goodmis.org> wrote:
> > That would be wonderful.  
> 
> Oh, the irony ;-)
> 

Sorry, I couldn't resist ;-)

-- Steve

> > -- Steve
> >
> > On September 26, 2019 12:18:18 PM GMT+02:00, Geert Uytterhoeven <geert@linux-m68k.org> wrote:  
> 
> [...]
> 
> > >And if it's done right, it may solve (or make it harder to show) other
> > >bad review
> > >behavior, like:
> > >  - Top posting review comments,
> > >- Quoting a 2000-line patch when commenting on a single line deep down.  
> 
> Gr{oetje,eeting}s,
> 
>                         Geert
> 

Re: Kernel development collaboration platform wish list

[Dmitry Vyukov]

On Sun, Sep 29, 2019 at 12:16 AM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Thu, 26 Sep 2019 15:39:54 +0200
> Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> > On Thu, Sep 26, 2019 at 3:02 PM Steven Rostedt <rostedt@goodmis.org> wrote:
> > > That would be wonderful.
> >
> > Oh, the irony ;-)
> >
>
> Sorry, I couldn't resist ;-)
>
> -- Steve
>
> > > -- Steve
> > >
> > > On September 26, 2019 12:18:18 PM GMT+02:00, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> >
> > [...]
> >
> > > >And if it's done right, it may solve (or make it harder to show) other
> > > >bad review
> > > >behavior, like:
> > > >  - Top posting review comments,
> > > >- Quoting a 2000-line patch when commenting on a single line deep down.

Hi,

I've aggregated the requirements that people mentioned so far here:
https://github.com/dvyukov/kit/blob/master/doc/references.md
I think we need some centralized docs related to this effort.
Information scattered across multiple long email threads and some
private discussions won't be manageable. In future we can extend and
refine the list, put more structure (e.g. v1 vs v2 requirements), etc.

Also references to other development support systems:
https://github.com/dvyukov/kit/blob/master/doc/requirements.md
(people keep mentioning more and more, I've found some interesting
ones e.g. git-appraise, so I thought it would be good to collect the
list)

Why is this on my private github? It was just easier to put an initial
version there. If there is a better shared place, I am happy to move
this.

Re: Kernel development collaboration platform wish list

[Daniel Axtens]

"Rafael J. Wysocki" <rjw@rjwysocki.net> writes:

> On Friday, September 13, 2019 10:22:20 AM CEST Rafael J. Wysocki wrote:
>> Hi All,
>> 
>> During the Maintainers Summit session yesterday I started to create a wish list
>> for the new kernel development collaboration platform to be created (and to
>> replace the multiple pieces of tooling in use today).  I also asked Bjorn,
>> Jiri, Greg and Shuah for input and here's the reslut:
>> 
>> 1. Compatible with e-mail
>> 
>>   (a) E-mail send to it stored and included automatically; appears as part of
>>       the normal flow.
>> 
>>   (b) Automatic e-mail responses
>>       If e-mail is sent to it, the sender will get all responses to it in the
>>       given thread by e-mail.
>> 
>> 2. History tracking
>> 
>>   (a) Should be able to track revisions of a given patch series (or patch) down
>>       to the initial submission.
>> 
>> 3. Integration with git
>> 
>>   (a) Should be able to create git commits from patches (or patch series)
>>       tracked by it if pointed to a git branch (either locally or remotely).
>> 
>>   (b) Link tags pointing back to it should be added automatically to git
>>       commits created from patches tracked by it.
>> 
>> 4. Distributed
>> 
>>   (a) Support for running offline.
>> 
>>   (b) Support for batch updates.
>> 
>>   (c) CL-frendly.
>> 
>> 5. Patchwork-like features
>> 
>>   (a) Delegation support.
>> 
>>   (b) Support for bundles and patch series manipulation.
>> 
>>   (c) Smart mbox (download all selected patches).
>
> And one more item from myself:
>
> (d) Support for "maintainer views"
>
>     That is, by default subsystem maintainers should see patches, bug reports
>     etc against the code maintained by them, with the possibility to extend the
>     view to also see the other submissions.
>
>     [That kind of is the case in Patchwork today when patches sent to different
>      mailing lists show up under different "projects", but the problem in there
>      is that copies of one patch appear under multiple "projects" as different
>      entities if sent to multiple lists.]

/me puts on pw maintainer hat

This is true, they are each individual database entries. This is largely
because projects will often change the state of patches differently. (An
example is a patch sent to multiple lists to collect ACKs before being
merged.)

Unsurprisingly, there's not really a lot of funding for pw development
at the moment, I'm doing it mostly as a hobby - but once we smash out a
bit of technical debt I'm slogging through it should be easier to
contribute. We also have a pretty full-featured API that I would
encourage people to check out (e.g. https://patchwork.kernel.org/api/) -
we already have a few lists using this for CI with projects like
snowpatch or custom GitLab scripts - just check out the linuxppc
patchwork for an example.

Kind regards,
Daniel

>
>> 
>> 6. Easy to set up (especially for local installations)
>> 
>> 7. Bug tracking support
>> 
>> I guess there are more items to be added to this list, so please extend it if
>> you have any ideas and we'll see where this goes. :-)

Re: Kernel development collaboration platform wish list

[Daniel Axtens]

Eric Wong <e@80x24.org> writes:

> "Theodore Y. Ts'o" <tytso@mit.edu> wrote:
>> One of the things I'd like to add as a strong desire is the ability to
>> review patches via web if that's what users would like to do.  There
>> are some real benefits for web-based review.  It means that if you
>> need to see greater context, it's relatively easy to do this.  It also
>> is convenient to be able to see the conversation for a particular hunk
>> of code right alongside the code.
>
> I've already started on that with public-inbox; it's not enabled
> on lore, yet(*); but public-inbox.org/git can use "git apply"
> and dfpre:/dfpost: search prefixes to reconstruct git blobs out
> of emailed patches.
>
> For example, a self-rejected patch I posted in 2016:
>
>   https://public-inbox.org/git/20160711210243.GA1604@whir/
>
> If you follow the link at the hunk header offset "+3,7",
> it'll bring you to:
>
>   https://public-inbox.org/git/8d27707/s/?b=http-walker.c#n4
>
>   The bottom of that page has a a "debug log:" which tells
>   how the blob is reconstructed using the email and pre-existing
>   blob.
>
> Since it can recreate blobs using patches, the next step is to
> support showing "git diff" against reconstructed blobs with for
> arbitrary contexts; but I got side-tracked from that earlier
> this year...
>
> (*) lore could configure coderepo associations, but that's a
>     bear with the amount of repos kernel.org has...
>
>> It's clear that whatever we do, it needs to be compatible with e-mail.
>> That's very clear.  But it would be useful if we can support both the
>> e-mail and web-based review.  There have some prototypes that have
>> been floated which shows that it is at least possible; perhaps
>> imperfectly, but something which provides a bidrectional gateway
>> between those who perfer to use e-mail and those that prefer to use a
>> web-based UI would be able to do it.
>
> A hacker-oriented a webmail UI could be derived from
> public-inbox and completely interoperable with other mail
> servers (it would also support SMTP/IMAP, of course).
>
>> There will be many potential kernel contributors who will be used to
>> web-based UI's such as those that are available on github.  So while
>> remaining e-mail compatible, having some way of allow as many
>> operations to be done via web interfaces might help us get some newer
>> developers who are more comfortable to living on the web than some of
>> us more senior developers who remember when "gopher" was a text-based
>> search engine, and not a mascot for the Go programming langauge.  :-)
>
> Unfortunate, but yeah.  Any web-based UIs ought to include some
> subtle (or maybe not-so-subtle) hints/pointers towards
> cheaper-to-run-and-likely-more-powerful tools and practices.
>
> I can't prove it, but it seems like public-inbox has been
> successful at promoting the use of Message-IDs in URLs and
> git-send-email :)

FYI, the next release of Patchwork will move to message-id based URLs
rather than numeric IDs. (Old URLs will still work of course! They will
redirect to new URLs.)

Regards,
Daniel

Re: Kernel development collaboration platform wish list

[Andrew Donnellan]

On 1/10/19 5:52 am, Daniel Axtens wrote:
>> (d) Support for "maintainer views"
>>
>>      That is, by default subsystem maintainers should see patches, bug reports
>>      etc against the code maintained by them, with the possibility to extend the
>>      view to also see the other submissions.
>>
>>      [That kind of is the case in Patchwork today when patches sent to different
>>       mailing lists show up under different "projects", but the problem in there
>>       is that copies of one patch appear under multiple "projects" as different
>>       entities if sent to multiple lists.]
> 
> /me puts on pw maintainer hat
> 
> This is true, they are each individual database entries. This is largely
> because projects will often change the state of patches differently. (An
> example is a patch sent to multiple lists to collect ACKs before being
> merged.)

We really should get around to adding inter-project links for matching 
message-IDs.

-- 
Andrew Donnellan              OzLabs, ADL Canberra
ajd@linux.ibm.com             IBM Australia Limited

Re: Kernel development collaboration platform wish list

[Daniel Axtens]

Andrew Donnellan <ajd@linux.ibm.com> writes:

> On 1/10/19 5:52 am, Daniel Axtens wrote:
>>> (d) Support for "maintainer views"
>>>
>>>      That is, by default subsystem maintainers should see patches, bug reports
>>>      etc against the code maintained by them, with the possibility to extend the
>>>      view to also see the other submissions.
>>>
>>>      [That kind of is the case in Patchwork today when patches sent to different
>>>       mailing lists show up under different "projects", but the problem in there
>>>       is that copies of one patch appear under multiple "projects" as different
>>>       entities if sent to multiple lists.]
>> 
>> /me puts on pw maintainer hat
>> 
>> This is true, they are each individual database entries. This is largely
>> because projects will often change the state of patches differently. (An
>> example is a patch sent to multiple lists to collect ACKs before being
>> merged.)
>
> We really should get around to adding inter-project links for matching 
> message-IDs.

Sure, we just have to get the UI right so that people aren't overwhelmed
in the case where a patch is sent to a dozen different lists.

Regards,
Daniel

>
> -- 
> Andrew Donnellan              OzLabs, ADL Canberra
> ajd@linux.ibm.com             IBM Australia Limited

Re: Kernel development collaboration platform wish list

[Rafael J. Wysocki]

On Friday, September 13, 2019 1:37:53 PM CEST Laurent Pinchart wrote:
> Hi Rafael,
> 
> On Fri, Sep 13, 2019 at 10:22:20AM +0200, Rafael J. Wysocki wrote:
> > Hi All,
> > 
> > During the Maintainers Summit session yesterday I started to create a wish list
> > for the new kernel development collaboration platform to be created (and to
> > replace the multiple pieces of tooling in use today).  I also asked Bjorn,
> > Jiri, Greg and Shuah for input and here's the reslut:
> > 
> > 1. Compatible with e-mail
> > 
> >   (a) E-mail send to it stored and included automatically; appears as part of
> >       the normal flow.
> > 
> >   (b) Automatic e-mail responses
> >       If e-mail is sent to it, the sender will get all responses to it in the
> >       given thread by e-mail.
> > 
> > 2. History tracking
> > 
> >   (a) Should be able to track revisions of a given patch series (or patch) down
> >       to the initial submission.
> > 
> > 3. Integration with git
> > 
> >   (a) Should be able to create git commits from patches (or patch series)
> >       tracked by it if pointed to a git branch (either locally or remotely).
> > 
> >   (b) Link tags pointing back to it should be added automatically to git
> >       commits created from patches tracked by it.
> 
> In addition to a link, should the tag message contain a copy of the
> cover letter for the patch series ?

Well, that too unless there is a sufficiently straightforward way to get from
a patch (tracked by it) to the series cover letter.  For instance, if the cover
letter is one mouse click away from the patch, it may not be worth
duplicating is the tag message.

Re: Kernel development collaboration platform wish list

[Rafael J. Wysocki]

On Monday, September 23, 2019 6:20:01 PM CEST Theodore Y. Ts'o wrote:
> One of the things I'd like to add as a strong desire is the ability to
> review patches via web if that's what users would like to do.  There
> are some real benefits for web-based review.  It means that if you
> need to see greater context, it's relatively easy to do this.  It also
> is convenient to be able to see the conversation for a particular hunk
> of code right alongside the code.

I totally agree.

That may not be clear from my original posting (sorry about that), but it
was my basic assumption that the web-based functionality would be there.

> It's clear that whatever we do, it needs to be compatible with e-mail.
> That's very clear.  But it would be useful if we can support both the
> e-mail and web-based review.

Right.

> There have some prototypes that have
> been floated which shows that it is at least possible; perhaps
> imperfectly, but something which provides a bidrectional gateway
> between those who perfer to use e-mail and those that prefer to use a
> web-based UI would be able to do it.
> 
> There will be many potential kernel contributors who will be used to
> web-based UI's such as those that are available on github.  So while
> remaining e-mail compatible, having some way of allow as many
> operations to be done via web interfaces might help us get some newer
> developers who are more comfortable to living on the web than some of
> us more senior developers who remember when "gopher" was a text-based
> search engine, and not a mascot for the Go programming langauge.  :-)

I agree.

Cheers,
Rafael

Re: Kernel development collaboration platform wish list

[Daniel Axtens]

"Rafael J. Wysocki" <rjw@rjwysocki.net> writes:

> On Monday, September 23, 2019 6:20:01 PM CEST Theodore Y. Ts'o wrote:
>> One of the things I'd like to add as a strong desire is the ability to
>> review patches via web if that's what users would like to do.  There
>> are some real benefits for web-based review.  It means that if you
>> need to see greater context, it's relatively easy to do this.  It also
>> is convenient to be able to see the conversation for a particular hunk
>> of code right alongside the code.
>
> I totally agree.
>
> That may not be clear from my original posting (sorry about that), but it
> was my basic assumption that the web-based functionality would be there.
>
>> It's clear that whatever we do, it needs to be compatible with e-mail.
>> That's very clear.  But it would be useful if we can support both the
>> e-mail and web-based review.
>
> Right.

A long time ago now (before I was involved) there was talk of extending
patchwork to allow logged in users to comment on a patch in the web UI
and have patchwork send the comment as an email on your behalf. This was
in the happier(?) days before we had SPF and DMARC, so I don't know how
feasible it would be today. (Maybe the emails would just have to come
from 'Patchwork on behalf of J. R. Developer'.) But could web-based
review be as simple as a web form that sends an email?

(I'm not advocating that this be put in patchwork straight away or
necessarily at all, but I think a proof of concept that lives off to the
side of patchwork could be an excellent use of the patchwork API!)

Regards,
Daniel

>> There have some prototypes that have
>> been floated which shows that it is at least possible; perhaps
>> imperfectly, but something which provides a bidrectional gateway
>> between those who perfer to use e-mail and those that prefer to use a
>> web-based UI would be able to do it.
>> 
>> There will be many potential kernel contributors who will be used to
>> web-based UI's such as those that are available on github.  So while
>> remaining e-mail compatible, having some way of allow as many
>> operations to be done via web interfaces might help us get some newer
>> developers who are more comfortable to living on the web than some of
>> us more senior developers who remember when "gopher" was a text-based
>> search engine, and not a mascot for the Go programming langauge.  :-)
>
> I agree.
>
> Cheers,
> Rafael

Re: Kernel development collaboration platform wish list

[Rafael J. Wysocki]

On Wednesday, October 2, 2019 4:22:11 PM CEST Daniel Axtens wrote:
> "Rafael J. Wysocki" <rjw@rjwysocki.net> writes:
> 
> > On Monday, September 23, 2019 6:20:01 PM CEST Theodore Y. Ts'o wrote:
> >> One of the things I'd like to add as a strong desire is the ability to
> >> review patches via web if that's what users would like to do.  There
> >> are some real benefits for web-based review.  It means that if you
> >> need to see greater context, it's relatively easy to do this.  It also
> >> is convenient to be able to see the conversation for a particular hunk
> >> of code right alongside the code.
> >
> > I totally agree.
> >
> > That may not be clear from my original posting (sorry about that), but it
> > was my basic assumption that the web-based functionality would be there.
> >
> >> It's clear that whatever we do, it needs to be compatible with e-mail.
> >> That's very clear.  But it would be useful if we can support both the
> >> e-mail and web-based review.
> >
> > Right.
> 
> A long time ago now (before I was involved) there was talk of extending
> patchwork to allow logged in users to comment on a patch in the web UI
> and have patchwork send the comment as an email on your behalf.

I actually would appreciate something like that. :-)

Today, if I look at a patch in Patchwork and want to make comments on it, I
need to find it in an email client and do that from there which at least is
somewhat cumbersome.

> This was in the happier(?) days before we had SPF and DMARC, so I don't know
> how feasible it would be today. (Maybe the emails would just have to come
> from 'Patchwork on behalf of J. R. Developer'.)

That would work for me.

> But could web-based review be as simple as a web form that sends an email?

That might work too I suppose, but I would like it to open up with the body of
the message I'm replaying to quoted with "> " (so things don't need to be
copied from there by hand).

> (I'm not advocating that this be put in patchwork straight away or
> necessarily at all, but I think a proof of concept that lives off to the
> side of patchwork could be an excellent use of the patchwork API!)

Sure.

Cheers,
Rafael