ksummit.lists.linux.dev archive mirror
 help / color / mirror / Atom feed
* [Ksummit-discuss] Topics for the Maintainer's Summit
@ 2019-08-30  3:17 Theodore Y. Ts'o
  2019-08-30 12:01 ` Wolfram Sang
                   ` (2 more replies)
  0 siblings, 3 replies; 37+ messages in thread
From: Theodore Y. Ts'o @ 2019-08-30  3:17 UTC (permalink / raw)
  To: ksummit-discuss

The following topics have been proposed for the maintainer's summit on
this list:

* Squashing Bugs!   (Shuah Kahn)
    How do we deal with the high volume of bugs reported (especially
    by automated systems like syzbot)

* Depth of the "pull network"	     (James Bottomley)
    Should we be encouraging more people to send pull requests
    to maintainers and sub-maintainers (and sub-sub-maintainers),
    versus a more "flat tree" model where people send pull requests
    directly to Linus?

* Stable Kernel Process Automation and Improvement (Sasha Levin)
    What remaining pain points are there?  How can we make it better?

* Talking in Code or talking Code (Shuah Kahn)
    This was a suggestion about a specific LPC proposal; the core
    suggestion was talkinig about our e-mail conversation styles
    on the mailing list.   We have a similar KS track talk:
    "The list is our process: An analysis of the kernel's
    email-based development process"

* Patch version changes in commit logs?   (Shuah Kahn)
    How to make information about how commit has changed while being
    developed.  (A solution which has already been adopted by some
    maintainers is to use the Link: tag in the commit discussion).
    There have been a more recent discussion in this past week under
    subject line "Allowing something Change-Id (or something like it)
    in kernel commits".
    
Some of these topics have already been mostly resolved via e-mail
discussion.  Which topics do people deserves more discussion?

Are there some additional topics that you'd like to suggest that we
discuss at the maintainer's summit?

					- Ted

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-08-30  3:17 [Ksummit-discuss] Topics for the Maintainer's Summit Theodore Y. Ts'o
@ 2019-08-30 12:01 ` Wolfram Sang
  2019-08-30 13:58 ` Shuah Khan
  2019-08-30 13:58 ` Bjorn Helgaas
  2 siblings, 0 replies; 37+ messages in thread
From: Wolfram Sang @ 2019-08-30 12:01 UTC (permalink / raw)
  To: Theodore Y. Ts'o; +Cc: ksummit-discuss

[-- Attachment #1: Type: text/plain, Size: 426 bytes --]

Hi Ted,

> The following topics have been proposed for the maintainer's summit on
> this list:

There was also:

* Keeping reviews meaningful (Wolfram Sang)

    What is the current common sense of reading tags like Acked-by and
    Rev-by and also of accepting or rejecting them?

So, not directly about the theses I provided as a spark, but rather the
questions which showed up during the discussion.

Regards,

   Wolfram


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-08-30  3:17 [Ksummit-discuss] Topics for the Maintainer's Summit Theodore Y. Ts'o
  2019-08-30 12:01 ` Wolfram Sang
@ 2019-08-30 13:58 ` Shuah Khan
  2019-08-30 14:36   ` shuah
  2019-08-30 13:58 ` Bjorn Helgaas
  2 siblings, 1 reply; 37+ messages in thread
From: Shuah Khan @ 2019-08-30 13:58 UTC (permalink / raw)
  To: Theodore Y. Ts'o, ksummit-discuss

On 8/29/19 9:17 PM, Theodore Y. Ts'o wrote:
> The following topics have been proposed for the maintainer's summit on
> this list:
> 
> * Squashing Bugs!   (Shuah Kahn)
>      How do we deal with the high volume of bugs reported (especially
>      by automated systems like syzbot)
> 
> * Depth of the "pull network"	     (James Bottomley)
>      Should we be encouraging more people to send pull requests
>      to maintainers and sub-maintainers (and sub-sub-maintainers),
>      versus a more "flat tree" model where people send pull requests
>      directly to Linus?
> 
> * Stable Kernel Process Automation and Improvement (Sasha Levin)
>      What remaining pain points are there?  How can we make it better?
> 
> * Talking in Code or talking Code (Shuah Kahn)
>      This was a suggestion about a specific LPC proposal; the core
>      suggestion was talkinig about our e-mail conversation styles
>      on the mailing list.   We have a similar KS track talk:
>      "The list is our process: An analysis of the kernel's
>      email-based development process"

Let's drop this topic. I asked the researcher to focus on clarity of
communication for this talk. Researcher has been looking for tools
that can help analyze clarity of communication in our emails, and
hasn't been successful. More like the analysis too noisy to make
sense of it. As a result, there hasn't been much progress in the
research in the clarity of communication.

> 
> * Patch version changes in commit logs?   (Shuah Kahn)
>      How to make information about how commit has changed while being
>      developed.  (A solution which has already been adopted by some
>      maintainers is to use the Link: tag in the commit discussion).
>      There have been a more recent discussion in this past week under
>      subject line "Allowing something Change-Id (or something like it)
>      in kernel commits".
>   

Looks like this has been resolved with adapting the Link: tag. I would
say we can summarize this and provide a link to the script to share
more easily than digging through the ksummit-discuss threads.

> Some of these topics have already been mostly resolved via e-mail
> discussion.  Which topics do people deserves more discussion?
> 
> Are there some additional topics that you'd like to suggest that we
> discuss at the maintainer's summit?
> 
> 					- Ted
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
> 

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-08-30  3:17 [Ksummit-discuss] Topics for the Maintainer's Summit Theodore Y. Ts'o
  2019-08-30 12:01 ` Wolfram Sang
  2019-08-30 13:58 ` Shuah Khan
@ 2019-08-30 13:58 ` Bjorn Helgaas
  2019-09-02 15:09   ` Shuah Khan
  2019-09-02 20:42   ` Dave Airlie
  2 siblings, 2 replies; 37+ messages in thread
From: Bjorn Helgaas @ 2019-08-30 13:58 UTC (permalink / raw)
  To: Theodore Y. Ts'o; +Cc: ksummit-discuss

On Thu, Aug 29, 2019 at 11:17:20PM -0400, Theodore Y. Ts'o wrote:
> ...
> Are there some additional topics that you'd like to suggest that we
> discuss at the maintainer's summit?

I don't have an effective workflow for managing incoming patches.  I
use a hodge-podge of patchwork, gmail, mutt, and ugly private scripts
to put patches on topic branches, review them, polish them, merge them
together into a "-next" branch, generate pull requests, etc.

I wish there were a collection of the workflows and scripts people
use, maybe even in the kernel sources so they could be shared and
improved.  Some short screencasts could help visualize and pull things
together.  I know a lot of this stuff is "out there" somewhere, but
I'm not aware of any organized collection.

Bjorn

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-08-30 13:58 ` Shuah Khan
@ 2019-08-30 14:36   ` shuah
  0 siblings, 0 replies; 37+ messages in thread
From: shuah @ 2019-08-30 14:36 UTC (permalink / raw)
  To: Shuah Khan, Theodore Y. Ts'o, ksummit-discuss

On 8/30/19 7:58 AM, Shuah Khan wrote:
> On 8/29/19 9:17 PM, Theodore Y. Ts'o wrote:
>> The following topics have been proposed for the maintainer's summit on
>> this list:
>>
>> * Squashing Bugs!   (Shuah Kahn)
>>      How do we deal with the high volume of bugs reported (especially
>>      by automated systems like syzbot)
>>

I am exploring and discussing bringing syzbot reproducers and add them
to kselftest regression tests with Dmitry. I will have to create a few
kselftest framework hooks to run these perhaps. We can discuss that when
we talk about syzbot bugs and how to keep up with them.

thanks,
-- Shuah

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-08-30 13:58 ` Bjorn Helgaas
@ 2019-09-02 15:09   ` Shuah Khan
  2019-09-02 20:42   ` Dave Airlie
  1 sibling, 0 replies; 37+ messages in thread
From: Shuah Khan @ 2019-09-02 15:09 UTC (permalink / raw)
  To: Bjorn Helgaas, Theodore Y. Ts'o; +Cc: ksummit-discuss

On 8/30/19 7:58 AM, Bjorn Helgaas wrote:
> On Thu, Aug 29, 2019 at 11:17:20PM -0400, Theodore Y. Ts'o wrote:
>> ...
>> Are there some additional topics that you'd like to suggest that we
>> discuss at the maintainer's summit?
> 
> I don't have an effective workflow for managing incoming patches.  I
> use a hodge-podge of patchwork, gmail, mutt, and ugly private scripts
> to put patches on topic branches, review them, polish them, merge them
> together into a "-next" branch, generate pull requests, etc.
> 
> I wish there were a collection of the workflows and scripts people
> use, maybe even in the kernel sources so they could be shared and
> improved.  Some short screencasts could help visualize and pull things
> together.  I know a lot of this stuff is "out there" somewhere, but
> I'm not aware of any organized collection.
> 

This is a great idea. Folds in to my request make the script that pulls
in the Link tag widely available.


Patch version changes in commit logs?   (Shuah Kahn)
      How to make information about how commit has changed while being
      developed.  (A solution which has already been adopted by some
      maintainers is to use the Link: tag in the commit discussion).
      There have been a more recent discussion in this past week under
      subject line "Allowing something Change-Id (or something like it)
      in kernel commits".

I have a work-flow that works and I would like to learn from others and
improve mine.

thanks,
-- Shuah

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-08-30 13:58 ` Bjorn Helgaas
  2019-09-02 15:09   ` Shuah Khan
@ 2019-09-02 20:42   ` Dave Airlie
  2019-09-02 22:22     ` Theodore Y. Ts'o
  1 sibling, 1 reply; 37+ messages in thread
From: Dave Airlie @ 2019-09-02 20:42 UTC (permalink / raw)
  To: Bjorn Helgaas; +Cc: ksummit-discuss

[-- Attachment #1: Type: text/plain, Size: 1461 bytes --]

On Friday, 30 August 2019, Bjorn Helgaas <helgaas@kernel.org> wrote:

> On Thu, Aug 29, 2019 at 11:17:20PM -0400, Theodore Y. Ts'o wrote:
> > ...
> > Are there some additional topics that you'd like to suggest that we
> > discuss at the maintainer's summit?
>
> I don't have an effective workflow for managing incoming patches.  I
> use a hodge-podge of patchwork, gmail, mutt, and ugly private scripts
> to put patches on topic branches, review them, polish them, merge them
> together into a "-next" branch, generate pull requests, etc.
>
> I wish there were a collection of the workflows and scripts people
> use, maybe even in the kernel sources so they could be shared and
> improved.  Some short screencasts could help visualize and pull things
> together.  I know a lot of this stuff is "out there" somewhere, but
> I'm not aware of any organized collection.


These are quite drm specific but they do mean myself and Daniel can operate
seamlessly, and all i915 and drm misc maintainers and committers use the
same enforced workflow. We hope to move to gitlab at some point and may try
and use the same interface or not.

 https://drm.pages.freedesktop.org/maintainer-tools/index.html

Happy to give more info at maintainer summit, but we have gotten negative
feedback in the past from some community members who wanted to point out at
length that drm didnt invent group maintainership first, i still have no
idea of the relevancy of the comment.

Dave.

[-- Attachment #2: Type: text/html, Size: 1899 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-02 20:42   ` Dave Airlie
@ 2019-09-02 22:22     ` Theodore Y. Ts'o
  2019-09-03  2:35       ` Olof Johansson
  2019-09-03 13:29       ` Laura Abbott
  0 siblings, 2 replies; 37+ messages in thread
From: Theodore Y. Ts'o @ 2019-09-02 22:22 UTC (permalink / raw)
  To: Dave Airlie; +Cc: Bjorn Helgaas, ksummit-discuss

On Tue, Sep 03, 2019 at 06:42:55AM +1000, Dave Airlie wrote:
> On Friday, 30 August 2019, Bjorn Helgaas <helgaas@kernel.org> wrote:
> 
> > On Thu, Aug 29, 2019 at 11:17:20PM -0400, Theodore Y. Ts'o wrote:
> > > ...
> > > Are there some additional topics that you'd like to suggest that we
> > > discuss at the maintainer's summit?
> >
> > I don't have an effective workflow for managing incoming patches.  I
> > use a hodge-podge of patchwork, gmail, mutt, and ugly private scripts
> > to put patches on topic branches, review them, polish them, merge them
> > together into a "-next" branch, generate pull requests, etc.
> >
> > I wish there were a collection of the workflows and scripts people
> > use, maybe even in the kernel sources so they could be shared and
> > improved.  Some short screencasts could help visualize and pull things
> > together.  I know a lot of this stuff is "out there" somewhere, but
> > I'm not aware of any organized collection.
> 
> 
> These are quite drm specific but they do mean myself and Daniel can operate
> seamlessly, and all i915 and drm misc maintainers and committers use the
> same enforced workflow. We hope to move to gitlab at some point and may try
> and use the same interface or not.
> 
>  https://drm.pages.freedesktop.org/maintainer-tools/index.html
> 
> Happy to give more info at maintainer summit, but we have gotten negative
> feedback in the past from some community members who wanted to point out at
> length that drm didnt invent group maintainership first, i still have no
> idea of the relevancy of the comment.

Are there are other people who have interest in sharing their
workflow?  I'm wonder if it might be useful to schedule time during
the kernel summit, so it's open for more people to benefit from this
sharing?  (Also note that Kernel Summit track sessions will be video
taped for posterity, while Maintainer Summit discussions are *not*
recorded.)

					- Ted

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-02 22:22     ` Theodore Y. Ts'o
@ 2019-09-03  2:35       ` Olof Johansson
  2019-09-03  3:05         ` Randy Dunlap
  2019-09-03 13:29       ` Laura Abbott
  1 sibling, 1 reply; 37+ messages in thread
From: Olof Johansson @ 2019-09-03  2:35 UTC (permalink / raw)
  To: Theodore Y. Ts'o; +Cc: Bjorn Helgaas, ksummit-discuss

On Mon, Sep 2, 2019 at 3:22 PM Theodore Y. Ts'o <tytso@mit.edu> wrote:
>
> On Tue, Sep 03, 2019 at 06:42:55AM +1000, Dave Airlie wrote:
> > On Friday, 30 August 2019, Bjorn Helgaas <helgaas@kernel.org> wrote:
> >
> > > On Thu, Aug 29, 2019 at 11:17:20PM -0400, Theodore Y. Ts'o wrote:
> > > > ...
> > > > Are there some additional topics that you'd like to suggest that we
> > > > discuss at the maintainer's summit?
> > >
> > > I don't have an effective workflow for managing incoming patches.  I
> > > use a hodge-podge of patchwork, gmail, mutt, and ugly private scripts
> > > to put patches on topic branches, review them, polish them, merge them
> > > together into a "-next" branch, generate pull requests, etc.
> > >
> > > I wish there were a collection of the workflows and scripts people
> > > use, maybe even in the kernel sources so they could be shared and
> > > improved.  Some short screencasts could help visualize and pull things
> > > together.  I know a lot of this stuff is "out there" somewhere, but
> > > I'm not aware of any organized collection.
> >
> >
> > These are quite drm specific but they do mean myself and Daniel can operate
> > seamlessly, and all i915 and drm misc maintainers and committers use the
> > same enforced workflow. We hope to move to gitlab at some point and may try
> > and use the same interface or not.
> >
> >  https://drm.pages.freedesktop.org/maintainer-tools/index.html
> >
> > Happy to give more info at maintainer summit, but we have gotten negative
> > feedback in the past from some community members who wanted to point out at
> > length that drm didnt invent group maintainership first, i still have no
> > idea of the relevancy of the comment.
>
> Are there are other people who have interest in sharing their
> workflow?  I'm wonder if it might be useful to schedule time during
> the kernel summit, so it's open for more people to benefit from this
> sharing?  (Also note that Kernel Summit track sessions will be video
> taped for posterity, while Maintainer Summit discussions are *not*
> recorded.)

Sharing workflow sessions are a repeating theme, but I think there's
still a good amount of interest in them since things change over time,
and there's always a lot to learn from how others deal with things.

I've found that sharing exact tool suites tends to be hard, people are
often comfortable with the pile-of-scripts they have. But there's
still value in seeing how others have solved things, and borrow ideas
or pieces of the workflow.

Steven's ktest that's in the kernel tree is a good example -- I like
the idea, but it didn't do quite what I needed, and it was easier to
just roll my own back when I first looked at using it. It doesn't mean
others won't reuse it 100%, and it doesn't mean it's not a good idea
to share them.

I agree that it's probably a great idea to do on the wider KS forum
instead, for wider audience. Maybe a BoF-style talk with show-and-tell
and/or others also showing what and how they do it is useful?


-Olof

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-03  2:35       ` Olof Johansson
@ 2019-09-03  3:05         ` Randy Dunlap
  0 siblings, 0 replies; 37+ messages in thread
From: Randy Dunlap @ 2019-09-03  3:05 UTC (permalink / raw)
  To: Olof Johansson, Theodore Y. Ts'o; +Cc: Bjorn Helgaas, ksummit-discuss

On 9/2/19 7:35 PM, Olof Johansson wrote:
> On Mon, Sep 2, 2019 at 3:22 PM Theodore Y. Ts'o <tytso@mit.edu> wrote:
>>
>> On Tue, Sep 03, 2019 at 06:42:55AM +1000, Dave Airlie wrote:
>>> On Friday, 30 August 2019, Bjorn Helgaas <helgaas@kernel.org> wrote:
>>>
>>>> On Thu, Aug 29, 2019 at 11:17:20PM -0400, Theodore Y. Ts'o wrote:
>>>>> ...
>>>>> Are there some additional topics that you'd like to suggest that we
>>>>> discuss at the maintainer's summit?
>>>>
>>>> I don't have an effective workflow for managing incoming patches.  I
>>>> use a hodge-podge of patchwork, gmail, mutt, and ugly private scripts
>>>> to put patches on topic branches, review them, polish them, merge them
>>>> together into a "-next" branch, generate pull requests, etc.
>>>>
>>>> I wish there were a collection of the workflows and scripts people
>>>> use, maybe even in the kernel sources so they could be shared and
>>>> improved.  Some short screencasts could help visualize and pull things
>>>> together.  I know a lot of this stuff is "out there" somewhere, but
>>>> I'm not aware of any organized collection.
>>>
>>>
>>> These are quite drm specific but they do mean myself and Daniel can operate
>>> seamlessly, and all i915 and drm misc maintainers and committers use the
>>> same enforced workflow. We hope to move to gitlab at some point and may try
>>> and use the same interface or not.
>>>
>>>  https://drm.pages.freedesktop.org/maintainer-tools/index.html
>>>
>>> Happy to give more info at maintainer summit, but we have gotten negative
>>> feedback in the past from some community members who wanted to point out at
>>> length that drm didnt invent group maintainership first, i still have no
>>> idea of the relevancy of the comment.
>>
>> Are there are other people who have interest in sharing their
>> workflow?  I'm wonder if it might be useful to schedule time during
>> the kernel summit, so it's open for more people to benefit from this
>> sharing?  (Also note that Kernel Summit track sessions will be video
>> taped for posterity, while Maintainer Summit discussions are *not*
>> recorded.)
> 
> Sharing workflow sessions are a repeating theme, but I think there's
> still a good amount of interest in them since things change over time,
> and there's always a lot to learn from how others deal with things.
> 
> I've found that sharing exact tool suites tends to be hard, people are
> often comfortable with the pile-of-scripts they have. But there's
> still value in seeing how others have solved things, and borrow ideas
> or pieces of the workflow.
> 
> Steven's ktest that's in the kernel tree is a good example -- I like
> the idea, but it didn't do quite what I needed, and it was easier to
> just roll my own back when I first looked at using it. It doesn't mean
> others won't reuse it 100%, and it doesn't mean it's not a good idea
> to share them.
> 
> I agree that it's probably a great idea to do on the wider KS forum
> instead, for wider audience. Maybe a BoF-style talk with show-and-tell
> and/or others also showing what and how they do it is useful?

Please just make sure that it is recorded for non-attendees to be able
to make use of it.

-- 
~Randy

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-02 22:22     ` Theodore Y. Ts'o
  2019-09-03  2:35       ` Olof Johansson
@ 2019-09-03 13:29       ` Laura Abbott
  2019-09-03 16:07         ` Linus Torvalds
  1 sibling, 1 reply; 37+ messages in thread
From: Laura Abbott @ 2019-09-03 13:29 UTC (permalink / raw)
  To: Theodore Y. Ts'o, Dave Airlie; +Cc: Bjorn Helgaas, ksummit-discuss

On 9/2/19 6:22 PM, Theodore Y. Ts'o wrote:
> On Tue, Sep 03, 2019 at 06:42:55AM +1000, Dave Airlie wrote:
>> On Friday, 30 August 2019, Bjorn Helgaas <helgaas@kernel.org> wrote:
>>
>>> On Thu, Aug 29, 2019 at 11:17:20PM -0400, Theodore Y. Ts'o wrote:
>>>> ...
>>>> Are there some additional topics that you'd like to suggest that we
>>>> discuss at the maintainer's summit?
>>>
>>> I don't have an effective workflow for managing incoming patches.  I
>>> use a hodge-podge of patchwork, gmail, mutt, and ugly private scripts
>>> to put patches on topic branches, review them, polish them, merge them
>>> together into a "-next" branch, generate pull requests, etc.
>>>
>>> I wish there were a collection of the workflows and scripts people
>>> use, maybe even in the kernel sources so they could be shared and
>>> improved.  Some short screencasts could help visualize and pull things
>>> together.  I know a lot of this stuff is "out there" somewhere, but
>>> I'm not aware of any organized collection.
>>
>>
>> These are quite drm specific but they do mean myself and Daniel can operate
>> seamlessly, and all i915 and drm misc maintainers and committers use the
>> same enforced workflow. We hope to move to gitlab at some point and may try
>> and use the same interface or not.
>>
>>   https://drm.pages.freedesktop.org/maintainer-tools/index.html
>>
>> Happy to give more info at maintainer summit, but we have gotten negative
>> feedback in the past from some community members who wanted to point out at
>> length that drm didnt invent group maintainership first, i still have no
>> idea of the relevancy of the comment.
> 
> Are there are other people who have interest in sharing their
> workflow?  I'm wonder if it might be useful to schedule time during
> the kernel summit, so it's open for more people to benefit from this
> sharing?  (Also note that Kernel Summit track sessions will be video
> taped for posterity, while Maintainer Summit discussions are *not*
> recorded.)

It's great that we can share these workflows but it still feels like a
bit of an anti-pattern that we even _need_ to do so. The problem is
everyone has their own 'pet' workflow and while you can certainly
adopt one, I think we're at the point where we do need something in
common for all workflows for the sake of automation (see the number
of testing topics that are about automation)

Maybe part of this discussion should be if we can't mandate a single
workflow, what parts of a maintainer workflow should be common for
the benefit of everyone?

Thanks,
Laura

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-03 13:29       ` Laura Abbott
@ 2019-09-03 16:07         ` Linus Torvalds
  2019-09-03 17:27           ` Konstantin Ryabitsev
  2019-09-05  7:01           ` Jani Nikula
  0 siblings, 2 replies; 37+ messages in thread
From: Linus Torvalds @ 2019-09-03 16:07 UTC (permalink / raw)
  To: Laura Abbott; +Cc: Bjorn Helgaas, ksummit-discuss

On Tue, Sep 3, 2019 at 6:29 AM Laura Abbott <labbott@redhat.com> wrote:
>
> It's great that we can share these workflows but it still feels like a
> bit of an anti-pattern that we even _need_ to do so.

I don't think we really can mandate some of the tooling details - and
details is what a lot of this is all about.

Some people have their workflow very much tied to some very personal
preferences - like particular email readers, editors etc.

Think back on the days when people used to read email inside of GNU
emacs and use a lot of the random Emacs functionality. There was no
way you could convince those people to do anything else ("everything
at my fingertips in the same environment") and there was no way you
could convince anybody else to use that crazy workflow.

I don't think there are a lot of those "everything inside GNU emacs"
people around any more, but some might exist, and even ignoring that
kind of thing, depending on which MUA you use some things are simply
<i>much</i> more convenient than others. I think a lot of us basically
live in our mail readers, and then it makes a huge difference whether
you're using Mutt or whether you're reading mail inside the web
interface of gmail. All those people who use automation from inside
their mail readers (much more limited than the GNU emacs example, but
still things like "pipe these 50 emails to Xyz") can't just be told
"now you have to use the web interface for patchwork and/or a tool
that downloads things that way.

I think some of the push-back to the GPU people wasn't from them not
inventing the group maintainership like Dave said, but from that being
presented as some kind of "this is the way to do it". When it's just
_one_ way to do it, and other groups have completely different
infrastructure and models..

So I don't think we can force some workflows.

We can force some ground rules. The whole "has to be in next" thing is
pretty separate from how the patches are managed, for example.

And "it has to be visible on a public list for review, and for
lore.kernel.org to archive the discussion, because things need not
just review, but _outside_ review" is pretty obvious for any big
changes.

But even that second "obvious" thing equally obviously cannot be
applied to _every_ patch. Even if you ignore the special embargo
cases, you just have a lot of patches that are local fixes, rather
than big new features. We can't tell people "you can't fix an obvious
bug without having it reviewd on the mailing list first". That would
be frustrating any sane developer if we tried to make that a hard
rule. So even the "obvious" workflow that we all think about for big
development simply can't be some kind of "this is how it must be done"
rule.

                      Linus

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-03 16:07         ` Linus Torvalds
@ 2019-09-03 17:27           ` Konstantin Ryabitsev
  2019-09-03 17:40             ` Bjorn Helgaas
                               ` (5 more replies)
  2019-09-05  7:01           ` Jani Nikula
  1 sibling, 6 replies; 37+ messages in thread
From: Konstantin Ryabitsev @ 2019-09-03 17:27 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Bjorn Helgaas, ksummit-discuss

On Tue, Sep 03, 2019 at 09:07:00AM -0700, Linus Torvalds wrote:
>I think some of the push-back to the GPU people wasn't from them not
>inventing the group maintainership like Dave said, but from that being
>presented as some kind of "this is the way to do it". When it's just
>_one_ way to do it, and other groups have completely different
>infrastructure and models..
>
>So I don't think we can force some workflows.

For quite some time now I've been trying to fund some client-side 
tooling development around public-inbox (the software that drives 
lore.kernel.org). Eric Wong (the principal author of public-inbox), and 
I have had lengthy chats about potential functionality of such tool, and 
what we envision can be described as "local patchwork with a mutt-like 
interface":

- It would use public-inbox repositories to track new patches and 
  conversations, so it would no longer be necessary to subscribe to the 
  actual mailing list(s). Getting new mail would be equivalent to a "git 
  pull".
- It would have an equivalent of notmuch search, so instead of needing 
  to go to lore.kernel.org, you could search the entire mailing list 
  locally and perform actions on the results found.
- Just like Patchwork, it would keep track of new patches and series of 
  patches, recognize when new patch/series revisions are posted, track 
  reviewed-by's, tested-by's, etc, and provide useful maintainer 
  functionality, such as showing interdiffs between revisions.
- Patches and series can be pre-filtered by keywords or file paths (e.g.  
  if you're only interested in arch/arm64/mm/.*, the tool would ignore 
  any patches/revisions not touching files in that dir).
- It would support creating workflows and conditional response actions, 
  e.g. "create new branch, apply this series, run these test suites; if 
  tests succeed, merge into branch `for-linus`; if merge successful, 
  reply to submitter with 'thanks, applied!'; if all went well, archive 
  the series; if any steps failed, flag the series for my review".
- The workflows would run in the background, including using external 
  systems if preferred. Maintainers can contribute their workflows to a 
  shared repository so others can easily copy and adapt them.

That's obviously not a complete list, but it seems to me that something 
like this would be quite welcome by a lot of maintainers (at least,
everyone I've talked to about this got really excited). Eric Wong is 
quite willing to work on something like this, but he is not in a 
position to donate so much of his time and effort (especially on top of 
maintaining public-inbox) -- so if we want to see this happen, we need 
to come up with some funds.

I've inquired internally at the Foundation, and while there's general 
willingness to fund such initiatives, the People In Charge Of Money want 
to see a buy-in from maintainers. The natural instinct is to talk to 
Greg, but I believe he's quite happy with his workflow, so while I'm 
sure he'd be happy to feign excitement, he's unlikely to be interested 
in the tool. Linus is not the right person to talk to either, because he 
doesn't deal with patches and tests, so wouldn't benefit from such tool.

So, my plan was to track down Shuah (who's also at the Foundation) and 
Laura (who is on the TAB) at the upcoming summit to float this idea with 
them to see what they think. However, since we're talking about 
lore.kernel.org, tooling and workflows quite a bit already, I figured 
I'll bring this up here as well.

It just seems that every maintainer I spoke with is generally making 
things "sort-of work well enough" by applying a lot of baling wire 
around mail clients, patchwork.kernel.org, gitlab, or all of the above, 
and I'm wondering if everyone is happy to do that, or only doing that 
because a good tool written to fit with the "kernel development model" 
doesn't exist.

So:

- would a tool with such functionality be useful, or would every 
  maintainer prefer to continue doing their own thing (in slightly 
  different ways)?
- would you (or your employer entity) be willing to participate in a 
  fundraiser to help fund the development of such tool (in case we 
  cannot get the LF to fully fund it)?
- would it be okay if the tool is written in NPM/javascript?

Okay, just kidding about the NPM bit. ;)

-K

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-03 17:27           ` Konstantin Ryabitsev
@ 2019-09-03 17:40             ` Bjorn Helgaas
  2019-09-06 10:21               ` Rob Herring
  2019-09-03 17:57             ` Mark Brown
                               ` (4 subsequent siblings)
  5 siblings, 1 reply; 37+ messages in thread
From: Bjorn Helgaas @ 2019-09-03 17:40 UTC (permalink / raw)
  To: Linus Torvalds, Laura Abbott, Bjorn Helgaas, ksummit-discuss

On Tue, Sep 3, 2019 at 12:27 PM Konstantin Ryabitsev
<konstantin@linuxfoundation.org> wrote:

> For quite some time now I've been trying to fund some client-side
> tooling development around public-inbox (the software that drives
> lore.kernel.org). Eric Wong (the principal author of public-inbox), and
> I have had lengthy chats about potential functionality of such tool, and
> what we envision can be described as "local patchwork with a mutt-like
> interface":
>
> - It would use public-inbox repositories to track new patches and
>   conversations, so it would no longer be necessary to subscribe to the
>   actual mailing list(s). Getting new mail would be equivalent to a "git
>   pull".
> - It would have an equivalent of notmuch search, so instead of needing
>   to go to lore.kernel.org, you could search the entire mailing list
>   locally and perform actions on the results found.
> - Just like Patchwork, it would keep track of new patches and series of
>   patches, recognize when new patch/series revisions are posted, track
>   reviewed-by's, tested-by's, etc, and provide useful maintainer
>   functionality, such as showing interdiffs between revisions.
> - Patches and series can be pre-filtered by keywords or file paths (e.g.
>   if you're only interested in arch/arm64/mm/.*, the tool would ignore
>   any patches/revisions not touching files in that dir).
> - It would support creating workflows and conditional response actions,
>   e.g. "create new branch, apply this series, run these test suites; if
>   tests succeed, merge into branch `for-linus`; if merge successful,
>   reply to submitter with 'thanks, applied!'; if all went well, archive
>   the series; if any steps failed, flag the series for my review".
> - The workflows would run in the background, including using external
>   systems if preferred. Maintainers can contribute their workflows to a
>   shared repository so others can easily copy and adapt them.
>
> That's obviously not a complete list, but it seems to me that something
> like this would be quite welcome by a lot of maintainers (at least,
> everyone I've talked to about this got really excited). Eric Wong is
> quite willing to work on something like this, but he is not in a
> position to donate so much of his time and effort (especially on top of
> maintaining public-inbox) -- so if we want to see this happen, we need
> to come up with some funds.
>
> I've inquired internally at the Foundation, and while there's general
> willingness to fund such initiatives, the People In Charge Of Money want
> to see a buy-in from maintainers. The natural instinct is to talk to
> Greg, but I believe he's quite happy with his workflow, so while I'm
> sure he'd be happy to feign excitement, he's unlikely to be interested
> in the tool. Linus is not the right person to talk to either, because he
> doesn't deal with patches and tests, so wouldn't benefit from such tool.
>
> So, my plan was to track down Shuah (who's also at the Foundation) and
> Laura (who is on the TAB) at the upcoming summit to float this idea with
> them to see what they think. However, since we're talking about
> lore.kernel.org, tooling and workflows quite a bit already, I figured
> I'll bring this up here as well.
>
> It just seems that every maintainer I spoke with is generally making
> things "sort-of work well enough" by applying a lot of baling wire
> around mail clients, patchwork.kernel.org, gitlab, or all of the above,
> and I'm wondering if everyone is happy to do that, or only doing that
> because a good tool written to fit with the "kernel development model"
> doesn't exist.
>
> So:
>
> - would a tool with such functionality be useful, or would every
>   maintainer prefer to continue doing their own thing (in slightly
>   different ways)?

I would find something like this incredibly useful.  I currently use
patchwork, but I am really sick of the only-when-online, mouse-around,
clickety-click, wait-for-the-web model.

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-03 17:27           ` Konstantin Ryabitsev
  2019-09-03 17:40             ` Bjorn Helgaas
@ 2019-09-03 17:57             ` Mark Brown
  2019-09-03 18:14             ` Dan Williams
                               ` (3 subsequent siblings)
  5 siblings, 0 replies; 37+ messages in thread
From: Mark Brown @ 2019-09-03 17:57 UTC (permalink / raw)
  To: Linus Torvalds, Laura Abbott, Bjorn Helgaas, ksummit-discuss

[-- Attachment #1: Type: text/plain, Size: 576 bytes --]

On Tue, Sep 03, 2019 at 01:27:08PM -0400, Konstantin Ryabitsev wrote:

> For quite some time now I've been trying to fund some client-side tooling
> development around public-inbox (the software that drives lore.kernel.org).
> Eric Wong (the principal author of public-inbox), and I have had lengthy
> chats about potential functionality of such tool, and what we envision can
> be described as "local patchwork with a mutt-like interface":

This all sounds incredibly useful, I'd be super happy to see it - the
initial proposal does a lot of what my existing scripting does.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-03 17:27           ` Konstantin Ryabitsev
  2019-09-03 17:40             ` Bjorn Helgaas
  2019-09-03 17:57             ` Mark Brown
@ 2019-09-03 18:14             ` Dan Williams
  2019-09-03 21:59             ` Wolfram Sang
                               ` (2 subsequent siblings)
  5 siblings, 0 replies; 37+ messages in thread
From: Dan Williams @ 2019-09-03 18:14 UTC (permalink / raw)
  To: Linus Torvalds, Laura Abbott, Bjorn Helgaas, ksummit-discuss

On Tue, Sep 3, 2019 at 10:27 AM Konstantin Ryabitsev
<konstantin@linuxfoundation.org>
[..]
> So:
>
> - would a tool with such functionality be useful, or would every
>   maintainer prefer to continue doing their own thing (in slightly
>   different ways)?

Yes, I would consider switching to this. The kernel.org patchwork-bot
+ the getpatchwork tool [1] does some of this for me, but lossless
patch reception, sharing rules and triggers for patches (not  just
git-hooks) is a powerful superset.

[1]: https://github.com/getpatchwork/git-pw

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-03 17:27           ` Konstantin Ryabitsev
                               ` (2 preceding siblings ...)
  2019-09-03 18:14             ` Dan Williams
@ 2019-09-03 21:59             ` Wolfram Sang
  2019-09-04  8:34             ` Jan Kara
  2019-09-04 12:08             ` Laurent Pinchart
  5 siblings, 0 replies; 37+ messages in thread
From: Wolfram Sang @ 2019-09-03 21:59 UTC (permalink / raw)
  To: Konstantin Ryabitsev; +Cc: ksummit-discuss

[-- Attachment #1: Type: text/plain, Size: 504 bytes --]


> For quite some time now I've been trying to fund some client-side tooling
> development around public-inbox (the software that drives lore.kernel.org).
> Eric Wong (the principal author of public-inbox), and I have had lengthy
> chats about potential functionality of such tool, and what we envision can
> be described as "local patchwork with a mutt-like interface":

Thanks, Konstantin, this sounds like something I would like to try and
see if it can replace my current "works-good-enough" setup.


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-03 17:27           ` Konstantin Ryabitsev
                               ` (3 preceding siblings ...)
  2019-09-03 21:59             ` Wolfram Sang
@ 2019-09-04  8:34             ` Jan Kara
  2019-09-04 12:08             ` Laurent Pinchart
  5 siblings, 0 replies; 37+ messages in thread
From: Jan Kara @ 2019-09-04  8:34 UTC (permalink / raw)
  To: Konstantin Ryabitsev; +Cc: Bjorn Helgaas, ksummit-discuss

On Tue 03-09-19 13:27:08, Konstantin Ryabitsev wrote:
> - would a tool with such functionality be useful, or would every  maintainer
> prefer to continue doing their own thing (in slightly  different ways)?

I'm more often on the 'review stuff' side of things rather than juggling
huge amounts of patches but having ability to easily go to previous
versions of series / seeing interdiffs would be certainly useful for me and
you seem to offer that :).

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-03 17:27           ` Konstantin Ryabitsev
                               ` (4 preceding siblings ...)
  2019-09-04  8:34             ` Jan Kara
@ 2019-09-04 12:08             ` Laurent Pinchart
  2019-09-04 13:47               ` Konstantin Ryabitsev
  5 siblings, 1 reply; 37+ messages in thread
From: Laurent Pinchart @ 2019-09-04 12:08 UTC (permalink / raw)
  To: Konstantin Ryabitsev; +Cc: Bjorn Helgaas, ksummit-discuss

Hi Konstantin,

On Tue, Sep 03, 2019 at 01:27:08PM -0400, Konstantin Ryabitsev wrote:
> On Tue, Sep 03, 2019 at 09:07:00AM -0700, Linus Torvalds wrote:
> > I think some of the push-back to the GPU people wasn't from them not
> > inventing the group maintainership like Dave said, but from that being
> > presented as some kind of "this is the way to do it". When it's just
> > _one_ way to do it, and other groups have completely different
> > infrastructure and models..
> >
> > So I don't think we can force some workflows.
> 
> For quite some time now I've been trying to fund some client-side 
> tooling development around public-inbox (the software that drives 
> lore.kernel.org). Eric Wong (the principal author of public-inbox), and 
> I have had lengthy chats about potential functionality of such tool, and 
> what we envision can be described as "local patchwork with a mutt-like 
> interface":
> 
> - It would use public-inbox repositories to track new patches and 
>   conversations, so it would no longer be necessary to subscribe to the 
>   actual mailing list(s). Getting new mail would be equivalent to a "git 
>   pull".
> - It would have an equivalent of notmuch search, so instead of needing 
>   to go to lore.kernel.org, you could search the entire mailing list 
>   locally and perform actions on the results found.
> - Just like Patchwork, it would keep track of new patches and series of 
>   patches, recognize when new patch/series revisions are posted, track 
>   reviewed-by's, tested-by's, etc, and provide useful maintainer 
>   functionality, such as showing interdiffs between revisions.

I've been thinking about this for about a year now. One issue that makes
this difficult is that many M[UTD]A software mangle e-mails in a way
that make extracting information automatically pretty painful. Google's
answer to this was Gerrit, which solved this particular issue, but
disrupted the e-mail-based workflow in a way that is not acceptable to
many developers (myself included). A better, in my opinion, solution
would have been standardisation of a format to exchange review
information. Quite obviously going for a markup language (be it XML,
json, yaml or whatever is hype today) is a no-go, given that we would
destroy everybody's workflow again. My idea was to use a review format
that is as close to e-mail as possible (with > quote markers and
everything that people are already familiar with). Of course M[UTD]A
software would still mangle it, but given reasonable M[TD]As, I think we
could teach some of the MUAs commonly used (mutt comes to mind) to
behave properly (through plugins, scripts, settings files, ...). E-mails
that would not be mangled through the process would be easily parsable
by the tool you would like to develop. That would not give us full
coverage, but if we manage to establish such an end-to-end solution, we
could then push it to more MUAs. This would allow to tackle this complex
problem one step at a time, while not alienating developers by asking
them to leave their MUA. Over time we could the develop more tooling
around that review exchange format, once a large enough portion of
exchanged reviews will follow it.

I know I'm dreaming, but is anyone else sharing this dream ?

> - Patches and series can be pre-filtered by keywords or file paths (e.g.  
>   if you're only interested in arch/arm64/mm/.*, the tool would ignore 
>   any patches/revisions not touching files in that dir).
> - It would support creating workflows and conditional response actions, 
>   e.g. "create new branch, apply this series, run these test suites; if 
>   tests succeed, merge into branch `for-linus`; if merge successful, 
>   reply to submitter with 'thanks, applied!'; if all went well, archive 
>   the series; if any steps failed, flag the series for my review".
> - The workflows would run in the background, including using external 
>   systems if preferred. Maintainers can contribute their workflows to a 
>   shared repository so others can easily copy and adapt them.
> 
> That's obviously not a complete list, but it seems to me that something 
> like this would be quite welcome by a lot of maintainers (at least,
> everyone I've talked to about this got really excited).

I'd say it sounds too good to be true. Being often on the move I would
love a tool that would let me work offline. Even when online, being able
to run queries locally and script actions without the need for a web
browser interface would be amazing.

> Eric Wong is 
> quite willing to work on something like this, but he is not in a 
> position to donate so much of his time and effort (especially on top of 
> maintaining public-inbox) -- so if we want to see this happen, we need 
> to come up with some funds.
> 
> I've inquired internally at the Foundation, and while there's general 
> willingness to fund such initiatives, the People In Charge Of Money want 
> to see a buy-in from maintainers. The natural instinct is to talk to 
> Greg, but I believe he's quite happy with his workflow, so while I'm 
> sure he'd be happy to feign excitement, he's unlikely to be interested 
> in the tool. Linus is not the right person to talk to either, because he 
> doesn't deal with patches and tests, so wouldn't benefit from such tool.
> 
> So, my plan was to track down Shuah (who's also at the Foundation) and 
> Laura (who is on the TAB) at the upcoming summit to float this idea with 
> them to see what they think. However, since we're talking about 
> lore.kernel.org, tooling and workflows quite a bit already, I figured 
> I'll bring this up here as well.
> 
> It just seems that every maintainer I spoke with is generally making 
> things "sort-of work well enough" by applying a lot of baling wire 
> around mail clients, patchwork.kernel.org, gitlab, or all of the above, 
> and I'm wondering if everyone is happy to do that, or only doing that 
> because a good tool written to fit with the "kernel development model" 
> doesn't exist.
> 
> So:
> 
> - would a tool with such functionality be useful, or would every 
>   maintainer prefer to continue doing their own thing (in slightly 
>   different ways)?
> - would you (or your employer entity) be willing to participate in a 
>   fundraiser to help fund the development of such tool (in case we 
>   cannot get the LF to fully fund it)?
> - would it be okay if the tool is written in NPM/javascript?
> 
> Okay, just kidding about the NPM bit. ;)

You won't fool us, everybody knows it will be written in Rust ;-)

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-04 12:08             ` Laurent Pinchart
@ 2019-09-04 13:47               ` Konstantin Ryabitsev
  2019-09-05  8:21                 ` Jani Nikula
  0 siblings, 1 reply; 37+ messages in thread
From: Konstantin Ryabitsev @ 2019-09-04 13:47 UTC (permalink / raw)
  To: Laurent Pinchart; +Cc: Bjorn Helgaas, ksummit-discuss

On Wed, Sep 04, 2019 at 03:08:43PM +0300, Laurent Pinchart wrote:
> > - Just like Patchwork, it would keep track of new patches and series of 
> >   patches, recognize when new patch/series revisions are posted, track 
> >   reviewed-by's, tested-by's, etc, and provide useful maintainer 
> >   functionality, such as showing interdiffs between revisions.
> 
> I've been thinking about this for about a year now. One issue that makes
> this difficult is that many M[UTD]A software mangle e-mails in a way
> that make extracting information automatically pretty painful. Google's
> answer to this was Gerrit, which solved this particular issue, but
> disrupted the e-mail-based workflow in a way that is not acceptable to
> many developers (myself included). A better, in my opinion, solution
> would have been standardisation of a format to exchange review
> information. Quite obviously going for a markup language (be it XML,
> json, yaml or whatever is hype today) is a no-go, given that we would
> destroy everybody's workflow again. My idea was to use a review format
> that is as close to e-mail as possible (with > quote markers and
> everything that people are already familiar with). Of course M[UTD]A
> software would still mangle it, but given reasonable M[TD]As, I think we
> could teach some of the MUAs commonly used (mutt comes to mind) to
> behave properly (through plugins, scripts, settings files, ...). E-mails
> that would not be mangled through the process would be easily parsable
> by the tool you would like to develop. That would not give us full
> coverage, but if we manage to establish such an end-to-end solution, we
> could then push it to more MUAs. This would allow to tackle this complex
> problem one step at a time, while not alienating developers by asking
> them to leave their MUA. Over time we could the develop more tooling
> around that review exchange format, once a large enough portion of
> exchanged reviews will follow it.

One way to prevent mail software from mangling message bodies is to send
them as multipart/signed -- at least most MTA/MDAs know not to touch
those. I know git-am handles multipart/signed patches just fine (though
it just ignores signatures), and most gui MUAs just shrug the signatures
off by showing them as useless attachments.

Doesn't help much for cases where people use their own MUAs to send
patches, but at least we can prevent the transmission/display parts of
the equation from messing with structured mail content.

(Of course, in my beautiful vision of the future we aren't using
mail clients at all any more, but let's leave that topic on the
sci-fi/fantasy shelf for now.)

> I'd say it sounds too good to be true. Being often on the move I would
> love a tool that would let me work offline. Even when online, being able
> to run queries locally and script actions without the need for a web
> browser interface would be amazing.

It is quite literally too good to be true, because this tool does not
exist. ;) However, having this feedback will hopefully help me make the
case for coming up with some funds to get things going.

Best regards,
-K

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-03 16:07         ` Linus Torvalds
  2019-09-03 17:27           ` Konstantin Ryabitsev
@ 2019-09-05  7:01           ` Jani Nikula
  2019-09-05 15:26             ` Theodore Y. Ts'o
  1 sibling, 1 reply; 37+ messages in thread
From: Jani Nikula @ 2019-09-05  7:01 UTC (permalink / raw)
  To: Linus Torvalds, Laura Abbott; +Cc: Bjorn Helgaas, ksummit-discuss

On Tue, 03 Sep 2019, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> I think some of the push-back to the GPU people wasn't from them not
> inventing the group maintainership like Dave said, but from that being
> presented as some kind of "this is the way to do it". When it's just
> _one_ way to do it, and other groups have completely different
> infrastructure and models..

At least I've tried (and likely also failed) to merely describe what we
do, what works for us, how we think we benefit, and how it just might
benefit others. It's just sad when the pushback is strong enough to
discourage people from sharing their experiences to begin with.

I know I've reduced talking about it outside of the GPU people bubble.

> And "it has to be visible on a public list for review, and for
> lore.kernel.org to archive the discussion, because things need not
> just review, but _outside_ review" is pretty obvious for any big
> changes.
>
> But even that second "obvious" thing equally obviously cannot be
> applied to _every_ patch. Even if you ignore the special embargo
> cases, you just have a lot of patches that are local fixes, rather
> than big new features. We can't tell people "you can't fix an obvious
> bug without having it reviewd on the mailing list first". That would
> be frustrating any sane developer if we tried to make that a hard
> rule. So even the "obvious" workflow that we all think about for big
> development simply can't be some kind of "this is how it must be done"
> rule.

Okay, I'll stick my neck out again.

In i915 it *is* a hard rule you can't push anything unless it was posted
and reviewed on the public list and passed CI first. No exceptions. Sure
it can be frustrating, but it's also so much less embarrassing when the
bug in the obvious fix gets caught in review/CI rather than in
mainline. And you tend to work on improving your process instead of
making more exceptions and taking shortcuts.

And since I'm one of those pesky "GPU people", YMMV a thousand times
over. ;)


BR,
Jani.

-- 
Jani Nikula, Intel Open Source Graphics Center

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-04 13:47               ` Konstantin Ryabitsev
@ 2019-09-05  8:21                 ` Jani Nikula
  2019-09-06 10:50                   ` Rob Herring
  0 siblings, 1 reply; 37+ messages in thread
From: Jani Nikula @ 2019-09-05  8:21 UTC (permalink / raw)
  To: Konstantin Ryabitsev, Laurent Pinchart; +Cc: Bjorn Helgaas, ksummit-discuss

On Wed, 04 Sep 2019, Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> On Wed, Sep 04, 2019 at 03:08:43PM +0300, Laurent Pinchart wrote:
>> > - Just like Patchwork, it would keep track of new patches and series of 
>> >   patches, recognize when new patch/series revisions are posted, track 
>> >   reviewed-by's, tested-by's, etc, and provide useful maintainer 
>> >   functionality, such as showing interdiffs between revisions.
>> 
>> I've been thinking about this for about a year now. One issue that makes
>> this difficult is that many M[UTD]A software mangle e-mails in a way
>> that make extracting information automatically pretty painful. Google's
>> answer to this was Gerrit, which solved this particular issue, but
>> disrupted the e-mail-based workflow in a way that is not acceptable to
>> many developers (myself included). A better, in my opinion, solution
>> would have been standardisation of a format to exchange review
>> information. Quite obviously going for a markup language (be it XML,
>> json, yaml or whatever is hype today) is a no-go, given that we would
>> destroy everybody's workflow again. My idea was to use a review format
>> that is as close to e-mail as possible (with > quote markers and
>> everything that people are already familiar with). Of course M[UTD]A
>> software would still mangle it, but given reasonable M[TD]As, I think we
>> could teach some of the MUAs commonly used (mutt comes to mind) to
>> behave properly (through plugins, scripts, settings files, ...). E-mails
>> that would not be mangled through the process would be easily parsable
>> by the tool you would like to develop. That would not give us full
>> coverage, but if we manage to establish such an end-to-end solution, we
>> could then push it to more MUAs. This would allow to tackle this complex
>> problem one step at a time, while not alienating developers by asking
>> them to leave their MUA. Over time we could the develop more tooling
>> around that review exchange format, once a large enough portion of
>> exchanged reviews will follow it.
>
> One way to prevent mail software from mangling message bodies is to send
> them as multipart/signed -- at least most MTA/MDAs know not to touch
> those. I know git-am handles multipart/signed patches just fine (though
> it just ignores signatures), and most gui MUAs just shrug the signatures
> off by showing them as useless attachments.
>
> Doesn't help much for cases where people use their own MUAs to send
> patches, but at least we can prevent the transmission/display parts of
> the equation from messing with structured mail content.
>
> (Of course, in my beautiful vision of the future we aren't using
> mail clients at all any more, but let's leave that topic on the
> sci-fi/fantasy shelf for now.)

The sci-fi doesn't turn to reality in massive disruptive jumps. There
are realistic intermediate steps that could be taken.

I have been, and still am, a proponent of email based review.

I've also contributed significantly to a MUA, and my observation is that
email is a massively distributed fuzzing project for email software that
allows message transmission in the sideband.

What if git push and pull operated on top of an unreliable and lossy
transmission channel, without integrity checks, that you had to
configure and set up yourself? That's pretty much what we're doing with
git send-email and am, isn't it?

Generally I think there are more issues in the submission side; there
are more people contributing than applying patches, more setups, more
configuration that can go wrong. Roughly speaking the masses of
contributors are less experienced than the maintainers. What if we tried
to provide a way to contribute using something based on git push
instead?

I'm sure you'll think of a thousand things that would not work and why
it would be just another broken github like thing, but consider this:

- The system would receive the changes by git push, and would mail out
  the patches to the relevant lists itself. It would have SMTP figured
  out. It would always mail the patches using the right git send-email
  options. It could always send a cover letter with the right diffstat,
  and with a git url to the commits.

- The system could decide what the relevant lists as well as maintainers
  to Cc are, using up-to-date info from MAINTAINERS. It could provide a
  way for maintainers and developers to opt in/out of Cc, in fine
  grained ways, instead of leaving that decision to the contributor.

- The system would know what the message-ids of the patches are, because
  it would generate them itself. Thus it would know what messages on the
  list are patches it sent, and which versions of the patches and/or
  series, and which replies are review. (It's incredibly hard for
  patchwork to identify patch series and their versions just by reading
  the list. It can get confused by review that contains a patch.)

- New versions of patch series could automatically contain a diff
  against the previous version of the patches/series. You could easily
  tell if the previous review still holds or needs checking.

- You'd still retain the familiar email based review, but it would be
  easier to find the various versions of the series, and you'd always
  have the changes readily available in a git repo. (This ties to a
  previous discussion about how to link patch series versions
  together. It could be all taken care of, automatically.)

- The maintainers could keep their email based flow, applying patches
  from the list. But there'd be fewer email related technical problems
  with them. Additionally, there'd be a way to pull the patches directly
  from a git tree, possibly pre-amended with the Reviewed-by, Tested-by,
  Link, etc. tags.

- You could add tons of useful git hooks on the contributions, ranging
  from pre-merge testing to notifying linked bugs and whatnot.

Note that I'm not suggesting contributors should have git repos from
which they send pull requests. Instead, you'd have a centralized repo
for the project where you can push your submission. Sure, lots of
details to work out there. But the git send-email part is, IMHO, a
broken part of our process, even if the changes keep being distributed
as emailed patches. It just doesn't seem that way to the people on this
list, because we've figured all of this out eons ago for ourselves. We
forget the long tail of contributors that we always brag about.


BR,
Jani.


-- 
Jani Nikula, Intel Open Source Graphics Center

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-05  7:01           ` Jani Nikula
@ 2019-09-05 15:26             ` Theodore Y. Ts'o
  0 siblings, 0 replies; 37+ messages in thread
From: Theodore Y. Ts'o @ 2019-09-05 15:26 UTC (permalink / raw)
  To: Jani Nikula; +Cc: Bjorn Helgaas, ksummit-discuss

On Thu, Sep 05, 2019 at 10:01:26AM +0300, Jani Nikula wrote:
> 
> At least I've tried (and likely also failed) to merely describe what we
> do, what works for us, how we think we benefit, and how it just might
> benefit others. It's just sad when the pushback is strong enough to
> discourage people from sharing their experiences to begin with.
> 
> I know I've reduced talking about it outside of the GPU people bubble.

I seem to recall at least one over-enthusiastic driver developer who
was so sure that group maintainership was The Way To Go that there may
have been, statements of the effect that everyone else should follow
in their footsteps.  No, not you, but there is a reason why there has
been that push back.

The bigger issue, however, is that there are those who are convinced
that it is a bug that different subsystems have different workflows,
and want to try to converge everyone to A Single Way Of Doing Things.
In some cases, such as trying to define exactly how the Link: or
Change-Id: or what URL to use with the Link: tag, before we even have
had a lot of experience with how things will work.

Others (and I happen to fall in that camp) believe that we should
allow people to experiment, because until we have lived experience
with these different workflows, we won't know how well they work.  If
it is obviously a better way of doing things, then people will adopt
it.  We don't need to legislate rules saying all maintainers must
follow exactly the same workflow, because they may be facing different
set of constraints.

They might not have access to the same continuous testing
infrastructure, especially for subsystems where special hardware is
required.  Or maybe they don't have enough developers and time of
those developers to for group maintainership requiring all patches be
reviewed.  Saying that we're going to force everyone to do things
Exactly The Same Way is not going to end well.

Admittedly, there are some downsides to per-subsystem variations.  It
does make it harder contributors to understand what they need to do in
order to get a patch accepted.  On the other hand, I believe that the
biggest problems in this area is not so much the workflow, but rather
how strict reviewers when they do their review.


For example, consider a new file system used on million devices all
over the world, and which has been consistently getting fix ups and
where the maintainer is very responsive.  What is the threshold before
(a) it gets accepted into staging, (b) it gets accepted as an "all
grown up file system"?  Some people seem to want near perfection
before it should be allowed into the kernel --- and that's why some
file systems have gone into the staging tree.  Some have argued that
file systems should never go through staging, but the reason why they
*do* is because many of these people are those who like to hold new
file systems to super-strict criteria, and others believe that staging
is a better alternative than an out-of-tree file system.

That's a much more profound disagreement than what workflow a
subsystem uses for review and testing, and what the level is of
nit-pickiness that should be used before allowing a commit or a new
subsystem into the tree is not something that can be easily
legislated.

(My opinion is that we are really hazing the prospective maintainer to
make sure they are going to stick around, and that it isn't going to
be a drive by contribution.  I also think we've elevated the criteria
for acceptance much too high, but at least amongst file system
developers, or at least the ones who speak up the most on fsdevel, I
fear I am in the minority.)

					- Ted

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-03 17:40             ` Bjorn Helgaas
@ 2019-09-06 10:21               ` Rob Herring
  2019-09-19  1:47                 ` Bjorn Helgaas
  0 siblings, 1 reply; 37+ messages in thread
From: Rob Herring @ 2019-09-06 10:21 UTC (permalink / raw)
  To: bjorn; +Cc: Bjorn Helgaas, ksummit-discuss

On Tue, Sep 3, 2019 at 6:40 PM Bjorn Helgaas <bjorn.helgaas@gmail.com> wrote:
>
> On Tue, Sep 3, 2019 at 12:27 PM Konstantin Ryabitsev
> <konstantin@linuxfoundation.org> wrote:
>
> > For quite some time now I've been trying to fund some client-side
> > tooling development around public-inbox (the software that drives
> > lore.kernel.org). Eric Wong (the principal author of public-inbox), and
> > I have had lengthy chats about potential functionality of such tool, and
> > what we envision can be described as "local patchwork with a mutt-like
> > interface":
> >
> > - It would use public-inbox repositories to track new patches and
> >   conversations, so it would no longer be necessary to subscribe to the
> >   actual mailing list(s). Getting new mail would be equivalent to a "git
> >   pull".
> > - It would have an equivalent of notmuch search, so instead of needing
> >   to go to lore.kernel.org, you could search the entire mailing list
> >   locally and perform actions on the results found.
> > - Just like Patchwork, it would keep track of new patches and series of
> >   patches, recognize when new patch/series revisions are posted, track
> >   reviewed-by's, tested-by's, etc, and provide useful maintainer
> >   functionality, such as showing interdiffs between revisions.
> > - Patches and series can be pre-filtered by keywords or file paths (e.g.
> >   if you're only interested in arch/arm64/mm/.*, the tool would ignore
> >   any patches/revisions not touching files in that dir).
> > - It would support creating workflows and conditional response actions,
> >   e.g. "create new branch, apply this series, run these test suites; if
> >   tests succeed, merge into branch `for-linus`; if merge successful,
> >   reply to submitter with 'thanks, applied!'; if all went well, archive
> >   the series; if any steps failed, flag the series for my review".
> > - The workflows would run in the background, including using external
> >   systems if preferred. Maintainers can contribute their workflows to a
> >   shared repository so others can easily copy and adapt them.
> >
> > That's obviously not a complete list, but it seems to me that something
> > like this would be quite welcome by a lot of maintainers (at least,
> > everyone I've talked to about this got really excited). Eric Wong is
> > quite willing to work on something like this, but he is not in a
> > position to donate so much of his time and effort (especially on top of
> > maintaining public-inbox) -- so if we want to see this happen, we need
> > to come up with some funds.
> >
> > I've inquired internally at the Foundation, and while there's general
> > willingness to fund such initiatives, the People In Charge Of Money want
> > to see a buy-in from maintainers. The natural instinct is to talk to
> > Greg, but I believe he's quite happy with his workflow, so while I'm
> > sure he'd be happy to feign excitement, he's unlikely to be interested
> > in the tool. Linus is not the right person to talk to either, because he
> > doesn't deal with patches and tests, so wouldn't benefit from such tool.
> >
> > So, my plan was to track down Shuah (who's also at the Foundation) and
> > Laura (who is on the TAB) at the upcoming summit to float this idea with
> > them to see what they think. However, since we're talking about
> > lore.kernel.org, tooling and workflows quite a bit already, I figured
> > I'll bring this up here as well.
> >
> > It just seems that every maintainer I spoke with is generally making
> > things "sort-of work well enough" by applying a lot of baling wire
> > around mail clients, patchwork.kernel.org, gitlab, or all of the above,
> > and I'm wondering if everyone is happy to do that, or only doing that
> > because a good tool written to fit with the "kernel development model"
> > doesn't exist.
> >
> > So:
> >
> > - would a tool with such functionality be useful, or would every
> >   maintainer prefer to continue doing their own thing (in slightly
> >   different ways)?
>
> I would find something like this incredibly useful.  I currently use
> patchwork, but I am really sick of the only-when-online, mouse-around,
> clickety-click, wait-for-the-web model.

You might like my set of bailing wire using patchwork and mutt. It
works offline if you download the patchwork state beforehand and
queues up state changes. The basic flow is:

Load the "New" list from PW (my PW instance is pre-filtered on paths,
so I don't have to sort thru everything on the DT list)
Check for multiple versions of patches, auto email on failure to add
my review tag, check for already applied (to next).
Iterate thru the patch list:
  - Run checkpatch.pl
  - open mutt for each patch. Mutt has the full DT list, so I can look
at the rest of the series if I want.
  - After exiting mutt, prompt for PW state change
  - Possibly apply it
  - Generate replies for applied, reviewed-by or acked-by

Happy to demo it at LPC if you are interested. You can find it
here[1]. The main script is pw-review.

Of course I would happily switch to something else like this proposal
if it shrinks the scripts I have to maintain. Especially for
generating quoted email replies as dealing with mime, utf-8, base64,
quoted printable is "fun".

Rob

[1] https://gitlab.com/robherring/pw-utils

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-05  8:21                 ` Jani Nikula
@ 2019-09-06 10:50                   ` Rob Herring
  2019-09-06 19:21                     ` Linus Torvalds
       [not found]                     ` <20190911095305.36104206A1@mail.kernel.org>
  0 siblings, 2 replies; 37+ messages in thread
From: Rob Herring @ 2019-09-06 10:50 UTC (permalink / raw)
  To: Jani Nikula; +Cc: ksummit, Bjorn Helgaas, Konstantin Ryabitsev

On Thu, Sep 5, 2019 at 9:20 AM Jani Nikula <jani.nikula@intel.com> wrote:
>
> On Wed, 04 Sep 2019, Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> > On Wed, Sep 04, 2019 at 03:08:43PM +0300, Laurent Pinchart wrote:
> >> > - Just like Patchwork, it would keep track of new patches and series of
> >> >   patches, recognize when new patch/series revisions are posted, track
> >> >   reviewed-by's, tested-by's, etc, and provide useful maintainer
> >> >   functionality, such as showing interdiffs between revisions.
> >>
> >> I've been thinking about this for about a year now. One issue that makes
> >> this difficult is that many M[UTD]A software mangle e-mails in a way
> >> that make extracting information automatically pretty painful. Google's
> >> answer to this was Gerrit, which solved this particular issue, but
> >> disrupted the e-mail-based workflow in a way that is not acceptable to
> >> many developers (myself included). A better, in my opinion, solution
> >> would have been standardisation of a format to exchange review
> >> information. Quite obviously going for a markup language (be it XML,
> >> json, yaml or whatever is hype today) is a no-go, given that we would
> >> destroy everybody's workflow again. My idea was to use a review format
> >> that is as close to e-mail as possible (with > quote markers and
> >> everything that people are already familiar with). Of course M[UTD]A
> >> software would still mangle it, but given reasonable M[TD]As, I think we
> >> could teach some of the MUAs commonly used (mutt comes to mind) to
> >> behave properly (through plugins, scripts, settings files, ...). E-mails
> >> that would not be mangled through the process would be easily parsable
> >> by the tool you would like to develop. That would not give us full
> >> coverage, but if we manage to establish such an end-to-end solution, we
> >> could then push it to more MUAs. This would allow to tackle this complex
> >> problem one step at a time, while not alienating developers by asking
> >> them to leave their MUA. Over time we could the develop more tooling
> >> around that review exchange format, once a large enough portion of
> >> exchanged reviews will follow it.
> >
> > One way to prevent mail software from mangling message bodies is to send
> > them as multipart/signed -- at least most MTA/MDAs know not to touch
> > those. I know git-am handles multipart/signed patches just fine (though
> > it just ignores signatures), and most gui MUAs just shrug the signatures
> > off by showing them as useless attachments.
> >
> > Doesn't help much for cases where people use their own MUAs to send
> > patches, but at least we can prevent the transmission/display parts of
> > the equation from messing with structured mail content.
> >
> > (Of course, in my beautiful vision of the future we aren't using
> > mail clients at all any more, but let's leave that topic on the
> > sci-fi/fantasy shelf for now.)
>
> The sci-fi doesn't turn to reality in massive disruptive jumps. There
> are realistic intermediate steps that could be taken.
>
> I have been, and still am, a proponent of email based review.
>
> I've also contributed significantly to a MUA, and my observation is that
> email is a massively distributed fuzzing project for email software that
> allows message transmission in the sideband.
>
> What if git push and pull operated on top of an unreliable and lossy
> transmission channel, without integrity checks, that you had to
> configure and set up yourself? That's pretty much what we're doing with
> git send-email and am, isn't it?
>
> Generally I think there are more issues in the submission side; there
> are more people contributing than applying patches, more setups, more
> configuration that can go wrong. Roughly speaking the masses of
> contributors are less experienced than the maintainers. What if we tried
> to provide a way to contribute using something based on git push
> instead?
>
> I'm sure you'll think of a thousand things that would not work and why
> it would be just another broken github like thing, but consider this:
>
> - The system would receive the changes by git push, and would mail out
>   the patches to the relevant lists itself. It would have SMTP figured
>   out. It would always mail the patches using the right git send-email
>   options. It could always send a cover letter with the right diffstat,
>   and with a git url to the commits.

Independent of the exact process, a git branch for every series would
be great. As a maintainer, I would love to be able to do 'git fetch
some-remote <message-id>'. I don't really care to write and maintain
code to apply series and figure out what branch they apply to. That
code already exists and I'm sure is more robust. If the submitter
provides the branch to begin with in a automatable way, then great,
but that's a much bigger process change.

> - The system could decide what the relevant lists as well as maintainers
>   to Cc are, using up-to-date info from MAINTAINERS. It could provide a
>   way for maintainers and developers to opt in/out of Cc, in fine
>   grained ways, instead of leaving that decision to the contributor.
>
> - The system would know what the message-ids of the patches are, because
>   it would generate them itself. Thus it would know what messages on the
>   list are patches it sent, and which versions of the patches and/or
>   series, and which replies are review. (It's incredibly hard for
>   patchwork to identify patch series and their versions just by reading
>   the list. It can get confused by review that contains a patch.)
>
> - New versions of patch series could automatically contain a diff
>   against the previous version of the patches/series. You could easily
>   tell if the previous review still holds or needs checking.
>
> - You'd still retain the familiar email based review, but it would be
>   easier to find the various versions of the series, and you'd always
>   have the changes readily available in a git repo. (This ties to a
>   previous discussion about how to link patch series versions
>   together. It could be all taken care of, automatically.)
>
> - The maintainers could keep their email based flow, applying patches
>   from the list. But there'd be fewer email related technical problems
>   with them. Additionally, there'd be a way to pull the patches directly
>   from a git tree, possibly pre-amended with the Reviewed-by, Tested-by,
>   Link, etc. tags.
>
> - You could add tons of useful git hooks on the contributions, ranging
>   from pre-merge testing to notifying linked bugs and whatnot.
>
> Note that I'm not suggesting contributors should have git repos from
> which they send pull requests. Instead, you'd have a centralized repo
> for the project where you can push your submission. Sure, lots of
> details to work out there. But the git send-email part is, IMHO, a
> broken part of our process, even if the changes keep being distributed
> as emailed patches. It just doesn't seem that way to the people on this
> list, because we've figured all of this out eons ago for ourselves. We
> forget the long tail of contributors that we always brag about.

I certainly agree that the steps between having a git branch ready and
sending the patches could be improved. If we can automate taking a git
branch and sending the emails on the server side, we could do it on
the client side too. The same problems will exist and need to be
solved: get_maintainers.pl is not completely accurate, who to Cc on
individual patches vs. series, writing version history, etc.

Rob

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-06 10:50                   ` Rob Herring
@ 2019-09-06 19:21                     ` Linus Torvalds
  2019-09-06 19:53                       ` Olof Johansson
       [not found]                     ` <20190911095305.36104206A1@mail.kernel.org>
  1 sibling, 1 reply; 37+ messages in thread
From: Linus Torvalds @ 2019-09-06 19:21 UTC (permalink / raw)
  To: Rob Herring; +Cc: Bjorn Helgaas, Konstantin Ryabitsev, ksummit

On Fri, Sep 6, 2019 at 3:51 AM Rob Herring <robherring2@gmail.com> wrote:
>
> Independent of the exact process, a git branch for every series would
> be great.

That actually sounds really nice. Especially if the cover-letter is
then done as a tag (probably not signed), so that when you fetch it
you get the overview automatically - and if you actually do "git pull"
it would fill the merge editor with the cover letter thing.

Even if you then don't really merge it as-is, it would be a lovely way
to just get the whole series to work with locally.

Of course, I'm likely biased. Since I do almost everything with git
(occasional random one-off patches and Andrew's patch-bomb being the
exceptions), I end up just doing a lot of "git fetch" and then looking
at the results. Despite still probably being able to edit patches in
my sleep after decades of looking at them, these days I find that
easier and more powerful to look at things in git than working on
patches.

I end up often doing things like just doing "gitk ..FETCH_HEAD" and
then increasing the context window to see more of the code around the
patch etc.

Of course, right now I only do it with people who use git branches
(doing the "git fetch" for the next pull request while the previous is
going through my build tests, or when people post pointers WIP
branches etc). Being able to do it for random 50-patch series sounds
lovely, particularly when you then can limit the gitk to only the
parts you care about, while _having_ the whole series.

                 Linus

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-06 19:21                     ` Linus Torvalds
@ 2019-09-06 19:53                       ` Olof Johansson
  2019-09-09  8:40                         ` Jani Nikula
  0 siblings, 1 reply; 37+ messages in thread
From: Olof Johansson @ 2019-09-06 19:53 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Bjorn Helgaas, ksummit, Konstantin Ryabitsev

On Fri, Sep 6, 2019 at 12:22 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Fri, Sep 6, 2019 at 3:51 AM Rob Herring <robherring2@gmail.com> wrote:
> >
> > Independent of the exact process, a git branch for every series would
> > be great.
>
> That actually sounds really nice. Especially if the cover-letter is
> then done as a tag (probably not signed), so that when you fetch it
> you get the overview automatically - and if you actually do "git pull"
> it would fill the merge editor with the cover letter thing.
>
> Even if you then don't really merge it as-is, it would be a lovely way
> to just get the whole series to work with locally.
>
> Of course, I'm likely biased. Since I do almost everything with git
> (occasional random one-off patches and Andrew's patch-bomb being the
> exceptions), I end up just doing a lot of "git fetch" and then looking
> at the results. Despite still probably being able to edit patches in
> my sleep after decades of looking at them, these days I find that
> easier and more powerful to look at things in git than working on
> patches.
>
> I end up often doing things like just doing "gitk ..FETCH_HEAD" and
> then increasing the context window to see more of the code around the
> patch etc.
>
> Of course, right now I only do it with people who use git branches
> (doing the "git fetch" for the next pull request while the previous is
> going through my build tests, or when people post pointers WIP
> branches etc). Being able to do it for random 50-patch series sounds
> lovely, particularly when you then can limit the gitk to only the
> parts you care about, while _having_ the whole series.

Applying patches to branches with automation is something that's been
on my todo list for a while as well -- especially since having a git
branch pre-staged makes some things easier (running checks, linters,
checkpatch, whatnot) with the way most CI tools work. I'd love to see
this happen. Patchwork has hooks so you can have these "checkers"
report back status, but it can be done over email as well, of course.

Random observation: We're slowly migrating closer to the "web" based
model of github/gitlab/bitbucket where changes come in via a merge
request + branch, but we would be reconstructing it out of email with
the cover letter equivalent of the merge request description, etc.
That's obviously not a problem, just an interesting observation. The
final step of merging it in is still manual in our setup, and that's
what most maintainers still prefer; the "hands off" part of the web
model where you don't actively download and look at the code is what
feels less careful and involved at least for me. Plus the fact that
the master contents of the tree would reside up somewhere on the
internet instead of on the maintainers locally controlled machine with
the trust complications involved in that.


-Olof

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-06 19:53                       ` Olof Johansson
@ 2019-09-09  8:40                         ` Jani Nikula
  2019-09-09  9:49                           ` Geert Uytterhoeven
  0 siblings, 1 reply; 37+ messages in thread
From: Jani Nikula @ 2019-09-09  8:40 UTC (permalink / raw)
  To: Olof Johansson, Linus Torvalds
  Cc: Bjorn Helgaas, Konstantin Ryabitsev, ksummit

On Fri, 06 Sep 2019, Olof Johansson <olof@lixom.net> wrote:
> Random observation: We're slowly migrating closer to the "web" based
> model of github/gitlab/bitbucket where changes come in via a merge
> request + branch, but we would be reconstructing it out of email with
> the cover letter equivalent of the merge request description, etc.
> That's obviously not a problem, just an interesting observation.

Well, as I tried to explain up-thread, I think it *is* a problem we're
building infrastructure on top of git send-email and am, while we have
git push and pull. Trying to reconstruct everything from email is
problematic because it is lossy.

We can still have the review on emailed patches, and we could still use
git am to apply patches from email, with better reliability if the
sending was done by a service in, say, kernel.org control. Though if we
had the series automatically available in a branch, I'd think people
would move over to picking up the commits from git. And email would only
be used for communication, not data transfer.

> The final step of merging it in is still manual in our setup, and
> that's what most maintainers still prefer; the "hands off" part of the
> web model where you don't actively download and look at the code is
> what feels less careful and involved at least for me. Plus the fact
> that the master contents of the tree would reside up somewhere on the
> internet instead of on the maintainers locally controlled machine with
> the trust complications involved in that.

I'm suggesting maintainers would still have their trees wherever they
feel comfortable having them. I find it hard to understand why emailed
patches would somehow be inherently safer and more trustworthy than git
pull.

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Graphics Center

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-09  8:40                         ` Jani Nikula
@ 2019-09-09  9:49                           ` Geert Uytterhoeven
  2019-09-09 10:16                             ` Konstantin Ryabitsev
  0 siblings, 1 reply; 37+ messages in thread
From: Geert Uytterhoeven @ 2019-09-09  9:49 UTC (permalink / raw)
  To: Jani Nikula; +Cc: Bjorn Helgaas, ksummit, Konstantin Ryabitsev

Hi Jani,

On Mon, Sep 9, 2019 at 10:39 AM Jani Nikula <jani.nikula@intel.com> wrote:
> On Fri, 06 Sep 2019, Olof Johansson <olof@lixom.net> wrote:
> > Random observation: We're slowly migrating closer to the "web" based
> > model of github/gitlab/bitbucket where changes come in via a merge
> > request + branch, but we would be reconstructing it out of email with
> > the cover letter equivalent of the merge request description, etc.
> > That's obviously not a problem, just an interesting observation.
>
> Well, as I tried to explain up-thread, I think it *is* a problem we're
> building infrastructure on top of git send-email and am, while we have
> git push and pull. Trying to reconstruct everything from email is
> problematic because it is lossy.
>
> We can still have the review on emailed patches, and we could still use
> git am to apply patches from email, with better reliability if the
> sending was done by a service in, say, kernel.org control. Though if we
> had the series automatically available in a branch, I'd think people
> would move over to picking up the commits from git. And email would only
> be used for communication, not data transfer.

Do we trust the branch to contain the exact same commits as the
patches reviewed on the mailing list?
For an automatic service on kernel.org, we could.
For branches provided manually by the submitter, or elsewhere, we cannot.

Note that we already trust patchwork, assuming it received the exact
same patch as our inboxes.  But for patchwork, the human factor is not
involved, so human mistakes are assumed to be absent.

> > The final step of merging it in is still manual in our setup, and
> > that's what most maintainers still prefer; the "hands off" part of the
> > web model where you don't actively download and look at the code is
> > what feels less careful and involved at least for me. Plus the fact
> > that the master contents of the tree would reside up somewhere on the
> > internet instead of on the maintainers locally controlled machine with
> > the trust complications involved in that.
>
> I'm suggesting maintainers would still have their trees wherever they
> feel comfortable having them. I find it hard to understand why emailed
> patches would somehow be inherently safer and more trustworthy than git
> pull.

The emailed patch is what has been reviewed.
For (sub)maintainers, we trust that the branch they provide contains the
right commits.  Still, mistakes happens (check how many pull requests
Linus complains about due to wrong/missing branch/tag).
For random contributors, we do not.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-09  9:49                           ` Geert Uytterhoeven
@ 2019-09-09 10:16                             ` Konstantin Ryabitsev
  2019-09-09 10:59                               ` Geert Uytterhoeven
  0 siblings, 1 reply; 37+ messages in thread
From: Konstantin Ryabitsev @ 2019-09-09 10:16 UTC (permalink / raw)
  To: Geert Uytterhoeven; +Cc: Bjorn Helgaas, ksummit

On Mon, Sep 09, 2019 at 11:49:48AM +0200, Geert Uytterhoeven wrote:
> > We can still have the review on emailed patches, and we could still use
> > git am to apply patches from email, with better reliability if the
> > sending was done by a service in, say, kernel.org control. Though if we
> > had the series automatically available in a branch, I'd think people
> > would move over to picking up the commits from git. And email would only
> > be used for communication, not data transfer.
> 
> Do we trust the branch to contain the exact same commits as the
> patches reviewed on the mailing list?
> For an automatic service on kernel.org, we could.

But we really shouldn't, considering kernel.org is the exact kind of
target that attackers would go after if it was implicitly trusted by
developers. Once patches have been reviewed by maintainers and merged
into their tree, we should be using cryptographic attestation for all
git-centric operations after that -- regardless of whether you pulled
from kernel.org or any other location.

Kernel.org is just there to simplify the moving of bits and we shouldn't
make it a source of trust.

-K

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-09 10:16                             ` Konstantin Ryabitsev
@ 2019-09-09 10:59                               ` Geert Uytterhoeven
  2019-09-09 12:37                                 ` Konstantin Ryabitsev
  0 siblings, 1 reply; 37+ messages in thread
From: Geert Uytterhoeven @ 2019-09-09 10:59 UTC (permalink / raw)
  To: Konstantin Ryabitsev; +Cc: Bjorn Helgaas, ksummit

Hi Konstantin,

On Mon, Sep 9, 2019 at 12:16 PM Konstantin Ryabitsev
<konstantin@linuxfoundation.org> wrote:
> On Mon, Sep 09, 2019 at 11:49:48AM +0200, Geert Uytterhoeven wrote:
> > > We can still have the review on emailed patches, and we could still use
> > > git am to apply patches from email, with better reliability if the
> > > sending was done by a service in, say, kernel.org control. Though if we
> > > had the series automatically available in a branch, I'd think people
> > > would move over to picking up the commits from git. And email would only
> > > be used for communication, not data transfer.
> >
> > Do we trust the branch to contain the exact same commits as the
> > patches reviewed on the mailing list?
> > For an automatic service on kernel.org, we could.
>
> But we really shouldn't, considering kernel.org is the exact kind of
> target that attackers would go after if it was implicitly trusted by

Sure.

> developers. Once patches have been reviewed by maintainers and merged
> into their tree, we should be using cryptographic attestation for all
> git-centric operations after that -- regardless of whether you pulled
> from kernel.org or any other location.

We already use cryptographic attestations for the later operations.

So the weakest link seems to be the step between public review and
import into git by the maintainer.  E.g.
  - The review chain on multiple submissions: Vn+1 may contain an evil
     change that was not in Vn.  As this happens in public, it may be
     noticed by reviewers.
  - The path between patch submission and git-am:  if a patchwork
    instance is compromised, evil changes may sneak in.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-09 10:59                               ` Geert Uytterhoeven
@ 2019-09-09 12:37                                 ` Konstantin Ryabitsev
  0 siblings, 0 replies; 37+ messages in thread
From: Konstantin Ryabitsev @ 2019-09-09 12:37 UTC (permalink / raw)
  To: Geert Uytterhoeven; +Cc: Bjorn Helgaas, ksummit

On Mon, Sep 09, 2019 at 12:59:26PM +0200, Geert Uytterhoeven wrote:
> So the weakest link seems to be the step between public review and
> import into git by the maintainer.  E.g.
>   - The review chain on multiple submissions: Vn+1 may contain an evil
>      change that was not in Vn.  As this happens in public, it may be
>      noticed by reviewers.

Theoretically, a tool that is able to show interdiffs between
patches/series would be able to help identify those.

>   - The path between patch submission and git-am:  if a patchwork
>     instance is compromised, evil changes may sneak in.

I'm not sure we can fix this problem in any way that would be
meaningful. I did have some thoughts about this -- for example, we could
run an API service on lore.kernel.org that would allow issuing queries
like:
- do you have a record of a patch
  - with patch-id c052719edde5b0c648b2e4629a7fbd72fd948652
  - sent by shengjiu.wang@nxp.com
  - sent to list with id linux-kernel.vger.kernel.org
  - sent on 2019-09-09

If the query returns "True" then you know that at least lore.kernel.org
has received the same patch, but I'm not sure if we'd be solving
anything in the grander scheme of things. We still have to trust
kernel.org, plus an attacker could have sent a malicious patch to LKML
with a forged "From" to make this check return true. Nobody really reads
LKML, so sneaking something unnoticed would be fairly easy.

Alternatively, a developer could require that all submitted patches
must include a cryptographic signature. It could even be made relatively
painless, especially if we taught git tools to do it.

For git-format-patch we'd need something like this:

- generate a patch-id (using `git patch-id --stable`)
- generate a signify-compatible signature for the patch-id
- add 'Patch-id-sig' and 'Patch-id-key' into the basement (under "---")

E.g., for the same patch as the example I used above:

------
Subject: [PATCH 1/3] ASoC: fsl_asrc: Use in(out)put_format instead of in(out)put_word_width

snd_pcm_format_t is more formal than enum asrc_word_width, which has
two property, width and physical width, which is more accurate than
enum asrc_word_width. So it is better to use in(out)put_format
instead of in(out)put_word_width.

Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
---
 sound/soc/fsl/fsl_asrc.c | 56 +++++++++++++++++++++++++++-------------
 sound/soc/fsl/fsl_asrc.h |  4 +--
 2 files changed, 40 insertions(+), 20 deletions(-)

 Patch-id-sig: RWT9fcUvSnHPLrCB+miMdp13r39W9az07CWW+b4OLz5zdtPUaj0N9qnfdNB+cbs8T1jYzHPIWfaf+B6z/ZvSVG9rfE1/Xx6+EgU=
 Patch-id-key: RWT9fcUvSnHPLqqyfLbkGBMEscBWciFFp2iBj2XnZPzW69OVIoYwZ25q

diff --git a/sound/soc/fsl/fsl_asrc.c b/sound/soc/fsl/fsl_asrc.c
...etc...
------

Then we teach git-am to keep a trust-on-first-use (TOFU) database of
public keys, so that:

- if there are no entries matching the email in From:, the key is
  added and automatically considered trusted
- if there are entries matching From:, the public key is compared
  with the TOFU database
  - if match, the key is used to validate that the patch-id hasn't
    changed
  - if no match, generate a warning and let the developer decide if they
    trust the new key or not
- if there are entries matching From: in the local TOFU db but the patch
  does not have a "Patch-id-sig", generate a warning

This would be more workable than using PGP signatures, but this would
still be a significant pain point for everyone, so I'm not sure how many
would consider using this. What seems reasonable from the security
high-horse I'm sitting on is not always reasonable for others.
:)

-K

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
       [not found]                     ` <20190911095305.36104206A1@mail.kernel.org>
@ 2019-09-11 11:03                       ` Christoph Hellwig
  2019-09-13  8:19                       ` Matthias Brugger
  1 sibling, 0 replies; 37+ messages in thread
From: Christoph Hellwig @ 2019-09-11 11:03 UTC (permalink / raw)
  To: Stephen Boyd; +Cc: Bjorn Helgaas, ksummit, Konstantin Ryabitsev

On Wed, Sep 11, 2019 at 02:53:04AM -0700, Stephen Boyd wrote:
> > Independent of the exact process, a git branch for every series would
> > be great. As a maintainer, I would love to be able to do 'git fetch
> > some-remote <message-id>'. I don't really care to write and maintain
> > code to apply series and figure out what branch they apply to. That
> > code already exists and I'm sure is more robust.
> 
> +1. It would be huge if 'git am' could gain the ability to apply an mbox
> (which it can already do) and parse out the tags to add them to all the
> right patches. I have a script that mostly does this but it needs some
> more work because sometimes people reply to the cover letter and say
> their reviewed-by tag applies to patches 1-3, 5 and 6 and parsing that
> isn't necessarily easy.

Yes, that would help a lot.  Any ignoring cover letters and allowing
for normal diff fuzz so that it doesn't completely fail with the
slightest movement in lines.  And a vaguely git-rebase like way
to resolve conflicts instead of the current mess requring a manual
patch application.
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
       [not found]                     ` <20190911095305.36104206A1@mail.kernel.org>
  2019-09-11 11:03                       ` Christoph Hellwig
@ 2019-09-13  8:19                       ` Matthias Brugger
  1 sibling, 0 replies; 37+ messages in thread
From: Matthias Brugger @ 2019-09-13  8:19 UTC (permalink / raw)
  To: Stephen Boyd, Rob Herring, Jani Nikula
  Cc: Bjorn Helgaas, ksummit, Konstantin Ryabitsev



On 11/09/2019 11:53, Stephen Boyd wrote:
> Quoting Rob Herring (2019-09-06 03:50:47)
>>
>> Independent of the exact process, a git branch for every series would
>> be great. As a maintainer, I would love to be able to do 'git fetch
>> some-remote <message-id>'. I don't really care to write and maintain
>> code to apply series and figure out what branch they apply to. That
>> code already exists and I'm sure is more robust.
> 
> +1. It would be huge if 'git am' could gain the ability to apply an mbox
> (which it can already do) and parse out the tags to add them to all the
> right patches. I have a script that mostly does this but it needs some
> more work because sometimes people reply to the cover letter and say
> their reviewed-by tag applies to patches 1-3, 5 and 6 and parsing that
> isn't necessarily easy.
> 
>> If the submitter
>> provides the branch to begin with in a automatable way, then great,
>> but that's a much bigger process change.
> 
> I've been formatting patches with the --base option for a few months now
> so that the information about the base commit to apply the patches to is
> part of the cover letter or after the patch if it's a single patch. The
> one missing piece is I don't have a database of patches that are in
> -next or being discussed on the list that these patches may depend on
> (indicated by prerequisite-patch-id:). Of course, this changes the
> process a bit so unless we can somehow force this option on for all git
> users in the kernel and require new users to do this then we're not
> really able to do much. My script falls back to the clk/master branch
> (typically -rc1) so that there's a sane base to patch against when this
> information is missing.
> 
>>
>>> - The system could decide what the relevant lists as well as maintainers
>>>   to Cc are, using up-to-date info from MAINTAINERS. It could provide a
>>>   way for maintainers and developers to opt in/out of Cc, in fine
>>>   grained ways, instead of leaving that decision to the contributor.
>>>
>>> - The system would know what the message-ids of the patches are, because
>>>   it would generate them itself. Thus it would know what messages on the
>>>   list are patches it sent, and which versions of the patches and/or
>>>   series, and which replies are review. (It's incredibly hard for
>>>   patchwork to identify patch series and their versions just by reading
>>>   the list. It can get confused by review that contains a patch.)
>>>
>>> - New versions of patch series could automatically contain a diff
>>>   against the previous version of the patches/series. You could easily
>>>   tell if the previous review still holds or needs checking.
> 
> I do this by applying all patch series and grabbing the version tag out
> of the PATCHv<N> part and using 'git range-diff' to see what's changed.
> I don't do this very often though because it's a huge pain. We could ask
> developers to use --interdiff on 'git format-patch' but again this is a
> process roadblock.
> 
>>>
>>> - You'd still retain the familiar email based review, but it would be
>>>   easier to find the various versions of the series, and you'd always
>>>   have the changes readily available in a git repo. (This ties to a
>>>   previous discussion about how to link patch series versions
>>>   together. It could be all taken care of, automatically.)
>>>
>>> - The maintainers could keep their email based flow, applying patches
>>>   from the list. But there'd be fewer email related technical problems
>>>   with them. Additionally, there'd be a way to pull the patches directly
>>>   from a git tree, possibly pre-amended with the Reviewed-by, Tested-by,
>>>   Link, etc. tags.
>>>
>>> - You could add tons of useful git hooks on the contributions, ranging
>>>   from pre-merge testing to notifying linked bugs and whatnot.
>>>
>>> Note that I'm not suggesting contributors should have git repos from
>>> which they send pull requests. Instead, you'd have a centralized repo
>>> for the project where you can push your submission. Sure, lots of
>>> details to work out there. But the git send-email part is, IMHO, a
>>> broken part of our process, even if the changes keep being distributed
>>> as emailed patches. It just doesn't seem that way to the people on this
>>> list, because we've figured all of this out eons ago for ourselves. We
>>> forget the long tail of contributors that we always brag about.
>>
>> I certainly agree that the steps between having a git branch ready and
>> sending the patches could be improved. If we can automate taking a git
>> branch and sending the emails on the server side, we could do it on
>> the client side too. The same problems will exist and need to be
>> solved: get_maintainers.pl is not completely accurate, who to Cc on
>> individual patches vs. series, writing version history, etc.
> 
> The git project has this "bridge", sort of. It's called GitGitGadget[1].
> It would be awesome if we could have something similar for the kernel so
> that we can get more contributors through the 'github' model and try to
> nudge them to use email at the same time.
> 

I think that would help to lower the burden to contribute to the kernel.
Actually my experience with people starting to work with the kernel is not so
much the technical part they are struggling with. But more all the processes we
have around the kernel. If we could get an "intelligent" GitGitGadget type of
service, that could help all the developers that come from a github-centered
background.

> [1] https://gitgitgadget.github.io
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
> 
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-06 10:21               ` Rob Herring
@ 2019-09-19  1:47                 ` Bjorn Helgaas
  2019-09-19 20:52                   ` Rob Herring
  0 siblings, 1 reply; 37+ messages in thread
From: Bjorn Helgaas @ 2019-09-19  1:47 UTC (permalink / raw)
  To: Rob Herring; +Cc: Bjorn Helgaas, ksummit-discuss

On Fri, Sep 6, 2019 at 5:21 AM Rob Herring <robherring2@gmail.com> wrote:
> You might like my set of bailing wire using patchwork and mutt. It
> works offline if you download the patchwork state beforehand and
> queues up state changes. The basic flow is:
>
> Load the "New" list from PW (my PW instance is pre-filtered on paths,
> so I don't have to sort thru everything on the DT list)
> Check for multiple versions of patches, auto email on failure to add
> my review tag, check for already applied (to next).
> Iterate thru the patch list:
>   - Run checkpatch.pl
>   - open mutt for each patch. Mutt has the full DT list, so I can look
> at the rest of the series if I want.
>   - After exiting mutt, prompt for PW state change
>   - Possibly apply it
>   - Generate replies for applied, reviewed-by or acked-by
>
> Happy to demo it at LPC if you are interested. You can find it
> here[1]. The main script is pw-review.

Thanks for the demo at LPC!  I'm trying to understand how all the
pieces fit together.

How do you download the patchwork state beforehand for working
offline?  For me, actually working offline is nice but rare; my
complaint is that I have to wait for every little interaction
(delegating, superseding, changing state, etc) to talk to the server.
The waits aren't long, but they make the whole process feel sluggish.

You mentioned some CI bits (to run checkpatch, change patchwork state,
etc).  Is there a way to look at that?  I'm guessing you also have
some mutt keybindings or macros?

Is http://patchwork.ozlabs.org/project/devicetree-bindings/list/ the
patchwork you're using?  ISTR one that showed the CI results.

I guess you keep your mbox trimmed somehow?  Starting mutt on my
linux-pci folder takes 5-10 seconds.  But for this purpose there
wouldn't really be a need to have the *entire* history, I guess.
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-19  1:47                 ` Bjorn Helgaas
@ 2019-09-19 20:52                   ` Rob Herring
  2019-09-20 13:37                     ` Mark Brown
  0 siblings, 1 reply; 37+ messages in thread
From: Rob Herring @ 2019-09-19 20:52 UTC (permalink / raw)
  To: bjorn; +Cc: Bjorn Helgaas, ksummit-discuss

On Wed, Sep 18, 2019 at 8:48 PM Bjorn Helgaas <bjorn.helgaas@gmail.com> wrote:
>
> On Fri, Sep 6, 2019 at 5:21 AM Rob Herring <robherring2@gmail.com> wrote:
> > You might like my set of bailing wire using patchwork and mutt. It
> > works offline if you download the patchwork state beforehand and
> > queues up state changes. The basic flow is:
> >
> > Load the "New" list from PW (my PW instance is pre-filtered on paths,
> > so I don't have to sort thru everything on the DT list)
> > Check for multiple versions of patches, auto email on failure to add
> > my review tag, check for already applied (to next).
> > Iterate thru the patch list:
> >   - Run checkpatch.pl
> >   - open mutt for each patch. Mutt has the full DT list, so I can look
> > at the rest of the series if I want.
> >   - After exiting mutt, prompt for PW state change
> >   - Possibly apply it
> >   - Generate replies for applied, reviewed-by or acked-by
> >
> > Happy to demo it at LPC if you are interested. You can find it
> > here[1]. The main script is pw-review.
>
> Thanks for the demo at LPC!  I'm trying to understand how all the
> pieces fit together.
>
> How do you download the patchwork state beforehand for working
> offline?  For me, actually working offline is nice but rare; my
> complaint is that I have to wait for every little interaction
> (delegating, superseding, changing state, etc) to talk to the server.
> The waits aren't long, but they make the whole process feel sluggish.

I just run 'pwclient list' formatted so I can parse it and dump into a
file. After that, the server interaction is mainly just doing
'pwclient update' commands in the review loop. In the offline case,
instead of running the commands, I just save them to another file to
run later.

> You mentioned some CI bits (to run checkpatch, change patchwork state,
> etc).  Is there a way to look at that?  I'm guessing you also have
> some mutt keybindings or macros?

Basically, I run this script which can run either locally on your
system or as a CI job:
https://gitlab.com/robherring/pw-utils/blob/master/pw-checks

This is the CI job:
https://gitlab.com/robherring/linux-dt-review/-/jobs/299584584

Either way, checks get added to the patch state. For example:
https://patchwork.ozlabs.org/patch/1164550/


A somewhat design goal I had was to not tie this into mutt too much.
About all I have is a git am key binding, but now I usually apply
using 'pwclient git-am' so a I get the tags. That's one thing that
doesn't work offline. Not a big deal for me as most things go thru
other maintainers. I just leave anything I'm applying pending and go
thru them again when online. It wouldn't be too hard to just download
all the patches from patchwork up front and then use that to apply
patches.

> Is http://patchwork.ozlabs.org/project/devicetree-bindings/list/ the
> patchwork you're using?  ISTR one that showed the CI results.

Yes.

> I guess you keep your mbox trimmed somehow?  Starting mutt on my
> linux-pci folder takes 5-10 seconds.  But for this purpose there
> wouldn't really be a need to have the *entire* history, I guess.

Yeah, gmail limits it for me.

BTW, I'm using maildir currently. I switched from mbox at some point
as I had some issues with searching the mbox.

Rob
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [Ksummit-discuss] Topics for the Maintainer's Summit
  2019-09-19 20:52                   ` Rob Herring
@ 2019-09-20 13:37                     ` Mark Brown
  0 siblings, 0 replies; 37+ messages in thread
From: Mark Brown @ 2019-09-20 13:37 UTC (permalink / raw)
  To: Rob Herring; +Cc: Bjorn Helgaas, ksummit-discuss


[-- Attachment #1.1: Type: text/plain, Size: 810 bytes --]

On Thu, Sep 19, 2019 at 03:52:34PM -0500, Rob Herring wrote:

> A somewhat design goal I had was to not tie this into mutt too much.
> About all I have is a git am key binding, but now I usually apply
> using 'pwclient git-am' so a I get the tags. That's one thing that
> doesn't work offline. Not a big deal for me as most things go thru
> other maintainers. I just leave anything I'm applying pending and go
> thru them again when online. It wouldn't be too hard to just download
> all the patches from patchwork up front and then use that to apply
> patches.

What I've been doing with that is saving to a mailbox and then having a
script that goes through and looks up the messages in the mailbox when
I'm back online.  If I decide I want to actually apply things while
offline I just do the tags by hand.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 186 bytes --]

_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

^ permalink raw reply	[flat|nested] 37+ messages in thread

end of thread, other threads:[~2019-09-20 13:37 UTC | newest]

Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-30  3:17 [Ksummit-discuss] Topics for the Maintainer's Summit Theodore Y. Ts'o
2019-08-30 12:01 ` Wolfram Sang
2019-08-30 13:58 ` Shuah Khan
2019-08-30 14:36   ` shuah
2019-08-30 13:58 ` Bjorn Helgaas
2019-09-02 15:09   ` Shuah Khan
2019-09-02 20:42   ` Dave Airlie
2019-09-02 22:22     ` Theodore Y. Ts'o
2019-09-03  2:35       ` Olof Johansson
2019-09-03  3:05         ` Randy Dunlap
2019-09-03 13:29       ` Laura Abbott
2019-09-03 16:07         ` Linus Torvalds
2019-09-03 17:27           ` Konstantin Ryabitsev
2019-09-03 17:40             ` Bjorn Helgaas
2019-09-06 10:21               ` Rob Herring
2019-09-19  1:47                 ` Bjorn Helgaas
2019-09-19 20:52                   ` Rob Herring
2019-09-20 13:37                     ` Mark Brown
2019-09-03 17:57             ` Mark Brown
2019-09-03 18:14             ` Dan Williams
2019-09-03 21:59             ` Wolfram Sang
2019-09-04  8:34             ` Jan Kara
2019-09-04 12:08             ` Laurent Pinchart
2019-09-04 13:47               ` Konstantin Ryabitsev
2019-09-05  8:21                 ` Jani Nikula
2019-09-06 10:50                   ` Rob Herring
2019-09-06 19:21                     ` Linus Torvalds
2019-09-06 19:53                       ` Olof Johansson
2019-09-09  8:40                         ` Jani Nikula
2019-09-09  9:49                           ` Geert Uytterhoeven
2019-09-09 10:16                             ` Konstantin Ryabitsev
2019-09-09 10:59                               ` Geert Uytterhoeven
2019-09-09 12:37                                 ` Konstantin Ryabitsev
     [not found]                     ` <20190911095305.36104206A1@mail.kernel.org>
2019-09-11 11:03                       ` Christoph Hellwig
2019-09-13  8:19                       ` Matthias Brugger
2019-09-05  7:01           ` Jani Nikula
2019-09-05 15:26             ` Theodore Y. Ts'o

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).