RFC improving amount of content in 5.11

RFC improving amount of content in 5.11

[Tom Rix]

I am disappointed with how little content is making it into 5.10


So I was wondering what we can do generally and i can do specifically

to improve this.


My comment

Though we are a low volume list, anything non trivial takes about 8 revisions.

My suggestion is that we all try to give the developer our big first

pass review within a week of the patch landing and try to cut the

revisions down to 3.


Tom

Re: RFC improving amount of content in 5.11

[Moritz Fischer]

Hi Tom,

On Mon, Sep 14, 2020 at 01:29:47PM -0700, Tom Rix wrote:
> I am disappointed with how little content is making it into 5.10

One comment I've gotten from Greg in the past is to not hold on to
patches so long, so the pull request this weekend was me trying to a
first set of changes out there. This doesn't mean it has to be the only
content that goes into 5.10 (Note how the pull request said: "First set
of changes for the 5.10 merge window").

> So I was wondering what we can do generally and i can do specifically
> to improve this.
> 
> My comment
> Though we are a low volume list, anything non trivial takes about 8 revisions.
> My suggestion is that we all try to give the developer our big first
> pass review within a week of the patch landing and try to cut the
> revisions down to 3.

It's unfortunate that it takes so long to get things moving, I agree,
but with everything that's going on - bear in mind people deal different
with situations like the present - it is what it is.

My current dayjob doesn't pay me for working on this so the time I dedicate
to this comes out of my spare time and weekends - Personally I'd rather
not burn out and keep functioning in the long run.

Thanks,
Moritz

Re: RFC improving amount of content in 5.11

[Tom Rix]

On 9/14/20 2:10 PM, Moritz Fischer wrote:
> Hi Tom,
>
> On Mon, Sep 14, 2020 at 01:29:47PM -0700, Tom Rix wrote:
>> I am disappointed with how little content is making it into 5.10
> One comment I've gotten from Greg in the past is to not hold on to
> patches so long, so the pull request this weekend was me trying to a
> first set of changes out there. This doesn't mean it has to be the only
> content that goes into 5.10 (Note how the pull request said: "First set
> of changes for the 5.10 merge window").


Let me try to explain why I am asking for input on how to improve the amount of content.

The rough planning i do in my head.  A release is about 2-3 months.

A non trival change takes 8 revisions, with about 1 week per revision.

Gives us 1 or 2 changes per release.

In the easy case, a new card is in the same family, will have 4 new ip blocks

and a change to glue it all together change, 5 patch sets.

So we can handle 1 or 2 cards year.

But if we can cut the review down to 2 weeks, we could do maybe 5-10 cards per year.


Then the downside if we do not keep up.

every card has a custom out of tree driver available on a limited set of distros.

which i believe is the current state of things.


>
>> So I was wondering what we can do generally and i can do specifically
>> to improve this.
>>
>> My comment
>> Though we are a low volume list, anything non trivial takes about 8 revisions.
>> My suggestion is that we all try to give the developer our big first
>> pass review within a week of the patch landing and try to cut the
>> revisions down to 3.
> It's unfortunate that it takes so long to get things moving, I agree,
> but with everything that's going on - bear in mind people deal different
> with situations like the present - it is what it is.
>
> My current dayjob doesn't pay me for working on this so the time I dedicate
> to this comes out of my spare time and weekends - Personally I'd rather
> not burn out and keep functioning in the long run.

I understand, in the past i have worked as a maintainer when it was not my day job, it's hard.

I am fortunate, fpga kernel and userspace is my day job.  Over the last couple of months, i have been

consistently spending a couple hours a day fixing random kernel problems as well as getting linux-fpga

reviews out within a day or two so i know i have the bandwidth to devote.


So I am asking what else can I do ?

Would helping out with staging the PR's be help ?

Could i move up to a maintainer ?

Tom

> Thanks,
> Moritz
>

Re: RFC improving amount of content in 5.11

[Moritz Fischer]

Tom,

On Tue, Sep 15, 2020 at 01:58:52PM -0700, Tom Rix wrote:

> A non trival change takes 8 revisions, with about 1 week per revision.

I don't consider that to be out of the norm, especially if you want
multiple people to give feedback on a changeset. This is a result of the
distributed nature of people working across several timezones.

I generally prefer to go a bit slower and get it right rather than
having to redo or realize we got the interfaces wrong -- some of which
have to stay stable.

> Gives us 1 or 2 changes per release.
> 
> In the easy case, a new card is in the same family, will have 4 new ip blocks
> 
> and a change to glue it all together change, 5 patch sets.

So far I haven't seen that happen that many times.

> So we can handle 1 or 2 cards year.

Again I haven't seen more than that in the past.
> 
> But if we can cut the review down to 2 weeks, we could do maybe 5-10 cards per year.
> 
> 
> Then the downside if we do not keep up.
> 
> every card has a custom out of tree driver available on a limited set of distros.
> 
> which i believe is the current state of things.

Tbh, this is easy to fix as vendor by just submitting the code earlier
and in smaller chunks. People can send out RFCs early and then we can
discuss designs and not just show up with 20+ patch series and expect them
to be merged as is (ideally within 2-3 revisions) even more so if they
span several subsystems.

The kernel never has cared about corporate timelines, and as vendor if
you care about timely hardware support (and want to avoid out-of-tree
nightmares) start early with your upstreaming efforts. That has always
been the case.

> >> So I was wondering what we can do generally and i can do specifically
> >> to improve this.
> >>
> >> My comment
> >> Though we are a low volume list, anything non trivial takes about 8 revisions.
> >> My suggestion is that we all try to give the developer our big first
> >> pass review within a week of the patch landing and try to cut the
> >> revisions down to 3.
> > It's unfortunate that it takes so long to get things moving, I agree,
> > but with everything that's going on - bear in mind people deal different
> > with situations like the present - it is what it is.
> >
> > My current dayjob doesn't pay me for working on this so the time I dedicate
> > to this comes out of my spare time and weekends - Personally I'd rather
> > not burn out and keep functioning in the long run.
> 
> I understand, in the past i have worked as a maintainer when it was not my day job, it's hard.
> 
> I am fortunate, fpga kernel and userspace is my day job.  Over the last couple of months, i have been
> 
> consistently spending a couple hours a day fixing random kernel problems as well as getting linux-fpga
> 
> reviews out within a day or two so i know i have the bandwidth to devote.
> 
> 
> So I am asking what else can I do ?
> 
> Would helping out with staging the PR's be help ?

As you pointed out above, the bottleneck is review velocity, I don't
know what staging PRs helps with that.

> 
> Could i move up to a maintainer ?

The problem is I'd still like to review the patches that go into my
subsystem. I appreciate your help with the reviews, and it's been
helpful so far. I don't think having an addtional maintainer will help
with that at this point.

- Moritz

Re: RFC improving amount of content in 5.11

[Tom Rix]

On 9/15/20 2:33 PM, Moritz Fischer wrote:
> Tom,
>
> On Tue, Sep 15, 2020 at 01:58:52PM -0700, Tom Rix wrote:
>
>> A non trival change takes 8 revisions, with about 1 week per revision.
> I don't consider that to be out of the norm, especially if you want
> multiple people to give feedback on a changeset. This is a result of the
> distributed nature of people working across several timezones.
>
> I generally prefer to go a bit slower and get it right rather than
> having to redo or realize we got the interfaces wrong -- some of which
> have to stay stable.
>
>> Gives us 1 or 2 changes per release.
>>
>> In the easy case, a new card is in the same family, will have 4 new ip blocks
>>
>> and a change to glue it all together change, 5 patch sets.
> So far I haven't seen that happen that many times.
>
>> So we can handle 1 or 2 cards year.
> Again I haven't seen more than that in the past.
>> But if we can cut the review down to 2 weeks, we could do maybe 5-10 cards per year.
>>
>>
>> Then the downside if we do not keep up.
>>
>> every card has a custom out of tree driver available on a limited set of distros.
>>
>> which i believe is the current state of things.
> Tbh, this is easy to fix as vendor by just submitting the code earlier
> and in smaller chunks. People can send out RFCs early and then we can
> discuss designs and not just show up with 20+ patch series and expect them
> to be merged as is (ideally within 2-3 revisions) even more so if they
> span several subsystems.
>
> The kernel never has cared about corporate timelines, and as vendor if
> you care about timely hardware support (and want to avoid out-of-tree
> nightmares) start early with your upstreaming efforts. That has always
> been the case.
>
>>>> So I was wondering what we can do generally and i can do specifically
>>>> to improve this.
>>>>
>>>> My comment
>>>> Though we are a low volume list, anything non trivial takes about 8 revisions.
>>>> My suggestion is that we all try to give the developer our big first
>>>> pass review within a week of the patch landing and try to cut the
>>>> revisions down to 3.
>>> It's unfortunate that it takes so long to get things moving, I agree,
>>> but with everything that's going on - bear in mind people deal different
>>> with situations like the present - it is what it is.
>>>
>>> My current dayjob doesn't pay me for working on this so the time I dedicate
>>> to this comes out of my spare time and weekends - Personally I'd rather
>>> not burn out and keep functioning in the long run.
>> I understand, in the past i have worked as a maintainer when it was not my day job, it's hard.
>>
>> I am fortunate, fpga kernel and userspace is my day job.  Over the last couple of months, i have been
>>
>> consistently spending a couple hours a day fixing random kernel problems as well as getting linux-fpga
>>
>> reviews out within a day or two so i know i have the bandwidth to devote.
>>
>>
>> So I am asking what else can I do ?
>>
>> Would helping out with staging the PR's be help ?
> As you pointed out above, the bottleneck is review velocity, I don't
> know what staging PRs helps with that.
>
>> Could i move up to a maintainer ?
> The problem is I'd still like to review the patches that go into my
> subsystem. I appreciate your help with the reviews, and it's been
> helpful so far. I don't think having an addtional maintainer will help
> with that at this point.

We agree slow reviews are throttling the content in the releases.

Is this a temporary situation with your work or is it steady state?


Are slow reviews the only problem ?

Which is getting back to my original RFC on how can we improve the amount of content in the releases ?

Tom

>
> - Moritz
>

Re: RFC improving amount of content in 5.11

[Moritz Fischer]

Tom,

On Wed, Sep 16, 2020 at 08:07:42AM -0700, Tom Rix wrote:
> 
> On 9/15/20 2:33 PM, Moritz Fischer wrote:
> > Tom,
> >
> > On Tue, Sep 15, 2020 at 01:58:52PM -0700, Tom Rix wrote:
> >
> >> A non trival change takes 8 revisions, with about 1 week per revision.
> > I don't consider that to be out of the norm, especially if you want
> > multiple people to give feedback on a changeset. This is a result of the
> > distributed nature of people working across several timezones.
> >
> > I generally prefer to go a bit slower and get it right rather than
> > having to redo or realize we got the interfaces wrong -- some of which
> > have to stay stable.
> >
> >> Gives us 1 or 2 changes per release.
> >>
> >> In the easy case, a new card is in the same family, will have 4 new ip blocks
> >>
> >> and a change to glue it all together change, 5 patch sets.
> > So far I haven't seen that happen that many times.
> >
> >> So we can handle 1 or 2 cards year.
> > Again I haven't seen more than that in the past.
> >> But if we can cut the review down to 2 weeks, we could do maybe 5-10 cards per year.
> >>
> >>
> >> Then the downside if we do not keep up.
> >>
> >> every card has a custom out of tree driver available on a limited set of distros.
> >>
> >> which i believe is the current state of things.
> > Tbh, this is easy to fix as vendor by just submitting the code earlier
> > and in smaller chunks. People can send out RFCs early and then we can
> > discuss designs and not just show up with 20+ patch series and expect them
> > to be merged as is (ideally within 2-3 revisions) even more so if they
> > span several subsystems.
> >
> > The kernel never has cared about corporate timelines, and as vendor if
> > you care about timely hardware support (and want to avoid out-of-tree
> > nightmares) start early with your upstreaming efforts. That has always
> > been the case.
> >
> >>>> So I was wondering what we can do generally and i can do specifically
> >>>> to improve this.
> >>>>
> >>>> My comment
> >>>> Though we are a low volume list, anything non trivial takes about 8 revisions.
> >>>> My suggestion is that we all try to give the developer our big first
> >>>> pass review within a week of the patch landing and try to cut the
> >>>> revisions down to 3.
> >>> It's unfortunate that it takes so long to get things moving, I agree,
> >>> but with everything that's going on - bear in mind people deal different
> >>> with situations like the present - it is what it is.
> >>>
> >>> My current dayjob doesn't pay me for working on this so the time I dedicate
> >>> to this comes out of my spare time and weekends - Personally I'd rather
> >>> not burn out and keep functioning in the long run.
> >> I understand, in the past i have worked as a maintainer when it was not my day job, it's hard.
> >>
> >> I am fortunate, fpga kernel and userspace is my day job.  Over the last couple of months, i have been
> >>
> >> consistently spending a couple hours a day fixing random kernel problems as well as getting linux-fpga
> >>
> >> reviews out within a day or two so i know i have the bandwidth to devote.
> >>
> >>
> >> So I am asking what else can I do ?
> >>
> >> Would helping out with staging the PR's be help ?
> > As you pointed out above, the bottleneck is review velocity, I don't
> > know what staging PRs helps with that.
> >
> >> Could i move up to a maintainer ?
> > The problem is I'd still like to review the patches that go into my
> > subsystem. I appreciate your help with the reviews, and it's been
> > helpful so far. I don't think having an addtional maintainer will help
> > with that at this point.
> 
> We agree slow reviews are throttling the content in the releases.
> 
> Is this a temporary situation with your work or is it steady state?

Tbh, I don't appreciate the tone you're taking with your emails:

Starting a conversation with how disappointed you are is generally not a
great way to get people on board with anything.

I'll let you know when I need help beyond the reviews, as I already told
you earlier in the replies to your off-list emails.

I am not generally opposed to the idea of bringing on new maintainers --
Hao has done a great job for the DFL code so far -- but as of now I do
not see an immediate benefit (or need) in terms of process to adding more
FPGA maintainers.

> Are slow reviews the only problem ?

Since the FPGA pull requests go through Greg's tree they need to be sent
out earlier than a pull request to Linus, if you send out a patchset
around rc4 don't expect it to go in in that release if it requires a
non-trivial amount of review -- if you have patches just send them.

> Which is getting back to my original RFC on how can we improve the amount of content in the releases ?

Send patches earlier (ideally start with an RFC if you intend larger
changes) and in smaller batches, which will save you time later in the
process.

- Moritz

Re: RFC improving amount of content in 5.11

[Tom Rix]

On 9/16/20 11:01 PM, Moritz Fischer wrote:
> Tom,
>
> On Wed, Sep 16, 2020 at 08:07:42AM -0700, Tom Rix wrote:
>> On 9/15/20 2:33 PM, Moritz Fischer wrote:
>>> Tom,
>>>
>>> On Tue, Sep 15, 2020 at 01:58:52PM -0700, Tom Rix wrote:
>>>
>>>> A non trival change takes 8 revisions, with about 1 week per revision.
>>> I don't consider that to be out of the norm, especially if you want
>>> multiple people to give feedback on a changeset. This is a result of the
>>> distributed nature of people working across several timezones.
>>>
>>> I generally prefer to go a bit slower and get it right rather than
>>> having to redo or realize we got the interfaces wrong -- some of which
>>> have to stay stable.
>>>
>>>> Gives us 1 or 2 changes per release.
>>>>
>>>> In the easy case, a new card is in the same family, will have 4 new ip blocks
>>>>
>>>> and a change to glue it all together change, 5 patch sets.
>>> So far I haven't seen that happen that many times.
>>>
>>>> So we can handle 1 or 2 cards year.
>>> Again I haven't seen more than that in the past.
>>>> But if we can cut the review down to 2 weeks, we could do maybe 5-10 cards per year.
>>>>
>>>>
>>>> Then the downside if we do not keep up.
>>>>
>>>> every card has a custom out of tree driver available on a limited set of distros.
>>>>
>>>> which i believe is the current state of things.
>>> Tbh, this is easy to fix as vendor by just submitting the code earlier
>>> and in smaller chunks. People can send out RFCs early and then we can
>>> discuss designs and not just show up with 20+ patch series and expect them
>>> to be merged as is (ideally within 2-3 revisions) even more so if they
>>> span several subsystems.
>>>
>>> The kernel never has cared about corporate timelines, and as vendor if
>>> you care about timely hardware support (and want to avoid out-of-tree
>>> nightmares) start early with your upstreaming efforts. That has always
>>> been the case.
>>>
>>>>>> So I was wondering what we can do generally and i can do specifically
>>>>>> to improve this.
>>>>>>
>>>>>> My comment
>>>>>> Though we are a low volume list, anything non trivial takes about 8 revisions.
>>>>>> My suggestion is that we all try to give the developer our big first
>>>>>> pass review within a week of the patch landing and try to cut the
>>>>>> revisions down to 3.
>>>>> It's unfortunate that it takes so long to get things moving, I agree,
>>>>> but with everything that's going on - bear in mind people deal different
>>>>> with situations like the present - it is what it is.
>>>>>
>>>>> My current dayjob doesn't pay me for working on this so the time I dedicate
>>>>> to this comes out of my spare time and weekends - Personally I'd rather
>>>>> not burn out and keep functioning in the long run.
>>>> I understand, in the past i have worked as a maintainer when it was not my day job, it's hard.
>>>>
>>>> I am fortunate, fpga kernel and userspace is my day job.  Over the last couple of months, i have been
>>>>
>>>> consistently spending a couple hours a day fixing random kernel problems as well as getting linux-fpga
>>>>
>>>> reviews out within a day or two so i know i have the bandwidth to devote.
>>>>
>>>>
>>>> So I am asking what else can I do ?
>>>>
>>>> Would helping out with staging the PR's be help ?
>>> As you pointed out above, the bottleneck is review velocity, I don't
>>> know what staging PRs helps with that.
>>>
>>>> Could i move up to a maintainer ?
>>> The problem is I'd still like to review the patches that go into my
>>> subsystem. I appreciate your help with the reviews, and it's been
>>> helpful so far. I don't think having an addtional maintainer will help
>>> with that at this point.
>> We agree slow reviews are throttling the content in the releases.
>>
>> Is this a temporary situation with your work or is it steady state?
> Tbh, I don't appreciate the tone you're taking with your emails:
>
> Starting a conversation with how disappointed you are is generally not a
> great way to get people on board with anything.
>
> I'll let you know when I need help beyond the reviews, as I already told
> you earlier in the replies to your off-list emails.
>
> I am not generally opposed to the idea of bringing on new maintainers --
> Hao has done a great job for the DFL code so far -- but as of now I do
> not see an immediate benefit (or need) in terms of process to adding more
> FPGA maintainers.
>
>> Are slow reviews the only problem ?
> Since the FPGA pull requests go through Greg's tree they need to be sent
> out earlier than a pull request to Linus, if you send out a patchset
> around rc4 don't expect it to go in in that release if it requires a
> non-trivial amount of review -- if you have patches just send them.
>
>> Which is getting back to my original RFC on how can we improve the amount of content in the releases ?
> Send patches earlier (ideally start with an RFC if you intend larger
> changes) and in smaller batches, which will save you time later in the
> process.

I am sorry, this is difficult i just did not know where to start, i know this is stressful.

My ask for more content is the first step in a bigger ask.

My expectation is that developers should be able to develop first in the kernel.

This a win for everyone, it supports the your call for small changes and rfc's, kills off oot drivers etc.

Having reviewed xilinx and intel dev repos I know we are far far from this.


Thanks for bearing with me,

Tom

>
> - Moritz
>