PECI patchset status

PECI patchset status

[Joel Stanley]

Hello OpenBMCers,

The PECI patchset has been carried in the openbmc tree in some form
since it was first posted in December 2017.

It has not made it upstream in that time, placing the maintenance
burden on me as the kernel maintainer. Generally this isn't a large
amount of work, although in some cases it has held up releasing kernel
branches. Today I noticed that the icotl number it chose has been
claimed by a different ioctl in linux-next, meaning we are guaranteed
to have future kernel/userspace incompatibility.

OpenBMC has strong rules about upstreaming kernel patches, and in
particular userspace facing code, to avoid this issue.

Given the lack of progress I propose dropping it from the OpenBMC
kernel tree until it is merged upstream. This would necessitate
removing tiogapass from the OpenBMC CI, as it relies on PECI support
in the kernel to build.

If you have an interest in the patchset staying in openbmc, we would
need someone (or a team) to take the patchset (v11 is the latest[1])
and submit for inclusion in the mainline kernel, including an entry in
MAINTAINERS to commit to future maintenance. Now is the perfect time
to submit for inclusion in 5.10.

[1] https://lore.kernel.org/linux-hwmon/20191211194624.2872-1-jae.hyun.yoo@linux.intel.com/

Cheers,

Joel

RE: PECI patchset status

[Mihm, James]

Thank you Joel for carrying this patchset, and Intel does have an interest in getting our patchsets upstreamed.

Since Intel has a large set of patches that need to be upstreamed our plan is to fork the kernel in github/intel-bmc and apply the intel patchsets. Upstream recipes can then pull the kernel from the intel fork with the intel patches. Intel will ensure that this fork tracks the openbmc kernel version and maintain the intel patchsets while in the process of getting them upstreamed.

Thanks, James.

-----Original Message-----
From: openbmc <openbmc-bounces+james.mihm=intel.com@lists.ozlabs.org> On Behalf Of Joel Stanley
Sent: Monday, August 31, 2020 1:57 AM
To: OpenBMC Maillist <openbmc@lists.ozlabs.org>
Subject: PECI patchset status

Hello OpenBMCers,

The PECI patchset has been carried in the openbmc tree in some form
since it was first posted in December 2017.

It has not made it upstream in that time, placing the maintenance
burden on me as the kernel maintainer. Generally this isn't a large
amount of work, although in some cases it has held up releasing kernel
branches. Today I noticed that the icotl number it chose has been
claimed by a different ioctl in linux-next, meaning we are guaranteed
to have future kernel/userspace incompatibility.

OpenBMC has strong rules about upstreaming kernel patches, and in
particular userspace facing code, to avoid this issue.

Given the lack of progress I propose dropping it from the OpenBMC
kernel tree until it is merged upstream. This would necessitate
removing tiogapass from the OpenBMC CI, as it relies on PECI support
in the kernel to build.

If you have an interest in the patchset staying in openbmc, we would
need someone (or a team) to take the patchset (v11 is the latest[1])
and submit for inclusion in the mainline kernel, including an entry in
MAINTAINERS to commit to future maintenance. Now is the perfect time
to submit for inclusion in 5.10.

[1] https://lore.kernel.org/linux-hwmon/20191211194624.2872-1-jae.hyun.yoo@linux.intel.com/

Cheers,

Joel

Re: PECI patchset status

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 1073 bytes --]

On Thu, Sep 03, 2020 at 05:57:48AM +0000, Mihm, James wrote:
> Thank you Joel for carrying this patchset, and Intel does have an interest in getting our patchsets upstreamed.
> 
> Since Intel has a large set of patches that need to be upstreamed our plan is to fork the kernel in github/intel-bmc and apply the intel patchsets. Upstream recipes can then pull the kernel from the intel fork with the intel patches. Intel will ensure that this fork tracks the openbmc kernel version and maintain the intel patchsets while in the process of getting them upstreamed.

Rather than create a separate fork of the kernel, is there something
that could be done here to have someone from Intel work with Joel on
preparing the patches?  When a new kernel comes out, Joel can ensure it
works on the base AST2xxx system design and before we move all the
systems to it, someone from Intel can rebase the non-upstreamed patches
they are carrying?  This hopefully reduces some of the burden on Joel
and stops us from further fragmenting the community.

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: PECI patchset status

[Vernon Mauery]

On 03-Sep-2020 10:27 AM, Patrick Williams wrote:
>On Thu, Sep 03, 2020 at 05:57:48AM +0000, Mihm, James wrote:
>> Thank you Joel for carrying this patchset, and Intel does have an interest in getting our patchsets upstreamed.
>>
>> Since Intel has a large set of patches that need to be upstreamed our plan is to fork the kernel in github/intel-bmc and apply the intel patchsets. Upstream recipes can then pull the kernel from the intel fork with the intel patches. Intel will ensure that this fork tracks the openbmc kernel version and maintain the intel patchsets while in the process of getting them upstreamed.
>
>Rather than create a separate fork of the kernel, is there something
>that could be done here to have someone from Intel work with Joel on
>preparing the patches?  When a new kernel comes out, Joel can ensure it
>works on the base AST2xxx system design and before we move all the
>systems to it, someone from Intel can rebase the non-upstreamed patches
>they are carrying?  This hopefully reduces some of the burden on Joel
>and stops us from further fragmenting the community.

Keep in mind that Intel does not plan to keep the fork around 
indefinitely. The hope is to fully upstream all of the patches that we 
have outstanding. Our intention is not to fragment the community, but to 
provide a mechanism to continue to move forward while still providing a 
way for other users to build the intel-platforms target.

As an added feature, having our full kernel source in a publicly 
available tree will allow us to upstream more features that depend on 
kernel support that is not currently available.

--Vernon

Re: PECI patchset status

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 3004 bytes --]

On Thu, Sep 03, 2020 at 10:15:56AM -0700, Vernon Mauery wrote:
> On 03-Sep-2020 10:27 AM, Patrick Williams wrote:
> >On Thu, Sep 03, 2020 at 05:57:48AM +0000, Mihm, James wrote:
> >Rather than create a separate fork of the kernel, is there something
> >that could be done here to have someone from Intel work with Joel on
> >preparing the patches?  When a new kernel comes out, Joel can ensure it
> >works on the base AST2xxx system design and before we move all the
> >systems to it, someone from Intel can rebase the non-upstreamed patches
> >they are carrying?  This hopefully reduces some of the burden on Joel
> >and stops us from further fragmenting the community.
> 
> Keep in mind that Intel does not plan to keep the fork around 
> indefinitely. The hope is to fully upstream all of the patches that we 
> have outstanding. Our intention is not to fragment the community, but to 
> provide a mechanism to continue to move forward while still providing a 
> way for other users to build the intel-platforms target.
> 
> As an added feature, having our full kernel source in a publicly 
> available tree will allow us to upstream more features that depend on 
> kernel support that is not currently available.

I'm not really following this last paragraph.  I suppose you're saying
that you have kernel changes that are not in openbmc/linux that add
additional features?  Why aren't they in openbmc/linux?  I thought there
was a process for getting code in that isn't quite ready for
upstreaming, as long as there is progress towards that?  Is there some
list of what these features are and what the upstreaming state is,
because this original thread was about PECI, but you're implying there
is much more.

If the process isn't working for the community shouldn't we discuss
improving that to something that does work?  It seems like your team has
decided to go to the nuclear option of forking after Joel has proposed
dropping a patchset that he's been carrying for three months short of
three years.

Joel does great work of keeping the kernel up to date, both on a major
release and picking up supplemental fixes.  Is Intel committing to this
same level of support that Joel is currently providing for your own
fork?

Past performance doesn't really give me a lot of confidence that this
will be a short-term fork.  In December 2019, Joel raised this exact
same problem with the PECI driver[1] and it was promised that there
would be forward progress "within a week"[2].  One week later, there was
a v11 of the patches posted[3] and we got some good comments from a
variety of upstream maintainers.  Since then, there has been zero
activity.  Shouldn't we have seen a v12 pretty quickly after that?

1. https://lists.ozlabs.org/pipermail/openbmc/2019-December/019684.html
2. https://lists.ozlabs.org/pipermail/openbmc/2019-December/019728.html
3. https://lists.ozlabs.org/pipermail/openbmc/2019-December/019823.html

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: PECI patchset status

[Ed Tanous]

On Fri, Sep 4, 2020 at 9:36 AM Patrick Williams <patrick@stwcx.xyz> wrote:
>
> On Thu, Sep 03, 2020 at 10:15:56AM -0700, Vernon Mauery wrote:
> > On 03-Sep-2020 10:27 AM, Patrick Williams wrote:
> > >On Thu, Sep 03, 2020 at 05:57:48AM +0000, Mihm, James wrote:
> > >Rather than create a separate fork of the kernel, is there something
> > >that could be done here to have someone from Intel work with Joel on
> > >preparing the patches?  When a new kernel comes out, Joel can ensure it
> > >works on the base AST2xxx system design and before we move all the
> > >systems to it, someone from Intel can rebase the non-upstreamed patches
> > >they are carrying?  This hopefully reduces some of the burden on Joel
> > >and stops us from further fragmenting the community.
> >
> > Keep in mind that Intel does not plan to keep the fork around
> > indefinitely. The hope is to fully upstream all of the patches that we
> > have outstanding. Our intention is not to fragment the community, but to
> > provide a mechanism to continue to move forward while still providing a
> > way for other users to build the intel-platforms target.
> >
> > As an added feature, having our full kernel source in a publicly
> > available tree will allow us to upstream more features that depend on
> > kernel support that is not currently available.
>
> I'm not really following this last paragraph.  I suppose you're saying
> that you have kernel changes that are not in openbmc/linux that add
> additional features?  Why aren't they in openbmc/linux?  I thought there
> was a process for getting code in that isn't quite ready for
> upstreaming, as long as there is progress towards that?  Is there some
> list of what these features are and what the upstreaming state is,
> because this original thread was about PECI, but you're implying there
> is much more.

I pulled down the tree the other day and counted 84 independent
patches.  How many of these have been submitted to upstream?

>
> If the process isn't working for the community shouldn't we discuss
> improving that to something that does work?  It seems like your team has
> decided to go to the nuclear option of forking after Joel has proposed
> dropping a patchset that he's been carrying for three months short of
> three years.
>
> Joel does great work of keeping the kernel up to date, both on a major
> release and picking up supplemental fixes.  Is Intel committing to this
> same level of support that Joel is currently providing for your own
> fork?
>
> Past performance doesn't really give me a lot of confidence that this
> will be a short-term fork.  In December 2019, Joel raised this exact
> same problem with the PECI driver[1] and it was promised that there
> would be forward progress "within a week"[2].  One week later, there was
> a v11 of the patches posted[3] and we got some good comments from a
> variety of upstream maintainers.  Since then, there has been zero
> activity.  Shouldn't we have seen a v12 pretty quickly after that?

+1.  I'm really wondering what's holding this up.

>
> 1. https://lists.ozlabs.org/pipermail/openbmc/2019-December/019684.html
> 2. https://lists.ozlabs.org/pipermail/openbmc/2019-December/019728.html
> 3. https://lists.ozlabs.org/pipermail/openbmc/2019-December/019823.html
>
> --
> Patrick Williams

Public forks (Re: PECI patchset status)

[krtaylor]

On 9/3/20 12:15 PM, Vernon Mauery wrote:
> On 03-Sep-2020 10:27 AM, Patrick Williams wrote:
>> On Thu, Sep 03, 2020 at 05:57:48AM +0000, Mihm, James wrote:
>>> Thank you Joel for carrying this patchset, and Intel does have an 
>>> interest in getting our patchsets upstreamed.
>>>
>>> Since Intel has a large set of patches that need to be upstreamed our 
>>> plan is to fork the kernel in github/intel-bmc and apply the intel 
>>> patchsets. Upstream recipes can then pull the kernel from the intel 
>>> fork with the intel patches. Intel will ensure that this fork tracks 
>>> the openbmc kernel version and maintain the intel patchsets while in 
>>> the process of getting them upstreamed.
>>
>> Rather than create a separate fork of the kernel, is there something
>> that could be done here to have someone from Intel work with Joel on
>> preparing the patches?  When a new kernel comes out, Joel can ensure it
>> works on the base AST2xxx system design and before we move all the
>> systems to it, someone from Intel can rebase the non-upstreamed patches
>> they are carrying?  This hopefully reduces some of the burden on Joel
>> and stops us from further fragmenting the community.
> 
> Keep in mind that Intel does not plan to keep the fork around 
> indefinitely. The hope is to fully upstream all of the patches that we 
> have outstanding. Our intention is not to fragment the community, but to 
> provide a mechanism to continue to move forward while still providing a 
> way for other users to build the intel-platforms target.


In 20+ years of open source development I have never seen a fork like 
this actually improve anything for anyone involved.

It fragments the community and winds up being a PITA for the company 
that forked since they now have to figure out how to reconcile the two 
trees.

As I said from day one with this fork, if there is a problem with the 
speed of the community ("moving forward"), then the problem is probably 
with planning and transparency. If we are not talking about n+1 release 
features, then it is already too late.

Open source really does work, it starts with an commitment from everyone 
to look out for the community first.

All IMHO,
Kurt Taylor (krtaylor)

> As an added feature, having our full kernel source in a publicly 
> available tree will allow us to upstream more features that depend on 
> kernel support that is not currently available.
> 
> --Vernon

RE: PECI patchset status

[Mihm, James]

>-----Original Message-----
>From: Patrick Williams <patrick@stwcx.xyz>
>Sent: Friday, September 4, 2020 9:35 AM
>To: Vernon Mauery <vernon.mauery@linux.intel.com>
>Cc: Mihm, James <james.mihm@intel.com>; OpenBMC Maillist
><openbmc@lists.ozlabs.org>
>Subject: Re: PECI patchset status
>
>On Thu, Sep 03, 2020 at 10:15:56AM -0700, Vernon Mauery wrote:
>> On 03-Sep-2020 10:27 AM, Patrick Williams wrote:
>> >On Thu, Sep 03, 2020 at 05:57:48AM +0000, Mihm, James wrote:
>> >Rather than create a separate fork of the kernel, is there something
>> >that could be done here to have someone from Intel work with Joel on
>> >preparing the patches?  When a new kernel comes out, Joel can ensure it
>> >works on the base AST2xxx system design and before we move all the
>> >systems to it, someone from Intel can rebase the non-upstreamed patches
>> >they are carrying?  This hopefully reduces some of the burden on Joel
>> >and stops us from further fragmenting the community.
>>
>> Keep in mind that Intel does not plan to keep the fork around
>> indefinitely. The hope is to fully upstream all of the patches that we
>> have outstanding. Our intention is not to fragment the community, but to
>> provide a mechanism to continue to move forward while still providing a
>> way for other users to build the intel-platforms target.
>>
>> As an added feature, having our full kernel source in a publicly
>> available tree will allow us to upstream more features that depend on
>> kernel support that is not currently available.
>
>I'm not really following this last paragraph.  I suppose you're saying
>that you have kernel changes that are not in openbmc/linux that add
>additional features?  Why aren't they in openbmc/linux?  I thought there
>was a process for getting code in that isn't quite ready for
>upstreaming, as long as there is progress towards that?  Is there some
>list of what these features are and what the upstreaming state is,
>because this original thread was about PECI, but you're implying there
>is much more.
>
>If the process isn't working for the community shouldn't we discuss
>improving that to something that does work?  It seems like your team has
>decided to go to the nuclear option of forking after Joel has proposed
>dropping a patchset that he's been carrying for three months short of
>three years.
>
>Joel does great work of keeping the kernel up to date, both on a major
>release and picking up supplemental fixes.  Is Intel committing to this
>same level of support that Joel is currently providing for your own
>fork?
>
>Past performance doesn't really give me a lot of confidence that this
>will be a short-term fork.  In December 2019, Joel raised this exact
>same problem with the PECI driver[1] and it was promised that there
>would be forward progress "within a week"[2].  One week later, there was
>a v11 of the patches posted[3] and we got some good comments from a
>variety of upstream maintainers.  Since then, there has been zero
>activity.  Shouldn't we have seen a v12 pretty quickly after that?
>

[James Mihm] 
As a product development team, we need to balance between product deliverables and upstreaming. Where product deliverables and schedules have taken priority over upstreaming.
We are working on a prioritized kernel upstreaming plan and will be sharing with the community after internal reviews. 

Regarding the PECI v11 driver, the kernel maintainers reached out to us with additional requirements before we could proceed with v12. We are taking steps to meet those requirements.

The Intel fork of openbmc/linux is to allow anyone needing to build openbmc for an Intel server and to be able to do so without having to apply patches. Placing the burden of maintaining the 84+ Intel patches against the kernel on ourselves while working through the upstreaming process. Intel would own rebasing the Intel fork with openbmc/linux. The lifespan of this fork would be as short as possible and is dependent upon our execution to upstream our patchsets.

Would it be acceptable for all of the 84+ Intel patches to reside in the openbmc repo while we work through the upstreaming process? 
Some of the patches require design changes and will take much longer to upstream.

James

>1. https://lists.ozlabs.org/pipermail/openbmc/2019-December/019684.html
>2. https://lists.ozlabs.org/pipermail/openbmc/2019-December/019728.html
>3. https://lists.ozlabs.org/pipermail/openbmc/2019-December/019823.html
>
>--
>Patrick Williams

Re: PECI patchset status

[Joel Stanley]

On Fri, 11 Sep 2020 at 01:20, Mihm, James <james.mihm@intel.com> wrote:

> Would it be acceptable for all of the 84+ Intel patches to reside in the openbmc repo while we work through the upstreaming process?
> Some of the patches require design changes and will take much longer to upstream.

This is the intent of the openbmc tree. THe patches need to be posted,
in reviewable series, to be included.

As discussed in this thread regarding the PECI patches, they have been
around for a long time without submission. They will need to be posted
upstream again before I put them in the OpenBMC tree.

Early September, when this thread was started, was a great time to
make the PECI submission in terms of the upstream kernel development
cycle. The next best time is now.

Cheers,

Joel

Re: PECI patchset status

[Jae Hyun Yoo]

Hi Joel,

On 9/28/2020 10:49 PM, Joel Stanley wrote:
> On Fri, 11 Sep 2020 at 01:20, Mihm, James <james.mihm@intel.com> wrote:
> 
>> Would it be acceptable for all of the 84+ Intel patches to reside in the openbmc repo while we work through the upstreaming process?
>> Some of the patches require design changes and will take much longer to upstream.
> 
> This is the intent of the openbmc tree. THe patches need to be posted,
> in reviewable series, to be included.
> 
> As discussed in this thread regarding the PECI patches, they have been
> around for a long time without submission. They will need to be posted
> upstream again before I put them in the OpenBMC tree.
> 
> Early September, when this thread was started, was a great time to
> make the PECI submission in terms of the upstream kernel development
> cycle. The next best time is now.

Thank you for carrying the out-of-tree PECI patch set so far. I really
appreciate it.

I agree with you that the PECI upstreaming should make a progress now
but I'm still in unbalanced task priorities. I'll try to make it done as
soon as possible but it's not available right now.

As James already said, we'll use a forked tree if PECI patches are
dropped off from OpenBMC linux tree v5.9, and it'll not be a permanent
fork but just for a temporary and alternative way to minimize impact of
dropping off the PECI patch set. Thanks for your understanding in
advance.

Best,

Jae