All of lore.kernel.org
 help / color / mirror / Atom feed
* Reducing fragmentation in OpenBMC
@ 2020-05-15  9:03 Jeremy Kerr
  2020-05-15 10:45 ` Jeremy Kerr
  2020-05-19  0:53 ` Andrew Geissler
  0 siblings, 2 replies; 10+ messages in thread
From: Jeremy Kerr @ 2020-05-15  9:03 UTC (permalink / raw)
  To: openbmc

[-- Attachment #1: Type: text/plain, Size: 3357 bytes --]

Hi OpenBMCers,

I've been working on some different areas of OpenBMC recently, and have
noticed that there seems to be increasing fragmentation between various
platform and infrastructure code.

One of the main impacts I've seen as a consequence is that it's
becoming incredibly difficult for new adopters and contributors to do a
platform port.

I definitely understand that contributing companies have their own
product timelines and priorities, which doesn't always correlate with
OpenBMC development directions. We need to allow that for the project
as a whole to be viable. But I still think there's scope to both
improve the coherence of the OpenBMC work, while also allowing variance
where justified, in a way that makes sense.

To pick a couple of examples and consequences:

 - there are a lot of out-of-tree patches around. While this isn't
necessarily a problem in its own right, some of these are fundamental
to make the upstream platforms work. For anyone adopting those
platforms as a reference, it means that they have a non-functional
platform, or have to use the non-upstream repos.

 - there's increasing amounts of code that require or implement non-
standard behaviour; For example, dbus-sensors exposes and consumes dbus
interfaces that are not described by phosphor-dbus-interfaces. This
means that the divergence is then needed in other projects too. If
those standards don't suit actual requirements, then we should look at
updating them.

Just to be 100% clear though, I am definitely not singling-out the
meta-intel platform support; it's just what I've been experimenting
with recently. Intel have done great upstream work on OpenBMC, and
these issues are only apparent because of the work already done. It
just feels like we're 90% there, and the rest would make the world of
difference for new users.

So, a few things that I think may help the situation:

 - Adherence to standards. Being a little more strict about what
comprises an OpenBMC implementation will go a long way to prevent
future incompatibilities, and means that all of our interface
implementations automatically document their expected behaviour
(because that's in the standard).

 - Identification of a set of reference platforms. If we can point
adopters to a platform that provides a recommended (and somewhat
"supported") way of doing things, that would prevent a lot of confusion
around different implementations, and how to best work with the options
available

 - Documentation of interactions between components. There's some great
documentation on how our components work, but not a whole lot on how
they fit together. Without this, it means a lot of jumping around
between repos, trying to find the "other side" of each interface.

 - Keep pushing on upstream. Sometimes this can delay things, but I
really think that's almost always false economy; the out-of-tree
patches will have to be addressed at some point, and that job just gets
more involved as time passes. Even engaging other community members to
help out would be great.

Finally, I don't mean this to sound like a bunch of complaints - I'm
keen to put in a bunch of time to address things. I'd just like to
start some discussion on how best to do that, before I do so.

So - any thoughts on how we can improve this? Comments / disagreements
/ questions always welcome.

Cheers,


Jeremy

[-- Attachment #2: Type: text/html, Size: 8677 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Reducing fragmentation in OpenBMC
  2020-05-15  9:03 Reducing fragmentation in OpenBMC Jeremy Kerr
@ 2020-05-15 10:45 ` Jeremy Kerr
  2020-05-15 18:17   ` Jae Hyun Yoo
  2020-05-19  0:53 ` Andrew Geissler
  1 sibling, 1 reply; 10+ messages in thread
From: Jeremy Kerr @ 2020-05-15 10:45 UTC (permalink / raw)
  To: openbmc

Hi all,

>  - Keep pushing on upstream. Sometimes this can delay things, but I
> really think that's almost always false economy; the out-of-tree
> patches will have to be addressed at some point, and that job just 
> gets more involved as time passes.

One of the lagging items here is the amount of kernel patches pending
in:

 
https://github.com/Intel-BMC/openbmc/tree/intel/meta-openbmc-mods/meta-common/recipes-kernel/linux/linux-aspeed

Intel folks: any objections if I grab select patches from there and
work on the upstreaming process? Or are you already working on this?

Regards,


Jeremy

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Reducing fragmentation in OpenBMC
  2020-05-15 10:45 ` Jeremy Kerr
@ 2020-05-15 18:17   ` Jae Hyun Yoo
  2020-05-15 21:07     ` Vijay Khemka
  0 siblings, 1 reply; 10+ messages in thread
From: Jae Hyun Yoo @ 2020-05-15 18:17 UTC (permalink / raw)
  To: openbmc

Hi Jeremy,

On 5/15/2020 3:45 AM, Jeremy Kerr wrote:
> Hi all,
> 
>>   - Keep pushing on upstream. Sometimes this can delay things, but I
>> really think that's almost always false economy; the out-of-tree
>> patches will have to be addressed at some point, and that job just
>> gets more involved as time passes.
> 
> One of the lagging items here is the amount of kernel patches pending
> in:
> 
>   
> https://github.com/Intel-BMC/openbmc/tree/intel/meta-openbmc-mods/meta-common/recipes-kernel/linux/linux-aspeed
> 
> Intel folks: any objections if I grab select patches from there and
> work on the upstreaming process? Or are you already working on this?
> 

First of all, thank you for your initiating this discussion. Obviously,
it should be done but I couldn't put enough time to make it.

I don't have any objection on that. It would be really helpful if you
grab patches from there for upstreaming. Please let me know if you need
any help or clarification from me or from Intel during the upstreaming.

Thanks,

Jae

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Reducing fragmentation in OpenBMC
  2020-05-15 18:17   ` Jae Hyun Yoo
@ 2020-05-15 21:07     ` Vijay Khemka
  0 siblings, 0 replies; 10+ messages in thread
From: Vijay Khemka @ 2020-05-15 21:07 UTC (permalink / raw)
  To: Jae Hyun Yoo, openbmc



On 5/15/20, 11:19 AM, "openbmc on behalf of Jae Hyun Yoo" <openbmc-bounces+vijaykhemka=fb.com@lists.ozlabs.org on behalf of jae.hyun.yoo@linux.intel.com> wrote:

    Hi Jeremy,
    
    On 5/15/2020 3:45 AM, Jeremy Kerr wrote:
    > Hi all,
    > 
    >>   - Keep pushing on upstream. Sometimes this can delay things, but I
    >> really think that's almost always false economy; the out-of-tree
    >> patches will have to be addressed at some point, and that job just
    >> gets more involved as time passes.
    > 
    > One of the lagging items here is the amount of kernel patches pending
    > in:
    > 
    >   
    > https://github.com/Intel-BMC/openbmc/tree/intel/meta-openbmc-mods/meta-common/recipes-kernel/linux/linux-aspeed
    > 
    > Intel folks: any objections if I grab select patches from there and
    > work on the upstreaming process? Or are you already working on this?
    > 
    
    First of all, thank you for your initiating this discussion. Obviously,
    it should be done but I couldn't put enough time to make it.
    
    I don't have any objection on that. It would be really helpful if you
    grab patches from there for upstreaming. Please let me know if you need
    any help or clarification from me or from Intel during the upstreaming.
    
    Thanks,
    
    Jae
    
If there are any other feature patches available in Intel-BMC tree which can be 
upstreamed to openbmc repo, please let community know so anyone who has
bandwidth can grab those patches and upstream it rather than making a copy
to their local repo.


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Reducing fragmentation in OpenBMC
  2020-05-15  9:03 Reducing fragmentation in OpenBMC Jeremy Kerr
  2020-05-15 10:45 ` Jeremy Kerr
@ 2020-05-19  0:53 ` Andrew Geissler
  2020-05-19  2:25   ` 郁雷
  1 sibling, 1 reply; 10+ messages in thread
From: Andrew Geissler @ 2020-05-19  0:53 UTC (permalink / raw)
  To: Jeremy Kerr; +Cc: openbmc



> On May 15, 2020, at 4:03 AM, Jeremy Kerr <jk@ozlabs.org> wrote:
> 
> So, a few things that I think may help the situation:
> 
>  - Adherence to standards. Being a little more strict about what
> comprises an OpenBMC implementation will go a long way to prevent
> future incompatibilities, and means that all of our interface
> implementations automatically document their expected behaviour
> (because that's in the standard).

I know phosphor-dbus-interfaces has always been a bit onerous. I do feel like
we get some good stuff in the reviews though. It also ensures we have
documentation  of our interfaces. The cross-repo dependencies we
get are a bit frustrating (i.e. need to get interface merged and bubbled into
openbmc before you can implement). There’s also no versioning concept so
if an interface needs to be changed, it’s a huge pain. Ideas on how we could
make this easier but keep the benefits? Or people that don’t use it and just
define their own interfaces, any improvements we could make to get
you to use it?

> 
>  - Identification of a set of reference platforms. If we can point
> adopters to a platform that provides a recommended (and somewhat
> "supported") way of doing things, that would prevent a lot of confusion
> around different implementations, and how to best work with the options
> available

This is definitely a big one. I’m not sure the solution though. There’s a
divergence between platforms. I know there’s some effort to to converge a bit
(entity manager usage) but  we seem to still be going through that phase where
we have multiple implementations of things and we’ve just got to let them work
themselves out. That can be confusing to new people coming in. A lack of an
affordable reference platform we can point people to is something that comes up
often in the community.

> 
>  - Documentation of interactions between components. There's some great
> documentation on how our components work, but not a whole lot on how
> they fit together. Without this, it means a lot of jumping around
> between repos, trying to find the "other side" of each interface.
> 
>  - Keep pushing on upstream. Sometimes this can delay things, but I
> really think that's almost always false economy; the out-of-tree
> patches will have to be addressed at some point, and that job just gets
> more involved as time passes. Even engaging other community members to
> help out would be great.
> 
> Finally, I don't mean this to sound like a bunch of complaints - I'm
> keen to put in a bunch of time to address things. I'd just like to
> start some discussion on how best to do that, before I do so.

Good topics and thanks for starting the discussion Jeremy!

> 
> So - any thoughts on how we can improve this? Comments / disagreements
> / questions always welcome.
> 
> Cheers,
> 
> 
> Jeremy

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Reducing fragmentation in OpenBMC
  2020-05-19  0:53 ` Andrew Geissler
@ 2020-05-19  2:25   ` 郁雷
  2020-05-19 12:50     ` Brad Bishop
  0 siblings, 1 reply; 10+ messages in thread
From: 郁雷 @ 2020-05-19  2:25 UTC (permalink / raw)
  To: Andrew Geissler; +Cc: Jeremy Kerr, openbmc

On Tue, May 19, 2020 at 8:53 AM Andrew Geissler <geissonator@gmail.com> wrote:

> I know phosphor-dbus-interfaces has always been a bit onerous. I do feel like
> we get some good stuff in the reviews though. It also ensures we have
> documentation  of our interfaces. The cross-repo dependencies we
> get are a bit frustrating (i.e. need to get interface merged and bubbled into
> openbmc before you can implement). There’s also no versioning concept so
> if an interface needs to be changed, it’s a huge pain. Ideas on how we could
> make this easier but keep the benefits? Or people that don’t use it and just
> define their own interfaces, any improvements we could make to get
> you to use it?
>

This usually involves the repo CI.
If we could implement "Cross-repo dependencies", making the Jenkins
job able to pick the "dependent" revision of phosphor-dbus-interfaces
(or sdbusplus, or else), and build a docker container with the
dependencies to run the repo CI, the issue could be resolved.
For example:
* A change in phosphor-dbus-interfaces is submitted to gerrit with
revision `abcd` and Change-Id `wxyz`, which is not yet merged;
* A change in repo is submitted to gerrit, with commit message
containing `Depends-On: wxyz`

The Jenkins repo CI job needs to figure out that this commit depends
on `wxyz`, which is phosphor-dbus-interfaces.
Then it needs to build the container with `abcd` revision of
phosphor-dbus-interfaces, and run the repo CI.

It requires non-trivial changes to the Jenkins job though.

-- 
BRs,
Lei YU

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Reducing fragmentation in OpenBMC
  2020-05-19  2:25   ` 郁雷
@ 2020-05-19 12:50     ` Brad Bishop
  2020-05-20  2:25       ` 郁雷
  0 siblings, 1 reply; 10+ messages in thread
From: Brad Bishop @ 2020-05-19 12:50 UTC (permalink / raw)
  To: 郁雷, Andrew Geissler; +Cc: openbmc

On Tue, 2020-05-19 at 10:25 +0800, 郁雷 wrote:
> On Tue, May 19, 2020 at 8:53 AM Andrew Geissler <
> geissonator@gmail.com> wrote:
> 
> > I know phosphor-dbus-interfaces has always been a bit onerous. I do
> > feel like
> > we get some good stuff in the reviews though. It also ensures we
> > have
> > documentation  of our interfaces. The cross-repo dependencies we
> > get are a bit frustrating (i.e. need to get interface merged and
> > bubbled into
> > openbmc before you can implement). There’s also no versioning
> > concept so
> > if an interface needs to be changed, it’s a huge pain. Ideas on how
> > we could
> > make this easier but keep the benefits? Or people that don’t use it
> > and just
> > define their own interfaces, any improvements we could make to get
> > you to use it?
> > 
> 
> This usually involves the repo CI.
> If we could implement "Cross-repo dependencies", making the Jenkins
> job able to pick the "dependent" revision of phosphor-dbus-interfaces
> (or sdbusplus, or else), and build a docker container with the
> dependencies to run the repo CI, the issue could be resolved.

This would be a nice feature to have in our CI when cross repo
dependencies come up.  But I don't think  having that would give us
free license to add cross repo dependencies everywhere though - I would
like to see us come up with a system that avoids the need for cross-
repo dependencies in the first place.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Reducing fragmentation in OpenBMC
  2020-05-19 12:50     ` Brad Bishop
@ 2020-05-20  2:25       ` 郁雷
  2020-05-20 20:06         ` Vernon Mauery
  0 siblings, 1 reply; 10+ messages in thread
From: 郁雷 @ 2020-05-20  2:25 UTC (permalink / raw)
  To: Brad Bishop; +Cc: Andrew Geissler, openbmc

On Tue, May 19, 2020 at 8:50 PM Brad Bishop <bradleyb@fuzziesquirrel.com> wrote:
>
> On Tue, 2020-05-19 at 10:25 +0800, 郁雷 wrote:
> > On Tue, May 19, 2020 at 8:53 AM Andrew Geissler <
> > geissonator@gmail.com> wrote:
> >
> > > I know phosphor-dbus-interfaces has always been a bit onerous. I do
> > > feel like
> > > we get some good stuff in the reviews though. It also ensures we
> > > have
> > > documentation  of our interfaces. The cross-repo dependencies we
> > > get are a bit frustrating (i.e. need to get interface merged and
> > > bubbled into
> > > openbmc before you can implement). There’s also no versioning
> > > concept so
> > > if an interface needs to be changed, it’s a huge pain. Ideas on how
> > > we could
> > > make this easier but keep the benefits? Or people that don’t use it
> > > and just
> > > define their own interfaces, any improvements we could make to get
> > > you to use it?
> > >
> >
> > This usually involves the repo CI.
> > If we could implement "Cross-repo dependencies", making the Jenkins
> > job able to pick the "dependent" revision of phosphor-dbus-interfaces
> > (or sdbusplus, or else), and build a docker container with the
> > dependencies to run the repo CI, the issue could be resolved.
>
> This would be a nice feature to have in our CI when cross repo
> dependencies come up.  But I don't think  having that would give us
> free license to add cross repo dependencies everywhere though - I would
> like to see us come up with a system that avoids the need for cross-
> repo dependencies in the first place.

As Andrew indicates, phosphor-dbus-interfaces is the major cross repo
dependency already.

-- 
BRs,
Lei YU

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Reducing fragmentation in OpenBMC
  2020-05-20  2:25       ` 郁雷
@ 2020-05-20 20:06         ` Vernon Mauery
  0 siblings, 0 replies; 10+ messages in thread
From: Vernon Mauery @ 2020-05-20 20:06 UTC (permalink / raw)
  To: 郁雷; +Cc: Brad Bishop, openbmc

On 20-May-2020 10:25 AM, 郁雷 wrote:
>On Tue, May 19, 2020 at 8:50 PM Brad Bishop <bradleyb@fuzziesquirrel.com> wrote:
>>
>> On Tue, 2020-05-19 at 10:25 +0800, 郁雷 wrote:
>> > On Tue, May 19, 2020 at 8:53 AM Andrew Geissler <
>> > geissonator@gmail.com> wrote:
>> >
>> > > I know phosphor-dbus-interfaces has always been a bit onerous. I do
>> > > feel like
>> > > we get some good stuff in the reviews though. It also ensures we
>> > > have
>> > > documentation  of our interfaces. The cross-repo dependencies we
>> > > get are a bit frustrating (i.e. need to get interface merged and
>> > > bubbled into
>> > > openbmc before you can implement). There’s also no versioning
>> > > concept so
>> > > if an interface needs to be changed, it’s a huge pain. Ideas on how
>> > > we could
>> > > make this easier but keep the benefits? Or people that don’t use it
>> > > and just
>> > > define their own interfaces, any improvements we could make to get
>> > > you to use it?
>> > >
>> >
>> > This usually involves the repo CI.
>> > If we could implement "Cross-repo dependencies", making the Jenkins
>> > job able to pick the "dependent" revision of phosphor-dbus-interfaces
>> > (or sdbusplus, or else), and build a docker container with the
>> > dependencies to run the repo CI, the issue could be resolved.
>>
>> This would be a nice feature to have in our CI when cross repo
>> dependencies come up.  But I don't think  having that would give us
>> free license to add cross repo dependencies everywhere though - I would
>> like to see us come up with a system that avoids the need for cross-
>> repo dependencies in the first place.
>
>As Andrew indicates, phosphor-dbus-interfaces is the major cross repo
>dependency already.

Would it be possible to have the D-Bus interfaces in the repo that 
provides it? If the yaml->c++ generator is used (sdbusplus) then it 
could be run IN that repo as part of the build. If the yaml->c++ 
generator is not used (sdbusplus-asio) then maybe we could figure out 
some auto-generated unit tests that validate that the provider actually 
does serve the interface as it is defined in the yaml.

This would make it harder to find where the interface is, but it would 
reduce the dependency problem. Also, it would make it difficult for 
multiple things that provide the same interface.

Just throwing ideas out there.

--Vernon

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Reducing fragmentation in OpenBMC
@ 2020-05-15 10:21 郁雷
  0 siblings, 0 replies; 10+ messages in thread
From: 郁雷 @ 2020-05-15 10:21 UTC (permalink / raw)
  To: Jeremy Kerr; +Cc: openbmc

On Fri, May 15, 2020 at 5:04 PM Jeremy Kerr <jk@ozlabs.org> wrote:
>
> So - any thoughts on how we can improve this? Comments / disagreements
> / questions always welcome.

I am all in for making the effort to upstream the separated projects.
E.g. one candidate step could be to upstream the static flash layout
and the code update in Intel-BMC/openbmc, which uses a different
layout and different update method than the upstream.
It seems to uses a single "big" chip to support A/B side of BMC
firmware, and at runtime, the rofs is mounted in ram so it could
safely update the rofs at runtime. All of these are good features that
are not implemented in OpenBMC upstream.

If one of the Intel developers could provide some initial patches, I
could test and work on shepherding those upstream.

--
BRs,
Lei YU

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2020-05-20 20:06 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-15  9:03 Reducing fragmentation in OpenBMC Jeremy Kerr
2020-05-15 10:45 ` Jeremy Kerr
2020-05-15 18:17   ` Jae Hyun Yoo
2020-05-15 21:07     ` Vijay Khemka
2020-05-19  0:53 ` Andrew Geissler
2020-05-19  2:25   ` 郁雷
2020-05-19 12:50     ` Brad Bishop
2020-05-20  2:25       ` 郁雷
2020-05-20 20:06         ` Vernon Mauery
2020-05-15 10:21 郁雷

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.