OpenBMC Upstream Reference System(s)

OpenBMC Upstream Reference System(s)

[Andrew Geissler]

Greetings,

One thing that has come up a few times, and was touched on in last weeks
Infrastructure workgroup (and todays community meeting), was what is our
community reference system(s)? i.e. What do we as a community test against
and guarantee works when we do a tag or release?

If you look at something like https://openpower.xyz/job/openbmc-build/ you
will see a variety of systems. Romulus is put through QEMU CI and another,
Witherspoon, is put through HW CI currently. This list has been mostly created
by IBM, and so it brings up the question of what's our future plan here.

First, what's the criteria for adding a system to officially being supported by
the upstream community?
- Covers as much OpenBMC function as possible?
- Is on hardware that anyone can easily (and cheaply) get their hands on?
- Has consistent upstream support and a person or company to support HW CI?

The witherspoon and romulus systems definitely are well supported by IBM in
the upstream community but they are not all that cheap. There was a rasberry
pi port out there that would be cheap and easy to get access to but not cover
as much OpenBMC function. There's something like
https://www.raptorcs.com/content/TLSDS3/intro.html which is a bit more
accessible but I don't believe the code has been upstreamed.

One point made in last weeks Infrastructure workgroup is we should shoot
for a minimum amount of hardware that covers as much function as possible.
So a matrix of some sort that covers our major functional areas. In general,
having a system on the upstream support list is like getting your code
upstreamed,
once in, you have a guarantee that all future code will support it, so maybe
this is more of a community reward to get your system on the list?

Just throwing ideas out there, input and ideas appreciated.

Andrew

References:
https://github.com/openbmc/openbmc/wiki/OpenBMC-Infrastructure-Workgroup

Re: OpenBMC Upstream Reference System(s)

[Khetan, Sharad]

Hi Andrew, 
I would like to see (and will support) an Intel platform. It may be Wolf Pass or something else. I am not sure if the requirements and expectations of supporting such a platform. 
I will discuss internally and get back. 
Thanks,
-Sharad

> On Dec 11, 2018, at 12:36 AM, Andrew Geissler <geissonator@gmail.com> wrote:
> 
> Greetings,
> 
> One thing that has come up a few times, and was touched on in last weeks
> Infrastructure workgroup (and todays community meeting), was what is our
> community reference system(s)? i.e. What do we as a community test against
> and guarantee works when we do a tag or release?
> 
> If you look at something like https://openpower.xyz/job/openbmc-build/ you
> will see a variety of systems. Romulus is put through QEMU CI and another,
> Witherspoon, is put through HW CI currently. This list has been mostly created
> by IBM, and so it brings up the question of what's our future plan here.
> 
> First, what's the criteria for adding a system to officially being supported by
> the upstream community?
> - Covers as much OpenBMC function as possible?
> - Is on hardware that anyone can easily (and cheaply) get their hands on?
> - Has consistent upstream support and a person or company to support HW CI?
> 
> The witherspoon and romulus systems definitely are well supported by IBM in
> the upstream community but they are not all that cheap. There was a rasberry
> pi port out there that would be cheap and easy to get access to but not cover
> as much OpenBMC function. There's something like
> https://www.raptorcs.com/content/TLSDS3/intro.html which is a bit more
> accessible but I don't believe the code has been upstreamed.
> 
> One point made in last weeks Infrastructure workgroup is we should shoot
> for a minimum amount of hardware that covers as much function as possible.
> So a matrix of some sort that covers our major functional areas. In general,
> having a system on the upstream support list is like getting your code
> upstreamed,
> once in, you have a guarantee that all future code will support it, so maybe
> this is more of a community reward to get your system on the list?
> 
> Just throwing ideas out there, input and ideas appreciated.
> 
> Andrew
> 
> References:
> https://github.com/openbmc/openbmc/wiki/OpenBMC-Infrastructure-Workgroup

Re: OpenBMC Upstream Reference System(s)

[Joseph Reynolds]

On 2018-12-10 13:05, Andrew Geissler wrote:
> Greetings,
> 
> One thing that has come up a few times, and was touched on in last 
> weeks
> Infrastructure workgroup (and todays community meeting), was what is 
> our
> community reference system(s)? i.e. What do we as a community test 
> against
> and guarantee works when we do a tag or release?
> 
> If you look at something like https://openpower.xyz/job/openbmc-build/ 
> you
> will see a variety of systems. Romulus is put through QEMU CI and 
> another,
> Witherspoon, is put through HW CI currently. This list has been mostly 
> created
> by IBM, and so it brings up the question of what's our future plan 
> here.
> 
> First, what's the criteria for adding a system to officially being 
> supported by
> the upstream community?
> - Covers as much OpenBMC function as possible?
> - Is on hardware that anyone can easily (and cheaply) get their hands 
> on?
> - Has consistent upstream support and a person or company to support HW 
> CI?


At today's Infrastructure Workgroup I mentioned four levels of machine 
support, where each level builds on the previous level, providing a path 
to get more support for your machine.  Here are my ideas, provided for 
discussion:

1. Source code - The machine has a repo in the openbmc organization.  
DETAILS: The machine has an github.com/openbmc/meta-MACHINE repository 
with a BitBake layer configuration which defines the meta-phosphor layer 
and can build the obmc-phosphor-image.  Expectations are that the code 
builds and is maintained, e.g., maintainers respond to a call for 
maintenance.

2. CI Builds - The openbmc-phosphor-image is built for the machine as 
part of Jenkins CI testing, and maintainers address failures in their 
layer.  DETAILS: Build failures will result in a -1 Gerrit code review 
score, which may prevent merging.  Because the image costs CPU cycles to 
build (one build per Gerrit patch set), is there an expectation that the 
team behind the machine provides a similar amount of value to the 
project?  Examples: donate the use of servers to the project, or have 
significant contributions to common functions, for example, 
meta-phosphor.

3. CI Tested - The machine is booted and tested as part of the Jenkins 
CI testing, and maintainers address failures in their layer.  DETAILS: 
Testcase failures will result in a -1 Gerrit code review score, which 
may prevent merging.  The expectation is the sponsoring organization 
will donate the use of physical hardware which they may host behind 
their company firewall.

4. Reference platform - The machine is an OpenBMC reference platform.  
DETAILS: I assume the TSC would maintain the list of reference platforms 
based on community input.  I see one criterion (CI Tested) and several 
factors in the TSC's decision:
  - The machine has active development (git commits) and good support (CI 
failures are addressed).
  - The machine can be obtained, e.g., purchased.
  - The machine has a simulator, e.g., QEMU.
  - The machine is different than existing reference platforms.

What value is being a reference platform beyond being tested by CI?  
Marketing?

- Joseph

> The witherspoon and romulus systems definitely are well supported by 
> IBM in
> the upstream community but they are not all that cheap. There was a 
> rasberry
> pi port out there that would be cheap and easy to get access to but not 
> cover
> as much OpenBMC function. There's something like
> https://www.raptorcs.com/content/TLSDS3/intro.html which is a bit more
> accessible but I don't believe the code has been upstreamed.
> 
> One point made in last weeks Infrastructure workgroup is we should 
> shoot
> for a minimum amount of hardware that covers as much function as 
> possible.
> So a matrix of some sort that covers our major functional areas. In 
> general,
> having a system on the upstream support list is like getting your code
> upstreamed,
> once in, you have a guarantee that all future code will support it, so 
> maybe
> this is more of a community reward to get your system on the list?
> 
> Just throwing ideas out there, input and ideas appreciated.
> 
> Andrew
> 
> References:
> https://github.com/openbmc/openbmc/wiki/OpenBMC-Infrastructure-Workgroup

Re: OpenBMC Upstream Reference System(s)

[Andrew Jeffery]

.
> 
> What value is being a reference platform beyond being tested by CI?  
> Marketing?

In my opinion is less about marketing and more about enabling potential consumers/contributors to OpenBMC evaluate whether it's fit for their purpose. A reference platform is a platform that we can recommend that will best enable them to make that decision.

Ideally it should be cheap and incorporate a BMC and host system (i.e. the RaspberryPi port doesn't really meet the bar), and allow the user to exercise most of all of OpenBMC's features

Andrew