Re: Xen and safety certification, Minutes of the meeting on Apr 4th

[Stefano Stabellini <sstabellini@kernel.org>]

On Fri, 6 Apr 2018, Artem Mygaiev wrote:
> > > > 2) Create a subset of functions that need to go through certifications
> > > > Next step: create a small Kconfig. We could use the Renesas Rcar as
> > > > reference. We need a discussion about the features we need, for
> > > > example real-time schedulers, do we need them or not?
> > > 
> > 
> > Identifying this subset is very important. My recommendation would be to
> > identify the very smallest subset to start with that supports a single, high
> > value use case, which I would suggest is consolidation of Linux and
> > real-time applications with mixed criticality, but not necessarily shared/PV
> > I/O, onto a single processing cluster. Identifying the highest reasonable
> > safety criticality to support would also be very helpful.
> > 
> 
> Unfortunately in mixed criticality systems (at least in automotive) we see a
> lot of attention to performance and , so processing cluster partitioning may
> not be well accepted in the industry

Sorry, I didn't quite understand your comment. Are you saying that
statically partitioning a cluster into VMs, for example with
vcpu-pinning or the null scheduler, in a way to have a total number of
vcpus equal to the total number of pcpus, is not acceptable because it
leads to lower hardware utilization? We need nr_vcpus > nr_pcpus?


> > At the Xen level, you might get away with just the null scheduler if VMs are
> > pinned to their own cores (and jitter caused by contention on the bus and in
> > the cache is acceptable). However, to do CAST-32a type scheduling
> > (effectively time slicing the SoC between your VMs), an updated ARINC-653
> > scheduler would be needed.
> > 
> 
> We are now looking into RTDS as a possible solution for industrial or
> automotive domains. Also , from our experience bus/cache contention in systems
> with high load is actually an issue... Looking into that, too

Bus/cache contention is where issues can become very board specific. It
is also why we'll need to narrow down a small set of boards initially.


> > > 
> > > @Stefano agreed to drive this.
> > > The minimal configuration does impact 1 and 2, which is why I moved this
> > > first.
> > > 
> > > We should probably agree a basic process: aka
> > > * Measure baseline size in KSLOC
> > > * Remove some feature
> > > * Measure reduction in KSLOC
> > > And record the data somewhere

I am happy to drive the discussion. I was already planning to submit a
small kconfig and a LOC counter to the Xen build. I wrote down my name
on the wikipage next to this item.

I understand that good real-time support is critical in the provided
configuration. I am happy to work with others to help improve it.



> > > > 1) Requirements to the code, a subset of MISRA for ASIL B Next step:
> > > > get more information about requirements and publish it to xen-devel.
> > > 
> > > I see a few problems here:
> > > 
> > > * The MISCRA 2012 spec has to be bought and it is rather big (100's of
> > > pages):
> > > so, I don't think it is practical to work from the spec
> > > 
> > > * Some coding style patterns will likely be perceived as odd and
> > > unreasonable by community members: as some common code would be
> > > affected we cannot treat this in isolation say on ARM only. Although it is
> > > recognized that some of the coding style patterns may not make sense,
> > > compliance to MISRA is necessary and cannot normally be discussed away.
> > > 
> > > * PRQA has set up an environment and initial MISRA compliance report for
> > > a Xen on ARM build
> > > ** The question is what (if anything) can be shared publicly
> > > ** The other open question is whether we can come to some sort of longer
> > > term agreement between the Xen Project and PRQA to use their tools
> > > ** As an aside, what PRQA have done would need to reflect what we do in
> > > step 2 is. We also want to minimize the work for PRQA: in other words, it
> > > has to be very simple to enable the minimal config coming out of task 2
> > > such that PRQA can
> > > ** As far as I recall 90% of all MISRA violations come down to around 70
> > > issues. A large number are in tools
> > > ** Also, I believe that MISRA compliance tools will likely lead to a large
> > > amount of false positives, due to the distributed nature of Xen: process
> > > boundaries, kernel/user space boundaries, etc. would all lead to false
> > > positives, which somehow have to be managed.
> > > 
> > > ACTION => Lars to follow up with Paul Luperto from PRQA
> > > 
> > > * An approach that may be manageable would be to look at the most
> > > common MISRA violations and work backwards from there.
> > > ** This would make the problem more manageable and mean people
> > > wouldn't have to read a long spec
> > > ** Discussing a small set of issues, would give us a sense of whether/what
> > > type of disagreements there are and how we resolve them.
> > > ** We should focus prioritize based on:
> > > a) Address/discuss the most frequently occurring issues first
> > > b) Address/discuss issues in common code first
> > > 
> > > At the very least (and for now in absence of the capability to check
> > > compliance), I would need someone who has access to MISRA compliance
> > > tools, to drive such an effort.

I wrote "Lars" near this item in
https://wiki.xenproject.org/wiki/Safety_Certification_Challenges, just
as a reference to where the ball is at the moment.



> > > > 3) Understand how to address dom0. FreeRTOS Dom0 sounds like a good
> > > > solution.
> > > > Next step: reach out to Dornerworks and/or others that worked with
> > > > FreeRTOS on Xen before. Figure out whether FreeRTOS is actually a
> > > > suitable solution and what needs to be done to run FreeRTOS as Dom0.
> > > 
> > > Some things to check at this stage:
> > > a) I believe there is a safety certified version of FreeRTOS - I could not
> > > find
> > > much, except for https://www.freertos.org/FreeRTOS-
> > > Plus/Safety_Critical_Certified/SafeRTOS-Safety-Critical-Certification.shtml
> > > -
> > > which describes SafeRTOS a commercial safety certified FreeRTOS and
> > > (mostly) API compliant version of FreeRTOS. Or am I missing something
> > > here?
> > > b) There is a DomU capable version from Galois (Jonathan Docherty CC'ed) -
> > > I don't know whether others also have such versions
> > 
> > I ported the version of FreeRTOS that Xilinx distributes with their SDK to
> > run as a domU on the ZUS+ in 2016 and round tripped the change set back to
> > Richard Barry.
> > I've also heard interest in running RTEMS as a guest OS.
> > 
> 
> We've had experience in running QNX in domu, but that was not very welcomed by
> BB QSSL folks back then :) They dont really like OSS
> 
> > Since I do not think that a previously certified OS will be available for
> > free, I see 3 general approaches wrt dom0:
> > 1) Find and certify an open source OS. My guess is this will not be Linux
> > due to code base size. POSIX support a plus.
> > 2) Use a commercially available, previously certified OS for dom0. DW ported
> > VxWorks to run on Xen in 2017 and uc/OS-III in 2016.
> > 3) Go with a dom0-less solution; bootloader starts up the necessary VMs
> > based on a static configuration.
> > 
> > The XL toolstack in its current form will likely cause cert issues and will
> > probably need to be stripped down and/or rewritten.
> > Bootloader (U-Boot, GRUB, or whatever) will also need to be certified.
> > 
> 
> We'd like to explore both FreeRTOS in dom0 and dom0-less options. I think
> there were some patches while ago for dom0-less xen.

"Dom0-less" is a great name actually :-)

Up until now, we discussed this topic under the name of "create multiple
guests from device tree". There are no patches (as far as I know), but
it was submitted as the Xen on ARM project for Outreachy this year.
There are patches for a different project to setup shared memory regions
from the xl config file (no need for grant table or xenbus support).


> We plan to analyze efforts to port FreeRTOS as dom0 OS

Great! I think it makes sense to start from that. I wrote "Artem" down
in the wikipage
(https://wiki.xenproject.org/wiki/Safety_Certification_Challenges) as
the reference contact for the dom0 stuff. Keep us in the loop as Julien
and I are very interested in it.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel