[Ksummit-discuss] Self nomination

[Ksummit-discuss] Self nomination

[Darren Hart]

I'd like to self nominate for this kernel summit this year.

I'm the platform-drivers-x86 maintainer, and am interested in discussions
regarding:

  - maintainer group process and tooling
  - easing the non-kernel-code barriers to contribution
    (eliminating "mechanical failures")

I'm involved with various efforts around Real-Time and Safety Critical Linux and
would be interested in discussions around:

  - Upstreaming PREEMPT_RT
  - Developing a "safety culture" and any overlap that may have with security

Finally, as part of my platform enabling responsibilities, I'm interested in any
developments in the stable process.

Regards,

-- 
Darren Hart
Intel Open Source Technology Center

Re: [Ksummit-discuss] Self nomination

[Linus Walleij]

On Wed, Jul 27, 2016 at 6:46 AM, Darren Hart <dvhart@infradead.org> wrote:

>   - Developing a "safety culture" and any overlap that may have with security

Do you mean safety as in "Linux in airbags and smoke detectors"?

This area is interesting for GPIO and IIO as well for natural reasons:
these systems all tend to use GPIO and sensors. (Albeit more often
than not with some horrific userspace hodge-podge but this is not the
time to be grumpy about that.)

This presentation appeared at LinuxCon Japan (got the link from
my colleague Takahiro Akashi):

Qualifying Linux for Functional Safety
http://events.linuxfoundation.org/sites/events/files/slides/20160713_SIL2LinuxMP_Min_ALS_0.9_pub.pdf

I have been in contact with a Swedish company working in the fire alarm
business as well.

My overall feeling is that world regulations (standards) on safety-critical
software seem to be centered around code inspection.

These regulations approach is not to trust any third party but have all
code inspected by independent reviewers for functional safety. All the
time. For every new deployment. Kernel, libc, busybox userspace.
(Or whatever they use.)

Thus Hitachi developed this minimizing stripping out all not-compiled code
and #ifdefs etc too to get down the code size they have to manually inspect
to a minimum. (It easily translates into work hours.)

My loose thoughts on the issue is twofold:

- We will have an influx of professional safety reviewers that do not
  share their review comments with us, instead apply fixes locally and
  not upstream. This is potentially dangerous if the next reviewer for
  a safety-critical system misses the same bug. (Not to say unethical
  vs the community but I have come to understand that some people
  out there do not care about that.) So we need to send a message to
  the safety-critical industry that any issues found in such safety
  inspections need to go upstream pronto. No vendor tree:ing of this.

- Can we record external inspection-only code reviews done by these
  independent code reviewers (post-minimization) into the kernel (etc) git?
  That I guess is pretty useful for building formal trust for the code,
  but I never heard of git annotations to some random code lines like
  that.

- Should minimization be a part of the kernel standard tooling for use
  cases like this?

Incidentally that may overlap with the footprint minimizing goal: if you can
configure code out (such as the modular syscalls things that Nico has
been working on), that makes this kind of code minimization easier and
may employ similar tooling.

Yours,
Linus Walleij

[Ksummit-discuss] Linux and Safety Industry reviews, was: Self nomination

[Jason Cooper]

Linus, Darren,

I'm also tangentially interested in this discussion from a security point
of view.  Failing in a deterministic way is a desirable trait in both
the security and safety industries.

On Wed, Jul 27, 2016 at 11:25:52AM +0200, Linus Walleij wrote:
> On Wed, Jul 27, 2016 at 6:46 AM, Darren Hart <dvhart@infradead.org> wrote:
> 
> >   - Developing a "safety culture" and any overlap that may have with security
...
> My loose thoughts on the issue is twofold:
> 
> - We will have an influx of professional safety reviewers that do not
>   share their review comments with us, instead apply fixes locally and
>   not upstream. This is potentially dangerous if the next reviewer for
>   a safety-critical system misses the same bug. 

I thought the safety reviewers were an independent third party?  Wouldn't
the company attempting to get the product certified be the one making
the changes?

Regardless, identifying these third party reviewers and asking them to
'Cc linux-safety@v.k.o' or similar on bugs they find may be a good place
to start.  The trick is increasing our awareness of bugs with the least
impact to their workflow.

We would also need to anticipate that a significant number of those
reports would be located in non-upstreamed vendor drivers/features.

> - Can we record external inspection-only code reviews done by these
>   independent code reviewers (post-minimization) into the kernel (etc) git?
>   That I guess is pretty useful for building formal trust for the code,
>   but I never heard of git annotations to some random code lines like
>   that.

How do they mark up $codebase currently?  It would be helpful to hear a
walk through of their typical workflow.

> - Should minimization be a part of the kernel standard tooling for use
>   cases like this?

I think so.  This also has other benefits.  I've been putting some
thought towards "how to make the engineers job easier when justifying a
kernel update"  If a vendor has a minimal config for their current
version, I could imagine a 'git diff v4.6..v4.7 | ./scripts/configdiff
...' that would trim the diff between two kernel versions down to only
the config-enabled code.  Bonus points for taking 'git log --oneline
v4.6..v4.7' output and trimming to relevant commits.

This would reduce hours needed to review the changes, help focus QA
testing, and quantify the benefits of upgrading.  All of which makes it
easier for vendors to update kernels instead of getting 'stuck'.

thx,

Jason.

Re: [Ksummit-discuss] Linux and Safety Industry reviews, was: Self nomination

[Darren Hart]

On Wed, Jul 27, 2016 at 01:46:02PM +0000, Jason Cooper wrote:
> Linus, Darren,
> 
> I'm also tangentially interested in this discussion from a security point
> of view.  Failing in a deterministic way is a desirable trait in both
> the security and safety industries.

I'm considering proposing a tech topic, sounds like there may be sufficient
interest.

> 
> On Wed, Jul 27, 2016 at 11:25:52AM +0200, Linus Walleij wrote:
> > On Wed, Jul 27, 2016 at 6:46 AM, Darren Hart <dvhart@infradead.org> wrote:
> > 
> > >   - Developing a "safety culture" and any overlap that may have with security
> ...
> > My loose thoughts on the issue is twofold:
> > 
> > - We will have an influx of professional safety reviewers that do not
> >   share their review comments with us, instead apply fixes locally and
> >   not upstream. This is potentially dangerous if the next reviewer for
> >   a safety-critical system misses the same bug. 
> 
> I thought the safety reviewers were an independent third party?  Wouldn't

Yes, the various German and Switzerland TUeV organizations for example.

> the company attempting to get the product certified be the one making
> the changes?

That's the traditional approach, but that is impractical for something as
complex as Linux. Like most arguments for open source software, it behooves the
industry to collaborate on the common needs, such as a compliance route, data
package, evidences, and argumentation for the Linux kernel, glibc, and basic
runtime environment. This is the goal of the OSADL SIL2LinuxMP project:

http://www.osadl.org/SIL2LinuxMP.sil2-linux-project.0.html

> 
> Regardless, identifying these third party reviewers and asking them to
> 'Cc linux-safety@v.k.o' or similar on bugs they find may be a good place
> to start.  The trick is increasing our awareness of bugs with the least
> impact to their workflow.

This sounds like a good idea to me, I'll discuss with the OSADL safety
coordinator and see if he has any concerns.

> 
> We would also need to anticipate that a significant number of those
> reports would be located in non-upstreamed vendor drivers/features.

Like bug against non-upstream drivers, there isn't anything we can really do
there. Also, the compliance route for Linux will depend heavily on code that is
upstream as the development process and requisite code review, static analysis,
etc. is part of the evidence that will be used to complete the data package for
qualification. Non-upstream code will not be able to use the same compliance
route, and will most likely have to take a more traditional route.

> 
> > - Can we record external inspection-only code reviews done by these
> >   independent code reviewers (post-minimization) into the kernel (etc) git?
> >   That I guess is pretty useful for building formal trust for the code,
> >   but I never heard of git annotations to some random code lines like
> >   that.
> 
> How do they mark up $codebase currently?  It would be helpful to hear a
> walk through of their typical workflow.

I'll follow up.

> 
> > - Should minimization be a part of the kernel standard tooling for use
> >   cases like this?
> 

Yes. A team at Hitachi is also working tooling which performs some preprocessing
of the code to further minimize and remove used code. They are an active
contributor to the SIL2LinuxMP project and presented their work at the
Automotive Linux Summit in Japan earlier this month.

> I think so.  This also has other benefits.  I've been putting some
> thought towards "how to make the engineers job easier when justifying a
> kernel update"  If a vendor has a minimal config for their current
> version, I could imagine a 'git diff v4.6..v4.7 | ./scripts/configdiff
> ...' that would trim the diff between two kernel versions down to only
> the config-enabled code.  Bonus points for taking 'git log --oneline
> v4.6..v4.7' output and trimming to relevant commits.
> 
> This would reduce hours needed to review the changes, help focus QA
> testing, and quantify the benefits of upgrading.  All of which makes it
> easier for vendors to update kernels instead of getting 'stuck'.
> 
> thx,
> 
> Jason.
> 

-- 
Darren Hart
Intel Open Source Technology Center

Re: [Ksummit-discuss] Linux and Safety Industry reviews, was: Self nomination

[Darren Hart]

On Wed, Jul 27, 2016 at 01:46:02PM +0000, Jason Cooper wrote:
> Linus, Darren,
> 
> I'm also tangentially interested in this discussion from a security point
> of view.  Failing in a deterministic way is a desirable trait in both
> the security and safety industries.
> 
> On Wed, Jul 27, 2016 at 11:25:52AM +0200, Linus Walleij wrote:
> > On Wed, Jul 27, 2016 at 6:46 AM, Darren Hart <dvhart@infradead.org> wrote:
> > 
> > >   - Developing a "safety culture" and any overlap that may have with security
> ...
> > My loose thoughts on the issue is twofold:
> > 
> > - We will have an influx of professional safety reviewers that do not
> >   share their review comments with us, instead apply fixes locally and
> >   not upstream. This is potentially dangerous if the next reviewer for
> >   a safety-critical system misses the same bug. 
> 
> I thought the safety reviewers were an independent third party?  Wouldn't
> the company attempting to get the product certified be the one making
> the changes?
> 
> Regardless, identifying these third party reviewers and asking them to
> 'Cc linux-safety@v.k.o' or similar on bugs they find may be a good place
> to start.  The trick is increasing our awareness of bugs with the least
> impact to their workflow.
> 
> We would also need to anticipate that a significant number of those
> reports would be located in non-upstreamed vendor drivers/features.
> 
> > - Can we record external inspection-only code reviews done by these
> >   independent code reviewers (post-minimization) into the kernel (etc) git?
> >   That I guess is pretty useful for building formal trust for the code,
> >   but I never heard of git annotations to some random code lines like
> >   that.
> 
> How do they mark up $codebase currently?  It would be helpful to hear a
> walk through of their typical workflow.

A response from Nicholas McGuire (SIL2LinuxMP Safety Coordinator) responded
with:

"
We are feeding all findings back as much as we can - regarding the
cert data package its a lot on the process we run and the evidence
we gather - so most of that will not go into the kernel documentation
in any resonable form - but it could help to cleanup the DLC a bit
mostly small things like
"

He also said he would be willing to join us in Santa Fe for a TECH topic. I'll
write one up quickly to get it in today.


> 
> > - Should minimization be a part of the kernel standard tooling for use
> >   cases like this?
> 
> I think so.  This also has other benefits.  I've been putting some
> thought towards "how to make the engineers job easier when justifying a
> kernel update"  If a vendor has a minimal config for their current
> version, I could imagine a 'git diff v4.6..v4.7 | ./scripts/configdiff
> ...' that would trim the diff between two kernel versions down to only
> the config-enabled code.  Bonus points for taking 'git log --oneline
> v4.6..v4.7' output and trimming to relevant commits.
> 
> This would reduce hours needed to review the changes, help focus QA
> testing, and quantify the benefits of upgrading.  All of which makes it
> easier for vendors to update kernels instead of getting 'stuck'.
> 
> thx,
> 
> Jason.
> 

-- 
Darren Hart
Intel Open Source Technology Center

Re: [Ksummit-discuss] Self nomination

[Darren Hart]

On Wed, Jul 27, 2016 at 11:25:52AM +0200, Linus Walleij wrote:
> On Wed, Jul 27, 2016 at 6:46 AM, Darren Hart <dvhart@infradead.org> wrote:
> 
> >   - Developing a "safety culture" and any overlap that may have with security
> 
> Do you mean safety as in "Linux in airbags and smoke detectors"?

It's more like Linux in consolidated automotive ECUs, industrial robot and
control, medical devices, avionics, etc. When the computational requirements of
safety critical systems exceed the capabilities of the traditional MCUs.
Automotive and Industrial Control are two of the driving forces currently.

> 
> This area is interesting for GPIO and IIO as well for natural reasons:
> these systems all tend to use GPIO and sensors. (Albeit more often
> than not with some horrific userspace hodge-podge but this is not the
> time to be grumpy about that.)
> 
> This presentation appeared at LinuxCon Japan (got the link from
> my colleague Takahiro Akashi):
> 
> Qualifying Linux for Functional Safety
> http://events.linuxfoundation.org/sites/events/files/slides/20160713_SIL2LinuxMP_Min_ALS_0.9_pub.pdf
> 

There are others, but this one is the core of the approach to a successful
compliance route, and is based on the most rigorous and experienced approach.
Nicholas is a well respected leader in this area. It may be worth inviting him
to participate in this tech topic. I'll see what his interest level is.

> I have been in contact with a Swedish company working in the fire alarm
> business as well.
> 
> My overall feeling is that world regulations (standards) on safety-critical
> software seem to be centered around code inspection.
> 
> These regulations approach is not to trust any third party but have all
> code inspected by independent reviewers for functional safety. All the
> time. For every new deployment. Kernel, libc, busybox userspace.
> (Or whatever they use.)

This is true, and all changes (fixes, features, security patches, etc.) are
treated as modifications. The compliance route, however, for complex software
and complex hardware is an active area of development (see SIL2LinuxMP above),
as these standards (IEC 61508) were not developed with such software or hardware
in mind.

> 
> Thus Hitachi developed this minimizing stripping out all not-compiled code
> and #ifdefs etc too to get down the code size they have to manually inspect
> to a minimum. (It easily translates into work hours.)
>
> My loose thoughts on the issue is twofold:
> 
> - We will have an influx of professional safety reviewers that do not
>   share their review comments with us, instead apply fixes locally and
>   not upstream. This is potentially dangerous if the next reviewer for
>   a safety-critical system misses the same bug. (Not to say unethical
>   vs the community but I have come to understand that some people
>   out there do not care about that.) So we need to send a message to
>   the safety-critical industry that any issues found in such safety
>   inspections need to go upstream pronto. No vendor tree:ing of this.

SIL2LinuxMP will help significantly with this, and the TUeV is happy to see open
source entering the Functional Safety space as the developer culture rewards
finding and fixing issues. Bugs can be found and fixed early, rather than
waiting for accidents and deaths to drive changes.

> 
> - Can we record external inspection-only code reviews done by these
>   independent code reviewers (post-minimization) into the kernel (etc) git?
>   That I guess is pretty useful for building formal trust for the code,
>   but I never heard of git annotations to some random code lines like
>   that.
> 
> - Should minimization be a part of the kernel standard tooling for use
>   cases like this?
> 
> Incidentally that may overlap with the footprint minimizing goal: if you can
> configure code out (such as the modular syscalls things that Nico has
> been working on), that makes this kind of code minimization easier and
> may employ similar tooling.

The SIL2LinuxMP project works with a minimized Linux kernel of ~400k loc.

> 
> Yours,
> Linus Walleij
> 

-- 
Darren Hart
Intel Open Source Technology Center

Re: [Ksummit-discuss] Self nomination

[Geert Uytterhoeven]

On Wed, Jul 27, 2016 at 11:25 AM, Linus Walleij
<linus.walleij@linaro.org> wrote:
> Qualifying Linux for Functional Safety
> http://events.linuxfoundation.org/sites/events/files/slides/20160713_SIL2LinuxMP_Min_ALS_0.9_pub.pdf

404

http://events.linuxfoundation.org/sites/events/files/slides/20160713_SIL2LinuxMP_Min_ALS_1.1.pdf

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds