[Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Peter Huewe]

Hi,

one topic I would like to discuss is: when is it time to remove support for old hardware?
Should we support as much hardware as possible forever, or does it make sense to cut down the support?
If so, when would be appropriate?

One concrete example is in the TPM subsystem - we have support for TPM1.1b based devices there, which haven't been manufactured since probably 10 years.
These still compile, load and probably function well enough but are not really tested or reviewed on a regular basis.
Maybe they even have some issues, security or otherwise.

We haven't removed them yet, despite plans to do so, as there still might be some users out there.
This lead to more a complicated code for the 'generic' tpm drivers as they still have to support 'old' workarounds.

I think the same applies to the whole kernel in general and other subsystems - do we have to much old code lying around, waiting to rot and blocking better code?
How should we aproach phasing out support for old hardware?


Thanks,
Peter

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Linus Walleij]

On Mon, Sep 10, 2018 at 4:04 PM Peter Huewe <PeterHuewe@gmx.de> wrote:

> one topic I would like to discuss is: when is it time to remove support for old hardware?
> Should we support as much hardware as possible forever, or does it make sense to cut down the support?
> If so, when would be appropriate?

I've been thinking a lot about it. I just recently prepared a talk on the
subject.

My thinking has been something along the lines: if it has active
users we don't delete it, by the same token that we don't destroy
kernel interfaces that have active users.

Of course the definition of "users" is a bit complex there. In our
opinion the "users" are those who regularly compile new kernels
and test them, finds regressions, bisects problems and generally
do a good job in the community. Not ranting bloggers.

By this reasoning it is absolutely forbidden to propose something
like to delete arch/m68k (the last CPU M68060 produced in
1994 IIUC) as long as these maintainers put in an honest effort to
keep running that kernel on increasingly aging and hard to obtain
hardware like Apollo/Domain or Amiga 4000.

One day the last m68k users stops responding to regressions and
then it can be deleted.

My ARMv4 is another example, but I can point at new devices
beging deployed as we speak, using that ISA, even though it is
from 1999. So it has many active users (and maintainers).

(CC:ing the m68k maintainer, who is an awesome guy.)

Yours,
Linus Walleij

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Arnd Bergmann]

On Mon, Sep 10, 2018 at 5:35 PM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> On Mon, Sep 10, 2018 at 4:04 PM Peter Huewe <PeterHuewe@gmx.de> wrote:
>
> > one topic I would like to discuss is: when is it time to remove support for old hardware?
> > Should we support as much hardware as possible forever, or does it make sense to cut down the support?
> > If so, when would be appropriate?
>
> I've been thinking a lot about it. I just recently prepared a talk on the
> subject.
>
> My thinking has been something along the lines: if it has active
> users we don't delete it, by the same token that we don't destroy
> kernel interfaces that have active users.
>
> Of course the definition of "users" is a bit complex there. In our
> opinion the "users" are those who regularly compile new kernels
> and test them, finds regressions, bisects problems and generally
> do a good job in the community. Not ranting bloggers.
>
> By this reasoning it is absolutely forbidden to propose something
> like to delete arch/m68k (the last CPU M68060 produced in
> 1994 IIUC) as long as these maintainers put in an honest effort to
> keep running that kernel on increasingly aging and hard to obtain
> hardware like Apollo/Domain or Amiga 4000.
>
> One day the last m68k users stops responding to regressions and
> then it can be deleted.

When I prepared the series to remove eight unmaintained architectures,
that is roughly what I did. I also managed to contact at least one
core developer for each of those architectures (or at least one
person representing the company owning that IP), and it was all
done in agreement with them.

One architecture (unicore32) got saved in the last minute by
the maintainer saying that he'd rather keep it in tree, despite not
even having a publicly available toolchain with sources.

For architecture support, I don't think we are removing anything
in the next few years now, but there are two notable categories
of platform ports that are definitely obsolete, and we may want
to consider how to manage their remaining lives:

- Server architectures without a future: alpha, parisc, (mostly)
  ia64, (arguably) sparc are in a similar category as m68k;
  they are kept alive by enthusiasts that still have the hardware,
  but nobody in their right minds is putting new workloads on
  them.

- In some architectures (arm, mips, ppc, sh, m68k), we support
   many different machines, but only a small number of those
   machines are actively being used by anyone. We did a big
   cleanup on ARM a few years ago, and ppc dropped some
   machines during the conversion to DT, but in some of the
   others that never happened, and some have died in the
   meantime (or in some cases died just after they got merged,
   such as arm/mach-axxia).

Making the correct call on which platforms exactly are dead
takes a lot of knowledge and work to get right, e.g. you want
to also remove all hacks in device drivers that were added for
a platform when that gets removed.

One big problem with removing code in general is that code that
becomes unmaintained often also has nobody left that knows
about it being unmaintained.

> My ARMv4 is another example, but I can point at new devices
> beging deployed as we speak, using that ISA, even though it is
> from 1999. So it has many active users (and maintainers).

Note that even though gcc is dropping ARMv4 support from new
compilers, you can still use old toolchains, and there are tricks to
make ARMv4T binary code work on ARMv4. However, if gcc
ever stops supporting ARMv4T, this becomes a problem. My guess
is that will take another 10 years though, and we might have
removed some or all of the individual ARMv4 platforms by then.

      Arnd

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Guenter Roeck]

On Mon, Sep 10, 2018 at 11:40:59PM +0200, Arnd Bergmann wrote:
> 
> One architecture (unicore32) got saved in the last minute by
> the maintainer saying that he'd rather keep it in tree, despite not
> even having a publicly available toolchain with sources.
> 

unicore32 no longer builds with the mainline kernel because the
(private) toolchain is too old (gcc < 4.6), so it may be time
to revisit that decision.

Guenter

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Geert Uytterhoeven]

Hi Günter,

On Tue, Sep 11, 2018 at 12:02 AM Guenter Roeck <linux@roeck-us.net> wrote:
> On Mon, Sep 10, 2018 at 11:40:59PM +0200, Arnd Bergmann wrote:
> > One architecture (unicore32) got saved in the last minute by
> > the maintainer saying that he'd rather keep it in tree, despite not
> > even having a publicly available toolchain with sources.
>
> unicore32 no longer builds with the mainline kernel because the
> (private) toolchain is too old (gcc < 4.6), so it may be time
> to revisit that decision.

I'm still building mainline with gcc-4.1.2, using:

    git revert 815f0ddb346c1960 # "include/linux/compiler*.h: make
compiler-*.h mutually exclusive"
    git revert cafa0010cd51fb71 # "Raise the minimum required gcc
version to 4.6"

leading to one recent bug fix:

    https://lore.kernel.org/lkml/20180823212738.18431-1-geert@linux-m68k.org/

and two code improvements:

    https://lore.kernel.org/lkml/20180823212436.17423-1-geert@linux-m68k.org/
    https://lore.kernel.org/lkml/20180823213027.18856-1-geert@linux-m68k.org/

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Guenter Roeck]

Hi Geert,

On Tue, Sep 11, 2018 at 10:49:25AM +0200, Geert Uytterhoeven wrote:
> Hi Günter,
> 
> On Tue, Sep 11, 2018 at 12:02 AM Guenter Roeck <linux@roeck-us.net> wrote:
> > On Mon, Sep 10, 2018 at 11:40:59PM +0200, Arnd Bergmann wrote:
> > > One architecture (unicore32) got saved in the last minute by
> > > the maintainer saying that he'd rather keep it in tree, despite not
> > > even having a publicly available toolchain with sources.
> >
> > unicore32 no longer builds with the mainline kernel because the
> > (private) toolchain is too old (gcc < 4.6), so it may be time
> > to revisit that decision.
> 
> I'm still building mainline with gcc-4.1.2, using:
> 
>     git revert 815f0ddb346c1960 # "include/linux/compiler*.h: make
> compiler-*.h mutually exclusive"
>     git revert cafa0010cd51fb71 # "Raise the minimum required gcc
> version to 4.6"
> 
> leading to one recent bug fix:
> 
>     https://lore.kernel.org/lkml/20180823212738.18431-1-geert@linux-m68k.org/
> 
> and two code improvements:
> 
>     https://lore.kernel.org/lkml/20180823212436.17423-1-geert@linux-m68k.org/
>     https://lore.kernel.org/lkml/20180823213027.18856-1-geert@linux-m68k.org/
> 

Interesting. How to you manage to not hit the problem reported with
https://lkml.org/lkml/2018/8/14/472 ?

Thanks,
Guenter

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Geert Uytterhoeven]

Hi Günter,

On Tue, Sep 11, 2018 at 7:27 PM Guenter Roeck <linux@roeck-us.net> wrote:
> On Tue, Sep 11, 2018 at 10:49:25AM +0200, Geert Uytterhoeven wrote:
> > On Tue, Sep 11, 2018 at 12:02 AM Guenter Roeck <linux@roeck-us.net> wrote:
> > > On Mon, Sep 10, 2018 at 11:40:59PM +0200, Arnd Bergmann wrote:
> > > > One architecture (unicore32) got saved in the last minute by
> > > > the maintainer saying that he'd rather keep it in tree, despite not
> > > > even having a publicly available toolchain with sources.
> > >
> > > unicore32 no longer builds with the mainline kernel because the
> > > (private) toolchain is too old (gcc < 4.6), so it may be time
> > > to revisit that decision.
> >
> > I'm still building mainline with gcc-4.1.2, using:
> >
> >     git revert 815f0ddb346c1960 # "include/linux/compiler*.h: make
> > compiler-*.h mutually exclusive"
> >     git revert cafa0010cd51fb71 # "Raise the minimum required gcc
> > version to 4.6"
> >
> > leading to one recent bug fix:
> >
> >     https://lore.kernel.org/lkml/20180823212738.18431-1-geert@linux-m68k.org/
> >
> > and two code improvements:
> >
> >     https://lore.kernel.org/lkml/20180823212436.17423-1-geert@linux-m68k.org/
> >     https://lore.kernel.org/lkml/20180823213027.18856-1-geert@linux-m68k.org/
>
> Interesting. How to you manage to not hit the problem reported with
> https://lkml.org/lkml/2018/8/14/472 ?

Sorry, I forgot I have also applied Andrew's wibble patch:
https://lore.kernel.org/lkml/20180814160208.4f4dd7ca142912f5894ddddd@linux-foundation.org/

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Guenter Roeck]

On Tue, Sep 11, 2018 at 07:58:48PM +0200, Geert Uytterhoeven wrote:
> Hi Günter,
> 
> On Tue, Sep 11, 2018 at 7:27 PM Guenter Roeck <linux@roeck-us.net> wrote:
> > On Tue, Sep 11, 2018 at 10:49:25AM +0200, Geert Uytterhoeven wrote:
> > > On Tue, Sep 11, 2018 at 12:02 AM Guenter Roeck <linux@roeck-us.net> wrote:
> > > > On Mon, Sep 10, 2018 at 11:40:59PM +0200, Arnd Bergmann wrote:
> > > > > One architecture (unicore32) got saved in the last minute by
> > > > > the maintainer saying that he'd rather keep it in tree, despite not
> > > > > even having a publicly available toolchain with sources.
> > > >
> > > > unicore32 no longer builds with the mainline kernel because the
> > > > (private) toolchain is too old (gcc < 4.6), so it may be time
> > > > to revisit that decision.
> > >
> > > I'm still building mainline with gcc-4.1.2, using:
> > >
> > >     git revert 815f0ddb346c1960 # "include/linux/compiler*.h: make
> > > compiler-*.h mutually exclusive"
> > >     git revert cafa0010cd51fb71 # "Raise the minimum required gcc
> > > version to 4.6"
> > >
> > > leading to one recent bug fix:
> > >
> > >     https://lore.kernel.org/lkml/20180823212738.18431-1-geert@linux-m68k.org/
> > >
> > > and two code improvements:
> > >
> > >     https://lore.kernel.org/lkml/20180823212436.17423-1-geert@linux-m68k.org/
> > >     https://lore.kernel.org/lkml/20180823213027.18856-1-geert@linux-m68k.org/
> >
> > Interesting. How to you manage to not hit the problem reported with
> > https://lkml.org/lkml/2018/8/14/472 ?
> 
> Sorry, I forgot I have also applied Andrew's wibble patch:
> https://lore.kernel.org/lkml/20180814160208.4f4dd7ca142912f5894ddddd@linux-foundation.org/
> 

Hmm. I don't really want to have to maintain a set of patches on top upstream
and stable kernels to be able to run my tests. I do have a set of configuration
options to apply, but anything beyond that is, in my opinion, out of scope
for (at least my) testing.

Thanks,
Guenter

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Daniel Vetter]

On Tue, Sep 11, 2018 at 8:27 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> On Tue, Sep 11, 2018 at 07:58:48PM +0200, Geert Uytterhoeven wrote:
>> Hi Günter,
>>
>> On Tue, Sep 11, 2018 at 7:27 PM Guenter Roeck <linux@roeck-us.net> wrote:
>> > On Tue, Sep 11, 2018 at 10:49:25AM +0200, Geert Uytterhoeven wrote:
>> > > On Tue, Sep 11, 2018 at 12:02 AM Guenter Roeck <linux@roeck-us.net> wrote:
>> > > > On Mon, Sep 10, 2018 at 11:40:59PM +0200, Arnd Bergmann wrote:
>> > > > > One architecture (unicore32) got saved in the last minute by
>> > > > > the maintainer saying that he'd rather keep it in tree, despite not
>> > > > > even having a publicly available toolchain with sources.
>> > > >
>> > > > unicore32 no longer builds with the mainline kernel because the
>> > > > (private) toolchain is too old (gcc < 4.6), so it may be time
>> > > > to revisit that decision.
>> > >
>> > > I'm still building mainline with gcc-4.1.2, using:
>> > >
>> > >     git revert 815f0ddb346c1960 # "include/linux/compiler*.h: make
>> > > compiler-*.h mutually exclusive"
>> > >     git revert cafa0010cd51fb71 # "Raise the minimum required gcc
>> > > version to 4.6"
>> > >
>> > > leading to one recent bug fix:
>> > >
>> > >     https://lore.kernel.org/lkml/20180823212738.18431-1-geert@linux-m68k.org/
>> > >
>> > > and two code improvements:
>> > >
>> > >     https://lore.kernel.org/lkml/20180823212436.17423-1-geert@linux-m68k.org/
>> > >     https://lore.kernel.org/lkml/20180823213027.18856-1-geert@linux-m68k.org/
>> >
>> > Interesting. How to you manage to not hit the problem reported with
>> > https://lkml.org/lkml/2018/8/14/472 ?
>>
>> Sorry, I forgot I have also applied Andrew's wibble patch:
>> https://lore.kernel.org/lkml/20180814160208.4f4dd7ca142912f5894ddddd@linux-foundation.org/
>>
>
> Hmm. I don't really want to have to maintain a set of patches on top upstream
> and stable kernels to be able to run my tests. I do have a set of configuration
> options to apply, but anything beyond that is, in my opinion, out of scope
> for (at least my) testing.

We've given that notion up years ago :-/

There's a constantly included topic branch with hacks-for-CI with a
random assortment of reverts, patches stuck in forever-limbo and
things where we've been yelled at too much and it's just not worth the
bother. If our CI is down, there's a 50 people team becoming rapidly
unhappy, we can't affort to wait even for maintainers who are on top
of their stuff. Rapidly here means: a few hours after CI is
down/backlogged, it's the only topic on our irc channel.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Linus Walleij]

Including Alexander Sverdlin, Lukasz Majewski  and Jonas
Jensen here, they may or may not be able to share some of
their industrial IoT experience. (Contract terms with vendors
may make it necessary to stay silent sometimes.)

On Mon, Sep 10, 2018 at 11:41 PM Arnd Bergmann <arnd@arndb.de> wrote:
> On Mon, Sep 10, 2018 at 5:35 PM Linus Walleij <linus.walleij@linaro.org> wrote:

> > My ARMv4 is another example, but I can point at new devices
> > beging deployed as we speak, using that ISA, even though it is
> > from 1999. So it has many active users (and maintainers).
>
> Note that even though gcc is dropping ARMv4 support from new
> compilers, you can still use old toolchains, and there are tricks to
> make ARMv4T binary code work on ARMv4. However, if gcc
> ever stops supporting ARMv4T, this becomes a problem. My guess
> is that will take another 10 years though, and we might have
> removed some or all of the individual ARMv4 platforms by then.

ARMv4 is becoming a trouble, not that it is hard to maintain,
actually we're on top of things there. The problem is that among
the FA526 systems from Faraday and the ARMv4T in EP93xx
there are very serious IoT deployments that have been going
on for soon 15 years and continuing.

New MOXA ART ARMv4 FA526 systems are being deployed
in buildings across the planet as we speak. They just replaced
one in the office block where I sit, that is how I got to know.

These are mostly for ventilation and
similar systems but also heavy duty from Liebherr controlling
unspecified hydraulic systems. The ventilation systems are
definately Internet-connected, I don't know about the others.

These pose an increasing security threat, and for that reason
I personally feel it is irresponsible to remove the option to
create new kernels and upgrade these devices.

I think for depreciation one has to be aware that some archs
used in IoT deployments have life cycles of 20-30 or more years,
whereas some tablet or handset SoCs may be something like
5 years maximum before maintainers get annoyed that you
even use them.

Sometimes I get the feeling that people focused on desktops
or servers suffer from velocitate (speed blindness) and think
everybody is like them. (Well don't we all.)

With all the hoopla about IoT in the business right now since
a year or two back, the question of their extremely long life
cycle and effect on development has not really been
considered AFAICT those are some of the most important
systems to keep maintained.

Yours,
Linus Walleij

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 3860 bytes --]

Hi Linus,

> Including Alexander Sverdlin, Lukasz Majewski  and Jonas
> Jensen here, they may or may not be able to share some of
> their industrial IoT experience. (Contract terms with vendors
> may make it necessary to stay silent sometimes.)

Thank you for putting me on CC.

> 
> On Mon, Sep 10, 2018 at 11:41 PM Arnd Bergmann <arnd@arndb.de> wrote:
> > On Mon, Sep 10, 2018 at 5:35 PM Linus Walleij
> > <linus.walleij@linaro.org> wrote:  
> 
> > > My ARMv4 is another example, but I can point at new devices
> > > beging deployed as we speak, using that ISA, even though it is
> > > from 1999. So it has many active users (and maintainers).  
> >
> > Note that even though gcc is dropping ARMv4 support from new
> > compilers, you can still use old toolchains, and there are tricks to
> > make ARMv4T binary code work on ARMv4. However, if gcc
> > ever stops supporting ARMv4T, this becomes a problem. My guess
> > is that will take another 10 years though, and we might have
> > removed some or all of the individual ARMv4 platforms by then.  
> 
> ARMv4 is becoming a trouble, not that it is hard to maintain,
> actually we're on top of things there. The problem is that among
> the FA526 systems from Faraday and the ARMv4T in EP93xx
> there are very serious IoT deployments that have been going
> on for soon 15 years and continuing.

The ARMv4T shall be supported. The 15 years may be a bare minimum for
some "heavy duty/industrial" systems...

It may happen that the same HW (certified already) will be used for 30+
years (with the SW BSP replaced a few times).

> 
> New MOXA ART ARMv4 FA526 systems are being deployed
> in buildings across the planet as we speak. They just replaced
> one in the office block where I sit, that is how I got to know.
> 
> These are mostly for ventilation and
> similar systems but also heavy duty from Liebherr controlling
> unspecified hydraulic systems. The ventilation systems are
> definately Internet-connected, I don't know about the others.

The "unspecified hydraulic systems" do have a way to update the SW
(which is already done).

> 
> These pose an increasing security threat, and for that reason
> I personally feel it is irresponsible to remove the option to
> create new kernels and upgrade these devices.

Frankly speaking the possibility to have the new kernel in a relatively
easy (and cheap) way was a strong motivator to add support to mainline.

> 
> I think for depreciation one has to be aware that some archs
> used in IoT deployments have life cycles of 20-30 or more years,
> whereas some tablet or handset SoCs may be something like
> 5 years maximum before maintainers get annoyed that you
> even use them.
> 
> Sometimes I get the feeling that people focused on desktops
> or servers suffer from velocitate (speed blindness) and think
> everybody is like them. (Well don't we all.)

I do must agree here. In general ARM9 (v4T) will stay with us for a
long, long time.

In the kernel community we pose a lot of attention to security (for
example the prompt reaction on meltdown/spectre), but in the same time
we tend to forget about the "long lived" devices and force their
maintainers to use 2.6.x kernels..... (or even 2.4.x).

> 
> With all the hoopla about IoT in the business right now since
> a year or two back, the question of their extremely long life
> cycle and effect on development has not really been
> considered AFAICT those are some of the most important
> systems to keep maintained.

+1

> 
> Yours,
> Linus Walleij




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Greg KH]

On Tue, Sep 11, 2018 at 11:37:25AM +0200, Lukasz Majewski wrote:
> In the kernel community we pose a lot of attention to security (for
> example the prompt reaction on meltdown/spectre), but in the same time
> we tend to forget about the "long lived" devices and force their
> maintainers to use 2.6.x kernels..... (or even 2.4.x).

We care, but really, how much can we do here?

I've been working a lot with the Adroid ecosystem to try to help fix
their bad habits of "grab a random kernel and ship it and never update
it" by providing longer lived kernels that they can constantly update
their devices to.

But their lifetimes is much shorter compared to yours, and I have no
insight into what kernels are being used, what configurations you all
care about, and how long you need/want them updated.

Working with really old kernels like you have, without hardware
available to test is a hard task.  If your hardware is in a system like
kernelci, then you can be sure that any new kernel will work properly
with your system and then you might not want to have to stay with really
old kernels that no one can maintain :)

There's a Linux Foundation project, "CIP" that wants to maintain kernels
for devices like what you are making for 20+ years.  They are having the
problems of not knowing exactly what platforms they wish to support, but
their goal is good, hopefully they eventually nail something down and we
can work together.  Perhaps you should contact them to try to help solve
this issue for everyone?

thanks,

greg k-h

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Laurent Pinchart]

Hi Greg,

On Tuesday, 11 September 2018 22:33:08 EEST Greg KH wrote:
> On Tue, Sep 11, 2018 at 11:37:25AM +0200, Lukasz Majewski wrote:
> > In the kernel community we pose a lot of attention to security (for
> > example the prompt reaction on meltdown/spectre), but in the same time
> > we tend to forget about the "long lived" devices and force their
> > maintainers to use 2.6.x kernels..... (or even 2.4.x).
> 
> We care, but really, how much can we do here?
> 
> I've been working a lot with the Adroid ecosystem to try to help fix
> their bad habits of "grab a random kernel and ship it and never update
> it" by providing longer lived kernels that they can constantly update
> their devices to.
> 
> But their lifetimes is much shorter compared to yours, and I have no
> insight into what kernels are being used, what configurations you all
> care about, and how long you need/want them updated.
> 
> Working with really old kernels like you have, without hardware
> available to test is a hard task.  If your hardware is in a system like
> kernelci, then you can be sure that any new kernel will work properly
> with your system and then you might not want to have to stay with really
> old kernels that no one can maintain :)
> 
> There's a Linux Foundation project, "CIP" that wants to maintain kernels
> for devices like what you are making for 20+ years.  They are having the
> problems of not knowing exactly what platforms they wish to support, but
> their goal is good, hopefully they eventually nail something down and we
> can work together.  Perhaps you should contact them to try to help solve
> this issue for everyone?

I may be wrong, but I understand Lukasz's comment as the exact opposite: we 
forget about long-lived devices and drop their support while they're still in 
active use, forcing vendors to start using old and unsupported kernels. If a 
large number of ARMv4(T) devices are still being actively deployed and 
maintain, we should treat them as first-class citizens.

-- 
Regards,

Laurent Pinchart

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Thomas Gleixner]

On Wed, 12 Sep 2018, Laurent Pinchart wrote:

> Hi Greg,
> 
> On Tuesday, 11 September 2018 22:33:08 EEST Greg KH wrote:
> > On Tue, Sep 11, 2018 at 11:37:25AM +0200, Lukasz Majewski wrote:
> > > In the kernel community we pose a lot of attention to security (for
> > > example the prompt reaction on meltdown/spectre), but in the same time
> > > we tend to forget about the "long lived" devices and force their
> > > maintainers to use 2.6.x kernels..... (or even 2.4.x).
> > 
> > We care, but really, how much can we do here?
> > 
> > I've been working a lot with the Adroid ecosystem to try to help fix
> > their bad habits of "grab a random kernel and ship it and never update
> > it" by providing longer lived kernels that they can constantly update
> > their devices to.
> > 
> > But their lifetimes is much shorter compared to yours, and I have no
> > insight into what kernels are being used, what configurations you all
> > care about, and how long you need/want them updated.
> > 
> > Working with really old kernels like you have, without hardware
> > available to test is a hard task.  If your hardware is in a system like
> > kernelci, then you can be sure that any new kernel will work properly
> > with your system and then you might not want to have to stay with really
> > old kernels that no one can maintain :)
> > 
> > There's a Linux Foundation project, "CIP" that wants to maintain kernels
> > for devices like what you are making for 20+ years.  They are having the
> > problems of not knowing exactly what platforms they wish to support, but
> > their goal is good, hopefully they eventually nail something down and we
> > can work together.  Perhaps you should contact them to try to help solve
> > this issue for everyone?
> 
> I may be wrong, but I understand Lukasz's comment as the exact opposite: we 
> forget about long-lived devices and drop their support while they're still in 
> active use, forcing vendors to start using old and unsupported kernels. If a 
> large number of ARMv4(T) devices are still being actively deployed and 
> maintain, we should treat them as first-class citizens.

But that does not mean, that we have to support ancient compilers
forever. If that stuff needs to be treated as first class citizens then
someone who has vested interest in this needs to fix that. That's none of
our business, really.

Thanks,

	tglx

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Geert Uytterhoeven]

Hi Thomas,

On Tue, Sep 11, 2018 at 11:50 PM Thomas Gleixner <tglx@linutronix.de> wrote:
> On Wed, 12 Sep 2018, Laurent Pinchart wrote:
> > On Tuesday, 11 September 2018 22:33:08 EEST Greg KH wrote:
> > > On Tue, Sep 11, 2018 at 11:37:25AM +0200, Lukasz Majewski wrote:
> > > > In the kernel community we pose a lot of attention to security (for
> > > > example the prompt reaction on meltdown/spectre), but in the same time
> > > > we tend to forget about the "long lived" devices and force their
> > > > maintainers to use 2.6.x kernels..... (or even 2.4.x).
> > >
> > > We care, but really, how much can we do here?
> > >
> > > I've been working a lot with the Adroid ecosystem to try to help fix
> > > their bad habits of "grab a random kernel and ship it and never update
> > > it" by providing longer lived kernels that they can constantly update
> > > their devices to.
> > >
> > > But their lifetimes is much shorter compared to yours, and I have no
> > > insight into what kernels are being used, what configurations you all
> > > care about, and how long you need/want them updated.
> > >
> > > Working with really old kernels like you have, without hardware
> > > available to test is a hard task.  If your hardware is in a system like
> > > kernelci, then you can be sure that any new kernel will work properly
> > > with your system and then you might not want to have to stay with really
> > > old kernels that no one can maintain :)
> > >
> > > There's a Linux Foundation project, "CIP" that wants to maintain kernels
> > > for devices like what you are making for 20+ years.  They are having the
> > > problems of not knowing exactly what platforms they wish to support, but
> > > their goal is good, hopefully they eventually nail something down and we
> > > can work together.  Perhaps you should contact them to try to help solve
> > > this issue for everyone?
> >
> > I may be wrong, but I understand Lukasz's comment as the exact opposite: we
> > forget about long-lived devices and drop their support while they're still in
> > active use, forcing vendors to start using old and unsupported kernels. If a
> > large number of ARMv4(T) devices are still being actively deployed and
> > maintain, we should treat them as first-class citizens.
>
> But that does not mean, that we have to support ancient compilers
> forever. If that stuff needs to be treated as first class citizens then
> someone who has vested interest in this needs to fix that. That's none of
> our business, really.

The issue here is that gcc dropped armv4 (not v4t AFAIK) support.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Arnd Bergmann]

On Wed, Sep 12, 2018 at 8:41 AM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> On Tue, Sep 11, 2018 at 11:50 PM Thomas Gleixner <tglx@linutronix.de> wrote:
> > On Wed, 12 Sep 2018, Laurent Pinchart wrote:
> > > On Tuesday, 11 September 2018 22:33:08 EEST Greg KH wrote:
> > But that does not mean, that we have to support ancient compilers
> > forever. If that stuff needs to be treated as first class citizens then
> > someone who has vested interest in this needs to fix that. That's none of
> > our business, really.
>
> The issue here is that gcc dropped armv4 (not v4t AFAIK) support.

Let me clarify here: we are talking to the gcc developers all the time,
and we can still change the plans here if we decide that we want it
to be dropped later.

There is currently no plan to deprecate ARMv4T support in gcc.

ARMv2/3/4 have been marked "deprecated" since gcc-6 two years
ago. This is the official way for gcc to figure out whether anyone still
needs it. If someone steps up and says they need to keep it for future
releases (and ideally ensure that it keeps getting tested properly),
ARMv4 can stay.

gcc-8 (released this spring) still contains (deprecated) support for
ARMv2 and up. gcc-9 (the development branch) has removed
support for ARMv2 and ARMv3, but not ARMv4. As a result, the
ARM610 and ARM710 processors cannot be supported any more
with gcc-9. This may be a problem for the RiscOS folks, but Linux
has dropped support for both of these years ago.

We do see one regression with the removal of ARMv3 support:
The arch/arm/mach-rpc/ target (using the SA110 CPU) will not
be buildable with gcc-9 (or clang, for that matter), but require a
gcc-8 at most (some other versions were broken in the past as
well, but 6/7/8 should all work). If the next compiler deprecation
waves are similar to the current one (dropping support for 12
year old compilers, while still allowing 7 year old one, i.e. gcc-4.6
from 2011 instead of gcc-4.1 from 2006), that puts the final end
of life of mach-rpc between 2026 and 2031, which I feel is an
acceptable trade-off (apparently keeping it working did cause
significant work in gcc), especially considering this is hardware
introduced in 1994!

Regarding ARMv4 (i.e. FA526 and StrongARM) vs ARMv4T
(all ARM7 and the early ARM9 we support), the kernel build isn't
actually different at all, as long as we turn off
CONFIG_ARM_THUMB. The main difference in the code produced
by gcc is the 'bx' instruction for branches, and the linker will turn
those into ARMv4-compatible branches with the '-Wl,--fix-v4bx'
option. This means that even after gcc removes the deprecated
ARMv4 target, and we stop supporting older compilers that still
had it, we can keep building new kernels for fa526 as long as
gcc supports ARMv4T and the linker supports --fix-v4bx.
Alternatively we can ask the compiler folks to un-deprecate
ARMv4.

      Arnd

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Greg KH]

On Wed, Sep 12, 2018 at 12:39:34AM +0300, Laurent Pinchart wrote:
> Hi Greg,
> 
> On Tuesday, 11 September 2018 22:33:08 EEST Greg KH wrote:
> > On Tue, Sep 11, 2018 at 11:37:25AM +0200, Lukasz Majewski wrote:
> > > In the kernel community we pose a lot of attention to security (for
> > > example the prompt reaction on meltdown/spectre), but in the same time
> > > we tend to forget about the "long lived" devices and force their
> > > maintainers to use 2.6.x kernels..... (or even 2.4.x).
> > 
> > We care, but really, how much can we do here?
> > 
> > I've been working a lot with the Adroid ecosystem to try to help fix
> > their bad habits of "grab a random kernel and ship it and never update
> > it" by providing longer lived kernels that they can constantly update
> > their devices to.
> > 
> > But their lifetimes is much shorter compared to yours, and I have no
> > insight into what kernels are being used, what configurations you all
> > care about, and how long you need/want them updated.
> > 
> > Working with really old kernels like you have, without hardware
> > available to test is a hard task.  If your hardware is in a system like
> > kernelci, then you can be sure that any new kernel will work properly
> > with your system and then you might not want to have to stay with really
> > old kernels that no one can maintain :)
> > 
> > There's a Linux Foundation project, "CIP" that wants to maintain kernels
> > for devices like what you are making for 20+ years.  They are having the
> > problems of not knowing exactly what platforms they wish to support, but
> > their goal is good, hopefully they eventually nail something down and we
> > can work together.  Perhaps you should contact them to try to help solve
> > this issue for everyone?
> 
> I may be wrong, but I understand Lukasz's comment as the exact opposite: we 
> forget about long-lived devices and drop their support while they're still in 
> active use, forcing vendors to start using old and unsupported kernels. If a 
> large number of ARMv4(T) devices are still being actively deployed and 
> maintain, we should treat them as first-class citizens.

No, I never want to drop support for anything that anyone is actually
using.  When I drop support for hardware/protocols, it goes through the
staging tree for a few months to see if anyone says anything.  If not,
then it gets removed.  And even then, if someone comes back and says
"hey I use that!" it's a simple revert to get it back.

The problem is if no one _tells_ us they are using something.  And
that's not something we can fix without help from the developers/users
of those hardware platforms talking to us.

thanks,

greg k-h

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Peter Huewe]

Am 12. September 2018 08:26:31 MESZ schrieb Greg KH <gregkh@linuxfoundation.org>:
>On Wed, Sep 12, 2018 at 12:39:34AM +0300, Laurent Pinchart wrote:
>> Hi Greg,
>> 
>> On Tuesday, 11 September 2018 22:33:08 EEST Greg KH wrote:
>> > On Tue, Sep 11, 2018 at 11:37:25AM +0200, Lukasz Majewski wrote:
>> > > In the kernel community we pose a lot of attention to security
>(for
>> > > example the prompt reaction on meltdown/spectre), but in the same
>time
>> > > we tend to forget about the "long lived" devices and force their
>> > > maintainers to use 2.6.x kernels..... (or even 2.4.x).
>> > 
>> > We care, but really, how much can we do here?
>> > 
>> > I've been working a lot with the Adroid ecosystem to try to help
>fix
>> > their bad habits of "grab a random kernel and ship it and never
>update
>> > it" by providing longer lived kernels that they can constantly
>update
>> > their devices to.
>> > 
>> > But their lifetimes is much shorter compared to yours, and I have
>no
>> > insight into what kernels are being used, what configurations you
>all
>> > care about, and how long you need/want them updated.
>> > 
>> > Working with really old kernels like you have, without hardware
>> > available to test is a hard task.  If your hardware is in a system
>like
>> > kernelci, then you can be sure that any new kernel will work
>properly
>> > with your system and then you might not want to have to stay with
>really
>> > old kernels that no one can maintain :)
>> > 
>> > There's a Linux Foundation project, "CIP" that wants to maintain
>kernels
>> > for devices like what you are making for 20+ years.  They are
>having the
>> > problems of not knowing exactly what platforms they wish to
>support, but
>> > their goal is good, hopefully they eventually nail something down
>and we
>> > can work together.  Perhaps you should contact them to try to help
>solve
>> > this issue for everyone?
>> 
>> I may be wrong, but I understand Lukasz's comment as the exact
>opposite: we 
>> forget about long-lived devices and drop their support while they're
>still in 
>> active use, forcing vendors to start using old and unsupported
>kernels. If a 
>> large number of ARMv4(T) devices are still being actively deployed
>and 
>> maintain, we should treat them as first-class citizens.
>
>No, I never want to drop support for anything that anyone is actually
>using.  When I drop support for hardware/protocols, it goes through the
>staging tree for a few months to see if anyone says anything.  If not,
>then it gets removed.  And even then, if someone comes back and says
>"hey I use that!" it's a simple revert to get it back.
Can/should everything that we think is broken/unused be moved to staging first?
Isn't this unecessary churn of code?
The mandarory todo file would then consist of which action items?

>
>The problem is if no one _tells_ us they are using something.  And
>that's not something we can fix without help from the developers/users
>of those hardware platforms talking to us.

Would it make sense to introduce something like the staging warning ('this driver is from stagin directory' message in dmesg) for support deprecation?

So if a maintainer, who probably has the best *available* picture about hw usage, thinks it's about time to remove the support, he adds a depecration flag to kconfig, which then adds code to emit a warning to dmesg.
Users then get at least informed that deprecation would affect them, once they boot a newer kernel.
"Foobar support is about to be removed from kernel - please object to xyz if you see this message"

If after e.g. 10 kernel cycles no one objected, it can be assumed that no one cared too much about it.
Even newbie users tend to have a look in dmesg from time to time - atleast I teach that to my students - and a warning usually gets noticed.

We (used to) have similar warnings in the kernel log if a driver uses old apis ("this driver still uses old v4l..." ), why not introduce something similar for support deprecation?



Just an idea.
Peter

-- 
Sent from my mobile

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Greg KH]

On Wed, Sep 12, 2018 at 08:49:05AM +0200, Peter Huewe wrote:
> 
> 
> Am 12. September 2018 08:26:31 MESZ schrieb Greg KH <gregkh@linuxfoundation.org>:
> >On Wed, Sep 12, 2018 at 12:39:34AM +0300, Laurent Pinchart wrote:
> >> Hi Greg,
> >> 
> >> On Tuesday, 11 September 2018 22:33:08 EEST Greg KH wrote:
> >> > On Tue, Sep 11, 2018 at 11:37:25AM +0200, Lukasz Majewski wrote:
> >> > > In the kernel community we pose a lot of attention to security
> >(for
> >> > > example the prompt reaction on meltdown/spectre), but in the same
> >time
> >> > > we tend to forget about the "long lived" devices and force their
> >> > > maintainers to use 2.6.x kernels..... (or even 2.4.x).
> >> > 
> >> > We care, but really, how much can we do here?
> >> > 
> >> > I've been working a lot with the Adroid ecosystem to try to help
> >fix
> >> > their bad habits of "grab a random kernel and ship it and never
> >update
> >> > it" by providing longer lived kernels that they can constantly
> >update
> >> > their devices to.
> >> > 
> >> > But their lifetimes is much shorter compared to yours, and I have
> >no
> >> > insight into what kernels are being used, what configurations you
> >all
> >> > care about, and how long you need/want them updated.
> >> > 
> >> > Working with really old kernels like you have, without hardware
> >> > available to test is a hard task.  If your hardware is in a system
> >like
> >> > kernelci, then you can be sure that any new kernel will work
> >properly
> >> > with your system and then you might not want to have to stay with
> >really
> >> > old kernels that no one can maintain :)
> >> > 
> >> > There's a Linux Foundation project, "CIP" that wants to maintain
> >kernels
> >> > for devices like what you are making for 20+ years.  They are
> >having the
> >> > problems of not knowing exactly what platforms they wish to
> >support, but
> >> > their goal is good, hopefully they eventually nail something down
> >and we
> >> > can work together.  Perhaps you should contact them to try to help
> >solve
> >> > this issue for everyone?
> >> 
> >> I may be wrong, but I understand Lukasz's comment as the exact
> >opposite: we 
> >> forget about long-lived devices and drop their support while they're
> >still in 
> >> active use, forcing vendors to start using old and unsupported
> >kernels. If a 
> >> large number of ARMv4(T) devices are still being actively deployed
> >and 
> >> maintain, we should treat them as first-class citizens.
> >
> >No, I never want to drop support for anything that anyone is actually
> >using.  When I drop support for hardware/protocols, it goes through the
> >staging tree for a few months to see if anyone says anything.  If not,
> >then it gets removed.  And even then, if someone comes back and says
> >"hey I use that!" it's a simple revert to get it back.
> Can/should everything that we think is broken/unused be moved to staging first?

It has been used this way, but no, it's not necessary.  Some maintainers
do it to give people a chance to notice easier than the code just
disappearing one release.

> Isn't this unecessary churn of code?

'git mv' is very simple to use :)

> The mandarory todo file would then consist of which action items?

Look at the previous examples of this happening for how this is usually
worded.  Something like "this code is obsolete or broken and will be
deleted on XXX unless someone notifies us that they use it and can help
out with fixing these issues: ..."

> >The problem is if no one _tells_ us they are using something.  And
> >that's not something we can fix without help from the developers/users
> >of those hardware platforms talking to us.
> 
> Would it make sense to introduce something like the staging warning ('this driver is from stagin directory' message in dmesg) for support deprecation?

We have that today, with a taint flag, have you not noticed it? :)

> So if a maintainer, who probably has the best *available* picture about hw usage, thinks it's about time to remove the support, he adds a depecration flag to kconfig, which then adds code to emit a warning to dmesg.
> Users then get at least informed that deprecation would affect them, once they boot a newer kernel.
> "Foobar support is about to be removed from kernel - please object to xyz if you see this message"

No one reads kernel log messages :)

> If after e.g. 10 kernel cycles no one objected, it can be assumed that no one cared too much about it.

10 kernel cycles is how many _years_?  We normally wait 6-8 months
instead (3 releases or so.)

> Even newbie users tend to have a look in dmesg from time to time - atleast I teach that to my students - and a warning usually gets noticed.
> 
> We (used to) have similar warnings in the kernel log if a driver uses old apis ("this driver still uses old v4l..." ), why not introduce something similar for support deprecation?

No one used to care or do anything about those old api messages, as was
evident when no one ever fixed up the code, so that way really does not
work well.

thanks,

greg k-h

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 534 bytes --]

On Tue, Sep 11, 2018 at 10:37:36AM +0200, Linus Walleij wrote:

> Sometimes I get the feeling that people focused on desktops
> or servers suffer from velocitate (speed blindness) and think
> everybody is like them. (Well don't we all.)

Conversely, the approaches that keep stable kernels going also get
applied to designing hardware that people intend to be stable - if 
it works well enough now why try something new, especially where that
new stuff is most likely going to be more expensive to either buy or
work with?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Linus Walleij]

Found some moxa.com mail addresses in the kernel log,
so including them. You can speak about the status of the
MOXA ART deployments if you like, I am talking about


On Tue, Sep 11, 2018 at 12:52 PM Mark Brown <broonie@kernel.org> wrote:
> On Tue, Sep 11, 2018 at 10:37:36AM +0200, Linus Walleij wrote:
>
> > Sometimes I get the feeling that people focused on desktops
> > or servers suffer from velocitate (speed blindness) and think
> > everybody is like them. (Well don't we all.)
>
> Conversely, the approaches that keep stable kernels going also get
> applied to designing hardware that people intend to be stable - if
> it works well enough now why try something new, especially where that
> new stuff is most likely going to be more expensive to either buy or
> work with?

Exactly. I think this is what happened at MOXA with the ART SoC.

It seems that MOXA made that with a bunch of consultants from
Faraday in Taiwan supplying their IP blocks and know-how. Since
then they have had zero incentive to upgrade the SoC to a newer
version, as this SoC handles the ethernet, GPIO and serial lines
they need just fine.

They recently made a new design using Freescale (now NXP)
LS1021a dualcore A7, but I can see that the ART (ARMv4) in
EM-1240 and their EP93xx products (ARMv4T) are still in active
deployment and even recommended for new projects, samples can
be easily obtained.[1]

And this is good stuff, it's rock solid at this point. I just wish and
hope that they really invest in using the very latest kernels
also for these elder boards even though I rarely hear from them
on the mailing lists.

At companies that have their internal ASIC design teams I
sometimes get the feeling that these are churning out newer
and more powerful ASICs as a therapeutic exercise just to
keep their skills up. But I hope it is pure prejudice on my
part.

Yours,
Linus Walleij

[1] https://www.moxa.com/product/compact_fanless_computers.htm

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[SZ Lin (林上智)]

Hi,

<snip>

> Found some moxa.com mail addresses in the kernel log, so including them. You can speak
> about the status of the MOXA ART deployments if you like, I am talking about

Sorry for the late reply and thanks for pinging me.

Please note that the opinions are my own and not the views of my employer.

> On Tue, Sep 11, 2018 at 12:52 PM Mark Brown <broonie@kernel.org> wrote:
> > On Tue, Sep 11, 2018 at 10:37:36AM +0200, Linus Walleij wrote:
> >
> > > Sometimes I get the feeling that people focused on desktops or
> > > servers suffer from velocitate (speed blindness) and think everybody
> > > is like them. (Well don't we all.)
> >
> > Conversely, the approaches that keep stable kernels going also get
> > applied to designing hardware that people intend to be stable - if it
> > works well enough now why try something new, especially where that new
> > stuff is most likely going to be more expensive to either buy or work
> > with?
> 
> Exactly. I think this is what happened at MOXA with the ART SoC.
> 
> It seems that MOXA made that with a bunch of consultants from Faraday in Taiwan
> supplying their IP blocks and know-how. Since then they have had zero incentive to
> upgrade the SoC to a newer version, as this SoC handles the ethernet, GPIO and serial
> lines they need just fine.

Indeed.

> 
> They recently made a new design using Freescale (now NXP) LS1021a dualcore A7, but I
> can see that the ART (ARMv4) in
> EM-1240 and their EP93xx products (ARMv4T) are still in active deployment and even
> recommended for new projects, samples can be easily obtained.[1]

Both of them work well with variety of vertical markets, it depends on application
needs.

For example, Moxart is not suitable for performance-related applications which are
using gigabit Ethernet and so on.

> 
> And this is good stuff, it's rock solid at this point. I just wish and hope that they really
> invest in using the very latest kernels also for these elder boards even though I rarely hear
> from them on the mailing lists.

To have long-term maintenance kernel is very important for industrial application,
that is why Moxa joined CIP [1].

[1] https://www.cip-project.org/announcement/2018/01/18/industry-leader-moxa-joins-civil-infrastructure-platform-project

> 
> At companies that have their internal ASIC design teams I sometimes get the feeling that
> these are churning out newer and more powerful ASICs as a therapeutic exercise just to
> keep their skills up. But I hope it is pure prejudice on my part.

It's not true for Moxa, Moxa added some specific Industrial-related IP into Moxart.

SZ
> 
> Yours,
> Linus Walleij
> 
> [1] https://www.moxa.com/product/compact_fanless_computers.htm

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Thomas Gleixner]

[-- Attachment #1: Type: text/plain, Size: 1348 bytes --]

On Thu, 27 Sep 2018, SZ Lin (林上智) wrote:

> > And this is good stuff, it's rock solid at this point. I just wish and
> > hope that they really invest in using the very latest kernels also for
> > these elder boards even though I rarely hear from them on the mailing
> > lists.
> 
> To have long-term maintenance kernel is very important for industrial application,
> that is why Moxa joined CIP [1].

That's not the question here. The question here is how do we keep old
hardware supported in the latest kernel and who makes sure that it is
supported.

CIP is trying to keep a dead kernel in zombie state for a very long time
and as recent events (the whole speculation mess) have shown this is not
really a sane approach because backporting of invasive security fixes is
anything between impossible, incomplete and outright dangerous.

So for events like Meltdown/Spectre/L1TF the right answer is to be able to
switch to a recent kernel on that old hardware, but that requires that the
old hardware is still properly supported.

That's what people are asking here: What to do with old hardware support in
the latest kernel. If it's just bitrotting because nobody cares, then we
rather remove it. If it should be kept supported, then someone has to step
up and make sure it is, manufacturer being the obvious candidate.

Thanks,

	tglx

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Arnd Bergmann]

On Tue, Sep 11, 2018 at 10:37 AM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> Including Alexander Sverdlin, Lukasz Majewski  and Jonas
> Jensen here, they may or may not be able to share some of
> their industrial IoT experience. (Contract terms with vendors
> may make it necessary to stay silent sometimes.)

Thanks

> On Mon, Sep 10, 2018 at 11:41 PM Arnd Bergmann <arnd@arndb.de> wrote:
> > On Mon, Sep 10, 2018 at 5:35 PM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> > > My ARMv4 is another example, but I can point at new devices
> > > beging deployed as we speak, using that ISA, even though it is
> > > from 1999. So it has many active users (and maintainers).
> >
> > Note that even though gcc is dropping ARMv4 support from new
> > compilers, you can still use old toolchains, and there are tricks to
> > make ARMv4T binary code work on ARMv4. However, if gcc
> > ever stops supporting ARMv4T, this becomes a problem. My guess
> > is that will take another 10 years though, and we might have
> > removed some or all of the individual ARMv4 platforms by then.
>
> ARMv4 is becoming a trouble, not that it is hard to maintain,
> actually we're on top of things there. The problem is that among
> the FA526 systems from Faraday and the ARMv4T in EP93xx
> there are very serious IoT deployments that have been going
> on for soon 15 years and continuing.
>
> New MOXA ART ARMv4 FA526 systems are being deployed
> in buildings across the planet as we speak. They just replaced
> one in the office block where I sit, that is how I got to know.
>
> These are mostly for ventilation and
> similar systems but also heavy duty from Liebherr controlling
> unspecified hydraulic systems. The ventilation systems are
> definately Internet-connected, I don't know about the others.
>
> These pose an increasing security threat, and for that reason
> I personally feel it is irresponsible to remove the option to
> create new kernels and upgrade these devices.

I think it's likely that each of those platforms will get removed
eventually, but we probably need to get better about documenting
the constraints when we add a platform so we have a chance
to find out if there are remaining users when it gets in the way
10 or 15 years later. One common trick is to find something
that is known to have been broken for years as a proof that
nobody cares about it in mainline (we try not to break
things intentionally but after years of untested cleanup patches,
something is likely to break eventually). This of course prevents
you from upgrading kernels only every few years to the latest
LTS version.

Generally speaking, you always have a mix of two models
in (good) products: you provide upgrades based on the
latest kernels as long as it makes sense, but eventually
you settle on some stable/lts/slts version that hopefully
gives you security updates to the end of the expected life
of the hardware, with otherwise minimal changes.

Ideally we'd coordinate between kernel developers and
downstream users here, so we can do our best to keep
a platform alive up to a particular release that then becomes
the base for the SLTS release to last the remaining hardware
life after users are no longer planning to upgrade kernel versions.
This again breaks down when you have the next Spectre
or y2038 type event that forces everyone to upgrade.

I wonder if we can assume that there is a natural limit of
18-20 years between such events that prevents people
from running really old software in a safe way ;-)

> I think for depreciation one has to be aware that some archs
> used in IoT deployments have life cycles of 20-30 or more years,
> whereas some tablet or handset SoCs may be something like
> 5 years maximum before maintainers get annoyed that you
> even use them.

If you have specific examples for the 20-30 year deployment
that you know is running Linux on ARM9, please point me to
pictures that I can include them in my y2038 presentation.

        Arnd

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Alexander Sverdlin]

Hi!

On 11/09/2018 13:53, Arnd Bergmann wrote:
> If you have specific examples for the 20-30 year deployment
> that you know is running Linux on ARM9, please point me to
> pictures that I can include them in my y2038 presentation.

These devices were installed since 2006 all over Russia (mostly
in block houses):

http://wiki.tekon.ru/index.php?title=%D0%9A%D0%A3%D0%9D-IPM
http://wiki.tekon.ru/index.php?title=%D0%A4%D0%B0%D0%B9%D0%BB:Kun_ipmv1.jpg
http://wiki.tekon.ru/index.php?title=%D0%A4%D0%B0%D0%B9%D0%BB:Kun_ipmv3.jpg

They are still in service and their operators are not motivated to
swap them for newer generations. It could happen that they will approach
their 20 years anniversary in service. But I really hope it will not
be 20 more years from now :)

--
Alex.

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Deprecation / Removal of old hardware support

[Alexander Sverdlin]

Hello!

On 11/09/2018 10:37, Linus Walleij wrote:
> Including Alexander Sverdlin, Lukasz Majewski  and Jonas
> Jensen here, they may or may not be able to share some of
> their industrial IoT experience. (Contract terms with vendors
> may make it necessary to stay silent sometimes.)

Yes, that's the reason I took the maintainer role for EP93xx.
I'm as well aware of several thousands of industrial devices which are
expected to run 24/7 for the next 5 years at least.
Linux firmware on them is still updated from time to time.

[...]

> ARMv4 is becoming a trouble, not that it is hard to maintain,
> actually we're on top of things there. The problem is that among
> the FA526 systems from Faraday and the ARMv4T in EP93xx
> there are very serious IoT deployments that have been going
> on for soon 15 years and continuing.

--
Alex.