(Resend) 2038 Kernel Summit Discussion Fodder

(Resend) 2038 Kernel Summit Discussion Fodder

[John Stultz]

(Forgive the resend, somehow despite being in plain-text mode, the gmail
compose interface somehow added some html-ism that caused lkml to bounce
this)

Next week at Kernel Summit, I'm hoping to have a discussion on 2038 during
one of the unconference slots, so I wanted to send out some background
information about the various proposed approaches (as well as send this out
for wider input, since apparently there won't be much libc representation
at KS).

So 2038 brings the end of time for 32bit architectures.  It being some
twenty four years ahead, it may seem like there is plenty of time for folks
to migrate to 64bit architectures that are (mostly) unaffected by this
issue. However, 32bit processors are still being produced today in
extremely high volumes, and many of those systems are being used in
commercial, industrial and medical environments, where these systems may be
quite literally embedded into the walls and machinery and are expected to
run for 25 years or more. As these small systems become more and more
pervasive, the risks of major trouble in 2038 grow. And that’s to say
nothing of the impact on future classic-car resale prices for fancy cars
like the Tesla when the high end in-dash display won’t work (gasp!).

Thus, the “just upgrade to 64bit” solution isn’t really sufficient, and we
need to find a transition plan soon, which will allow 32bit cpus being sold
today and in the future to function correctly past 2038.

Other OSes have already started rolling out solutions. NetBSD switched to
64bit time_t in 2012, providing a compatibility layer for old applications.
Then in 2013, OpenBSD switched its 32bit systems time_t to use long long.
You can read more about the OpenBSD transition here:
http://www.openbsd.org/papers/eurobsdcon_2013_time_t/. One item to note is
that NetBSD preserved compatibility with 2038 unsafe applications, while
OpenBSD did not (seeing the certainty of excluding unsafe applications as a
feature over compatibility). For Linux, we obviously want to maintain
compatibility, but I think its important we recognise we’re not the first
movers here, and many of the discussions we’ll be having have already been
had in those communities. I also think we should be sure to credit the
NetBSD and OpenBSD devs for jumping in on this early and working to ensure
issues in the userspace applications we share are already addressed.

Also, just to clarify, as time related discussions can bring out a laundry
list of issues, I would like to focus this discussion on providing a 2038
solution for existing interfaces and applications in a way that ideally
doesn't require modifying application source code.  While there will be
plenty of places where applications have cast or stored time_t values
explicitly as longs, and for those applications, deep modifications will be
necessary. But I’d like to avoid getting into new-interface discussions,
like exporting ktime_t like nanosecond interfaces instead of timepsecs,
unifying time-stamping formats, or methods for avoiding leapseconds. Those
are all interesting issues, and I’d be up for discussing them separately,
but those issue apply equally to 32bit and 64bit systems, and really aren't
2038 specific, so I think its best to separate them out.

I think any solution to the 2038 issue for Linux, from the kernel’s
perspective, will have a number of phases:

1) In-kernel correctness: Making sure the kernel itself handles the 2038
rollover correctly. As noted in LWN (https://lwn.net/Articles/607741/) this
work has begun and portions have been merged. Personally, I don’t really
see this as that interesting of a discussion topic, and with the exception
of dealing with external representations of time like on-disk filesystem
formats, etc, it is for the most part a matter of just going through,
finding and fixing things (with a few extra complications around ioctls).
Mostly this will revolve around adding explicitly sized 64bit time_t types
(and 32bit types for compatibility), and migrating in-kernel users over to
the explicitly sized implementations, allowing us to validate conversion by
eventually removing the in-kernel time_t type, of course finding non-time_t
custom storage uses will be a long tail of work.

2) Userspace ABI modifications: This includes how we expose the new 64bit
time_t and related structures to userland via syscalls and ioctls, and how
we preserve compatibility to older applications. This is probably the most
complex issue, as the different choices have large impacts to how the
transition in userspace is done.

3) Aiding in validating userspace correctness: While we want to preserve
compatibility, there is the very real aspect that 2038 unsafe applications
(ie: almost all 32bit applications today) are terminally broken in 2038.
While we can try to ensure that we don’t break those applications
prematurely, we do want to ensure folks using unsafe applications are aware
and motivated to upgrade to 2038-safe versions quickly. This might include
warnings when unsafe syscalls are used, or options to disable unsafe
syscalls, as well as maybe debug modes for testing where if you set the
date to past 2038, the kernel will be extra verbose if any truncated
absolute time_t values are observed (hinting that an application cast to a
long and back).

So what I’d like to cover in this mail, are some discussion starters around
ABI modifications that I think we should discuss at Kernel Summit, in order
to make sure we have a clear path forward, as what we decide for the ABI
modifications, has large impacts on how we help ensure userspace is correct
or not.


>From discussions so far, it seems the preferred change to the userspace
interface is what I’ll call the “Large File” method, as it follows the
approach used for large file support:

Create new 64bit time_t/timespec/timeval/etc variants for syscalls, while
preserving existing interfaces. This has some complexity around IOCTLs, but
that can mostly be handled by creating new ioctl numbers while preserving
the old ones. Since we’re only modifying time types, we’ll also need to add
compat versions for many of these syscalls for 64bit native systems.

Libc then introduces versioned symbols, and a new compile options to allow
applications to be built for “large time”. New and old applications could
then share the same libc.

The benefits of this approach is is simply and minimally extends the
current 32 bit environment, without any effect on existing applications
which continue to work. Most of the complexity is in the libc library and
its build environment.

The downsides to this approach is that as it follows the large-file
approach, it has many of the same problems as large-file support, in that
the transition to large-file has been slow and is still ongoing. Also,
since this solution focuses on libc, there is also the problem of existing
3rd party libraries, which have no way of knowing which sized time is being
used, will break. So all libraries that do anything with time will then
have to implement their own versioned interfaces. This approach also makes
it a little more difficult to audit that a system is 2038 safe, without
running it and looking for issues.


A potential alternative I’d like to also propose is the “Libc Version Bump”
approach.

Basically this is the same as the above, where the kernel provides both
legacy and new time_t related interfaces. However, the libc would make a
version break, migrating to using 64bit time_t types and syscalls. Legacy
applications would still work using the old glibc version, but this would
provide a stronger line in the sand between 2038 safe and unsafe
applications and libraries, making it easier to avoid mixing the two.
NetBSD developers discussed this same approach back in 2008 here:
https://mail-index.netbsd.org/tech-userlevel/2008/03/22/msg000231.html

The downsides here is, for legacy application support, one would have to
have all the requisite legacy libraries also installed, which will add a
burden to distro vendors. However, this extra storage overhead would likely
be a positive motivator to get applications rebuilt and migrated to new
version. Additionally, for 3rd party libraries built against the new libc
version, the libraries may need to do a version bump themselves, in order
to be able to co-exist with versions built against the previous libc. This
approach also assumes that libraries that use time_t related values would
have a libc dependency.

A more aggressive version of the previous proposal is what I’m calling the
“New Virtual-Architecture” approach, basically extending the versioning
control from the linker down into the kernel as well.  It would be adding a
new “virtual-architecture” to the kernel, not entirely unlike how x32 is
supported on x86_64 systems.  We would create entirely new ABI and
architecture name in the kernel (think something like “armllt” or
“i386llt”). We would preserve compatibility for legacy applications via
personalities, similar mechanism as the compat_ interface used to support
32bit applications on 64bit kernels. In this case, we wouldn’t introduce
new 64 bit syscalls in the kernel, as the existing interfaces would just be
typed correctly for our new virtual architecture, but we would have
duplicate syscall interfaces via the compat interfaces. The extra
complexity would also be that we would have to support new 32bit compat
environment on 64bit systems. Userspace would be completely rebuilt to
support the new -llt architecture, and compatibility for legacy
applications would be done via the same multiarch packaging as is done now
for running 32bit applications on 64bit systems.

The pros for this case is that it would be very easy to audit that
applications have migrated to the new 64bit time_t ABI. Additionally since
we know which type the application is in the kernel, it would make
problematic compatability areas like IOCTLS easier to deal with utilizing a
flag in the task structure.

The downsides here are many. The distros will probably hate this idea, as
it requires rebuilding the world, and maintaining another legacy
architecture support. I’m also not completely sure how robust multi-arch
packaging is in the face of having to handle 3-4 architectures on one
system. On the kernel side, it also adds more complexity, where we have to
add even more complex compat support for 64bit systems to handle all the
various 32bit applications possible.

That said the practical reality isn't much further from the the “Libc
Version Bump” approach, since legacy support will need legacy versions of
all dependent libraries there a well. Similarly the additional storage
required to support legacy applications is a positive motivator to get
folks to move away from unsafe legacy applications. I also personally like
the clarity the new virtual architecture brings, and that it would allow a
kernel option to disable 2038-unsafe legacy support.

With any of these approaches, we still have quite a bit of work to just get
the kernel in-shape internally. And with the exception of the “virtual
arch” approach, the changes on the kernel side are basically the same. The
big thing we probably want to avoid is requiring any sort of flagday for
distros, and instead allowing them each to migrate to the new solution
individually (but hopefully not taking too long). Even so, I think having a
clear vision for how userspace will make this transition is important, so
hopefully during the Kernel Summit discussion we can come to consensus on
what approach to take moving forward.

Anyway, this is probably more then enough to read and think about in the
next week. I’d be very interested in further thoughts or alternative
proposals to discuss.

Thanks to Thomas Gleixner, Arnd Bergmann, Mark Brown, Joseph Myers and
others for their thoughts and proposals which I used to create this summary.

Also, Arnd has been keeping 2038 related details (most usefully on the
non-time_t 2038 concerns) on the kenrelnewbies wiki here:
http://kernelnewbies.org/y2038

thanks
-john

Re: [Ksummit-discuss] (Resend) 2038 Kernel Summit Discussion Fodder

[josh]

On Tue, Aug 12, 2014 at 05:08:53PM -0700, John Stultz wrote:
> Also, just to clarify, as time related discussions can bring out a laundry
> list of issues, I would like to focus this discussion on providing a 2038
> solution for existing interfaces and applications in a way that ideally
> doesn't require modifying application source code.  While there will be
> plenty of places where applications have cast or stored time_t values
> explicitly as longs, and for those applications, deep modifications will be
> necessary. But I’d like to avoid getting into new-interface discussions,
> like exporting ktime_t like nanosecond interfaces instead of timepsecs,
> unifying time-stamping formats, or methods for avoiding leapseconds. Those
> are all interesting issues, and I’d be up for discussing them separately,
> but those issue apply equally to 32bit and 64bit systems, and really aren't
> 2038 specific, so I think its best to separate them out.

That's understandable.  However, I wonder to what extent we could
support unmodified source code via libc wrappers (since code calling
syscalls directly can't work completely unmodified), while using better
interfaces for new syscalls.  Given syscalls written in terms of (for
instance) nanoseconds rather than timespec values, it seems
straightforward enough for libc to provide compatibility interfaces.

> From discussions so far, it seems the preferred change to the userspace
> interface is what I’ll call the “Large File” method, as it follows the
> approach used for large file support:
> 
> Create new 64bit time_t/timespec/timeval/etc variants for syscalls, while
> preserving existing interfaces. This has some complexity around IOCTLs, but
> that can mostly be handled by creating new ioctl numbers while preserving
> the old ones. Since we’re only modifying time types, we’ll also need to add
> compat versions for many of these syscalls for 64bit native systems.
> 
> Libc then introduces versioned symbols, and a new compile options to allow
> applications to be built for “large time”. New and old applications could
> then share the same libc.
> 
> The benefits of this approach is is simply and minimally extends the
> current 32 bit environment, without any effect on existing applications
> which continue to work. Most of the complexity is in the libc library and
> its build environment.
> 
> The downsides to this approach is that as it follows the large-file
> approach, it has many of the same problems as large-file support, in that
> the transition to large-file has been slow and is still ongoing. Also,
> since this solution focuses on libc, there is also the problem of existing
> 3rd party libraries, which have no way of knowing which sized time is being
> used, will break. So all libraries that do anything with time will then
> have to implement their own versioned interfaces. This approach also makes
> it a little more difficult to audit that a system is 2038 safe, without
> running it and looking for issues.

If we go this route, we should also provide a "depends on EMBEDDED"
Kconfig option that omits all of the compatibility support, for systems
that have fully migrated to new userspace.

> A potential alternative I’d like to also propose is the “Libc Version Bump”
> approach.
> 
> Basically this is the same as the above, where the kernel provides both
> legacy and new time_t related interfaces. However, the libc would make a
> version break, migrating to using 64bit time_t types and syscalls. Legacy
> applications would still work using the old glibc version, but this would
> provide a stronger line in the sand between 2038 safe and unsafe
> applications and libraries, making it easier to avoid mixing the two.
> NetBSD developers discussed this same approach back in 2008 here:
> https://mail-index.netbsd.org/tech-userlevel/2008/03/22/msg000231.html
> 
> The downsides here is, for legacy application support, one would have to
> have all the requisite legacy libraries also installed, which will add a
> burden to distro vendors. However, this extra storage overhead would likely
> be a positive motivator to get applications rebuilt and migrated to new
> version. Additionally, for 3rd party libraries built against the new libc
> version, the libraries may need to do a version bump themselves, in order
> to be able to co-exist with versions built against the previous libc. This
> approach also assumes that libraries that use time_t related values would
> have a libc dependency.

The migration pain here doesn't seem worth it at all.

> A more aggressive version of the previous proposal is what I’m calling the
> “New Virtual-Architecture” approach, basically extending the versioning
> control from the linker down into the kernel as well.  It would be adding a
> new “virtual-architecture” to the kernel, not entirely unlike how x32 is
> supported on x86_64 systems.  We would create entirely new ABI and
> architecture name in the kernel (think something like “armllt” or
> “i386llt”). We would preserve compatibility for legacy applications via
> personalities, similar mechanism as the compat_ interface used to support
> 32bit applications on 64bit kernels. In this case, we wouldn’t introduce
> new 64 bit syscalls in the kernel, as the existing interfaces would just be
> typed correctly for our new virtual architecture, but we would have
> duplicate syscall interfaces via the compat interfaces. The extra
> complexity would also be that we would have to support new 32bit compat
> environment on 64bit systems. Userspace would be completely rebuilt to
> support the new -llt architecture, and compatibility for legacy
> applications would be done via the same multiarch packaging as is done now
> for running 32bit applications on 64bit systems.

I wonder: could we make this new architecture effectively use the
signatures of the 64-bit syscalls (similar to x32), just with a 32-bit
calling convention?

- Josh Triplett

Re: [Ksummit-discuss] (Resend) 2038 Kernel Summit Discussion Fodder

[Andy Lutomirski]

On Tue, Aug 12, 2014 at 6:33 PM,  <josh@joshtriplett.org> wrote:
> On Tue, Aug 12, 2014 at 05:08:53PM -0700, John Stultz wrote:
>> Also, just to clarify, as time related discussions can bring out a laundry
>> list of issues, I would like to focus this discussion on providing a 2038
>> solution for existing interfaces and applications in a way that ideally
>> doesn't require modifying application source code.  While there will be
>> plenty of places where applications have cast or stored time_t values
>> explicitly as longs, and for those applications, deep modifications will be
>> necessary. But I’d like to avoid getting into new-interface discussions,
>> like exporting ktime_t like nanosecond interfaces instead of timepsecs,
>> unifying time-stamping formats, or methods for avoiding leapseconds. Those
>> are all interesting issues, and I’d be up for discussing them separately,
>> but those issue apply equally to 32bit and 64bit systems, and really aren't
>> 2038 specific, so I think its best to separate them out.
>
> That's understandable.  However, I wonder to what extent we could
> support unmodified source code via libc wrappers (since code calling
> syscalls directly can't work completely unmodified), while using better
> interfaces for new syscalls.  Given syscalls written in terms of (for
> instance) nanoseconds rather than timespec values, it seems
> straightforward enough for libc to provide compatibility interfaces.
>
>> From discussions so far, it seems the preferred change to the userspace
>> interface is what I’ll call the “Large File” method, as it follows the
>> approach used for large file support:
>>
>> Create new 64bit time_t/timespec/timeval/etc variants for syscalls, while
>> preserving existing interfaces. This has some complexity around IOCTLs, but
>> that can mostly be handled by creating new ioctl numbers while preserving
>> the old ones. Since we’re only modifying time types, we’ll also need to add
>> compat versions for many of these syscalls for 64bit native systems.
>>
>> Libc then introduces versioned symbols, and a new compile options to allow
>> applications to be built for “large time”. New and old applications could
>> then share the same libc.
>>
>> The benefits of this approach is is simply and minimally extends the
>> current 32 bit environment, without any effect on existing applications
>> which continue to work. Most of the complexity is in the libc library and
>> its build environment.
>>
>> The downsides to this approach is that as it follows the large-file
>> approach, it has many of the same problems as large-file support, in that
>> the transition to large-file has been slow and is still ongoing. Also,
>> since this solution focuses on libc, there is also the problem of existing
>> 3rd party libraries, which have no way of knowing which sized time is being
>> used, will break. So all libraries that do anything with time will then
>> have to implement their own versioned interfaces. This approach also makes
>> it a little more difficult to audit that a system is 2038 safe, without
>> running it and looking for issues.
>
> If we go this route, we should also provide a "depends on EMBEDDED"
> Kconfig option that omits all of the compatibility support, for systems
> that have fully migrated to new userspace.
>
>> A potential alternative I’d like to also propose is the “Libc Version Bump”
>> approach.
>>
>> Basically this is the same as the above, where the kernel provides both
>> legacy and new time_t related interfaces. However, the libc would make a
>> version break, migrating to using 64bit time_t types and syscalls. Legacy
>> applications would still work using the old glibc version, but this would
>> provide a stronger line in the sand between 2038 safe and unsafe
>> applications and libraries, making it easier to avoid mixing the two.
>> NetBSD developers discussed this same approach back in 2008 here:
>> https://mail-index.netbsd.org/tech-userlevel/2008/03/22/msg000231.html
>>
>> The downsides here is, for legacy application support, one would have to
>> have all the requisite legacy libraries also installed, which will add a
>> burden to distro vendors. However, this extra storage overhead would likely
>> be a positive motivator to get applications rebuilt and migrated to new
>> version. Additionally, for 3rd party libraries built against the new libc
>> version, the libraries may need to do a version bump themselves, in order
>> to be able to co-exist with versions built against the previous libc. This
>> approach also assumes that libraries that use time_t related values would
>> have a libc dependency.
>
> The migration pain here doesn't seem worth it at all.
>
>> A more aggressive version of the previous proposal is what I’m calling the
>> “New Virtual-Architecture” approach, basically extending the versioning
>> control from the linker down into the kernel as well.  It would be adding a
>> new “virtual-architecture” to the kernel, not entirely unlike how x32 is
>> supported on x86_64 systems.  We would create entirely new ABI and
>> architecture name in the kernel (think something like “armllt” or
>> “i386llt”). We would preserve compatibility for legacy applications via
>> personalities, similar mechanism as the compat_ interface used to support
>> 32bit applications on 64bit kernels. In this case, we wouldn’t introduce
>> new 64 bit syscalls in the kernel, as the existing interfaces would just be
>> typed correctly for our new virtual architecture, but we would have
>> duplicate syscall interfaces via the compat interfaces. The extra
>> complexity would also be that we would have to support new 32bit compat
>> environment on 64bit systems. Userspace would be completely rebuilt to
>> support the new -llt architecture, and compatibility for legacy
>> applications would be done via the same multiarch packaging as is done now
>> for running 32bit applications on 64bit systems.
>
> I wonder: could we make this new architecture effectively use the
> signatures of the 64-bit syscalls (similar to x32), just with a 32-bit
> calling convention?

Doesn't x32 do the reverse?  It invokes *compat* syscalls using a
64-bit calling convention.

--Andy

Re: [Ksummit-discuss] (Resend) 2038 Kernel Summit Discussion Fodder

[John Stultz]

On 08/12/2014 06:33 PM, josh@joshtriplett.org wrote:
> On Tue, Aug 12, 2014 at 05:08:53PM -0700, John Stultz wrote:
>> Also, just to clarify, as time related discussions can bring out a laundry
>> list of issues, I would like to focus this discussion on providing a 2038
>> solution for existing interfaces and applications in a way that ideally
>> doesn't require modifying application source code.  While there will be
>> plenty of places where applications have cast or stored time_t values
>> explicitly as longs, and for those applications, deep modifications will be
>> necessary. But I’d like to avoid getting into new-interface discussions,
>> like exporting ktime_t like nanosecond interfaces instead of timepsecs,
>> unifying time-stamping formats, or methods for avoiding leapseconds. Those
>> are all interesting issues, and I’d be up for discussing them separately,
>> but those issue apply equally to 32bit and 64bit systems, and really aren't
>> 2038 specific, so I think its best to separate them out.
> That's understandable.  However, I wonder to what extent we could
> support unmodified source code via libc wrappers (since code calling
> syscalls directly can't work completely unmodified), while using better
> interfaces for new syscalls.  Given syscalls written in terms of (for
> instance) nanoseconds rather than timespec values, it seems
> straightforward enough for libc to provide compatibility interfaces.

So we'd be then introducing new syscalls not just for 32bit but 64bit
systems as well, and its a fairly substantial list:
    http://kernelnewbies.org/y2038

Again, I worry that just getting time_t to be 64bits is a big enough
problem to solve without trying to mix in interface preference
discussions along, adding totally new interfaces, coordinating all the
libcs out there to provide wrappers, and then deprecating the old
interfaces.

If there are more desirable interfaces wanted, I really think that needs
to be a separate topic from fixing the ones we have.


>
>> From discussions so far, it seems the preferred change to the userspace
>> interface is what I’ll call the “Large File” method, as it follows the
>> approach used for large file support:
>>
>> Create new 64bit time_t/timespec/timeval/etc variants for syscalls, while
>> preserving existing interfaces. This has some complexity around IOCTLs, but
>> that can mostly be handled by creating new ioctl numbers while preserving
>> the old ones. Since we’re only modifying time types, we’ll also need to add
>> compat versions for many of these syscalls for 64bit native systems.
>>
>> Libc then introduces versioned symbols, and a new compile options to allow
>> applications to be built for “large time”. New and old applications could
>> then share the same libc.
>>
>> The benefits of this approach is is simply and minimally extends the
>> current 32 bit environment, without any effect on existing applications
>> which continue to work. Most of the complexity is in the libc library and
>> its build environment.
>>
>> The downsides to this approach is that as it follows the large-file
>> approach, it has many of the same problems as large-file support, in that
>> the transition to large-file has been slow and is still ongoing. Also,
>> since this solution focuses on libc, there is also the problem of existing
>> 3rd party libraries, which have no way of knowing which sized time is being
>> used, will break. So all libraries that do anything with time will then
>> have to implement their own versioned interfaces. This approach also makes
>> it a little more difficult to audit that a system is 2038 safe, without
>> running it and looking for issues.
> If we go this route, we should also provide a "depends on EMBEDDED"
> Kconfig option that omits all of the compatibility support, for systems
> that have fully migrated to new userspace.

Right, though I think for any of the solutions, having methods to
actually remove support for the 2038 unsafe interfaces is important to
help ensure systems are completely converted.


>
>> A potential alternative I’d like to also propose is the “Libc Version Bump”
>> approach.
>>
>> Basically this is the same as the above, where the kernel provides both
>> legacy and new time_t related interfaces. However, the libc would make a
>> version break, migrating to using 64bit time_t types and syscalls. Legacy
>> applications would still work using the old glibc version, but this would
>> provide a stronger line in the sand between 2038 safe and unsafe
>> applications and libraries, making it easier to avoid mixing the two.
>> NetBSD developers discussed this same approach back in 2008 here:
>> https://mail-index.netbsd.org/tech-userlevel/2008/03/22/msg000231.html
>>
>> The downsides here is, for legacy application support, one would have to
>> have all the requisite legacy libraries also installed, which will add a
>> burden to distro vendors. However, this extra storage overhead would likely
>> be a positive motivator to get applications rebuilt and migrated to new
>> version. Additionally, for 3rd party libraries built against the new libc
>> version, the libraries may need to do a version bump themselves, in order
>> to be able to co-exist with versions built against the previous libc. This
>> approach also assumes that libraries that use time_t related values would
>> have a libc dependency.
> The migration pain here doesn't seem worth it at all.

Hmm. I see this as a more mechanical process, rather then the
large-file-style support, which requires libraries to all add versioned
interfaces. This approach uses the library version number to in-effect
version the interfaces. But it does require version bumps everywhere.

The basic point here is that for compatibility we have to pick a method
to chose the new or old interfaces. That can be internal to the
libraries at build time w/ versioned symbols, external to the libraries
by the linker, using their named version numbers, or by the kernel
loader itself with the virtual architecture.

Each step along the gradient has different trade-offs in how mechanical
it can be vs the level of duplication needed to support it. I think the
libc bump has the best balance, but think the distro folks might have
better insight as to the pain of what I'm suggesting.


>
>> A more aggressive version of the previous proposal is what I’m calling the
>> “New Virtual-Architecture” approach, basically extending the versioning
>> control from the linker down into the kernel as well.  It would be adding a
>> new “virtual-architecture” to the kernel, not entirely unlike how x32 is
>> supported on x86_64 systems.  We would create entirely new ABI and
>> architecture name in the kernel (think something like “armllt” or
>> “i386llt”). We would preserve compatibility for legacy applications via
>> personalities, similar mechanism as the compat_ interface used to support
>> 32bit applications on 64bit kernels. In this case, we wouldn’t introduce
>> new 64 bit syscalls in the kernel, as the existing interfaces would just be
>> typed correctly for our new virtual architecture, but we would have
>> duplicate syscall interfaces via the compat interfaces. The extra
>> complexity would also be that we would have to support new 32bit compat
>> environment on 64bit systems. Userspace would be completely rebuilt to
>> support the new -llt architecture, and compatibility for legacy
>> applications would be done via the same multiarch packaging as is done now
>> for running 32bit applications on 64bit systems.
> I wonder: could we make this new architecture effectively use the
> signatures of the 64-bit syscalls (similar to x32), just with a 32-bit
> calling convention?

So if you mean on 64bit systems, having the 2038-safe 32bit compat use
the same 64bit entry-points, I think that would be possible for some of
the syscalls, but there's some where other data fields in structures
won't be expected to be 64bit. But I might not be understanding what
your suggesting.

thanks
-john

Re: [Ksummit-discuss] (Resend) 2038 Kernel Summit Discussion Fodder

[Catalin Marinas]

On Wed, Aug 13, 2014 at 02:37:11AM +0100, Andy Lutomirski wrote:
> On Tue, Aug 12, 2014 at 6:33 PM,  <josh@joshtriplett.org> wrote:
> > On Tue, Aug 12, 2014 at 05:08:53PM -0700, John Stultz wrote:
> >> A more aggressive version of the previous proposal is what I’m calling the
> >> “New Virtual-Architecture” approach, basically extending the versioning
> >> control from the linker down into the kernel as well.  It would be adding a
> >> new “virtual-architecture” to the kernel, not entirely unlike how x32 is
> >> supported on x86_64 systems.  We would create entirely new ABI and
> >> architecture name in the kernel (think something like “armllt” or
> >> “i386llt”). We would preserve compatibility for legacy applications via
> >> personalities, similar mechanism as the compat_ interface used to support
> >> 32bit applications on 64bit kernels. In this case, we wouldn’t introduce
> >> new 64 bit syscalls in the kernel, as the existing interfaces would just be
> >> typed correctly for our new virtual architecture, but we would have
> >> duplicate syscall interfaces via the compat interfaces. The extra
> >> complexity would also be that we would have to support new 32bit compat
> >> environment on 64bit systems. Userspace would be completely rebuilt to
> >> support the new -llt architecture, and compatibility for legacy
> >> applications would be done via the same multiarch packaging as is done now
> >> for running 32bit applications on 64bit systems.
> >
> > I wonder: could we make this new architecture effectively use the
> > signatures of the 64-bit syscalls (similar to x32), just with a 32-bit
> > calling convention?
> 
> Doesn't x32 do the reverse?  It invokes *compat* syscalls using a
> 64-bit calling convention.

Not entirely. It uses 64-bit syscalls primarily with compat only when
sharing structures containing pointers. We are working on something
similar for arm64 (called the ILP32 ABI).

-- 
Catalin

Re: [Ksummit-discuss] (Resend) 2038 Kernel Summit Discussion Fodder

[Arnd Bergmann]

On Tuesday 12 August 2014 20:45:03 John Stultz wrote:
> On 08/12/2014 06:33 PM, josh@joshtriplett.org wrote:
> > On Tue, Aug 12, 2014 at 05:08:53PM -0700, John Stultz wrote:
> >> Also, just to clarify, as time related discussions can bring out a laundry
> >> list of issues, I would like to focus this discussion on providing a 2038
> >> solution for existing interfaces and applications in a way that ideally
> >> doesn't require modifying application source code.  While there will be
> >> plenty of places where applications have cast or stored time_t values
> >> explicitly as longs, and for those applications, deep modifications will be
> >> necessary. But I’d like to avoid getting into new-interface discussions,
> >> like exporting ktime_t like nanosecond interfaces instead of timepsecs,
> >> unifying time-stamping formats, or methods for avoiding leapseconds. Those
> >> are all interesting issues, and I’d be up for discussing them separately,
> >> but those issue apply equally to 32bit and 64bit systems, and really aren't
> >> 2038 specific, so I think its best to separate them out.
> > That's understandable.  However, I wonder to what extent we could
> > support unmodified source code via libc wrappers (since code calling
> > syscalls directly can't work completely unmodified), while using better
> > interfaces for new syscalls.  Given syscalls written in terms of (for
> > instance) nanoseconds rather than timespec values, it seems
> > straightforward enough for libc to provide compatibility interfaces.
> 
> So we'd be then introducing new syscalls not just for 32bit but 64bit
> systems as well, and its a fairly substantial list:
>     http://kernelnewbies.org/y2038
> 
> Again, I worry that just getting time_t to be 64bits is a big enough
> problem to solve without trying to mix in interface preference
> discussions along, adding totally new interfaces, coordinating all the
> libcs out there to provide wrappers, and then deprecating the old
> interfaces.
> 
> If there are more desirable interfaces wanted, I really think that needs
> to be a separate topic from fixing the ones we have.

Agreed. I originally argued the same ways as Josh in the conversations
we had a few weeks ago, but I think you are right.

It essentially comes down to the question what code paths we want to
change. Introducing completely new syscall ABIs means we also have to
change all 64-bit architectures to support those, while the plain
1:1 replacement means we only impact the 32-bit native and compat
(under a 64-bit kernel) cases that actually want this change.

> > If we go this route, we should also provide a "depends on EMBEDDED"
> > Kconfig option that omits all of the compatibility support, for systems
> > that have fully migrated to new userspace.
> 
> Right, though I think for any of the solutions, having methods to
> actually remove support for the 2038 unsafe interfaces is important to
> help ensure systems are completely converted.

Yes. The way I'd do this is to introduce a new 'CONFIG_COMPAT_TIME' symbol
that initially designates the architectures that are have assigned syscall
numbers to the newly introduced calls, turning e.g.

SYSCALL_DEFINE4(clock_nanosleep, const clockid_t, which_clock, int, flags,
                const struct timespec __user *, rqtp,
                struct timespec __user *, rmtp)
{
	...
}

into this:

#if defined(CONFIG_COMPAT_TIME) || defined(CONFIG_64BIT)
#define __kernel_timespec __kernel_timespec64
#else
#define __kernel_timespec __kernel_timespec32
#endif

#ifdef CONFIG_COMPAT_TIME
SYSCALL_DEFINE4(compat_clock_nanosleep, const clockid_t, which_clock, int, flags,
                const struct __kernel_timespec32 __user *, rqtp,
                struct __kernel_timespec32 __user *, rmtp)
{
	...
}
#endif

SYSCALL_DEFINE4(clock_nanosleep, const clockid_t, which_clock, int, flags,
                const struct __kernel_timespec __user *, rqtp,
                struct __kernel_timespec __user *, rmtp)
{
	...
}

Each 32-bit architecture (one at a time) has to change its syscall table
so the __NR_clock_nanosleep entry points to the sys_compat_clock_nanosleep
function, and it needs to assign a new number for sys_clock_nanosleep,
and also do the same thing for all other syscalls involving time, and then
set CONFIG_COMPAT_TIME unconditionally.

Once all architectures do this, we can turn CONFIG_COMPAT_TIME into a
user-selectable option and turn it off for any system that wants to ensure
it either works now or doesn't work, but does not break in 24 years from
now.

> > I wonder: could we make this new architecture effectively use the
> > signatures of the 64-bit syscalls (similar to x32), just with a 32-bit
> > calling convention?
> 
> So if you mean on 64bit systems, having the 2038-safe 32bit compat use
> the same 64bit entry-points, I think that would be possible for some of
> the syscalls, but there's some where other data fields in structures
> won't be expected to be 64bit. But I might not be understanding what
> your suggesting.

Of the syscalls I've looked at, all would be able to do it this way:
you already have to introduce a new data structure at the libc-kernel
interface, with the libc potentially translating it into a different
format at least for legacy applications. An interesting case is 'struct
stat': All architectures that don't use the asm-generic/stat.h file
have their own incompatible definition of this structure, and the
asm-generic one gets it wrong too (this is my fault I admit). For the
existing 32-bit architectures that also come with a 64-bit counterpart,
we have the choice to make each one use whatever their 64-bit ABI
uses, or alternatively make them all use the same structure. Either
way makes sense to me.

	Arnd

Re: (Resend) 2038 Kernel Summit Discussion Fodder

[Arnd Bergmann]

On Tuesday 12 August 2014 17:08:53 John Stultz wrote:

> Next week at Kernel Summit, I'm hoping to have a discussion on 2038 during
> one of the unconference slots, so I wanted to send out some background
> information about the various proposed approaches (as well as send this out
> for wider input, since apparently there won't be much libc representation
> at KS).

Thanks a lot for doing the writeup. I've been on vacation the past two weeks,
so I couldn't really contribute much during that time, but a lot of what
you write here is what we discussed earlier in private.

> 1) In-kernel correctness: Making sure the kernel itself handles the 2038
> rollover correctly. As noted in LWN (https://lwn.net/Articles/607741/) this
> work has begun and portions have been merged. Personally, I don’t really
> see this as that interesting of a discussion topic, and with the exception
> of dealing with external representations of time like on-disk filesystem
> formats, etc, it is for the most part a matter of just going through,
> finding and fixing things (with a few extra complications around ioctls).
> Mostly this will revolve around adding explicitly sized 64bit time_t types
> (and 32bit types for compatibility), and migrating in-kernel users over to
> the explicitly sized implementations, allowing us to validate conversion by
> eventually removing the in-kernel time_t type, of course finding non-time_t
> custom storage uses will be a long tail of work.

Some more thoughts on how I think this can proceed:

I think the most important tool here is compile-time validation, by ensuring
no use of time_t or a derived structure (timespec, timeval, itimerspec, ...)
can be compiled if you are building a kernel that you want to survive y2038.

Any drivers or file systems that we don't expect to need at that time
(e.g. ext2/3 rather than ext4) can keep using the unsafe interfaces
but will be disabled in Kconfig or removed from the code using #ifdef
in case we are building a 64-bit-time-only kernel.

> From discussions so far, it seems the preferred change to the userspace
> interface is what I’ll call the “Large File” method, as it follows the
> approach used for large file support:
> 
> Create new 64bit time_t/timespec/timeval/etc variants for syscalls, while
> preserving existing interfaces. This has some complexity around IOCTLs, but
> that can mostly be handled by creating new ioctl numbers while preserving
> the old ones. Since we’re only modifying time types, we’ll also need to add
> compat versions for many of these syscalls for 64bit native systems.

Josh Triplett already brought up the point of introducing completely new
interfaces for some cases and having libc emulate the old calls. A variation
of this would be to not add any 64-bit version of syscalls that are already
outdated. For example, we don't really need a 64-bit gettimeofday system
call, because we can easily implement it through clock_gettime.

I'd rather not introduce a 64-bit version of struct timeval if all syscalls
using it already have a replacement using timespec.

> The downsides here are many. The distros will probably hate this idea, as
> it requires rebuilding the world, and maintaining another legacy
> architecture support. I’m also not completely sure how robust multi-arch
> packaging is in the face of having to handle 3-4 architectures on one
> system. On the kernel side, it also adds more complexity, where we have to
> add even more complex compat support for 64bit systems to handle all the
> various 32bit applications possible.
>
> That said the practical reality isn't much further from the the “Libc
> Version Bump” approach, since legacy support will need legacy versions of
> all dependent libraries there a well. Similarly the additional storage
> required to support legacy applications is a positive motivator to get
> folks to move away from unsafe legacy applications. I also personally like
> the clarity the new virtual architecture brings, and that it would allow a
> kernel option to disable 2038-unsafe legacy support.

As mentioned above, I think the last point is not unique to this approach,
we can and should have that option in any case.

Between the "large file" approach and the "libc version bump" approach,
there doesn't have to be such a strict distinction. As you write, they
are the same from a kernel perspective, and I would expect most libc
implementations other than glibc (i.e. uclibc, bionic, newlib, musl, ...)
to only support the second approach for practical reasons.

In case of glibc, the preferred approach seems to be the "large file"
method, which means that any function passing a time value will have
at least two versions. However, you should be able to build a glibc
binary that only supports the most recent version of these interfaces,
effectively dropping support for the old ABI.

At a distro level, anybody can make the decision when to stop supporting
the old ABI independently: Gentoo can do this as an option as soon
as it works, while Debian can make a 10-year plan for a seamless
migration and Fedora can decide to dodge the problem by never even
enabling 64-bit time and just dropping support for 32-bit architectures
at some point before 2038, following what RHEL7 already did.

Most importantly, the embedded distros that most care about really long
term support (openembedded, buildroot, openwrt, ptxdist, ...) can just
drop 32-bit time_t entirely by switching the default to 64-bit time_t in
a new source release, independent of whether glibc gives them the
option for backwards compatibility or not.

	Arnd

Re: (Resend) 2038 Kernel Summit Discussion Fodder

[Pavel Machek]

Hi!

> There was also some pre-discussion before the sessions started around
> why we don't just change time_t to be unsigned. From the kernel's
> point of view this would be mostly fine, since dates before 1970 are
> not considered valid for any internal uses (with the exception of some
> filesystem timestamps). However, the problem with this approach is
> that userspace may want to handle dates prior to 1970, so this would
> eliminate that possibility. And removing the sign could cause problems
> with existing comparison logic. There is also the fact that having the
> same time unit range on 32bit and 64bit systems avoids complicating
> how timestamps are interpreted between architectures, where as having
> it be unsigned on 32bit but signed on 64bit would likely cause
> confusion. It is quite likely that using unsigned timestamps will be a
> useful solution for cases where timestamps cannot be converted to
> 64bits, but from the kernel perspective if we are going to change the
> abi, we should probably go all the way to 64bits. There seemed to be
> no disagreement here.
> 
> For the rest of the session, I opened it up for further thoughts or
> ideas. While there wasn't any new proposals, there was a question as
> to if anyone will really be running 32bit hardware in 2038, which made
> some folks point out that as systems get smaller there are likely to
> be tiny embedded platforms using Linux. The point that these systems

I'm pretty sure people will run 32bit hardware in 2038... that is not even
a question.

Today, there's good chance there's linux somewhere in your car. (Dashboard,
entertainment system). People like to keep cars from 1910 working, and I suspect
that is not going to change.

So yes, in 2038 people will be running 32bit linux.

Whether there will be people putting 32bit linux into new devices is a question,
but I suspect answer is still yes.
										Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: (Resend) 2038 Kernel Summit Discussion Fodder

[John Stultz]

On Tue, Aug 12, 2014 at 5:08 PM, John Stultz <john.stultz@linaro.org> wrote:
> So what I’d like to cover in this mail, are some discussion starters around
> ABI modifications that I think we should discuss at Kernel Summit, in order
> to make sure we have a clear path forward, as what we decide for the ABI
> modifications, has large impacts on how we help ensure userspace is correct
> or not.
>
>
> From discussions so far, it seems the preferred change to the userspace
> interface is what I’ll call the “Large File” method, as it follows the
> approach used for large file support:
>
> Create new 64bit time_t/timespec/timeval/etc variants for syscalls, while
> preserving existing interfaces. This has some complexity around IOCTLs, but
> that can mostly be handled by creating new ioctl numbers while preserving
> the old ones. Since we’re only modifying time types, we’ll also need to add
> compat versions for many of these syscalls for 64bit native systems.
>
> Libc then introduces versioned symbols, and a new compile options to allow
> applications to be built for “large time”. New and old applications could
> then share the same libc.
>
> The benefits of this approach is is simply and minimally extends the
> current 32 bit environment, without any effect on existing applications
> which continue to work. Most of the complexity is in the libc library and
> its build environment.
>
> The downsides to this approach is that as it follows the large-file
> approach, it has many of the same problems as large-file support, in that
> the transition to large-file has been slow and is still ongoing. Also,
> since this solution focuses on libc, there is also the problem of existing
> 3rd party libraries, which have no way of knowing which sized time is being
> used, will break. So all libraries that do anything with time will then
> have to implement their own versioned interfaces. This approach also makes
> it a little more difficult to audit that a system is 2038 safe, without
> running it and looking for issues.
>
>
> A potential alternative I’d like to also propose is the “Libc Version Bump”
> approach.
>
> Basically this is the same as the above, where the kernel provides both
> legacy and new time_t related interfaces. However, the libc would make a
> version break, migrating to using 64bit time_t types and syscalls. Legacy
> applications would still work using the old glibc version, but this would
> provide a stronger line in the sand between 2038 safe and unsafe
> applications and libraries, making it easier to avoid mixing the two.
> NetBSD developers discussed this same approach back in 2008 here:
> https://mail-index.netbsd.org/tech-userlevel/2008/03/22/msg000231.html
>
> The downsides here is, for legacy application support, one would have to
> have all the requisite legacy libraries also installed, which will add a
> burden to distro vendors. However, this extra storage overhead would likely
> be a positive motivator to get applications rebuilt and migrated to new
> version. Additionally, for 3rd party libraries built against the new libc
> version, the libraries may need to do a version bump themselves, in order
> to be able to co-exist with versions built against the previous libc. This
> approach also assumes that libraries that use time_t related values would
> have a libc dependency.
>
> A more aggressive version of the previous proposal is what I’m calling the
> “New Virtual-Architecture” approach, basically extending the versioning
> control from the linker down into the kernel as well.  It would be adding a
> new “virtual-architecture” to the kernel, not entirely unlike how x32 is
> supported on x86_64 systems.  We would create entirely new ABI and
> architecture name in the kernel (think something like “armllt” or
> “i386llt”). We would preserve compatibility for legacy applications via
> personalities, similar mechanism as the compat_ interface used to support
> 32bit applications on 64bit kernels. In this case, we wouldn’t introduce
> new 64 bit syscalls in the kernel, as the existing interfaces would just be
> typed correctly for our new virtual architecture, but we would have
> duplicate syscall interfaces via the compat interfaces. The extra
> complexity would also be that we would have to support new 32bit compat
> environment on 64bit systems. Userspace would be completely rebuilt to
> support the new -llt architecture, and compatibility for legacy
> applications would be done via the same multiarch packaging as is done now
> for running 32bit applications on 64bit systems.
>
> The pros for this case is that it would be very easy to audit that
> applications have migrated to the new 64bit time_t ABI. Additionally since
> we know which type the application is in the kernel, it would make
> problematic compatability areas like IOCTLS easier to deal with utilizing a
> flag in the task structure.
>
> The downsides here are many. The distros will probably hate this idea, as
> it requires rebuilding the world, and maintaining another legacy
> architecture support. I’m also not completely sure how robust multi-arch
> packaging is in the face of having to handle 3-4 architectures on one
> system. On the kernel side, it also adds more complexity, where we have to
> add even more complex compat support for 64bit systems to handle all the
> various 32bit applications possible.
>
> That said the practical reality isn't much further from the the “Libc
> Version Bump” approach, since legacy support will need legacy versions of
> all dependent libraries there a well. Similarly the additional storage
> required to support legacy applications is a positive motivator to get
> folks to move away from unsafe legacy applications. I also personally like
> the clarity the new virtual architecture brings, and that it would allow a
> kernel option to disable 2038-unsafe legacy support.
>
> With any of these approaches, we still have quite a bit of work to just get
> the kernel in-shape internally. And with the exception of the “virtual
> arch” approach, the changes on the kernel side are basically the same. The
> big thing we probably want to avoid is requiring any sort of flagday for
> distros, and instead allowing them each to migrate to the new solution
> individually (but hopefully not taking too long). Even so, I think having a
> clear vision for how userspace will make this transition is important, so
> hopefully during the Kernel Summit discussion we can come to consensus on
> what approach to take moving forward.

So I just wanted to send a (late) follow up here to summarize my
recollection of the discussion from the Kernel Summit workshop
session. Since there wasn't representation from the userspace world,
there was only a brief 30 minute session to try to get feedback on the
above proposals, and have a bit of open discussion to allow for folks
to air any ideas that hadn't already been proposed.

To start, I expresses what I felt to be the need for whatever solution
we choose, it should ideally be easy to audit in order for folks to be
able to validate that a system has or has not been updated to use
64bit time.  The counterpoint raised is that userspace may still do
problematic things like storing timestamps on disk or elsewhere in
32bit structures, so the value of any trivial auditing may not be that
great.

>From the discussion on the list, the "libc version bump" approach is
unlikely, and consensus in the session is that it would be a userspace
decision anyway. The "virtual architecture" approach, didn't seem to
have many fans either. Linus cut through the discussion with the
request that we not over-design this, just provide the 64bit
interfaces and let userspace do what it wants. So that resolved my
main open questions, and gave a clear way forward from the kernel's
perspective, with userland (or atleast glibc) likely taking the
"large-file style" approach.

There was also some pre-discussion before the sessions started around
why we don't just change time_t to be unsigned. From the kernel's
point of view this would be mostly fine, since dates before 1970 are
not considered valid for any internal uses (with the exception of some
filesystem timestamps). However, the problem with this approach is
that userspace may want to handle dates prior to 1970, so this would
eliminate that possibility. And removing the sign could cause problems
with existing comparison logic. There is also the fact that having the
same time unit range on 32bit and 64bit systems avoids complicating
how timestamps are interpreted between architectures, where as having
it be unsigned on 32bit but signed on 64bit would likely cause
confusion. It is quite likely that using unsigned timestamps will be a
useful solution for cases where timestamps cannot be converted to
64bits, but from the kernel perspective if we are going to change the
abi, we should probably go all the way to 64bits. There seemed to be
no disagreement here.

For the rest of the session, I opened it up for further thoughts or
ideas. While there wasn't any new proposals, there was a question as
to if anyone will really be running 32bit hardware in 2038, which made
some folks point out that as systems get smaller there are likely to
be tiny embedded platforms using Linux. The point that these systems
are likely never to be updated brought out a discussion as to if 2038
is even the least of our worries, given the security issues around
these abandoned systems. Debate of the risks of light-bulb hacks
ensued. This also raised an interesting but further tangential point
that apparently there is talk of legislative enforced obsolescence on
devices that don't get updates.

<sarcasm> So clearly the solution is to simply make it illegal to hit
any 2038 issue. Then there's nothing to worry about. :) </sarcasm>

So that's everything I recall. It was a fairly short session. I'm
hoping Thomas will be able to have a similar session with more
userspace representation at Plumbers, but given the attendance caps
were hit early, I'm suspecting will have to have continued discussions
on the list and wherever we can next year.

I'll be continuing to work (along with Arnd and others) on getting the
kernel internals migrated to 64bit time, and then eventually exporting
64bit syscalls for userspace to use. Further discussion, feedback and
help is of course welcome!

thanks
-john

Re: (Resend) 2038 Kernel Summit Discussion Fodder

[One Thousand Gnomes]

> Today, there's good chance there's linux somewhere in your car. (Dashboard,
> entertainment system). People like to keep cars from 1910 working, and I suspect
> that is not going to change.
> 
> So yes, in 2038 people will be running 32bit linux.
> 
> Whether there will be people putting 32bit linux into new devices is a question,
> but I suspect answer is still yes.

I'm currently trying to add 64bit longlong support to an initial PCC 8086
compiler port so I can fix that for some 16bit projects 8-)

My 8bit machines are mostly 2038 safe already.

Alan

Re: (Resend) 2038 Kernel Summit Discussion Fodder

[H. Peter Anvin]

On 09/08/2014 10:55 AM, One Thousand Gnomes wrote:
>> Today, there's good chance there's linux somewhere in your car. (Dashboard,
>> entertainment system). People like to keep cars from 1910 working, and I suspect
>> that is not going to change.
>>
>> So yes, in 2038 people will be running 32bit linux.
>>
>> Whether there will be people putting 32bit linux into new devices is a question,
>> but I suspect answer is still yes.
> 
> I'm currently trying to add 64bit longlong support to an initial PCC 8086
> compiler port so I can fix that for some 16bit projects 8-)
> 
> My 8bit machines are mostly 2038 safe already.
> 

Back in 1985 I decided to Y2K-test my ABC800 8-bit computer.  It passed
(they had implemented a windowed year solution which breaks in 2070 or
2080, as I recall.)

	-hpa