Re: [PATCH] Y2038: provide kernel support indication

Re: [PATCH] Y2038: provide kernel support indication

[Arnd Bergmann]

On Wed, Dec 5, 2018 at 12:49 PM Albert ARIBAUD <albert.aribaud@3adev.fr> wrote:
> On Tue, 25 Sep 2018 22:14:38 +0200, Arnd Bergmann <arnd@arndb.de> wrote :
> > On Tue, Sep 25, 2018 at 9:36 PM Albert ARIBAUD <albert.aribaud@3adev.fr> wrote:
> > I was undecided on this issue actually: it does make some sense to
> > me to have the syscall macros be the same in the long run, so that
> > glibc doesn't have to pick between __NR_clock_gettime and
> > __NR_clock_gettime64 in a few years, after all supported kernels
> > support __NR_clock_gettime64. Also, 64-bit architectures won't
> > reuse the number space that we reserve for the new calls on 32-bit
> > architectures, so the addition is essentially free.
> >
> > I don't have a strong preference here, and can do whichever
> > you like better in glibc.
>
> There does not seem to have been any other comment on this for a while,
> so, do we go for this approach? That is:
>
> - on 64-bit architectures, the kernel will define _NR symbols identical
>   to those added for Y2038, so that, for instance, on X86_64, there
>   will be an _NR_clock_gettime64 symbol (which will point to the same
>   syscall number as _NR_clock_gettime does).

There are a few new considerations for things that happened over the
last few months, so I'd still like to get more people involved before
we finally decide on this one:

- For cleaning up the kernel headers to make them more easily
  usable in glibc, I think it makes a lot of sense to define the new
  new macro names (as I said above). If we do that, maybe we should
  do it for other system calls (not time related) as well, in particular
  fcntl64, {f,}statfs64, {f,}truncate64, llseek, sendfile64, fstat{at,}64,
  mmap2, and fadvise64 as well as the associated structures.
  I already plan to do it independently for socketcall and ipc, which
  should be replaced with the separate entry points whenever we
  have the recent enough kernel.

- The work that Firoz Khan did for generating the asm/unistd.h
  kernel headers and syscall tables on all architectures makes it
  a little awkward to redirect new syscall macros to existing numbers,
  as the table file format uses the number as the primary key.
  The most logical way to do this in the kernel now is actually to
  have the newly assigned number point to the same syscall
  implementation on all architectures, both 32-bit and 64-bit.
  Not providing the number on 64-bit at all is fairly easy, but the
  redirect would require a permanent workaround on ppc, sparc,
  parisc, s390, and riscv as well as any future 64-bit architectures.

- Any Lutomirksi has some ideas for cleaning up the x86 syscall
  table, which may be relevant here. Adding him to Cc.

> - on 64-bit architectures, GLIBC will set __ASSUME_KERNEL_Y2038_SUPPORT
>   once the minimal kernel supported version for these architectures
>   provides the 'new' syscall symbols.
>
> - on all architectures, if __ASSUME_KERNEL_Y2038_SUPPORT is defined,
>   then GLIBC assumes the 'new' syscall symbols exist and can be used.

We have a couple of system calls that we add even on 64-bit architectures,
including statx() and likely the new version of waitid()/getrusage().
I think for those the logic has to be different, so a 64-bit architecture
will have to e.g. fall back to calling fstatat() when statx() is unavailable
on an older kernel.

> - on all architectures, if __ASSUME_KERNEL_Y2038_SUPPORT is NOT defined,
>   then GLIBC will try to use new syscalls for which a _NR symbol is
>   provided by the build-time kernel, and on ENOSYS, will fallback to old
>   syscalls.

I think it's important that we only do this when we are called from
an application with 64-bit time_t: When a user calls gettimeofday()
with 64-bit time_t, that should try clock_gettime64(2) and fall back
to clock_gettime(2) or gettimeofday(2) if that is unavailable. However
an application using gettimeofday() with a 32-bit time_t should
directly call gettimeofday(2) in the kernel but not try clock_gettime64()
first. Without that distinction, the kernel cannot return -ENOSYS
if we configure it to break compatibility with 32-bit time_t interfaces
and the application might seem to work correctly despite being
broken in 2038.

      Arnd

Re: [PATCH] Y2038: provide kernel support indication

[Joseph Myers]

On Wed, 5 Dec 2018, Arnd Bergmann wrote:

>   The most logical way to do this in the kernel now is actually to
>   have the newly assigned number point to the same syscall
>   implementation on all architectures, both 32-bit and 64-bit.

If you have (redundant) new numbers on 64-bit, glibc should not use those 
at all on 64-bit until the minimum kernel version is new enough to be sure 
they are available, to avoid unnecessary runtime fallback code.  Cf. how 
on socketcall architectures it made sense to keep using just socketcall 
for accept4 / recvmmsg / sendmmsg, rather than using the separate syscalls 
with runtime fallback to socketcall (in the cases where syscalls were 
added later than socketcall support), unless the configured minimum kernel 
version is recent enough to ensure the syscall is available.

(That does not prevent glibc having local #undef / #define to be able to 
use the new *names* unconditionally while still using the old numbers, if 
that proves useful.)

> I think it's important that we only do this when we are called from
> an application with 64-bit time_t: When a user calls gettimeofday()
> with 64-bit time_t, that should try clock_gettime64(2) and fall back
> to clock_gettime(2) or gettimeofday(2) if that is unavailable. However
> an application using gettimeofday() with a 32-bit time_t should
> directly call gettimeofday(2) in the kernel but not try clock_gettime64()
> first. Without that distinction, the kernel cannot return -ENOSYS
> if we configure it to break compatibility with 32-bit time_t interfaces
> and the application might seem to work correctly despite being
> broken in 2038.

By design, all nontrivial 32-bit time interfaces in glibc for 32-bit 
architectures should end up as simple wrappers round the implementations 
using 64-bit time internally, to avoid duplication of complicated code 
(previous versions of the Y2038 patches e.g. duplicated hundreds of lines 
of pthread_mutex_timedlock implementation, which is clearly not a 
maintainable approach).  That means many 32-bit interfaces *will* end up 
using 64-bit syscalls (with fallback to 32-bit if the 64-bit syscalls are 
unavailable at runtime), even if some simple functions that really are 
just one syscall for 32-bit still call the old syscall.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [PATCH] Y2038: provide kernel support indication

[Arnd Bergmann]

On Wed, Dec 5, 2018 at 2:48 PM Joseph Myers <joseph@codesourcery.com> wrote:
> On Wed, 5 Dec 2018, Arnd Bergmann wrote:
>
> > I think it's important that we only do this when we are called from
> > an application with 64-bit time_t: When a user calls gettimeofday()
> > with 64-bit time_t, that should try clock_gettime64(2) and fall back
> > to clock_gettime(2) or gettimeofday(2) if that is unavailable. However
> > an application using gettimeofday() with a 32-bit time_t should
> > directly call gettimeofday(2) in the kernel but not try clock_gettime64()
> > first. Without that distinction, the kernel cannot return -ENOSYS
> > if we configure it to break compatibility with 32-bit time_t interfaces
> > and the application might seem to work correctly despite being
> > broken in 2038.
>
> By design, all nontrivial 32-bit time interfaces in glibc for 32-bit
> architectures should end up as simple wrappers round the implementations
> using 64-bit time internally, to avoid duplication of complicated code
> (previous versions of the Y2038 patches e.g. duplicated hundreds of lines
> of pthread_mutex_timedlock implementation, which is clearly not a
> maintainable approach).  That means many 32-bit interfaces *will* end up
> using 64-bit syscalls (with fallback to 32-bit if the 64-bit syscalls are
> unavailable at runtime), even if some simple functions that really are
> just one syscall for 32-bit still call the old syscall.

That would make it very hard to deploy a glibc based system
with strict y2038 requirements. If we can no longer guarantee the
basic assumption that old applications keep using the old system
calls, then I think we instead need a way to build glibc itself without
support for the 32-bit time_t interfaces, e.g. using a --disable-time32
configuration switch to force a link-time error for any application
or library that has not been built against the 64-bit time_t interfaces.

The default would of course remain providing backwards compatibility
with all applications to allow a gradual migration from 32-bit time_t
to 64-bit time_t. If all 32-bit time_t calls are just wrappers around
either the internal implementation using 64-bit time_t or around
a simple system call, then it should at least be easy to make them
optional for users that need them not work.

Would that be something you can do?

      Arnd

Re: [PATCH] Y2038: provide kernel support indication

[Joseph Myers]

On Wed, 5 Dec 2018, Arnd Bergmann wrote:

> That would make it very hard to deploy a glibc based system
> with strict y2038 requirements. If we can no longer guarantee the
> basic assumption that old applications keep using the old system
> calls, then I think we instead need a way to build glibc itself without
> support for the 32-bit time_t interfaces, e.g. using a --disable-time32
> configuration switch to force a link-time error for any application
> or library that has not been built against the 64-bit time_t interfaces.

(There's never been a guarantee of keeping using the same system calls.  
Cf. people trying to restrict the system calls some code can use and 
running into issues with moves to use e.g. *at syscalls unconditionally on 
all architectures to implement the non-*at functions.)

The set of interfaces that involve 32-bit time_t or other structures 
including it is well-defined; an external checker can examine the dynamic 
symbol table of any dynamically linked 32-bit application or shared 
library to see if it has references to any of the symbols in question, 
without needing to run the code at all (and so detect issues in code that 
might not be frequently executed, not just in whatever code runs when 
you're testing for 32-bit time uses).  (Or if you control the build of the 
whole system, do a local change to make a glibc header use #error if 
_TIME_BITS is not defined to 64.)

I'd expect it to be a long time before we might consider making 32-bit 
time_t (and thus 32-bit off_t) subject to something like 
--enable-obsolete-rpc / --enable-obsolete-nsl being needed to make the 
32-bit interfaces available at (static) link time at all.

-- 
Joseph S. Myers
joseph@codesourcery.com