[Y2038] Question regarding support of old time interfaces beyond y2038

[Y2038] Question regarding support of old time interfaces beyond y2038

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 2372 bytes --]

Dear Arnd,

In your "playground" repository [1] (branch: y2038), the time functions
(stime, settimeofday, etc) are not converted in Linux to be Y2038 aware
(as for example clock_settime{64}() is).

I've also searched on the Internet and I've found some old discussions
regarding them:

SHA1:  d33c577cccd0b3e5bb2425f85037f26714a59363 [2]
From commit message:

"The time, stime, utime, utimes, and futimesat system calls are only
used on older architectures, and we do not provide y2038 safe variants
of them, as they are replaced by clock_gettime64, clock_settime64,
and utimensat_time64."

Moreover, the stime has been even explicitly marked as obsolete [3].


From other discussion [4] - regarding the following system calls:
 time, stime, gettimeofday, settimeofday, adjtimex, nanosleep, alarm,
 getitimer, setitimer, select, utime, utimes, futimesat, and
 {old,new}{l,f,}stat{,64}.

"These all pass 32-bit time_t arguments on 32-bit
 architectures and are replaced by other interfaces (e.g. posix
 timers and clocks, statx). C libraries implementing 64-bit time_t in
 32-bit architectures have to implement the handles by wrapping
 around the newer interfaces."




Has something changed since then? Has any new idea for conversion
emerged?



After observing the development of y2038 on playground [1], I can deduce
that new interfaces are only going to be supported and converted
(clock_settime64/clock_gettime64, etc.)

Considering the above - would it be best to drop Y2038 support on 32
bit machines for old syscalls (stime and friends) and for some others
(settimeofday/gettimeofday) write Y2038 wrappers based on new time
kernel API (clock_gettime/settime) in the C library (i.e. glibc)?




Note:

[1] -
https://git.kernel.org/pub/scm/linux/kernel/git/arnd/playground.git/tree/kernel/time/time.c?h=y2038
[2] - git://git.kernel.org/pub/scm/linux/kernel/git/arnd/playground.git
[3] - 
https://elixir.bootlin.com/linux/v2.6.32/source/arch/arm/include/asm/unistd.h#L419
[4] - https://lists.linaro.org/pipermail/y2038/2017-November/002387.html


Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Y2038] Question regarding support of old time interfaces beyond y2038

[Zack Weinberg]

On Tue, Mar 5, 2019 at 10:24 AM Lukasz Majewski <lukma@denx.de> wrote:
> From other discussion [4] - regarding the following system calls:
>  time, stime, gettimeofday, settimeofday, adjtimex, nanosleep, alarm,
>  getitimer, setitimer, select, utime, utimes, futimesat, and
>  {old,new}{l,f,}stat{,64}.
>
> "These all pass 32-bit time_t arguments on 32-bit
>  architectures and are replaced by other interfaces (e.g. posix
>  timers and clocks, statx). C libraries implementing 64-bit time_t in
>  32-bit architectures have to implement the handles by wrapping
>  around the newer interfaces."

1) We should be clear that most of these will continue to be supported
as C library interfaces even if they are not system calls.  Some of
them are obsolete enough and/or rarely used enough that we might not
bother (the older ways to set the system clock, for instance).

2) I know of one case where the new interfaces don't cover all of the
functionality of the old ones: timers started by setitimer continue to
run after an execve, timers started by timer_create don't.  This means
setitimer(ITIMER_VIRTUAL) can be used to impose a CPU time limit on a
program you didn't write, and timer_create can't.  If new kernels are
not going to have setitimer as a primitive, we need some other way of
getting the same effect.

zw

Re: [Y2038] Question regarding support of old time interfaces beyond y2038

[Ben Hutchings]

On Tue, 2019-03-05 at 11:05 -0500, Zack Weinberg wrote:
> > On Tue, Mar 5, 2019 at 10:24 AM Lukasz Majewski <lukma@denx.de> wrote:
> > From other discussion [4] - regarding the following system calls:
> >  time, stime, gettimeofday, settimeofday, adjtimex, nanosleep, alarm,
> >  getitimer, setitimer, select, utime, utimes, futimesat, and
> >  {old,new}{l,f,}stat{,64}.
> > 
> > "These all pass 32-bit time_t arguments on 32-bit
> >  architectures and are replaced by other interfaces (e.g. posix
> >  timers and clocks, statx). C libraries implementing 64-bit time_t in
> >  32-bit architectures have to implement the handles by wrapping
> >  around the newer interfaces."
> 
> 1) We should be clear that most of these will continue to be supported
> as C library interfaces even if they are not system calls.  Some of
> them are obsolete enough and/or rarely used enough that we might not
> bother (the older ways to set the system clock, for instance).
> 
> 2) I know of one case where the new interfaces don't cover all of the
> functionality of the old ones: timers started by setitimer continue to
> run after an execve, timers started by timer_create don't.  This means
> setitimer(ITIMER_VIRTUAL) can be used to impose a CPU time limit on a
> program you didn't write, and timer_create can't.  If new kernels are
> not going to have setitimer as a primitive, we need some other way of
> getting the same effect.

{get,set}itimer() are still implemented on all architectures, and I
don't see any sign that that's going to change.  There aren't 64-bit
versions on 32-bit architectures though.  This is explained in the
message for commit 48166e6ea47d23984f0b481ca199250e1ce0730a:

"...these can all be safely implemented in the C library by wrapping
around the existing system calls because the 32-bit time_t they pass
only counts elapsed time, not time since the epoch."

Ben.

-- 
Ben Hutchings, Software Developer                         Codethink Ltd
https://www.codethink.co.uk/                 Dale House, 35 Dale Street
                                     Manchester, M1 2HF, United Kingdom

Re: [Y2038] Question regarding support of old time interfaces beyond y2038

[Arnd Bergmann]

On Tue, Mar 5, 2019 at 4:24 PM Lukasz Majewski <lukma@denx.de> wrote:
>
> Dear Arnd,
>
> In your "playground" repository [1] (branch: y2038), the time functions
> (stime, settimeofday, etc) are not converted in Linux to be Y2038 aware
> (as for example clock_settime{64}() is).

Correct. FWIW, this is now merged into the mainline kernel.

> I've also searched on the Internet and I've found some old discussions
> regarding them:
>
> SHA1:  d33c577cccd0b3e5bb2425f85037f26714a59363 [2]
> From commit message:
>
> "The time, stime, utime, utimes, and futimesat system calls are only
> used on older architectures, and we do not provide y2038 safe variants
> of them, as they are replaced by clock_gettime64, clock_settime64,
> and utimensat_time64."
>
> Moreover, the stime has been even explicitly marked as obsolete [3].
>
>
> From other discussion [4] - regarding the following system calls:
>  time, stime, gettimeofday, settimeofday, adjtimex, nanosleep, alarm,
>  getitimer, setitimer, select, utime, utimes, futimesat, and
>  {old,new}{l,f,}stat{,64}.
>
> "These all pass 32-bit time_t arguments on 32-bit
>  architectures and are replaced by other interfaces (e.g. posix
>  timers and clocks, statx). C libraries implementing 64-bit time_t in
>  32-bit architectures have to implement the handles by wrapping
>  around the newer interfaces."
>
>
>
>
> Has something changed since then? Has any new idea for conversion
> emerged?

No, this has been the plan for many years now.

> After observing the development of y2038 on playground [1], I can deduce
> that new interfaces are only going to be supported and converted
> (clock_settime64/clock_gettime64, etc.)
>
> Considering the above - would it be best to drop Y2038 support on 32
> bit machines for old syscalls (stime and friends) and for some others
> (settimeofday/gettimeofday) write Y2038 wrappers based on new time
> kernel API (clock_gettime/settime) in the C library (i.e. glibc)?

There are multiple dimensions to what you are asking here:

- On the user space interface, the C library (glibc, musl, uclibc, ...)
  implements a set of interfaces for time management. The
  set that is implemented here is defined by POSIX and other
  standards and decided by the respective C library implementation.
  All functions that get implemented here have to use the same
  definition of time_t however, so if there is both a clock_gettime()
  function and a time() function, they must either both use 32-bit
  time_t or both must use 64-bit time_t. Both can be implemented
  on top of any kernel interface for getting the time (time, gettimeofday,
  clock_gettime, clock_gettime64), but the only sensible implementation
  is to use clock_gettime64() in order to have the full range and
  resolution.

- The kernel has a growing set of system calls, i.e. we tend to
  only add new ones but not take old ones away. In many cases,
  a new syscall is a superset of the old one (e.g. oldstat, newstat
  stat64, xstat), any architecture that had an old version has
  to keep it around, but new architectures only ever provide the
  most recent variant.

     Arnd

Re: [Y2038] Question regarding support of old time interfaces beyond y2038

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 4960 bytes --]

Hi Arnd,

> On Tue, Mar 5, 2019 at 4:24 PM Lukasz Majewski <lukma@denx.de> wrote:
> >
> > Dear Arnd,
> >
> > In your "playground" repository [1] (branch: y2038), the time
> > functions (stime, settimeofday, etc) are not converted in Linux to
> > be Y2038 aware (as for example clock_settime{64}() is).  
> 
> Correct. FWIW, this is now merged into the mainline kernel.
> 

The arch/arm/tools/syscall.tbl
linux-next 20190306
SHA1: cf08baa29613dd899954089e7cc7dba1d478b365

Has now:

403	common	clock_gettime64			sys_clock_gettime
404	common	clock_settime64			sys_clock_settime
405	common	clock_adjtime64			sys_clock_adjtime
406	common	clock_getres_time64		sys_clock_getres
407	common	clock_nanosleep_time64		sys_clock_nanosleep
408	common	timer_gettime64			sys_timer_gettime
409	common	timer_settime64			sys_timer_settime
410	common	timerfd_gettime64		sys_timerfd_gettime
411	common	timerfd_settime64		sys_timerfd_settime
412	common	utimensat_time64		sys_utimensat
413	common	pselect6_time64			sys_pselect6
414	common	ppoll_time64			sys_ppoll
416	common	io_pgetevents_time64		sys_io_pgetevents
417	common	recvmmsg_time64			sys_recvmmsg
418	common	mq_timedsend_time64		sys_mq_timedsend
419	common	mq_timedreceive_time64		sys_mq_timedreceive
420	common	semtimedop_time64		sys_semtimedop
421	common	rt_sigtimedwait_time64		sys_rt_sigtimedwait
422	common	futex_time64			sys_futex
423	common	sched_rr_get_interval_time64
sys_sched_rr_get_interval

> > I've also searched on the Internet and I've found some old
> > discussions regarding them:
> >
> > SHA1:  d33c577cccd0b3e5bb2425f85037f26714a59363 [2]
> > From commit message:
> >
> > "The time, stime, utime, utimes, and futimesat system calls are only
> > used on older architectures, and we do not provide y2038 safe
> > variants of them, as they are replaced by clock_gettime64,
> > clock_settime64, and utimensat_time64."
> >
> > Moreover, the stime has been even explicitly marked as obsolete [3].
> >
> >
> > From other discussion [4] - regarding the following system calls:
> >  time, stime, gettimeofday, settimeofday, adjtimex, nanosleep,
> > alarm, getitimer, setitimer, select, utime, utimes, futimesat, and
> >  {old,new}{l,f,}stat{,64}.
> >
> > "These all pass 32-bit time_t arguments on 32-bit
> >  architectures and are replaced by other interfaces (e.g. posix
> >  timers and clocks, statx). C libraries implementing 64-bit time_t
> > in 32-bit architectures have to implement the handles by wrapping
> >  around the newer interfaces."
> >
> >
> >
> >
> > Has something changed since then? Has any new idea for conversion
> > emerged?  
> 
> No, this has been the plan for many years now.

Ok.

> 
> > After observing the development of y2038 on playground [1], I can
> > deduce that new interfaces are only going to be supported and
> > converted (clock_settime64/clock_gettime64, etc.)
> >
> > Considering the above - would it be best to drop Y2038 support on 32
> > bit machines for old syscalls (stime and friends) and for some
> > others (settimeofday/gettimeofday) write Y2038 wrappers based on
> > new time kernel API (clock_gettime/settime) in the C library (i.e.
> > glibc)?  
> 
> There are multiple dimensions to what you are asking here:
> 
> - On the user space interface, the C library (glibc, musl,
> uclibc, ...) implements a set of interfaces for time management. The
>   set that is implemented here is defined by POSIX and other
>   standards and decided by the respective C library implementation.
>   All functions that get implemented here have to use the same
>   definition of time_t however, so if there is both a clock_gettime()
>   function and a time() function, they must either both use 32-bit
>   time_t or both must use 64-bit time_t. Both can be implemented
>   on top of any kernel interface for getting the time (time,
> gettimeofday, clock_gettime, clock_gettime64), but the only sensible
> implementation is to use clock_gettime64() in order to have the full
> range and resolution.

I see. I just would like to be sure that the "old" API calls (for
example the settimeofday, gettimeofday, stime, etc) are not recommended
(planned?) for conversion to have 64 bit interfaces (like
clock_{get|set}time).

> 
> - The kernel has a growing set of system calls, i.e. we tend to
>   only add new ones but not take old ones away. In many cases,
>   a new syscall is a superset of the old one (e.g. oldstat, newstat
>   stat64, xstat), any architecture that had an old version has
>   to keep it around, but new architectures only ever provide the
>   most recent variant.
> 
>      Arnd




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Y2038] Question regarding support of old time interfaces beyond y2038

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 1838 bytes --]

Hi Zack,

> On Tue, Mar 5, 2019 at 10:24 AM Lukasz Majewski <lukma@denx.de> wrote:
> > From other discussion [4] - regarding the following system calls:
> >  time, stime, gettimeofday, settimeofday, adjtimex, nanosleep,
> > alarm, getitimer, setitimer, select, utime, utimes, futimesat, and
> >  {old,new}{l,f,}stat{,64}.
> >
> > "These all pass 32-bit time_t arguments on 32-bit
> >  architectures and are replaced by other interfaces (e.g. posix
> >  timers and clocks, statx). C libraries implementing 64-bit time_t
> > in 32-bit architectures have to implement the handles by wrapping
> >  around the newer interfaces."  
> 
> 1) We should be clear that most of these will continue to be supported
> as C library interfaces even if they are not system calls.  Some of
> them are obsolete enough and/or rarely used enough that we might not
> bother (the older ways to set the system clock, for instance).

The question here is about the decision if even the old time APIs shall
be supported on 32 bit systems which are going to be Y2038 proof (like
the 'stime').

> 
> 2) I know of one case where the new interfaces don't cover all of the
> functionality of the old ones: timers started by setitimer continue to
> run after an execve, timers started by timer_create don't.  This means
> setitimer(ITIMER_VIRTUAL) can be used to impose a CPU time limit on a
> program you didn't write, and timer_create can't.  If new kernels are
> not going to have setitimer as a primitive, we need some other way of
> getting the same effect.
> 
> zw




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Y2038] Question regarding support of old time interfaces beyond y2038

[Arnd Bergmann]

On Thu, Mar 7, 2019 at 8:53 AM Lukasz Majewski <lukma@denx.de> wrote:
>
> Hi Zack,
>
> > On Tue, Mar 5, 2019 at 10:24 AM Lukasz Majewski <lukma@denx.de> wrote:
> > > From other discussion [4] - regarding the following system calls:
> > >  time, stime, gettimeofday, settimeofday, adjtimex, nanosleep,
> > > alarm, getitimer, setitimer, select, utime, utimes, futimesat, and
> > >  {old,new}{l,f,}stat{,64}.
> > >
> > > "These all pass 32-bit time_t arguments on 32-bit
> > >  architectures and are replaced by other interfaces (e.g. posix
> > >  timers and clocks, statx). C libraries implementing 64-bit time_t
> > > in 32-bit architectures have to implement the handles by wrapping
> > >  around the newer interfaces."
> >
> > 1) We should be clear that most of these will continue to be supported
> > as C library interfaces even if they are not system calls.  Some of
> > them are obsolete enough and/or rarely used enough that we might not
> > bother (the older ways to set the system clock, for instance).
>
> The question here is about the decision if even the old time APIs shall
> be supported on 32 bit systems which are going to be Y2038 proof (like
> the 'stime').

See my other reply. In the kernel, it won't be supported (the old syscall
is of course still there, but we may have an option to remove all time32
interfaces). In glibc, it's probably there in a y2038-safe way since it is
there now, other C libraries may take other decisions that are independent
of y2038.

      Arnd

Re: [Y2038] Question regarding support of old time interfaces beyond y2038

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 2956 bytes --]

Hi Arnd,

> On Thu, Mar 7, 2019 at 8:53 AM Lukasz Majewski <lukma@denx.de> wrote:
> >
> > Hi Zack,
> >  
> > > On Tue, Mar 5, 2019 at 10:24 AM Lukasz Majewski <lukma@denx.de>
> > > wrote:  
> > > > From other discussion [4] - regarding the following system
> > > > calls: time, stime, gettimeofday, settimeofday, adjtimex,
> > > > nanosleep, alarm, getitimer, setitimer, select, utime, utimes,
> > > > futimesat, and {old,new}{l,f,}stat{,64}.
> > > >
> > > > "These all pass 32-bit time_t arguments on 32-bit
> > > >  architectures and are replaced by other interfaces (e.g. posix
> > > >  timers and clocks, statx). C libraries implementing 64-bit
> > > > time_t in 32-bit architectures have to implement the handles by
> > > > wrapping around the newer interfaces."  
> > >
> > > 1) We should be clear that most of these will continue to be
> > > supported as C library interfaces even if they are not system
> > > calls.  Some of them are obsolete enough and/or rarely used
> > > enough that we might not bother (the older ways to set the system
> > > clock, for instance).  
> >
> > The question here is about the decision if even the old time APIs
> > shall be supported on 32 bit systems which are going to be Y2038
> > proof (like the 'stime').  
> 
> See my other reply. In the kernel, it won't be supported (the old
> syscall is of course still there, but we may have an option to remove
> all time32 interfaces). 

To be more specific:

I'm thinking of settimeofday/gettimeofday syscalls.

In the kernel we use internally do_sys_settimeofday64() to support
clock_settime() and settimeofday()

The internal (in-kernel) representation for those two is struct
timespec64.

If I may ask - why settimeofday64() and gettimeofday64() are not
implemented?

Is it because the same result can be achieved with clock_settime64(tv64)
+ settimeofday(NULL, tz) ?
(The drawback is two syscalls instead of one).




I've also stumbled upon the __kernel_timex introduction on the
playground branch:

"time: Add struct __kernel_timex"
2c620ff93d9fbd5d644760d4c21d389078ec1080

This one introduces the:
struct __kernel_timex_timeval {
	__kernel_time64_t tv_sec;
	long long tv_usec;
};

This code is "protected" by CONFIG_64BIT_TIME. 

Is there any plan to explicitly introduce:

struct __kernel_timeval {
       __kernel_time64_t tv_sec;
       long ong tv_usec;
}

and convert settimeofday()/gettimeofday() ?


Thanks in advance for your help.

> In glibc, it's probably there in a y2038-safe
> way since it is there now, other C libraries may take other decisions
> that are independent of y2038.
> 
>       Arnd




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Y2038] Question regarding support of old time interfaces beyond y2038

[Arnd Bergmann]

On Thu, Mar 7, 2019 at 3:43 PM Lukasz Majewski <lukma@denx.de> wrote:
> > On Thu, Mar 7, 2019 at 8:53 AM Lukasz Majewski <lukma@denx.de> wrote:
> > > > On Tue, Mar 5, 2019 at 10:24 AM Lukasz Majewski <lukma@denx.de>

> To be more specific:
>
> I'm thinking of settimeofday/gettimeofday syscalls.
>
> In the kernel we use internally do_sys_settimeofday64() to support
> clock_settime() and settimeofday()
>
> The internal (in-kernel) representation for those two is struct
> timespec64.
>
> If I may ask - why settimeofday64() and gettimeofday64() are not
> implemented?
>
> Is it because the same result can be achieved with clock_settime64(tv64)
> + settimeofday(NULL, tz) ?
> (The drawback is two syscalls instead of one).

Yes, that is the idea. I don't see the drawback as significant here,
since settimeofday() is not performance critical.

> I've also stumbled upon the __kernel_timex introduction on the
> playground branch:
>
> "time: Add struct __kernel_timex"
> 2c620ff93d9fbd5d644760d4c21d389078ec1080
>
> This one introduces the:
> struct __kernel_timex_timeval {
>         __kernel_time64_t tv_sec;
>         long long tv_usec;
> };
>
> This code is "protected" by CONFIG_64BIT_TIME.
>
> Is there any plan to explicitly introduce:
>
> struct __kernel_timeval {
>        __kernel_time64_t tv_sec;
>        long ong tv_usec;
> }
>
> and convert settimeofday()/gettimeofday() ?

No, see above. Basically all system calls that take a 'timeval'
already have a replacement that uses a 'timespec' with
nanosecond resolution, so the idea was that by not having
a new timeval, we make sure to catch any calls that need a
nanosecond based version as well.

clock_adjtime() is a bit of a special case here because
it uses a timeval structure but passes nanoseconds in it
when ADJ_NANO is set.

      Arnd

Re: [Y2038] Question regarding support of old time interfaces beyond y2038

[Joseph Myers]

On Thu, 7 Mar 2019, Lukasz Majewski wrote:

> > 1) We should be clear that most of these will continue to be supported
> > as C library interfaces even if they are not system calls.  Some of
> > them are obsolete enough and/or rarely used enough that we might not
> > bother (the older ways to set the system clock, for instance).
> 
> The question here is about the decision if even the old time APIs shall
> be supported on 32 bit systems which are going to be Y2038 proof (like
> the 'stime').

The glibc API should support the same set of functions both with and 
without _TIME_BITS=64.

I think it would be reasonable to obsolete the stime function in glibc 
(meaning turn it into a compat symbol, not available for linking new 
programs and not present at all for new architectures).  But that's 
orthogonal to supporting 64-bit times on 32-bit platforms in glibc.  If 
stime is obsoleted before (or in the same release as) that 64-bit time 
support, no 64-bit version of stime is needed in glibc.  If obsoleted in a 
later release, glibc would need to get a 64-bit version (and both versions 
would turn into compat symbols if the interface is obsoleted).

-- 
Joseph S. Myers
joseph@codesourcery.com