Y2038, glibc and utmp/utmpx on 64bit architectures

Y2038, glibc and utmp/utmpx on 64bit architectures

[Thorsten Kukuk]

Hi,

I hope everybody is aware of the Y2038 problem. And not only for 32bit
architectures, but also 64bit architectures are not ready today, at
least not if they use glibc.
glibc uses for compatibility with 32bit userland applications 32bit
values for time_t and other variables even on 64bit systems.
Affected is everything around utmp/utmpx, wtmp/wtmpx and lastlog.

I wrote a blog about how to solve that for utmp/utmpx by using the data
from systemd-logind instead:
https://www.thkukuk.de/blog/Y2038_glibc_utmp_64bit/

A detailed analysis also for wtmp and lastlog, which have the same
problems, can be found here:
https://github.com/thkukuk/utmpx/blob/main/Y2038.md


   Thorsten

-- 
Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies
SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany
Managing Director: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman
(HRB 36809, AG Nürnberg)

Re: Y2038, glibc and utmp/utmpx on 64bit architectures

[A. Wilcox]

On Mar 3, 2023, at 4:46 AM, Thorsten Kukuk <kukuk@suse.com> wrote:
> Hi,
> 
> I hope everybody is aware of the Y2038 problem. And not only for 32bit
> architectures, but also 64bit architectures are not ready today, at
> least not if they use glibc.
> glibc uses for compatibility with 32bit userland applications 32bit
> values for time_t and other variables even on 64bit systems.
> Affected is everything around utmp/utmpx, wtmp/wtmpx and lastlog.
> 
> I wrote a blog about how to solve that for utmp/utmpx by using the data
> from systemd-logind instead:
> https://www.thkukuk.de/blog/Y2038_glibc_utmp_64bit/
> 
> A detailed analysis also for wtmp and lastlog, which have the same
> problems, can be found here:
> https://github.com/thkukuk/utmpx/blob/main/Y2038.md
> 
> 
>   Thorsten
> 
> -- 
> Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies
> SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany
> Managing Director: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman
> (HRB 36809, AG Nürnberg)

Hi Thorsten,

Please don’t require systemd for utmpx features.  It is not exactly
accurate that musl does not support utmp.  The musl view is that utmp
*in the libc* is insecure, but can be implemented securely using an
external process.  That process is the utmps package that you found.  It
does not require s6, other than skalibs (which is not a very heavy
dependency at all).

What if I also told you that systemd itself is a replacement for
`s6-ipcserver`? :)  All you need is a socket unit with Accept=Yes.  I
could wire up an example Fedora container that you could play around
with, using utmps and a custom built coreutils/util-linux against it,
over the weekend if it would help sway anyone’s mind.

I really don’t think it is appropriate to outright remove POSIX standard
interfaces from Linux, replacing them with non-standard systemd APIs.
The number of packages that use utmpx are numerous and far beyond what
you probably realise:

* tcsh uses it for its custom lastlog primitive.

* AccountsService uses wtmp.

* lynx uses it.

* net-snap uses it for exposing utmp information over MIBs.

* Xterm, urxvt, etc use it to update information on logged in shells.

* Python psutil package uses it to display status information.

* SDDM display manager updates wtmp/utmp for logged in sessions.

* lsof tool.

* libutempter, which is used by tmux and Konsole to update utmp.

* X11VNC, OpenSSH, procps, sysklogd, sudo…

These are just the packages we have in Adélie, and we are a small
distribution, which is also built on musl libc and has full utmpx
support.

It is much better to provide a secure way for the standard POSIX utmpx
header, than try to replace it and add all those conditionals to all
those packages.  Remember that in addition to distros that don’t have
systemd, there are other systems (BSD, Illumos, even Mac OS for some of
those) that will never have logind APIs, so you are asking all those
packages to special-case glibc Linux…

Best,
-A.

--
A. Wilcox (they/them)
SW Engineering: C/C++, DevOps, POSIX
Wilcox Technologies Inc.

Re: Y2038, glibc and utmp/utmpx on 64bit architectures

[Thorsten Kukuk]

Hi,

On Fri, Mar 03, A. Wilcox wrote:

> On Mar 3, 2023, at 4:46 AM, Thorsten Kukuk <kukuk@suse.com> wrote:
> > Hi,
> > 
> > I hope everybody is aware of the Y2038 problem. And not only for 32bit
> > architectures, but also 64bit architectures are not ready today, at
> > least not if they use glibc.
> > glibc uses for compatibility with 32bit userland applications 32bit
> > values for time_t and other variables even on 64bit systems.
> > Affected is everything around utmp/utmpx, wtmp/wtmpx and lastlog.
> > 
> > I wrote a blog about how to solve that for utmp/utmpx by using the data
> > from systemd-logind instead:
> > https://www.thkukuk.de/blog/Y2038_glibc_utmp_64bit/
> > 
> > A detailed analysis also for wtmp and lastlog, which have the same
> > problems, can be found here:
> > https://github.com/thkukuk/utmpx/blob/main/Y2038.md
> > 
> > 
> >   Thorsten
> > 
> > -- 
> > Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies
> > SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany
> > Managing Director: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman
> > (HRB 36809, AG Nürnberg)
> 
> Hi Thorsten,
> 
> Please don’t require systemd for utmpx features.  It is not exactly
> accurate that musl does not support utmp.  The musl view is that utmp
> *in the libc* is insecure, but can be implemented securely using an
> external process.  That process is the utmps package that you found.  It
> does not require s6, other than skalibs (which is not a very heavy
> dependency at all).

Exactly what you write is what I wrote. musl libc itself does not
support utmp, but there is utmps.
So I don't understand your complain.

> What if I also told you that systemd itself is a replacement for
> `s6-ipcserver`? :)  All you need is a socket unit with Accept=Yes.  I
> could wire up an example Fedora container that you could play around
> with, using utmps and a custom built coreutils/util-linux against it,
> over the weekend if it would help sway anyone’s mind.

Not really, as I wrote in my blog. utmps does not solve all problems
with utmp/utmps and it's just another daemon doing the same things as
systemd-logind is already doing.

> I really don’t think it is appropriate to outright remove POSIX standard
> interfaces from Linux, replacing them with non-standard systemd APIs.

Nobody wants to remove the current utmp code (ok, not quite correct,
glibc developers plan to remove it from glibc, as it will stop working
in a few years) and feel free to convert all the code out there to use
the utmpx and not utmp interface.

> The number of packages that use utmpx are numerous and far beyond what
> you probably realise:

No, I know how many tools are using it and for what, look at the
document I did reference in my blog.

> * tcsh uses it for its custom lastlog primitive.

It's on the list.

> * AccountsService uses wtmp.

It's on the list, it just looks when a user did logout.
Your mistake: utmp is not wtmp.

> * net-snap uses it for exposing utmp information over MIBs.

Do you mean net-snmp? It's only using utmp to count the number of logged
in users. And as I explained in my blog, with utmp this number is very
likely even wrong.

> * Xterm, urxvt, etc use it to update information on logged in shells.

They use it to create fake logged in users for wall. And since Desktop
users hate to be spamed with wall messages on all ttys, tools try to be
clever and ignore such fake entries.
And if you use GNOME, tmux or anything else: they don't create fake
entries.

> * Python psutil package uses it to display status information.

It's on my list

> * SDDM display manager updates wtmp/utmp for logged in sessions.

It's all on the list.

> * lsof tool.

Not the version I have.

> * libutempter, which is used by tmux and Konsole to update utmp.

It's on the list.

> * X11VNC, OpenSSH, procps, sysklogd, sudo…

It's all on the list.

Next time please read everything, saves you a lot of time ;)

  Thorsten

-- 
Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies
SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany
Managing Director: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman
(HRB 36809, AG Nürnberg)

Re: Y2038, glibc and utmp/utmpx on 64bit architectures

[Anna]

[-- Attachment #1: Type: text/plain, Size: 1227 bytes --]

> Not really, as I wrote in my blog. utmps does not solve all problems
> with utmp/utmps and it's just another daemon doing the same things as
> systemd-logind is already doing.

In the blog post, it was included the following:
> Proposal
>    1. Change all applications, which read utmp, to query systemd-logind instead
>    2. Stop writing utmp entries after we are sure nobody uses them anymore

> [...]

> What if I don’t use systemd?

> This heavily depends on the libc and init system you are using. For s6 exists e.g.
> a secure utmp (utmps) implementation, while on the other side musl libc has no
> support for utmp/wtmp at all.

My issue with this is basically leaving everyone that doesn't use
systemd to fend for themselves. We patch those apps to use logind,
remove utmp and then what does non-systemd distros do?

While having two daemons (umtps and logind) at the same time is not that
great of a solution either, it would be better to have an interface that
can work with different backends, so that in a systemd system, you'd
just use logind, but in a non-systemd system, you'd use a standalone
daemon, such as utmps, both using options using the same API on the
application side.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: Y2038, glibc and utmp/utmpx on 64bit architectures

[Bruno Haible]

Thorsten Kukuk wrote:
> > I really don’t think it is appropriate to outright remove POSIX standard
> > interfaces from Linux, replacing them with non-standard systemd APIs.
> 
> Nobody wants to remove the current utmp code (ok, not quite correct,
> glibc developers plan to remove it from glibc, as it will stop working
> in a few years) and feel free to convert all the code out there to use
> the utmpx and not utmp interface.

I don't understand: Do the glibc developers plan to remove the utmpx
interface as well (together with utmp interface)?

If no, then
  - Why does your blog post https://github.com/thkukuk/utmpx/blob/main/Y2038.md
    say "It looks like the glibc developers don't want to solve this problem
    but instead deprecate the utmp.h/utmpx.h/lastlog.h interface."?
  - What about the '#if __WORDSIZE_TIME64_COMPAT32' in
    /usr/include/x86_64-linux-gnu/bits/utmpx.h ? Isn't it as nasty as the
    '#if __WORDSIZE_TIME64_COMPAT32' in
    /usr/include/x86_64-linux-gnu/bits/utmp.h ?

If yes, then
  - What is the point of your suggestion to "use the utmpx and not utmp
    interface", above?
  - Since there is no "FUTURE DIRECTIONS" in POSIX
    https://pubs.opengroup.org/onlinepubs/9699919799.2018edition/functions/endutxent.html
    will the utmpx interface get deprecated in POSIX, or stay as it is?
    Is the Austin Group already involved?

Bruno

Re: Y2038, glibc and utmp/utmpx on 64bit architectures

[Thorsten Kukuk]

On Fri, Mar 03, Bruno Haible wrote:

> Thorsten Kukuk wrote:
> > > I really don’t think it is appropriate to outright remove POSIX standard
> > > interfaces from Linux, replacing them with non-standard systemd APIs.
> > 
> > Nobody wants to remove the current utmp code (ok, not quite correct,
> > glibc developers plan to remove it from glibc, as it will stop working
> > in a few years) and feel free to convert all the code out there to use
> > the utmpx and not utmp interface.
> 
> I don't understand: Do the glibc developers plan to remove the utmpx
> interface as well (together with utmp interface)?

In glibc, utmpx is just an alias for utmp.

> If yes, then
>   - What is the point of your suggestion to "use the utmpx and not utmp
>     interface", above?

utmps only supports the utmpx interface, not utmp. So if you want to use
utmps, you need to convert all source code to utmpx.
I did not recommend to use utmpx as this is with glibc identical with
utmp with exact the same problems.

>   - Since there is no "FUTURE DIRECTIONS" in POSIX
>     https://pubs.opengroup.org/onlinepubs/9699919799.2018edition/functions/endutxent.html
>     will the utmpx interface get deprecated in POSIX, or stay as it is?
>     Is the Austin Group already involved?

Linux is not POSIX conform, never was and there was never the plan to
become, but it tries to be as compatible with POSIX as possible where
it makes sense. If it does not make sense, POSIX will not be used. You
should be able to find several examples in glibc, where interfaces
derivate slightly from POSIX because the POSIX interface didn't made any
sense.
Parts of POSIX are really old (don't know how old utmp/utmpx are, but
they did exist already since a long time before I started to work
with Unix, and that's really a long time ago) and things are changing.
Today, utmp/utmpx create more problems then it solve. Many features of
it where never used on Linux or are meanwhile no longer used, at least
not with systemd.
There is just no benefit from it anymore. But on the other side, all the
problems with utmp/utmpx stay. Like the need for applications with
special rights to write to that file, which is security wise a
nightmare. Especially if that is an X11 application.
Or the format, a string can be nul terminated, or be as long as the
field. It's a nightmare to parse such entries in a secure way and often done
wrong.
Or the limitation of the length of the username, which creates regular
bug reports by users, which don't understand this limitation. Which is
only there because of utmp, the full rest of the system does not have
such a limitation of the username.

If somebody really needs a POSIX conform utmpx implementation for Linux:
I'm pretty sure that it is no problem to create a library where the
functions fetches the data from logind and not a file, like utmps is
doing. But he need to be aware: such an interface will introduce all the
useless limitations and problems of utmp/utmpx again.

  Thorsten

-- 
Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies
SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany
Managing Director: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman
(HRB 36809, AG Nürnberg)

Re: Y2038, glibc and utmp/utmpx on 64bit architectures

[Bruno Haible]

Thorsten Kukuk wrote:
> > Do the glibc developers plan to remove the utmpx
> > interface as well (together with utmp interface)?
> 
> In glibc, utmpx is just an alias for utmp.
> 
> > If yes, then
> >   - What is the point of your suggestion to "use the utmpx and not utmp
> >     interface", above?
> 
> utmps only supports the utmpx interface, not utmp. So if you want to use
> utmps, you need to convert all source code to utmpx.

Thanks for the explanations. It's clear now.

> >   - Since there is no "FUTURE DIRECTIONS" in POSIX
> >     https://pubs.opengroup.org/onlinepubs/9699919799.2018edition/functions/endutxent.html
> >     will the utmpx interface get deprecated in POSIX, or stay as it is?
> >     Is the Austin Group already involved?
> 
> Linux is not POSIX conform, never was and there was never the plan to
> become, but it tries to be as compatible with POSIX as possible where
> it makes sense. If it does not make sense, POSIX will not be used. You
> should be able to find several examples in glibc, where interfaces
> derivate slightly from POSIX because the POSIX interface didn't made any
> sense.
> Parts of POSIX are really old (don't know how old utmp/utmpx are, but
> they did exist already since a long time before I started to work
> with Unix, and that's really a long time ago) and things are changing.
> Today, utmp/utmpx create more problems then it solve. Many features of
> it where never used on Linux or are meanwhile no longer used, at least
> not with systemd.
> There is just no benefit from it anymore. ...

I don't disagree with that.

Just that, as part of keeping Linux + glibc as close a possible to POSIX
over the long term, when we remove a feature from glibc that is POSIX-
standardized, we should also remove (or at least mark as LEGACY) this
part from POSIX. POSIX evolves, partially based on our inputs. I don't see
the deprecation of [1][2] on the Austin Group's agenda so far [3].

Bruno

[1] https://pubs.opengroup.org/onlinepubs/9699919799.2018edition/basedefs/utmpx.h.html
[2] https://pubs.opengroup.org/onlinepubs/9699919799.2018edition/functions/endutxent.html
[3] https://www.austingroupbugs.net/view_all_bug_page.php