linux interprets an fcntl int arg as long

linux interprets an fcntl int arg as long

[Szabolcs Nagy]

in short, F_ADD_SEALS fcntl cmd is documented to take int arg,
but linux mm/memfd.c has

        switch (cmd) {
        case F_ADD_SEALS:
                /* disallow upper 32bit */
                if (arg > UINT_MAX)
                        return -EINVAL;

fcntl is variadic:

  int fcntl(int fd, int cmd, ... /* arg */);

and arg is either int or pointer in the current documentation.
the libc does not know which (except for existing commands,
but there can be future extensions).

so glibc just assumes pointer arg and passes it down to the
kernel. musl uses unsigned long arg, but either way depending
on the vararg abi rules of the target the top bits of an int
arg can be non-zero when passed to the kernel. (in principle
it could crash too: variadic args only supposed to work when
the type is right, but in existing abis this does not seem to
be a problem.)

e.g. the following fails with EINVAL:

  int fd = memfd_create("test", MFD_CLOEXEC|MFD_ALLOW_SEALING);
  int r = fcntl(fd, F_ADD_SEALS, F_SEAL_WRITE | 0xF00000000);

and such fcntl call can happen with c code that just passes
F_SEAL_WRITE since it is an int and e.g. with aarch64 pcs rules
it is passed in a register where top bits can be non-zero
(unlikely in practice but valid).

i think there are general issues with variadic syscalls:

1) existing varargs that are documented as int should be
interpret as int on the kernel side (the libc cannot fix this
up reliably). i.e. linux should fix the F_ADD_SEALS code.

2) variadic args ideally should not have mixed type and the
type that is used for future extensions should be documented.

3) macro defines for varargs that are long should have type
long and not int in the uapi. (affects some prctl flags)
otherwise users must cast manually to the right type.

thanks.

Re: linux interprets an fcntl int arg as long

[Theodore Ts'o]

On Mon, Oct 31, 2022 at 12:44:59PM +0000, Szabolcs Nagy wrote:
> and such fcntl call can happen with c code that just passes
> F_SEAL_WRITE since it is an int and e.g. with aarch64 pcs rules
> it is passed in a register where top bits can be non-zero
> (unlikely in practice but valid).

In Linux's aarch64 ABI, an int is a 4-byte value.  It is *not* an
8-byte value.  So passing in "F_SEAL_WRITE | 0xF00000000" as an int
(as in your example) is simply not valid thing for the userspace
program to do.

Now, if there is a C program which has "int c = F_SEAL_WRITE", if the
PCS allows the compiler to pass a function paramter c --- for example
f(a, b, c) --- where the 4-byte paramter 'c' is placed in a 64-bit
register where the high bits of the 64-bit register contains non-zero
garbage values, I would argue that this is a bug in the PCS and/or the
compiler.

						- Ted

Re: linux interprets an fcntl int arg as long

[Szabolcs Nagy]

The 10/31/2022 21:46, Theodore Ts'o wrote:
> On Mon, Oct 31, 2022 at 12:44:59PM +0000, Szabolcs Nagy wrote:
> > and such fcntl call can happen with c code that just passes
> > F_SEAL_WRITE since it is an int and e.g. with aarch64 pcs rules
> > it is passed in a register where top bits can be non-zero
> > (unlikely in practice but valid).
> 
> In Linux's aarch64 ABI, an int is a 4-byte value.  It is *not* an
> 8-byte value.  So passing in "F_SEAL_WRITE | 0xF00000000" as an int
> (as in your example) is simply not valid thing for the userspace
> program to do.
> 
> Now, if there is a C program which has "int c = F_SEAL_WRITE", if the
> PCS allows the compiler to pass a function paramter c --- for example
> f(a, b, c) --- where the 4-byte paramter 'c' is placed in a 64-bit
> register where the high bits of the 64-bit register contains non-zero
> garbage values, I would argue that this is a bug in the PCS and/or the
> compiler.

the callee uses va_arg(ap, type) to get the argument,
and if the type is wider than what was actually passed
then anything can happen. in practice what happens is
that the top bits can be non-zero.

many pcs are affected (aarch64 is the one i know well,
but at least x86_64, arm are affected too). and even if
it was aarch64 pcs only, it is incompetent to say that
the pcs is wrong: that's a constraint we are working with.

the kernel must not read a wider type than what it
documents as argument to variadic functions in the c api.
(it does not make much sense to expect anything there
anyway, but it can break userspace)

RE: linux interprets an fcntl int arg as long

[David Laight]

From: Szabolcs Nagy
> Sent: 01 November 2022 09:11
> 
> The 10/31/2022 21:46, Theodore Ts'o wrote:
> > On Mon, Oct 31, 2022 at 12:44:59PM +0000, Szabolcs Nagy wrote:
> > > and such fcntl call can happen with c code that just passes
> > > F_SEAL_WRITE since it is an int and e.g. with aarch64 pcs rules
> > > it is passed in a register where top bits can be non-zero
> > > (unlikely in practice but valid).
> >
> > In Linux's aarch64 ABI, an int is a 4-byte value.  It is *not* an
> > 8-byte value.  So passing in "F_SEAL_WRITE | 0xF00000000" as an int
> > (as in your example) is simply not valid thing for the userspace
> > program to do.
> >
> > Now, if there is a C program which has "int c = F_SEAL_WRITE", if the
> > PCS allows the compiler to pass a function paramter c --- for example
> > f(a, b, c) --- where the 4-byte paramter 'c' is placed in a 64-bit
> > register where the high bits of the 64-bit register contains non-zero
> > garbage values, I would argue that this is a bug in the PCS and/or the
> > compiler.
> 
> the callee uses va_arg(ap, type) to get the argument,
> and if the type is wider than what was actually passed
> then anything can happen. in practice what happens is
> that the top bits can be non-zero.
> 
> many pcs are affected (aarch64 is the one i know well,
> but at least x86_64, arm are affected too). and even if
> it was aarch64 pcs only, it is incompetent to say that
> the pcs is wrong: that's a constraint we are working with.
> 
> the kernel must not read a wider type than what it
> documents as argument to variadic functions in the c api.
> (it does not make much sense to expect anything there
> anyway, but it can break userspace)

The Linux kernel just assumes that the varargs call looks like
a non-varags call with the same parameters.
(It doesn't use va_arg())
All syscall arguments are passed in registers (unlike BSDs
where they can also be on the user stack).
On 64bit systems the same registers are expected to be used
for 64bit and 32bit integers and for pointers.
32bit values usually get masked because they get passed to
a function with an 'int' argument.

If any fcntl() calls require a 64bit value and the C ABI
might leave non-zero high bits in an register containing
a 32bit value (esp. to a varargs function) then the calling
code will need to cast such arguments to 64 bits.

OTOH I suspect the argument is either absent, int or pointer.
So it should mask the value to 32 bits.

Note that there are ABI where 'int' and 'pointer' get passed
in different registers.
Fortunately none will support Linux!

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

Re: linux interprets an fcntl int arg as long

['Szabolcs Nagy']

The 11/01/2022 10:02, David Laight wrote:
> From: Szabolcs Nagy
> > Sent: 01 November 2022 09:11
> > 
> > The 10/31/2022 21:46, Theodore Ts'o wrote:
> > > On Mon, Oct 31, 2022 at 12:44:59PM +0000, Szabolcs Nagy wrote:
> > > > and such fcntl call can happen with c code that just passes
> > > > F_SEAL_WRITE since it is an int and e.g. with aarch64 pcs rules
> > > > it is passed in a register where top bits can be non-zero
> > > > (unlikely in practice but valid).
> > >
> > > In Linux's aarch64 ABI, an int is a 4-byte value.  It is *not* an
> > > 8-byte value.  So passing in "F_SEAL_WRITE | 0xF00000000" as an int
> > > (as in your example) is simply not valid thing for the userspace
> > > program to do.
> > >
> > > Now, if there is a C program which has "int c = F_SEAL_WRITE", if the
> > > PCS allows the compiler to pass a function paramter c --- for example
> > > f(a, b, c) --- where the 4-byte paramter 'c' is placed in a 64-bit
> > > register where the high bits of the 64-bit register contains non-zero
> > > garbage values, I would argue that this is a bug in the PCS and/or the
> > > compiler.
> > 
> > the callee uses va_arg(ap, type) to get the argument,
> > and if the type is wider than what was actually passed
> > then anything can happen. in practice what happens is
> > that the top bits can be non-zero.
> > 
> > many pcs are affected (aarch64 is the one i know well,
> > but at least x86_64, arm are affected too). and even if
> > it was aarch64 pcs only, it is incompetent to say that
> > the pcs is wrong: that's a constraint we are working with.
> > 
> > the kernel must not read a wider type than what it
> > documents as argument to variadic functions in the c api.
> > (it does not make much sense to expect anything there
> > anyway, but it can break userspace)
> 
> The Linux kernel just assumes that the varargs call looks like
> a non-varags call with the same parameters.
> (It doesn't use va_arg())
> All syscall arguments are passed in registers (unlike BSDs
> where they can also be on the user stack).
> On 64bit systems the same registers are expected to be used
> for 64bit and 32bit integers and for pointers.
> 32bit values usually get masked because they get passed to
> a function with an 'int' argument.
> 
> If any fcntl() calls require a 64bit value and the C ABI
> might leave non-zero high bits in an register containing
> a 32bit value (esp. to a varargs function) then the calling
> code will need to cast such arguments to 64 bits.

the entire point of my mail is that it is not possible
to tell in the libc if the vararg is pointer or int.

so in case a user passed an int, the libc cannot fix
that up, like it usually does for other cases where
linux syscall abi is incompatible with the c api.

let me go through step by step what is going on:

user code:
----------
int fcntl(int, int, ...);
void foobar(int fd, int foo, long arg)
{
	/* top bits of arg are used by the user (non-zero),
	   they are not for the syscall. */

	/* bottom bits of arg are passed to fcntl:
	   the compiler passes arg unmodified in a register
	   where top bits are set, but that's fine: the callee
	   only supposed to use the bottom 32bits.  */
	fcntl(fd, F_ADD_SEALS, (int)arg);
}

libc code:
----------
long internal_syscall(int, long, long, long, long, long, long);
int fcntl(int fd, int cmd, ...)
{
	va_list ap;
	va_start(ap, cmd);
	/* this is non-conforming C: wrong type in va_arg,
	   but that's not relevant since libc can implement
	   this as target specific asm, the important bit is
	   that the correct type is not known: libc cannot
	   replicate the kernel side dispatch logic because
	   new cmd can be introduced in the future with
	   arbitrary type arg.

	   top 32bits of arg are non-zero, libc cannot
	   zero them here as arg may be long or pointer.  */
	long arg = va_arg(ap, long);
	va_end(ap);
	return internal_syscall(SYS_fcntl, fd, cmd, arg, 0, 0, 0);
}

kernel code:
------------
SYSCALL_DEFINE3(fcntl, unsigned int, fd, unsigned int, cmd, unsigned long, arg)
{
	...
	switch (cmd) {
	...
	/* this part is in memfd_fcntl.  */
	case F_ADD_SEALS:
		/* uses arg as long instead of the documented
		   int type and fails because top bits are set.  */
	}
}



> 
> OTOH I suspect the argument is either absent, int or pointer.
> So it should mask the value to 32 bits.

how does that work on 64bit targets?

and how does it work on 32bit targets if a new cmd
is introduced with short int argument (posix does
not guarantee that it won't)?

> 
> Note that there are ABI where 'int' and 'pointer' get passed
> in different registers.
> Fortunately none will support Linux!

what i explained above is not theoretical abi or libc:
this is how things work now in practice on x86_64 and
aarch64 and it is clearly broken.

normally when linux does something silly like this the
libc fixes it up (since we want to support old kernels),
but in this case that would mean replicating the entire
ioctl, prctl, fcntl, etc dispatch logic in userspace
which is impractical and cannot be future proof.

so going forward i'd like linux devs to consider the
3 points i raised in my original mail. and fix
memfd_fcntl accordingly.

thanks.


complete example (tested on aarch64 and x86_64):

$ cat a.c
#define _GNU_SOURCE 1
#include <stdio.h>
#include <fcntl.h>
#include <sys/mman.h>

int g = 0;
__attribute__((noinline)) /* pretend to be in another TU. */
int foobar(int fd, int c, long arg)
{
        if (arg & 0x100000000) /* use top bits for something */
                g+=c;
        return fcntl(fd, F_ADD_SEALS, (int)arg);
}

int main (int argc, char *argv[])
{
  int fd = memfd_create ("tst", MFD_CLOEXEC|MFD_ALLOW_SEALING);
  if (fd < 0 ) return -1;
  int r = foobar(fd, argc, F_SEAL_WRITE | 0xF00000000);
  printf("%d %d %m\n", g, r);
  return 0;
}
$ gcc -O2 a.c
$ ./a.out
1 -1 Invalid argument
$ strace ./a.out 2>&1 |grep fcntl
fcntl(3, F_ADD_SEALS, F_SEAL_WRITE|0xf00000000) = -1 EINVAL (Invalid argument)

RE: linux interprets an fcntl int arg as long

[David Laight]

From: 'Szabolcs Nagy' <szabolcs.nagy@arm.com>
> Sent: 01 November 2022 11:45
> 
> The 11/01/2022 10:02, David Laight wrote:
> > From: Szabolcs Nagy
> > > Sent: 01 November 2022 09:11
> > >
> > > The 10/31/2022 21:46, Theodore Ts'o wrote:
> > > > On Mon, Oct 31, 2022 at 12:44:59PM +0000, Szabolcs Nagy wrote:
> > > > > and such fcntl call can happen with c code that just passes
> > > > > F_SEAL_WRITE since it is an int and e.g. with aarch64 pcs rules
> > > > > it is passed in a register where top bits can be non-zero
> > > > > (unlikely in practice but valid).
> > > >
> > > > In Linux's aarch64 ABI, an int is a 4-byte value.  It is *not* an
> > > > 8-byte value.  So passing in "F_SEAL_WRITE | 0xF00000000" as an int
> > > > (as in your example) is simply not valid thing for the userspace
> > > > program to do.
> > > >
> > > > Now, if there is a C program which has "int c = F_SEAL_WRITE", if the
> > > > PCS allows the compiler to pass a function paramter c --- for example
> > > > f(a, b, c) --- where the 4-byte paramter 'c' is placed in a 64-bit
> > > > register where the high bits of the 64-bit register contains non-zero
> > > > garbage values, I would argue that this is a bug in the PCS and/or the
> > > > compiler.
> > >
> > > the callee uses va_arg(ap, type) to get the argument,
> > > and if the type is wider than what was actually passed
> > > then anything can happen. in practice what happens is
> > > that the top bits can be non-zero.
> > >
> > > many pcs are affected (aarch64 is the one i know well,
> > > but at least x86_64, arm are affected too). and even if
> > > it was aarch64 pcs only, it is incompetent to say that
> > > the pcs is wrong: that's a constraint we are working with.
> > >
> > > the kernel must not read a wider type than what it
> > > documents as argument to variadic functions in the c api.
> > > (it does not make much sense to expect anything there
> > > anyway, but it can break userspace)
> >
> > The Linux kernel just assumes that the varargs call looks like
> > a non-varags call with the same parameters.
> > (It doesn't use va_arg())
> > All syscall arguments are passed in registers (unlike BSDs
> > where they can also be on the user stack).
> > On 64bit systems the same registers are expected to be used
> > for 64bit and 32bit integers and for pointers.
> > 32bit values usually get masked because they get passed to
> > a function with an 'int' argument.
> >
> > If any fcntl() calls require a 64bit value and the C ABI
> > might leave non-zero high bits in an register containing
> > a 32bit value (esp. to a varargs function) then the calling
> > code will need to cast such arguments to 64 bits.
> 
> the entire point of my mail is that it is not possible
> to tell in the libc if the vararg is pointer or int.
> 
> so in case a user passed an int, the libc cannot fix
> that up, like it usually does for other cases where
> linux syscall abi is incompatible with the c api.
> 
> let me go through step by step what is going on:
...
> long internal_syscall(int, long, long, long, long, long, long);
> int fcntl(int fd, int cmd, ...)
> {
> 	va_list ap;
> 	va_start(ap, cmd);
> 	/* this is non-conforming C: wrong type in va_arg,
> 	   but that's not relevant since libc can implement
> 	   this as target specific asm, the important bit is
> 	   that the correct type is not known: libc cannot
> 	   replicate the kernel side dispatch logic because
> 	   new cmd can be introduced in the future with
> 	   arbitrary type arg.
> 
> 	   top 32bits of arg are non-zero, libc cannot
> 	   zero them here as arg may be long or pointer.  */
> 	long arg = va_arg(ap, long);

Here libc has to assume that int, long and pointer are passed
the same way.
This is true for everything Linux actually runs on.
But wouldn't be if, for example, a 64bit arch just pushed
32bit arguments on stack.
Or if a 32 bit one passed integer and pointer args in different
types of registers.
But all 64bit ones use the same GP registers for int/long/pointer.

> kernel code:
> ------------
> SYSCALL_DEFINE3(fcntl, unsigned int, fd, unsigned int, cmd, unsigned long, arg)
> {

That is just a wrapper and calls do_fcntl().
which needs changing to be add:
	arg &= ~0U;
before the switch(cmd) {

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

Re: linux interprets an fcntl int arg as long

['Szabolcs Nagy']

The 11/01/2022 12:19, David Laight wrote:
> From: 'Szabolcs Nagy' <szabolcs.nagy@arm.com>
> > SYSCALL_DEFINE3(fcntl, unsigned int, fd, unsigned int, cmd, unsigned long, arg)
> > {
> 
> That is just a wrapper and calls do_fcntl().
> which needs changing to be add:
> 	arg &= ~0U;
> before the switch(cmd) {

this makes sense.
i thought previously you meant masking in userspace.

although if you mask there then many linux internal
apis could use int instead of long arg.

do you want me to post a patch? (i'm not a kernel
developer)

Re: linux interprets an fcntl int arg as long

[Mark Rutland]

On Tue, Nov 01, 2022 at 12:19:51PM +0000, David Laight wrote:
> From: 'Szabolcs Nagy' <szabolcs.nagy@arm.com>
> > Sent: 01 November 2022 11:45
> > 
> > The 11/01/2022 10:02, David Laight wrote:
> > > From: Szabolcs Nagy

> > kernel code:
> > ------------
> > SYSCALL_DEFINE3(fcntl, unsigned int, fd, unsigned int, cmd, unsigned long, arg)
> > {
> 
> That is just a wrapper and calls do_fcntl().
> which needs changing to be add:
> 	arg &= ~0U;
> before the switch(cmd) {

Just to check, do you mean the switch in do_fcntl(), or the switch within memfd_fcntl() ?

The former handles other APIs which do expect arg to be a long (e.g.
F_SET_RW_HINT and F_GET_RW_HINT expect it to hold a full 64-bit pointer), so
that'd break things.

The latter would work (as would casting arg to int when calling
memfd_add_seals()).

Mark.

RE: linux interprets an fcntl int arg as long

[David Laight]

From: Mark Rutland
> Sent: 01 November 2022 13:13
> 
> On Tue, Nov 01, 2022 at 12:19:51PM +0000, David Laight wrote:
> > From: 'Szabolcs Nagy' <szabolcs.nagy@arm.com>
> > > Sent: 01 November 2022 11:45
> > >
> > > The 11/01/2022 10:02, David Laight wrote:
> > > > From: Szabolcs Nagy
> 
> > > kernel code:
> > > ------------
> > > SYSCALL_DEFINE3(fcntl, unsigned int, fd, unsigned int, cmd, unsigned long, arg)
> > > {
> >
> > That is just a wrapper and calls do_fcntl().
> > which needs changing to be add:
> > 	arg &= ~0U;
> > before the switch(cmd) {
> 
> Just to check, do you mean the switch in do_fcntl(), or the switch within memfd_fcntl() ?
> 
> The former handles other APIs which do expect arg to be a long (e.g.
> F_SET_RW_HINT and F_GET_RW_HINT expect it to hold a full 64-bit pointer), so
> that'd break things.

The assignment to argv is earlier.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

RE: linux interprets an fcntl int arg as long

[David Laight]

From: David Laight
> Sent: 01 November 2022 13:30
> 
> From: Mark Rutland
> > Sent: 01 November 2022 13:13
> >
> > On Tue, Nov 01, 2022 at 12:19:51PM +0000, David Laight wrote:
> > > From: 'Szabolcs Nagy' <szabolcs.nagy@arm.com>
> > > > Sent: 01 November 2022 11:45
> > > >
> > > > The 11/01/2022 10:02, David Laight wrote:
> > > > > From: Szabolcs Nagy
> >
> > > > kernel code:
> > > > ------------
> > > > SYSCALL_DEFINE3(fcntl, unsigned int, fd, unsigned int, cmd, unsigned long, arg)
> > > > {
> > >
> > > That is just a wrapper and calls do_fcntl().
> > > which needs changing to be add:
> > > 	arg &= ~0U;
> > > before the switch(cmd) {
> >
> > Just to check, do you mean the switch in do_fcntl(), or the switch within memfd_fcntl() ?
> >
> > The former handles other APIs which do expect arg to be a long (e.g.
> > F_SET_RW_HINT and F_GET_RW_HINT expect it to hold a full 64-bit pointer), so
> > that'd break things.
> 
> The assignment to argv is earlier.

Clearly I meant argp :-)

Alternatively all the helper functions should be changed to
have an 'unsigned int' argument instead of 'unsigned long'.

That might show up any that try to read user buffers.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)