Re: Argument type for FS_IOC_GETFLAGS/FS_IOC_SETFLAGS ioctls

[Theodore Ts'o <tytso@mit.edu>]

On Tue, Nov 26, 2013 at 05:01:41PM -0800, Darrick J. Wong wrote:
> > Most of the userland code seems to pass an int to this ioctl, but a few
> > others (e.g.: bup, libexplain) passes a long. While it doesn't make a
> > difference on little endian machines, it does make a difference on
> > 64-bit big endian machines.
> > 
> > Could you please tell me if I am wrong in my analysis or if there is a
> > actually real problem?
> 
> It also causes problems with FUSE, because the kernel fuse driver expects to be
> able to transfer a ulong to and from userspace, but chattr & friends only
> allocate an int on the stack, so stack mashing seems to happen.
> 
> I complained to tytso about it on linux-ext4 a while ago, he suggested
> special-casing fuse... I haven't gotten around to doing that.

This is a mistake that was made a long, LONG, LONG time ago.  And so
there are huge numbers of userspace programs which are using an int,
and we change it to be a long, it will break those userspace programs
for ALL platforms where sizeof(int) != sizeof(long).  This includes all
64-bit x86 systems, for which there are quite a few.  :-)

Yes, it's unfortunate that programs that programs that try to use a
long are breaking on 64-bit big endian machines, but (a) there are
much fewer of them, and (b) they are only breaking on big endian
machines, as opposed the much bigger class of userspace programs which
would break on the proposed change for ALL 64-bit platforms, including
x86-64.  And like it or not, there are a lot more linux machines
running x86-64, compared to those running linux on big-endian PowerPC.
(Of course, the little-endian ppc machines which IBM is now pushing
for Linux in data centers will be just fine.  :-P)

If people really cared, we could allocate a new ioctl codepoint, and
then teach the kernel to support the new ioctl number, and then
gradually change userspace to first try the new ioctl, and if that
failed go back to the old one.  The coversion progress would take 5-10
years (there are still sites running RHEL 3, and RHEL 4 after all),
and it wouldn't help existing userspace programs, that would still be
using the old code point.  Hence my recommendation that the path of
least resistence is to fix the kernel FUSE code, instead of trying to
change the world.

Regards,

					- Ted

P.S.  If we were going to create a new ioctl, what I'd suggest is that
the new ioctl explicitly use a 64-bit type, instead of using "long" or
"int", to avoid the compat ioctl hair to allow 64-bit kernels to
support 32-bit userspace programs.