Linux-ext4 Archive on lore.kernel.org
 help / color / Atom feed
* [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+
@ 2019-12-24 14:07 bugzilla-daemon
  2019-12-24 14:07 ` [Bug 205957] " bugzilla-daemon
                   ` (16 more replies)
  0 siblings, 17 replies; 18+ messages in thread
From: bugzilla-daemon @ 2019-12-24 14:07 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

            Bug ID: 205957
           Summary: Ext4 64 bit hash breaks 32 bit glibc 2.28+
           Product: File System
           Version: 2.5
    Kernel Version: 5.3.16 (any version)
          Hardware: ARM
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: ext4
          Assignee: fs_ext4@kernel-bugs.osdl.org
          Reporter: aladjev.andrew@gmail.com
        Regression: No

Hello. Please see the following glibc issue:
https://sourceware.org/bugzilla/show_bug.cgi?id=23960.

I am running arm system using qemu on host system with x86_64 kernel. I've
received 64 bits value for "d_off" instead of 32 bits from "getdents64".

Than I've tried i386 instead of arm and it works fine. I've received 32 bits
value for "d_off" from "getdents64".

I think that the problem is here
arch/x86/include/generated/uapi/asm/unistd_x32.h:
#define __NR_getdents64 (__X32_SYSCALL_BIT + 217)

arch/x86/include/asm/compat.h:
static inline bool in_x32_syscall(void)
{
#ifdef CONFIG_X86_X32_ABI
        if (task_pt_regs(current)->orig_ax & __X32_SYSCALL_BIT)
                return true;
#endif
        return false;
}

fs/ext4/dir.c:
static inline int is_32bit_api(void)
{
#ifdef CONFIG_COMPAT
        return in_compat_syscall();
#else
        return (BITS_PER_LONG == 32);
#endif
}

static inline loff_t hash2pos(struct file *filp, __u32 major, __u32 minor)
{
        if ((filp->f_mode & FMODE_32BITHASH) ||
            (!(filp->f_mode & FMODE_64BITHASH) && is_32bit_api()))
                return major >> 1;
        else
                return ((__u64)(major >> 1) << 32) | (__u64)minor;
}

I think that i386 makes a special syscall with "__X32_SYSCALL_BIT" enabled.
This special bit makes "in_x32_syscall", "in_compat_syscall" and "is_32bit_api"
to return true. This thing affects "hash2pos" and it works like
"FMODE_32BITHASH" is enabled.

ARM has no such special system, it uses generic "in_compat_syscall".
include/linux/compat.h:
* For most but not all architectures, "am I in a compat syscall?" and
* "am I a compat task?" are the same question.  For architectures on which
* they aren't the same question, arch code can override in_compat_syscall.

I don't know how to fix this issue.

1. Is it possible to implement special arm syscalls in the same way as x86?

2. Is it possible to add special compatibility syscall like
"__NR_compat_getdents64" that will make generic "in_compat_syscall" return
true?

3. Is it possible to force enable FMODE_32BITHASH for arm syscalls in runtime
in another way?

4. We can't shift "d_off >> 32" now, because we will have to restore "<< 32" in
all third party applications (not only glibc). Is it possible to replace
"((__u64)(major >> 1) << 32) | (__u64)minor" with "((__u64)minor << 32) |
(__u64)(major >> 1)"? So we will just ignore minor overflow.

5. Is it possible to refactor ext4 and remove hash from pos completely? It
provides so much pain.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
@ 2019-12-24 14:07 ` " bugzilla-daemon
  2019-12-24 15:09 ` bugzilla-daemon
                   ` (15 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2019-12-24 14:07 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

aladjev.andrew@gmail.com (aladjev.andrew@gmail.com) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                URL|                            |https://sourceware.org/bugz
                   |                            |illa/show_bug.cgi?id=23960

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
  2019-12-24 14:07 ` [Bug 205957] " bugzilla-daemon
@ 2019-12-24 15:09 ` bugzilla-daemon
  2019-12-24 18:16 ` bugzilla-daemon
                   ` (14 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2019-12-24 15:09 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #1 from dflogeras2@gmail.com ---
I too have been dealing with this situation in a few Qemu+chroot environments I
use for 32bit arm on a x86_64 host.  I have asked about it via this Qemu bug:

https://bugs.launchpad.net/qemu/+bug/1805913


as well as the glibc discussion in this bug may provide more insight:

https://sourceware.org/bugzilla/show_bug.cgi?id=23960


and finally this older LKML discussion has some more information:

https://lkml.org/lkml/2018/12/27/155

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
  2019-12-24 14:07 ` [Bug 205957] " bugzilla-daemon
  2019-12-24 15:09 ` bugzilla-daemon
@ 2019-12-24 18:16 ` bugzilla-daemon
  2019-12-25 22:50 ` bugzilla-daemon
                   ` (13 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2019-12-24 18:16 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

Andreas Dilger (adilger.kernelbugzilla@dilger.ca) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |adilger.kernelbugzilla@dilg
                   |                            |er.ca

--- Comment #2 from Andreas Dilger (adilger.kernelbugzilla@dilger.ca) ---
IMHO, it is broken to be calling a 64-bit interface like getdents64() and then
be unhappy when it is returning 64-bit values.  As previously stated, it would
be possible to add a "32bitapi" mount option to force ext4 to always return
32-bit offset values, as is done with NFS. 

The alternative is for QEMU's telldir() to see that d_off is a 64-bit value,
but is exporting a 32-bit interface and downshift the offset to fit into a
32-bit field. It would store a '64BITOFFSET' flag in the file descriptor in
this case, and if seekdir() is called on that fd it will upshift the offset to
a 64-bit value again.  That will lose the low bits of the offset, but that is
unlikely to be noticeable until there are more than 65000 entries in a
directory.  Since the hash values are uniformly distributed, they will almost
immediately exceed 32 bits, and telldir() is uncommon for use after the first
entry is read, so this detection will work reliably.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
                   ` (2 preceding siblings ...)
  2019-12-24 18:16 ` bugzilla-daemon
@ 2019-12-25 22:50 ` bugzilla-daemon
  2019-12-25 22:51 ` bugzilla-daemon
                   ` (12 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2019-12-25 22:50 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #3 from aladjev.andrew@gmail.com (aladjev.andrew@gmail.com) ---
Created attachment 286447
  --> https://bugzilla.kernel.org/attachment.cgi?id=286447&action=edit
getdents64_x32 kernel patch

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
                   ` (3 preceding siblings ...)
  2019-12-25 22:50 ` bugzilla-daemon
@ 2019-12-25 22:51 ` bugzilla-daemon
  2019-12-25 22:51 ` bugzilla-daemon
                   ` (11 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2019-12-25 22:51 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #4 from aladjev.andrew@gmail.com (aladjev.andrew@gmail.com) ---
Created attachment 286449
  --> https://bugzilla.kernel.org/attachment.cgi?id=286449&action=edit
getdents64_x32 glibc patch

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
                   ` (4 preceding siblings ...)
  2019-12-25 22:51 ` bugzilla-daemon
@ 2019-12-25 22:51 ` bugzilla-daemon
  2019-12-25 22:52 ` bugzilla-daemon
                   ` (10 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2019-12-25 22:51 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #5 from aladjev.andrew@gmail.com (aladjev.andrew@gmail.com) ---
Created attachment 286451
  --> https://bugzilla.kernel.org/attachment.cgi?id=286451&action=edit
getdents64_x32 qemu patch

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
                   ` (5 preceding siblings ...)
  2019-12-25 22:51 ` bugzilla-daemon
@ 2019-12-25 22:52 ` bugzilla-daemon
  2019-12-25 22:52 ` bugzilla-daemon
                   ` (9 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2019-12-25 22:52 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #6 from aladjev.andrew@gmail.com (aladjev.andrew@gmail.com) ---
Created attachment 286453
  --> https://bugzilla.kernel.org/attachment.cgi?id=286453&action=edit
getdents64_x32 libseccomp patch

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
                   ` (6 preceding siblings ...)
  2019-12-25 22:52 ` bugzilla-daemon
@ 2019-12-25 22:52 ` bugzilla-daemon
  2019-12-25 22:52 ` bugzilla-daemon
                   ` (8 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2019-12-25 22:52 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #7 from aladjev.andrew@gmail.com (aladjev.andrew@gmail.com) ---
Created attachment 286455
  --> https://bugzilla.kernel.org/attachment.cgi?id=286455&action=edit
getdents64_x32 go patch

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
                   ` (7 preceding siblings ...)
  2019-12-25 22:52 ` bugzilla-daemon
@ 2019-12-25 22:52 ` bugzilla-daemon
  2019-12-25 22:53 ` bugzilla-daemon
                   ` (7 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2019-12-25 22:52 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #8 from aladjev.andrew@gmail.com (aladjev.andrew@gmail.com) ---
Created attachment 286457
  --> https://bugzilla.kernel.org/attachment.cgi?id=286457&action=edit
getdents64_x32 bildah patch

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
                   ` (8 preceding siblings ...)
  2019-12-25 22:52 ` bugzilla-daemon
@ 2019-12-25 22:53 ` bugzilla-daemon
  2019-12-28  7:50 ` bugzilla-daemon
                   ` (6 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2019-12-25 22:53 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #9 from aladjev.andrew@gmail.com (aladjev.andrew@gmail.com) ---
I've created heavy (but easy) bunch of patches that fixes issue. This is the
order of right patches applyment (for gentoo):

+----------------------------------------------------------------------------+
|                                                                            |
| amd64 host:            +-------------------------------------------------+ |
|                        |                                                 | |
| linux-headers: kernel  | amd64 container:      +-----------------------+ | |
| gentoo-sources: kernel |                       |                       | | |
| libseccomp             | linux-headers: kernel | arm container (qemu): | | |
| go                     | qemu                  |                       | | |
| buildah                |                       | linux-headers: kernel | | |
|                        |                       | glibc                 | | |
|                        |                       |                       | | |
|                        |                       +-----------------------+ | |
|                        |                                                 | |
|                        +-------------------------------------------------+ |
|                                                                            |
+----------------------------------------------------------------------------+

You can find full example here
https://github.com/andrew-aladev/test-images/tree/master/cross/arm-unknown-linux-gnueabi.
Now I am able to emerge several application inside arm container (qemu), so
glibc with "getdents64_x32" works perfect.

I understand that kernel developers don't like additional syscalls, so they
won't accept such kernel patch. But this syscall is the easiest way for me to
provide a proof of concept.

How to understand these patches? Please read "glibc.patch". Our amd64 kernel is
the common kernel for amd64, i386 and arm (via qemu). amd64 is using
"getdents64" and arm (without LFS) is using "getdents64_x32". We need to have 2
syscalls simultaneously: "getdents64" and "getdents64_x32" or maybe single
syscall with some "x32" flag. I think this will be the most reliable solution.

I want to ask glibc and kernel developers to cooperate and find the right
solution. This is important not only for arm, it looks like this issue affects
any x32 abi.

Thank you.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
                   ` (9 preceding siblings ...)
  2019-12-25 22:53 ` bugzilla-daemon
@ 2019-12-28  7:50 ` bugzilla-daemon
  2019-12-28  9:00 ` bugzilla-daemon
                   ` (5 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2019-12-28  7:50 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #10 from Andreas Dilger (adilger.kernelbugzilla@dilger.ca) ---
IMHO, adding a new syscall for this is a lot of complexity that could be
avoided.  Using an ioctl() or fadvise() to set the "32-bitness" of the file
descriptor would seem like a simpler implementation that could be handled
directly by an ext4 patch rather than having to change every architecture just
to pass this flag.

Then, in the cases that QEMU is running in the confusing "I want 32-bit values
returned from a 64-bit system call" mode, it would call the ioctl(fd,
FS_IOC_32BITHASH) once before calling getdents64(fd, ...).

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
                   ` (10 preceding siblings ...)
  2019-12-28  7:50 ` bugzilla-daemon
@ 2019-12-28  9:00 ` bugzilla-daemon
  2019-12-28 11:20 ` bugzilla-daemon
                   ` (4 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2019-12-28  9:00 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #11 from aladjev.andrew@gmail.com (aladjev.andrew@gmail.com) ---
I think ext4 patch for qemu will be too weak solution, kernel wants to be a
black box for users and hide x32 implementation details. It will save
opportunity to maneuver for ext4 developers in future.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
                   ` (11 preceding siblings ...)
  2019-12-28  9:00 ` bugzilla-daemon
@ 2019-12-28 11:20 ` bugzilla-daemon
  2020-01-05 21:47 ` bugzilla-daemon
                   ` (3 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2019-12-28 11:20 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #12 from Andreas Dilger (adilger.kernelbugzilla@dilger.ca) ---
Created attachment 286491
  --> https://bugzilla.kernel.org/attachment.cgi?id=286491&action=edit
add ioctl(EXT4_IOC_SET_DIRLIMIT) to limit directory cookie size

Totally untested prototype patch to add ioctl(fd, EXT4_IOC_SET_DIRLIMIT, 32)
that could be used by glibc/QEMU to force ext4 to return a 32-bit directory
offset cookie for the 64-bit getdirent64() syscall on a per-fd basis.

The glibc patch would look something like the following, though this could
potentially be done only once per open:

 +  // It is affected by "__USE_FILE_OFFSET64" and "__USE_LARGEFILE64".
 +  if (sizeof (outp->u.d_off) != sizeof (inp->k.d_off))
 +    (void *) ioctl (fd, EXT4_IOC_SET_DIRLIMIT, 32); // ignore error return
 +
    retval = INLINE_SYSCALL_CALL (getdents64, fd, kbuf, kbytes);


If glibc/QEMU want ext4 to be totally transparent w.r.t. the behavior of the
syscalls, then glibc/QEMU would need set the kernel task state so that
is_32bit_api()->in_compat_syscall() return true and hash2pos() and related
functions can determine this directly to return a 32-bit pos in a manner
similar to x86.  That is outside my area of expertise, so I can't really
suggest how it might be done.  It would also be possible to add some different
logic to is_32bit_api() for ARM/etc. so that it is more transparent to
userspace, but I don't know what it should check.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
                   ` (12 preceding siblings ...)
  2019-12-28 11:20 ` bugzilla-daemon
@ 2020-01-05 21:47 ` bugzilla-daemon
  2020-01-05 22:08 ` bugzilla-daemon
                   ` (2 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2020-01-05 21:47 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #13 from aladjev.andrew@gmail.com (aladjev.andrew@gmail.com) ---
I am trying to work around mips issues and find this one
https://www.linux-mips.org/archives/linux-mips/2012-11/msg00031.html. This
issue from 2012 is the complete copy of our current issue. I can say that old
issue with mips is still alive: readdir is still broken on mips n32.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
                   ` (13 preceding siblings ...)
  2020-01-05 21:47 ` bugzilla-daemon
@ 2020-01-05 22:08 ` bugzilla-daemon
  2020-01-08 23:11 ` bugzilla-daemon
  2020-01-09 20:02 ` bugzilla-daemon
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2020-01-05 22:08 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #14 from John Paul Adrian Glaubitz (glaubitz@physik.fu-berlin.de) ---
Thanks for working on this! I think the suffix _x32 might be misleading as this
is the name of the 32-bit ABI on x86_64, see:
https://en.wikipedia.org/wiki/X32_ABI.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
                   ` (14 preceding siblings ...)
  2020-01-05 22:08 ` bugzilla-daemon
@ 2020-01-08 23:11 ` bugzilla-daemon
  2020-01-09 20:02 ` bugzilla-daemon
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2020-01-08 23:11 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #15 from aladjev.andrew@gmail.com (aladjev.andrew@gmail.com) ---
I've investigated mips related linux-user qemu and glibc source and mosaic
looks complete.

See qemu:
https://github.com/qemu/qemu/blob/master/linux-user/syscall.c#L9465-L9536
emulates getdents via getdents or getdents64, but don't emulate getdents64
using getdents.

Is it possible to emulate getdents64 using getdents? Yes, it was implemented in
glibc for mips 64:
https://github.com/bminor/glibc/blob/master/sysdeps/unix/sysv/linux/mips/mips64/getdents64.c#L49-L139
This emulation has been done because getdents64 was not available for some
ancient kernels on mips n64.

But this emulation should be a part of kernel. It has almost nothing to do with
qemu or glibc. Qemu should just launch compat syscall instead of regular and
that's it. Please see the following easy example:
https://github.com/torvalds/linux/blob/master/fs/read_write.c#L322-L332

SYSCALL_DEFINE3(lseek, unsigned int, fd, off_t, offset, unsigned int, whence)
{
        return ksys_lseek(fd, offset, whence);
}

#ifdef CONFIG_COMPAT
COMPAT_SYSCALL_DEFINE3(lseek, unsigned int, fd, compat_off_t, offset, unsigned
int, whence)
{
        return ksys_lseek(fd, offset, whence);
}
#endif

We need to declear compat syscall in the same way, like:

COMPAT_SYSCALL_DEFINE3(getdents64, unsigned int, fd, struct
compat_linux_dirent64 __user *, dirent, unsigned int, count)

"COMPAT_SYSCALL_DEFINE" will guarantee that "in_compat_syscall" will return
true and ext4 will use 32 bit hash.

Than we need to add this syscall to
https://github.com/torvalds/linux/blob/master/arch/x86/entry/syscalls/syscall_64.tbl#L361-L402

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [Bug 205957] Ext4 64 bit hash breaks 32 bit glibc 2.28+
  2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
                   ` (15 preceding siblings ...)
  2020-01-08 23:11 ` bugzilla-daemon
@ 2020-01-09 20:02 ` bugzilla-daemon
  16 siblings, 0 replies; 18+ messages in thread
From: bugzilla-daemon @ 2020-01-09 20:02 UTC (permalink / raw)
  To: linux-ext4

https://bugzilla.kernel.org/show_bug.cgi?id=205957

--- Comment #16 from aladjev.andrew@gmail.com (aladjev.andrew@gmail.com) ---
We don't need to create a patch for kernel about this compat syscall, because
it already supports "__X32_SYSCALL_BIT". We need just to enable it using
"CONFIG_X86_X32=y" kernel config. Recompiled kernel will expose all x86 related
syscalls in x32 mode.

I've tested that "__X32_SYSCALL_BIT + 217" works completely the same as my
"SYS_getdents64_x32". Function "in_x32_syscall" works perfect.

We can patch glibc to redirect getdents64 syscall to x32 compatible one. My
previous assumption was wrong. Patching qemu will be very bad solution, because
it means that large file support will be broken in all applications, not only
glibc.

For now only glibc wants to convert dirent64 to dirent just by copying fields.
Regular application don't want to emulate getdents using getdents64 and it will
not touch dirent64. So we shouldn't interfere in qemu syscalls.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, back to index

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-24 14:07 [Bug 205957] New: Ext4 64 bit hash breaks 32 bit glibc 2.28+ bugzilla-daemon
2019-12-24 14:07 ` [Bug 205957] " bugzilla-daemon
2019-12-24 15:09 ` bugzilla-daemon
2019-12-24 18:16 ` bugzilla-daemon
2019-12-25 22:50 ` bugzilla-daemon
2019-12-25 22:51 ` bugzilla-daemon
2019-12-25 22:51 ` bugzilla-daemon
2019-12-25 22:52 ` bugzilla-daemon
2019-12-25 22:52 ` bugzilla-daemon
2019-12-25 22:52 ` bugzilla-daemon
2019-12-25 22:53 ` bugzilla-daemon
2019-12-28  7:50 ` bugzilla-daemon
2019-12-28  9:00 ` bugzilla-daemon
2019-12-28 11:20 ` bugzilla-daemon
2020-01-05 21:47 ` bugzilla-daemon
2020-01-05 22:08 ` bugzilla-daemon
2020-01-08 23:11 ` bugzilla-daemon
2020-01-09 20:02 ` bugzilla-daemon

Linux-ext4 Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-ext4/0 linux-ext4/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-ext4 linux-ext4/ https://lore.kernel.org/linux-ext4 \
		linux-ext4@vger.kernel.org
	public-inbox-index linux-ext4

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-ext4


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git