[Buildroot] uclibc-ng built with wrong _FILE_OFFSET_BITS

[Buildroot] uclibc-ng built with wrong _FILE_OFFSET_BITS

[Max Filippov]

Hello,

I've tried to run uclibc-ng tests on xtensa in a buildroot-built rootfs
and found misc/tst-preadvwritev failing with the following message:

  first buffer from first preadv different than expected

Digging deeper into it I've found that the test is built with the following
definition in the command line: -D_FILE_OFFSET_BITS=64,
due to the following line in the uclibc-ng-test.mk:

  UCLIBC_EXTRA_CFLAGS="$(TARGET_CFLAGS)"

but uclibc-ng itself does not have -D_FILE_OFFSET_BITS=64,
because of the following definition in the uclibc-ng.mk:

  UCLIBC_EXTRA_CFLAGS="$(TARGET_ABI)"

uclibc-ng defaults to 32-bit off_t type then.
I'm not sure what's broken here, buildroot or uclibc-ng. Ideas?

-- 
Thanks.
-- Max

[Buildroot] uclibc-ng built with wrong _FILE_OFFSET_BITS

[Max Filippov]

On Thu, May 9, 2019 at 11:34 AM Max Filippov <jcmvbkbc@gmail.com> wrote:
> I've tried to run uclibc-ng tests on xtensa in a buildroot-built rootfs
> and found misc/tst-preadvwritev failing with the following message:
>
>   first buffer from first preadv different than expected
>
> Digging deeper into it I've found that the test is built with the following
> definition in the command line: -D_FILE_OFFSET_BITS=64,
> due to the following line in the uclibc-ng-test.mk:
>
>   UCLIBC_EXTRA_CFLAGS="$(TARGET_CFLAGS)"
>
> but uclibc-ng itself does not have -D_FILE_OFFSET_BITS=64,
> because of the following definition in the uclibc-ng.mk:
>
>   UCLIBC_EXTRA_CFLAGS="$(TARGET_ABI)"
>
> uclibc-ng defaults to 32-bit off_t type then.
> I'm not sure what's broken here, buildroot or uclibc-ng. Ideas?

Ok, I see something definitely strange in the uclibc-ng:
ublibc-ng does not honor -D_FILE_OFFSET_BITS=64
when it's passed to it in the make command, because
the following fragment of the include/features.h that defines
__USE_FILE_OFFSET64 based on the value of
_FILE_OFFSET_BITS

#if defined _FILE_OFFSET_BITS && _FILE_OFFSET_BITS == 64
# define __USE_FILE_OFFSET64    1
#endif

is neutralized by the following fragment at the end of the same file:

#ifdef _LIBC
#  undef _FILE_OFFSET_BITS
#  undef __USE_FILE_OFFSET64
# include <libc-internal.h>
#endif

--
Thanks.
-- Max

[Buildroot] [uclibc-ng-devel] uclibc-ng built with wrong _FILE_OFFSET_BITS

[Max Filippov]

On Fri, May 10, 2019 at 6:32 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
>
> If that can be of any help, this test seems to PASS on armv7 arch:
> https://uclibc-ng-ci.sionneau.net:8443/job/uclibc-ng-multiarch/arch=armv7/19/console
>
> However this bugs reproduces for mips32r6 :
> https://uclibc-ng-ci.sionneau.net:8443/job/uclibc-ng-multiarch/19/arch=mips32r6/console
>
> Also, I can confirm I am able to reproduce the issue with
> buildroot+uclibc-ng by building the qemu_xtensa_lx60_defconfig buildroot
> config

Thanks for the confirmation.
I've spent some more time on this issue and I hope have got a better
understanding of it. So:
- uclibc-ng itself is not meant to be compiled differently with
-D_FILE_OFFSET_BITS=64
  and without it. It is meant to provide both LFS and non-LFS function versions,
  like ftruncate and ftruncate64.
- the library user on the other hand gets different set of
declarations depending
  on the presence of -D_FILE_OFFSET_BITS=64, that results in a call to different
  functions.
- however this is not the case for preadv/pwritev: there's only one
definition for
  each of these functions. When libc is built these functions assume 32-bit
  off_t, but when the user code is built, off_t size is selected by the
  _FILE_OFFSET_BITS resulting in declaration that does not match definition.

I guess the right way to fix it is using __off64_t in declarations and
definitions
of both preadv/pwritev. With this change the test passes on xtensa.

-- 
Thanks.
-- Max