xfstest and chromeos

xfstest and chromeos

[Gwendal Grignou]

I am adding xfstest to chromeos test image, to improve chromeos
filesystems test coverage.
Doing so, I encountered to issues, documented in
https://chromium-review.googlesource.com/#/c/424372
and
https://chromium-review.googlesource.com/#/c/424191

https://chromium-review.googlesource.com/#/c/424372
- To circuvent SELinux protection we add a mount context option,
requesting to use nfs_t context. However on chromeos, we have no nfs
security context - we are not expecting usings nfs - leading to a
mount error. Shall we try to use another context or check beforehand
if a context is needed for mount?

https://chromium-review.googlesource.com/#/c/424191
- Chromeos needs libtool to (cross-)compile packages and put them at
the right place. I needed to move archived aclocal.m4 to acinclude.m4.
Does it make sense to require aclocal to run to compile xfstests?

Gwendal.

Re: xfstest and chromeos

[Eryu Guan]

On Mon, Jan 23, 2017 at 09:57:15AM -0800, Gwendal Grignou wrote:
> I am adding xfstest to chromeos test image, to improve chromeos
> filesystems test coverage.
> Doing so, I encountered to issues, documented in
> https://chromium-review.googlesource.com/#/c/424372
> and
> https://chromium-review.googlesource.com/#/c/424191
> 
> https://chromium-review.googlesource.com/#/c/424372
> - To circuvent SELinux protection we add a mount context option,
> requesting to use nfs_t context. However on chromeos, we have no nfs
> security context - we are not expecting usings nfs - leading to a
> mount error. Shall we try to use another context or check beforehand
> if a context is needed for mount?

The selinux context is there because without it some quota or xattr
tests will fail due to extra xattr set by selinux.

I'm not sure what context works for chromeos, but I think we can make
SELINUX_MOUNT_OPTIONS configurable, and it defaults to current value. So
that chromeos could use any context that works for it.

> 
> https://chromium-review.googlesource.com/#/c/424191
> - Chromeos needs libtool to (cross-)compile packages and put them at
> the right place. I needed to move archived aclocal.m4 to acinclude.m4.
> Does it make sense to require aclocal to run to compile xfstests?

Sorry, I'm not that familiar with all the autotools build system.. I may
need some time to look into it. Or someone else who is familiar with it
could provide some inputs here?

Thanks,
Eryu

Re: xfstest and chromeos

[Mike Frysinger]

On Wed, Jan 25, 2017 at 7:14 PM, Eryu Guan wrote:
> On Mon, Jan 23, 2017 at 09:57:15AM -0800, Gwendal Grignou wrote:
>> I am adding xfstest to chromeos test image, to improve chromeos
>> filesystems test coverage.
>> Doing so, I encountered to issues, documented in
>> https://chromium-review.googlesource.com/#/c/424372
>> and
>> https://chromium-review.googlesource.com/#/c/424191
>>
>> https://chromium-review.googlesource.com/#/c/424372
>> - To circuvent SELinux protection we add a mount context option,
>> requesting to use nfs_t context. However on chromeos, we have no nfs
>> security context - we are not expecting usings nfs - leading to a
>> mount error. Shall we try to use another context or check beforehand
>> if a context is needed for mount?
>
> The selinux context is there because without it some quota or xattr
> tests will fail due to extra xattr set by selinux.
>
> I'm not sure what context works for chromeos, but I think we can make
> SELINUX_MOUNT_OPTIONS configurable, and it defaults to current value. So
> that chromeos could use any context that works for it.

that sounds fine.  something like:
  if [ -x /usr/sbin/selinuxenabled ] && /usr/sbin/selinuxenabled; then
    : ${SELINUX_MOUNT_OPTIONS:="-o context=system_u:object_r:nfs_t:s0"}
    export SELINUX_MOUNT_OPTIONS
  fi

>> https://chromium-review.googlesource.com/#/c/424191
>> - Chromeos needs libtool to (cross-)compile packages and put them at
>> the right place. I needed to move archived aclocal.m4 to acinclude.m4.
>> Does it make sense to require aclocal to run to compile xfstests?
>
> Sorry, I'm not that familiar with all the autotools build system.. I may
> need some time to look into it. Or someone else who is familiar with it
> could provide some inputs here?

i've long been hesitant to touch the xfs build code because it looks
like SGI/XFS peeps have spent a lot of time avoiding using autotools,
or maybe the code is so old it predates support on the autotools side.
either way, it's not clear what exactly the goal is here.

Gwendal's patch needs a bit more work, but if people are open to
depending on libtool in this way (which would fix cross-compiling
problems), then i can update his patch and send it out.
-mike

Re: xfstest and chromeos

[Theodore Ts'o]

On Mon, Jan 23, 2017 at 09:57:15AM -0800, Gwendal Grignou wrote:
> https://chromium-review.googlesource.com/#/c/424191
> - Chromeos needs libtool to (cross-)compile packages and put them at
> the right place. I needed to move archived aclocal.m4 to acinclude.m4.
> Does it make sense to require aclocal to run to compile xfstests?

I'm kind of curious why you need libtool to be able to cross-compile
packages.

I've been cross-compiling xfstests to armhf, ppcle, and arm64 without
any issues.   

See:   https://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/tree/build-all

Are you trying to build with shared libraries, or some such?

							- Ted

Re: xfstest and chromeos

[Mike Frysinger]

On Thu, Jan 26, 2017 at 5:45 PM, Theodore Ts'o wrote:
> On Mon, Jan 23, 2017 at 09:57:15AM -0800, Gwendal Grignou wrote:
>> https://chromium-review.googlesource.com/#/c/424191
>> - Chromeos needs libtool to (cross-)compile packages and put them at
>> the right place. I needed to move archived aclocal.m4 to acinclude.m4.
>> Does it make sense to require aclocal to run to compile xfstests?
>
> I'm kind of curious why you need libtool to be able to cross-compile
> packages.
>
> I've been cross-compiling xfstests to armhf, ppcle, and arm64 without
> any issues.
>
> See:   https://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/tree/build-all
>
> Are you trying to build with shared libraries, or some such?

it isn't about "needing" libtool, nor what we're attempting to do.  we
don't care if xfstools uses libtool or plain automake rules or
something else.  however, xfstests itself already utilizes `libtool`
to compile+link things, and indeed to also create libs like libtest.la
and libdmtest.la.

the host `libtool` is specifically generated with settings that target
the host system only.  just grep for your tuple in there (e.g. `grep
x86_64 /usr/bin/libtool`) to see.  for simpler builds, the differences
might go unnoticed as a happy accident.

by using LT_INIT in autoconf, it'll include a libtool script that
configure will create & use automatically tailored for the current
target.
-mike

Re: xfstest and chromeos

[Theodore Ts'o]

On Thu, Jan 26, 2017 at 10:40:39PM -1000, Mike Frysinger wrote:
> On Thu, Jan 26, 2017 at 5:45 PM, Theodore Ts'o wrote:
> > On Mon, Jan 23, 2017 at 09:57:15AM -0800, Gwendal Grignou wrote:
> >> https://chromium-review.googlesource.com/#/c/424191
> >> - Chromeos needs libtool to (cross-)compile packages and put them at
> >> the right place. I needed to move archived aclocal.m4 to acinclude.m4.
> >> Does it make sense to require aclocal to run to compile xfstests?
> >
> > I'm kind of curious why you need libtool to be able to cross-compile
> > packages.
> >
> > I've been cross-compiling xfstests to armhf, ppcle, and arm64 without
> > any issues.
> >
> > See:   https://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/tree/build-all
> >
> > Are you trying to build with shared libraries, or some such?
> 
> it isn't about "needing" libtool, nor what we're attempting to do.  we
> don't care if xfstools uses libtool or plain automake rules or
> something else.  however, xfstests itself already utilizes `libtool`
> to compile+link things, and indeed to also create libs like libtest.la
> and libdmtest.la.

Right, but I'm able to create libtool without any problems and messing
with aclocal.m4 and acinclude.m4, et. al.  I probably am relying on
libtoolize from the local Debian or Guubuntu build host, though.  Is
that the issue you're having with ChromeOS?  That you're using a
different version of the libtool/autoconf macros?

						- Ted

Re: xfstest and chromeos

[Mike Frysinger]

On Fri, Jan 27, 2017 at 6:37 AM, Theodore Ts'o wrote:
> On Thu, Jan 26, 2017 at 10:40:39PM -1000, Mike Frysinger wrote:
> > On Thu, Jan 26, 2017 at 5:45 PM, Theodore Ts'o wrote:
> > > On Mon, Jan 23, 2017 at 09:57:15AM -0800, Gwendal Grignou wrote:
> > >> https://chromium-review.googlesource.com/#/c/424191
> > >> - Chromeos needs libtool to (cross-)compile packages and put them at
> > >> the right place. I needed to move archived aclocal.m4 to acinclude.m4.
> > >> Does it make sense to require aclocal to run to compile xfstests?
> > >
> > > I'm kind of curious why you need libtool to be able to cross-compile
> > > packages.
> > >
> > > I've been cross-compiling xfstests to armhf, ppcle, and arm64 without
> > > any issues.
> > >
> > > See:   https://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/tree/build-all
> > >
> > > Are you trying to build with shared libraries, or some such?
> >
> > it isn't about "needing" libtool, nor what we're attempting to do.  we
> > don't care if xfstools uses libtool or plain automake rules or
> > something else.  however, xfstests itself already utilizes `libtool`
> > to compile+link things, and indeed to also create libs like libtest.la
> > and libdmtest.la.
>
> Right, but I'm able to create libtool without any problems and messing
> with aclocal.m4 and acinclude.m4, et. al.  I probably am relying on
> libtoolize from the local Debian or Guubuntu build host, though.  Is
> that the issue you're having with ChromeOS?  That you're using a
> different version of the libtool/autoconf macros?

if we wanted, we could do all this manually too, but i don't think it
makes sense to force every developer to learn autotools and how to set
up a local copy.  Gwendal's patch makes things "just work" w/out
having to learn anything new.  the overhead is, arguably, negligible.
-mike

Re: xfstest and chromeos

[Gwendal Grignou]

On Fri, Jan 27, 2017 at 10:02 AM, Mike Frysinger <vapier@chromium.org> wrote:
> On Fri, Jan 27, 2017 at 6:37 AM, Theodore Ts'o wrote:
>> On Thu, Jan 26, 2017 at 10:40:39PM -1000, Mike Frysinger wrote:
>> > On Thu, Jan 26, 2017 at 5:45 PM, Theodore Ts'o wrote:
>> > > On Mon, Jan 23, 2017 at 09:57:15AM -0800, Gwendal Grignou wrote:
>> > >> https://chromium-review.googlesource.com/#/c/424191
>> > >> - Chromeos needs libtool to (cross-)compile packages and put them at
>> > >> the right place. I needed to move archived aclocal.m4 to acinclude.m4.
>> > >> Does it make sense to require aclocal to run to compile xfstests?
>> > >
>> > > I'm kind of curious why you need libtool to be able to cross-compile
>> > > packages.
>> > >
>> > > I've been cross-compiling xfstests to armhf, ppcle, and arm64 without
>> > > any issues.
>> > >
>> > > See:   https://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/tree/build-all
>> > >
>> > > Are you trying to build with shared libraries, or some such?
>> >
>> > it isn't about "needing" libtool, nor what we're attempting to do.  we
>> > don't care if xfstools uses libtool or plain automake rules or
>> > something else.  however, xfstests itself already utilizes `libtool`
>> > to compile+link things, and indeed to also create libs like libtest.la
>> > and libdmtest.la.
>>
>> Right, but I'm able to create libtool without any problems and messing
>> with aclocal.m4 and acinclude.m4, et. al.  I probably am relying on
>> libtoolize from the local Debian or Guubuntu build host, though.  Is
>> that the issue you're having with ChromeOS?  That you're using a
>> different version of the libtool/autoconf macros?

Without the patch described in c/424191, emerging xfstest fails during
the install phase with:

>>> Install xfstests-9999 into /build/cave/tmp/portage/app-benchmarks/xfstests-9999/image/ category app-benchmarks
make -j1 install
Building include
...
Building ...
/usr/bin/gmake --no-print-directory Q=@ -C include install
gmake[1]: Nothing to be done for 'install'.
/usr/bin/gmake --no-print-directory Q=@ -C lib install
gmake[1]: Nothing to be done for 'install'.
/usr/bin/gmake --no-print-directory Q=@ -C ltp install
../install-sh -o gwendal -g portage -m 755 -d /var/lib/xfstests/ltp
/usr/bin/libtool --quiet --mode=install ../install-sh -o gwendal -g
portage -m 755 doio fsstress fsx growfiles iogen aio-stress
/var/lib/xfstests/ltp
Usage: /usr/bin/libtool [OPTION]... [MODE-ARG]...
Try 'libtool --help' for more information.
libtool:   error:'/var/lib/xfstests/ltp' is not a directory


The problem is in include/builddefs.in with PKG_LIB_DIR     =
/var/lib/@pkg_name@
As we cross compile into /build/<board>/..., we don't want to put data
on the local machine in /var.

Just prefixing  PKG_LIB_DIR with $(DESTDIR) is not good enough,
Although DESTDIR is defined it points to the directory is compiled:
/build/cave/tmp/portage/app-benchmarks/xfstests-9999/image/.
we are still using the default libtool in (/usr/bin/libtool).
By using LT_INIT, we build a libtool tailored to the environment and use it.
Looking at the difference between local libtool.sh and the default
one, the default settings are not correct: wrong compiler, not using
clang, different compiler flags,...

Gwendal.


>
> if we wanted, we could do all this manually too, but i don't think it
> makes sense to force every developer to learn autotools and how to set
> up a local copy.  Gwendal's patch makes things "just work" w/out
> having to learn anything new.  the overhead is, arguably, negligible.
> -mike

Re: xfstest and chromeos

[Theodore Ts'o]

On Fri, Jan 27, 2017 at 08:02:30AM -1000, Mike Frysinger wrote:
> > Right, but I'm able to create libtool without any problems and messing
> > with aclocal.m4 and acinclude.m4, et. al.  I probably am relying on
> > libtoolize from the local Debian or Guubuntu build host, though.  Is
> > that the issue you're having with ChromeOS?  That you're using a
> > different version of the libtool/autoconf macros?
> 
> if we wanted, we could do all this manually too, but i don't think it
> makes sense to force every developer to learn autotools and how to set
> up a local copy.  Gwendal's patch makes things "just work" w/out
> having to learn anything new.  the overhead is, arguably, negligible.

Most developers are forced to learn autotools, since it's generally
considered good practice by people who use libtools at all to not
check in build artifacts into the git tree.  And developers can also
check out my xfstests-bld git tree if they wanted something completely
turnkey.   Not that it's all that hard:

if test -z "$SKIP_XFSTESTS" ; then
    build_start "xfstests"
    (cd xfstests-dev; \
     ver=$(git describe --always --dirty); echo "xfstests	$ver ($(git log -1 --pretty=%cD))" > ../xfstests.ver ; \
     autoheader ; autoconf ; \
     CFLAGS="$LCF -I$DESTDIR/include -fno-stack-protector" \
     CPPFLAGS="-I$DESTDIR/include" \
     LDFLAGS="$LLF -static -L$DESTDIR/lib" \
     LIBS=-lpthread \
     ./configure $cross ; \
     make $J LLDFLAGS=$EXEC_LLDFLAGS PLDLIBS="$android_compat" \
	  LIBTOOL="/usr/bin/libtool --tag=CC" BUILD_VERBOSE=1)
fi

I personally don't mind checking in generated build artificats such as
"configure" into the git tree --- see how I do things with e2fsprogs
--- but I also have the good taste to absolutely consider libtool
and automake as abominations from hell.  :-)

If you want to try to convince Darrin and From: Theodore Ts'o <tytso@mit.edu>
To: Mike Frysinger <vapier@chromium.org>
Cc: Gwendal Grignou <gwendal@chromium.org>, fstests@vger.kernel.org
Bcc: tytso@mit.edu
Subject: Re: xfstest and chromeos
Reply-To: 
In-Reply-To: <CAAbOScmd_j-HgZj929fbWzs3JjLkB817dJ_A-ghLyD7kHjh7GA@mail.gmail.com>

On Fri, Jan 27, 2017 at 08:02:30AM -1000, Mike Frysinger wrote:
> > Right, but I'm able to create libtool without any problems and messing
> > with aclocal.m4 and acinclude.m4, et. al.  I probably am relying on
> > libtoolize from the local Debian or Guubuntu build host, though.  Is
> > that the issue you're having with ChromeOS?  That you're using a
> > different version of the libtool/autoconf macros?
> 
> if we wanted, we could do all this manually too, but i don't think it
> makes sense to force every developer to learn autotools and how to set
> up a local copy.  Gwendal's patch makes things "just work" w/out
> having to learn anything new.  the overhead is, arguably, negligible.

Most developers are forced to learn autotools, since it's generally
considered good practice by people who use libtools at all to not
check in build artifacts into the git tree.  And developers can also
check out my xfstests-bld git tree if they wanted something completely
turnkey.   Not that it's all that hard:

if test -z "$SKIP_XFSTESTS" ; then
    build_start "xfstests"
    (cd xfstests-dev; \
     autoheader ; autoconf ; \
     CFLAGS="$LCF -I$DESTDIR/include -fno-stack-protector" \
     CPPFLAGS="-I$DESTDIR/include" \
     LDFLAGS="$LLF -static -L$DESTDIR/lib" \
     LIBS=-lpthread \
     ./configure $cross ; \
     make $J LLDFLAGS=$EXEC_LLDFLAGS PLDLIBS="$android_compat" \
	  LIBTOOL="/usr/bin/libtool --tag=CC" BUILD_VERBOSE=1)
fi

At least for xfstests, "autoheader ; autoconf ; configure ; make"
should be sufficient.  It's really not that hard.  And for cross
compilations, it's just:

    cross="--build=$(config.guess) --host=$CROSS_COMPILE"

and then using "configure $cross" instead.  I guess I'm not
understanding why it's considered tricky.

						- Ted

P.S.  I personally don't mind checking in generated build artificats
such as "configure" into the git tree --- see how I do things with
e2fsprogs --- but I also have the good taste to absolutely consider
libtool and automake as abominations from hell.  :-)

Especially given that xfstests and xfsprogs only need to work on Linux
these days (we have enough other Linixisms that it would be hopeless
to get it to work on, say, BSD or Open Solaris), in my opinion the
right thing to do would be to completely ditch automake and libtool;
e2fsprogs autoconf mainly for things like "--enable-elf-shlibs" and
using Makefile fragments included as part of the configure process is
plenty to handle shared libraries.  But that's up to Darrick and Eryu
to decide.  :-)

[PATCH] Make SELinux protection conditional

[Gwendal Grignou]

When selinux utilities are present, xfstests add options to help circumvent
selinux protection. However, on Chrome OS, it leads to the opposite effect,
when it prevents mount to succeed.

BUG=chromium:669641
TEST=xfstest test ext4/001 completes where it use to display:
common/rc: could not mount /dev/loop29 on /usr/local/autotest/tmp/xfstests_TEST

Signed-off-by: Gwendal Grignou <gwendal@chromium.org>
---
 common/config | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/common/config b/common/config
index fb60216c..0c7335ad 100644
--- a/common/config
+++ b/common/config
@@ -35,6 +35,7 @@
 # RMT_TAPE_DEV -    the remote tape device for the xfsdump tests
 # RMT_IRIXTAPE_DEV- the IRIX remote tape device for the xfsdump tests
 # RMT_TAPE_USER -   remote user for tape device
+# SELINUX_MOUNT_OPTIONS - Options to use when SELinux is enabled.
 #
 # - These can be added to $HOST_CONFIG_DIR (witch default to ./config)
 #   below or a separate local configuration file can be used (using
@@ -262,7 +263,7 @@ esac
 # So, mount with a context, and they won't be created
 # # nfs_t is a "liberal" context so we can use it.
 if [ -x /usr/sbin/selinuxenabled ] && /usr/sbin/selinuxenabled; then
-	SELINUX_MOUNT_OPTIONS="-o context=system_u:object_r:nfs_t:s0"
+	: ${SELINUX_MOUNT_OPTIONS:="-o context=system_u:object_r:nfs_t:s0"}
 	export SELINUX_MOUNT_OPTIONS
 fi
 
-- 
2.12.0.rc1.440.g5b76565f74-goog

Re: xfstest and chromeos

[Mike Frysinger]

On Fri, Jan 27, 2017 at 2:10 PM, Theodore Ts'o wrote:
> On Fri, Jan 27, 2017 at 08:02:30AM -1000, Mike Frysinger wrote:
>> > Right, but I'm able to create libtool without any problems and messing
>> > with aclocal.m4 and acinclude.m4, et. al.  I probably am relying on
>> > libtoolize from the local Debian or Guubuntu build host, though.  Is
>> > that the issue you're having with ChromeOS?  That you're using a
>> > different version of the libtool/autoconf macros?
>>
>> if we wanted, we could do all this manually too, but i don't think it
>> makes sense to force every developer to learn autotools and how to set
>> up a local copy.  Gwendal's patch makes things "just work" w/out
>> having to learn anything new.  the overhead is, arguably, negligible.
>
> Most developers are forced to learn autotools, since it's generally
> considered good practice by people who use libtools at all to not
> check in build artifacts into the git tree.

i have no idea what you're trying to argue here.
(1) xfstests already uses autotools.  disliking autotools doesn't change that.
(2) developers don't need to learn anything new.  you always run
`autoreconf -fi`.  FIN.  there is no need to figure out which specific
sub-tools to run (autoheader, autoconf, libtoolize, whatever), or in
what order.  if you're worried about people not knowing how to run
this one command, then we can add an "autogen.sh" script to the root
to run `autoreconf -fi` for you.  that's a semi-common practice in
autotool based projects.
(3) no one (on the CrOS side) is arguing for committing generated
artifacts into git.  that's a terrible practice regardless of
autotools.

> Especially given that xfstests and xfsprogs only need to work on Linux
> these days (we have enough other Linixisms that it would be hopeless
> to get it to work on, say, BSD or Open Solaris), in my opinion the
> right thing to do would be to completely ditch automake and libtool;
> e2fsprogs autoconf mainly for things like "--enable-elf-shlibs" and
> using Makefile fragments included as part of the configure process is
> plenty to handle shared libraries.  But that's up to Darrick and Eryu
> to decide.  :-)

so we trade one pile of standard scripts/hacks that the rest of the
world uses for a pile of non-standard/unique scripts/hacks that only
one project uses ?  that doesn't sound like an improvement, it just
sounds like "this is the pile i know".  except in the case of
xfstests, it isn't something we know.
-mike

[PATCH] Code to allow cros-compilation on chromeOS

[Gwendal Grignou]

- Request LIBTOOL to be used
- Set topbuildir based on a Makefile variable to call libtool
- Use /usr/local instead of /var for xfstest final location
- Move macros from aclocal.m4 to acinclude.m4, aclocal.m4 is autogenerated.

TEST=Compile in chromeOS chroot.
TEST=Using a snippet of build-all at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_tytso_xfstests-2Dbld&d=DwIBAg&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=-yMrTV4jriXR7ieyzzjV-QgHBD0UDw8ixoR77aMeAHE&m=VdzD4uuj5Zks7OsGcUBKCHDDJig5g9qTLjbRfNI6wd0&s=6l8TxAqq_8ZTrjDLZIBViElUbXQKiiDmRP5ZUFGC44c&e= 
After adding config.sub and config.guess with:
cp /usr/share/misc/config.sub /usr/share/misc/config.guess
Verified that compilation of xfstest works:
ver=$(git describe --always --dirty); echo "xfstests       $ver ($(git log -1 --pretty=%cD))" > ../xfstests.ver ;
autoheader ; autoconf ;
CFLAGS="$LCF -I$DESTDIR/include -fno-stack-protector"
CPPFLAGS="-I$DESTDIR/include"
LDFLAGS="$LLF -static -L$DESTDIR/lib"
LIBS=-lpthread
./configure
make -j40 LLDFLAGS=$EXEC_LLDFLAGS LIBTOOL="/usr/bin/libtool --tag=CC" BUILD_VERBOSE=1

Signed-off-by: Gwendal Grignou <gwendal@chromium.org>
---
 acinclude.m4         | 30 ++++++++++++++++++++++++++++++
 aclocal.m4           | 50 --------------------------------------------------
 configure.ac         |  3 ++-
 include/builddefs.in |  7 +++++++
 4 files changed, 39 insertions(+), 51 deletions(-)
 create mode 100644 acinclude.m4
 delete mode 100644 aclocal.m4

diff --git a/acinclude.m4 b/acinclude.m4
new file mode 100644
index 00000000..666f4069
--- /dev/null
+++ b/acinclude.m4
@@ -0,0 +1,30 @@
+dnl Copyright (C) 2016 Free Software Foundation, Inc.
+dnl This file is free software; the Free Software Foundation
+dnl gives unlimited permission to copy and/or distribute it,
+dnl with or without modifications, as long as this notice is preserved.
+AC_DEFUN([AC_PACKAGE_WANT_LINUX_FIEMAP_H],
+  [ AC_CHECK_HEADERS([linux/fiemap.h], [ have_fiemap=true ], [ have_fiemap=false ])
+    AC_SUBST(have_fiemap)
+  ])
+
+AC_DEFUN([AC_PACKAGE_WANT_LINUX_PRCTL_H],
+  [ AC_CHECK_HEADERS([sys/prctl.h], [ have_prctl=true ], [ have_prctl=false ])
+    AC_SUBST(have_prctl)
+  ])
+
+AC_DEFUN([AC_PACKAGE_WANT_LINUX_FS_H],
+  [ AC_CHECK_HEADER([linux/fs.h])
+  ])
+
+AC_DEFUN([AC_PACKAGE_WANT_FALLOCATE],
+  [ AC_MSG_CHECKING([for fallocate])
+    AC_TRY_LINK([
+#define _GNU_SOURCE
+#define _FILE_OFFSET_BITS 64
+#include <fcntl.h>
+#include <linux/falloc.h> ],
+      [ fallocate(0, 0, 0, 0); ],
+      [ have_fallocate=true; AC_MSG_RESULT(yes) ],
+      [ have_fallocate=false; AC_MSG_RESULT(no) ])
+    AC_SUBST(have_fallocate)
+  ])
diff --git a/aclocal.m4 b/aclocal.m4
deleted file mode 100644
index f3412e19..00000000
--- a/aclocal.m4
+++ /dev/null
@@ -1,50 +0,0 @@
-# generated automatically by aclocal 1.11 -*- Autoconf -*-
-
-# Copyright (C) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004,
-# 2005, 2006, 2007, 2008, 2009  Free Software Foundation, Inc.
-# This file is free software; the Free Software Foundation
-# gives unlimited permission to copy and/or distribute it,
-# with or without modifications, as long as this notice is preserved.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY, to the extent permitted by law; without
-# even the implied warranty of MERCHANTABILITY or FITNESS FOR A
-# PARTICULAR PURPOSE.
-
-AC_DEFUN([AC_PACKAGE_WANT_LINUX_FIEMAP_H],
-  [ AC_CHECK_HEADERS([linux/fiemap.h], [ have_fiemap=true ], [ have_fiemap=false ])
-    AC_SUBST(have_fiemap)
-  ])
-
-AC_DEFUN([AC_PACKAGE_WANT_LINUX_PRCTL_H],
-  [ AC_CHECK_HEADERS([sys/prctl.h], [ have_prctl=true ], [ have_prctl=false ])
-    AC_SUBST(have_prctl)
-  ])
-
-AC_DEFUN([AC_PACKAGE_WANT_LINUX_FS_H],
-  [ AC_CHECK_HEADER([linux/fs.h])
-  ])
-
-AC_DEFUN([AC_PACKAGE_WANT_FALLOCATE],
-  [ AC_MSG_CHECKING([for fallocate])
-    AC_TRY_LINK([
-#define _GNU_SOURCE
-#define _FILE_OFFSET_BITS 64
-#include <fcntl.h>
-#include <linux/falloc.h> ],
-      [ fallocate(0, 0, 0, 0); ],
-      [ have_fallocate=true; AC_MSG_RESULT(yes) ],
-      [ have_fallocate=false; AC_MSG_RESULT(no) ])
-    AC_SUBST(have_fallocate)
-  ])
-m4_include([m4/multilib.m4])
-m4_include([m4/package_acldev.m4])
-m4_include([m4/package_aiodev.m4])
-m4_include([m4/package_attrdev.m4])
-m4_include([m4/package_dmapidev.m4])
-m4_include([m4/package_gdbmdev.m4])
-m4_include([m4/package_globals.m4])
-m4_include([m4/package_ssldev.m4])
-m4_include([m4/package_utilies.m4])
-m4_include([m4/package_uuiddev.m4])
-m4_include([m4/package_xfslibs.m4])
diff --git a/configure.ac b/configure.ac
index fa48d2f8..d8393b1f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,5 +1,6 @@
 AC_INIT([xfstests], [1.1.1])
-AC_PREREQ(2.50)
+AC_CONFIG_MACRO_DIRS([m4])
+LT_INIT
 AC_CONFIG_SRCDIR([src/xfsctl.c])
 AC_PACKAGE_GLOBALS(xfstests)
 AC_PACKAGE_UTILITIES(xfstests)
diff --git a/include/builddefs.in b/include/builddefs.in
index 24f838f5..48d35311 100644
--- a/include/builddefs.in
+++ b/include/builddefs.in
@@ -7,6 +7,8 @@
 ifndef _BUILDDEFS_INCLUDED_
 _BUILDDEFS_INCLUDED_ = 1
 
+top_builddir=$(TOPDIR)
+
 DEBUG = @debug_build@
 OPTIMIZER = @opt_build@
 MALLOCLIB = @malloc_lib@
@@ -35,7 +37,12 @@ PKG_DISTRIBUTION= @pkg_distribution@
 PKG_SBIN_DIR    = @sbindir@
 # A bit of a hack; by rights only state should probably go here
 # But for now ...
+ifdef DESTDIR
+PKG_LIB_DIR     = $(DESTDIR)/usr/local/@pkg_name@
+else
 PKG_LIB_DIR     = /var/lib/@pkg_name@
+endif
+
 
 CC              = @cc@
 AWK             = @awk@
-- 
2.12.0.367.g23dc2f6d3c-goog

--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  https://urldefense.proofpoint.com/v2/url?u=http-3A__vger.kernel.org_majordomo-2Dinfo.html&d=DwIBAg&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=-yMrTV4jriXR7ieyzzjV-QgHBD0UDw8ixoR77aMeAHE&m=VdzD4uuj5Zks7OsGcUBKCHDDJig5g9qTLjbRfNI6wd0&s=BJTV13qlnHCdnyn5nHqRq2kdxeX7INMAvsOFAsOKAiw&e= 

Re: [PATCH] Code to allow cros-compilation on chromeOS

[Eryu Guan]

On Mon, Mar 20, 2017 at 02:20:45PM -0700, Gwendal Grignou wrote:
> - Request LIBTOOL to be used
> - Set topbuildir based on a Makefile variable to call libtool
> - Use /usr/local instead of /var for xfstest final location
> - Move macros from aclocal.m4 to acinclude.m4, aclocal.m4 is autogenerated.
> 
> TEST=Compile in chromeOS chroot.
> TEST=Using a snippet of build-all at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_tytso_xfstests-2Dbld&d=DwIBAg&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=-yMrTV4jriXR7ieyzzjV-QgHBD0UDw8ixoR77aMeAHE&m=CfsgeRzY4RIPfAUQqWlSEu0A-gXQ3jhYRYWh8njYK1Q&s=wJL4qnWAA1n2ek4-DxvFWIGPw8HbXGuN7Pl2SSpK6No&e= 
> After adding config.sub and config.guess with:
> cp /usr/share/misc/config.sub /usr/share/misc/config.guess
> Verified that compilation of xfstest works:
> ver=$(git describe --always --dirty); echo "xfstests       $ver ($(git log -1 --pretty=%cD))" > ../xfstests.ver ;
> autoheader ; autoconf ;
> CFLAGS="$LCF -I$DESTDIR/include -fno-stack-protector"
> CPPFLAGS="-I$DESTDIR/include"
> LDFLAGS="$LLF -static -L$DESTDIR/lib"
> LIBS=-lpthread
> ./configure
> make -j40 LLDFLAGS=$EXEC_LLDFLAGS LIBTOOL="/usr/bin/libtool --tag=CC" BUILD_VERBOSE=1
> 
> Signed-off-by: Gwendal Grignou <gwendal@chromium.org>

I'm not an autotools expert, and I really appreciate if someone could
help review build/packaging patches, thanks in advance!

But I did hit errors when "make" on default fedora25 and RHEL7 hosts
(which worked fine prior to this patch), I tried to "fix" them but
failed.

[root@bootp-73-5-211 fstests]# make
autoheader
autoconf
configure.ac:2: error: possibly undefined macro: AC_CONFIG_MACRO_DIRS
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.ac:5: error: possibly undefined macro: AC_PACKAGE_GLOBALS
configure.ac:6: error: possibly undefined macro: AC_PACKAGE_UTILITIES
configure.ac:43: error: possibly undefined macro: AC_PACKAGE_NEED_UUIDCOMPARE
...

Seems AC_CONFIG_MACRO_DIRS is only available starting from autoconf
2.70, but f25 and rhel7 ship autoconf 2.69 by default.

After changing AC_CONFIG_MACRO_DIRS to AC_CONFIG_MACRO_DIR, above error
is gone, but other errors remain.

[root@bootp-73-5-211 fstests]# make
autoheader
autoconf
configure.ac:5: error: possibly undefined macro: AC_PACKAGE_GLOBALS
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.ac:6: error: possibly undefined macro: AC_PACKAGE_UTILITIES
configure.ac:43: error: possibly undefined macro: AC_PACKAGE_NEED_UUIDCOMPARE
...

These macros are all defined in m4/ directory, so seems .m4 files are
not included correctly, because aclocal.m4 was not generated
automatically.

After doing this change to Makefile, I got 'configure' file generated,
but configure process failed again, because of missing config.sub file.

diff --git a/Makefile b/Makefile
index 30d8747..aa8db92 100644
--- a/Makefile
+++ b/Makefile
@@ -75,7 +75,7 @@ else
 clean:  # if configure hasn't run, nothing to clean
 endif
 
-configure: configure.ac
+configure: aclocal.m4 configure.ac
        autoheader
        autoconf
 
@@ -84,7 +84,7 @@ include/builddefs include/config.h: configure
                 --libexecdir=/usr/lib
 
 aclocal.m4::
-       aclocal --acdir=`pwd`/m4 --output=$@
+       aclocal -I`pwd`/m4 --output=$@
 
 depend: include/builddefs $(addsuffix -depend,$(SUBDIRS))
 

[root@bootp-73-5-211 fstests]# make
aclocal -I`pwd`/m4 --output=aclocal.m4
autoheader
autoconf
./configure \   
                --libexecdir=/usr/lib
configure: error: cannot run /bin/sh ./config.sub
Makefile:83: recipe for target 'include/builddefs' failed
make: *** [include/builddefs] Error 127


I noticed that you copied config.sub and config.guess manually to
xfstests directory, but this would break existing build process which
doesn't expect to do that. And forcing users to do manually copy files
to meet build requirements doesn't seem right to me either.

Can you please take a look at these issues?

Thanks,
Eryu
--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  https://urldefense.proofpoint.com/v2/url?u=http-3A__vger.kernel.org_majordomo-2Dinfo.html&d=DwIBAg&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=-yMrTV4jriXR7ieyzzjV-QgHBD0UDw8ixoR77aMeAHE&m=CfsgeRzY4RIPfAUQqWlSEu0A-gXQ3jhYRYWh8njYK1Q&s=59ruBSBPZdkktlllER_f54rd80OfRe1pqnP1pe02x0Y&e= 

Re: [PATCH] Code to allow cros-compilation on chromeOS

[Gwendal Grignou]

On Thu, Mar 23, 2017 at 9:23 PM, Eryu Guan <eguan@redhat.com> wrote:
> On Mon, Mar 20, 2017 at 02:20:45PM -0700, Gwendal Grignou wrote:
>> - Request LIBTOOL to be used
>> - Set topbuildir based on a Makefile variable to call libtool
>> - Use /usr/local instead of /var for xfstest final location
>> - Move macros from aclocal.m4 to acinclude.m4, aclocal.m4 is autogenerated.
>>
>> TEST=Compile in chromeOS chroot.
>> TEST=Using a snippet of build-all at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_tytso_xfstests-2Dbld&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=-yMrTV4jriXR7ieyzzjV-QgHBD0UDw8ixoR77aMeAHE&m=cWDBHwtFLpBlLHXoeZBA9cB2fFFnzyQQHiVoJWzLrKQ&s=Vl5D6Bv5s825YH2xomziAfhfhQC318syauikRsaz3i4&e= 
>> After adding config.sub and config.guess with:
>> cp /usr/share/misc/config.sub /usr/share/misc/config.guess
>> Verified that compilation of xfstest works:
>> ver=$(git describe --always --dirty); echo "xfstests       $ver ($(git log -1 --pretty=%cD))" > ../xfstests.ver ;
>> autoheader ; autoconf ;
>> CFLAGS="$LCF -I$DESTDIR/include -fno-stack-protector"
>> CPPFLAGS="-I$DESTDIR/include"
>> LDFLAGS="$LLF -static -L$DESTDIR/lib"
>> LIBS=-lpthread
>> ./configure
>> make -j40 LLDFLAGS=$EXEC_LLDFLAGS LIBTOOL="/usr/bin/libtool --tag=CC" BUILD_VERBOSE=1
>>
>> Signed-off-by: Gwendal Grignou <gwendal@chromium.org>
>
> I'm not an autotools expert, and I really appreciate if someone could
> help review build/packaging patches, thanks in advance!
>
> But I did hit errors when "make" on default fedora25 and RHEL7 hosts
> (which worked fine prior to this patch), I tried to "fix" them but
> failed.
>
> [root@bootp-73-5-211 fstests]# make
> autoheader
> autoconf
> configure.ac:2: error: possibly undefined macro: AC_CONFIG_MACRO_DIRS
>       If this token and others are legitimate, please use m4_pattern_allow.
>       See the Autoconf documentation.
> configure.ac:5: error: possibly undefined macro: AC_PACKAGE_GLOBALS
> configure.ac:6: error: possibly undefined macro: AC_PACKAGE_UTILITIES
> configure.ac:43: error: possibly undefined macro: AC_PACKAGE_NEED_UUIDCOMPARE
> ...
>
> Seems AC_CONFIG_MACRO_DIRS is only available starting from autoconf
> 2.70, but f25 and rhel7 ship autoconf 2.69 by default.
Fixed.
>
> After changing AC_CONFIG_MACRO_DIRS to AC_CONFIG_MACRO_DIR, above error
> is gone, but other errors remain.
>
> [root@bootp-73-5-211 fstests]# make
> autoheader
> autoconf
> configure.ac:5: error: possibly undefined macro: AC_PACKAGE_GLOBALS
>       If this token and others are legitimate, please use m4_pattern_allow.
>       See the Autoconf documentation.
> configure.ac:6: error: possibly undefined macro: AC_PACKAGE_UTILITIES
> configure.ac:43: error: possibly undefined macro: AC_PACKAGE_NEED_UUIDCOMPARE
> ...
>
> These macros are all defined in m4/ directory, so seems .m4 files are
> not included correctly, because aclocal.m4 was not generated
> automatically.
>
> After doing this change to Makefile, I got 'configure' file generated,
> but configure process failed again, because of missing config.sub file.
>
> diff --git a/Makefile b/Makefile
> index 30d8747..aa8db92 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -75,7 +75,7 @@ else
>  clean:  # if configure hasn't run, nothing to clean
>  endif
>
> -configure: configure.ac
> +configure: aclocal.m4 configure.ac
>         autoheader
>         autoconf
>
> @@ -84,7 +84,7 @@ include/builddefs include/config.h: configure
>                  --libexecdir=/usr/lib
>
>  aclocal.m4::
> -       aclocal --acdir=`pwd`/m4 --output=$@
> +       aclocal -I`pwd`/m4 --output=$@
>
>  depend: include/builddefs $(addsuffix -depend,$(SUBDIRS))
>
>
> [root@bootp-73-5-211 fstests]# make
> aclocal -I`pwd`/m4 --output=aclocal.m4
> autoheader
> autoconf
> ./configure \
>                 --libexecdir=/usr/lib
> configure: error: cannot run /bin/sh ./config.sub
> Makefile:83: recipe for target 'include/builddefs' failed
> make: *** [include/builddefs] Error 127
>
>
> I noticed that you copied config.sub and config.guess manually to
> xfstests directory, but this would break existing build process which
> doesn't expect to do that. And forcing users to do manually copy files
> to meet build requirements doesn't seem right to me either.
>
> Can you please take a look at these issues?
Yes, I did not do enough changes. I replaced autoheader and autoconf
in Makefile with autoreconf and libtoolize (similar to how gentoo
would compile xfstests) and it produces a valid ./configure.
The drawback is install_sh is overwritten by libtoolize, to libtool
has now to be used.

I already sent a patch, but in response to a bounced email, sorry.

Gwendal.
>
> Thanks,
> Eryu
--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  https://urldefense.proofpoint.com/v2/url?u=http-3A__vger.kernel.org_majordomo-2Dinfo.html&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=-yMrTV4jriXR7ieyzzjV-QgHBD0UDw8ixoR77aMeAHE&m=cWDBHwtFLpBlLHXoeZBA9cB2fFFnzyQQHiVoJWzLrKQ&s=wtkAuBYysOPpJn7-du-mZTQXHR-jotTqEgoe8hz1mSA&e=