[Buildroot] [Bug 9386] New: ubinize fails with or without custom config

[Buildroot] [Bug 9386] New: ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

            Bug ID: 9386
           Summary: ubinize fails with or without custom config
           Product: buildroot
           Version: 2016.08
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: Other
          Assignee: unassigned at buildroot.uclibc.org
          Reporter: yurovsky at gmail.com
                CC: buildroot at uclibc.org
  Target Milestone: ---

Created attachment 6781
  --> https://bugs.busybox.net/attachment.cgi?id=6781&action=edit
my experimental fix or workaround

I'm unable to build a UBI image regardless of whether I use the provided
ubinize.cfg or the new custom config option.  In case it's useful, the build
host is Fedora 24.

Sanitized output:

output/foo/build/_device_table.txt'
iniparser: cannot open output/foo/build/ubinize.cfg
ubinize: error!: cannot load the input ini file "output/build/ubinize.cfg"

I think that the issue is that ubinize.cfg is being copied and edited under
fakeroot and the permissions mask is wrong.  As an experiment I added another
routine that's called directly (rather than printed into _fakeroot.sh) and
moved the ubi.mk _CMD to my new _SETUP_CMD and that resolves the problem.

I'm not sure if something like this is a reasonable solution and I don't
understand why copying/sed/etc is done under fakeroot to begin with:

--- a/buildroot/fs/common.mk
+++ b/buildroot/fs/common.mk
@@ -95,6 +95,7 @@
        $$(foreach s,$$(call qstrip,$$(BR2_ROOTFS_POST_FAKEROOT_SCRIPT)),\
                echo "echo '$$(TERM_BOLD)>>>   Executing fakeroot script
$$(s)$$(TERM_RESET)'" >> $$(FAKEROOT_SCRIPT); \
                echo $$(s) $$(TARGET_DIR) $$(BR2_ROOTFS_POST_SCRIPT_ARGS) >>
$$(FAKEROOT_SCRIPT)$$(sep))
+       $$(call $$(ROOTFS_$(2)_SETUP_CMD))
        $$(call PRINTF,$$(ROOTFS_$(2)_CMD)) >> $$(FAKEROOT_SCRIPT)
        chmod a+x $$(FAKEROOT_SCRIPT)
        PATH=$$(BR_PATH) $$(HOST_DIR)/usr/bin/fakeroot -- $$(FAKEROOT_SCRIPT)
--- a/buildroot/fs/ubifs/ubi.mk
+++ b/buildroot/fs/ubifs/ubi.mk
@@ -20,7 +20,7 @@
 UBINIZE_CONFIG_FILE_PATH = fs/ubifs/ubinize.cfg
 endif

-define ROOTFS_UBI_CMD
+define ROOTFS_UBI_SETUP_CMD
        $(INSTALL) -m 0644 $(UBINIZE_CONFIG_FILE_PATH) $(BUILD_DIR)/ubinize.cfg
        $(SED) 's;BR2_ROOTFS_UBIFS_PATH;$@fs;' $(BUILD_DIR)/ubinize.cfg
        $(HOST_DIR)/usr/sbin/ubinize -o $@ $(UBI_UBINIZE_OPTS)
$(BUILD_DIR)/ubinize.cfg

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

Yann E. MORIN <yann.morin.1998@free.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |yann.morin.1998 at free.fr

--- Comment #1 from Yann E. MORIN <yann.morin.1998@free.fr> ---
Andrey,

Thanks for your report.

I've tried to run atmel_sama5d4_xplained_defconfig that uses ubinize,
but was not able to reproduce your issue.

Note that we already had a very similar issue a while back, but it was
marked as "worksforme" and closed because we had no precise explanations
on how to reproduce:
    https://bugs.busybox.net/show_bug.cgi?id=9096

Since you have that issue now, would you care to provide a defconfig
file that exhibits the problem, so we can try to reproduce and
invetigate further, please?

Regards,
Yann E. MORIN.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

--- Comment #2 from Andrey Yurovsky <yurovsky@gmail.com> ---
Created attachment 6786
  --> https://bugs.busybox.net/attachment.cgi?id=6786&action=edit
approximate config

I've attached an approximate defconfig (sanitized to remove paths that have to
do with my product specifically, sorry for the inconvenience).  The SoC is an
Atmel AT91SAM9G20 so it's similar to the config you're testing with.  One
difference is there's both a CPIO/initramfs and a UBI image being generated but
hopefully that's not important.

I'm building with atmel_sama5d4_xplained_defconfig to see if I can reproduce
the problem, sorry for not doing that first.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

--- Comment #3 from Yann E. MORIN <yann.morin.1998@free.fr> ---
Andrey,

There's not much we can do with this config file: all path are
pointing to non-existing files, so the build fails horribly...

Let's see if you can reproduce with the sama5d4 board.

> sorry for not doing that first

No problem! ;-)

Regards,
Yann E. MORIN.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

--- Comment #4 from Andrey Yurovsky <yurovsky@gmail.com> ---
Ok, I've reproduced it with your defconfig and I've left all paths in here.  To
build I just ran:

make atmel_sama5d4_xplained_defconfig
make

And in the end the output is:

>>>   Generating root filesystem image rootfs.ubi
rm -f /home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs
rm -f
/home/ayurovsky/dev/build-system/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM
rm -f /home/ayurovsky/dev/build-system/buildroot/output/build/_users_table.txt
echo '#!/bin/sh' >
/home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs
echo "set -e" >>
/home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs
echo "chown -h -R 0:0 /home/ayurovsky/dev/build-system/buildroot/output/target"
>> /home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs
printf '   \n' >>
/home/ayurovsky/dev/build-system/buildroot/output/build/_users_table.txt
PATH="/home/ayurovsky/dev/build-system/buildroot/output/host/bin:/home/ayurovsky/dev/build-system/buildroot/output/host/sbin:/home/ayurovsky/dev/build-system/buildroot/output/host/usr/bin:/home/ayurovsky/dev/build-system/buildroot/output/host/usr/sbin:/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/ayurovsky/.local/bin:/home/ayurovsky/bin"
/home/ayurovsky/dev/build-system/buildroot/support/scripts/mkusers
/home/ayurovsky/dev/build-system/buildroot/output/build/_users_table.txt
/home/ayurovsky/dev/build-system/buildroot/output/target >>
/home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs
cat system/device_table.txt >
/home/ayurovsky/dev/build-system/buildroot/output/build/_device_table.txt
printf '        /bin/busybox                     f 4755 0  0 - - - - -\n
/dev/console c 622 0 0 5 1 - - -\n\n' >>
/home/ayurovsky/dev/build-system/buildroot/output/build/_device_table.txt
echo "/home/ayurovsky/dev/build-system/buildroot/output/host/usr/bin/makedevs
-d /home/ayurovsky/dev/build-system/buildroot/output/build/_device_table.txt
/home/ayurovsky/dev/build-system/buildroot/output/target" >>
/home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs
printf '        /usr/bin/install -m 0644 fs/ubifs/ubinize.cfg
/home/ayurovsky/dev/build-system/buildroot/output/build/ubinize.cfg\n    
/usr/bin/sed -i -e
'\''s;BR2_ROOTFS_UBIFS_PATH;/home/ayurovsky/dev/build-system/buildroot/output/images/rootfs.ubifs;'\''
/home/ayurovsky/dev/build-system/buildroot/output/build/ubinize.cfg\n
/home/ayurovsky/dev/build-system/buildroot/output/host/usr/sbin/ubinize -o
/home/ayurovsky/dev/build-system/buildroot/output/images/rootfs.ubi -m 0x1000
-p 0x40000 
/home/ayurovsky/dev/build-system/buildroot/output/build/ubinize.cfg\n      rm
/home/ayurovsky/dev/build-system/buildroot/output/build/ubinize.cfg\n' >>
/home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs
chmod a+x /home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs
PATH="/home/ayurovsky/dev/build-system/buildroot/output/host/bin:/home/ayurovsky/dev/build-system/buildroot/output/host/sbin:/home/ayurovsky/dev/build-system/buildroot/output/host/usr/bin:/home/ayurovsky/dev/build-system/buildroot/output/host/usr/sbin:/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/ayurovsky/.local/bin:/home/ayurovsky/bin"
/home/ayurovsky/dev/build-system/buildroot/output/host/usr/bin/fakeroot --
/home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs
rootdir=/home/ayurovsky/dev/build-system/buildroot/output/target
table='/home/ayurovsky/dev/build-system/buildroot/output/build/_device_table.txt'
iniparser: cannot open
/home/ayurovsky/dev/build-system/buildroot/output/build/ubinize.cfg
ubinize: error!: cannot load the input ini file
"/home/ayurovsky/dev/build-system/buildroot/output/build/ubinize.cfg"
/usr/bin/install -m 0644 support/misc/target-dir-warning.txt
/home/ayurovsky/dev/build-system/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

--- Comment #5 from Arnout Vandecappelle <arnout@mind.be> ---
Nothing weird in your log...

Can you check the permissions of output/build/ubinize.cfg? And its contents?

Do you have ACLs, SMACK, SELinux or anything like that?

I really don't see how running ubinize from without or outside of fakeroot
could make a difference...

Maybe add an strace in front of the ubinize command in ubi.mk to see what is
really going on?


I do think it's a good idea to do the ubinize outside of fakeroot. However, I'd
like to understand why this is happening - if it happens for ubinize, it could
happen for something else as well...

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

--- Comment #6 from Andrey Yurovsky <yurovsky@gmail.com> ---
I have SELinux but it's set to 'permissive' right now, nothing else is
configured differently from a standard Fedora 24 install.

ubi.mk rm's the ubinize.cfg file, if I comment that out it'll be left behind
and then:

$ stat output/build/ubinize.cfg 
  File: 'output/build/ubinize.cfg'
  Size: 124             Blocks: 8          IO Block: 4096   regular file
Device: fd03h/64771d    Inode: 1596887     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/ayurovsky)   Gid: ( 1000/ayurovsky)
Context: unconfined_u:object_r:user_home_t:s0
Access: 2016-10-27 15:10:37.142923074 -0700
Modify: 2016-10-27 15:10:37.142923074 -0700
Change: 2016-10-27 15:10:37.142923074 -0700
 Birth: -

What I can't explain is that if instead I leave everything alone and just
insert a line like:

stat $(BUILD_DIR)/ubinize.cf

in ubi.mk (in its _CMD), stat reports that the path doesn't exist (as does
ubinize).  So why does it exist if I remove the last line in _CMD (the rm) but
it doesn't exist after the first line (the $(INSTALL)) ... ?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

--- Comment #7 from Andrey Yurovsky <yurovsky@gmail.com> ---
Ok, I found a better workaround that may explain some of this (plus the
original workaround I posted must be wrong, please ignore that).

If I install (via dnf in my case) fakeroot and then in fs/common.mk change:

PATH=$$(BR_PATH) $$(HOST_DIR)/usr/bin/fakeroot -- $$(FAKEROOT_SCRIPT)

to just:

fakeroot -- $$(FAKEROOT_SCRIPT)

...everything works fine.  So the issue has something to do with the host tools
provided fakeroot and/or the path handed to that.  They're both version 1.20.2
so I'm guessing it's some path or environment issue.

For now I'll patch common.mk to eliminate the buildroot-provided fakeroot but I
would love to help get to the bottom of this (or understand what I'm doing
wrong).  Thanks!

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

--- Comment #8 from Yann E. MORIN <yann.morin.1998@free.fr> ---
Andrey,

What distribution are you using?

Do you have LD_LIBRARY_PATH and/or LD_PRELOAD in your environment?
If so, that are their values?

Regards,
Yann E. MORIN.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

Gustavo Zacarias <gustavo@zacarias.com.ar> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |gustavo at zacarias.com.ar

--- Comment #9 from Gustavo Zacarias <gustavo@zacarias.com.ar> ---
Can reproduce on fc23.

Upstream related bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=1238802
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=799858

It seems the selinux context isn't picked up by fakeroot and causes trouble.
Unfortunately there seems to be no patch at this time to address this, so
running ubinize outside fakeroot looks like the only option.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

--- Comment #10 from Yann E. MORIN <yann.morin.1998@free.fr> ---
Gustavo, Andrey,

OK, so this is a known issue with fakeroot relating to xattrs.
There is nothing we can do in Buildroot, except as you said,
to run outside fakeroot.

But that solution means we need to tweak our fs infra just
for the sake of a bug in fakeroot. That's not nice...

However, following the bug report Gustavo copied, I got to:
https://www.yoctoproject.org/tools-resources/projects/pseudo

As they say:

> Pseudo is a program that can either be used directly or as
> an LD_PRELOAD, which allows these operations to succeed as
> if the user did have system administrator privileges even
> though they are an ordinary user.
>
> Pseudo has a lot of similarities to fakeroot but is a new
> implementation that improves on the problems seen using
> fakeroot.

Maybe we should also go with pseudo in Buildroot, and ditch
fakeroot? I'll try to have a look at that, and see if I can
come up with a patch.

Regards,
Yann E. MORIN.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

--- Comment #11 from Yann E. MORIN <yann.morin.1998@free.fr> ---
Andrey, All,

I just sent two patches to try and fix this issue:
    https://patchwork.ozlabs.org/patch/688725/
    https://patchwork.ozlabs.org/patch/688726/

Could you try them and report, please?

Regards,
Yann E. MORIN.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #12 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> ---
Fixed by switching from fakeroot to pseudo to build the root filesystem, done
in commit
https://git.buildroot.org/buildroot/commit/?id=8035ceb56cc3ba2541aba0d0ead3f29900d9e310.
Thanks Yann for providing the patches, and thanks Andrey for reporting the
problem in the first place!

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

--- Comment #13 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> ---
I just tested the following defconfig:

BR2_arm=y
BR2_cortex_a8=y
BR2_TOOLCHAIN_EXTERNAL=y
# BR2_TARGET_ROOTFS_TAR is not set
BR2_TARGET_ROOTFS_UBIFS=y
BR2_TARGET_ROOTFS_UBI=y

and built it under a Fedora 24 machine, and it built just fine, with Buildroot
2016.08.1 (which uses fakeroot)

So I'm not sure why we have moved to pseudo because of this bug.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

--- Comment #14 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> ---
Hum, my apologies, there is indeed the message:

iniparser: cannot open /home/test/buildroot/output/build/ubinize.cfg
ubinize: error!: cannot load the input ini file
"/home/test/buildroot/output/build/ubinize.cfg"

and indeed, the UBIFS image is produced but not the UBI image. So I can
reproduce the issue.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

--- Comment #15 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> ---
And indeed, installing Fedora's fakeroot, and changing
$$(HOST_DIR)/usr/bin/fakeroot to just /usr/bin/fakeroot makes everything work.

So there is a difference in the fakeroot source code, or the way we build it,
we makes our fakeroot behave badly on SELinux enabled systems.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

--- Comment #16 from Peter Korsgaard <jacmet@uclibc.org> ---
Switching to pseudo brought a number of issues, so I have reverted that and
instead added a workaround to the ubifs package:

https://git.buildroot.org/buildroot/commit/?id=01354e1a0f4333ffee40a20260a5689e3846560c

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9386] ubinize fails with or without custom config

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9386

--- Comment #17 from Andrey Yurovsky <yurovsky@gmail.com> ---
Thanks, that sed -i behavior issue makes a lot more sense and definitely
explains it.

-- 
You are receiving this mail because:
You are on the CC list for the bug.