LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Guenter Roeck <linux@roeck-us.net>
To: Sven Schmidt <4sschmid@informatik.uni-hamburg.de>
Cc: Arnd Bergmann <arnd@arndb.de>,
	Andrew Morton <akpm@linux-foundation.org>,
	linux-kernel@vger.kernel.org, Ley Foon Tan <lftan@altera.com>,
	nios2-dev@lists.rocketboards.org
Subject: Re: nios2 crash/hang in mainline due to 'lib: update LZ4 compressor module'
Date: Mon, 27 Feb 2017 12:37:19 -0800
Message-ID: <20170227203719.GA5823@roeck-us.net> (raw)
In-Reply-To: <20170227193455.GA25066@bierbaron.springfield.local>

On Mon, Feb 27, 2017 at 08:34:55PM +0100, Sven Schmidt wrote:
> Hi Guenter,
> 
> thanks for your testing!
> 
> I must admit, I'm fairly new to kernel development and a little overwhelmed by all that tools used.
> So I do not really know how to reproduce your test using your script. I installed qemu from the master branch and buildroot.
> Unfortunately, that's the point I'm stuck. I would be grateful if you provide me some lead how to continue.
> Would I make the kernel using ARCH=nios2 and a defconf and pass it to qemu? What arguments do I provide to that script
> (especially, the machine param)?
> 

run-qemu-nios2.sh doesn't need any parameters, though you would have to update
PATH_NIOS2 to match your toolchain and QEMU to match the qemu binary location.
Otherwise just run the script from your linux repository.

You can also build a nios2 image using 10m50_defconfig and run qemu directly.
Just remember to enable CONFIG_NIOS2_PASS_CMDLINE=y and CONFIG_BLK_DEV_INITRD=y.
CONFIG_BLK_DEV_INITRD=y enables CONFIG_RD_LZ4 which triggers the problem.

path-to-qemu/qemu-system-nios2 -M 10m50-ghrd -kernel vmlinux -no-reboot \
	-dtb arch/nios2/boot/dts/10m50_devboard.dtb \
	--append "rdinit=/sbin/init" \
	-initrd busybox-nios2.cpio \
	-nographic -monitor none

should do it (assuming you copied the root file system from
https://github.com/groeck/linux-build-test/blob/master/rootfs/nios2/busybox-nios2.cpio).

> On Sun, Feb 26, 2017 at 01:03:38PM -0800, Guenter Roeck wrote:
> > Hi Sven,
> > 
> > my qemu test for nios2 started failing with commit 4e1a33b105dd ("lib:
> > update LZ4 compressor module"). The test hangs early during boot before
> > any console output is seen. Reverting the offending patch as well as the
> > subsequent lz4 related patches fixes the problem. Disabling CONFIG_RD_LZ4
> > and with it other LZ4 options also fixes it (as does adding "return -EINVAL;"
> > at the top of the LZ4 decompression code). For reference, bisect log
> > is attached.
> > 
> 
> So, seems like it's the decompressor? Which decompression code do you mean exactly? LZ4_decompress_fast/_safe/_generic?
> Since the decompression functions worked fine in all previous tests, and this is a problem during boot, my first guess would
> be the lib/decompress_unlz4.c, providing the functions for decompressing a lz4-compressed kernel image.
> But then it should only result in problems when the kernel image is compressed, wouldn't it?
> 
I am booting the uncompressed kernel. Also, if I disable CONFIG_RD_LZ4
in the configuration, everything works just fine. Just the _presence_ of
the decompression code seems to trigger the problem. No idea if enabling
CONFIG_RD_LZ4 results in some LZ4 compressed code to be generated (I do
see usr/initramfs_data.cpio.lz4).

I added "return -EINVAL;" to the top of LZ4_decompress_generic(), which
also helped. Adding it to the individual decompression functions seemed
to be an on/off thing; sometimes it helped, sometimes not.

> > I tried with buildroot toolchains using gcc 6.1.0 as well as 6.3.0
> > and binutils 2.26.1. Scripts used to run the tests are available at
> > https://github.com/groeck/linux-build-test/tree/master/rootfs/nios2.
> > Qemu is from qemu mainline or qemu v2.8 with nios2 patches applied.
> > 
> > I tried to track down the problem, with no success. Just the presence
> > of the LZ4 code seems to be sufficient to cause the problem; I have
> > no idea why that would be the case.
> > 
> 
> Maybe there's someone who has an idea and/or is experiencing similar issues. Hopefully, we can track this down.
> 

Agreed. For my part I am pretty much out of ideas. I could explicitly
disable CONFIG_RD_LZ4 in my tests, but that would really just defeat
the purpose.

Guenter

> > Please let me know if there is anything I can do to help tracking down
> > the problem.
> > 
> > Thanks,
> > Guenter
> > 
> > ---
> > # bad: [c4f3f22eddc982d247ffe2a6690c3e4a5c46dd09] Merge tag 'linux-kselftest-4.11-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
> > # good: [9e314890292c0dd357eadef6a043704fa0b4c157] Merge tag 'openrisc-for-linus' of git://github.com/openrisc/linux
> > git bisect start 'HEAD' '9e31489'
> > # bad: [7067739df23ffd641ca99c967830e0ed2ba39eab] Merge branch 'i2c/for-4.11' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux
> > git bisect bad 7067739df23ffd641ca99c967830e0ed2ba39eab
> > # good: [c5adae9583ef6985875532904160c6bf9f07b453] lib: add CONFIG_TEST_SORT to enable self-test of sort()
> > git bisect good c5adae9583ef6985875532904160c6bf9f07b453
> > # bad: [edccb59429657b09806146339e2b27594c1d1da0] Merge tag 'fbdev-v4.11' of git://github.com/bzolnier/linux
> > git bisect bad edccb59429657b09806146339e2b27594c1d1da0
> > # good: [72db33355c1431fefcabb06c9c25705e8226eed5] fbdev: ssd1307fb: Start to use gpiod API for reset gpio
> > git bisect good 72db33355c1431fefcabb06c9c25705e8226eed5
> > # bad: [95330473636e5e4546f94874c957c3be66bb2140] checkpatch: remove false unbalanced braces warning
> > git bisect bad 95330473636e5e4546f94874c957c3be66bb2140
> > # bad: [69c78423b8f439b077929410bdf8f88e7031b891] lib/lz4: remove back-compat wrappers
> > git bisect bad 69c78423b8f439b077929410bdf8f88e7031b891
> > # bad: [e23d54e48346e775be53b3cc25a95d65da960393] lib/decompress_unlz4: change module to work with new LZ4 module version
> > git bisect bad e23d54e48346e775be53b3cc25a95d65da960393
> > # bad: [4e1a33b105ddf201f66dcc44490c6086a25eca0b] lib: update LZ4 compressor module
> > git bisect bad 4e1a33b105ddf201f66dcc44490c6086a25eca0b
> > # good: [8893f519330bb073a49c5b4676fce4be6f1be15d] lib/test_sort.c: make it explicitly non-modular
> > git bisect good 8893f519330bb073a49c5b4676fce4be6f1be15d
> > # first bad commit: [4e1a33b105ddf201f66dcc44490c6086a25eca0b] lib: update LZ4 compressor module
> 
> Thank you,
> 
> Sven

  reply index

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-02-26 21:03 Guenter Roeck
2017-02-27 19:34 ` Sven Schmidt
2017-02-27 20:37   ` Guenter Roeck [this message]
2017-02-28 15:53 ` Tobias Klauser
2017-02-28 17:53   ` Sandra Loosemore
2017-02-28 18:14     ` Guenter Roeck
2017-03-01 18:58       ` Sven Schmidt
2017-03-01 19:45         ` Guenter Roeck
2017-03-02 16:38           ` Tobias Klauser
2017-03-03  3:04             ` Guenter Roeck
2017-03-07 12:46               ` Tobias Klauser
2017-03-08  4:12                 ` Guenter Roeck
2017-03-09 13:20                 ` Guenter Roeck
2017-03-09 14:43                   ` Tobias Klauser
2017-03-09 18:49                     ` Guenter Roeck
2017-03-01 22:50         ` Sandra Loosemore
2017-03-02 13:30           ` Tobias Klauser
2017-02-28 17:57   ` Guenter Roeck

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170227203719.GA5823@roeck-us.net \
    --to=linux@roeck-us.net \
    --cc=4sschmid@informatik.uni-hamburg.de \
    --cc=akpm@linux-foundation.org \
    --cc=arnd@arndb.de \
    --cc=lftan@altera.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=nios2-dev@lists.rocketboards.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git