LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Guenter Roeck <linux@roeck-us.net>
To: Tobias Klauser <tklauser@distanz.ch>
Cc: Sven Schmidt <4sschmid@informatik.uni-hamburg.de>,
	Sandra Loosemore <sandra@codesourcery.com>,
	Arnd Bergmann <arnd@arndb.de>,
	Andrew Morton <akpm@linux-foundation.org>,
	linux-kernel@vger.kernel.org, Ley Foon Tan <lftan@altera.com>,
	nios2-dev@lists.rocketboards.org
Subject: Re: nios2 crash/hang in mainline due to 'lib: update LZ4 compressor module'
Date: Thu, 2 Mar 2017 19:04:41 -0800
Message-ID: <1ad19c21-6f6e-4516-7df5-d3536df9f4ee@roeck-us.net> (raw)
In-Reply-To: <20170302163813.GE27998@distanz.ch>

On 03/02/2017 08:38 AM, Tobias Klauser wrote:
> On 2017-03-01 at 20:45:21 +0100, Guenter Roeck <linux@roeck-us.net> wrote:
>> On Wed, Mar 01, 2017 at 07:58:17PM +0100, Sven Schmidt wrote:
>>> Hi Guenter, Tobias and Sandra,
>>>
>>> thanks for your effort here.
>>>
>>> On Tue, Feb 28, 2017 at 10:14:13AM -0800, Guenter Roeck wrote:
>>>> On Tue, Feb 28, 2017 at 10:53:56AM -0700, Sandra Loosemore wrote:
>>>>> On 02/28/2017 08:53 AM, Tobias Klauser wrote:
>>>>>> (adding Sandra Loosemore to Cc due to possible relation to gcc/binutils
>>>>>> for nios2)
>>>>>>
>>>>>> On 2017-02-26 at 22:03:38 +0100, Guenter Roeck <linux@roeck-us.net> wrote:
>>>>>>> Hi Sven,
>>>>>>>
>>>>>>> my qemu test for nios2 started failing with commit 4e1a33b105dd ("lib:
>>>>>>> update LZ4 compressor module"). The test hangs early during boot before
>>>>>>> any console output is seen. Reverting the offending patch as well as the
>>>>>>> subsequent lz4 related patches fixes the problem. Disabling CONFIG_RD_LZ4
>>>>>>> and with it other LZ4 options also fixes it (as does adding "return -EINVAL;"
>>>>>>> at the top of the LZ4 decompression code). For reference, bisect log
>>>>>>> is attached.
>>>>>>>
>>>>>>> I tried with buildroot toolchains using gcc 6.1.0 as well as 6.3.0
>>>>>>> and binutils 2.26.1. Scripts used to run the tests are available at
>>>>>>> https://github.com/groeck/linux-build-test/tree/master/rootfs/nios2.
>>>>>>> Qemu is from qemu mainline or qemu v2.8 with nios2 patches applied.
>>>>>>
>>>>>> Looks like this is somehow related to gcc/binutils. Using GCC 4.8.3 and
>>>>>> binutils 2.24.51 (both from from Sourcery CodeBench Lite 2014.05) I can
>>>>>> get a kernel booting on latest master branch. AFAICT, none of the
>>>>>> LZ4_decompress_* functions are called during boot.
>>>>>>
>>>
>>> It seems a bit strange that code which is not actually called causes problems like that.
>>>
>> Yes, it is, though it is always possible. The code isn't exactly easy to
>> understand; there may be some hidden caveats such as global variables. It may
>> also be that some jump target exceeds its range (though why that would only
>> be seen with the LZ4 code is another question), or that the compiler gets
>> confused by the forced inlines (disabling that didn't make a difference,
>> though, nor did disabling -O3).
>>
>>> Please let me know if and how I may help you figure out what's happening, especially
>>> regarding the differences between the previous LZ4 and the current implementation.
>>>
>>
>> For my part I am all but clueless. Unless someone has an idea, we may to
>> disable LZ4 support for nios2 for the time being. Does anyone have thoughts
>> on that ? Of course, that would not help if the problem also affects
>> recent gcc/binutil versions on other architectures.
>
> After some further investigations, I'd say this isn't "caused" by LZ4
> specifically but by a more general problem with one of the nios2 arch
> specific tools involved.
>
> I manually enabled random additional CONFIG_* options and in some cases
> I got the kernel to boot (with CONFIG_RD_LZ4 enabled and no return
> -EINVAL in place) while in others I didn't. So I'd rather suspect this
> problem to be connected to the size or structure of the generated vmlinux
> image.
>
> Or could this even be a problem with qemu? Did anyone already verify
> this on the 10m50 devboard? (Unfortunately I don't have any nios2
> devboard available right now, otherwise I would have done this...)
>

That is of course always possible.

> Other than that I'm also becoming all but clueless... One option I
> thought of was using the QEMU monitor to dump the CPU state after the
> hang but so far I didn't manage to get it to work (hints appreciated ;)
>

Something like

qemu-system-nios2 -M 10m50-ghrd -kernel vmlinux -no-reboot \
	-dtb arch/nios2/boot/dts/10m50_devboard.dtb \
	--append "rdinit=/sbin/init" -initrd busybox-nios2.cpio

gives you a qemu monitor window. Use "info registers" to see registers.
Looks like it is stuck in init_bootmem_core, or at least that is what it
shows for me.

Guenter

  reply index

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-02-26 21:03 Guenter Roeck
2017-02-27 19:34 ` Sven Schmidt
2017-02-27 20:37   ` Guenter Roeck
2017-02-28 15:53 ` Tobias Klauser
2017-02-28 17:53   ` Sandra Loosemore
2017-02-28 18:14     ` Guenter Roeck
2017-03-01 18:58       ` Sven Schmidt
2017-03-01 19:45         ` Guenter Roeck
2017-03-02 16:38           ` Tobias Klauser
2017-03-03  3:04             ` Guenter Roeck [this message]
2017-03-07 12:46               ` Tobias Klauser
2017-03-08  4:12                 ` Guenter Roeck
2017-03-09 13:20                 ` Guenter Roeck
2017-03-09 14:43                   ` Tobias Klauser
2017-03-09 18:49                     ` Guenter Roeck
2017-03-01 22:50         ` Sandra Loosemore
2017-03-02 13:30           ` Tobias Klauser
2017-02-28 17:57   ` Guenter Roeck

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1ad19c21-6f6e-4516-7df5-d3536df9f4ee@roeck-us.net \
    --to=linux@roeck-us.net \
    --cc=4sschmid@informatik.uni-hamburg.de \
    --cc=akpm@linux-foundation.org \
    --cc=arnd@arndb.de \
    --cc=lftan@altera.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=nios2-dev@lists.rocketboards.org \
    --cc=sandra@codesourcery.com \
    --cc=tklauser@distanz.ch \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git