Crash in -next due to 'MIPS: Add cacheinfo support'

Crash in -next due to 'MIPS: Add cacheinfo support'

[Guenter Roeck]

Hi Justin,

The 32-bit qemu mips runtime tests for -next at kerneltests.org are crashing
for non-SMP builds as follows.

cacheinfo: Failed to find cpu0 device node
cacheinfo: Unable to detect cache hierarchy for CPU 0
CPU 0 Unable to handle kernel paging request at virtual address 00000004, epc ==
8013c6a8, ra == 8013cb54
Oops[#1]:
CPU: 0 PID: 5 Comm: kworker/u2:0 Not tainted 4.10.0-rc7-next-20170208 #1
Workqueue: events_unbound call_usermodehelper_exec_work
task: 8704a420 task.stack: 87068000
$ 0   : 00000000 80770000 873a8204 8700c214
$ 4   : 00000000 873a8200 00000000 ffff00fe
$ 8   : 8706bfe0 0000a400 fffffffc 8704a459
$12   : 00000000 00000000 00000000 00000720
$16   : 873a8200 87045880 00000000 8700c200
$20   : 87014000 87014005 8700c200 8700c200
$24   : 00000000 8014d1e8                  
$28   : 87068000 8706be70 87045880 8013cb54
Hi    : 00077698
Lo    : ab801af8
epc   : 8013c6a8 process_one_work+0xd4/0x408
ra    : 8013cb54 worker_thread+0x178/0x598
Status: 1000a402	KERNEL EXL 
Cause : 0080000c (ExcCode 03)
BadVA : 00000004
PrId  : 00019300 (MIPS 24Kc)
Modules linked in:
Process kworker/u2:0 (pid: 5, threadinfo=87068000, task=8704a420, tls=00000000)
Stack : 80700000 8700c214 00000088 80700000 8700c200 8700c200 8700c200 87045898
        80700000 8700c214 00000088 80700000 8700c200 8013cb54 8013c9dc 8704fe00
	87045600 87045700 80700000 80700000 80700000 80676560 87045600 87045700
	87045618 87045880 8013c9dc 8704fe00 80670000 80760000 806e0000 80142958
	00000000 00000000 00000000 00000000 80142850 80142850 87045700 8704b380
	...
Call Trace:
[<8013c6a8>] process_one_work+0xd4/0x408
[<8013cb54>] worker_thread+0x178/0x598
[<80142958>] kthread+0x108/0x138
[<80105e18>] ret_from_kernel_thread+0x14/0x1c
Code: 8e030008  8e040004  8e150000 <ac830004> 7eb51900  ac640000  ae020004 16400094  ae020008 

Bisect points to 'MIPS: Add cacheinfo support'; see below for details.
Reverting this patch fixes te problem.

Configuration is pretty much malta_defconfig with SMP disabled.
I can provide more details if needed.

Thanks,
Guenter

---
# bad: [e3e6c5f3544c5d05c6b3b309a34f4f2c3537e993] Add linux-next specific files for 20170208
# good: [d5adbfcd5f7bcc6fa58a41c5c5ada0e5c826ce2c] Linux 4.10-rc7
git bisect start 'HEAD' 'v4.10-rc7'
# bad: [403e468309f9e2b2dbe264be1ad29b1486ed720e] Merge remote-tracking branch 'crypto/master'
git bisect bad 403e468309f9e2b2dbe264be1ad29b1486ed720e
# bad: [44448c3f07a3ae77164de4405fa97baca4f103f5] Merge remote-tracking branch 'hid/for-next'
git bisect bad 44448c3f07a3ae77164de4405fa97baca4f103f5
# good: [dd4318312c6fc5c00ae7619f875fb73538a2c1e6] Merge remote-tracking branch 'omap/for-next'
git bisect good dd4318312c6fc5c00ae7619f875fb73538a2c1e6
# bad: [2fe482530119a6d07acca625b29d527dc22435e6] Merge remote-tracking branch 'powerpc/next'
git bisect bad 2fe482530119a6d07acca625b29d527dc22435e6
# good: [8576190cecf3370ce07ed04178cb3ecbf17dc2d9] Merge remote-tracking branch 'clk/clk-next'
git bisect good 8576190cecf3370ce07ed04178cb3ecbf17dc2d9
# bad: [bf0505dfeac3f7df81b62d9d81987aaef6739ffd] MIPS: Fix special case in 64 bit IP checksumming.
git bisect bad bf0505dfeac3f7df81b62d9d81987aaef6739ffd
# good: [e89ef66d7682f031f026eee6bba03c8c2248d2a9] MIPS: init: Ensure reserved memory regions are not added to bootmem
git bisect good e89ef66d7682f031f026eee6bba03c8c2248d2a9
# bad: [a8b3b0c94ac282628f0668d1366239a3fa72dc9d] MIPS: Netlogic: Fix assembler warning from smpboot.S
git bisect bad a8b3b0c94ac282628f0668d1366239a3fa72dc9d
# bad: [07724712bf22aec5fe44e5d399f115755f436721] MIPS: ralink: Add missing symbol for highmem support.
git bisect bad 07724712bf22aec5fe44e5d399f115755f436721
# good: [856b0f591e951a234d6642cc466023df182eb51c] MIPS: kexec: add debug info about the new kexec'ed image
git bisect good 856b0f591e951a234d6642cc466023df182eb51c
# bad: [2517caf19dbfac3b39f2db5500c5fd03c4370e81] MIPS: ralink: Add missing I2C and I2S clocks.
git bisect bad 2517caf19dbfac3b39f2db5500c5fd03c4370e81
# bad: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
git bisect bad ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d
# good: [0014dea6b71ae3e0384c3358f632b4728c3d432e] MIPS: generic/kexec: add support for a DTB passed in a separate buffer
git bisect good 0014dea6b71ae3e0384c3358f632b4728c3d432e
# first bad commit: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support

Re: Crash in -next due to 'MIPS: Add cacheinfo support'

[Justin Chen]

Hello Guenter,

I am unable to reproduce the problem. May you give me more details?

I rebased my patch onto of 4.10-rc7 and compiled with malta defconfig
with disabled SMP. Then booted with qemu. Doesn't seem to have an
error with the way I have it set up.

I am using the command:
qemu-system-mipsel -kernel vmlinux -nographic -smp 0

Thanks,
Justin

On Wed, Feb 8, 2017 at 3:45 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> Hi Justin,
>
> The 32-bit qemu mips runtime tests for -next at kerneltests.org are crashing
> for non-SMP builds as follows.
>
> cacheinfo: Failed to find cpu0 device node
> cacheinfo: Unable to detect cache hierarchy for CPU 0
> CPU 0 Unable to handle kernel paging request at virtual address 00000004, epc ==
> 8013c6a8, ra == 8013cb54
> Oops[#1]:
> CPU: 0 PID: 5 Comm: kworker/u2:0 Not tainted 4.10.0-rc7-next-20170208 #1
> Workqueue: events_unbound call_usermodehelper_exec_work
> task: 8704a420 task.stack: 87068000
> $ 0   : 00000000 80770000 873a8204 8700c214
> $ 4   : 00000000 873a8200 00000000 ffff00fe
> $ 8   : 8706bfe0 0000a400 fffffffc 8704a459
> $12   : 00000000 00000000 00000000 00000720
> $16   : 873a8200 87045880 00000000 8700c200
> $20   : 87014000 87014005 8700c200 8700c200
> $24   : 00000000 8014d1e8
> $28   : 87068000 8706be70 87045880 8013cb54
> Hi    : 00077698
> Lo    : ab801af8
> epc   : 8013c6a8 process_one_work+0xd4/0x408
> ra    : 8013cb54 worker_thread+0x178/0x598
> Status: 1000a402        KERNEL EXL
> Cause : 0080000c (ExcCode 03)
> BadVA : 00000004
> PrId  : 00019300 (MIPS 24Kc)
> Modules linked in:
> Process kworker/u2:0 (pid: 5, threadinfo=87068000, task=8704a420, tls=00000000)
> Stack : 80700000 8700c214 00000088 80700000 8700c200 8700c200 8700c200 87045898
>         80700000 8700c214 00000088 80700000 8700c200 8013cb54 8013c9dc 8704fe00
>         87045600 87045700 80700000 80700000 80700000 80676560 87045600 87045700
>         87045618 87045880 8013c9dc 8704fe00 80670000 80760000 806e0000 80142958
>         00000000 00000000 00000000 00000000 80142850 80142850 87045700 8704b380
>         ...
> Call Trace:
> [<8013c6a8>] process_one_work+0xd4/0x408
> [<8013cb54>] worker_thread+0x178/0x598
> [<80142958>] kthread+0x108/0x138
> [<80105e18>] ret_from_kernel_thread+0x14/0x1c
> Code: 8e030008  8e040004  8e150000 <ac830004> 7eb51900  ac640000  ae020004 16400094  ae020008
>
> Bisect points to 'MIPS: Add cacheinfo support'; see below for details.
> Reverting this patch fixes te problem.
>
> Configuration is pretty much malta_defconfig with SMP disabled.
> I can provide more details if needed.
>
> Thanks,
> Guenter
>
> ---
> # bad: [e3e6c5f3544c5d05c6b3b309a34f4f2c3537e993] Add linux-next specific files for 20170208
> # good: [d5adbfcd5f7bcc6fa58a41c5c5ada0e5c826ce2c] Linux 4.10-rc7
> git bisect start 'HEAD' 'v4.10-rc7'
> # bad: [403e468309f9e2b2dbe264be1ad29b1486ed720e] Merge remote-tracking branch 'crypto/master'
> git bisect bad 403e468309f9e2b2dbe264be1ad29b1486ed720e
> # bad: [44448c3f07a3ae77164de4405fa97baca4f103f5] Merge remote-tracking branch 'hid/for-next'
> git bisect bad 44448c3f07a3ae77164de4405fa97baca4f103f5
> # good: [dd4318312c6fc5c00ae7619f875fb73538a2c1e6] Merge remote-tracking branch 'omap/for-next'
> git bisect good dd4318312c6fc5c00ae7619f875fb73538a2c1e6
> # bad: [2fe482530119a6d07acca625b29d527dc22435e6] Merge remote-tracking branch 'powerpc/next'
> git bisect bad 2fe482530119a6d07acca625b29d527dc22435e6
> # good: [8576190cecf3370ce07ed04178cb3ecbf17dc2d9] Merge remote-tracking branch 'clk/clk-next'
> git bisect good 8576190cecf3370ce07ed04178cb3ecbf17dc2d9
> # bad: [bf0505dfeac3f7df81b62d9d81987aaef6739ffd] MIPS: Fix special case in 64 bit IP checksumming.
> git bisect bad bf0505dfeac3f7df81b62d9d81987aaef6739ffd
> # good: [e89ef66d7682f031f026eee6bba03c8c2248d2a9] MIPS: init: Ensure reserved memory regions are not added to bootmem
> git bisect good e89ef66d7682f031f026eee6bba03c8c2248d2a9
> # bad: [a8b3b0c94ac282628f0668d1366239a3fa72dc9d] MIPS: Netlogic: Fix assembler warning from smpboot.S
> git bisect bad a8b3b0c94ac282628f0668d1366239a3fa72dc9d
> # bad: [07724712bf22aec5fe44e5d399f115755f436721] MIPS: ralink: Add missing symbol for highmem support.
> git bisect bad 07724712bf22aec5fe44e5d399f115755f436721
> # good: [856b0f591e951a234d6642cc466023df182eb51c] MIPS: kexec: add debug info about the new kexec'ed image
> git bisect good 856b0f591e951a234d6642cc466023df182eb51c
> # bad: [2517caf19dbfac3b39f2db5500c5fd03c4370e81] MIPS: ralink: Add missing I2C and I2S clocks.
> git bisect bad 2517caf19dbfac3b39f2db5500c5fd03c4370e81
> # bad: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
> git bisect bad ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d
> # good: [0014dea6b71ae3e0384c3358f632b4728c3d432e] MIPS: generic/kexec: add support for a DTB passed in a separate buffer
> git bisect good 0014dea6b71ae3e0384c3358f632b4728c3d432e
> # first bad commit: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
>
>

Re: Crash in -next due to 'MIPS: Add cacheinfo support'

[Guenter Roeck]

On 02/09/2017 04:01 PM, Justin Chen wrote:
> Hello Guenter,
>
> I am unable to reproduce the problem. May you give me more details?
>
The scripts I am using are available at

https://github.com/groeck/linux-build-test/tree/master/rootfs

in the mips and mipsel subdirectories (both crash). Individual
build logs are always available at kerneltests.org/builders,
in the 'next' column.

Note: If I build the image with my configuration, and run
your qemu command, I get a different crash:

...
ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0x1088 irq 15
CPU 0 Unable to handle kernel paging request at virtual address fffffffc, epc == 80341d68, ra == 803eae70
Oops[#1]:
CPU: 0 PID: 1 Comm: swapper Not tainted 4.10.0-rc7-next-20170209 #2
task: 8704b8a0 task.stack: 8704c000
$ 0   : 00000000 0000006f fffffffc 00000000
$ 4   : 87044980 8069ee48 8069ee49 00000004
$ 8   : 00000000 8741c990 00000000 00000040
$12   : 00000000 00000001 00000000 fffffffe
$16   : 8070ae14 80742fb8 80738190 00000000
$20   : 806e0000 8071c2fc 80770000 00000008
$24   : 00000110 00000000
$28   : 8704c000 8704fe10 80754728 803eae70
Hi    : 0000000f
Lo    : 8f8f8f9f
epc   : 80341d68 kset_find_obj+0xc/0xa4
ra    : 803eae70 driver_find+0x1c/0x48
Status: 1000a403	KERNEL EXL IE
Cause : 00800008 (ExcCode 02)
BadVA : fffffffc
PrId  : 00019300 (MIPS 24Kc)
Modules linked in:
Process swapper (pid: 1, threadinfo=8704c000, task=8704b8a0, tls=00000000)
Stack : 80710000 00000000 806e0000 8044ba18 8070ae14 803eaf10 00000000 00000000
         80770000 80742fb8 80770000 80770000 80742fb8 801005dc 8708b900 810017e0
         8704fe68 806f0000 806ee894 81041c41 80600600 80141fb8 00000000 80740000
         806e0000 80720000 00000000 00000006 806765e4 00000006 00000017 80670000
         80676cf0 80680000 80770000 80742fb8 00000006 80742f98 80770000 80742fb8
         ...
Call Trace:
[<80341d68>] kset_find_obj+0xc/0xa4
[<803eae70>] driver_find+0x1c/0x48
[<803eaf10>] driver_register+0x74/0x128
[<801005dc>] do_one_initcall+0x48/0x16c
[<8071cd18>] kernel_init_freeable+0x170/0x22c
[<805e1d30>] kernel_init+0x14/0x108
[<80105e18>] ret_from_kernel_thread+0x14/0x1c
Code: 8c830000  10830012  2462fffc <8c430000> 1060000c  00a03021  90670000  90c10000  24630001

---[ end trace c22dffa22f0b1376 ]---
Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b

---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b

Guenter

> I rebased my patch onto of 4.10-rc7 and compiled with malta defconfig
> with disabled SMP. Then booted with qemu. Doesn't seem to have an
> error with the way I have it set up.
>
> I am using the command:
> qemu-system-mipsel -kernel vmlinux -nographic -smp 0
>
> Thanks,
> Justin
>
> On Wed, Feb 8, 2017 at 3:45 PM, Guenter Roeck <linux@roeck-us.net> wrote:
>> Hi Justin,
>>
>> The 32-bit qemu mips runtime tests for -next at kerneltests.org are crashing
>> for non-SMP builds as follows.
>>
>> cacheinfo: Failed to find cpu0 device node
>> cacheinfo: Unable to detect cache hierarchy for CPU 0
>> CPU 0 Unable to handle kernel paging request at virtual address 00000004, epc ==
>> 8013c6a8, ra == 8013cb54
>> Oops[#1]:
>> CPU: 0 PID: 5 Comm: kworker/u2:0 Not tainted 4.10.0-rc7-next-20170208 #1
>> Workqueue: events_unbound call_usermodehelper_exec_work
>> task: 8704a420 task.stack: 87068000
>> $ 0   : 00000000 80770000 873a8204 8700c214
>> $ 4   : 00000000 873a8200 00000000 ffff00fe
>> $ 8   : 8706bfe0 0000a400 fffffffc 8704a459
>> $12   : 00000000 00000000 00000000 00000720
>> $16   : 873a8200 87045880 00000000 8700c200
>> $20   : 87014000 87014005 8700c200 8700c200
>> $24   : 00000000 8014d1e8
>> $28   : 87068000 8706be70 87045880 8013cb54
>> Hi    : 00077698
>> Lo    : ab801af8
>> epc   : 8013c6a8 process_one_work+0xd4/0x408
>> ra    : 8013cb54 worker_thread+0x178/0x598
>> Status: 1000a402        KERNEL EXL
>> Cause : 0080000c (ExcCode 03)
>> BadVA : 00000004
>> PrId  : 00019300 (MIPS 24Kc)
>> Modules linked in:
>> Process kworker/u2:0 (pid: 5, threadinfo=87068000, task=8704a420, tls=00000000)
>> Stack : 80700000 8700c214 00000088 80700000 8700c200 8700c200 8700c200 87045898
>>         80700000 8700c214 00000088 80700000 8700c200 8013cb54 8013c9dc 8704fe00
>>         87045600 87045700 80700000 80700000 80700000 80676560 87045600 87045700
>>         87045618 87045880 8013c9dc 8704fe00 80670000 80760000 806e0000 80142958
>>         00000000 00000000 00000000 00000000 80142850 80142850 87045700 8704b380
>>         ...
>> Call Trace:
>> [<8013c6a8>] process_one_work+0xd4/0x408
>> [<8013cb54>] worker_thread+0x178/0x598
>> [<80142958>] kthread+0x108/0x138
>> [<80105e18>] ret_from_kernel_thread+0x14/0x1c
>> Code: 8e030008  8e040004  8e150000 <ac830004> 7eb51900  ac640000  ae020004 16400094  ae020008
>>
>> Bisect points to 'MIPS: Add cacheinfo support'; see below for details.
>> Reverting this patch fixes te problem.
>>
>> Configuration is pretty much malta_defconfig with SMP disabled.
>> I can provide more details if needed.
>>
>> Thanks,
>> Guenter
>>
>> ---
>> # bad: [e3e6c5f3544c5d05c6b3b309a34f4f2c3537e993] Add linux-next specific files for 20170208
>> # good: [d5adbfcd5f7bcc6fa58a41c5c5ada0e5c826ce2c] Linux 4.10-rc7
>> git bisect start 'HEAD' 'v4.10-rc7'
>> # bad: [403e468309f9e2b2dbe264be1ad29b1486ed720e] Merge remote-tracking branch 'crypto/master'
>> git bisect bad 403e468309f9e2b2dbe264be1ad29b1486ed720e
>> # bad: [44448c3f07a3ae77164de4405fa97baca4f103f5] Merge remote-tracking branch 'hid/for-next'
>> git bisect bad 44448c3f07a3ae77164de4405fa97baca4f103f5
>> # good: [dd4318312c6fc5c00ae7619f875fb73538a2c1e6] Merge remote-tracking branch 'omap/for-next'
>> git bisect good dd4318312c6fc5c00ae7619f875fb73538a2c1e6
>> # bad: [2fe482530119a6d07acca625b29d527dc22435e6] Merge remote-tracking branch 'powerpc/next'
>> git bisect bad 2fe482530119a6d07acca625b29d527dc22435e6
>> # good: [8576190cecf3370ce07ed04178cb3ecbf17dc2d9] Merge remote-tracking branch 'clk/clk-next'
>> git bisect good 8576190cecf3370ce07ed04178cb3ecbf17dc2d9
>> # bad: [bf0505dfeac3f7df81b62d9d81987aaef6739ffd] MIPS: Fix special case in 64 bit IP checksumming.
>> git bisect bad bf0505dfeac3f7df81b62d9d81987aaef6739ffd
>> # good: [e89ef66d7682f031f026eee6bba03c8c2248d2a9] MIPS: init: Ensure reserved memory regions are not added to bootmem
>> git bisect good e89ef66d7682f031f026eee6bba03c8c2248d2a9
>> # bad: [a8b3b0c94ac282628f0668d1366239a3fa72dc9d] MIPS: Netlogic: Fix assembler warning from smpboot.S
>> git bisect bad a8b3b0c94ac282628f0668d1366239a3fa72dc9d
>> # bad: [07724712bf22aec5fe44e5d399f115755f436721] MIPS: ralink: Add missing symbol for highmem support.
>> git bisect bad 07724712bf22aec5fe44e5d399f115755f436721
>> # good: [856b0f591e951a234d6642cc466023df182eb51c] MIPS: kexec: add debug info about the new kexec'ed image
>> git bisect good 856b0f591e951a234d6642cc466023df182eb51c
>> # bad: [2517caf19dbfac3b39f2db5500c5fd03c4370e81] MIPS: ralink: Add missing I2C and I2S clocks.
>> git bisect bad 2517caf19dbfac3b39f2db5500c5fd03c4370e81
>> # bad: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
>> git bisect bad ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d
>> # good: [0014dea6b71ae3e0384c3358f632b4728c3d432e] MIPS: generic/kexec: add support for a DTB passed in a separate buffer
>> git bisect good 0014dea6b71ae3e0384c3358f632b4728c3d432e
>> # first bad commit: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
>>
>>
>

Re: Crash in -next due to 'MIPS: Add cacheinfo support'

[James Hogan]

[-- Attachment #1: Type: text/plain, Size: 8320 bytes --]

Hi Guenter,

On Thu, Feb 09, 2017 at 08:50:04PM -0800, Guenter Roeck wrote:
> On 02/09/2017 04:01 PM, Justin Chen wrote:
> > Hello Guenter,
> >
> > I am unable to reproduce the problem. May you give me more details?
> >
> The scripts I am using are available at
> 
> https://github.com/groeck/linux-build-test/tree/master/rootfs
> 
> in the mips and mipsel subdirectories (both crash). Individual
> build logs are always available at kerneltests.org/builders,
> in the 'next' column.

Did you find it 100% reproducible? I was trying to reproduce yesterday
evening, and did hit it a few times, but then it stopped happening
before I could try and figure out what was going wrong.

I presume from your run-qemu-mipsel.sh script its a vanilla 2.7
qemu-system-mipsel?

Cheers
James

> 
> Note: If I build the image with my configuration, and run
> your qemu command, I get a different crash:
> 
> ...
> ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0x1088 irq 15
> CPU 0 Unable to handle kernel paging request at virtual address fffffffc, epc == 80341d68, ra == 803eae70
> Oops[#1]:
> CPU: 0 PID: 1 Comm: swapper Not tainted 4.10.0-rc7-next-20170209 #2
> task: 8704b8a0 task.stack: 8704c000
> $ 0   : 00000000 0000006f fffffffc 00000000
> $ 4   : 87044980 8069ee48 8069ee49 00000004
> $ 8   : 00000000 8741c990 00000000 00000040
> $12   : 00000000 00000001 00000000 fffffffe
> $16   : 8070ae14 80742fb8 80738190 00000000
> $20   : 806e0000 8071c2fc 80770000 00000008
> $24   : 00000110 00000000
> $28   : 8704c000 8704fe10 80754728 803eae70
> Hi    : 0000000f
> Lo    : 8f8f8f9f
> epc   : 80341d68 kset_find_obj+0xc/0xa4
> ra    : 803eae70 driver_find+0x1c/0x48
> Status: 1000a403	KERNEL EXL IE
> Cause : 00800008 (ExcCode 02)
> BadVA : fffffffc
> PrId  : 00019300 (MIPS 24Kc)
> Modules linked in:
> Process swapper (pid: 1, threadinfo=8704c000, task=8704b8a0, tls=00000000)
> Stack : 80710000 00000000 806e0000 8044ba18 8070ae14 803eaf10 00000000 00000000
>          80770000 80742fb8 80770000 80770000 80742fb8 801005dc 8708b900 810017e0
>          8704fe68 806f0000 806ee894 81041c41 80600600 80141fb8 00000000 80740000
>          806e0000 80720000 00000000 00000006 806765e4 00000006 00000017 80670000
>          80676cf0 80680000 80770000 80742fb8 00000006 80742f98 80770000 80742fb8
>          ...
> Call Trace:
> [<80341d68>] kset_find_obj+0xc/0xa4
> [<803eae70>] driver_find+0x1c/0x48
> [<803eaf10>] driver_register+0x74/0x128
> [<801005dc>] do_one_initcall+0x48/0x16c
> [<8071cd18>] kernel_init_freeable+0x170/0x22c
> [<805e1d30>] kernel_init+0x14/0x108
> [<80105e18>] ret_from_kernel_thread+0x14/0x1c
> Code: 8c830000  10830012  2462fffc <8c430000> 1060000c  00a03021  90670000  90c10000  24630001
> 
> ---[ end trace c22dffa22f0b1376 ]---
> Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> 
> ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> 
> Guenter
> 
> > I rebased my patch onto of 4.10-rc7 and compiled with malta defconfig
> > with disabled SMP. Then booted with qemu. Doesn't seem to have an
> > error with the way I have it set up.
> >
> > I am using the command:
> > qemu-system-mipsel -kernel vmlinux -nographic -smp 0
> >
> > Thanks,
> > Justin
> >
> > On Wed, Feb 8, 2017 at 3:45 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> >> Hi Justin,
> >>
> >> The 32-bit qemu mips runtime tests for -next at kerneltests.org are crashing
> >> for non-SMP builds as follows.
> >>
> >> cacheinfo: Failed to find cpu0 device node
> >> cacheinfo: Unable to detect cache hierarchy for CPU 0
> >> CPU 0 Unable to handle kernel paging request at virtual address 00000004, epc ==
> >> 8013c6a8, ra == 8013cb54
> >> Oops[#1]:
> >> CPU: 0 PID: 5 Comm: kworker/u2:0 Not tainted 4.10.0-rc7-next-20170208 #1
> >> Workqueue: events_unbound call_usermodehelper_exec_work
> >> task: 8704a420 task.stack: 87068000
> >> $ 0   : 00000000 80770000 873a8204 8700c214
> >> $ 4   : 00000000 873a8200 00000000 ffff00fe
> >> $ 8   : 8706bfe0 0000a400 fffffffc 8704a459
> >> $12   : 00000000 00000000 00000000 00000720
> >> $16   : 873a8200 87045880 00000000 8700c200
> >> $20   : 87014000 87014005 8700c200 8700c200
> >> $24   : 00000000 8014d1e8
> >> $28   : 87068000 8706be70 87045880 8013cb54
> >> Hi    : 00077698
> >> Lo    : ab801af8
> >> epc   : 8013c6a8 process_one_work+0xd4/0x408
> >> ra    : 8013cb54 worker_thread+0x178/0x598
> >> Status: 1000a402        KERNEL EXL
> >> Cause : 0080000c (ExcCode 03)
> >> BadVA : 00000004
> >> PrId  : 00019300 (MIPS 24Kc)
> >> Modules linked in:
> >> Process kworker/u2:0 (pid: 5, threadinfo=87068000, task=8704a420, tls=00000000)
> >> Stack : 80700000 8700c214 00000088 80700000 8700c200 8700c200 8700c200 87045898
> >>         80700000 8700c214 00000088 80700000 8700c200 8013cb54 8013c9dc 8704fe00
> >>         87045600 87045700 80700000 80700000 80700000 80676560 87045600 87045700
> >>         87045618 87045880 8013c9dc 8704fe00 80670000 80760000 806e0000 80142958
> >>         00000000 00000000 00000000 00000000 80142850 80142850 87045700 8704b380
> >>         ...
> >> Call Trace:
> >> [<8013c6a8>] process_one_work+0xd4/0x408
> >> [<8013cb54>] worker_thread+0x178/0x598
> >> [<80142958>] kthread+0x108/0x138
> >> [<80105e18>] ret_from_kernel_thread+0x14/0x1c
> >> Code: 8e030008  8e040004  8e150000 <ac830004> 7eb51900  ac640000  ae020004 16400094  ae020008
> >>
> >> Bisect points to 'MIPS: Add cacheinfo support'; see below for details.
> >> Reverting this patch fixes te problem.
> >>
> >> Configuration is pretty much malta_defconfig with SMP disabled.
> >> I can provide more details if needed.
> >>
> >> Thanks,
> >> Guenter
> >>
> >> ---
> >> # bad: [e3e6c5f3544c5d05c6b3b309a34f4f2c3537e993] Add linux-next specific files for 20170208
> >> # good: [d5adbfcd5f7bcc6fa58a41c5c5ada0e5c826ce2c] Linux 4.10-rc7
> >> git bisect start 'HEAD' 'v4.10-rc7'
> >> # bad: [403e468309f9e2b2dbe264be1ad29b1486ed720e] Merge remote-tracking branch 'crypto/master'
> >> git bisect bad 403e468309f9e2b2dbe264be1ad29b1486ed720e
> >> # bad: [44448c3f07a3ae77164de4405fa97baca4f103f5] Merge remote-tracking branch 'hid/for-next'
> >> git bisect bad 44448c3f07a3ae77164de4405fa97baca4f103f5
> >> # good: [dd4318312c6fc5c00ae7619f875fb73538a2c1e6] Merge remote-tracking branch 'omap/for-next'
> >> git bisect good dd4318312c6fc5c00ae7619f875fb73538a2c1e6
> >> # bad: [2fe482530119a6d07acca625b29d527dc22435e6] Merge remote-tracking branch 'powerpc/next'
> >> git bisect bad 2fe482530119a6d07acca625b29d527dc22435e6
> >> # good: [8576190cecf3370ce07ed04178cb3ecbf17dc2d9] Merge remote-tracking branch 'clk/clk-next'
> >> git bisect good 8576190cecf3370ce07ed04178cb3ecbf17dc2d9
> >> # bad: [bf0505dfeac3f7df81b62d9d81987aaef6739ffd] MIPS: Fix special case in 64 bit IP checksumming.
> >> git bisect bad bf0505dfeac3f7df81b62d9d81987aaef6739ffd
> >> # good: [e89ef66d7682f031f026eee6bba03c8c2248d2a9] MIPS: init: Ensure reserved memory regions are not added to bootmem
> >> git bisect good e89ef66d7682f031f026eee6bba03c8c2248d2a9
> >> # bad: [a8b3b0c94ac282628f0668d1366239a3fa72dc9d] MIPS: Netlogic: Fix assembler warning from smpboot.S
> >> git bisect bad a8b3b0c94ac282628f0668d1366239a3fa72dc9d
> >> # bad: [07724712bf22aec5fe44e5d399f115755f436721] MIPS: ralink: Add missing symbol for highmem support.
> >> git bisect bad 07724712bf22aec5fe44e5d399f115755f436721
> >> # good: [856b0f591e951a234d6642cc466023df182eb51c] MIPS: kexec: add debug info about the new kexec'ed image
> >> git bisect good 856b0f591e951a234d6642cc466023df182eb51c
> >> # bad: [2517caf19dbfac3b39f2db5500c5fd03c4370e81] MIPS: ralink: Add missing I2C and I2S clocks.
> >> git bisect bad 2517caf19dbfac3b39f2db5500c5fd03c4370e81
> >> # bad: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
> >> git bisect bad ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d
> >> # good: [0014dea6b71ae3e0384c3358f632b4728c3d432e] MIPS: generic/kexec: add support for a DTB passed in a separate buffer
> >> git bisect good 0014dea6b71ae3e0384c3358f632b4728c3d432e
> >> # first bad commit: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
> >>
> >>
> >
> 
> 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: Crash in -next due to 'MIPS: Add cacheinfo support'

[James Hogan]

[-- Attachment #1: Type: text/plain, Size: 8320 bytes --]

Hi Guenter,

On Thu, Feb 09, 2017 at 08:50:04PM -0800, Guenter Roeck wrote:
> On 02/09/2017 04:01 PM, Justin Chen wrote:
> > Hello Guenter,
> >
> > I am unable to reproduce the problem. May you give me more details?
> >
> The scripts I am using are available at
> 
> https://github.com/groeck/linux-build-test/tree/master/rootfs
> 
> in the mips and mipsel subdirectories (both crash). Individual
> build logs are always available at kerneltests.org/builders,
> in the 'next' column.

Did you find it 100% reproducible? I was trying to reproduce yesterday
evening, and did hit it a few times, but then it stopped happening
before I could try and figure out what was going wrong.

I presume from your run-qemu-mipsel.sh script its a vanilla 2.7
qemu-system-mipsel?

Cheers
James

> 
> Note: If I build the image with my configuration, and run
> your qemu command, I get a different crash:
> 
> ...
> ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0x1088 irq 15
> CPU 0 Unable to handle kernel paging request at virtual address fffffffc, epc == 80341d68, ra == 803eae70
> Oops[#1]:
> CPU: 0 PID: 1 Comm: swapper Not tainted 4.10.0-rc7-next-20170209 #2
> task: 8704b8a0 task.stack: 8704c000
> $ 0   : 00000000 0000006f fffffffc 00000000
> $ 4   : 87044980 8069ee48 8069ee49 00000004
> $ 8   : 00000000 8741c990 00000000 00000040
> $12   : 00000000 00000001 00000000 fffffffe
> $16   : 8070ae14 80742fb8 80738190 00000000
> $20   : 806e0000 8071c2fc 80770000 00000008
> $24   : 00000110 00000000
> $28   : 8704c000 8704fe10 80754728 803eae70
> Hi    : 0000000f
> Lo    : 8f8f8f9f
> epc   : 80341d68 kset_find_obj+0xc/0xa4
> ra    : 803eae70 driver_find+0x1c/0x48
> Status: 1000a403	KERNEL EXL IE
> Cause : 00800008 (ExcCode 02)
> BadVA : fffffffc
> PrId  : 00019300 (MIPS 24Kc)
> Modules linked in:
> Process swapper (pid: 1, threadinfo=8704c000, task=8704b8a0, tls=00000000)
> Stack : 80710000 00000000 806e0000 8044ba18 8070ae14 803eaf10 00000000 00000000
>          80770000 80742fb8 80770000 80770000 80742fb8 801005dc 8708b900 810017e0
>          8704fe68 806f0000 806ee894 81041c41 80600600 80141fb8 00000000 80740000
>          806e0000 80720000 00000000 00000006 806765e4 00000006 00000017 80670000
>          80676cf0 80680000 80770000 80742fb8 00000006 80742f98 80770000 80742fb8
>          ...
> Call Trace:
> [<80341d68>] kset_find_obj+0xc/0xa4
> [<803eae70>] driver_find+0x1c/0x48
> [<803eaf10>] driver_register+0x74/0x128
> [<801005dc>] do_one_initcall+0x48/0x16c
> [<8071cd18>] kernel_init_freeable+0x170/0x22c
> [<805e1d30>] kernel_init+0x14/0x108
> [<80105e18>] ret_from_kernel_thread+0x14/0x1c
> Code: 8c830000  10830012  2462fffc <8c430000> 1060000c  00a03021  90670000  90c10000  24630001
> 
> ---[ end trace c22dffa22f0b1376 ]---
> Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> 
> ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> 
> Guenter
> 
> > I rebased my patch onto of 4.10-rc7 and compiled with malta defconfig
> > with disabled SMP. Then booted with qemu. Doesn't seem to have an
> > error with the way I have it set up.
> >
> > I am using the command:
> > qemu-system-mipsel -kernel vmlinux -nographic -smp 0
> >
> > Thanks,
> > Justin
> >
> > On Wed, Feb 8, 2017 at 3:45 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> >> Hi Justin,
> >>
> >> The 32-bit qemu mips runtime tests for -next at kerneltests.org are crashing
> >> for non-SMP builds as follows.
> >>
> >> cacheinfo: Failed to find cpu0 device node
> >> cacheinfo: Unable to detect cache hierarchy for CPU 0
> >> CPU 0 Unable to handle kernel paging request at virtual address 00000004, epc ==
> >> 8013c6a8, ra == 8013cb54
> >> Oops[#1]:
> >> CPU: 0 PID: 5 Comm: kworker/u2:0 Not tainted 4.10.0-rc7-next-20170208 #1
> >> Workqueue: events_unbound call_usermodehelper_exec_work
> >> task: 8704a420 task.stack: 87068000
> >> $ 0   : 00000000 80770000 873a8204 8700c214
> >> $ 4   : 00000000 873a8200 00000000 ffff00fe
> >> $ 8   : 8706bfe0 0000a400 fffffffc 8704a459
> >> $12   : 00000000 00000000 00000000 00000720
> >> $16   : 873a8200 87045880 00000000 8700c200
> >> $20   : 87014000 87014005 8700c200 8700c200
> >> $24   : 00000000 8014d1e8
> >> $28   : 87068000 8706be70 87045880 8013cb54
> >> Hi    : 00077698
> >> Lo    : ab801af8
> >> epc   : 8013c6a8 process_one_work+0xd4/0x408
> >> ra    : 8013cb54 worker_thread+0x178/0x598
> >> Status: 1000a402        KERNEL EXL
> >> Cause : 0080000c (ExcCode 03)
> >> BadVA : 00000004
> >> PrId  : 00019300 (MIPS 24Kc)
> >> Modules linked in:
> >> Process kworker/u2:0 (pid: 5, threadinfo=87068000, task=8704a420, tls=00000000)
> >> Stack : 80700000 8700c214 00000088 80700000 8700c200 8700c200 8700c200 87045898
> >>         80700000 8700c214 00000088 80700000 8700c200 8013cb54 8013c9dc 8704fe00
> >>         87045600 87045700 80700000 80700000 80700000 80676560 87045600 87045700
> >>         87045618 87045880 8013c9dc 8704fe00 80670000 80760000 806e0000 80142958
> >>         00000000 00000000 00000000 00000000 80142850 80142850 87045700 8704b380
> >>         ...
> >> Call Trace:
> >> [<8013c6a8>] process_one_work+0xd4/0x408
> >> [<8013cb54>] worker_thread+0x178/0x598
> >> [<80142958>] kthread+0x108/0x138
> >> [<80105e18>] ret_from_kernel_thread+0x14/0x1c
> >> Code: 8e030008  8e040004  8e150000 <ac830004> 7eb51900  ac640000  ae020004 16400094  ae020008
> >>
> >> Bisect points to 'MIPS: Add cacheinfo support'; see below for details.
> >> Reverting this patch fixes te problem.
> >>
> >> Configuration is pretty much malta_defconfig with SMP disabled.
> >> I can provide more details if needed.
> >>
> >> Thanks,
> >> Guenter
> >>
> >> ---
> >> # bad: [e3e6c5f3544c5d05c6b3b309a34f4f2c3537e993] Add linux-next specific files for 20170208
> >> # good: [d5adbfcd5f7bcc6fa58a41c5c5ada0e5c826ce2c] Linux 4.10-rc7
> >> git bisect start 'HEAD' 'v4.10-rc7'
> >> # bad: [403e468309f9e2b2dbe264be1ad29b1486ed720e] Merge remote-tracking branch 'crypto/master'
> >> git bisect bad 403e468309f9e2b2dbe264be1ad29b1486ed720e
> >> # bad: [44448c3f07a3ae77164de4405fa97baca4f103f5] Merge remote-tracking branch 'hid/for-next'
> >> git bisect bad 44448c3f07a3ae77164de4405fa97baca4f103f5
> >> # good: [dd4318312c6fc5c00ae7619f875fb73538a2c1e6] Merge remote-tracking branch 'omap/for-next'
> >> git bisect good dd4318312c6fc5c00ae7619f875fb73538a2c1e6
> >> # bad: [2fe482530119a6d07acca625b29d527dc22435e6] Merge remote-tracking branch 'powerpc/next'
> >> git bisect bad 2fe482530119a6d07acca625b29d527dc22435e6
> >> # good: [8576190cecf3370ce07ed04178cb3ecbf17dc2d9] Merge remote-tracking branch 'clk/clk-next'
> >> git bisect good 8576190cecf3370ce07ed04178cb3ecbf17dc2d9
> >> # bad: [bf0505dfeac3f7df81b62d9d81987aaef6739ffd] MIPS: Fix special case in 64 bit IP checksumming.
> >> git bisect bad bf0505dfeac3f7df81b62d9d81987aaef6739ffd
> >> # good: [e89ef66d7682f031f026eee6bba03c8c2248d2a9] MIPS: init: Ensure reserved memory regions are not added to bootmem
> >> git bisect good e89ef66d7682f031f026eee6bba03c8c2248d2a9
> >> # bad: [a8b3b0c94ac282628f0668d1366239a3fa72dc9d] MIPS: Netlogic: Fix assembler warning from smpboot.S
> >> git bisect bad a8b3b0c94ac282628f0668d1366239a3fa72dc9d
> >> # bad: [07724712bf22aec5fe44e5d399f115755f436721] MIPS: ralink: Add missing symbol for highmem support.
> >> git bisect bad 07724712bf22aec5fe44e5d399f115755f436721
> >> # good: [856b0f591e951a234d6642cc466023df182eb51c] MIPS: kexec: add debug info about the new kexec'ed image
> >> git bisect good 856b0f591e951a234d6642cc466023df182eb51c
> >> # bad: [2517caf19dbfac3b39f2db5500c5fd03c4370e81] MIPS: ralink: Add missing I2C and I2S clocks.
> >> git bisect bad 2517caf19dbfac3b39f2db5500c5fd03c4370e81
> >> # bad: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
> >> git bisect bad ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d
> >> # good: [0014dea6b71ae3e0384c3358f632b4728c3d432e] MIPS: generic/kexec: add support for a DTB passed in a separate buffer
> >> git bisect good 0014dea6b71ae3e0384c3358f632b4728c3d432e
> >> # first bad commit: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
> >>
> >>
> >
> 
> 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: Crash in -next due to 'MIPS: Add cacheinfo support'

[James Hogan]

[-- Attachment #1: Type: text/plain, Size: 8864 bytes --]

On Fri, Feb 10, 2017 at 09:40:11AM +0000, James Hogan wrote:
> Hi Guenter,
> 
> On Thu, Feb 09, 2017 at 08:50:04PM -0800, Guenter Roeck wrote:
> > On 02/09/2017 04:01 PM, Justin Chen wrote:
> > > Hello Guenter,
> > >
> > > I am unable to reproduce the problem. May you give me more details?
> > >
> > The scripts I am using are available at
> > 
> > https://github.com/groeck/linux-build-test/tree/master/rootfs
> > 
> > in the mips and mipsel subdirectories (both crash). Individual
> > build logs are always available at kerneltests.org/builders,
> > in the 'next' column.
> 
> Did you find it 100% reproducible? I was trying to reproduce yesterday
> evening, and did hit it a few times, but then it stopped happening
> before I could try and figure out what was going wrong.

I switched to gcc 5.2 (buildroot toolchain) for building kernel, and now
it reproduces every time :)

Cheers
James

> 
> I presume from your run-qemu-mipsel.sh script its a vanilla 2.7
> qemu-system-mipsel?
> 
> Cheers
> James
> 
> > 
> > Note: If I build the image with my configuration, and run
> > your qemu command, I get a different crash:
> > 
> > ...
> > ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0x1088 irq 15
> > CPU 0 Unable to handle kernel paging request at virtual address fffffffc, epc == 80341d68, ra == 803eae70
> > Oops[#1]:
> > CPU: 0 PID: 1 Comm: swapper Not tainted 4.10.0-rc7-next-20170209 #2
> > task: 8704b8a0 task.stack: 8704c000
> > $ 0   : 00000000 0000006f fffffffc 00000000
> > $ 4   : 87044980 8069ee48 8069ee49 00000004
> > $ 8   : 00000000 8741c990 00000000 00000040
> > $12   : 00000000 00000001 00000000 fffffffe
> > $16   : 8070ae14 80742fb8 80738190 00000000
> > $20   : 806e0000 8071c2fc 80770000 00000008
> > $24   : 00000110 00000000
> > $28   : 8704c000 8704fe10 80754728 803eae70
> > Hi    : 0000000f
> > Lo    : 8f8f8f9f
> > epc   : 80341d68 kset_find_obj+0xc/0xa4
> > ra    : 803eae70 driver_find+0x1c/0x48
> > Status: 1000a403	KERNEL EXL IE
> > Cause : 00800008 (ExcCode 02)
> > BadVA : fffffffc
> > PrId  : 00019300 (MIPS 24Kc)
> > Modules linked in:
> > Process swapper (pid: 1, threadinfo=8704c000, task=8704b8a0, tls=00000000)
> > Stack : 80710000 00000000 806e0000 8044ba18 8070ae14 803eaf10 00000000 00000000
> >          80770000 80742fb8 80770000 80770000 80742fb8 801005dc 8708b900 810017e0
> >          8704fe68 806f0000 806ee894 81041c41 80600600 80141fb8 00000000 80740000
> >          806e0000 80720000 00000000 00000006 806765e4 00000006 00000017 80670000
> >          80676cf0 80680000 80770000 80742fb8 00000006 80742f98 80770000 80742fb8
> >          ...
> > Call Trace:
> > [<80341d68>] kset_find_obj+0xc/0xa4
> > [<803eae70>] driver_find+0x1c/0x48
> > [<803eaf10>] driver_register+0x74/0x128
> > [<801005dc>] do_one_initcall+0x48/0x16c
> > [<8071cd18>] kernel_init_freeable+0x170/0x22c
> > [<805e1d30>] kernel_init+0x14/0x108
> > [<80105e18>] ret_from_kernel_thread+0x14/0x1c
> > Code: 8c830000  10830012  2462fffc <8c430000> 1060000c  00a03021  90670000  90c10000  24630001
> > 
> > ---[ end trace c22dffa22f0b1376 ]---
> > Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> > 
> > ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> > 
> > Guenter
> > 
> > > I rebased my patch onto of 4.10-rc7 and compiled with malta defconfig
> > > with disabled SMP. Then booted with qemu. Doesn't seem to have an
> > > error with the way I have it set up.
> > >
> > > I am using the command:
> > > qemu-system-mipsel -kernel vmlinux -nographic -smp 0
> > >
> > > Thanks,
> > > Justin
> > >
> > > On Wed, Feb 8, 2017 at 3:45 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> > >> Hi Justin,
> > >>
> > >> The 32-bit qemu mips runtime tests for -next at kerneltests.org are crashing
> > >> for non-SMP builds as follows.
> > >>
> > >> cacheinfo: Failed to find cpu0 device node
> > >> cacheinfo: Unable to detect cache hierarchy for CPU 0
> > >> CPU 0 Unable to handle kernel paging request at virtual address 00000004, epc ==
> > >> 8013c6a8, ra == 8013cb54
> > >> Oops[#1]:
> > >> CPU: 0 PID: 5 Comm: kworker/u2:0 Not tainted 4.10.0-rc7-next-20170208 #1
> > >> Workqueue: events_unbound call_usermodehelper_exec_work
> > >> task: 8704a420 task.stack: 87068000
> > >> $ 0   : 00000000 80770000 873a8204 8700c214
> > >> $ 4   : 00000000 873a8200 00000000 ffff00fe
> > >> $ 8   : 8706bfe0 0000a400 fffffffc 8704a459
> > >> $12   : 00000000 00000000 00000000 00000720
> > >> $16   : 873a8200 87045880 00000000 8700c200
> > >> $20   : 87014000 87014005 8700c200 8700c200
> > >> $24   : 00000000 8014d1e8
> > >> $28   : 87068000 8706be70 87045880 8013cb54
> > >> Hi    : 00077698
> > >> Lo    : ab801af8
> > >> epc   : 8013c6a8 process_one_work+0xd4/0x408
> > >> ra    : 8013cb54 worker_thread+0x178/0x598
> > >> Status: 1000a402        KERNEL EXL
> > >> Cause : 0080000c (ExcCode 03)
> > >> BadVA : 00000004
> > >> PrId  : 00019300 (MIPS 24Kc)
> > >> Modules linked in:
> > >> Process kworker/u2:0 (pid: 5, threadinfo=87068000, task=8704a420, tls=00000000)
> > >> Stack : 80700000 8700c214 00000088 80700000 8700c200 8700c200 8700c200 87045898
> > >>         80700000 8700c214 00000088 80700000 8700c200 8013cb54 8013c9dc 8704fe00
> > >>         87045600 87045700 80700000 80700000 80700000 80676560 87045600 87045700
> > >>         87045618 87045880 8013c9dc 8704fe00 80670000 80760000 806e0000 80142958
> > >>         00000000 00000000 00000000 00000000 80142850 80142850 87045700 8704b380
> > >>         ...
> > >> Call Trace:
> > >> [<8013c6a8>] process_one_work+0xd4/0x408
> > >> [<8013cb54>] worker_thread+0x178/0x598
> > >> [<80142958>] kthread+0x108/0x138
> > >> [<80105e18>] ret_from_kernel_thread+0x14/0x1c
> > >> Code: 8e030008  8e040004  8e150000 <ac830004> 7eb51900  ac640000  ae020004 16400094  ae020008
> > >>
> > >> Bisect points to 'MIPS: Add cacheinfo support'; see below for details.
> > >> Reverting this patch fixes te problem.
> > >>
> > >> Configuration is pretty much malta_defconfig with SMP disabled.
> > >> I can provide more details if needed.
> > >>
> > >> Thanks,
> > >> Guenter
> > >>
> > >> ---
> > >> # bad: [e3e6c5f3544c5d05c6b3b309a34f4f2c3537e993] Add linux-next specific files for 20170208
> > >> # good: [d5adbfcd5f7bcc6fa58a41c5c5ada0e5c826ce2c] Linux 4.10-rc7
> > >> git bisect start 'HEAD' 'v4.10-rc7'
> > >> # bad: [403e468309f9e2b2dbe264be1ad29b1486ed720e] Merge remote-tracking branch 'crypto/master'
> > >> git bisect bad 403e468309f9e2b2dbe264be1ad29b1486ed720e
> > >> # bad: [44448c3f07a3ae77164de4405fa97baca4f103f5] Merge remote-tracking branch 'hid/for-next'
> > >> git bisect bad 44448c3f07a3ae77164de4405fa97baca4f103f5
> > >> # good: [dd4318312c6fc5c00ae7619f875fb73538a2c1e6] Merge remote-tracking branch 'omap/for-next'
> > >> git bisect good dd4318312c6fc5c00ae7619f875fb73538a2c1e6
> > >> # bad: [2fe482530119a6d07acca625b29d527dc22435e6] Merge remote-tracking branch 'powerpc/next'
> > >> git bisect bad 2fe482530119a6d07acca625b29d527dc22435e6
> > >> # good: [8576190cecf3370ce07ed04178cb3ecbf17dc2d9] Merge remote-tracking branch 'clk/clk-next'
> > >> git bisect good 8576190cecf3370ce07ed04178cb3ecbf17dc2d9
> > >> # bad: [bf0505dfeac3f7df81b62d9d81987aaef6739ffd] MIPS: Fix special case in 64 bit IP checksumming.
> > >> git bisect bad bf0505dfeac3f7df81b62d9d81987aaef6739ffd
> > >> # good: [e89ef66d7682f031f026eee6bba03c8c2248d2a9] MIPS: init: Ensure reserved memory regions are not added to bootmem
> > >> git bisect good e89ef66d7682f031f026eee6bba03c8c2248d2a9
> > >> # bad: [a8b3b0c94ac282628f0668d1366239a3fa72dc9d] MIPS: Netlogic: Fix assembler warning from smpboot.S
> > >> git bisect bad a8b3b0c94ac282628f0668d1366239a3fa72dc9d
> > >> # bad: [07724712bf22aec5fe44e5d399f115755f436721] MIPS: ralink: Add missing symbol for highmem support.
> > >> git bisect bad 07724712bf22aec5fe44e5d399f115755f436721
> > >> # good: [856b0f591e951a234d6642cc466023df182eb51c] MIPS: kexec: add debug info about the new kexec'ed image
> > >> git bisect good 856b0f591e951a234d6642cc466023df182eb51c
> > >> # bad: [2517caf19dbfac3b39f2db5500c5fd03c4370e81] MIPS: ralink: Add missing I2C and I2S clocks.
> > >> git bisect bad 2517caf19dbfac3b39f2db5500c5fd03c4370e81
> > >> # bad: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
> > >> git bisect bad ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d
> > >> # good: [0014dea6b71ae3e0384c3358f632b4728c3d432e] MIPS: generic/kexec: add support for a DTB passed in a separate buffer
> > >> git bisect good 0014dea6b71ae3e0384c3358f632b4728c3d432e
> > >> # first bad commit: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
> > >>
> > >>
> > >
> > 
> > 



[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: Crash in -next due to 'MIPS: Add cacheinfo support'

[James Hogan]

[-- Attachment #1: Type: text/plain, Size: 8864 bytes --]

On Fri, Feb 10, 2017 at 09:40:11AM +0000, James Hogan wrote:
> Hi Guenter,
> 
> On Thu, Feb 09, 2017 at 08:50:04PM -0800, Guenter Roeck wrote:
> > On 02/09/2017 04:01 PM, Justin Chen wrote:
> > > Hello Guenter,
> > >
> > > I am unable to reproduce the problem. May you give me more details?
> > >
> > The scripts I am using are available at
> > 
> > https://github.com/groeck/linux-build-test/tree/master/rootfs
> > 
> > in the mips and mipsel subdirectories (both crash). Individual
> > build logs are always available at kerneltests.org/builders,
> > in the 'next' column.
> 
> Did you find it 100% reproducible? I was trying to reproduce yesterday
> evening, and did hit it a few times, but then it stopped happening
> before I could try and figure out what was going wrong.

I switched to gcc 5.2 (buildroot toolchain) for building kernel, and now
it reproduces every time :)

Cheers
James

> 
> I presume from your run-qemu-mipsel.sh script its a vanilla 2.7
> qemu-system-mipsel?
> 
> Cheers
> James
> 
> > 
> > Note: If I build the image with my configuration, and run
> > your qemu command, I get a different crash:
> > 
> > ...
> > ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0x1088 irq 15
> > CPU 0 Unable to handle kernel paging request at virtual address fffffffc, epc == 80341d68, ra == 803eae70
> > Oops[#1]:
> > CPU: 0 PID: 1 Comm: swapper Not tainted 4.10.0-rc7-next-20170209 #2
> > task: 8704b8a0 task.stack: 8704c000
> > $ 0   : 00000000 0000006f fffffffc 00000000
> > $ 4   : 87044980 8069ee48 8069ee49 00000004
> > $ 8   : 00000000 8741c990 00000000 00000040
> > $12   : 00000000 00000001 00000000 fffffffe
> > $16   : 8070ae14 80742fb8 80738190 00000000
> > $20   : 806e0000 8071c2fc 80770000 00000008
> > $24   : 00000110 00000000
> > $28   : 8704c000 8704fe10 80754728 803eae70
> > Hi    : 0000000f
> > Lo    : 8f8f8f9f
> > epc   : 80341d68 kset_find_obj+0xc/0xa4
> > ra    : 803eae70 driver_find+0x1c/0x48
> > Status: 1000a403	KERNEL EXL IE
> > Cause : 00800008 (ExcCode 02)
> > BadVA : fffffffc
> > PrId  : 00019300 (MIPS 24Kc)
> > Modules linked in:
> > Process swapper (pid: 1, threadinfo=8704c000, task=8704b8a0, tls=00000000)
> > Stack : 80710000 00000000 806e0000 8044ba18 8070ae14 803eaf10 00000000 00000000
> >          80770000 80742fb8 80770000 80770000 80742fb8 801005dc 8708b900 810017e0
> >          8704fe68 806f0000 806ee894 81041c41 80600600 80141fb8 00000000 80740000
> >          806e0000 80720000 00000000 00000006 806765e4 00000006 00000017 80670000
> >          80676cf0 80680000 80770000 80742fb8 00000006 80742f98 80770000 80742fb8
> >          ...
> > Call Trace:
> > [<80341d68>] kset_find_obj+0xc/0xa4
> > [<803eae70>] driver_find+0x1c/0x48
> > [<803eaf10>] driver_register+0x74/0x128
> > [<801005dc>] do_one_initcall+0x48/0x16c
> > [<8071cd18>] kernel_init_freeable+0x170/0x22c
> > [<805e1d30>] kernel_init+0x14/0x108
> > [<80105e18>] ret_from_kernel_thread+0x14/0x1c
> > Code: 8c830000  10830012  2462fffc <8c430000> 1060000c  00a03021  90670000  90c10000  24630001
> > 
> > ---[ end trace c22dffa22f0b1376 ]---
> > Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> > 
> > ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> > 
> > Guenter
> > 
> > > I rebased my patch onto of 4.10-rc7 and compiled with malta defconfig
> > > with disabled SMP. Then booted with qemu. Doesn't seem to have an
> > > error with the way I have it set up.
> > >
> > > I am using the command:
> > > qemu-system-mipsel -kernel vmlinux -nographic -smp 0
> > >
> > > Thanks,
> > > Justin
> > >
> > > On Wed, Feb 8, 2017 at 3:45 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> > >> Hi Justin,
> > >>
> > >> The 32-bit qemu mips runtime tests for -next at kerneltests.org are crashing
> > >> for non-SMP builds as follows.
> > >>
> > >> cacheinfo: Failed to find cpu0 device node
> > >> cacheinfo: Unable to detect cache hierarchy for CPU 0
> > >> CPU 0 Unable to handle kernel paging request at virtual address 00000004, epc ==
> > >> 8013c6a8, ra == 8013cb54
> > >> Oops[#1]:
> > >> CPU: 0 PID: 5 Comm: kworker/u2:0 Not tainted 4.10.0-rc7-next-20170208 #1
> > >> Workqueue: events_unbound call_usermodehelper_exec_work
> > >> task: 8704a420 task.stack: 87068000
> > >> $ 0   : 00000000 80770000 873a8204 8700c214
> > >> $ 4   : 00000000 873a8200 00000000 ffff00fe
> > >> $ 8   : 8706bfe0 0000a400 fffffffc 8704a459
> > >> $12   : 00000000 00000000 00000000 00000720
> > >> $16   : 873a8200 87045880 00000000 8700c200
> > >> $20   : 87014000 87014005 8700c200 8700c200
> > >> $24   : 00000000 8014d1e8
> > >> $28   : 87068000 8706be70 87045880 8013cb54
> > >> Hi    : 00077698
> > >> Lo    : ab801af8
> > >> epc   : 8013c6a8 process_one_work+0xd4/0x408
> > >> ra    : 8013cb54 worker_thread+0x178/0x598
> > >> Status: 1000a402        KERNEL EXL
> > >> Cause : 0080000c (ExcCode 03)
> > >> BadVA : 00000004
> > >> PrId  : 00019300 (MIPS 24Kc)
> > >> Modules linked in:
> > >> Process kworker/u2:0 (pid: 5, threadinfo=87068000, task=8704a420, tls=00000000)
> > >> Stack : 80700000 8700c214 00000088 80700000 8700c200 8700c200 8700c200 87045898
> > >>         80700000 8700c214 00000088 80700000 8700c200 8013cb54 8013c9dc 8704fe00
> > >>         87045600 87045700 80700000 80700000 80700000 80676560 87045600 87045700
> > >>         87045618 87045880 8013c9dc 8704fe00 80670000 80760000 806e0000 80142958
> > >>         00000000 00000000 00000000 00000000 80142850 80142850 87045700 8704b380
> > >>         ...
> > >> Call Trace:
> > >> [<8013c6a8>] process_one_work+0xd4/0x408
> > >> [<8013cb54>] worker_thread+0x178/0x598
> > >> [<80142958>] kthread+0x108/0x138
> > >> [<80105e18>] ret_from_kernel_thread+0x14/0x1c
> > >> Code: 8e030008  8e040004  8e150000 <ac830004> 7eb51900  ac640000  ae020004 16400094  ae020008
> > >>
> > >> Bisect points to 'MIPS: Add cacheinfo support'; see below for details.
> > >> Reverting this patch fixes te problem.
> > >>
> > >> Configuration is pretty much malta_defconfig with SMP disabled.
> > >> I can provide more details if needed.
> > >>
> > >> Thanks,
> > >> Guenter
> > >>
> > >> ---
> > >> # bad: [e3e6c5f3544c5d05c6b3b309a34f4f2c3537e993] Add linux-next specific files for 20170208
> > >> # good: [d5adbfcd5f7bcc6fa58a41c5c5ada0e5c826ce2c] Linux 4.10-rc7
> > >> git bisect start 'HEAD' 'v4.10-rc7'
> > >> # bad: [403e468309f9e2b2dbe264be1ad29b1486ed720e] Merge remote-tracking branch 'crypto/master'
> > >> git bisect bad 403e468309f9e2b2dbe264be1ad29b1486ed720e
> > >> # bad: [44448c3f07a3ae77164de4405fa97baca4f103f5] Merge remote-tracking branch 'hid/for-next'
> > >> git bisect bad 44448c3f07a3ae77164de4405fa97baca4f103f5
> > >> # good: [dd4318312c6fc5c00ae7619f875fb73538a2c1e6] Merge remote-tracking branch 'omap/for-next'
> > >> git bisect good dd4318312c6fc5c00ae7619f875fb73538a2c1e6
> > >> # bad: [2fe482530119a6d07acca625b29d527dc22435e6] Merge remote-tracking branch 'powerpc/next'
> > >> git bisect bad 2fe482530119a6d07acca625b29d527dc22435e6
> > >> # good: [8576190cecf3370ce07ed04178cb3ecbf17dc2d9] Merge remote-tracking branch 'clk/clk-next'
> > >> git bisect good 8576190cecf3370ce07ed04178cb3ecbf17dc2d9
> > >> # bad: [bf0505dfeac3f7df81b62d9d81987aaef6739ffd] MIPS: Fix special case in 64 bit IP checksumming.
> > >> git bisect bad bf0505dfeac3f7df81b62d9d81987aaef6739ffd
> > >> # good: [e89ef66d7682f031f026eee6bba03c8c2248d2a9] MIPS: init: Ensure reserved memory regions are not added to bootmem
> > >> git bisect good e89ef66d7682f031f026eee6bba03c8c2248d2a9
> > >> # bad: [a8b3b0c94ac282628f0668d1366239a3fa72dc9d] MIPS: Netlogic: Fix assembler warning from smpboot.S
> > >> git bisect bad a8b3b0c94ac282628f0668d1366239a3fa72dc9d
> > >> # bad: [07724712bf22aec5fe44e5d399f115755f436721] MIPS: ralink: Add missing symbol for highmem support.
> > >> git bisect bad 07724712bf22aec5fe44e5d399f115755f436721
> > >> # good: [856b0f591e951a234d6642cc466023df182eb51c] MIPS: kexec: add debug info about the new kexec'ed image
> > >> git bisect good 856b0f591e951a234d6642cc466023df182eb51c
> > >> # bad: [2517caf19dbfac3b39f2db5500c5fd03c4370e81] MIPS: ralink: Add missing I2C and I2S clocks.
> > >> git bisect bad 2517caf19dbfac3b39f2db5500c5fd03c4370e81
> > >> # bad: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
> > >> git bisect bad ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d
> > >> # good: [0014dea6b71ae3e0384c3358f632b4728c3d432e] MIPS: generic/kexec: add support for a DTB passed in a separate buffer
> > >> git bisect good 0014dea6b71ae3e0384c3358f632b4728c3d432e
> > >> # first bad commit: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
> > >>
> > >>
> > >
> > 
> > 



[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: Crash in -next due to 'MIPS: Add cacheinfo support'

[Guenter Roeck]

Hi James,

On 02/10/2017 01:40 AM, James Hogan wrote:
> Hi Guenter,
>
> On Thu, Feb 09, 2017 at 08:50:04PM -0800, Guenter Roeck wrote:
>> On 02/09/2017 04:01 PM, Justin Chen wrote:
>>> Hello Guenter,
>>>
>>> I am unable to reproduce the problem. May you give me more details?
>>>
>> The scripts I am using are available at
>>
>> https://github.com/groeck/linux-build-test/tree/master/rootfs
>>
>> in the mips and mipsel subdirectories (both crash). Individual
>> build logs are always available at kerneltests.org/builders,
>> in the 'next' column.
>
> Did you find it 100% reproducible? I was trying to reproduce yesterday
> evening, and did hit it a few times, but then it stopped happening
> before I could try and figure out what was going wrong.
>
> I presume from your run-qemu-mipsel.sh script its a vanilla 2.7
> qemu-system-mipsel?
>

My builds are with qemu from https://github.com/groeck/qemu,
built with https://github.com/groeck/linux-build-test/blob/master/qemu/buildall.sh

No mips specific changes. I'll be happy to test with vanilla 2.7 or 2.8 if that
eases people's mind. The later log below is with vanilla qemu 2.5. gcc version
used to build the kernel is 5.2.0 from poky 2.0.

Guenter

> Cheers
> James
>
>>
>> Note: If I build the image with my configuration, and run
>> your qemu command, I get a different crash:
>>
>> ...
>> ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0x1088 irq 15
>> CPU 0 Unable to handle kernel paging request at virtual address fffffffc, epc == 80341d68, ra == 803eae70
>> Oops[#1]:
>> CPU: 0 PID: 1 Comm: swapper Not tainted 4.10.0-rc7-next-20170209 #2
>> task: 8704b8a0 task.stack: 8704c000
>> $ 0   : 00000000 0000006f fffffffc 00000000
>> $ 4   : 87044980 8069ee48 8069ee49 00000004
>> $ 8   : 00000000 8741c990 00000000 00000040
>> $12   : 00000000 00000001 00000000 fffffffe
>> $16   : 8070ae14 80742fb8 80738190 00000000
>> $20   : 806e0000 8071c2fc 80770000 00000008
>> $24   : 00000110 00000000
>> $28   : 8704c000 8704fe10 80754728 803eae70
>> Hi    : 0000000f
>> Lo    : 8f8f8f9f
>> epc   : 80341d68 kset_find_obj+0xc/0xa4
>> ra    : 803eae70 driver_find+0x1c/0x48
>> Status: 1000a403	KERNEL EXL IE
>> Cause : 00800008 (ExcCode 02)
>> BadVA : fffffffc
>> PrId  : 00019300 (MIPS 24Kc)
>> Modules linked in:
>> Process swapper (pid: 1, threadinfo=8704c000, task=8704b8a0, tls=00000000)
>> Stack : 80710000 00000000 806e0000 8044ba18 8070ae14 803eaf10 00000000 00000000
>>          80770000 80742fb8 80770000 80770000 80742fb8 801005dc 8708b900 810017e0
>>          8704fe68 806f0000 806ee894 81041c41 80600600 80141fb8 00000000 80740000
>>          806e0000 80720000 00000000 00000006 806765e4 00000006 00000017 80670000
>>          80676cf0 80680000 80770000 80742fb8 00000006 80742f98 80770000 80742fb8
>>          ...
>> Call Trace:
>> [<80341d68>] kset_find_obj+0xc/0xa4
>> [<803eae70>] driver_find+0x1c/0x48
>> [<803eaf10>] driver_register+0x74/0x128
>> [<801005dc>] do_one_initcall+0x48/0x16c
>> [<8071cd18>] kernel_init_freeable+0x170/0x22c
>> [<805e1d30>] kernel_init+0x14/0x108
>> [<80105e18>] ret_from_kernel_thread+0x14/0x1c
>> Code: 8c830000  10830012  2462fffc <8c430000> 1060000c  00a03021  90670000  90c10000  24630001
>>
>> ---[ end trace c22dffa22f0b1376 ]---
>> Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
>>
>> ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
>>
>> Guenter
>>
>>> I rebased my patch onto of 4.10-rc7 and compiled with malta defconfig
>>> with disabled SMP. Then booted with qemu. Doesn't seem to have an
>>> error with the way I have it set up.
>>>
>>> I am using the command:
>>> qemu-system-mipsel -kernel vmlinux -nographic -smp 0
>>>
>>> Thanks,
>>> Justin
>>>
>>> On Wed, Feb 8, 2017 at 3:45 PM, Guenter Roeck <linux@roeck-us.net> wrote:
>>>> Hi Justin,
>>>>
>>>> The 32-bit qemu mips runtime tests for -next at kerneltests.org are crashing
>>>> for non-SMP builds as follows.
>>>>
>>>> cacheinfo: Failed to find cpu0 device node
>>>> cacheinfo: Unable to detect cache hierarchy for CPU 0
>>>> CPU 0 Unable to handle kernel paging request at virtual address 00000004, epc ==
>>>> 8013c6a8, ra == 8013cb54
>>>> Oops[#1]:
>>>> CPU: 0 PID: 5 Comm: kworker/u2:0 Not tainted 4.10.0-rc7-next-20170208 #1
>>>> Workqueue: events_unbound call_usermodehelper_exec_work
>>>> task: 8704a420 task.stack: 87068000
>>>> $ 0   : 00000000 80770000 873a8204 8700c214
>>>> $ 4   : 00000000 873a8200 00000000 ffff00fe
>>>> $ 8   : 8706bfe0 0000a400 fffffffc 8704a459
>>>> $12   : 00000000 00000000 00000000 00000720
>>>> $16   : 873a8200 87045880 00000000 8700c200
>>>> $20   : 87014000 87014005 8700c200 8700c200
>>>> $24   : 00000000 8014d1e8
>>>> $28   : 87068000 8706be70 87045880 8013cb54
>>>> Hi    : 00077698
>>>> Lo    : ab801af8
>>>> epc   : 8013c6a8 process_one_work+0xd4/0x408
>>>> ra    : 8013cb54 worker_thread+0x178/0x598
>>>> Status: 1000a402        KERNEL EXL
>>>> Cause : 0080000c (ExcCode 03)
>>>> BadVA : 00000004
>>>> PrId  : 00019300 (MIPS 24Kc)
>>>> Modules linked in:
>>>> Process kworker/u2:0 (pid: 5, threadinfo=87068000, task=8704a420, tls=00000000)
>>>> Stack : 80700000 8700c214 00000088 80700000 8700c200 8700c200 8700c200 87045898
>>>>         80700000 8700c214 00000088 80700000 8700c200 8013cb54 8013c9dc 8704fe00
>>>>         87045600 87045700 80700000 80700000 80700000 80676560 87045600 87045700
>>>>         87045618 87045880 8013c9dc 8704fe00 80670000 80760000 806e0000 80142958
>>>>         00000000 00000000 00000000 00000000 80142850 80142850 87045700 8704b380
>>>>         ...
>>>> Call Trace:
>>>> [<8013c6a8>] process_one_work+0xd4/0x408
>>>> [<8013cb54>] worker_thread+0x178/0x598
>>>> [<80142958>] kthread+0x108/0x138
>>>> [<80105e18>] ret_from_kernel_thread+0x14/0x1c
>>>> Code: 8e030008  8e040004  8e150000 <ac830004> 7eb51900  ac640000  ae020004 16400094  ae020008
>>>>
>>>> Bisect points to 'MIPS: Add cacheinfo support'; see below for details.
>>>> Reverting this patch fixes te problem.
>>>>
>>>> Configuration is pretty much malta_defconfig with SMP disabled.
>>>> I can provide more details if needed.
>>>>
>>>> Thanks,
>>>> Guenter
>>>>
>>>> ---
>>>> # bad: [e3e6c5f3544c5d05c6b3b309a34f4f2c3537e993] Add linux-next specific files for 20170208
>>>> # good: [d5adbfcd5f7bcc6fa58a41c5c5ada0e5c826ce2c] Linux 4.10-rc7
>>>> git bisect start 'HEAD' 'v4.10-rc7'
>>>> # bad: [403e468309f9e2b2dbe264be1ad29b1486ed720e] Merge remote-tracking branch 'crypto/master'
>>>> git bisect bad 403e468309f9e2b2dbe264be1ad29b1486ed720e
>>>> # bad: [44448c3f07a3ae77164de4405fa97baca4f103f5] Merge remote-tracking branch 'hid/for-next'
>>>> git bisect bad 44448c3f07a3ae77164de4405fa97baca4f103f5
>>>> # good: [dd4318312c6fc5c00ae7619f875fb73538a2c1e6] Merge remote-tracking branch 'omap/for-next'
>>>> git bisect good dd4318312c6fc5c00ae7619f875fb73538a2c1e6
>>>> # bad: [2fe482530119a6d07acca625b29d527dc22435e6] Merge remote-tracking branch 'powerpc/next'
>>>> git bisect bad 2fe482530119a6d07acca625b29d527dc22435e6
>>>> # good: [8576190cecf3370ce07ed04178cb3ecbf17dc2d9] Merge remote-tracking branch 'clk/clk-next'
>>>> git bisect good 8576190cecf3370ce07ed04178cb3ecbf17dc2d9
>>>> # bad: [bf0505dfeac3f7df81b62d9d81987aaef6739ffd] MIPS: Fix special case in 64 bit IP checksumming.
>>>> git bisect bad bf0505dfeac3f7df81b62d9d81987aaef6739ffd
>>>> # good: [e89ef66d7682f031f026eee6bba03c8c2248d2a9] MIPS: init: Ensure reserved memory regions are not added to bootmem
>>>> git bisect good e89ef66d7682f031f026eee6bba03c8c2248d2a9
>>>> # bad: [a8b3b0c94ac282628f0668d1366239a3fa72dc9d] MIPS: Netlogic: Fix assembler warning from smpboot.S
>>>> git bisect bad a8b3b0c94ac282628f0668d1366239a3fa72dc9d
>>>> # bad: [07724712bf22aec5fe44e5d399f115755f436721] MIPS: ralink: Add missing symbol for highmem support.
>>>> git bisect bad 07724712bf22aec5fe44e5d399f115755f436721
>>>> # good: [856b0f591e951a234d6642cc466023df182eb51c] MIPS: kexec: add debug info about the new kexec'ed image
>>>> git bisect good 856b0f591e951a234d6642cc466023df182eb51c
>>>> # bad: [2517caf19dbfac3b39f2db5500c5fd03c4370e81] MIPS: ralink: Add missing I2C and I2S clocks.
>>>> git bisect bad 2517caf19dbfac3b39f2db5500c5fd03c4370e81
>>>> # bad: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
>>>> git bisect bad ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d
>>>> # good: [0014dea6b71ae3e0384c3358f632b4728c3d432e] MIPS: generic/kexec: add support for a DTB passed in a separate buffer
>>>> git bisect good 0014dea6b71ae3e0384c3358f632b4728c3d432e
>>>> # first bad commit: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
>>>>
>>>>
>>>
>>
>>

Re: Crash in -next due to 'MIPS: Add cacheinfo support'

[Guenter Roeck]

On Fri, Feb 10, 2017 at 10:39:52AM +0000, James Hogan wrote:
> On Fri, Feb 10, 2017 at 09:40:11AM +0000, James Hogan wrote:
> > Hi Guenter,
> > 
> > On Thu, Feb 09, 2017 at 08:50:04PM -0800, Guenter Roeck wrote:
> > > On 02/09/2017 04:01 PM, Justin Chen wrote:
> > > > Hello Guenter,
> > > >
> > > > I am unable to reproduce the problem. May you give me more details?
> > > >
> > > The scripts I am using are available at
> > > 
> > > https://github.com/groeck/linux-build-test/tree/master/rootfs
> > > 
> > > in the mips and mipsel subdirectories (both crash). Individual
> > > build logs are always available at kerneltests.org/builders,
> > > in the 'next' column.
> > 
> > Did you find it 100% reproducible? I was trying to reproduce yesterday
> > evening, and did hit it a few times, but then it stopped happening
> > before I could try and figure out what was going wrong.
> 
> I switched to gcc 5.2 (buildroot toolchain) for building kernel, and now
> it reproduces every time :)
> 
gcc 5.4.0 (binutils 2.26.1) also reliably crashes. The exact crash depends on
the kernel (-next is different to ToT + offending patch, qemu command line
makes a difference, qemu version makes a difference), but the crash is easy
to reproduce, at least for me.

Guenter

> Cheers
> James
> 
> > 
> > I presume from your run-qemu-mipsel.sh script its a vanilla 2.7
> > qemu-system-mipsel?
> > 
> > Cheers
> > James
> > 
> > > 
> > > Note: If I build the image with my configuration, and run
> > > your qemu command, I get a different crash:
> > > 
> > > ...
> > > ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0x1088 irq 15
> > > CPU 0 Unable to handle kernel paging request at virtual address fffffffc, epc == 80341d68, ra == 803eae70
> > > Oops[#1]:
> > > CPU: 0 PID: 1 Comm: swapper Not tainted 4.10.0-rc7-next-20170209 #2
> > > task: 8704b8a0 task.stack: 8704c000
> > > $ 0   : 00000000 0000006f fffffffc 00000000
> > > $ 4   : 87044980 8069ee48 8069ee49 00000004
> > > $ 8   : 00000000 8741c990 00000000 00000040
> > > $12   : 00000000 00000001 00000000 fffffffe
> > > $16   : 8070ae14 80742fb8 80738190 00000000
> > > $20   : 806e0000 8071c2fc 80770000 00000008
> > > $24   : 00000110 00000000
> > > $28   : 8704c000 8704fe10 80754728 803eae70
> > > Hi    : 0000000f
> > > Lo    : 8f8f8f9f
> > > epc   : 80341d68 kset_find_obj+0xc/0xa4
> > > ra    : 803eae70 driver_find+0x1c/0x48
> > > Status: 1000a403	KERNEL EXL IE
> > > Cause : 00800008 (ExcCode 02)
> > > BadVA : fffffffc
> > > PrId  : 00019300 (MIPS 24Kc)
> > > Modules linked in:
> > > Process swapper (pid: 1, threadinfo=8704c000, task=8704b8a0, tls=00000000)
> > > Stack : 80710000 00000000 806e0000 8044ba18 8070ae14 803eaf10 00000000 00000000
> > >          80770000 80742fb8 80770000 80770000 80742fb8 801005dc 8708b900 810017e0
> > >          8704fe68 806f0000 806ee894 81041c41 80600600 80141fb8 00000000 80740000
> > >          806e0000 80720000 00000000 00000006 806765e4 00000006 00000017 80670000
> > >          80676cf0 80680000 80770000 80742fb8 00000006 80742f98 80770000 80742fb8
> > >          ...
> > > Call Trace:
> > > [<80341d68>] kset_find_obj+0xc/0xa4
> > > [<803eae70>] driver_find+0x1c/0x48
> > > [<803eaf10>] driver_register+0x74/0x128
> > > [<801005dc>] do_one_initcall+0x48/0x16c
> > > [<8071cd18>] kernel_init_freeable+0x170/0x22c
> > > [<805e1d30>] kernel_init+0x14/0x108
> > > [<80105e18>] ret_from_kernel_thread+0x14/0x1c
> > > Code: 8c830000  10830012  2462fffc <8c430000> 1060000c  00a03021  90670000  90c10000  24630001
> > > 
> > > ---[ end trace c22dffa22f0b1376 ]---
> > > Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> > > 
> > > ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> > > 
> > > Guenter
> > > 
> > > > I rebased my patch onto of 4.10-rc7 and compiled with malta defconfig
> > > > with disabled SMP. Then booted with qemu. Doesn't seem to have an
> > > > error with the way I have it set up.
> > > >
> > > > I am using the command:
> > > > qemu-system-mipsel -kernel vmlinux -nographic -smp 0
> > > >
> > > > Thanks,
> > > > Justin
> > > >
> > > > On Wed, Feb 8, 2017 at 3:45 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> > > >> Hi Justin,
> > > >>
> > > >> The 32-bit qemu mips runtime tests for -next at kerneltests.org are crashing
> > > >> for non-SMP builds as follows.
> > > >>
> > > >> cacheinfo: Failed to find cpu0 device node
> > > >> cacheinfo: Unable to detect cache hierarchy for CPU 0
> > > >> CPU 0 Unable to handle kernel paging request at virtual address 00000004, epc ==
> > > >> 8013c6a8, ra == 8013cb54
> > > >> Oops[#1]:
> > > >> CPU: 0 PID: 5 Comm: kworker/u2:0 Not tainted 4.10.0-rc7-next-20170208 #1
> > > >> Workqueue: events_unbound call_usermodehelper_exec_work
> > > >> task: 8704a420 task.stack: 87068000
> > > >> $ 0   : 00000000 80770000 873a8204 8700c214
> > > >> $ 4   : 00000000 873a8200 00000000 ffff00fe
> > > >> $ 8   : 8706bfe0 0000a400 fffffffc 8704a459
> > > >> $12   : 00000000 00000000 00000000 00000720
> > > >> $16   : 873a8200 87045880 00000000 8700c200
> > > >> $20   : 87014000 87014005 8700c200 8700c200
> > > >> $24   : 00000000 8014d1e8
> > > >> $28   : 87068000 8706be70 87045880 8013cb54
> > > >> Hi    : 00077698
> > > >> Lo    : ab801af8
> > > >> epc   : 8013c6a8 process_one_work+0xd4/0x408
> > > >> ra    : 8013cb54 worker_thread+0x178/0x598
> > > >> Status: 1000a402        KERNEL EXL
> > > >> Cause : 0080000c (ExcCode 03)
> > > >> BadVA : 00000004
> > > >> PrId  : 00019300 (MIPS 24Kc)
> > > >> Modules linked in:
> > > >> Process kworker/u2:0 (pid: 5, threadinfo=87068000, task=8704a420, tls=00000000)
> > > >> Stack : 80700000 8700c214 00000088 80700000 8700c200 8700c200 8700c200 87045898
> > > >>         80700000 8700c214 00000088 80700000 8700c200 8013cb54 8013c9dc 8704fe00
> > > >>         87045600 87045700 80700000 80700000 80700000 80676560 87045600 87045700
> > > >>         87045618 87045880 8013c9dc 8704fe00 80670000 80760000 806e0000 80142958
> > > >>         00000000 00000000 00000000 00000000 80142850 80142850 87045700 8704b380
> > > >>         ...
> > > >> Call Trace:
> > > >> [<8013c6a8>] process_one_work+0xd4/0x408
> > > >> [<8013cb54>] worker_thread+0x178/0x598
> > > >> [<80142958>] kthread+0x108/0x138
> > > >> [<80105e18>] ret_from_kernel_thread+0x14/0x1c
> > > >> Code: 8e030008  8e040004  8e150000 <ac830004> 7eb51900  ac640000  ae020004 16400094  ae020008
> > > >>
> > > >> Bisect points to 'MIPS: Add cacheinfo support'; see below for details.
> > > >> Reverting this patch fixes te problem.
> > > >>
> > > >> Configuration is pretty much malta_defconfig with SMP disabled.
> > > >> I can provide more details if needed.
> > > >>
> > > >> Thanks,
> > > >> Guenter
> > > >>
> > > >> ---
> > > >> # bad: [e3e6c5f3544c5d05c6b3b309a34f4f2c3537e993] Add linux-next specific files for 20170208
> > > >> # good: [d5adbfcd5f7bcc6fa58a41c5c5ada0e5c826ce2c] Linux 4.10-rc7
> > > >> git bisect start 'HEAD' 'v4.10-rc7'
> > > >> # bad: [403e468309f9e2b2dbe264be1ad29b1486ed720e] Merge remote-tracking branch 'crypto/master'
> > > >> git bisect bad 403e468309f9e2b2dbe264be1ad29b1486ed720e
> > > >> # bad: [44448c3f07a3ae77164de4405fa97baca4f103f5] Merge remote-tracking branch 'hid/for-next'
> > > >> git bisect bad 44448c3f07a3ae77164de4405fa97baca4f103f5
> > > >> # good: [dd4318312c6fc5c00ae7619f875fb73538a2c1e6] Merge remote-tracking branch 'omap/for-next'
> > > >> git bisect good dd4318312c6fc5c00ae7619f875fb73538a2c1e6
> > > >> # bad: [2fe482530119a6d07acca625b29d527dc22435e6] Merge remote-tracking branch 'powerpc/next'
> > > >> git bisect bad 2fe482530119a6d07acca625b29d527dc22435e6
> > > >> # good: [8576190cecf3370ce07ed04178cb3ecbf17dc2d9] Merge remote-tracking branch 'clk/clk-next'
> > > >> git bisect good 8576190cecf3370ce07ed04178cb3ecbf17dc2d9
> > > >> # bad: [bf0505dfeac3f7df81b62d9d81987aaef6739ffd] MIPS: Fix special case in 64 bit IP checksumming.
> > > >> git bisect bad bf0505dfeac3f7df81b62d9d81987aaef6739ffd
> > > >> # good: [e89ef66d7682f031f026eee6bba03c8c2248d2a9] MIPS: init: Ensure reserved memory regions are not added to bootmem
> > > >> git bisect good e89ef66d7682f031f026eee6bba03c8c2248d2a9
> > > >> # bad: [a8b3b0c94ac282628f0668d1366239a3fa72dc9d] MIPS: Netlogic: Fix assembler warning from smpboot.S
> > > >> git bisect bad a8b3b0c94ac282628f0668d1366239a3fa72dc9d
> > > >> # bad: [07724712bf22aec5fe44e5d399f115755f436721] MIPS: ralink: Add missing symbol for highmem support.
> > > >> git bisect bad 07724712bf22aec5fe44e5d399f115755f436721
> > > >> # good: [856b0f591e951a234d6642cc466023df182eb51c] MIPS: kexec: add debug info about the new kexec'ed image
> > > >> git bisect good 856b0f591e951a234d6642cc466023df182eb51c
> > > >> # bad: [2517caf19dbfac3b39f2db5500c5fd03c4370e81] MIPS: ralink: Add missing I2C and I2S clocks.
> > > >> git bisect bad 2517caf19dbfac3b39f2db5500c5fd03c4370e81
> > > >> # bad: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
> > > >> git bisect bad ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d
> > > >> # good: [0014dea6b71ae3e0384c3358f632b4728c3d432e] MIPS: generic/kexec: add support for a DTB passed in a separate buffer
> > > >> git bisect good 0014dea6b71ae3e0384c3358f632b4728c3d432e
> > > >> # first bad commit: [ef462f3b64e9fb0c8e1cd5d60f5bd7f13ac2156d] MIPS: Add cacheinfo support
> > > >>
> > > >>
> > > >
> > > 
> > > 
> 
> 

Re: Crash in -next due to 'MIPS: Add cacheinfo support'

[Florian Fainelli]

On 02/10/2017 09:46 AM, Guenter Roeck wrote:
> On Fri, Feb 10, 2017 at 10:39:52AM +0000, James Hogan wrote:
>> On Fri, Feb 10, 2017 at 09:40:11AM +0000, James Hogan wrote:
>>> Hi Guenter,
>>>
>>> On Thu, Feb 09, 2017 at 08:50:04PM -0800, Guenter Roeck wrote:
>>>> On 02/09/2017 04:01 PM, Justin Chen wrote:
>>>>> Hello Guenter,
>>>>>
>>>>> I am unable to reproduce the problem. May you give me more details?
>>>>>
>>>> The scripts I am using are available at
>>>>
>>>> https://github.com/groeck/linux-build-test/tree/master/rootfs
>>>>
>>>> in the mips and mipsel subdirectories (both crash). Individual
>>>> build logs are always available at kerneltests.org/builders,
>>>> in the 'next' column.
>>>
>>> Did you find it 100% reproducible? I was trying to reproduce yesterday
>>> evening, and did hit it a few times, but then it stopped happening
>>> before I could try and figure out what was going wrong.
>>
>> I switched to gcc 5.2 (buildroot toolchain) for building kernel, and now
>> it reproduces every time :)
>>
> gcc 5.4.0 (binutils 2.26.1) also reliably crashes. The exact crash depends on
> the kernel (-next is different to ToT + offending patch, qemu command line
> makes a difference, qemu version makes a difference), but the crash is easy
> to reproduce, at least for me.

Just to clarify here, is the crash a combination of:

- the kernel image, based on different trees/branches
- different GCC versions
- different ways of invoking QEMU/QEMU version?

and essentially Justin's commit just made problem 1) to occur, but is
not the root cause of the crash you are seeing?
-- 
Florian

Re: Crash in -next due to 'MIPS: Add cacheinfo support'

[Guenter Roeck]

On Fri, Feb 10, 2017 at 11:15:31AM -0800, Florian Fainelli wrote:
> On 02/10/2017 09:46 AM, Guenter Roeck wrote:
> > On Fri, Feb 10, 2017 at 10:39:52AM +0000, James Hogan wrote:
> >> On Fri, Feb 10, 2017 at 09:40:11AM +0000, James Hogan wrote:
> >>> Hi Guenter,
> >>>
> >>> On Thu, Feb 09, 2017 at 08:50:04PM -0800, Guenter Roeck wrote:
> >>>> On 02/09/2017 04:01 PM, Justin Chen wrote:
> >>>>> Hello Guenter,
> >>>>>
> >>>>> I am unable to reproduce the problem. May you give me more details?
> >>>>>
> >>>> The scripts I am using are available at
> >>>>
> >>>> https://github.com/groeck/linux-build-test/tree/master/rootfs
> >>>>
> >>>> in the mips and mipsel subdirectories (both crash). Individual
> >>>> build logs are always available at kerneltests.org/builders,
> >>>> in the 'next' column.
> >>>
> >>> Did you find it 100% reproducible? I was trying to reproduce yesterday
> >>> evening, and did hit it a few times, but then it stopped happening
> >>> before I could try and figure out what was going wrong.
> >>
> >> I switched to gcc 5.2 (buildroot toolchain) for building kernel, and now
> >> it reproduces every time :)
> >>
> > gcc 5.4.0 (binutils 2.26.1) also reliably crashes. The exact crash depends on
> > the kernel (-next is different to ToT + offending patch, qemu command line
> > makes a difference, qemu version makes a difference), but the crash is easy
> > to reproduce, at least for me.
> 
> Just to clarify here, is the crash a combination of:
> 
> - the kernel image, based on different trees/branches

I tried with linux-next, and I tried with ToT with the offending patch
applied. Both fail reliably. Without the patch, both pass reliably.

> - different GCC versions

I can only say that I see it crashes with both gcc 5.2.0 and gcc 5.4.0.

> - different ways of invoking QEMU/QEMU version?
> 
Yes and no. One way of _not_ (or maybe not always) seeing this crash
is to use the bare-bone command line with qemu 2.7 or 2.8 (with no
root file system provided). This crashes because there is no root file
system, but not as described earlier. It does crash, though, when
providing a more comprehensive command line. All kernel versions
without this patch do not crash.

On the other side, blaming this on the qemu command line is akin to
saying that a crash only seen if a mouse is connected would be caused
by the mouse, so I am not entirely sure if that helps too much.
Also see below, regarding heisenbug.

> and essentially Justin's commit just made problem 1) to occur, but is
> not the root cause of the crash you are seeing?

That would not necessarily be my conclusion. Of course, the code appears
to be heavily SMP related, so it may well be that it exposes some
problem associated with cache handling or support in non-SMP configurations.

Of course, it might also be possible that there is a qemu problem somewhere
which only manifests itself on non-SMP mips images with Justin's commit
applied. That appears to be somewhat unlikely, though I have no hard data
supporting this guess.

I'll do some more testing and try to find the actual crash location.
Tricky though since it almost looks like there is a not completely
initialized workqueue. Making things worse, the problem "goes away"
if I add some debug log into process_one_work(), meaning there may
be a heisenbug.

Guenter

Re: Crash in -next due to 'MIPS: Add cacheinfo support'

[James Hogan]

[-- Attachment #1: Type: text/plain, Size: 4217 bytes --]

On Fri, Feb 10, 2017 at 02:11:52PM -0800, Guenter Roeck wrote:
> On Fri, Feb 10, 2017 at 11:15:31AM -0800, Florian Fainelli wrote:
> > On 02/10/2017 09:46 AM, Guenter Roeck wrote:
> > > On Fri, Feb 10, 2017 at 10:39:52AM +0000, James Hogan wrote:
> > >> On Fri, Feb 10, 2017 at 09:40:11AM +0000, James Hogan wrote:
> > >>> Hi Guenter,
> > >>>
> > >>> On Thu, Feb 09, 2017 at 08:50:04PM -0800, Guenter Roeck wrote:
> > >>>> On 02/09/2017 04:01 PM, Justin Chen wrote:
> > >>>>> Hello Guenter,
> > >>>>>
> > >>>>> I am unable to reproduce the problem. May you give me more details?
> > >>>>>
> > >>>> The scripts I am using are available at
> > >>>>
> > >>>> https://github.com/groeck/linux-build-test/tree/master/rootfs
> > >>>>
> > >>>> in the mips and mipsel subdirectories (both crash). Individual
> > >>>> build logs are always available at kerneltests.org/builders,
> > >>>> in the 'next' column.
> > >>>
> > >>> Did you find it 100% reproducible? I was trying to reproduce yesterday
> > >>> evening, and did hit it a few times, but then it stopped happening
> > >>> before I could try and figure out what was going wrong.
> > >>
> > >> I switched to gcc 5.2 (buildroot toolchain) for building kernel, and now
> > >> it reproduces every time :)
> > >>
> > > gcc 5.4.0 (binutils 2.26.1) also reliably crashes. The exact crash depends on
> > > the kernel (-next is different to ToT + offending patch, qemu command line
> > > makes a difference, qemu version makes a difference), but the crash is easy
> > > to reproduce, at least for me.
> > 
> > Just to clarify here, is the crash a combination of:
> > 
> > - the kernel image, based on different trees/branches
> 
> I tried with linux-next, and I tried with ToT with the offending patch
> applied. Both fail reliably. Without the patch, both pass reliably.
> 
> > - different GCC versions
> 
> I can only say that I see it crashes with both gcc 5.2.0 and gcc 5.4.0.
> 
> > - different ways of invoking QEMU/QEMU version?
> > 
> Yes and no. One way of _not_ (or maybe not always) seeing this crash
> is to use the bare-bone command line with qemu 2.7 or 2.8 (with no
> root file system provided). This crashes because there is no root file
> system, but not as described earlier. It does crash, though, when
> providing a more comprehensive command line. All kernel versions
> without this patch do not crash.
> 
> On the other side, blaming this on the qemu command line is akin to
> saying that a crash only seen if a mouse is connected would be caused
> by the mouse, so I am not entirely sure if that helps too much.
> Also see below, regarding heisenbug.
> 
> > and essentially Justin's commit just made problem 1) to occur, but is
> > not the root cause of the crash you are seeing?
> 
> That would not necessarily be my conclusion. Of course, the code appears
> to be heavily SMP related, so it may well be that it exposes some
> problem associated with cache handling or support in non-SMP configurations.
> 
> Of course, it might also be possible that there is a qemu problem somewhere
> which only manifests itself on non-SMP mips images with Justin's commit
> applied. That appears to be somewhat unlikely, though I have no hard data
> supporting this guess.
> 
> I'll do some more testing and try to find the actual crash location.
> Tricky though since it almost looks like there is a not completely
> initialized workqueue. Making things worse, the problem "goes away"
> if I add some debug log into process_one_work(), meaning there may
> be a heisenbug.

cracked it by moving around an early return error. populate_cache()
macro has multiple statements with no do while (0) around it. The
c->scache.waysize condition in populate_cache_leaves then only
conditionalises the first statement in the macro and in absense of l2
(or l3 for that matter) it'll continue to write beyond the end of the
array allocated in detect_cache_attributes(). Badness ensues.

The SMP calls in arch/mips/kernel/cacheinfo.c file are pretty redundant
too since all the cache info is read from the cpu info structures.

I'll write a patch. Thanks for reporting Guenter!

Cheers
James

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: Crash in -next due to 'MIPS: Add cacheinfo support'

[James Hogan]

[-- Attachment #1: Type: text/plain, Size: 4217 bytes --]

On Fri, Feb 10, 2017 at 02:11:52PM -0800, Guenter Roeck wrote:
> On Fri, Feb 10, 2017 at 11:15:31AM -0800, Florian Fainelli wrote:
> > On 02/10/2017 09:46 AM, Guenter Roeck wrote:
> > > On Fri, Feb 10, 2017 at 10:39:52AM +0000, James Hogan wrote:
> > >> On Fri, Feb 10, 2017 at 09:40:11AM +0000, James Hogan wrote:
> > >>> Hi Guenter,
> > >>>
> > >>> On Thu, Feb 09, 2017 at 08:50:04PM -0800, Guenter Roeck wrote:
> > >>>> On 02/09/2017 04:01 PM, Justin Chen wrote:
> > >>>>> Hello Guenter,
> > >>>>>
> > >>>>> I am unable to reproduce the problem. May you give me more details?
> > >>>>>
> > >>>> The scripts I am using are available at
> > >>>>
> > >>>> https://github.com/groeck/linux-build-test/tree/master/rootfs
> > >>>>
> > >>>> in the mips and mipsel subdirectories (both crash). Individual
> > >>>> build logs are always available at kerneltests.org/builders,
> > >>>> in the 'next' column.
> > >>>
> > >>> Did you find it 100% reproducible? I was trying to reproduce yesterday
> > >>> evening, and did hit it a few times, but then it stopped happening
> > >>> before I could try and figure out what was going wrong.
> > >>
> > >> I switched to gcc 5.2 (buildroot toolchain) for building kernel, and now
> > >> it reproduces every time :)
> > >>
> > > gcc 5.4.0 (binutils 2.26.1) also reliably crashes. The exact crash depends on
> > > the kernel (-next is different to ToT + offending patch, qemu command line
> > > makes a difference, qemu version makes a difference), but the crash is easy
> > > to reproduce, at least for me.
> > 
> > Just to clarify here, is the crash a combination of:
> > 
> > - the kernel image, based on different trees/branches
> 
> I tried with linux-next, and I tried with ToT with the offending patch
> applied. Both fail reliably. Without the patch, both pass reliably.
> 
> > - different GCC versions
> 
> I can only say that I see it crashes with both gcc 5.2.0 and gcc 5.4.0.
> 
> > - different ways of invoking QEMU/QEMU version?
> > 
> Yes and no. One way of _not_ (or maybe not always) seeing this crash
> is to use the bare-bone command line with qemu 2.7 or 2.8 (with no
> root file system provided). This crashes because there is no root file
> system, but not as described earlier. It does crash, though, when
> providing a more comprehensive command line. All kernel versions
> without this patch do not crash.
> 
> On the other side, blaming this on the qemu command line is akin to
> saying that a crash only seen if a mouse is connected would be caused
> by the mouse, so I am not entirely sure if that helps too much.
> Also see below, regarding heisenbug.
> 
> > and essentially Justin's commit just made problem 1) to occur, but is
> > not the root cause of the crash you are seeing?
> 
> That would not necessarily be my conclusion. Of course, the code appears
> to be heavily SMP related, so it may well be that it exposes some
> problem associated with cache handling or support in non-SMP configurations.
> 
> Of course, it might also be possible that there is a qemu problem somewhere
> which only manifests itself on non-SMP mips images with Justin's commit
> applied. That appears to be somewhat unlikely, though I have no hard data
> supporting this guess.
> 
> I'll do some more testing and try to find the actual crash location.
> Tricky though since it almost looks like there is a not completely
> initialized workqueue. Making things worse, the problem "goes away"
> if I add some debug log into process_one_work(), meaning there may
> be a heisenbug.

cracked it by moving around an early return error. populate_cache()
macro has multiple statements with no do while (0) around it. The
c->scache.waysize condition in populate_cache_leaves then only
conditionalises the first statement in the macro and in absense of l2
(or l3 for that matter) it'll continue to write beyond the end of the
array allocated in detect_cache_attributes(). Badness ensues.

The SMP calls in arch/mips/kernel/cacheinfo.c file are pretty redundant
too since all the cache info is read from the cpu info structures.

I'll write a patch. Thanks for reporting Guenter!

Cheers
James

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: Crash in -next due to 'MIPS: Add cacheinfo support'

[Florian Fainelli]

On 02/10/2017 02:39 PM, James Hogan wrote:
> On Fri, Feb 10, 2017 at 02:11:52PM -0800, Guenter Roeck wrote:
>> On Fri, Feb 10, 2017 at 11:15:31AM -0800, Florian Fainelli wrote:
>>> On 02/10/2017 09:46 AM, Guenter Roeck wrote:
>>>> On Fri, Feb 10, 2017 at 10:39:52AM +0000, James Hogan wrote:
>>>>> On Fri, Feb 10, 2017 at 09:40:11AM +0000, James Hogan wrote:
>>>>>> Hi Guenter,
>>>>>>
>>>>>> On Thu, Feb 09, 2017 at 08:50:04PM -0800, Guenter Roeck wrote:
>>>>>>> On 02/09/2017 04:01 PM, Justin Chen wrote:
>>>>>>>> Hello Guenter,
>>>>>>>>
>>>>>>>> I am unable to reproduce the problem. May you give me more details?
>>>>>>>>
>>>>>>> The scripts I am using are available at
>>>>>>>
>>>>>>> https://github.com/groeck/linux-build-test/tree/master/rootfs
>>>>>>>
>>>>>>> in the mips and mipsel subdirectories (both crash). Individual
>>>>>>> build logs are always available at kerneltests.org/builders,
>>>>>>> in the 'next' column.
>>>>>>
>>>>>> Did you find it 100% reproducible? I was trying to reproduce yesterday
>>>>>> evening, and did hit it a few times, but then it stopped happening
>>>>>> before I could try and figure out what was going wrong.
>>>>>
>>>>> I switched to gcc 5.2 (buildroot toolchain) for building kernel, and now
>>>>> it reproduces every time :)
>>>>>
>>>> gcc 5.4.0 (binutils 2.26.1) also reliably crashes. The exact crash depends on
>>>> the kernel (-next is different to ToT + offending patch, qemu command line
>>>> makes a difference, qemu version makes a difference), but the crash is easy
>>>> to reproduce, at least for me.
>>>
>>> Just to clarify here, is the crash a combination of:
>>>
>>> - the kernel image, based on different trees/branches
>>
>> I tried with linux-next, and I tried with ToT with the offending patch
>> applied. Both fail reliably. Without the patch, both pass reliably.
>>
>>> - different GCC versions
>>
>> I can only say that I see it crashes with both gcc 5.2.0 and gcc 5.4.0.
>>
>>> - different ways of invoking QEMU/QEMU version?
>>>
>> Yes and no. One way of _not_ (or maybe not always) seeing this crash
>> is to use the bare-bone command line with qemu 2.7 or 2.8 (with no
>> root file system provided). This crashes because there is no root file
>> system, but not as described earlier. It does crash, though, when
>> providing a more comprehensive command line. All kernel versions
>> without this patch do not crash.
>>
>> On the other side, blaming this on the qemu command line is akin to
>> saying that a crash only seen if a mouse is connected would be caused
>> by the mouse, so I am not entirely sure if that helps too much.
>> Also see below, regarding heisenbug.
>>
>>> and essentially Justin's commit just made problem 1) to occur, but is
>>> not the root cause of the crash you are seeing?
>>
>> That would not necessarily be my conclusion. Of course, the code appears
>> to be heavily SMP related, so it may well be that it exposes some
>> problem associated with cache handling or support in non-SMP configurations.
>>
>> Of course, it might also be possible that there is a qemu problem somewhere
>> which only manifests itself on non-SMP mips images with Justin's commit
>> applied. That appears to be somewhat unlikely, though I have no hard data
>> supporting this guess.
>>
>> I'll do some more testing and try to find the actual crash location.
>> Tricky though since it almost looks like there is a not completely
>> initialized workqueue. Making things worse, the problem "goes away"
>> if I add some debug log into process_one_work(), meaning there may
>> be a heisenbug.
> 
> cracked it by moving around an early return error. populate_cache()
> macro has multiple statements with no do while (0) around it. The
> c->scache.waysize condition in populate_cache_leaves then only
> conditionalises the first statement in the macro and in absense of l2
> (or l3 for that matter) it'll continue to write beyond the end of the
> array allocated in detect_cache_attributes(). Badness ensues.

Good catch, thanks James!

> 
> The SMP calls in arch/mips/kernel/cacheinfo.c file are pretty redundant
> too since all the cache info is read from the cpu info structures.
> 
> I'll write a patch. Thanks for reporting Guenter!
> 
> Cheers
> James
> 


-- 
Florian

Re: Crash in -next due to 'MIPS: Add cacheinfo support'

[Guenter Roeck]

On Fri, Feb 10, 2017 at 10:39:32PM +0000, James Hogan wrote:
> > 
> > > and essentially Justin's commit just made problem 1) to occur, but is
> > > not the root cause of the crash you are seeing?
> > 
> > That would not necessarily be my conclusion. Of course, the code appears
> > to be heavily SMP related, so it may well be that it exposes some
> > problem associated with cache handling or support in non-SMP configurations.
> > 
> > Of course, it might also be possible that there is a qemu problem somewhere
> > which only manifests itself on non-SMP mips images with Justin's commit
> > applied. That appears to be somewhat unlikely, though I have no hard data
> > supporting this guess.
> > 
> > I'll do some more testing and try to find the actual crash location.
> > Tricky though since it almost looks like there is a not completely
> > initialized workqueue. Making things worse, the problem "goes away"
> > if I add some debug log into process_one_work(), meaning there may
> > be a heisenbug.
> 
> cracked it by moving around an early return error. populate_cache()
> macro has multiple statements with no do while (0) around it. The
> c->scache.waysize condition in populate_cache_leaves then only
> conditionalises the first statement in the macro and in absense of l2
> (or l3 for that matter) it'll continue to write beyond the end of the
> array allocated in detect_cache_attributes(). Badness ensues.
> 
Outch. Yes, after you mention it, the problem is easy to see.

> The SMP calls in arch/mips/kernel/cacheinfo.c file are pretty redundant
> too since all the cache info is read from the cpu info structures.
> 
> I'll write a patch. Thanks for reporting Guenter!
> 
Thank you for tracking it down!

Guenter

[PATCH] MIPS: Fix cacheinfo overflow

[James Hogan]

The recently added MIPS cacheinfo support used a macro populate_cache()
to populate the cacheinfo structures depending on which caches are
present. However the macro contains multiple statements without
enclosing them in a do {} while (0) loop, so the L2 and L3 cache
conditionals in populate_cache_leaves() only conditionalised the first
statement in the macro.

This overflows the buffer allocated by detect_cache_attributes(),
resulting in boot failures under QEMU where neither the L2 or L2 caches
are present.

Enclose the macro statements in a do {} while (0) block to keep the
whole macro inside the conditionals.

Fixes: ef462f3b64e9 ("MIPS: Add cacheinfo support")
Reported-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Justin Chen <justin.chen@broadcom.com>
Cc: Florian Fainelli <f.fainelli@gmail.com>
Cc: linux-mips@linux-mips.org
Cc: bcm-kernel-feedback-list@broadcom.com
---
 arch/mips/kernel/cacheinfo.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/mips/kernel/cacheinfo.c b/arch/mips/kernel/cacheinfo.c
index a92bbbae969b..97d5239ca47b 100644
--- a/arch/mips/kernel/cacheinfo.c
+++ b/arch/mips/kernel/cacheinfo.c
@@ -17,6 +17,7 @@
 
 /* Populates leaf and increments to next leaf */
 #define populate_cache(cache, leaf, c_level, c_type)		\
+do {								\
 	leaf->type = c_type;					\
 	leaf->level = c_level;					\
 	leaf->coherency_line_size = c->cache.linesz;		\
@@ -24,7 +25,8 @@
 	leaf->ways_of_associativity = c->cache.ways;		\
 	leaf->size = c->cache.linesz * c->cache.sets *		\
 		c->cache.ways;					\
-	leaf++;
+	leaf++;							\
+} while (0)
 
 static int __init_cache_level(unsigned int cpu)
 {
-- 
2.11.0

[PATCH] MIPS: Fix cacheinfo overflow

[James Hogan]

The recently added MIPS cacheinfo support used a macro populate_cache()
to populate the cacheinfo structures depending on which caches are
present. However the macro contains multiple statements without
enclosing them in a do {} while (0) loop, so the L2 and L3 cache
conditionals in populate_cache_leaves() only conditionalised the first
statement in the macro.

This overflows the buffer allocated by detect_cache_attributes(),
resulting in boot failures under QEMU where neither the L2 or L2 caches
are present.

Enclose the macro statements in a do {} while (0) block to keep the
whole macro inside the conditionals.

Fixes: ef462f3b64e9 ("MIPS: Add cacheinfo support")
Reported-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Justin Chen <justin.chen@broadcom.com>
Cc: Florian Fainelli <f.fainelli@gmail.com>
Cc: linux-mips@linux-mips.org
Cc: bcm-kernel-feedback-list@broadcom.com
---
 arch/mips/kernel/cacheinfo.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/mips/kernel/cacheinfo.c b/arch/mips/kernel/cacheinfo.c
index a92bbbae969b..97d5239ca47b 100644
--- a/arch/mips/kernel/cacheinfo.c
+++ b/arch/mips/kernel/cacheinfo.c
@@ -17,6 +17,7 @@
 
 /* Populates leaf and increments to next leaf */
 #define populate_cache(cache, leaf, c_level, c_type)		\
+do {								\
 	leaf->type = c_type;					\
 	leaf->level = c_level;					\
 	leaf->coherency_line_size = c->cache.linesz;		\
@@ -24,7 +25,8 @@
 	leaf->ways_of_associativity = c->cache.ways;		\
 	leaf->size = c->cache.linesz * c->cache.sets *		\
 		c->cache.ways;					\
-	leaf++;
+	leaf++;							\
+} while (0)
 
 static int __init_cache_level(unsigned int cpu)
 {
-- 
2.11.0

Re: [PATCH] MIPS: Fix cacheinfo overflow

[Justin Chen]

Hello,

Thanks for the catch!

I actually submitted a v2 of the patch sometime back located here:
https://patchwork.linux-mips.org/patch/15107/

The v2 had some code review changes from Matt Redfearn. These changes 
indirectly got rid of the error, which was why I wasn't running into the 
crash.

Either way, whatever makes more sense, we can drop the other v1 patch 
and add v2 or just add this patch on top.

Thanks,
Justin

On 02/10/2017 03:01 PM, James Hogan wrote:
> The recently added MIPS cacheinfo support used a macro populate_cache()
> to populate the cacheinfo structures depending on which caches are
> present. However the macro contains multiple statements without
> enclosing them in a do {} while (0) loop, so the L2 and L3 cache
> conditionals in populate_cache_leaves() only conditionalised the first
> statement in the macro.
>
> This overflows the buffer allocated by detect_cache_attributes(),
> resulting in boot failures under QEMU where neither the L2 or L2 caches
> are present.
>
> Enclose the macro statements in a do {} while (0) block to keep the
> whole macro inside the conditionals.
>
> Fixes: ef462f3b64e9 ("MIPS: Add cacheinfo support")
> Reported-by: Guenter Roeck <linux@roeck-us.net>
> Signed-off-by: James Hogan <james.hogan@imgtec.com>
> Cc: Ralf Baechle <ralf@linux-mips.org>
> Cc: Justin Chen <justin.chen@broadcom.com>
> Cc: Florian Fainelli <f.fainelli@gmail.com>
> Cc: linux-mips@linux-mips.org
> Cc: bcm-kernel-feedback-list@broadcom.com
> ---
>  arch/mips/kernel/cacheinfo.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/arch/mips/kernel/cacheinfo.c b/arch/mips/kernel/cacheinfo.c
> index a92bbbae969b..97d5239ca47b 100644
> --- a/arch/mips/kernel/cacheinfo.c
> +++ b/arch/mips/kernel/cacheinfo.c
> @@ -17,6 +17,7 @@
>
>  /* Populates leaf and increments to next leaf */
>  #define populate_cache(cache, leaf, c_level, c_type)		\
> +do {								\
>  	leaf->type = c_type;					\
>  	leaf->level = c_level;					\
>  	leaf->coherency_line_size = c->cache.linesz;		\
> @@ -24,7 +25,8 @@
>  	leaf->ways_of_associativity = c->cache.ways;		\
>  	leaf->size = c->cache.linesz * c->cache.sets *		\
>  		c->cache.ways;					\
> -	leaf++;
> +	leaf++;							\
> +} while (0)
>
>  static int __init_cache_level(unsigned int cpu)
>  {
>

Re: [PATCH] MIPS: Fix cacheinfo overflow

[James Hogan]

[-- Attachment #1: Type: text/plain, Size: 1114 bytes --]

Hi Justin,

On Fri, Feb 10, 2017 at 03:10:47PM -0800, Justin Chen wrote:
> I actually submitted a v2 of the patch sometime back located here:
> https://patchwork.linux-mips.org/patch/15107/
> 
> The v2 had some code review changes from Matt Redfearn. These changes 
> indirectly got rid of the error, which was why I wasn't running into the 
> crash.

Ah yes. It looks like Ralf applied the original patch on January 3rd,
before the last 3 submissions of the patch.

Incidentally, in future I'd recommend incrementing the patch version
number each time you submit a new version, and mentioning what has
changed in a comment at the bottom of the commit message (anything after
"---" gets dropped when the patch is applied).

A random example:
https://patchwork.linux-mips.org/patch/15134/

> 
> Either way, whatever makes more sense, we can drop the other v1 patch 
> and add v2 or just add this patch on top.

Since the patch has already been applied for a while and the merge
window is imminent, I think its best to make do with fixup patches on
top this time around.

Thanks
James

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH] MIPS: Fix cacheinfo overflow

[James Hogan]

[-- Attachment #1: Type: text/plain, Size: 1114 bytes --]

Hi Justin,

On Fri, Feb 10, 2017 at 03:10:47PM -0800, Justin Chen wrote:
> I actually submitted a v2 of the patch sometime back located here:
> https://patchwork.linux-mips.org/patch/15107/
> 
> The v2 had some code review changes from Matt Redfearn. These changes 
> indirectly got rid of the error, which was why I wasn't running into the 
> crash.

Ah yes. It looks like Ralf applied the original patch on January 3rd,
before the last 3 submissions of the patch.

Incidentally, in future I'd recommend incrementing the patch version
number each time you submit a new version, and mentioning what has
changed in a comment at the bottom of the commit message (anything after
"---" gets dropped when the patch is applied).

A random example:
https://patchwork.linux-mips.org/patch/15134/

> 
> Either way, whatever makes more sense, we can drop the other v1 patch 
> and add v2 or just add this patch on top.

Since the patch has already been applied for a while and the merge
window is imminent, I think its best to make do with fixup patches on
top this time around.

Thanks
James

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH] MIPS: Fix cacheinfo overflow

[Justin Chen]

On 02/10/2017 03:44 PM, James Hogan wrote:
> Hi Justin,
>
> On Fri, Feb 10, 2017 at 03:10:47PM -0800, Justin Chen wrote:
>> I actually submitted a v2 of the patch sometime back located here:
>> https://patchwork.linux-mips.org/patch/15107/
>>
>> The v2 had some code review changes from Matt Redfearn. These changes
>> indirectly got rid of the error, which was why I wasn't running into the
>> crash.
>
> Ah yes. It looks like Ralf applied the original patch on January 3rd,
> before the last 3 submissions of the patch.
>
> Incidentally, in future I'd recommend incrementing the patch version
> number each time you submit a new version, and mentioning what has
> changed in a comment at the bottom of the commit message (anything after
> "---" gets dropped when the patch is applied).
>
> A random example:
> https://patchwork.linux-mips.org/patch/15134/
Got it. Sorry, will do next time!
>
>>
>> Either way, whatever makes more sense, we can drop the other v1 patch
>> and add v2 or just add this patch on top.
>
> Since the patch has already been applied for a while and the merge
> window is imminent, I think its best to make do with fixup patches on
> top this time around.
Sounds good to me.
>
> Thanks
> James
>

Re: [PATCH] MIPS: Fix cacheinfo overflow

[Guenter Roeck]

On 02/10/2017 03:01 PM, James Hogan wrote:
> The recently added MIPS cacheinfo support used a macro populate_cache()
> to populate the cacheinfo structures depending on which caches are
> present. However the macro contains multiple statements without
> enclosing them in a do {} while (0) loop, so the L2 and L3 cache
> conditionals in populate_cache_leaves() only conditionalised the first
> statement in the macro.
>
> This overflows the buffer allocated by detect_cache_attributes(),
> resulting in boot failures under QEMU where neither the L2 or L2 caches
> are present.
>
> Enclose the macro statements in a do {} while (0) block to keep the
> whole macro inside the conditionals.
>
> Fixes: ef462f3b64e9 ("MIPS: Add cacheinfo support")
> Reported-by: Guenter Roeck <linux@roeck-us.net>
> Signed-off-by: James Hogan <james.hogan@imgtec.com>

Tested-by: Guenter Roeck <linux@roeck-us.net>

> Cc: Ralf Baechle <ralf@linux-mips.org>
> Cc: Justin Chen <justin.chen@broadcom.com>
> Cc: Florian Fainelli <f.fainelli@gmail.com>
> Cc: linux-mips@linux-mips.org
> Cc: bcm-kernel-feedback-list@broadcom.com
> ---
>  arch/mips/kernel/cacheinfo.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/arch/mips/kernel/cacheinfo.c b/arch/mips/kernel/cacheinfo.c
> index a92bbbae969b..97d5239ca47b 100644
> --- a/arch/mips/kernel/cacheinfo.c
> +++ b/arch/mips/kernel/cacheinfo.c
> @@ -17,6 +17,7 @@
>
>  /* Populates leaf and increments to next leaf */
>  #define populate_cache(cache, leaf, c_level, c_type)		\
> +do {								\
>  	leaf->type = c_type;					\
>  	leaf->level = c_level;					\
>  	leaf->coherency_line_size = c->cache.linesz;		\
> @@ -24,7 +25,8 @@
>  	leaf->ways_of_associativity = c->cache.ways;		\
>  	leaf->size = c->cache.linesz * c->cache.sets *		\
>  		c->cache.ways;					\
> -	leaf++;
> +	leaf++;							\
> +} while (0)
>
>  static int __init_cache_level(unsigned int cpu)
>  {
>