From: Hangbin Liu <haliu@redhat.com>
To: CKI Project <cki-project@redhat.com>
Cc: Linux Stable maillist <stable@vger.kernel.org>,
Yi Zhang <yi.zhang@redhat.com>, Xiumei Mu <xmu@redhat.com>,
Jianwen Ji <jiji@redhat.com>
Subject: Re: ❌ FAIL: Stable queue: queue-5.3
Date: Thu, 7 Nov 2019 18:22:35 +0800 [thread overview]
Message-ID: <20191107102235.GR16190@dhcp-12-139.nay.redhat.com> (raw)
In-Reply-To: <cki.7C65793389.02658IUR51@redhat.com>
On Wed, Nov 06, 2019 at 08:56:04PM -0000, CKI Project wrote:
>
> Hello,
>
> We ran automated tests on a patchset that was proposed for merging into this
> kernel tree. The patches were applied to:
>
> Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
> Commit: 37b4d0c37c0b - Linux 5.3.9
>
> The results of these automated tests are provided below.
>
> Overall result: FAILED (see details below)
> Merge: OK
> Compile: OK
> Tests: FAILED
>
> All kernel binaries, config files, and logs are available for download here:
>
> https://artifacts.cki-project.org/pipelines/268208
>
> One or more kernel tests failed:
>
> ppc64le:
> ❌ Networking socket: fuzz
From the call trace this doesn't like a network issue...
[10171.661351] NET: Registered protocol family 5
[10171.935836] NET: Unregistered protocol family 5
[10172.056312] NET: Registered protocol family 5
[10172.086044] BUG: Unable to handle kernel data access at 0x29f54ed8fb2aa757
[10172.086068] Faulting instruction address: 0xc0000000006d7a28
[10172.086082] Oops: Kernel access of bad area, sig: 11 [#1]
[10172.086104] LE PAGE_SIZE=64K MMU=Radix MMU=Hash SMP NR_CPUS=1024 NUMA PowerNV
[10172.086119] Modules linked in: appletalk(+) netrom bluetooth ecdh_generic ecc atm smc ib_core mpls_router ip_tunnel af_key nfc rfkill pppoe pppox ppp_generic slhc psnap llc kcm ieee802154_socket ieee802154 fcrypt pcbc rxrpc rose ax25 can mlx4_en mlx4_core nls_utf8 isofs kvm_hv kvm minix binfmt_misc nfsv3 nfs_acl nfs lockd grace fscache sctp rds brd vfat fat btrfs xor zstd_compress raid6_pq zstd_decompress loop tun ip6table_nat ip6_tables xt_conntrack iptable_filter xt_MASQUERADE xt_comment iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 overlay fuse sunrpc bnx2x vmx_crypto at24 mdio regmap_i2c tg3 crct10dif_vpmsum ipmi_powernv ipmi_devintf ofpart powernv_flash opal_prd mtd rtc_opal i2c_opal ipmi_msghandler ip_tables xfs libcrc32c ast i2c_algo_bit drm_vram_helper ttm drm_kms_helper crc32c_vpmsum syscopyarea sysfillrect sysimgblt fb_sys_fops drm drm_panel_orientation_quirks i2c_core [last unloaded: dummy]
[10172.086377] CPU: 124 PID: 1324763 Comm: kworker/u258:0 Not tainted 5.3.9-fd272dc.cki #1
[10172.086420] Workqueue: writeback wb_workfn (flush-253:0)
[10172.086442] NIP: c0000000006d7a28 LR: c0000000006d7f00 CTR: c0000000006d38a0
[10172.086477] REGS: c000201ac2807380 TRAP: 0380 Not tainted (5.3.9-fd272dc.cki)
[10172.086501] MSR: 9000000000009033 <SF,HV,EE,ME,IR,DR,RI,LE> CR: 44008248 XER: 00000000
[10172.086539] CFAR: c0000000006d7a70 IRQMASK: 1
[10172.086539] GPR00: c0000000006d7f00 c000201ac2807610 c000000001771f00 c000001fe9a5d800
[10172.086539] GPR04: c000001fe9314e68 0000000000000000 0000000000000000 0000000000000270
[10172.086539] GPR08: c000001fea935800 c000001fe9a5d898 29f54ed8fb2aa71f c000201cae364cf8
[10172.086539] GPR12: 0000000000000040 c000201fff67c400 c00000000015adc8 c000001fc5177248
[10172.086539] GPR16: 0000000000000000 0000000000000000 0000000f00000204 002d444552414853
[10172.086539] GPR20: c000000000fe9618 c000201c57118580 0000000000000000 c000201cac3bdb70
[10172.086539] GPR24: 0000000000060800 c000001fc7a71800 c000201c76a64c00 c000201c98f63200
[10172.086539] GPR28: 0000000000001aee c000001fc7a71800 c000201c978f67c0 c000001fc7a71800
[10172.086782] NIP [c0000000006d7a28] bfq_find_set_group+0xa8/0x140
[10172.086816] LR [c0000000006d7f00] bfq_bic_update_cgroup+0x130/0x2f0
[10172.086849] Call Trace:
[10172.086868] [c000201ac2807610] [c0000000006d79e0] bfq_find_set_group+0x60/0x140 (unreliable)
[10172.086906] [c000201ac2807640] [c0000000006d7f00] bfq_bic_update_cgroup+0x130/0x2f0
[10172.086942] [c000201ac2807710] [c0000000006d2bdc] bfq_init_rq+0x11c/0xb30
[10172.086976] [c000201ac28077e0] [c0000000006d3a38] bfq_insert_requests+0x198/0x12e0
[10172.087013] [c000201ac2807940] [c0000000006a5274] blk_mq_sched_insert_requests+0xb4/0x1d0
[10172.087061] [c000201ac2807990] [c00000000069e720] blk_mq_flush_plug_list+0x260/0x330
[10172.087097] [c000201ac2807a20] [c00000000068ed58] blk_flush_plug_list+0x168/0x190
[10172.087122] [c000201ac2807a90] [c00000000068edb4] blk_finish_plug+0x34/0x50
[10172.087156] [c000201ac2807ab0] [c000000000483400] wb_writeback+0x200/0x470
[10172.087179] [c000201ac2807b70] [c000000000485228] wb_workfn+0x4f8/0x5c0
[10172.087203] [c000201ac2807c80] [c000000000151f6c] process_one_work+0x26c/0x510
[10172.087217] [c000201ac2807d20] [c000000000152298] worker_thread+0x88/0x5c0
[10172.087231] [c000201ac2807db0] [c00000000015af14] kthread+0x154/0x1a0
[10172.087266] [c000201ac2807e20] [c00000000000bed8] ret_from_kernel_thread+0x5c/0x64
[10172.087300] Instruction dump:
[10172.087319] e8e9489a 38e7004c 78e71f24 7c63382a 2c230000 418200a0 35230098 40820048
[10172.087347] 48000068 60000000 419e0008 e949ff68 <e94a0038> 2c2a0000 41820060 7d4a382a
[10172.087397] ---[ end trace ce87155ebd82719c ]---
[10172.239286]
[10232.098595] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[10232.098617] rcu: 124-...0: (0 ticks this GP) idle=45e/1/0x4000000000000000 softirq=65648/65648 fqs=3000
[10232.098636] (detected by 102, t=6002 jiffies, g=822369, q=4669)
[10232.098651] Sending NMI from CPU 102 to CPUs 124:
[10237.669520] CPU 124 didn't respond to backtrace IPI, inspecting paca.
[10237.669537] irq_soft_mask: 0x01 in_mce: 0 in_nmi: 0 current: 1324763 (kworker/u258:0)
[10237.669550] Back trace of paca->saved_r1 (0xc000201ac2806fe0) (possibly stale):
[10237.669572] Call Trace:
[10237.669579] [c000201ac2806fe0] [c000201ac2807080] 0xc000201ac2807080 (unreliable)
[10237.669593] [c000201ac2807020] [c0000000006d28b8] bfq_exit_icq_bfqq+0x68/0x220
[10237.669627] [c000201ac28070d0] [c0000000006d2a9c] bfq_exit_icq+0x2c/0x50
[10237.669639] [c000201ac2807100] [c000000000693734] put_io_context_active+0xc4/0x120
[10237.669674] [c000201ac2807140] [c00000000012ebf4] do_exit+0x764/0xe10
[10237.669685] [c000201ac2807200] [c00000000002a90c] oops_end+0x16c/0x1a0
[10237.669697] [c000201ac2807280] [c000000000075738] bad_page_fault+0x118/0x1d0
[10237.669720] [c000201ac28072f0] [c00000000007f218] do_bad_slb_fault+0x88/0xa0
[10237.669744] [c000201ac2807310] [c000000000008904] data_access_slb_common+0x174/0x180
[10237.669779] --- interrupt: 380 at bfq_find_set_group+0xa8/0x140
[10237.669779] LR = bfq_bic_update_cgroup+0x130/0x2f0
[10237.669827] [c000201ac2807610] [c0000000006d79e0] bfq_find_set_group+0x60/0x140 (unreliable)
[10237.669864] [c000201ac2807640] [c0000000006d7f00] bfq_bic_update_cgroup+0x130/0x2f0
[10237.669924] [c000201ac2807710] [c0000000006d2bdc] bfq_init_rq+0x11c/0xb30
[10237.669946] [c000201ac28077e0] [c0000000006d3a38] bfq_insert_requests+0x198/0x12e0
[10237.669981] [c000201ac2807940] [c0000000006a5274] blk_mq_sched_insert_requests+0xb4/0x1d0
[10237.670016] [c000201ac2807990] [c00000000069e720] blk_mq_flush_plug_list+0x260/0x330
[10237.670051] [c000201ac2807a20] [c00000000068ed58] blk_flush_plug_list+0x168/0x190
[10237.670086] [c000201ac2807a90] [c00000000068edb4] blk_finish_plug+0x34/0x50
[10237.670120] [c000201ac2807ab0] [c000000000483400] wb_writeback+0x200/0x470
[10237.670153] [c000201ac2807b70] [c000000000485228] wb_workfn+0x4f8/0x5c0
[10237.670187] [c000201ac2807c80] [c000000000151f6c] process_one_work+0x26c/0x510
[10237.670222] [c000201ac2807d20] [c000000000152298] worker_thread+0x88/0x5c0
[10237.670245] [c000201ac2807db0] [c00000000015af14] kthread+0x154/0x1a0
[10237.670278] [c000201ac2807e20] [c00000000000bed8] ret_from_kernel_thread+0x5c/0x64
[10237.670313] watchdog: CPU 19 self-detected hard LOCKUP @ _raw_spin_lock_irq+0x60/0x100
[10237.670314] watchdog: CPU 19 TB:5287683334232, last heartbeat TB:5281606533730 (11868ms ago)
[10237.670314] Modules linked in: appletalk(+) netrom bluetooth ecdh_generic ecc atm smc ib_core mpls_router ip_tunnel af_key nfc rfkill pppoe pppox ppp_generic slhc psnap llc kcm ieee802154_socket ieee802154 fcrypt pcbc rxrpc rose ax25 can mlx4_en mlx4_core nls_utf8 isofs kvm_hv kvm minix binfmt_misc nfsv3 nfs_acl nfs lockd grace fscache sctp rds brd vfat fat btrfs xor zstd_compress raid6_pq zstd_decompress loop tun ip6table_nat ip6_tables xt_conntrack iptable_filter xt_MASQUERADE xt_comment iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 overlay fuse sunrpc bnx2x vmx_crypto at24 mdio regmap_i2c tg3 crct10dif_vpmsum ipmi_powernv ipmi_devintf ofpart powernv_flash opal_prd mtd rtc_opal i2c_opal ipmi_msghandler ip_tables xfs libcrc32c ast i2c_algo_bit drm_vram_helper ttm drm_kms_helper crc32c_vpmsum syscopyarea sysfillrect sysimgblt fb_sys_fops drm drm_panel_orientation_quirks i2c_core [last unloaded: dummy]
[10237.670339] CPU: 19 PID: 1213 Comm: xfsaild/dm-0 Tainted: G D 5.3.9-fd272dc.cki #1
[10237.670340] NIP: c000000000d1ea70 LR: c0000000006ce1e8 CTR: c0000000006ce140
[10237.670341] REGS: c000001fffcd3d70 TRAP: 0900 Tainted: G D (5.3.9-fd272dc.cki)
[10237.670341] MSR: 9000000000009033 <SF,HV,EE,ME,IR,DR,RI,LE> CR: 44002422 XER: 20040000
[10237.670344] CFAR: c000000000d1ea98 IRQMASK: 1
[10237.670345] GPR00: c0000000006ce1e8 c000001fe9d8b7b0 c000000001771f00 c000001fc7a71bf0
[10237.670346] GPR04: 0000000000000000 0000000000000001 c000001fe9d8b910 c000001fea934000
[10237.670348] GPR08: 0000000000000000 000000008000007c 0000000080000013 c000000001799cf8
[10237.670349] GPR12: c0000000006ce140 c000001ffffe9a00 0000000000004000 0000000000000000
[10237.670351] GPR16: c000000001a385a8 0000000000000000 0000000000001001 0000000000010000
[10237.670353] GPR20: 0000000000000000 0000000000000000 000000000078c9e0 000000000000c000
[10237.670354] GPR24: 0000000000008000 c000001fe9314ef4 c000001fc7a71bf0 c000001fe9314e68
[10237.670356] GPR28: 0000000000000000 c000001f6d1f2940 c000001fc8d679f0 c00000000179bd00
[10237.670358] NIP [c000000000d1ea70] _raw_spin_lock_irq+0x60/0x100
[10237.670358] LR [c0000000006ce1e8] bfq_bio_merge+0xa8/0x1f0
[10237.670358] Call Trace:
[10237.670359] [c000001fe9d8b7b0] [c000001fe9d8b7f0] 0xc000001fe9d8b7f0 (unreliable)
[10237.670360] [c000001fe9d8b7f0] [c0000000006ce1dc] bfq_bio_merge+0x9c/0x1f0
[10237.670361] [c000001fe9d8b870] [c0000000006a4de8] __blk_mq_sched_bio_merge+0xa8/0x1d0
[10237.670361] [c000001fe9d8b8d0] [c00000000069da58] blk_mq_make_request+0xe8/0x780
[10237.670362] [c000001fe9d8b980] [c00000000068d1ac] generic_make_request+0xdc/0x380
[10237.670363] [c000001fe9d8ba00] [c00000000068d4a8] submit_bio+0x58/0x250
[10237.670364] [c000001fe9d8bac0] [c00800000d7f35a8] _xfs_buf_ioapply+0x3e0/0x4b0 [xfs]
[10237.670364] [c000001fe9d8bbf0] [c00800000d7f37b0] __xfs_buf_submit+0xc8/0x2d0 [xfs]
[10237.670365] [c000001fe9d8bc30] [c00800000d7f4020] xfs_buf_delwri_submit_buffers+0x138/0x300 [xfs]
[10237.670366] [c000001fe9d8bcd0] [c00800000d843090] xfsaild+0x2a8/0x980 [xfs]
[10237.670367] [c000001fe9d8bdb0] [c00000000015af14] kthread+0x154/0x1a0
[10237.670367] [c000001fe9d8be20] [c00000000000bed8] ret_from_kernel_thread+0x5c/0x64
[10237.670368] Instruction dump:
[10237.670369] 7c2004ac 2c290000 40820018 a12d1180 39290001 b12d1180 4e800020 60000000
[10237.670371] fbe1fff8 f821ffc1 3fe20003 3bff9e00 <7c210b78> e93f0000 75290010 41820014
[10237.670392] watchdog: CPU 56 self-detected hard LOCKUP @ _raw_spin_lock_irq+0x64/0x100
next prev parent reply other threads:[~2019-11-07 10:22 UTC|newest]
Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-06 20:56 ❌ FAIL: Stable queue: queue-5.3 CKI Project
2019-11-07 10:22 ` Hangbin Liu [this message]
-- strict thread matches above, loose matches on Subject: below --
2019-11-26 2:00 CKI Project
2019-11-22 1:23 CKI Project
2019-11-18 15:00 CKI Project
2019-11-19 0:07 ` [LTP] " Rachel Sibley
2019-11-19 12:37 ` Laura Abbott
2019-11-19 12:44 ` Greg KH
2019-11-19 12:50 ` Laura Abbott
2019-11-21 17:53 ` Greg KH
2019-11-11 21:58 CKI Project
2019-11-11 19:57 CKI Project
2019-11-11 15:36 CKI Project
2019-11-11 20:10 ` Rachel Sibley
2019-11-11 20:47 ` Laura Abbott
2019-11-11 12:50 CKI Project
2019-11-11 11:43 CKI Project
2019-11-10 16:12 CKI Project
2019-11-08 23:24 CKI Project
2019-11-08 21:16 CKI Project
2019-11-08 21:55 ` Ondrej Mosnacek
2019-11-08 5:51 CKI Project
2019-11-05 2:51 CKI Project
2019-11-05 1:14 CKI Project
2019-11-05 6:37 ` Greg KH
2019-11-05 8:15 ` Naresh Kamboju
2019-11-04 11:21 CKI Project
2019-10-30 20:12 CKI Project
2019-10-29 20:22 CKI Project
2019-10-29 17:48 CKI Project
2019-10-28 2:44 CKI Project
2019-10-28 2:30 CKI Project
2019-10-28 1:22 CKI Project
2019-10-27 23:47 CKI Project
2019-10-27 14:34 CKI Project
2019-10-27 9:18 CKI Project
2019-10-27 9:08 CKI Project
2019-10-27 9:20 ` Sasha Levin
2019-10-26 22:33 CKI Project
2019-10-27 7:02 ` Naresh Kamboju
2019-10-27 9:32 ` Jan Stancek
2019-10-25 11:25 CKI Project
2019-10-22 2:54 CKI Project
2019-10-16 14:03 CKI Project
2019-10-16 13:56 CKI Project
2019-10-15 22:30 CKI Project
2019-10-15 6:39 CKI Project
2019-10-15 4:49 CKI Project
2019-10-10 13:17 CKI Project
2019-10-10 10:26 CKI Project
2019-10-10 10:51 ` Veronika Kabatova
2019-10-10 6:43 CKI Project
2019-10-09 22:11 CKI Project
2019-10-09 22:44 ` Sasha Levin
2019-10-10 1:09 ` Don Zickus
2019-10-10 6:30 ` Naresh Kamboju
2019-10-16 17:25 ` ? " Jakub Racek
2019-10-08 19:17 ❌ " CKI Project
2019-10-08 19:10 CKI Project
2019-10-08 18:03 CKI Project
2019-10-08 18:57 ` Greg KH
2019-10-06 4:57 CKI Project
2019-10-01 19:25 CKI Project
2019-10-01 18:32 CKI Project
2019-10-01 19:19 ` Greg KH
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191107102235.GR16190@dhcp-12-139.nay.redhat.com \
--to=haliu@redhat.com \
--cc=cki-project@redhat.com \
--cc=jiji@redhat.com \
--cc=stable@vger.kernel.org \
--cc=xmu@redhat.com \
--cc=yi.zhang@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.