All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jan Stancek <jstancek@redhat.com>
To: CKI Project <cki-project@redhat.com>,
	Jaroslav Kysela <jkysela@redhat.com>,
	alsa-devel@alsa-project.org,
	LTP Mailing List <ltp@lists.linux.it>
Cc: Linux Stable maillist <stable@vger.kernel.org>,
	Memory Management <mm-qe@redhat.com>
Subject: Re: ❌ FAIL: Test report for kernel 5.3.9-rc1-dfe283e.cki (stable)
Date: Mon, 4 Nov 2019 08:35:51 -0500 (EST)	[thread overview]
Message-ID: <2029139028.10333037.1572874551626.JavaMail.zimbra@redhat.com> (raw)
In-Reply-To: <cki.1210A7ECB0.BD9Q3APV4K@redhat.com>



----- Original Message -----
> 
> Hello,
> 
> We ran automated tests on a recent commit from this kernel tree:
> 
>        Kernel repo:
>        git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
>             Commit: dfe283e9fdac - Linux 5.3.9-rc1
> 
> The results of these automated tests are provided below.
> 
>     Overall result: FAILED (see details below)
>              Merge: OK
>            Compile: OK
>              Tests: FAILED
> 
> All kernel binaries, config files, and logs are available for download here:
> 
>   https://artifacts.cki-project.org/pipelines/262380
> 
> One or more kernel tests failed:
> 
>     x86_64:
>      ❌ LTP lite
>

Not a 5.3 -stable regression.

Failure comes from test that sanity checks all /proc files by doing
1k read from each. There are couple issues it hits wrt. snd_hda_*.

Example reproducer:
  dd if=/sys/kernel/debug/regmap/hdaudioC0D3-hdaudio/access of=out.txt count=1 bs=1024 iflag=nonblock

It's slow and triggers soft lockups [1]. And it also requires lot
of memory, triggering OOMs on smaller VMs:
0x0000000024f0437b-0x000000001a32b1c8 1073745920 seq_read+0x131/0x400 pages=262144 vmalloc vpages N0=262144

I'm leaning towards skipping all regmap entries in this test.
Comments are welcomed.

Regards,
Jan

[1]
[15341.137116] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [dd:636573]
[15341.144141] Modules linked in: nls_utf8 isofs dummy minix binfmt_misc nfsv3 nfs_acl nfs lockd grace fscache sctp rds brd vfat fat btrfs xor zstd_compress raid6_pq zstd_decompress loop tu
n ip6table_nat ip6_tables xt_conntrack iptable_filter xt_MASQUERADE xt_comment iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 veth bridge stp llc overlay fuse snd_hda_codec_r
ealtek snd_hda_codec_generic sunrpc ledtrig_audio snd_hda_codec_hdmi snd_hda_intel snd_hda_codec edac_mce_amd kvm_amd snd_hda_core ccp snd_hwdep snd_pcm kvm snd_timer irqbypass joydev snd c
rct10dif_pclmul crc32_pclmul soundcore broadcom bcm_phy_lib ghash_clmulni_intel sp5100_tco fam15h_power k10temp tg3 wmi_bmof pcspkr i2c_piix4 acpi_cpufreq ip_tables xfs libcrc32c radeon ata
_generic i2c_algo_bit pata_acpi drm_kms_helper firewire_ohci ttm crc32c_intel serio_raw pata_atiixp firewire_core drm crc_itu_t wmi [last unloaded: can]
[15341.223635] CPU: 2 PID: 636573 Comm: dd Tainted: G             L    5.3.9-rc1-dfe283e.cki #1
[15341.232073] Hardware name: AMD Pike/Pike, BIOS RPK1506A 09/03/2014
[15341.238467] RIP: 0010:regmap_readable+0x15/0x60
[15341.242996] Code: 25 28 00 00 00 75 07 48 83 c4 10 5b 5d c3 e8 92 08 a6 ff 66 90 0f 1f 44 00 00 48 83 bf b0 01 00 00 00 74 31 8b 97 48 01 00 00 <39> f2 0f 92 c0 85 d2 0f 95 c2 20 d0 75 1
d 48 83 7f 70 00 74 01 c3
[15341.261765] RSP: 0018:ffffb3b100697dc8 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff13
[15341.269330] RAX: 0000000000000000 RBX: ffff8d1a63773400 RCX: 0000000000000b41
[15341.276723] RDX: 000000000fffffff RSI: 000000000f3b4139 RDI: ffff8d1a63773400
[15341.283858] RBP: 000000000f3b4139 R08: 0000000000000000 R09: 0000000000000000
[15341.290989] R10: 000000000000000f R11: ffff8d19901fffff R12: 0000000000000079
[15341.298114] R13: 000000000f3b4139 R14: 00000000ffffffff R15: 000000000000006e
[15341.305501] FS:  00007f4e067a0580(0000) GS:ffff8d1a6aa80000(0000) knlGS:0000000000000000
[15341.313583] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15341.319322] CR2: 00007f5083682dc0 CR3: 00000001287dc000 CR4: 00000000000406e0
[15341.326462] Call Trace:
[15341.328912]  regmap_volatile+0x4f/0xa0
[15341.332667]  regmap_access_show+0x70/0x100
[15341.336997]  seq_read+0xcd/0x400
[15341.340247]  full_proxy_read+0x57/0x70
[15341.343997]  vfs_read+0x9d/0x150
[15341.347231]  ksys_read+0x5f/0xe0
[15341.350473]  do_syscall_64+0x5f/0x1a0
[15341.354166]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[15341.359216] RIP: 0033:0x7f4e066823c2
[15341.362795] Code: c0 e9 c2 fe ff ff 50 48 8d 3d c2 0d 0a 00 e8 b5 f1 01 00 0f 1f 44 00 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 0f 05 <48> 3d 00 f0 ff ff 77 56 c3 0f 1f 44 00 0
0 48 83 ec 28 48 89 54 24
[15341.381852] RSP: 002b:00007ffc6ad30e88 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[15341.389427] RAX: ffffffffffffffda RBX: 0000000000000400 RCX: 00007f4e066823c2
[15341.396568] RDX: 0000000000000400 RSI: 0000561bc69f0000 RDI: 0000000000000000
[15341.403979] RBP: 0000561bc69f0000 R08: 0000561bc69efb60 R09: 00000000000000c0
[15341.411111] R10: 0000561bc69f0000 R11: 0000000000000246 R12: ffffffffffffffff
[15341.418244] R13: 0000000000000000 R14: 0000000000000000 R15: ffffffffffffffff


WARNING: multiple messages have this Message-ID (diff)
From: Jan Stancek <jstancek@redhat.com>
To: CKI Project <cki-project@redhat.com>,
	Jaroslav Kysela <jkysela@redhat.com>,
	alsa-devel@alsa-project.org,
	LTP Mailing List <ltp@lists.linux.it>
Cc: Memory Management <mm-qe@redhat.com>,
	Linux Stable maillist <stable@vger.kernel.org>
Subject: Re: [alsa-devel] ❌ FAIL: Test report for kernel 5.3.9-rc1-dfe283e.cki (stable)
Date: Mon, 4 Nov 2019 08:35:51 -0500 (EST)	[thread overview]
Message-ID: <2029139028.10333037.1572874551626.JavaMail.zimbra@redhat.com> (raw)
In-Reply-To: <cki.1210A7ECB0.BD9Q3APV4K@redhat.com>



----- Original Message -----
> 
> Hello,
> 
> We ran automated tests on a recent commit from this kernel tree:
> 
>        Kernel repo:
>        git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
>             Commit: dfe283e9fdac - Linux 5.3.9-rc1
> 
> The results of these automated tests are provided below.
> 
>     Overall result: FAILED (see details below)
>              Merge: OK
>            Compile: OK
>              Tests: FAILED
> 
> All kernel binaries, config files, and logs are available for download here:
> 
>   https://artifacts.cki-project.org/pipelines/262380
> 
> One or more kernel tests failed:
> 
>     x86_64:
>      ❌ LTP lite
>

Not a 5.3 -stable regression.

Failure comes from test that sanity checks all /proc files by doing
1k read from each. There are couple issues it hits wrt. snd_hda_*.

Example reproducer:
  dd if=/sys/kernel/debug/regmap/hdaudioC0D3-hdaudio/access of=out.txt count=1 bs=1024 iflag=nonblock

It's slow and triggers soft lockups [1]. And it also requires lot
of memory, triggering OOMs on smaller VMs:
0x0000000024f0437b-0x000000001a32b1c8 1073745920 seq_read+0x131/0x400 pages=262144 vmalloc vpages N0=262144

I'm leaning towards skipping all regmap entries in this test.
Comments are welcomed.

Regards,
Jan

[1]
[15341.137116] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [dd:636573]
[15341.144141] Modules linked in: nls_utf8 isofs dummy minix binfmt_misc nfsv3 nfs_acl nfs lockd grace fscache sctp rds brd vfat fat btrfs xor zstd_compress raid6_pq zstd_decompress loop tu
n ip6table_nat ip6_tables xt_conntrack iptable_filter xt_MASQUERADE xt_comment iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 veth bridge stp llc overlay fuse snd_hda_codec_r
ealtek snd_hda_codec_generic sunrpc ledtrig_audio snd_hda_codec_hdmi snd_hda_intel snd_hda_codec edac_mce_amd kvm_amd snd_hda_core ccp snd_hwdep snd_pcm kvm snd_timer irqbypass joydev snd c
rct10dif_pclmul crc32_pclmul soundcore broadcom bcm_phy_lib ghash_clmulni_intel sp5100_tco fam15h_power k10temp tg3 wmi_bmof pcspkr i2c_piix4 acpi_cpufreq ip_tables xfs libcrc32c radeon ata
_generic i2c_algo_bit pata_acpi drm_kms_helper firewire_ohci ttm crc32c_intel serio_raw pata_atiixp firewire_core drm crc_itu_t wmi [last unloaded: can]
[15341.223635] CPU: 2 PID: 636573 Comm: dd Tainted: G             L    5.3.9-rc1-dfe283e.cki #1
[15341.232073] Hardware name: AMD Pike/Pike, BIOS RPK1506A 09/03/2014
[15341.238467] RIP: 0010:regmap_readable+0x15/0x60
[15341.242996] Code: 25 28 00 00 00 75 07 48 83 c4 10 5b 5d c3 e8 92 08 a6 ff 66 90 0f 1f 44 00 00 48 83 bf b0 01 00 00 00 74 31 8b 97 48 01 00 00 <39> f2 0f 92 c0 85 d2 0f 95 c2 20 d0 75 1
d 48 83 7f 70 00 74 01 c3
[15341.261765] RSP: 0018:ffffb3b100697dc8 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff13
[15341.269330] RAX: 0000000000000000 RBX: ffff8d1a63773400 RCX: 0000000000000b41
[15341.276723] RDX: 000000000fffffff RSI: 000000000f3b4139 RDI: ffff8d1a63773400
[15341.283858] RBP: 000000000f3b4139 R08: 0000000000000000 R09: 0000000000000000
[15341.290989] R10: 000000000000000f R11: ffff8d19901fffff R12: 0000000000000079
[15341.298114] R13: 000000000f3b4139 R14: 00000000ffffffff R15: 000000000000006e
[15341.305501] FS:  00007f4e067a0580(0000) GS:ffff8d1a6aa80000(0000) knlGS:0000000000000000
[15341.313583] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15341.319322] CR2: 00007f5083682dc0 CR3: 00000001287dc000 CR4: 00000000000406e0
[15341.326462] Call Trace:
[15341.328912]  regmap_volatile+0x4f/0xa0
[15341.332667]  regmap_access_show+0x70/0x100
[15341.336997]  seq_read+0xcd/0x400
[15341.340247]  full_proxy_read+0x57/0x70
[15341.343997]  vfs_read+0x9d/0x150
[15341.347231]  ksys_read+0x5f/0xe0
[15341.350473]  do_syscall_64+0x5f/0x1a0
[15341.354166]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[15341.359216] RIP: 0033:0x7f4e066823c2
[15341.362795] Code: c0 e9 c2 fe ff ff 50 48 8d 3d c2 0d 0a 00 e8 b5 f1 01 00 0f 1f 44 00 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 0f 05 <48> 3d 00 f0 ff ff 77 56 c3 0f 1f 44 00 0
0 48 83 ec 28 48 89 54 24
[15341.381852] RSP: 002b:00007ffc6ad30e88 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[15341.389427] RAX: ffffffffffffffda RBX: 0000000000000400 RCX: 00007f4e066823c2
[15341.396568] RDX: 0000000000000400 RSI: 0000561bc69f0000 RDI: 0000000000000000
[15341.403979] RBP: 0000561bc69f0000 R08: 0000561bc69efb60 R09: 00000000000000c0
[15341.411111] R10: 0000561bc69f0000 R11: 0000000000000246 R12: ffffffffffffffff
[15341.418244] R13: 0000000000000000 R14: 0000000000000000 R15: ffffffffffffffff

_______________________________________________
Alsa-devel mailing list
Alsa-devel@alsa-project.org
https://mailman.alsa-project.org/mailman/listinfo/alsa-devel

WARNING: multiple messages have this Message-ID (diff)
From: Jan Stancek <jstancek@redhat.com>
To: ltp@lists.linux.it
Subject: [LTP]  ❌ FAIL: Test report for kernel 5.3.9-rc1-dfe283e.cki (stable)
Date: Mon, 4 Nov 2019 08:35:51 -0500 (EST)	[thread overview]
Message-ID: <2029139028.10333037.1572874551626.JavaMail.zimbra@redhat.com> (raw)
In-Reply-To: <cki.1210A7ECB0.BD9Q3APV4K@redhat.com>



----- Original Message -----
> 
> Hello,
> 
> We ran automated tests on a recent commit from this kernel tree:
> 
>        Kernel repo:
>        git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
>             Commit: dfe283e9fdac - Linux 5.3.9-rc1
> 
> The results of these automated tests are provided below.
> 
>     Overall result: FAILED (see details below)
>              Merge: OK
>            Compile: OK
>              Tests: FAILED
> 
> All kernel binaries, config files, and logs are available for download here:
> 
>   https://artifacts.cki-project.org/pipelines/262380
> 
> One or more kernel tests failed:
> 
>     x86_64:
>      ? LTP lite
>

Not a 5.3 -stable regression.

Failure comes from test that sanity checks all /proc files by doing
1k read from each. There are couple issues it hits wrt. snd_hda_*.

Example reproducer:
  dd if=/sys/kernel/debug/regmap/hdaudioC0D3-hdaudio/access of=out.txt count=1 bs=1024 iflag=nonblock

It's slow and triggers soft lockups [1]. And it also requires lot
of memory, triggering OOMs on smaller VMs:
0x0000000024f0437b-0x000000001a32b1c8 1073745920 seq_read+0x131/0x400 pages=262144 vmalloc vpages N0=262144

I'm leaning towards skipping all regmap entries in this test.
Comments are welcomed.

Regards,
Jan

[1]
[15341.137116] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [dd:636573]
[15341.144141] Modules linked in: nls_utf8 isofs dummy minix binfmt_misc nfsv3 nfs_acl nfs lockd grace fscache sctp rds brd vfat fat btrfs xor zstd_compress raid6_pq zstd_decompress loop tu
n ip6table_nat ip6_tables xt_conntrack iptable_filter xt_MASQUERADE xt_comment iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 veth bridge stp llc overlay fuse snd_hda_codec_r
ealtek snd_hda_codec_generic sunrpc ledtrig_audio snd_hda_codec_hdmi snd_hda_intel snd_hda_codec edac_mce_amd kvm_amd snd_hda_core ccp snd_hwdep snd_pcm kvm snd_timer irqbypass joydev snd c
rct10dif_pclmul crc32_pclmul soundcore broadcom bcm_phy_lib ghash_clmulni_intel sp5100_tco fam15h_power k10temp tg3 wmi_bmof pcspkr i2c_piix4 acpi_cpufreq ip_tables xfs libcrc32c radeon ata
_generic i2c_algo_bit pata_acpi drm_kms_helper firewire_ohci ttm crc32c_intel serio_raw pata_atiixp firewire_core drm crc_itu_t wmi [last unloaded: can]
[15341.223635] CPU: 2 PID: 636573 Comm: dd Tainted: G             L    5.3.9-rc1-dfe283e.cki #1
[15341.232073] Hardware name: AMD Pike/Pike, BIOS RPK1506A 09/03/2014
[15341.238467] RIP: 0010:regmap_readable+0x15/0x60
[15341.242996] Code: 25 28 00 00 00 75 07 48 83 c4 10 5b 5d c3 e8 92 08 a6 ff 66 90 0f 1f 44 00 00 48 83 bf b0 01 00 00 00 74 31 8b 97 48 01 00 00 <39> f2 0f 92 c0 85 d2 0f 95 c2 20 d0 75 1
d 48 83 7f 70 00 74 01 c3
[15341.261765] RSP: 0018:ffffb3b100697dc8 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff13
[15341.269330] RAX: 0000000000000000 RBX: ffff8d1a63773400 RCX: 0000000000000b41
[15341.276723] RDX: 000000000fffffff RSI: 000000000f3b4139 RDI: ffff8d1a63773400
[15341.283858] RBP: 000000000f3b4139 R08: 0000000000000000 R09: 0000000000000000
[15341.290989] R10: 000000000000000f R11: ffff8d19901fffff R12: 0000000000000079
[15341.298114] R13: 000000000f3b4139 R14: 00000000ffffffff R15: 000000000000006e
[15341.305501] FS:  00007f4e067a0580(0000) GS:ffff8d1a6aa80000(0000) knlGS:0000000000000000
[15341.313583] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15341.319322] CR2: 00007f5083682dc0 CR3: 00000001287dc000 CR4: 00000000000406e0
[15341.326462] Call Trace:
[15341.328912]  regmap_volatile+0x4f/0xa0
[15341.332667]  regmap_access_show+0x70/0x100
[15341.336997]  seq_read+0xcd/0x400
[15341.340247]  full_proxy_read+0x57/0x70
[15341.343997]  vfs_read+0x9d/0x150
[15341.347231]  ksys_read+0x5f/0xe0
[15341.350473]  do_syscall_64+0x5f/0x1a0
[15341.354166]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[15341.359216] RIP: 0033:0x7f4e066823c2
[15341.362795] Code: c0 e9 c2 fe ff ff 50 48 8d 3d c2 0d 0a 00 e8 b5 f1 01 00 0f 1f 44 00 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 0f 05 <48> 3d 00 f0 ff ff 77 56 c3 0f 1f 44 00 0
0 48 83 ec 28 48 89 54 24
[15341.381852] RSP: 002b:00007ffc6ad30e88 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[15341.389427] RAX: ffffffffffffffda RBX: 0000000000000400 RCX: 00007f4e066823c2
[15341.396568] RDX: 0000000000000400 RSI: 0000561bc69f0000 RDI: 0000000000000000
[15341.403979] RBP: 0000561bc69f0000 R08: 0000561bc69efb60 R09: 00000000000000c0
[15341.411111] R10: 0000561bc69f0000 R11: 0000000000000246 R12: ffffffffffffffff
[15341.418244] R13: 0000000000000000 R14: 0000000000000000 R15: ffffffffffffffff


  reply	other threads:[~2019-11-04 13:35 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-04  3:36 ❌ FAIL: Test report for kernel 5.3.9-rc1-dfe283e.cki (stable) CKI Project
2019-11-04  3:36 ` [LTP] " CKI Project
2019-11-04 13:35 ` Jan Stancek [this message]
2019-11-04 13:35   ` Jan Stancek
2019-11-04 13:35   ` [alsa-devel] " Jan Stancek
2019-11-04 13:51   ` Greg KH
2019-11-04 13:51     ` [LTP] " Greg KH
2019-11-04 13:51     ` [alsa-devel] " Greg KH
2019-11-04 14:28     ` Jan Stancek
2019-11-04 14:28       ` [LTP] " Jan Stancek
2019-11-04 14:28       ` [alsa-devel] " Jan Stancek
2019-11-04 14:59       ` Greg KH
2019-11-04 14:59         ` [LTP] " Greg KH
2019-11-04 14:59         ` [alsa-devel] " Greg KH
2019-11-04 15:25         ` Jan Stancek
2019-11-04 15:25           ` [LTP] " Jan Stancek
2019-11-04 15:25           ` [alsa-devel] " Jan Stancek
2019-11-04 16:30           ` Greg KH
2019-11-04 16:30             ` [LTP] " Greg KH
2019-11-04 16:30             ` [alsa-devel] " Greg KH
2019-11-04 17:02             ` Jan Stancek
2019-11-04 17:02               ` [LTP] " Jan Stancek
2019-11-04 17:02               ` [alsa-devel] " Jan Stancek
2019-11-04 20:32               ` Greg KH
2019-11-04 20:32                 ` [LTP] " Greg KH
2019-11-04 20:32                 ` [alsa-devel] " Greg KH
2019-11-07 11:05                 ` [LTP] " Richard Palethorpe
2019-11-07 11:05                   ` Richard Palethorpe
2019-11-07 11:05                   ` [alsa-devel] " Richard Palethorpe
2019-11-05  8:15   ` [alsa-devel] " Takashi Iwai
2019-11-05  8:15     ` [LTP] " Takashi Iwai
2019-11-05  8:15     ` Takashi Iwai

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2029139028.10333037.1572874551626.JavaMail.zimbra@redhat.com \
    --to=jstancek@redhat.com \
    --cc=alsa-devel@alsa-project.org \
    --cc=cki-project@redhat.com \
    --cc=jkysela@redhat.com \
    --cc=ltp@lists.linux.it \
    --cc=mm-qe@redhat.com \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.