Greetings, 0day kernel testing robot got the below dmesg and the first bad commit is https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master commit 3ba75830ce175550ef45c6524ec62faab8f62c1b Author: J. Bruce Fields AuthorDate: Fri May 17 09:03:38 2019 -0400 Commit: J. Bruce Fields CommitDate: Wed Jul 3 17:52:08 2019 -0400 nfsd4: drc containerization The nfsd duplicate reply cache should not be shared between network namespaces. The most straightforward way to fix this is just to move every global in the code to per-net-namespace memory, so that's what we do. Still todo: sort out which members of nfsd_stats should be global and which per-net-namespace. Signed-off-by: J. Bruce Fields b401170f6d nfsd: don't call nfsd_reply_cache_shutdown twice 3ba75830ce nfsd4: drc containerization 444fc5cde6 Merge tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm +----------------------------------------------------------------+------------+------------+------------+ | | b401170f6d | 3ba75830ce | 444fc5cde6 | +----------------------------------------------------------------+------------+------------+------------+ | boot_successes | 72 | 12 | 10 | | boot_failures | 3 | 13 | 9 | | BUG:kernel_hang_in_boot_stage | 3 | 1 | | | WARNING:possible_circular_locking_dependency_detected | 0 | 11 | 9 | | BUG:kernel_hang_in_test_stage | 0 | 1 | | | WARNING:stack_going_in_the_wrong_direction?ip=path_lookupat/0x | 0 | 1 | | +----------------------------------------------------------------+------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot [ 45.163710] Unable to find swap-space signature [ 45.375718] trinity-c1 (855): attempted to duplicate a private mapping with mremap. This is not supported. [ 46.055744] futex_wake_op: trinity-c1 tries to shift op by -209; fix this program [ 51.011723] [ 51.013378] ====================================================== [ 51.013875] WARNING: possible circular locking dependency detected [ 51.014378] 5.2.0-rc2 #1 Not tainted [ 51.014672] ------------------------------------------------------ [ 51.015182] trinity-c2/886 is trying to acquire lock: [ 51.015593] 000000005405f099 (slab_mutex){+.+.}, at: slab_attr_store+0xa2/0x130 [ 51.016190] [ 51.016190] but task is already holding lock: [ 51.016652] 00000000ac662005 (kn->count#43){++++}, at: kernfs_fop_write+0x286/0x500 [ 51.017266] [ 51.017266] which lock already depends on the new lock. [ 51.017266] [ 51.017909] [ 51.017909] the existing dependency chain (in reverse order) is: [ 51.018497] [ 51.018497] -> #1 (kn->count#43){++++}: [ 51.018956] __lock_acquire+0x7cf/0x1a20 [ 51.019317] lock_acquire+0x17d/0x390 [ 51.019658] __kernfs_remove+0x892/0xae0 [ 51.020020] kernfs_remove_by_name_ns+0x78/0x110 [ 51.020435] sysfs_remove_link+0x55/0xb0 [ 51.020832] sysfs_slab_add+0xc1/0x3e0 [ 51.021332] __kmem_cache_create+0x155/0x200 [ 51.021720] create_cache+0xf5/0x320 [ 51.022054] kmem_cache_create_usercopy+0x179/0x320 [ 51.022486] kmem_cache_create+0x1a/0x30 [ 51.022867] nfsd_reply_cache_init+0x278/0x560 [ 51.023266] nfsd_init_net+0x20f/0x5e0 [ 51.023623] ops_init+0xcb/0x4b0 [ 51.023928] setup_net+0x2fe/0x670 [ 51.024315] copy_net_ns+0x30a/0x3f0 [ 51.024653] create_new_namespaces+0x3c5/0x820 [ 51.025257] unshare_nsproxy_namespaces+0xd1/0x240 [ 51.025881] ksys_unshare+0x506/0x9c0 [ 51.026381] __x64_sys_unshare+0x3a/0x50 [ 51.026937] do_syscall_64+0x110/0x10b0 [ 51.027509] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 51.028175] [ 51.028175] -> #0 (slab_mutex){+.+.}: [ 51.028817] validate_chain+0x1c51/0x2cc0 [ 51.029422] __lock_acquire+0x7cf/0x1a20 [ 51.029947] lock_acquire+0x17d/0x390 [ 51.030438] __mutex_lock+0x100/0xfa0 [ 51.030995] mutex_lock_nested+0x27/0x30 [ 51.031516] slab_attr_store+0xa2/0x130 [ 51.032020] sysfs_kf_write+0x11d/0x180 [ 51.032529] kernfs_fop_write+0x32a/0x500 [ 51.033056] do_loop_readv_writev+0x21d/0x310 [ 51.033627] do_iter_write+0x2e5/0x380 [ 51.034148] vfs_writev+0x170/0x310 [ 51.034616] do_pwritev+0x13e/0x160 [ 51.035100] __x64_sys_pwritev+0xa3/0x110 [ 51.035633] do_syscall_64+0x110/0x10b0 [ 51.036200] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 51.036924] [ 51.036924] other info that might help us debug this: [ 51.036924] [ 51.037876] Possible unsafe locking scenario: [ 51.037876] [ 51.038556] CPU0 CPU1 [ 51.039130] ---- ---- [ 51.039676] lock(kn->count#43); [ 51.040084] lock(slab_mutex); [ 51.040597] lock(kn->count#43); [ 51.041062] lock(slab_mutex); [ 51.041320] [ 51.041320] *** DEADLOCK *** [ 51.041320] [ 51.041793] 3 locks held by trinity-c2/886: [ 51.042128] #0: 000000001f55e152 (sb_writers#5){.+.+}, at: vfs_writev+0x2b9/0x310 [ 51.042739] #1: 00000000c7d6c034 (&of->mutex){+.+.}, at: kernfs_fop_write+0x25b/0x500 [ 51.043400] #2: 00000000ac662005 (kn->count#43){++++}, at: kernfs_fop_write+0x286/0x500 [ 51.044044] [ 51.044044] stack backtrace: [ 51.044398] CPU: 0 PID: 886 Comm: trinity-c2 Not tainted 5.2.0-rc2 #1 [ 51.044911] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 51.045569] Call Trace: [ 51.045807] dump_stack+0x28/0x3d [ 51.046089] print_circular_bug+0x455/0x730 [ 51.046432] check_prev_add+0x22db/0x4470 [ 51.046764] ? is_bpf_text_address+0xc2/0x1c0 [ 51.047136] ? check_irq_usage+0xbb0/0xbb0 [ 51.047475] ? reacquire_held_locks+0x5b0/0x5b0 [ 51.047850] ? kernfs_fop_write+0x1e2/0x500 [ 51.048195] ? validate_chain+0x1c51/0x2cc0 [ 51.048591] validate_chain+0x1c51/0x2cc0 [ 51.048980] ? check_prev_add+0x4470/0x4470 [ 51.049324] ? stack_trace_snprint+0x150/0x150 [ 51.049690] __lock_acquire+0x7cf/0x1a20 [ 51.050018] lock_acquire+0x17d/0x390 [ 51.050322] ? slab_attr_store+0xa2/0x130 [ 51.050656] __mutex_lock+0x100/0xfa0 [ 51.050970] ? slab_attr_store+0xa2/0x130 [ 51.051299] ? slab_attr_store+0xa2/0x130 [ 51.051632] ? __mutex_add_waiter+0x150/0x150 [ 51.051987] ? lock_acquire+0x17d/0x390 [ 51.052302] ? kernfs_fop_write+0x286/0x500 [ 51.052650] ? mutex_lock_nested+0x27/0x30 [ 51.052987] mutex_lock_nested+0x27/0x30 [ 51.053311] slab_attr_store+0xa2/0x130 [ 51.053637] ? print_tracking+0xb0/0xb0 [ 51.053953] sysfs_kf_write+0x11d/0x180 [ 51.054274] ? sysfs_file_ops+0x1f0/0x1f0 [ 51.054608] kernfs_fop_write+0x32a/0x500 [ 51.054967] do_loop_readv_writev+0x21d/0x310 [ 51.055331] ? rw_verify_area+0x11b/0x340 [ 51.055663] do_iter_write+0x2e5/0x380 [ 51.055976] vfs_writev+0x170/0x310 [ 51.056267] ? __x32_compat_sys_writev+0xc0/0xc0 [ 51.056654] ? tracer_preempt_on+0x15/0x2b0 [ 51.056996] ? check_prev_add+0x4470/0x4470 [ 51.057339] ? trace_preempt_on+0x69/0x240 [ 51.057679] ? preempt_count_sub+0x186/0x280 [ 51.058031] ? __task_pid_nr_ns+0x26f/0x560 [ 51.058374] ? find_held_lock+0x3a/0x210 [ 51.058697] ? perf_syscall_enter+0xd1/0x640 [ 51.059069] ? __task_pid_nr_ns+0x26f/0x560 [ 51.059414] ? perf_call_bpf_enter+0x1b0/0x1b0 [ 51.059829] ? do_pwritev+0x13e/0x160 [ 51.060130] do_pwritev+0x13e/0x160 [ 51.060424] __x64_sys_pwritev+0xa3/0x110 [ 51.060754] ? rcu_read_lock_sched_held+0x197/0x1e0 [ 51.061151] do_syscall_64+0x110/0x10b0 [ 51.061468] ? perf_call_bpf_enter+0x1b0/0x1b0 [ 51.061881] ? entry_SYSCALL_64_after_hwframe+0x3e/0xbe [ 51.062305] ? syscall_return_slowpath+0x570/0x570 [ 51.062700] ? trace_hardirqs_off_caller+0x43/0x290 [ 51.063148] ? trace_hardirqs_off_thunk+0x1a/0x1c [ 51.063535] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 51.063965] RIP: 0033:0x457729 [ 51.064254] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 2b 84 00 00 c3 66 2e 0f 1f 84 00 00 00 00 [ 51.065721] RSP: 002b:00007ffc65002668 EFLAGS: 00000246 ORIG_RAX: 0000000000000128 [ 51.066325] RAX: ffffffffffffffda RBX: 0000000000000128 RCX: 0000000000457729 [ 51.066904] RDX: 0000000000000001 RSI: 00000000012be5a0 RDI: 000000000000010d [ 51.067478] RBP: 00007ffc65002710 R08: 0000000062626262 R09: 0000000000000016 [ 51.068172] R10: 00000000c9c9c9c9 R11: 0000000000000246 R12: 0000000000000002 [ 51.068789] R13: 00007fa1d2d6d058 R14: 0000000001057830 R15: 00007fa1d2d6d000 [ 53.943382] scsi_nl_rcv_msg: discarding partial skb [main] 10244 iterations. [F:7172 S:3031 HI:3142] [main] 10229 iterations. [F:7108 S:3063 HI:2080] # HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD git bisect start v5.3 v5.2 -- git bisect bad 5168afe6ef596eaf2ff7a533b780c79ce14445e4 # 02:17 B 0 1 17 0 Merge tag 'for-linus-20190726-2' of git://git.kernel.dk/linux-block git bisect bad 1fa91854dcdf7829e2694873b19f96b65947112d # 02:39 B 2 2 0 1 Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/ide git bisect good 9d22167f34305280c5dd57a74c21651da3c23015 # 03:03 G 24 0 0 0 Merge branch 'next-lsm' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security git bisect bad 4832a4dada1a2baefac76b70e4f3a78e71a7c35c # 03:25 B 1 3 0 0 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/hid/hid git bisect bad 29cd581b59496c26334c910a8b848baa81a6becd # 04:23 B 3 3 0 0 Merge tag 'm68k-for-v5.3-tag2' of git://git.kernel.org/pub/scm/linux/kernel/git/geert/linux-m68k git bisect good a47f5c56b2eb55290e2a8668e9ca9c029990dbf6 # 05:07 G 24 0 0 0 Merge tag 'iomap-5.3-merge-1' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux git bisect good 2e756758e5cb4ea29cba5865d00fad476ce94a93 # 06:05 G 23 0 0 0 Merge tag 'ext4_for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4 git bisect bad d2b6b4c832f7e3067709e8d4970b7b82b44419ac # 06:46 B 1 2 0 0 Merge tag 'nfsd-5.3' of git://linux-nfs.org/~bfields/linux git bisect good 0248a8be6d21dad72b9ce80a7565cf13c11509d8 # 07:19 G 25 0 0 0 Merge tag 'gfs2-for-5.3' of git://git.kernel.org/pub/scm/linux/kernel/git/gfs2/linux-gfs2 git bisect bad 59f8e91b75ecf16f22d62eca0659c13901eff5f3 # 08:26 B 2 3 0 0 nfsd4: use reference count to free client git bisect bad f85d93385e9fe6886a751f647f6812a89bf6bee3 # 08:39 B 5 7 0 0 locks: Cleanup lm_compare_owner and lm_owner_key git bisect bad 9de3ec1d57113e270638254868675d9c048b071e # 09:34 B 4 3 0 0 lockd: prepare nlm_lockowner for use by the server git bisect bad 3ba75830ce175550ef45c6524ec62faab8f62c1b # 09:54 B 4 4 0 0 nfsd4: drc containerization git bisect good b401170f6d9cd4270e0159d9b016f4ccb06caa85 # 10:37 G 23 0 0 0 nfsd: don't call nfsd_reply_cache_shutdown twice # first bad commit: [3ba75830ce175550ef45c6524ec62faab8f62c1b] nfsd4: drc containerization git bisect good b401170f6d9cd4270e0159d9b016f4ccb06caa85 # 10:45 G 70 0 1 1 nfsd: don't call nfsd_reply_cache_shutdown twice # extra tests with debug options git bisect bad 3ba75830ce175550ef45c6524ec62faab8f62c1b # 11:00 B 0 2 18 0 nfsd4: drc containerization # extra tests on head commit of linus/master git bisect bad 444fc5cde64330661bf59944c43844e7d4c2ccd8 # 11:44 B 9 8 0 0 Merge tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm # bad: [444fc5cde64330661bf59944c43844e7d4c2ccd8] Merge tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm # extra tests on linus/master # duplicated: [444fc5cde64330661bf59944c43844e7d4c2ccd8] Merge tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm # extra tests on linux-next/master # 119: [b0523c7b1c9d0edcd6c0fe6d2cb558a9ad5c60a8] Add linux-next specific files for 20200526 --- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/lkp@lists.01.org