* Re: circular lock warning at: btrfs_record_root_in_trans, kernel 5.0.0-0.rc4+
[not found] <CAJCQCtRDuss2vTYCA=W6ifXOyLkD-WzSS5ytfruAbWrDOoFx2w@mail.gmail.com>
@ 2019-02-14 8:06 ` Chris Murphy
2019-02-14 8:10 ` Chris Murphy
0 siblings, 1 reply; 2+ messages in thread
From: Chris Murphy @ 2019-02-14 8:06 UTC (permalink / raw)
To: Btrfs BTRFS
[-- Attachment #1: Type: text/plain, Size: 206 bytes --]
I'm still running into this with 5.0.0-rc6 and sssd. I'm not sure what
the issue is but it often does prevent sssd from starting up, so
that's not good.
https://bugzilla.redhat.com/show_bug.cgi?id=1677163
[-- Attachment #2: circularlockwarn5.0.0-rc6.txt --]
[-- Type: text/plain, Size: 14686 bytes --]
[ 53.313230] localhost.localdomain polkitd[721]: Started polkitd version 0.115
[ 53.460957] localhost.localdomain kernel:
[ 53.461002] localhost.localdomain kernel: ======================================================
[ 53.461056] localhost.localdomain kernel: WARNING: possible circular locking dependency detected
[ 53.461110] localhost.localdomain kernel: 5.0.0-0.rc6.git0.1.fc30.x86_64 #1 Not tainted
[ 53.461155] localhost.localdomain kernel: ------------------------------------------------------
[ 53.461207] localhost.localdomain kernel: sssd/700 is trying to acquire lock:
[ 53.461247] localhost.localdomain kernel: 0000000087d28c82 (&fs_info->reloc_mutex){+.+.}, at: btrfs_record_root_in_trans+0x44/0x70 [btrfs]
[ 53.461378] localhost.localdomain kernel:
but task is already holding lock:
[ 53.461426] localhost.localdomain kernel: 000000008b7fd9a3 (&mm->mmap_sem){++++}, at: vm_mmap_pgoff+0xad/0x120
[ 53.461494] localhost.localdomain kernel:
which lock already depends on the new lock.
[ 53.461561] localhost.localdomain kernel:
the existing dependency chain (in reverse order) is:
[ 53.461623] localhost.localdomain kernel:
-> #5 (&mm->mmap_sem){++++}:
[ 53.461674] localhost.localdomain kernel: _copy_from_user+0x1e/0x90
[ 53.461713] localhost.localdomain kernel: scsi_cmd_ioctl+0x218/0x440
[ 53.461777] localhost.localdomain kernel: cdrom_ioctl+0x3c/0x1068
[ 53.461817] localhost.localdomain kernel: sr_block_ioctl+0xa0/0xd0
[ 53.461855] localhost.localdomain kernel: blkdev_ioctl+0x611/0x9b0
[ 53.461892] localhost.localdomain kernel: block_ioctl+0x41/0x50
[ 53.461966] localhost.localdomain kernel: do_vfs_ioctl+0x408/0x750
[ 53.462005] localhost.localdomain kernel: ksys_ioctl+0x5e/0x90
[ 53.462039] localhost.localdomain kernel: __x64_sys_ioctl+0x16/0x20
[ 53.462078] localhost.localdomain kernel: do_syscall_64+0x5c/0xa0
[ 53.462115] localhost.localdomain kernel: entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 53.462163] localhost.localdomain kernel:
-> #4 (sr_mutex){+.+.}:
[ 53.462210] localhost.localdomain kernel: sr_block_open+0x81/0x100
[ 53.462247] localhost.localdomain kernel: __blkdev_get+0xed/0x590
[ 53.462282] localhost.localdomain kernel: blkdev_get+0x4a/0x380
[ 53.463369] localhost.localdomain kernel: do_dentry_open+0x144/0x370
[ 53.464475] localhost.localdomain kernel: path_openat+0x4e9/0xcb0
[ 53.465603] localhost.localdomain kernel: do_filp_open+0x91/0x100
[ 53.466752] localhost.localdomain kernel: do_sys_open+0x186/0x230
[ 53.467908] localhost.localdomain kernel: do_syscall_64+0x5c/0xa0
[ 53.469123] localhost.localdomain kernel: entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 53.470326] localhost.localdomain kernel:
-> #3 (&bdev->bd_mutex){+.+.}:
[ 53.472773] localhost.localdomain kernel: __blkdev_get+0x7a/0x590
[ 53.474084] localhost.localdomain kernel: blkdev_get+0x214/0x380
[ 53.475382] localhost.localdomain kernel: blkdev_get_by_path+0x46/0x80
[ 53.476746] localhost.localdomain kernel: btrfs_get_bdev_and_sb+0x1b/0xb0 [btrfs]
[ 53.478180] localhost.localdomain kernel: open_fs_devices+0x7a/0x2a0 [btrfs]
[ 53.479606] localhost.localdomain kernel: btrfs_open_devices+0x92/0xa0 [btrfs]
[ 53.481097] localhost.localdomain kernel: btrfs_mount_root+0x30b/0x680 [btrfs]
[ 53.482518] localhost.localdomain kernel: mount_fs+0x4c/0x167
[ 53.483945] localhost.localdomain kernel: vfs_kern_mount.part.0+0x4c/0x160
[ 53.485371] localhost.localdomain kernel: btrfs_mount+0x155/0x8e0 [btrfs]
[ 53.486737] localhost.localdomain kernel: mount_fs+0x4c/0x167
[ 53.488120] localhost.localdomain kernel: vfs_kern_mount.part.0+0x4c/0x160
[ 53.489463] localhost.localdomain kernel: do_mount+0x22d/0xda0
[ 53.490764] localhost.localdomain kernel: ksys_mount+0x7e/0xc0
[ 53.492044] localhost.localdomain kernel: __x64_sys_mount+0x21/0x30
[ 53.493290] localhost.localdomain kernel: do_syscall_64+0x5c/0xa0
[ 53.494505] localhost.localdomain kernel: entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 53.495710] localhost.localdomain kernel:
-> #2 (&fs_devs->device_list_mutex){+.+.}:
[ 53.498179] localhost.localdomain kernel: btrfs_run_dev_stats+0x42/0x3d0 [btrfs]
[ 53.499487] localhost.localdomain kernel: commit_cowonly_roots+0xb8/0x310 [btrfs]
[ 53.500784] localhost.localdomain kernel: btrfs_commit_transaction+0x353/0x9c0 [btrfs]
[ 53.502108] localhost.localdomain kernel: btrfs_sync_file+0x32d/0x460 [btrfs]
[ 53.503347] localhost.localdomain kernel: do_fsync+0x38/0x70
[ 53.504575] localhost.localdomain kernel: __x64_sys_fdatasync+0x13/0x20
[ 53.505818] localhost.localdomain kernel: do_syscall_64+0x5c/0xa0
[ 53.507095] localhost.localdomain kernel: entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 53.508366] localhost.localdomain kernel:
-> #1 (&fs_info->tree_log_mutex){+.+.}:
[ 53.510958] localhost.localdomain kernel: btrfs_commit_transaction+0x2f8/0x9c0 [btrfs]
[ 53.512317] localhost.localdomain kernel: btrfs_sync_file+0x32d/0x460 [btrfs]
[ 53.513628] localhost.localdomain kernel: do_fsync+0x38/0x70
[ 53.514980] localhost.localdomain kernel: __x64_sys_fdatasync+0x13/0x20
[ 53.516296] localhost.localdomain kernel: do_syscall_64+0x5c/0xa0
[ 53.517601] localhost.localdomain kernel: entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 53.518953] localhost.localdomain kernel:
-> #0 (&fs_info->reloc_mutex){+.+.}:
[ 53.521571] localhost.localdomain kernel: __mutex_lock+0x92/0x930
[ 53.523066] localhost.localdomain kernel: btrfs_record_root_in_trans+0x44/0x70 [btrfs]
[ 53.524489] localhost.localdomain kernel: start_transaction+0x95/0x4f0 [btrfs]
[ 53.525949] localhost.localdomain kernel: btrfs_dirty_inode+0x44/0xd0 [btrfs]
[ 53.527324] localhost.localdomain kernel: touch_atime+0xbe/0xe0
[ 53.528750] localhost.localdomain kernel: btrfs_file_mmap+0x3f/0x60 [btrfs]
[ 53.530179] localhost.localdomain kernel: mmap_region+0x410/0x660
[ 53.531576] localhost.localdomain kernel: do_mmap+0x38c/0x500
[ 53.532999] localhost.localdomain kernel: vm_mmap_pgoff+0xd7/0x120
[ 53.534373] localhost.localdomain kernel: ksys_mmap_pgoff+0x18a/0x240
[ 53.535750] localhost.localdomain kernel: do_syscall_64+0x5c/0xa0
[ 53.537158] localhost.localdomain kernel: entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 53.538544] localhost.localdomain kernel:
other info that might help us debug this:
[ 53.542685] localhost.localdomain kernel: Chain exists of:
&fs_info->reloc_mutex --> sr_mutex --> &mm->mmap_sem
[ 53.546850] localhost.localdomain kernel: Possible unsafe locking scenario:
[ 53.549643] localhost.localdomain kernel: CPU0 CPU1
[ 53.551063] localhost.localdomain kernel: ---- ----
[ 53.552436] localhost.localdomain kernel: lock(&mm->mmap_sem);
[ 53.553795] localhost.localdomain kernel: lock(sr_mutex);
[ 53.555223] localhost.localdomain kernel: lock(&mm->mmap_sem);
[ 53.556607] localhost.localdomain kernel: lock(&fs_info->reloc_mutex);
[ 53.558025] localhost.localdomain kernel:
*** DEADLOCK ***
[ 53.562057] localhost.localdomain kernel: 3 locks held by sssd/700:
[ 53.563384] localhost.localdomain kernel: #0: 000000008b7fd9a3 (&mm->mmap_sem){++++}, at: vm_mmap_pgoff+0xad/0x120
[ 53.564779] localhost.localdomain kernel: #1: 00000000af244331 (sb_writers#14){.+.+}, at: touch_atime+0x64/0xe0
[ 53.566222] localhost.localdomain kernel: #2: 000000002d3a3c88 (sb_internal){.+.+}, at: start_transaction+0x37f/0x4f0 [btrfs]
[ 53.567715] localhost.localdomain kernel:
stack backtrace:
[ 53.570509] localhost.localdomain kernel: CPU: 0 PID: 700 Comm: sssd Not tainted 5.0.0-0.rc6.git0.1.fc30.x86_64 #1
[ 53.572005] localhost.localdomain kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 0.0.0 02/06/2015
[ 53.573486] localhost.localdomain kernel: Call Trace:
[ 53.575093] localhost.localdomain kernel: dump_stack+0x85/0xc0
[ 53.576596] localhost.localdomain kernel: print_circular_bug.isra.0.cold+0x15c/0x195
[ 53.578112] localhost.localdomain kernel: __lock_acquire+0x134c/0x1660
[ 53.579583] localhost.localdomain kernel: ? mark_held_locks+0x57/0x80
[ 53.581088] localhost.localdomain kernel: ? lockdep_hardirqs_on+0xed/0x180
[ 53.582563] localhost.localdomain kernel: lock_acquire+0xa2/0x1b0
[ 53.584113] localhost.localdomain kernel: ? btrfs_record_root_in_trans+0x44/0x70 [btrfs]
[ 53.585651] localhost.localdomain kernel: ? btrfs_record_root_in_trans+0x44/0x70 [btrfs]
[ 53.587148] localhost.localdomain kernel: __mutex_lock+0x92/0x930
[ 53.588648] localhost.localdomain kernel: ? btrfs_record_root_in_trans+0x44/0x70 [btrfs]
[ 53.590163] localhost.localdomain kernel: ? rcu_read_lock_sched_held+0x6b/0x80
[ 53.591676] localhost.localdomain kernel: ? btrfs_record_root_in_trans+0x44/0x70 [btrfs]
[ 53.593188] localhost.localdomain kernel: ? sched_clock_cpu+0xc/0xc0
[ 53.594704] localhost.localdomain kernel: ? btrfs_record_root_in_trans+0x44/0x70 [btrfs]
[ 53.596276] localhost.localdomain kernel: btrfs_record_root_in_trans+0x44/0x70 [btrfs]
[ 53.597817] localhost.localdomain kernel: start_transaction+0x95/0x4f0 [btrfs]
[ 53.599354] localhost.localdomain kernel: ? ktime_get+0x120/0x130
[ 53.600897] localhost.localdomain kernel: btrfs_dirty_inode+0x44/0xd0 [btrfs]
[ 53.602450] localhost.localdomain kernel: touch_atime+0xbe/0xe0
[ 53.604055] localhost.localdomain kernel: btrfs_file_mmap+0x3f/0x60 [btrfs]
[ 53.605584] localhost.localdomain kernel: mmap_region+0x410/0x660
[ 53.607129] localhost.localdomain kernel: do_mmap+0x38c/0x500
[ 53.608635] localhost.localdomain kernel: vm_mmap_pgoff+0xd7/0x120
[ 53.610159] localhost.localdomain kernel: ksys_mmap_pgoff+0x18a/0x240
[ 53.611646] localhost.localdomain kernel: ? trace_hardirqs_off_thunk+0x1a/0x1c
[ 53.613188] localhost.localdomain kernel: do_syscall_64+0x5c/0xa0
[ 53.614676] localhost.localdomain kernel: entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 53.616201] localhost.localdomain kernel: RIP: 0033:0x7f308b4723b6
[ 53.617688] localhost.localdomain kernel: Code: 00 00 00 00 f3 0f 1e fa 41 f7 c1 ff 0f 00 00 75 2b 55 48 89 fd 53 89 cb 48 85 ff 74 37 41 89 da 48 89 ef b8 09 00 00 00 0>
[ 53.620958] localhost.localdomain kernel: RSP: 002b:00007fff897b87a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000009
[ 53.622639] localhost.localdomain kernel: RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f308b4723b6
[ 53.624370] localhost.localdomain kernel: RDX: 0000000000000003 RSI: 000000000013a000 RDI: 0000000000000000
[ 53.626114] localhost.localdomain kernel: RBP: 0000000000000000 R08: 0000000000000005 R09: 0000000000000000
[ 53.627817] localhost.localdomain kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 000000000013a000
[ 53.629531] localhost.localdomain kernel: R13: 000000000013a001 R14: 0000000000000001 R15: 00007fff897b89b8
[ 53.737935] localhost.localdomain polkitd[721]: Loading rules from directory /etc/polkit-1/rules.d
[ 53.742930] localhost.localdomain polkitd[721]: Loading rules from directory /usr/share/polkit-1/rules.d
[ 53.894596] localhost.localdomain polkitd[721]: Finished loading, compiling and executing 12 rules
[ 53.899396] localhost.localdomain audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=polkit comm="systemd" exe>
[ 53.899720] localhost.localdomain systemd[1]: Started Authorization Manager.
[ 53.905857] localhost.localdomain polkitd[721]: Acquired the name org.freedesktop.PolicyKit1 on the system bus
[ 54.035788] localhost.localdomain udisksd[679]: Error probing device: Error sending ATA command IDENTIFY PACKET DEVICE to /dev/sr0: Unexpected sense data returned:
0000: 70 00 05 00 00 00 00 0a 00 58 00 01 21 04 00 00 p........X..!...
0010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
(g-io-error-quark, 0)
[ 54.094206] localhost.localdomain sssd[700]: Starting up
[ 54.216577] localhost.localdomain udisksd[679]: Error probing device: Error sending ATA command IDENTIFY PACKET DEVICE to /dev/sr0: Unexpected sense data returned:
0000: 70 00 05 00 00 00 00 0a 00 58 00 01 21 04 00 00 p........X..!...
0010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
(g-io-error-quark, 0)
[ 54.308177] localhost.localdomain systemd[1]: Started Disk Manager.
[ 54.310193] localhost.localdomain audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=udisks2 comm="systemd" ex>
[ 54.312324] localhost.localdomain udisksd[679]: Acquired the name org.freedesktop.UDisks2 on the system message bus
[ 54.697052] localhost.localdomain systemd[1]: sssd.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
[ 54.697497] localhost.localdomain systemd[1]: sssd.service: Failed with result 'exit-code'.
[ 54.700548] localhost.localdomain systemd[1]: Failed to start System Security Services Daemon.
[ 54.709923] localhost.localdomain audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=sssd comm="systemd" exe=">
[ 54.711330] localhost.localdomain systemd[1]: Reached target User and Group Name Lookups.
^ permalink raw reply [flat|nested] 2+ messages in thread