* KASAN use after free in deferred close
@ 2021-05-18 22:42 ronnie sahlberg
2021-05-19 5:14 ` Rohith Surabattula
0 siblings, 1 reply; 4+ messages in thread
From: ronnie sahlberg @ 2021-05-18 22:42 UTC (permalink / raw)
To: linux-cifs, rohiths msft; +Cc: Steve French
List, Rorith,
I got a hit in KASAN for a use after free that looks like it is
related to the recent deferred close patches. Can you please have a
look?
[ 473.779989] run fstests generic/013 at 2021-05-19 08:27:00
[ 612.157429] ==================================================================
[ 612.158275] BUG: KASAN: use-after-free in process_one_work+0x90/0x9b0
[ 612.158801] Read of size 8 at addr ffff88810a31ca60 by task kworker/2:9/2382
[ 612.159611] CPU: 2 PID: 2382 Comm: kworker/2:9 Tainted: G
OE 5.13.0-rc2+ #98
[ 612.159623] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.14.0-1.fc33 04/01/2014
[ 612.159640] Workqueue: 0x0 (deferredclose)
[ 612.159669] Call Trace:
[ 612.159685] dump_stack+0xbb/0x107
[ 612.159711] print_address_description.constprop.0+0x18/0x140
[ 612.159733] ? process_one_work+0x90/0x9b0
[ 612.159743] ? process_one_work+0x90/0x9b0
[ 612.159754] kasan_report.cold+0x7c/0xd8
[ 612.159778] ? lock_is_held_type+0x80/0x130
[ 612.159789] ? process_one_work+0x90/0x9b0
[ 612.159812] kasan_check_range+0x145/0x1a0
[ 612.159834] process_one_work+0x90/0x9b0
[ 612.159877] ? pwq_dec_nr_in_flight+0x110/0x110
[ 612.159914] ? spin_bug+0x90/0x90
[ 612.159967] worker_thread+0x3b6/0x6c0
[ 612.160023] ? process_one_work+0x9b0/0x9b0
[ 612.160038] kthread+0x1dc/0x200
[ 612.160051] ? kthread_create_worker_on_cpu+0xd0/0xd0
[ 612.160092] ret_from_fork+0x1f/0x30
[ 612.160399] Allocated by task 2358:
[ 612.160757] kasan_save_stack+0x1b/0x40
[ 612.160768] __kasan_kmalloc+0x9b/0xd0
[ 612.160778] cifs_new_fileinfo+0xb0/0x960 [cifs]
[ 612.161170] cifs_open+0xadf/0xf20 [cifs]
[ 612.161421] do_dentry_open+0x2aa/0x6b0
[ 612.161432] path_openat+0xbd9/0xfa0
[ 612.161441] do_filp_open+0x11d/0x230
[ 612.161450] do_sys_openat2+0x115/0x240
[ 612.161460] __x64_sys_openat+0xce/0x140
[ 612.161470] do_syscall_64+0x3a/0x70
[ 612.161486] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 612.161721] Freed by task 2382:
[ 612.162241] kasan_save_stack+0x1b/0x40
[ 612.162253] kasan_set_track+0x1c/0x30
[ 612.162263] kasan_set_free_info+0x20/0x30
[ 612.162272] __kasan_slab_free+0x108/0x150
[ 612.162282] slab_free_freelist_hook+0xf9/0x2c0
[ 612.162294] kfree+0xce/0x350
[ 612.162302] _cifsFileInfo_put+0x42d/0x6a0 [cifs]
[ 612.162612] process_one_work+0x4f2/0x9b0
[ 612.162622] worker_thread+0x2d3/0x6c0
[ 612.162631] kthread+0x1dc/0x200
[ 612.162639] ret_from_fork+0x1f/0x30
[ 612.162989] Last potentially related work creation:
[ 612.163583] kasan_save_stack+0x1b/0x40
[ 612.163594] kasan_record_aux_stack+0xc1/0xd0
[ 612.163605] insert_work+0x32/0x160
[ 612.163614] __queue_work+0x35e/0x7e0
[ 612.163625] mod_delayed_work_on+0x98/0x110
[ 612.163635] cifs_close_all_deferred_files+0x8a/0xb0 [cifs]
[ 612.163888] cifs_unlink+0x20c/0x780 [cifs]
[ 612.164149] vfs_unlink+0x194/0x2e0
[ 612.164162] do_unlinkat+0x28b/0x400
[ 612.164172] do_syscall_64+0x3a/0x70
[ 612.164183] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 612.164557] Second to last potentially related work creation:
[ 612.165183] kasan_save_stack+0x1b/0x40
[ 612.165195] kasan_record_aux_stack+0xc1/0xd0
[ 612.165205] insert_work+0x32/0x160
[ 612.165215] __queue_work+0x35e/0x7e0
[ 612.165225] queue_delayed_work_on+0xa6/0xc0
[ 612.165235] cifs_close+0x18d/0x270 [cifs]
[ 612.165486] __fput+0x115/0x3d0
[ 612.165498] task_work_run+0x85/0xc0
[ 612.165510] exit_to_user_mode_prepare+0x1fd/0x200
[ 612.165520] syscall_exit_to_user_mode+0x27/0x70
[ 612.165531] do_syscall_64+0x47/0x70
[ 612.165542] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 612.165921] The buggy address belongs to the object at ffff88810a31c800
which belongs to the cache kmalloc-1k of size 1024
[ 612.167111] The buggy address is located 608 bytes inside of
1024-byte region [ffff88810a31c800, ffff88810a31cc00)
[ 612.168215] The buggy address belongs to the page:
[ 612.168794] page:00000000d0b7a3cf refcount:1 mapcount:0
mapping:0000000000000000 index:0x0 pfn:0x10a318
[ 612.168807] head:00000000d0b7a3cf order:3 compound_mapcount:0
compound_pincount:0
[ 612.168815] memcg:ffff88810d800901
[ 612.168822] flags:
0x17ffe000010200(slab|head|node=0|zone=2|lastcpupid=0x3fff)
[ 612.168835] raw: 0017ffe000010200 dead000000000100 dead000000000122
ffff888100042dc0
[ 612.168845] raw: 0000000000000000 0000000080100010 00000001ffffffff
ffff88810d800901
[ 612.168852] page dumped because: kasan: bad access detected
[ 612.169163] Memory state around the buggy address:
[ 612.169605] ffff88810a31c900: fb fb fb fb fb fb fb fb fb fb fb fb
fb fb fb fb
[ 612.170243] ffff88810a31c980: fb fb fb fb fb fb fb fb fb fb fb fb
fb fb fb fb
[ 612.170930] >ffff88810a31ca00: fb fb fb fb fb fb fb fb fb fb fb fb
fb fb fb fb
[ 612.171545] ^
[ 612.172068] ffff88810a31ca80: fb fb fb fb fb fb fb fb fb fb fb fb
fb fb fb fb
[ 612.172832] ffff88810a31cb00: fb fb fb fb fb fb fb fb fb fb fb fb
fb fb fb fb
[ 612.173683] ==================================================================
[ 612.174498] Disabling lock debugging due to kernel taint
regards
ronnie sahlberg
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: KASAN use after free in deferred close 2021-05-18 22:42 KASAN use after free in deferred close ronnie sahlberg @ 2021-05-19 5:14 ` Rohith Surabattula 2021-05-19 5:38 ` ronnie sahlberg 0 siblings, 1 reply; 4+ messages in thread From: Rohith Surabattula @ 2021-05-19 5:14 UTC (permalink / raw) To: ronnie sahlberg; +Cc: linux-cifs, Steve French Hi Ronnie, Did you hit the issue with the latest for-next? Do you have below patch in your code repo: https://git.samba.org/?p=sfrench/cifs-2.6.git;a=commit;h=e87dbd1cec70a32e670647f0bfb07e57cf974288 Regards, Rohith On Wed, May 19, 2021 at 4:12 AM ronnie sahlberg <ronniesahlberg@gmail.com> wrote: > > List, Rorith, > I got a hit in KASAN for a use after free that looks like it is > related to the recent deferred close patches. Can you please have a > look? > > [ 473.779989] run fstests generic/013 at 2021-05-19 08:27:00 > [ 612.157429] ================================================================== > [ 612.158275] BUG: KASAN: use-after-free in process_one_work+0x90/0x9b0 > [ 612.158801] Read of size 8 at addr ffff88810a31ca60 by task kworker/2:9/2382 > > [ 612.159611] CPU: 2 PID: 2382 Comm: kworker/2:9 Tainted: G > OE 5.13.0-rc2+ #98 > [ 612.159623] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), > BIOS 1.14.0-1.fc33 04/01/2014 > [ 612.159640] Workqueue: 0x0 (deferredclose) > [ 612.159669] Call Trace: > [ 612.159685] dump_stack+0xbb/0x107 > [ 612.159711] print_address_description.constprop.0+0x18/0x140 > [ 612.159733] ? process_one_work+0x90/0x9b0 > [ 612.159743] ? process_one_work+0x90/0x9b0 > [ 612.159754] kasan_report.cold+0x7c/0xd8 > [ 612.159778] ? lock_is_held_type+0x80/0x130 > [ 612.159789] ? process_one_work+0x90/0x9b0 > [ 612.159812] kasan_check_range+0x145/0x1a0 > [ 612.159834] process_one_work+0x90/0x9b0 > [ 612.159877] ? pwq_dec_nr_in_flight+0x110/0x110 > [ 612.159914] ? spin_bug+0x90/0x90 > [ 612.159967] worker_thread+0x3b6/0x6c0 > [ 612.160023] ? process_one_work+0x9b0/0x9b0 > [ 612.160038] kthread+0x1dc/0x200 > [ 612.160051] ? kthread_create_worker_on_cpu+0xd0/0xd0 > [ 612.160092] ret_from_fork+0x1f/0x30 > > [ 612.160399] Allocated by task 2358: > [ 612.160757] kasan_save_stack+0x1b/0x40 > [ 612.160768] __kasan_kmalloc+0x9b/0xd0 > [ 612.160778] cifs_new_fileinfo+0xb0/0x960 [cifs] > [ 612.161170] cifs_open+0xadf/0xf20 [cifs] > [ 612.161421] do_dentry_open+0x2aa/0x6b0 > [ 612.161432] path_openat+0xbd9/0xfa0 > [ 612.161441] do_filp_open+0x11d/0x230 > [ 612.161450] do_sys_openat2+0x115/0x240 > [ 612.161460] __x64_sys_openat+0xce/0x140 > [ 612.161470] do_syscall_64+0x3a/0x70 > [ 612.161486] entry_SYSCALL_64_after_hwframe+0x44/0xae > > [ 612.161721] Freed by task 2382: > [ 612.162241] kasan_save_stack+0x1b/0x40 > [ 612.162253] kasan_set_track+0x1c/0x30 > [ 612.162263] kasan_set_free_info+0x20/0x30 > [ 612.162272] __kasan_slab_free+0x108/0x150 > [ 612.162282] slab_free_freelist_hook+0xf9/0x2c0 > [ 612.162294] kfree+0xce/0x350 > [ 612.162302] _cifsFileInfo_put+0x42d/0x6a0 [cifs] > [ 612.162612] process_one_work+0x4f2/0x9b0 > [ 612.162622] worker_thread+0x2d3/0x6c0 > [ 612.162631] kthread+0x1dc/0x200 > [ 612.162639] ret_from_fork+0x1f/0x30 > > [ 612.162989] Last potentially related work creation: > [ 612.163583] kasan_save_stack+0x1b/0x40 > [ 612.163594] kasan_record_aux_stack+0xc1/0xd0 > [ 612.163605] insert_work+0x32/0x160 > [ 612.163614] __queue_work+0x35e/0x7e0 > [ 612.163625] mod_delayed_work_on+0x98/0x110 > [ 612.163635] cifs_close_all_deferred_files+0x8a/0xb0 [cifs] > [ 612.163888] cifs_unlink+0x20c/0x780 [cifs] > [ 612.164149] vfs_unlink+0x194/0x2e0 > [ 612.164162] do_unlinkat+0x28b/0x400 > [ 612.164172] do_syscall_64+0x3a/0x70 > [ 612.164183] entry_SYSCALL_64_after_hwframe+0x44/0xae > > [ 612.164557] Second to last potentially related work creation: > [ 612.165183] kasan_save_stack+0x1b/0x40 > [ 612.165195] kasan_record_aux_stack+0xc1/0xd0 > [ 612.165205] insert_work+0x32/0x160 > [ 612.165215] __queue_work+0x35e/0x7e0 > [ 612.165225] queue_delayed_work_on+0xa6/0xc0 > [ 612.165235] cifs_close+0x18d/0x270 [cifs] > [ 612.165486] __fput+0x115/0x3d0 > [ 612.165498] task_work_run+0x85/0xc0 > [ 612.165510] exit_to_user_mode_prepare+0x1fd/0x200 > [ 612.165520] syscall_exit_to_user_mode+0x27/0x70 > [ 612.165531] do_syscall_64+0x47/0x70 > [ 612.165542] entry_SYSCALL_64_after_hwframe+0x44/0xae > > [ 612.165921] The buggy address belongs to the object at ffff88810a31c800 > which belongs to the cache kmalloc-1k of size 1024 > [ 612.167111] The buggy address is located 608 bytes inside of > 1024-byte region [ffff88810a31c800, ffff88810a31cc00) > [ 612.168215] The buggy address belongs to the page: > [ 612.168794] page:00000000d0b7a3cf refcount:1 mapcount:0 > mapping:0000000000000000 index:0x0 pfn:0x10a318 > [ 612.168807] head:00000000d0b7a3cf order:3 compound_mapcount:0 > compound_pincount:0 > [ 612.168815] memcg:ffff88810d800901 > [ 612.168822] flags: > 0x17ffe000010200(slab|head|node=0|zone=2|lastcpupid=0x3fff) > [ 612.168835] raw: 0017ffe000010200 dead000000000100 dead000000000122 > ffff888100042dc0 > [ 612.168845] raw: 0000000000000000 0000000080100010 00000001ffffffff > ffff88810d800901 > [ 612.168852] page dumped because: kasan: bad access detected > > [ 612.169163] Memory state around the buggy address: > [ 612.169605] ffff88810a31c900: fb fb fb fb fb fb fb fb fb fb fb fb > fb fb fb fb > [ 612.170243] ffff88810a31c980: fb fb fb fb fb fb fb fb fb fb fb fb > fb fb fb fb > [ 612.170930] >ffff88810a31ca00: fb fb fb fb fb fb fb fb fb fb fb fb > fb fb fb fb > [ 612.171545] ^ > [ 612.172068] ffff88810a31ca80: fb fb fb fb fb fb fb fb fb fb fb fb > fb fb fb fb > [ 612.172832] ffff88810a31cb00: fb fb fb fb fb fb fb fb fb fb fb fb > fb fb fb fb > [ 612.173683] ================================================================== > [ 612.174498] Disabling lock debugging due to kernel taint > > > regards > ronnie sahlberg ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: KASAN use after free in deferred close 2021-05-19 5:14 ` Rohith Surabattula @ 2021-05-19 5:38 ` ronnie sahlberg 2021-05-20 15:19 ` Rohith Surabattula 0 siblings, 1 reply; 4+ messages in thread From: ronnie sahlberg @ 2021-05-19 5:38 UTC (permalink / raw) To: Rohith Surabattula; +Cc: linux-cifs, Steve French On Wed, May 19, 2021 at 3:14 PM Rohith Surabattula <rohiths.msft@gmail.com> wrote: > > Hi Ronnie, > > Did you hit the issue with the latest for-next? > Do you have below patch in your code repo: > https://git.samba.org/?p=sfrench/cifs-2.6.git;a=commit;h=e87dbd1cec70a32e670647f0bfb07e57cf974288 Yes, I got it at for-next at 93a47dd8 which is one commit after that one. so current for-next It triggers for me a minute or two into running generic/013 against a win16 server. > > Regards, > Rohith > > On Wed, May 19, 2021 at 4:12 AM ronnie sahlberg > <ronniesahlberg@gmail.com> wrote: > > > > List, Rorith, > > I got a hit in KASAN for a use after free that looks like it is > > related to the recent deferred close patches. Can you please have a > > look? > > > > [ 473.779989] run fstests generic/013 at 2021-05-19 08:27:00 > > [ 612.157429] ================================================================== > > [ 612.158275] BUG: KASAN: use-after-free in process_one_work+0x90/0x9b0 > > [ 612.158801] Read of size 8 at addr ffff88810a31ca60 by task kworker/2:9/2382 > > > > [ 612.159611] CPU: 2 PID: 2382 Comm: kworker/2:9 Tainted: G > > OE 5.13.0-rc2+ #98 > > [ 612.159623] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), > > BIOS 1.14.0-1.fc33 04/01/2014 > > [ 612.159640] Workqueue: 0x0 (deferredclose) > > [ 612.159669] Call Trace: > > [ 612.159685] dump_stack+0xbb/0x107 > > [ 612.159711] print_address_description.constprop.0+0x18/0x140 > > [ 612.159733] ? process_one_work+0x90/0x9b0 > > [ 612.159743] ? process_one_work+0x90/0x9b0 > > [ 612.159754] kasan_report.cold+0x7c/0xd8 > > [ 612.159778] ? lock_is_held_type+0x80/0x130 > > [ 612.159789] ? process_one_work+0x90/0x9b0 > > [ 612.159812] kasan_check_range+0x145/0x1a0 > > [ 612.159834] process_one_work+0x90/0x9b0 > > [ 612.159877] ? pwq_dec_nr_in_flight+0x110/0x110 > > [ 612.159914] ? spin_bug+0x90/0x90 > > [ 612.159967] worker_thread+0x3b6/0x6c0 > > [ 612.160023] ? process_one_work+0x9b0/0x9b0 > > [ 612.160038] kthread+0x1dc/0x200 > > [ 612.160051] ? kthread_create_worker_on_cpu+0xd0/0xd0 > > [ 612.160092] ret_from_fork+0x1f/0x30 > > > > [ 612.160399] Allocated by task 2358: > > [ 612.160757] kasan_save_stack+0x1b/0x40 > > [ 612.160768] __kasan_kmalloc+0x9b/0xd0 > > [ 612.160778] cifs_new_fileinfo+0xb0/0x960 [cifs] > > [ 612.161170] cifs_open+0xadf/0xf20 [cifs] > > [ 612.161421] do_dentry_open+0x2aa/0x6b0 > > [ 612.161432] path_openat+0xbd9/0xfa0 > > [ 612.161441] do_filp_open+0x11d/0x230 > > [ 612.161450] do_sys_openat2+0x115/0x240 > > [ 612.161460] __x64_sys_openat+0xce/0x140 > > [ 612.161470] do_syscall_64+0x3a/0x70 > > [ 612.161486] entry_SYSCALL_64_after_hwframe+0x44/0xae > > > > [ 612.161721] Freed by task 2382: > > [ 612.162241] kasan_save_stack+0x1b/0x40 > > [ 612.162253] kasan_set_track+0x1c/0x30 > > [ 612.162263] kasan_set_free_info+0x20/0x30 > > [ 612.162272] __kasan_slab_free+0x108/0x150 > > [ 612.162282] slab_free_freelist_hook+0xf9/0x2c0 > > [ 612.162294] kfree+0xce/0x350 > > [ 612.162302] _cifsFileInfo_put+0x42d/0x6a0 [cifs] > > [ 612.162612] process_one_work+0x4f2/0x9b0 > > [ 612.162622] worker_thread+0x2d3/0x6c0 > > [ 612.162631] kthread+0x1dc/0x200 > > [ 612.162639] ret_from_fork+0x1f/0x30 > > > > [ 612.162989] Last potentially related work creation: > > [ 612.163583] kasan_save_stack+0x1b/0x40 > > [ 612.163594] kasan_record_aux_stack+0xc1/0xd0 > > [ 612.163605] insert_work+0x32/0x160 > > [ 612.163614] __queue_work+0x35e/0x7e0 > > [ 612.163625] mod_delayed_work_on+0x98/0x110 > > [ 612.163635] cifs_close_all_deferred_files+0x8a/0xb0 [cifs] > > [ 612.163888] cifs_unlink+0x20c/0x780 [cifs] > > [ 612.164149] vfs_unlink+0x194/0x2e0 > > [ 612.164162] do_unlinkat+0x28b/0x400 > > [ 612.164172] do_syscall_64+0x3a/0x70 > > [ 612.164183] entry_SYSCALL_64_after_hwframe+0x44/0xae > > > > [ 612.164557] Second to last potentially related work creation: > > [ 612.165183] kasan_save_stack+0x1b/0x40 > > [ 612.165195] kasan_record_aux_stack+0xc1/0xd0 > > [ 612.165205] insert_work+0x32/0x160 > > [ 612.165215] __queue_work+0x35e/0x7e0 > > [ 612.165225] queue_delayed_work_on+0xa6/0xc0 > > [ 612.165235] cifs_close+0x18d/0x270 [cifs] > > [ 612.165486] __fput+0x115/0x3d0 > > [ 612.165498] task_work_run+0x85/0xc0 > > [ 612.165510] exit_to_user_mode_prepare+0x1fd/0x200 > > [ 612.165520] syscall_exit_to_user_mode+0x27/0x70 > > [ 612.165531] do_syscall_64+0x47/0x70 > > [ 612.165542] entry_SYSCALL_64_after_hwframe+0x44/0xae > > > > [ 612.165921] The buggy address belongs to the object at ffff88810a31c800 > > which belongs to the cache kmalloc-1k of size 1024 > > [ 612.167111] The buggy address is located 608 bytes inside of > > 1024-byte region [ffff88810a31c800, ffff88810a31cc00) > > [ 612.168215] The buggy address belongs to the page: > > [ 612.168794] page:00000000d0b7a3cf refcount:1 mapcount:0 > > mapping:0000000000000000 index:0x0 pfn:0x10a318 > > [ 612.168807] head:00000000d0b7a3cf order:3 compound_mapcount:0 > > compound_pincount:0 > > [ 612.168815] memcg:ffff88810d800901 > > [ 612.168822] flags: > > 0x17ffe000010200(slab|head|node=0|zone=2|lastcpupid=0x3fff) > > [ 612.168835] raw: 0017ffe000010200 dead000000000100 dead000000000122 > > ffff888100042dc0 > > [ 612.168845] raw: 0000000000000000 0000000080100010 00000001ffffffff > > ffff88810d800901 > > [ 612.168852] page dumped because: kasan: bad access detected > > > > [ 612.169163] Memory state around the buggy address: > > [ 612.169605] ffff88810a31c900: fb fb fb fb fb fb fb fb fb fb fb fb > > fb fb fb fb > > [ 612.170243] ffff88810a31c980: fb fb fb fb fb fb fb fb fb fb fb fb > > fb fb fb fb > > [ 612.170930] >ffff88810a31ca00: fb fb fb fb fb fb fb fb fb fb fb fb > > fb fb fb fb > > [ 612.171545] ^ > > [ 612.172068] ffff88810a31ca80: fb fb fb fb fb fb fb fb fb fb fb fb > > fb fb fb fb > > [ 612.172832] ffff88810a31cb00: fb fb fb fb fb fb fb fb fb fb fb fb > > fb fb fb fb > > [ 612.173683] ================================================================== > > [ 612.174498] Disabling lock debugging due to kernel taint > > > > > > regards > > ronnie sahlberg ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: KASAN use after free in deferred close 2021-05-19 5:38 ` ronnie sahlberg @ 2021-05-20 15:19 ` Rohith Surabattula 0 siblings, 0 replies; 4+ messages in thread From: Rohith Surabattula @ 2021-05-20 15:19 UTC (permalink / raw) To: ronnie sahlberg; +Cc: linux-cifs, Steve French [-- Attachment #1: Type: text/plain, Size: 6830 bytes --] Hi All, Attached the patch to address the use-after-free issue. Roonie, Can you please review and help in validating the fix. Regards, Rohith On Wed, May 19, 2021 at 11:08 AM ronnie sahlberg <ronniesahlberg@gmail.com> wrote: > > On Wed, May 19, 2021 at 3:14 PM Rohith Surabattula > <rohiths.msft@gmail.com> wrote: > > > > Hi Ronnie, > > > > Did you hit the issue with the latest for-next? > > Do you have below patch in your code repo: > > https://git.samba.org/?p=sfrench/cifs-2.6.git;a=commit;h=e87dbd1cec70a32e670647f0bfb07e57cf974288 > > Yes, I got it at for-next at 93a47dd8 which is one commit after that one. > so current for-next > > It triggers for me a minute or two into running generic/013 against a > win16 server. > > > > > > Regards, > > Rohith > > > > On Wed, May 19, 2021 at 4:12 AM ronnie sahlberg > > <ronniesahlberg@gmail.com> wrote: > > > > > > List, Rorith, > > > I got a hit in KASAN for a use after free that looks like it is > > > related to the recent deferred close patches. Can you please have a > > > look? > > > > > > [ 473.779989] run fstests generic/013 at 2021-05-19 08:27:00 > > > [ 612.157429] ================================================================== > > > [ 612.158275] BUG: KASAN: use-after-free in process_one_work+0x90/0x9b0 > > > [ 612.158801] Read of size 8 at addr ffff88810a31ca60 by task kworker/2:9/2382 > > > > > > [ 612.159611] CPU: 2 PID: 2382 Comm: kworker/2:9 Tainted: G > > > OE 5.13.0-rc2+ #98 > > > [ 612.159623] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), > > > BIOS 1.14.0-1.fc33 04/01/2014 > > > [ 612.159640] Workqueue: 0x0 (deferredclose) > > > [ 612.159669] Call Trace: > > > [ 612.159685] dump_stack+0xbb/0x107 > > > [ 612.159711] print_address_description.constprop.0+0x18/0x140 > > > [ 612.159733] ? process_one_work+0x90/0x9b0 > > > [ 612.159743] ? process_one_work+0x90/0x9b0 > > > [ 612.159754] kasan_report.cold+0x7c/0xd8 > > > [ 612.159778] ? lock_is_held_type+0x80/0x130 > > > [ 612.159789] ? process_one_work+0x90/0x9b0 > > > [ 612.159812] kasan_check_range+0x145/0x1a0 > > > [ 612.159834] process_one_work+0x90/0x9b0 > > > [ 612.159877] ? pwq_dec_nr_in_flight+0x110/0x110 > > > [ 612.159914] ? spin_bug+0x90/0x90 > > > [ 612.159967] worker_thread+0x3b6/0x6c0 > > > [ 612.160023] ? process_one_work+0x9b0/0x9b0 > > > [ 612.160038] kthread+0x1dc/0x200 > > > [ 612.160051] ? kthread_create_worker_on_cpu+0xd0/0xd0 > > > [ 612.160092] ret_from_fork+0x1f/0x30 > > > > > > [ 612.160399] Allocated by task 2358: > > > [ 612.160757] kasan_save_stack+0x1b/0x40 > > > [ 612.160768] __kasan_kmalloc+0x9b/0xd0 > > > [ 612.160778] cifs_new_fileinfo+0xb0/0x960 [cifs] > > > [ 612.161170] cifs_open+0xadf/0xf20 [cifs] > > > [ 612.161421] do_dentry_open+0x2aa/0x6b0 > > > [ 612.161432] path_openat+0xbd9/0xfa0 > > > [ 612.161441] do_filp_open+0x11d/0x230 > > > [ 612.161450] do_sys_openat2+0x115/0x240 > > > [ 612.161460] __x64_sys_openat+0xce/0x140 > > > [ 612.161470] do_syscall_64+0x3a/0x70 > > > [ 612.161486] entry_SYSCALL_64_after_hwframe+0x44/0xae > > > > > > [ 612.161721] Freed by task 2382: > > > [ 612.162241] kasan_save_stack+0x1b/0x40 > > > [ 612.162253] kasan_set_track+0x1c/0x30 > > > [ 612.162263] kasan_set_free_info+0x20/0x30 > > > [ 612.162272] __kasan_slab_free+0x108/0x150 > > > [ 612.162282] slab_free_freelist_hook+0xf9/0x2c0 > > > [ 612.162294] kfree+0xce/0x350 > > > [ 612.162302] _cifsFileInfo_put+0x42d/0x6a0 [cifs] > > > [ 612.162612] process_one_work+0x4f2/0x9b0 > > > [ 612.162622] worker_thread+0x2d3/0x6c0 > > > [ 612.162631] kthread+0x1dc/0x200 > > > [ 612.162639] ret_from_fork+0x1f/0x30 > > > > > > [ 612.162989] Last potentially related work creation: > > > [ 612.163583] kasan_save_stack+0x1b/0x40 > > > [ 612.163594] kasan_record_aux_stack+0xc1/0xd0 > > > [ 612.163605] insert_work+0x32/0x160 > > > [ 612.163614] __queue_work+0x35e/0x7e0 > > > [ 612.163625] mod_delayed_work_on+0x98/0x110 > > > [ 612.163635] cifs_close_all_deferred_files+0x8a/0xb0 [cifs] > > > [ 612.163888] cifs_unlink+0x20c/0x780 [cifs] > > > [ 612.164149] vfs_unlink+0x194/0x2e0 > > > [ 612.164162] do_unlinkat+0x28b/0x400 > > > [ 612.164172] do_syscall_64+0x3a/0x70 > > > [ 612.164183] entry_SYSCALL_64_after_hwframe+0x44/0xae > > > > > > [ 612.164557] Second to last potentially related work creation: > > > [ 612.165183] kasan_save_stack+0x1b/0x40 > > > [ 612.165195] kasan_record_aux_stack+0xc1/0xd0 > > > [ 612.165205] insert_work+0x32/0x160 > > > [ 612.165215] __queue_work+0x35e/0x7e0 > > > [ 612.165225] queue_delayed_work_on+0xa6/0xc0 > > > [ 612.165235] cifs_close+0x18d/0x270 [cifs] > > > [ 612.165486] __fput+0x115/0x3d0 > > > [ 612.165498] task_work_run+0x85/0xc0 > > > [ 612.165510] exit_to_user_mode_prepare+0x1fd/0x200 > > > [ 612.165520] syscall_exit_to_user_mode+0x27/0x70 > > > [ 612.165531] do_syscall_64+0x47/0x70 > > > [ 612.165542] entry_SYSCALL_64_after_hwframe+0x44/0xae > > > > > > [ 612.165921] The buggy address belongs to the object at ffff88810a31c800 > > > which belongs to the cache kmalloc-1k of size 1024 > > > [ 612.167111] The buggy address is located 608 bytes inside of > > > 1024-byte region [ffff88810a31c800, ffff88810a31cc00) > > > [ 612.168215] The buggy address belongs to the page: > > > [ 612.168794] page:00000000d0b7a3cf refcount:1 mapcount:0 > > > mapping:0000000000000000 index:0x0 pfn:0x10a318 > > > [ 612.168807] head:00000000d0b7a3cf order:3 compound_mapcount:0 > > > compound_pincount:0 > > > [ 612.168815] memcg:ffff88810d800901 > > > [ 612.168822] flags: > > > 0x17ffe000010200(slab|head|node=0|zone=2|lastcpupid=0x3fff) > > > [ 612.168835] raw: 0017ffe000010200 dead000000000100 dead000000000122 > > > ffff888100042dc0 > > > [ 612.168845] raw: 0000000000000000 0000000080100010 00000001ffffffff > > > ffff88810d800901 > > > [ 612.168852] page dumped because: kasan: bad access detected > > > > > > [ 612.169163] Memory state around the buggy address: > > > [ 612.169605] ffff88810a31c900: fb fb fb fb fb fb fb fb fb fb fb fb > > > fb fb fb fb > > > [ 612.170243] ffff88810a31c980: fb fb fb fb fb fb fb fb fb fb fb fb > > > fb fb fb fb > > > [ 612.170930] >ffff88810a31ca00: fb fb fb fb fb fb fb fb fb fb fb fb > > > fb fb fb fb > > > [ 612.171545] ^ > > > [ 612.172068] ffff88810a31ca80: fb fb fb fb fb fb fb fb fb fb fb fb > > > fb fb fb fb > > > [ 612.172832] ffff88810a31cb00: fb fb fb fb fb fb fb fb fb fb fb fb > > > fb fb fb fb > > > [ 612.173683] ================================================================== > > > [ 612.174498] Disabling lock debugging due to kernel taint > > > > > > > > > regards > > > ronnie sahlberg [-- Attachment #2: 0001-Fix-KASAN-identified-use-after-free-issue.patch --] [-- Type: application/octet-stream, Size: 5360 bytes --] From 560f8c51ce8edd47943c9aca4f18b5dbfd5b19ad Mon Sep 17 00:00:00 2001 From: Rohith Surabattula <rohiths@microsoft.com> Date: Thu, 20 May 2021 14:36:57 +0000 Subject: [PATCH] Fix KASAN identified use-after-free issue. [ 612.157429] ================================================================== [ 612.158275] BUG: KASAN: use-after-free in process_one_work+0x90/0x9b0 [ 612.158801] Read of size 8 at addr ffff88810a31ca60 by task kworker/2:9/2382 [ 612.159611] CPU: 2 PID: 2382 Comm: kworker/2:9 Tainted: G OE 5.13.0-rc2+ #98 [ 612.159623] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-1.fc33 04/01/2014 [ 612.159640] Workqueue: 0x0 (deferredclose) [ 612.159669] Call Trace: [ 612.159685] dump_stack+0xbb/0x107 [ 612.159711] print_address_description.constprop.0+0x18/0x140 [ 612.159733] ? process_one_work+0x90/0x9b0 [ 612.159743] ? process_one_work+0x90/0x9b0 [ 612.159754] kasan_report.cold+0x7c/0xd8 [ 612.159778] ? lock_is_held_type+0x80/0x130 [ 612.159789] ? process_one_work+0x90/0x9b0 [ 612.159812] kasan_check_range+0x145/0x1a0 [ 612.159834] process_one_work+0x90/0x9b0 [ 612.159877] ? pwq_dec_nr_in_flight+0x110/0x110 [ 612.159914] ? spin_bug+0x90/0x90 [ 612.159967] worker_thread+0x3b6/0x6c0 [ 612.160023] ? process_one_work+0x9b0/0x9b0 [ 612.160038] kthread+0x1dc/0x200 [ 612.160051] ? kthread_create_worker_on_cpu+0xd0/0xd0 [ 612.160092] ret_from_fork+0x1f/0x30 [ 612.160399] Allocated by task 2358: [ 612.160757] kasan_save_stack+0x1b/0x40 [ 612.160768] __kasan_kmalloc+0x9b/0xd0 [ 612.160778] cifs_new_fileinfo+0xb0/0x960 [cifs] [ 612.161170] cifs_open+0xadf/0xf20 [cifs] [ 612.161421] do_dentry_open+0x2aa/0x6b0 [ 612.161432] path_openat+0xbd9/0xfa0 [ 612.161441] do_filp_open+0x11d/0x230 [ 612.161450] do_sys_openat2+0x115/0x240 [ 612.161460] __x64_sys_openat+0xce/0x140 When mod_delayed_work is called to modify the delay of pending work, it might return false and queue a new work when pending work is already scheduled or when try to grab pending work failed. So, Increase the reference count when new work is scheduled to avoid use-after-free. Signed-off-by: Rohith Surabattula <rohiths@microsoft.com> --- fs/cifs/file.c | 20 +++++++++++++------- fs/cifs/misc.c | 12 ++++++++++-- 2 files changed, 23 insertions(+), 9 deletions(-) diff --git a/fs/cifs/file.c b/fs/cifs/file.c index 4cfa22cfbc90..36ee01cbf3a0 100644 --- a/fs/cifs/file.c +++ b/fs/cifs/file.c @@ -874,10 +874,6 @@ void smb2_deferred_work_close(struct work_struct *work) struct cifsFileInfo, deferred.work); spin_lock(&CIFS_I(d_inode(cfile->dentry))->deferred_lock); - if (!cfile->deferred_close_scheduled) { - spin_unlock(&CIFS_I(d_inode(cfile->dentry))->deferred_lock); - return; - } cifs_del_deferred_close(cfile); cfile->deferred_close_scheduled = false; spin_unlock(&CIFS_I(d_inode(cfile->dentry))->deferred_lock); @@ -904,8 +900,13 @@ int cifs_close(struct inode *inode, struct file *file) cifs_add_deferred_close(cfile, dclose); if (cfile->deferred_close_scheduled && delayed_work_pending(&cfile->deferred)) { - mod_delayed_work(deferredclose_wq, - &cfile->deferred, cifs_sb->ctx->acregmax); + /* + * If there is no pending work, mod_delayed_work queues new work. + * So, Increase the ref count to avoid use-after-free. + */ + if (!mod_delayed_work(deferredclose_wq, + &cfile->deferred, cifs_sb->ctx->acregmax)) + cifsFileInfo_get(cfile); } else { /* Deferred close for files */ queue_delayed_work(deferredclose_wq, @@ -4879,7 +4880,12 @@ void cifs_oplock_break(struct work_struct *work) if (is_deferred && cfile->deferred_close_scheduled && delayed_work_pending(&cfile->deferred)) { - mod_delayed_work(deferredclose_wq, &cfile->deferred, 0); + /* + * If there is no pending work, mod_delayed_work queues new work. + * So, Increase the ref count to avoid use-after-free. + */ + if (!mod_delayed_work(deferredclose_wq, &cfile->deferred, 0)) + cifsFileInfo_get(cfile); } spin_unlock(&CIFS_I(inode)->deferred_lock); _cifsFileInfo_put(cfile, false /* do not wait for ourself */, false); diff --git a/fs/cifs/misc.c b/fs/cifs/misc.c index cd705f8a4e31..7cd6c2164c68 100644 --- a/fs/cifs/misc.c +++ b/fs/cifs/misc.c @@ -674,6 +674,8 @@ cifs_add_pending_open(struct cifs_fid *fid, struct tcon_link *tlink, /* * Critical section which runs after acquiring deferred_lock. + * As there is no reference count on cifs_deferred_close, pdclose + * should not be used outside deferred_lock. */ bool cifs_is_deferred_close(struct cifsFileInfo *cfile, struct cifs_deferred_close **pdclose) @@ -754,8 +756,14 @@ cifs_close_all_deferred_files(struct cifs_tcon *tcon) list_for_each(tmp, &tcon->openFileList) { cfile = list_entry(tmp, struct cifsFileInfo, tlist); cinode = CIFS_I(d_inode(cfile->dentry)); - if (delayed_work_pending(&cfile->deferred)) - mod_delayed_work(deferredclose_wq, &cfile->deferred, 0); + if (delayed_work_pending(&cfile->deferred)) { + /* + * If there is no pending work, mod_delayed_work queues new work. + * So, Increase the ref count to avoid use-after-free. + */ + if (!mod_delayed_work(deferredclose_wq, &cfile->deferred, 0)) + cifsFileInfo_get(cfile); + } } spin_unlock(&tcon->open_file_lock); } -- 2.30.2 ^ permalink raw reply related [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-05-20 15:19 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-05-18 22:42 KASAN use after free in deferred close ronnie sahlberg 2021-05-19 5:14 ` Rohith Surabattula 2021-05-19 5:38 ` ronnie sahlberg 2021-05-20 15:19 ` Rohith Surabattula
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).