* KASAN use after free in deferred close
@ 2021-05-18 22:42 ronnie sahlberg
2021-05-19 5:14 ` Rohith Surabattula
0 siblings, 1 reply; 4+ messages in thread
From: ronnie sahlberg @ 2021-05-18 22:42 UTC (permalink / raw)
To: linux-cifs, rohiths msft; +Cc: Steve French
List, Rorith,
I got a hit in KASAN for a use after free that looks like it is
related to the recent deferred close patches. Can you please have a
look?
[ 473.779989] run fstests generic/013 at 2021-05-19 08:27:00
[ 612.157429] ==================================================================
[ 612.158275] BUG: KASAN: use-after-free in process_one_work+0x90/0x9b0
[ 612.158801] Read of size 8 at addr ffff88810a31ca60 by task kworker/2:9/2382
[ 612.159611] CPU: 2 PID: 2382 Comm: kworker/2:9 Tainted: G
OE 5.13.0-rc2+ #98
[ 612.159623] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.14.0-1.fc33 04/01/2014
[ 612.159640] Workqueue: 0x0 (deferredclose)
[ 612.159669] Call Trace:
[ 612.159685] dump_stack+0xbb/0x107
[ 612.159711] print_address_description.constprop.0+0x18/0x140
[ 612.159733] ? process_one_work+0x90/0x9b0
[ 612.159743] ? process_one_work+0x90/0x9b0
[ 612.159754] kasan_report.cold+0x7c/0xd8
[ 612.159778] ? lock_is_held_type+0x80/0x130
[ 612.159789] ? process_one_work+0x90/0x9b0
[ 612.159812] kasan_check_range+0x145/0x1a0
[ 612.159834] process_one_work+0x90/0x9b0
[ 612.159877] ? pwq_dec_nr_in_flight+0x110/0x110
[ 612.159914] ? spin_bug+0x90/0x90
[ 612.159967] worker_thread+0x3b6/0x6c0
[ 612.160023] ? process_one_work+0x9b0/0x9b0
[ 612.160038] kthread+0x1dc/0x200
[ 612.160051] ? kthread_create_worker_on_cpu+0xd0/0xd0
[ 612.160092] ret_from_fork+0x1f/0x30
[ 612.160399] Allocated by task 2358:
[ 612.160757] kasan_save_stack+0x1b/0x40
[ 612.160768] __kasan_kmalloc+0x9b/0xd0
[ 612.160778] cifs_new_fileinfo+0xb0/0x960 [cifs]
[ 612.161170] cifs_open+0xadf/0xf20 [cifs]
[ 612.161421] do_dentry_open+0x2aa/0x6b0
[ 612.161432] path_openat+0xbd9/0xfa0
[ 612.161441] do_filp_open+0x11d/0x230
[ 612.161450] do_sys_openat2+0x115/0x240
[ 612.161460] __x64_sys_openat+0xce/0x140
[ 612.161470] do_syscall_64+0x3a/0x70
[ 612.161486] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 612.161721] Freed by task 2382:
[ 612.162241] kasan_save_stack+0x1b/0x40
[ 612.162253] kasan_set_track+0x1c/0x30
[ 612.162263] kasan_set_free_info+0x20/0x30
[ 612.162272] __kasan_slab_free+0x108/0x150
[ 612.162282] slab_free_freelist_hook+0xf9/0x2c0
[ 612.162294] kfree+0xce/0x350
[ 612.162302] _cifsFileInfo_put+0x42d/0x6a0 [cifs]
[ 612.162612] process_one_work+0x4f2/0x9b0
[ 612.162622] worker_thread+0x2d3/0x6c0
[ 612.162631] kthread+0x1dc/0x200
[ 612.162639] ret_from_fork+0x1f/0x30
[ 612.162989] Last potentially related work creation:
[ 612.163583] kasan_save_stack+0x1b/0x40
[ 612.163594] kasan_record_aux_stack+0xc1/0xd0
[ 612.163605] insert_work+0x32/0x160
[ 612.163614] __queue_work+0x35e/0x7e0
[ 612.163625] mod_delayed_work_on+0x98/0x110
[ 612.163635] cifs_close_all_deferred_files+0x8a/0xb0 [cifs]
[ 612.163888] cifs_unlink+0x20c/0x780 [cifs]
[ 612.164149] vfs_unlink+0x194/0x2e0
[ 612.164162] do_unlinkat+0x28b/0x400
[ 612.164172] do_syscall_64+0x3a/0x70
[ 612.164183] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 612.164557] Second to last potentially related work creation:
[ 612.165183] kasan_save_stack+0x1b/0x40
[ 612.165195] kasan_record_aux_stack+0xc1/0xd0
[ 612.165205] insert_work+0x32/0x160
[ 612.165215] __queue_work+0x35e/0x7e0
[ 612.165225] queue_delayed_work_on+0xa6/0xc0
[ 612.165235] cifs_close+0x18d/0x270 [cifs]
[ 612.165486] __fput+0x115/0x3d0
[ 612.165498] task_work_run+0x85/0xc0
[ 612.165510] exit_to_user_mode_prepare+0x1fd/0x200
[ 612.165520] syscall_exit_to_user_mode+0x27/0x70
[ 612.165531] do_syscall_64+0x47/0x70
[ 612.165542] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 612.165921] The buggy address belongs to the object at ffff88810a31c800
which belongs to the cache kmalloc-1k of size 1024
[ 612.167111] The buggy address is located 608 bytes inside of
1024-byte region [ffff88810a31c800, ffff88810a31cc00)
[ 612.168215] The buggy address belongs to the page:
[ 612.168794] page:00000000d0b7a3cf refcount:1 mapcount:0
mapping:0000000000000000 index:0x0 pfn:0x10a318
[ 612.168807] head:00000000d0b7a3cf order:3 compound_mapcount:0
compound_pincount:0
[ 612.168815] memcg:ffff88810d800901
[ 612.168822] flags:
0x17ffe000010200(slab|head|node=0|zone=2|lastcpupid=0x3fff)
[ 612.168835] raw: 0017ffe000010200 dead000000000100 dead000000000122
ffff888100042dc0
[ 612.168845] raw: 0000000000000000 0000000080100010 00000001ffffffff
ffff88810d800901
[ 612.168852] page dumped because: kasan: bad access detected
[ 612.169163] Memory state around the buggy address:
[ 612.169605] ffff88810a31c900: fb fb fb fb fb fb fb fb fb fb fb fb
fb fb fb fb
[ 612.170243] ffff88810a31c980: fb fb fb fb fb fb fb fb fb fb fb fb
fb fb fb fb
[ 612.170930] >ffff88810a31ca00: fb fb fb fb fb fb fb fb fb fb fb fb
fb fb fb fb
[ 612.171545] ^
[ 612.172068] ffff88810a31ca80: fb fb fb fb fb fb fb fb fb fb fb fb
fb fb fb fb
[ 612.172832] ffff88810a31cb00: fb fb fb fb fb fb fb fb fb fb fb fb
fb fb fb fb
[ 612.173683] ==================================================================
[ 612.174498] Disabling lock debugging due to kernel taint
regards
ronnie sahlberg
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: KASAN use after free in deferred close
2021-05-18 22:42 KASAN use after free in deferred close ronnie sahlberg
@ 2021-05-19 5:14 ` Rohith Surabattula
2021-05-19 5:38 ` ronnie sahlberg
0 siblings, 1 reply; 4+ messages in thread
From: Rohith Surabattula @ 2021-05-19 5:14 UTC (permalink / raw)
To: ronnie sahlberg; +Cc: linux-cifs, Steve French
Hi Ronnie,
Did you hit the issue with the latest for-next?
Do you have below patch in your code repo:
https://git.samba.org/?p=sfrench/cifs-2.6.git;a=commit;h=e87dbd1cec70a32e670647f0bfb07e57cf974288
Regards,
Rohith
On Wed, May 19, 2021 at 4:12 AM ronnie sahlberg
<ronniesahlberg@gmail.com> wrote:
>
> List, Rorith,
> I got a hit in KASAN for a use after free that looks like it is
> related to the recent deferred close patches. Can you please have a
> look?
>
> [ 473.779989] run fstests generic/013 at 2021-05-19 08:27:00
> [ 612.157429] ==================================================================
> [ 612.158275] BUG: KASAN: use-after-free in process_one_work+0x90/0x9b0
> [ 612.158801] Read of size 8 at addr ffff88810a31ca60 by task kworker/2:9/2382
>
> [ 612.159611] CPU: 2 PID: 2382 Comm: kworker/2:9 Tainted: G
> OE 5.13.0-rc2+ #98
> [ 612.159623] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> BIOS 1.14.0-1.fc33 04/01/2014
> [ 612.159640] Workqueue: 0x0 (deferredclose)
> [ 612.159669] Call Trace:
> [ 612.159685] dump_stack+0xbb/0x107
> [ 612.159711] print_address_description.constprop.0+0x18/0x140
> [ 612.159733] ? process_one_work+0x90/0x9b0
> [ 612.159743] ? process_one_work+0x90/0x9b0
> [ 612.159754] kasan_report.cold+0x7c/0xd8
> [ 612.159778] ? lock_is_held_type+0x80/0x130
> [ 612.159789] ? process_one_work+0x90/0x9b0
> [ 612.159812] kasan_check_range+0x145/0x1a0
> [ 612.159834] process_one_work+0x90/0x9b0
> [ 612.159877] ? pwq_dec_nr_in_flight+0x110/0x110
> [ 612.159914] ? spin_bug+0x90/0x90
> [ 612.159967] worker_thread+0x3b6/0x6c0
> [ 612.160023] ? process_one_work+0x9b0/0x9b0
> [ 612.160038] kthread+0x1dc/0x200
> [ 612.160051] ? kthread_create_worker_on_cpu+0xd0/0xd0
> [ 612.160092] ret_from_fork+0x1f/0x30
>
> [ 612.160399] Allocated by task 2358:
> [ 612.160757] kasan_save_stack+0x1b/0x40
> [ 612.160768] __kasan_kmalloc+0x9b/0xd0
> [ 612.160778] cifs_new_fileinfo+0xb0/0x960 [cifs]
> [ 612.161170] cifs_open+0xadf/0xf20 [cifs]
> [ 612.161421] do_dentry_open+0x2aa/0x6b0
> [ 612.161432] path_openat+0xbd9/0xfa0
> [ 612.161441] do_filp_open+0x11d/0x230
> [ 612.161450] do_sys_openat2+0x115/0x240
> [ 612.161460] __x64_sys_openat+0xce/0x140
> [ 612.161470] do_syscall_64+0x3a/0x70
> [ 612.161486] entry_SYSCALL_64_after_hwframe+0x44/0xae
>
> [ 612.161721] Freed by task 2382:
> [ 612.162241] kasan_save_stack+0x1b/0x40
> [ 612.162253] kasan_set_track+0x1c/0x30
> [ 612.162263] kasan_set_free_info+0x20/0x30
> [ 612.162272] __kasan_slab_free+0x108/0x150
> [ 612.162282] slab_free_freelist_hook+0xf9/0x2c0
> [ 612.162294] kfree+0xce/0x350
> [ 612.162302] _cifsFileInfo_put+0x42d/0x6a0 [cifs]
> [ 612.162612] process_one_work+0x4f2/0x9b0
> [ 612.162622] worker_thread+0x2d3/0x6c0
> [ 612.162631] kthread+0x1dc/0x200
> [ 612.162639] ret_from_fork+0x1f/0x30
>
> [ 612.162989] Last potentially related work creation:
> [ 612.163583] kasan_save_stack+0x1b/0x40
> [ 612.163594] kasan_record_aux_stack+0xc1/0xd0
> [ 612.163605] insert_work+0x32/0x160
> [ 612.163614] __queue_work+0x35e/0x7e0
> [ 612.163625] mod_delayed_work_on+0x98/0x110
> [ 612.163635] cifs_close_all_deferred_files+0x8a/0xb0 [cifs]
> [ 612.163888] cifs_unlink+0x20c/0x780 [cifs]
> [ 612.164149] vfs_unlink+0x194/0x2e0
> [ 612.164162] do_unlinkat+0x28b/0x400
> [ 612.164172] do_syscall_64+0x3a/0x70
> [ 612.164183] entry_SYSCALL_64_after_hwframe+0x44/0xae
>
> [ 612.164557] Second to last potentially related work creation:
> [ 612.165183] kasan_save_stack+0x1b/0x40
> [ 612.165195] kasan_record_aux_stack+0xc1/0xd0
> [ 612.165205] insert_work+0x32/0x160
> [ 612.165215] __queue_work+0x35e/0x7e0
> [ 612.165225] queue_delayed_work_on+0xa6/0xc0
> [ 612.165235] cifs_close+0x18d/0x270 [cifs]
> [ 612.165486] __fput+0x115/0x3d0
> [ 612.165498] task_work_run+0x85/0xc0
> [ 612.165510] exit_to_user_mode_prepare+0x1fd/0x200
> [ 612.165520] syscall_exit_to_user_mode+0x27/0x70
> [ 612.165531] do_syscall_64+0x47/0x70
> [ 612.165542] entry_SYSCALL_64_after_hwframe+0x44/0xae
>
> [ 612.165921] The buggy address belongs to the object at ffff88810a31c800
> which belongs to the cache kmalloc-1k of size 1024
> [ 612.167111] The buggy address is located 608 bytes inside of
> 1024-byte region [ffff88810a31c800, ffff88810a31cc00)
> [ 612.168215] The buggy address belongs to the page:
> [ 612.168794] page:00000000d0b7a3cf refcount:1 mapcount:0
> mapping:0000000000000000 index:0x0 pfn:0x10a318
> [ 612.168807] head:00000000d0b7a3cf order:3 compound_mapcount:0
> compound_pincount:0
> [ 612.168815] memcg:ffff88810d800901
> [ 612.168822] flags:
> 0x17ffe000010200(slab|head|node=0|zone=2|lastcpupid=0x3fff)
> [ 612.168835] raw: 0017ffe000010200 dead000000000100 dead000000000122
> ffff888100042dc0
> [ 612.168845] raw: 0000000000000000 0000000080100010 00000001ffffffff
> ffff88810d800901
> [ 612.168852] page dumped because: kasan: bad access detected
>
> [ 612.169163] Memory state around the buggy address:
> [ 612.169605] ffff88810a31c900: fb fb fb fb fb fb fb fb fb fb fb fb
> fb fb fb fb
> [ 612.170243] ffff88810a31c980: fb fb fb fb fb fb fb fb fb fb fb fb
> fb fb fb fb
> [ 612.170930] >ffff88810a31ca00: fb fb fb fb fb fb fb fb fb fb fb fb
> fb fb fb fb
> [ 612.171545] ^
> [ 612.172068] ffff88810a31ca80: fb fb fb fb fb fb fb fb fb fb fb fb
> fb fb fb fb
> [ 612.172832] ffff88810a31cb00: fb fb fb fb fb fb fb fb fb fb fb fb
> fb fb fb fb
> [ 612.173683] ==================================================================
> [ 612.174498] Disabling lock debugging due to kernel taint
>
>
> regards
> ronnie sahlberg
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: KASAN use after free in deferred close
2021-05-19 5:14 ` Rohith Surabattula
@ 2021-05-19 5:38 ` ronnie sahlberg
2021-05-20 15:19 ` Rohith Surabattula
0 siblings, 1 reply; 4+ messages in thread
From: ronnie sahlberg @ 2021-05-19 5:38 UTC (permalink / raw)
To: Rohith Surabattula; +Cc: linux-cifs, Steve French
On Wed, May 19, 2021 at 3:14 PM Rohith Surabattula
<rohiths.msft@gmail.com> wrote:
>
> Hi Ronnie,
>
> Did you hit the issue with the latest for-next?
> Do you have below patch in your code repo:
> https://git.samba.org/?p=sfrench/cifs-2.6.git;a=commit;h=e87dbd1cec70a32e670647f0bfb07e57cf974288
Yes, I got it at for-next at 93a47dd8 which is one commit after that one.
so current for-next
It triggers for me a minute or two into running generic/013 against a
win16 server.
>
> Regards,
> Rohith
>
> On Wed, May 19, 2021 at 4:12 AM ronnie sahlberg
> <ronniesahlberg@gmail.com> wrote:
> >
> > List, Rorith,
> > I got a hit in KASAN for a use after free that looks like it is
> > related to the recent deferred close patches. Can you please have a
> > look?
> >
> > [ 473.779989] run fstests generic/013 at 2021-05-19 08:27:00
> > [ 612.157429] ==================================================================
> > [ 612.158275] BUG: KASAN: use-after-free in process_one_work+0x90/0x9b0
> > [ 612.158801] Read of size 8 at addr ffff88810a31ca60 by task kworker/2:9/2382
> >
> > [ 612.159611] CPU: 2 PID: 2382 Comm: kworker/2:9 Tainted: G
> > OE 5.13.0-rc2+ #98
> > [ 612.159623] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> > BIOS 1.14.0-1.fc33 04/01/2014
> > [ 612.159640] Workqueue: 0x0 (deferredclose)
> > [ 612.159669] Call Trace:
> > [ 612.159685] dump_stack+0xbb/0x107
> > [ 612.159711] print_address_description.constprop.0+0x18/0x140
> > [ 612.159733] ? process_one_work+0x90/0x9b0
> > [ 612.159743] ? process_one_work+0x90/0x9b0
> > [ 612.159754] kasan_report.cold+0x7c/0xd8
> > [ 612.159778] ? lock_is_held_type+0x80/0x130
> > [ 612.159789] ? process_one_work+0x90/0x9b0
> > [ 612.159812] kasan_check_range+0x145/0x1a0
> > [ 612.159834] process_one_work+0x90/0x9b0
> > [ 612.159877] ? pwq_dec_nr_in_flight+0x110/0x110
> > [ 612.159914] ? spin_bug+0x90/0x90
> > [ 612.159967] worker_thread+0x3b6/0x6c0
> > [ 612.160023] ? process_one_work+0x9b0/0x9b0
> > [ 612.160038] kthread+0x1dc/0x200
> > [ 612.160051] ? kthread_create_worker_on_cpu+0xd0/0xd0
> > [ 612.160092] ret_from_fork+0x1f/0x30
> >
> > [ 612.160399] Allocated by task 2358:
> > [ 612.160757] kasan_save_stack+0x1b/0x40
> > [ 612.160768] __kasan_kmalloc+0x9b/0xd0
> > [ 612.160778] cifs_new_fileinfo+0xb0/0x960 [cifs]
> > [ 612.161170] cifs_open+0xadf/0xf20 [cifs]
> > [ 612.161421] do_dentry_open+0x2aa/0x6b0
> > [ 612.161432] path_openat+0xbd9/0xfa0
> > [ 612.161441] do_filp_open+0x11d/0x230
> > [ 612.161450] do_sys_openat2+0x115/0x240
> > [ 612.161460] __x64_sys_openat+0xce/0x140
> > [ 612.161470] do_syscall_64+0x3a/0x70
> > [ 612.161486] entry_SYSCALL_64_after_hwframe+0x44/0xae
> >
> > [ 612.161721] Freed by task 2382:
> > [ 612.162241] kasan_save_stack+0x1b/0x40
> > [ 612.162253] kasan_set_track+0x1c/0x30
> > [ 612.162263] kasan_set_free_info+0x20/0x30
> > [ 612.162272] __kasan_slab_free+0x108/0x150
> > [ 612.162282] slab_free_freelist_hook+0xf9/0x2c0
> > [ 612.162294] kfree+0xce/0x350
> > [ 612.162302] _cifsFileInfo_put+0x42d/0x6a0 [cifs]
> > [ 612.162612] process_one_work+0x4f2/0x9b0
> > [ 612.162622] worker_thread+0x2d3/0x6c0
> > [ 612.162631] kthread+0x1dc/0x200
> > [ 612.162639] ret_from_fork+0x1f/0x30
> >
> > [ 612.162989] Last potentially related work creation:
> > [ 612.163583] kasan_save_stack+0x1b/0x40
> > [ 612.163594] kasan_record_aux_stack+0xc1/0xd0
> > [ 612.163605] insert_work+0x32/0x160
> > [ 612.163614] __queue_work+0x35e/0x7e0
> > [ 612.163625] mod_delayed_work_on+0x98/0x110
> > [ 612.163635] cifs_close_all_deferred_files+0x8a/0xb0 [cifs]
> > [ 612.163888] cifs_unlink+0x20c/0x780 [cifs]
> > [ 612.164149] vfs_unlink+0x194/0x2e0
> > [ 612.164162] do_unlinkat+0x28b/0x400
> > [ 612.164172] do_syscall_64+0x3a/0x70
> > [ 612.164183] entry_SYSCALL_64_after_hwframe+0x44/0xae
> >
> > [ 612.164557] Second to last potentially related work creation:
> > [ 612.165183] kasan_save_stack+0x1b/0x40
> > [ 612.165195] kasan_record_aux_stack+0xc1/0xd0
> > [ 612.165205] insert_work+0x32/0x160
> > [ 612.165215] __queue_work+0x35e/0x7e0
> > [ 612.165225] queue_delayed_work_on+0xa6/0xc0
> > [ 612.165235] cifs_close+0x18d/0x270 [cifs]
> > [ 612.165486] __fput+0x115/0x3d0
> > [ 612.165498] task_work_run+0x85/0xc0
> > [ 612.165510] exit_to_user_mode_prepare+0x1fd/0x200
> > [ 612.165520] syscall_exit_to_user_mode+0x27/0x70
> > [ 612.165531] do_syscall_64+0x47/0x70
> > [ 612.165542] entry_SYSCALL_64_after_hwframe+0x44/0xae
> >
> > [ 612.165921] The buggy address belongs to the object at ffff88810a31c800
> > which belongs to the cache kmalloc-1k of size 1024
> > [ 612.167111] The buggy address is located 608 bytes inside of
> > 1024-byte region [ffff88810a31c800, ffff88810a31cc00)
> > [ 612.168215] The buggy address belongs to the page:
> > [ 612.168794] page:00000000d0b7a3cf refcount:1 mapcount:0
> > mapping:0000000000000000 index:0x0 pfn:0x10a318
> > [ 612.168807] head:00000000d0b7a3cf order:3 compound_mapcount:0
> > compound_pincount:0
> > [ 612.168815] memcg:ffff88810d800901
> > [ 612.168822] flags:
> > 0x17ffe000010200(slab|head|node=0|zone=2|lastcpupid=0x3fff)
> > [ 612.168835] raw: 0017ffe000010200 dead000000000100 dead000000000122
> > ffff888100042dc0
> > [ 612.168845] raw: 0000000000000000 0000000080100010 00000001ffffffff
> > ffff88810d800901
> > [ 612.168852] page dumped because: kasan: bad access detected
> >
> > [ 612.169163] Memory state around the buggy address:
> > [ 612.169605] ffff88810a31c900: fb fb fb fb fb fb fb fb fb fb fb fb
> > fb fb fb fb
> > [ 612.170243] ffff88810a31c980: fb fb fb fb fb fb fb fb fb fb fb fb
> > fb fb fb fb
> > [ 612.170930] >ffff88810a31ca00: fb fb fb fb fb fb fb fb fb fb fb fb
> > fb fb fb fb
> > [ 612.171545] ^
> > [ 612.172068] ffff88810a31ca80: fb fb fb fb fb fb fb fb fb fb fb fb
> > fb fb fb fb
> > [ 612.172832] ffff88810a31cb00: fb fb fb fb fb fb fb fb fb fb fb fb
> > fb fb fb fb
> > [ 612.173683] ==================================================================
> > [ 612.174498] Disabling lock debugging due to kernel taint
> >
> >
> > regards
> > ronnie sahlberg
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: KASAN use after free in deferred close
2021-05-19 5:38 ` ronnie sahlberg
@ 2021-05-20 15:19 ` Rohith Surabattula
0 siblings, 0 replies; 4+ messages in thread
From: Rohith Surabattula @ 2021-05-20 15:19 UTC (permalink / raw)
To: ronnie sahlberg; +Cc: linux-cifs, Steve French
[-- Attachment #1: Type: text/plain, Size: 6830 bytes --]
Hi All,
Attached the patch to address the use-after-free issue.
Roonie,
Can you please review and help in validating the fix.
Regards,
Rohith
On Wed, May 19, 2021 at 11:08 AM ronnie sahlberg
<ronniesahlberg@gmail.com> wrote:
>
> On Wed, May 19, 2021 at 3:14 PM Rohith Surabattula
> <rohiths.msft@gmail.com> wrote:
> >
> > Hi Ronnie,
> >
> > Did you hit the issue with the latest for-next?
> > Do you have below patch in your code repo:
> > https://git.samba.org/?p=sfrench/cifs-2.6.git;a=commit;h=e87dbd1cec70a32e670647f0bfb07e57cf974288
>
> Yes, I got it at for-next at 93a47dd8 which is one commit after that one.
> so current for-next
>
> It triggers for me a minute or two into running generic/013 against a
> win16 server.
>
>
> >
> > Regards,
> > Rohith
> >
> > On Wed, May 19, 2021 at 4:12 AM ronnie sahlberg
> > <ronniesahlberg@gmail.com> wrote:
> > >
> > > List, Rorith,
> > > I got a hit in KASAN for a use after free that looks like it is
> > > related to the recent deferred close patches. Can you please have a
> > > look?
> > >
> > > [ 473.779989] run fstests generic/013 at 2021-05-19 08:27:00
> > > [ 612.157429] ==================================================================
> > > [ 612.158275] BUG: KASAN: use-after-free in process_one_work+0x90/0x9b0
> > > [ 612.158801] Read of size 8 at addr ffff88810a31ca60 by task kworker/2:9/2382
> > >
> > > [ 612.159611] CPU: 2 PID: 2382 Comm: kworker/2:9 Tainted: G
> > > OE 5.13.0-rc2+ #98
> > > [ 612.159623] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> > > BIOS 1.14.0-1.fc33 04/01/2014
> > > [ 612.159640] Workqueue: 0x0 (deferredclose)
> > > [ 612.159669] Call Trace:
> > > [ 612.159685] dump_stack+0xbb/0x107
> > > [ 612.159711] print_address_description.constprop.0+0x18/0x140
> > > [ 612.159733] ? process_one_work+0x90/0x9b0
> > > [ 612.159743] ? process_one_work+0x90/0x9b0
> > > [ 612.159754] kasan_report.cold+0x7c/0xd8
> > > [ 612.159778] ? lock_is_held_type+0x80/0x130
> > > [ 612.159789] ? process_one_work+0x90/0x9b0
> > > [ 612.159812] kasan_check_range+0x145/0x1a0
> > > [ 612.159834] process_one_work+0x90/0x9b0
> > > [ 612.159877] ? pwq_dec_nr_in_flight+0x110/0x110
> > > [ 612.159914] ? spin_bug+0x90/0x90
> > > [ 612.159967] worker_thread+0x3b6/0x6c0
> > > [ 612.160023] ? process_one_work+0x9b0/0x9b0
> > > [ 612.160038] kthread+0x1dc/0x200
> > > [ 612.160051] ? kthread_create_worker_on_cpu+0xd0/0xd0
> > > [ 612.160092] ret_from_fork+0x1f/0x30
> > >
> > > [ 612.160399] Allocated by task 2358:
> > > [ 612.160757] kasan_save_stack+0x1b/0x40
> > > [ 612.160768] __kasan_kmalloc+0x9b/0xd0
> > > [ 612.160778] cifs_new_fileinfo+0xb0/0x960 [cifs]
> > > [ 612.161170] cifs_open+0xadf/0xf20 [cifs]
> > > [ 612.161421] do_dentry_open+0x2aa/0x6b0
> > > [ 612.161432] path_openat+0xbd9/0xfa0
> > > [ 612.161441] do_filp_open+0x11d/0x230
> > > [ 612.161450] do_sys_openat2+0x115/0x240
> > > [ 612.161460] __x64_sys_openat+0xce/0x140
> > > [ 612.161470] do_syscall_64+0x3a/0x70
> > > [ 612.161486] entry_SYSCALL_64_after_hwframe+0x44/0xae
> > >
> > > [ 612.161721] Freed by task 2382:
> > > [ 612.162241] kasan_save_stack+0x1b/0x40
> > > [ 612.162253] kasan_set_track+0x1c/0x30
> > > [ 612.162263] kasan_set_free_info+0x20/0x30
> > > [ 612.162272] __kasan_slab_free+0x108/0x150
> > > [ 612.162282] slab_free_freelist_hook+0xf9/0x2c0
> > > [ 612.162294] kfree+0xce/0x350
> > > [ 612.162302] _cifsFileInfo_put+0x42d/0x6a0 [cifs]
> > > [ 612.162612] process_one_work+0x4f2/0x9b0
> > > [ 612.162622] worker_thread+0x2d3/0x6c0
> > > [ 612.162631] kthread+0x1dc/0x200
> > > [ 612.162639] ret_from_fork+0x1f/0x30
> > >
> > > [ 612.162989] Last potentially related work creation:
> > > [ 612.163583] kasan_save_stack+0x1b/0x40
> > > [ 612.163594] kasan_record_aux_stack+0xc1/0xd0
> > > [ 612.163605] insert_work+0x32/0x160
> > > [ 612.163614] __queue_work+0x35e/0x7e0
> > > [ 612.163625] mod_delayed_work_on+0x98/0x110
> > > [ 612.163635] cifs_close_all_deferred_files+0x8a/0xb0 [cifs]
> > > [ 612.163888] cifs_unlink+0x20c/0x780 [cifs]
> > > [ 612.164149] vfs_unlink+0x194/0x2e0
> > > [ 612.164162] do_unlinkat+0x28b/0x400
> > > [ 612.164172] do_syscall_64+0x3a/0x70
> > > [ 612.164183] entry_SYSCALL_64_after_hwframe+0x44/0xae
> > >
> > > [ 612.164557] Second to last potentially related work creation:
> > > [ 612.165183] kasan_save_stack+0x1b/0x40
> > > [ 612.165195] kasan_record_aux_stack+0xc1/0xd0
> > > [ 612.165205] insert_work+0x32/0x160
> > > [ 612.165215] __queue_work+0x35e/0x7e0
> > > [ 612.165225] queue_delayed_work_on+0xa6/0xc0
> > > [ 612.165235] cifs_close+0x18d/0x270 [cifs]
> > > [ 612.165486] __fput+0x115/0x3d0
> > > [ 612.165498] task_work_run+0x85/0xc0
> > > [ 612.165510] exit_to_user_mode_prepare+0x1fd/0x200
> > > [ 612.165520] syscall_exit_to_user_mode+0x27/0x70
> > > [ 612.165531] do_syscall_64+0x47/0x70
> > > [ 612.165542] entry_SYSCALL_64_after_hwframe+0x44/0xae
> > >
> > > [ 612.165921] The buggy address belongs to the object at ffff88810a31c800
> > > which belongs to the cache kmalloc-1k of size 1024
> > > [ 612.167111] The buggy address is located 608 bytes inside of
> > > 1024-byte region [ffff88810a31c800, ffff88810a31cc00)
> > > [ 612.168215] The buggy address belongs to the page:
> > > [ 612.168794] page:00000000d0b7a3cf refcount:1 mapcount:0
> > > mapping:0000000000000000 index:0x0 pfn:0x10a318
> > > [ 612.168807] head:00000000d0b7a3cf order:3 compound_mapcount:0
> > > compound_pincount:0
> > > [ 612.168815] memcg:ffff88810d800901
> > > [ 612.168822] flags:
> > > 0x17ffe000010200(slab|head|node=0|zone=2|lastcpupid=0x3fff)
> > > [ 612.168835] raw: 0017ffe000010200 dead000000000100 dead000000000122
> > > ffff888100042dc0
> > > [ 612.168845] raw: 0000000000000000 0000000080100010 00000001ffffffff
> > > ffff88810d800901
> > > [ 612.168852] page dumped because: kasan: bad access detected
> > >
> > > [ 612.169163] Memory state around the buggy address:
> > > [ 612.169605] ffff88810a31c900: fb fb fb fb fb fb fb fb fb fb fb fb
> > > fb fb fb fb
> > > [ 612.170243] ffff88810a31c980: fb fb fb fb fb fb fb fb fb fb fb fb
> > > fb fb fb fb
> > > [ 612.170930] >ffff88810a31ca00: fb fb fb fb fb fb fb fb fb fb fb fb
> > > fb fb fb fb
> > > [ 612.171545] ^
> > > [ 612.172068] ffff88810a31ca80: fb fb fb fb fb fb fb fb fb fb fb fb
> > > fb fb fb fb
> > > [ 612.172832] ffff88810a31cb00: fb fb fb fb fb fb fb fb fb fb fb fb
> > > fb fb fb fb
> > > [ 612.173683] ==================================================================
> > > [ 612.174498] Disabling lock debugging due to kernel taint
> > >
> > >
> > > regards
> > > ronnie sahlberg
[-- Attachment #2: 0001-Fix-KASAN-identified-use-after-free-issue.patch --]
[-- Type: application/octet-stream, Size: 5360 bytes --]
From 560f8c51ce8edd47943c9aca4f18b5dbfd5b19ad Mon Sep 17 00:00:00 2001
From: Rohith Surabattula <rohiths@microsoft.com>
Date: Thu, 20 May 2021 14:36:57 +0000
Subject: [PATCH] Fix KASAN identified use-after-free issue.
[ 612.157429] ==================================================================
[ 612.158275] BUG: KASAN: use-after-free in process_one_work+0x90/0x9b0
[ 612.158801] Read of size 8 at addr ffff88810a31ca60 by task kworker/2:9/2382
[ 612.159611] CPU: 2 PID: 2382 Comm: kworker/2:9 Tainted: G
OE 5.13.0-rc2+ #98
[ 612.159623] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.14.0-1.fc33 04/01/2014
[ 612.159640] Workqueue: 0x0 (deferredclose)
[ 612.159669] Call Trace:
[ 612.159685] dump_stack+0xbb/0x107
[ 612.159711] print_address_description.constprop.0+0x18/0x140
[ 612.159733] ? process_one_work+0x90/0x9b0
[ 612.159743] ? process_one_work+0x90/0x9b0
[ 612.159754] kasan_report.cold+0x7c/0xd8
[ 612.159778] ? lock_is_held_type+0x80/0x130
[ 612.159789] ? process_one_work+0x90/0x9b0
[ 612.159812] kasan_check_range+0x145/0x1a0
[ 612.159834] process_one_work+0x90/0x9b0
[ 612.159877] ? pwq_dec_nr_in_flight+0x110/0x110
[ 612.159914] ? spin_bug+0x90/0x90
[ 612.159967] worker_thread+0x3b6/0x6c0
[ 612.160023] ? process_one_work+0x9b0/0x9b0
[ 612.160038] kthread+0x1dc/0x200
[ 612.160051] ? kthread_create_worker_on_cpu+0xd0/0xd0
[ 612.160092] ret_from_fork+0x1f/0x30
[ 612.160399] Allocated by task 2358:
[ 612.160757] kasan_save_stack+0x1b/0x40
[ 612.160768] __kasan_kmalloc+0x9b/0xd0
[ 612.160778] cifs_new_fileinfo+0xb0/0x960 [cifs]
[ 612.161170] cifs_open+0xadf/0xf20 [cifs]
[ 612.161421] do_dentry_open+0x2aa/0x6b0
[ 612.161432] path_openat+0xbd9/0xfa0
[ 612.161441] do_filp_open+0x11d/0x230
[ 612.161450] do_sys_openat2+0x115/0x240
[ 612.161460] __x64_sys_openat+0xce/0x140
When mod_delayed_work is called to modify the delay of pending work,
it might return false and queue a new work when pending work is
already scheduled or when try to grab pending work failed.
So, Increase the reference count when new work is scheduled to
avoid use-after-free.
Signed-off-by: Rohith Surabattula <rohiths@microsoft.com>
---
fs/cifs/file.c | 20 +++++++++++++-------
fs/cifs/misc.c | 12 ++++++++++--
2 files changed, 23 insertions(+), 9 deletions(-)
diff --git a/fs/cifs/file.c b/fs/cifs/file.c
index 4cfa22cfbc90..36ee01cbf3a0 100644
--- a/fs/cifs/file.c
+++ b/fs/cifs/file.c
@@ -874,10 +874,6 @@ void smb2_deferred_work_close(struct work_struct *work)
struct cifsFileInfo, deferred.work);
spin_lock(&CIFS_I(d_inode(cfile->dentry))->deferred_lock);
- if (!cfile->deferred_close_scheduled) {
- spin_unlock(&CIFS_I(d_inode(cfile->dentry))->deferred_lock);
- return;
- }
cifs_del_deferred_close(cfile);
cfile->deferred_close_scheduled = false;
spin_unlock(&CIFS_I(d_inode(cfile->dentry))->deferred_lock);
@@ -904,8 +900,13 @@ int cifs_close(struct inode *inode, struct file *file)
cifs_add_deferred_close(cfile, dclose);
if (cfile->deferred_close_scheduled &&
delayed_work_pending(&cfile->deferred)) {
- mod_delayed_work(deferredclose_wq,
- &cfile->deferred, cifs_sb->ctx->acregmax);
+ /*
+ * If there is no pending work, mod_delayed_work queues new work.
+ * So, Increase the ref count to avoid use-after-free.
+ */
+ if (!mod_delayed_work(deferredclose_wq,
+ &cfile->deferred, cifs_sb->ctx->acregmax))
+ cifsFileInfo_get(cfile);
} else {
/* Deferred close for files */
queue_delayed_work(deferredclose_wq,
@@ -4879,7 +4880,12 @@ void cifs_oplock_break(struct work_struct *work)
if (is_deferred &&
cfile->deferred_close_scheduled &&
delayed_work_pending(&cfile->deferred)) {
- mod_delayed_work(deferredclose_wq, &cfile->deferred, 0);
+ /*
+ * If there is no pending work, mod_delayed_work queues new work.
+ * So, Increase the ref count to avoid use-after-free.
+ */
+ if (!mod_delayed_work(deferredclose_wq, &cfile->deferred, 0))
+ cifsFileInfo_get(cfile);
}
spin_unlock(&CIFS_I(inode)->deferred_lock);
_cifsFileInfo_put(cfile, false /* do not wait for ourself */, false);
diff --git a/fs/cifs/misc.c b/fs/cifs/misc.c
index cd705f8a4e31..7cd6c2164c68 100644
--- a/fs/cifs/misc.c
+++ b/fs/cifs/misc.c
@@ -674,6 +674,8 @@ cifs_add_pending_open(struct cifs_fid *fid, struct tcon_link *tlink,
/*
* Critical section which runs after acquiring deferred_lock.
+ * As there is no reference count on cifs_deferred_close, pdclose
+ * should not be used outside deferred_lock.
*/
bool
cifs_is_deferred_close(struct cifsFileInfo *cfile, struct cifs_deferred_close **pdclose)
@@ -754,8 +756,14 @@ cifs_close_all_deferred_files(struct cifs_tcon *tcon)
list_for_each(tmp, &tcon->openFileList) {
cfile = list_entry(tmp, struct cifsFileInfo, tlist);
cinode = CIFS_I(d_inode(cfile->dentry));
- if (delayed_work_pending(&cfile->deferred))
- mod_delayed_work(deferredclose_wq, &cfile->deferred, 0);
+ if (delayed_work_pending(&cfile->deferred)) {
+ /*
+ * If there is no pending work, mod_delayed_work queues new work.
+ * So, Increase the ref count to avoid use-after-free.
+ */
+ if (!mod_delayed_work(deferredclose_wq, &cfile->deferred, 0))
+ cifsFileInfo_get(cfile);
+ }
}
spin_unlock(&tcon->open_file_lock);
}
--
2.30.2
^ permalink raw reply related [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-05-20 15:19 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-18 22:42 KASAN use after free in deferred close ronnie sahlberg
2021-05-19 5:14 ` Rohith Surabattula
2021-05-19 5:38 ` ronnie sahlberg
2021-05-20 15:19 ` Rohith Surabattula
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).