All of lore.kernel.org
 help / color / mirror / Atom feed
* slab-out-of-bounds in iov_iter_revert()
@ 2020-09-11 21:59 Qian Cai
  2020-09-11 23:55 ` Al Viro
  0 siblings, 1 reply; 12+ messages in thread
From: Qian Cai @ 2020-09-11 21:59 UTC (permalink / raw)
  To: viro; +Cc: torvalds, vgoyal, miklos, linux-fsdevel, linux-kernel

Super easy to reproduce on today's mainline by just fuzzing for a few minutes
on virtiofs (if it ever matters). Any thoughts?

[  511.089112] BUG: KASAN: slab-out-of-bounds in iov_iter_revert+0xd8/0x3c0
iov_iter_revert at lib/iov_iter.c:1135
(inlined by) iov_iter_revert at lib/iov_iter.c:1080
[  511.092650] Read of size 8 at addr ffff88869e11dff8 by task trinity-c1/11868
[  511.096178] 
[  511.096897] CPU: 20 PID: 11868 Comm: trinity-c1 Not tainted 5.9.0-rc4+ #1
[  511.100257] Hardware name: Red Hat KVM, BIOS 1.14.0-1.module+el8.3.0+7638+07cf13d2 04/01/2014
[  511.103999] Call Trace:
[  511.105002]  dump_stack+0x7c/0xb0
[  511.106329]  ? iov_iter_revert+0xd8/0x3c0
[  511.107915]  print_address_description.constprop.7+0x1e/0x230
[  511.110193]  ? kmsg_dump_rewind_nolock+0x59/0x59
[  511.112038]  ? _raw_write_lock_irqsave+0xe0/0xe0
[  511.113890]  ? iov_iter_revert+0xd8/0x3c0
[  511.115469]  ? iov_iter_revert+0xd8/0x3c0
[  511.117082]  kasan_report.cold.9+0x37/0x86
[  511.118711]  ? do_readv+0x20/0x1b0
[  511.120078]  ? iov_iter_revert+0xd8/0x3c0
[  511.122614]  iov_iter_revert+0xd8/0x3c0
[  511.124673]  generic_file_read_iter+0x139/0x220
[  511.127386]  fuse_file_read_iter+0x239/0x270 [fuse]
[  511.130229]  ? fuse_direct_IO+0x600/0x600 [fuse]
[  511.133491]  ? rwsem_optimistic_spin+0x3d0/0x3d0
[  511.137177]  ? wake_up_q+0x92/0xd0
[  511.139702]  ? kasan_unpoison_shadow+0x30/0x40
[  511.142518]  do_iter_readv_writev+0x307/0x350
[  511.144850]  ? no_seek_end_llseek_size+0x20/0x20
[  511.147155]  do_iter_read+0x13f/0x2e0
[  511.148696]  vfs_readv+0xcc/0x130
[  511.150118]  ? compat_rw_copy_check_uvector+0x1e0/0x1e0
[  511.152300]  ? enqueue_hrtimer+0x60/0x100
[  511.154043]  ? hrtimer_start_range_ns+0x32f/0x4c0
[  511.157561]  ? hrtimer_run_softirq+0x100/0x100
[  511.161514]  ? _raw_spin_lock_irq+0x7b/0xd0
[  511.164570]  ? _raw_write_unlock_irqrestore+0x20/0x20
[  511.167568]  ? hrtimer_active+0x71/0xa0
[  511.169331]  ? mutex_lock+0x8e/0xe0
[  511.171694]  ? __mutex_lock_slowpath+0x10/0x10
[  511.174580]  ? perf_call_bpf_enter.isra.21+0x110/0x110
[  511.177926]  ? __fget_light+0xa3/0x100
[  511.179916]  do_readv+0xc1/0x1b0
[  511.181331]  ? vfs_readv+0x130/0x130
[  511.182867]  ? ktime_get_coarse_real_ts64+0x4a/0x70
[  511.185455]  do_syscall_64+0x33/0x40
[  511.188008]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  511.191314] RIP: 0033:0x7f11e9b4578d
[  511.193639] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d cb 56 2c 00 f7 d8 64 89 08
[  511.202148] RSP: 002b:00007fff9b5eec58 EFLAGS: 00000246 ORIG_RAX: 0000000000000013
[  511.205620] RAX: ffffffffffffffda RBX: 0000000000000013 RCX: 00007f11e9b4578d
[  511.210533] RDX: 0000000000000091 RSI: 0000000002c49450 RDI: 00000000000000e1
[  511.214992] RBP: 0000000000000013 R08: 000000008d8d8d8d R09: 00000000000002d2
[  511.218631] R10: 00000020845754a0 R11: 0000000000000246 R12: 0000000000000002
[  511.221595] R13: 00007f11ea227058 R14: 00007f11ea2356c0 R15: 00007f11ea227000
[  511.225949] 
[  511.227008] Allocated by task 11748:
[  511.229204]  kasan_save_stack+0x19/0x40
[  511.231404]  __kasan_kmalloc.constprop.8+0xc1/0xd0
[  511.234647]  perf_event_mmap+0x28f/0x5f0
[  511.237170]  mmap_region+0x1cc/0xa50
[  511.239192]  do_mmap+0x3e5/0x6a0
[  511.241337]  vm_mmap_pgoff+0x15f/0x1b0
[  511.243586]  ksys_mmap_pgoff+0x2d3/0x320
[  511.245903]  do_syscall_64+0x33/0x40
[  511.247914]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  511.250139] 
[  511.250797] Freed by task 11748:
[  511.252160]  kasan_save_stack+0x19/0x40
[  511.253775]  kasan_set_track+0x1c/0x30
[  511.255348]  kasan_set_free_info+0x1b/0x30
[  511.257072]  __kasan_slab_free+0x108/0x150
[  511.258785]  kfree+0x95/0x380
[  511.260050]  perf_event_mmap+0x4aa/0x5f0
[  511.261694]  mmap_region+0x1cc/0xa50
[  511.263198]  do_mmap+0x3e5/0x6a0
[  511.264564]  vm_mmap_pgoff+0x15f/0x1b0
[  511.266133]  ksys_mmap_pgoff+0x2d3/0x320
[  511.267773]  do_syscall_64+0x33/0x40
[  511.269276]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  511.272756] 
[  511.273583] The buggy address belongs to the object at ffff88869e11c000
[  511.273583]  which belongs to the cache kmalloc-4k of size 4096
[  511.281456] The buggy address is located 4088 bytes to the right of
[  511.281456]  4096-byte region [ffff88869e11c000, ffff88869e11d000)
[  511.288473] The buggy address belongs to the page:
[  511.291093] page:0000000073d20fbc refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x69e118
[  511.296681] head:0000000073d20fbc order:3 compound_mapcount:0 compound_pincount:0
[  511.301118] flags: 0x17ffffc0010200(slab|head)
[  511.303426] raw: 0017ffffc0010200 0000000000000000 0000000300000001 ffff888107c4ef80
[  511.307482] raw: 0000000000000000 0000000000040004 00000001ffffffff 0000000000000000
[  511.310957] page dumped because: kasan: bad access detected
[  511.313233] 
[  511.313867] Memory state around the buggy address:
[  511.315849]  ffff88869e11de80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[  511.318933]  ffff88869e11df00: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[  511.322715] >ffff88869e11df80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[  511.325993]                                                                 ^
[  511.330020]  ffff88869e11e000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[  511.334333]  ffff88869e11e080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: slab-out-of-bounds in iov_iter_revert()
  2020-09-11 21:59 slab-out-of-bounds in iov_iter_revert() Qian Cai
@ 2020-09-11 23:55 ` Al Viro
  2020-09-16 21:09   ` Qian Cai
  0 siblings, 1 reply; 12+ messages in thread
From: Al Viro @ 2020-09-11 23:55 UTC (permalink / raw)
  To: Qian Cai; +Cc: torvalds, vgoyal, miklos, linux-fsdevel, linux-kernel

On Fri, Sep 11, 2020 at 05:59:04PM -0400, Qian Cai wrote:
> Super easy to reproduce on today's mainline by just fuzzing for a few minutes
> on virtiofs (if it ever matters). Any thoughts?

Usually happens when ->direct_IO() fucks up and reports the wrong amount
of data written/read.  We had several bugs like that in the past - see
e.g. 85128b2be673 (fix nfs O_DIRECT advancing iov_iter too much).

Had there been any recent O_DIRECT-related patches on the filesystems
involved?

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: slab-out-of-bounds in iov_iter_revert()
  2020-09-11 23:55 ` Al Viro
@ 2020-09-16 21:09   ` Qian Cai
  2020-09-17  2:04     ` Al Viro
  0 siblings, 1 reply; 12+ messages in thread
From: Qian Cai @ 2020-09-16 21:09 UTC (permalink / raw)
  To: Al Viro; +Cc: torvalds, vgoyal, miklos, linux-fsdevel, linux-kernel

On Sat, 2020-09-12 at 00:55 +0100, Al Viro wrote:
> On Fri, Sep 11, 2020 at 05:59:04PM -0400, Qian Cai wrote:
> > Super easy to reproduce on today's mainline by just fuzzing for a few
> > minutes
> > on virtiofs (if it ever matters). Any thoughts?
> 
> Usually happens when ->direct_IO() fucks up and reports the wrong amount
> of data written/read.  We had several bugs like that in the past - see
> e.g. 85128b2be673 (fix nfs O_DIRECT advancing iov_iter too much).
> 
> Had there been any recent O_DIRECT-related patches on the filesystems
> involved?

This is only reproducible using FUSE/virtiofs so far, so I will stare at
fuse_direct_IO() until someone can beat me to it.


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: slab-out-of-bounds in iov_iter_revert()
  2020-09-16 21:09   ` Qian Cai
@ 2020-09-17  2:04     ` Al Viro
  2020-09-17  2:14       ` Al Viro
  0 siblings, 1 reply; 12+ messages in thread
From: Al Viro @ 2020-09-17  2:04 UTC (permalink / raw)
  To: Qian Cai; +Cc: torvalds, vgoyal, miklos, linux-fsdevel, linux-kernel

On Wed, Sep 16, 2020 at 05:09:49PM -0400, Qian Cai wrote:
> On Sat, 2020-09-12 at 00:55 +0100, Al Viro wrote:
> > On Fri, Sep 11, 2020 at 05:59:04PM -0400, Qian Cai wrote:
> > > Super easy to reproduce on today's mainline by just fuzzing for a few
> > > minutes
> > > on virtiofs (if it ever matters). Any thoughts?
> > 
> > Usually happens when ->direct_IO() fucks up and reports the wrong amount
> > of data written/read.  We had several bugs like that in the past - see
> > e.g. 85128b2be673 (fix nfs O_DIRECT advancing iov_iter too much).
> > 
> > Had there been any recent O_DIRECT-related patches on the filesystems
> > involved?
> 
> This is only reproducible using FUSE/virtiofs so far, so I will stare at
> fuse_direct_IO() until someone can beat me to it.

What happens there is that it tries to play with iov_iter_truncate() in
->direct_IO() without a corresponding iov_iter_reexpand().  Could you
check if the following helps?

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
---
diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index 6611ef3269a8..d3eec2e11975 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -3095,7 +3095,7 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 	loff_t pos = 0;
 	struct inode *inode;
 	loff_t i_size;
-	size_t count = iov_iter_count(iter);
+	size_t count = iov_iter_count(iter), shortened;
 	loff_t offset = iocb->ki_pos;
 	struct fuse_io_priv *io;
 
@@ -3111,7 +3111,8 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 		if (offset >= i_size)
 			return 0;
 		iov_iter_truncate(iter, fuse_round_up(ff->fc, i_size - offset));
-		count = iov_iter_count(iter);
+		shortened = count - iov_iter_count(iter);
+		count -= shortened;
 	}
 
 	io = kmalloc(sizeof(struct fuse_io_priv), GFP_KERNEL);
@@ -3177,6 +3178,8 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 		else if (ret < 0 && offset + count > i_size)
 			fuse_do_truncate(file);
 	}
+	if (shortened)
+		iov_iter_reexpand(iter, shortened);
 
 	return ret;
 }

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: slab-out-of-bounds in iov_iter_revert()
  2020-09-17  2:04     ` Al Viro
@ 2020-09-17  2:14       ` Al Viro
  2020-09-17 14:10         ` Qian Cai
  0 siblings, 1 reply; 12+ messages in thread
From: Al Viro @ 2020-09-17  2:14 UTC (permalink / raw)
  To: Qian Cai; +Cc: torvalds, vgoyal, miklos, linux-fsdevel, linux-kernel

On Thu, Sep 17, 2020 at 03:04:40AM +0100, Al Viro wrote:
> On Wed, Sep 16, 2020 at 05:09:49PM -0400, Qian Cai wrote:
> > On Sat, 2020-09-12 at 00:55 +0100, Al Viro wrote:
> > > On Fri, Sep 11, 2020 at 05:59:04PM -0400, Qian Cai wrote:
> > > > Super easy to reproduce on today's mainline by just fuzzing for a few
> > > > minutes
> > > > on virtiofs (if it ever matters). Any thoughts?
> > > 
> > > Usually happens when ->direct_IO() fucks up and reports the wrong amount
> > > of data written/read.  We had several bugs like that in the past - see
> > > e.g. 85128b2be673 (fix nfs O_DIRECT advancing iov_iter too much).
> > > 
> > > Had there been any recent O_DIRECT-related patches on the filesystems
> > > involved?
> > 
> > This is only reproducible using FUSE/virtiofs so far, so I will stare at
> > fuse_direct_IO() until someone can beat me to it.
> 
> What happens there is that it tries to play with iov_iter_truncate() in
> ->direct_IO() without a corresponding iov_iter_reexpand().  Could you
> check if the following helps?

Gyah...  Sorry, that should be

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
---
diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index 6611ef3269a8..92de6b9b06b0 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -3095,7 +3095,7 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 	loff_t pos = 0;
 	struct inode *inode;
 	loff_t i_size;
-	size_t count = iov_iter_count(iter);
+	size_t count = iov_iter_count(iter), shortened;
 	loff_t offset = iocb->ki_pos;
 	struct fuse_io_priv *io;
 
@@ -3111,7 +3111,8 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 		if (offset >= i_size)
 			return 0;
 		iov_iter_truncate(iter, fuse_round_up(ff->fc, i_size - offset));
-		count = iov_iter_count(iter);
+		shortened = count - iov_iter_count(iter);
+		count -= shortened;
 	}
 
 	io = kmalloc(sizeof(struct fuse_io_priv), GFP_KERNEL);
@@ -3177,6 +3178,7 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 		else if (ret < 0 && offset + count > i_size)
 			fuse_do_truncate(file);
 	}
+	iov_iter_reexpand(iter, iov_iter_count(iter) + shortened);
 
 	return ret;
 }

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: slab-out-of-bounds in iov_iter_revert()
  2020-09-17  2:14       ` Al Viro
@ 2020-09-17 14:10         ` Qian Cai
  2020-09-17 16:44           ` Al Viro
  0 siblings, 1 reply; 12+ messages in thread
From: Qian Cai @ 2020-09-17 14:10 UTC (permalink / raw)
  To: Al Viro; +Cc: torvalds, vgoyal, miklos, linux-fsdevel, linux-kernel

On Thu, 2020-09-17 at 03:14 +0100, Al Viro wrote:
> On Thu, Sep 17, 2020 at 03:04:40AM +0100, Al Viro wrote:
> > On Wed, Sep 16, 2020 at 05:09:49PM -0400, Qian Cai wrote:
> > > On Sat, 2020-09-12 at 00:55 +0100, Al Viro wrote:
> > > > On Fri, Sep 11, 2020 at 05:59:04PM -0400, Qian Cai wrote:
> > > > > Super easy to reproduce on today's mainline by just fuzzing for a few
> > > > > minutes
> > > > > on virtiofs (if it ever matters). Any thoughts?
> > > > 
> > > > Usually happens when ->direct_IO() fucks up and reports the wrong amount
> > > > of data written/read.  We had several bugs like that in the past - see
> > > > e.g. 85128b2be673 (fix nfs O_DIRECT advancing iov_iter too much).
> > > > 
> > > > Had there been any recent O_DIRECT-related patches on the filesystems
> > > > involved?
> > > 
> > > This is only reproducible using FUSE/virtiofs so far, so I will stare at
> > > fuse_direct_IO() until someone can beat me to it.
> > 
> > What happens there is that it tries to play with iov_iter_truncate() in
> > ->direct_IO() without a corresponding iov_iter_reexpand().  Could you
> > check if the following helps?
> 
> Gyah...  Sorry, that should be
> 
> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

This is now triggering:

[   81.942804] WARNING: CPU: 24 PID: 1545 at lib/iov_iter.c:1084 iov_iter_revert+0x245/0x8c0
[   81.942805] Modules linked in: af_key mpls_router ip_tunnel hidp cmtp kernelcapi bnep rfcomm bluetooth ecdh_generic ecc can_bcm can_raw can pptp gre l2tp_ppp l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel pppoe pppox ppp_generic slhc crypto_user ib_core nfnetlink scsi_transport_iscsi atm sctp rfkill nls_utf8 isofs intel_rapl_msr intel_rapl_common sb_edac kvm_intel kvm bochs_drm drm_vram_helper irqbypass drm_ttm_helper ttm rapl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm i2c_piix4 joydev pcspkr vfat fat ip_tables xfs libcrc32c sr_mod sd_mod cdrom t10_pi sg ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ata_piix virtiofs libata ghash_clmulni_intel fuse e1000 serio_raw sunrpc dm_mirror dm_region_hash dm_log dm_mod
[   81.942870] CPU: 24 PID: 1545 Comm: trinity-c0 Not tainted 5.9.0-rc5-iter+ #2
[   81.942871] Hardware name: Red Hat KVM, BIOS 1.14.0-1.module+el8.3.0+7638+07cf13d2 04/01/2014
[   81.942879] RIP: 0010:iov_iter_revert+0x245/0x8c0
[   81.942882] Code: 89 fa 48 c1 ea 03 80 3c 02 00 0f 85 ce 05 00 00 49 29 f5 48 89 6b 18 4c 89 6b 08 48 83 c4 30 5b 5d 41 5c 41 5d 41 5e 41 5f c3 <0f> 0b e9 3e ff ff ff 48 b8 00 00 00 00 00 fc ff df 48 8d 7b 18 48
[   81.942884] RSP: 0018:ffff8886bf637b60 EFLAGS: 00010286
[   81.942886] RAX: 000000000000028b RBX: ffff8886bf637dc8 RCX: 1ffff110d7ec6faf
[   81.942888] RDX: dffffc0000000000 RSI: ffffffffbe4a754d RDI: ffff8886bf637d68
[   81.942889] RBP: ffff8886bf637d68 R08: 0000000000000004 R09: ffffed10d7ec6ee2
[   81.942890] R10: 0000000000000003 R11: ffffed10d7ec6ee2 R12: 0000000000000000
[   81.942891] R13: ffff8886eaa62d80 R14: ffff8886bf637dd0 R15: ffff8886bf637d78
[   81.942893] FS:  00007fc78f78d740(0000) GS:ffff8887d2000000(0000) knlGS:0000000000000000
[   81.942897] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   81.942898] CR2: 0000000000000008 CR3: 00000007cadba005 CR4: 0000000000170ee0
[   81.942899] Call Trace:
[   81.942909]  generic_file_read_iter+0x23b/0x4b0
[   81.942918]  fuse_file_read_iter+0x280/0x4e0 [fuse]
[   81.942931]  ? fuse_direct_IO+0xd30/0xd30 [fuse]
[   81.942949]  ? _raw_spin_lock_irqsave+0x80/0xe0
[   81.942957]  ? timerqueue_add+0x15e/0x280
[   81.942960]  ? _raw_spin_lock_irqsave+0x80/0xe0
[   81.942966]  new_sync_read+0x3b7/0x620
[   81.942968]  ? __ia32_sys_llseek+0x2e0/0x2e0
[   81.942971] ==================================================================
[   81.942984] BUG: KASAN: slab-out-of-bounds in iov_iter_revert+0x693/0x8c0
[   81.942995]  ? hrtimer_start_range_ns+0x495/0x900
[   81.942997]  ? hrtimer_try_to_cancel+0x6d/0x330
[   81.943002] Read of size 8 at addr ffff88867e6a6ff8 by task trinity-c21/1600
[   81.943005]  ? hrtimer_forward+0x1b0/0x1b0

[   81.943026]  ? __fsnotify_update_child_dentry_flags.part.12+0x290/0x290
[   81.943030]  ? _cond_resched+0x15/0x30
[   81.943041]  ? __inode_security_revalidate+0x9d/0xc0
[   81.943047] CPU: 12 PID: 1600 Comm: trinity-c21 Not tainted 5.9.0-rc5-iter+ #2
[   81.943050] Hardware name: Red Hat KVM, BIOS 1.14.0-1.module+el8.3.0+7638+07cf13d2 04/01/2014
[   81.943053]  vfs_read+0x22b/0x460
[   81.943056]  ksys_read+0xe5/0x1b0
[   81.943058] Call Trace:
[   81.943061]  ? vfs_write+0x5e0/0x5e0
[   81.943069]  dump_stack+0x7c/0xb0
[   81.943074]  do_syscall_64+0x33/0x40
[   81.943077]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   81.943081] RIP: 0033:0x7fc78f09d78d
[   81.943084]  ? iov_iter_revert+0x693/0x8c0
[   81.943097]  print_address_description.constprop.7+0x1e/0x230
[   81.943100] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d cb 56 2c 00 f7 d8 64 89 01 48
[   81.943117]  ? kmsg_dump_rewind_nolock+0xd9/0xd9
[   81.943119] RSP: 002b:00007ffe01a9e168 EFLAGS: 00000246
[   81.943128]  ? _raw_write_lock_irqsave+0xe0/0xe0
[   81.943129]  ORIG_RAX: 0000000000000000
[   81.943133] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fc78f09d78d
[   81.943136] RDX: 000000000000028b RSI: 00007fc78d282000 RDI: 00000000000000f2
[   81.943140]  ? iov_iter_revert+0x693/0x8c0
[   81.943142] RBP: 0000000000000000 R08: 19d504e75f9b0c9b R09: 00000000000000ff
[   81.943146]  ? iov_iter_revert+0x693/0x8c0
[   81.943148] R10: 00000000d9d9d9d9 R11: 0000000000000246 R12: 0000000000000002
[   81.943153]  kasan_report.cold.9+0x37/0x7c
[   81.943155] R13: 00007fc78f786058 R14: 00007fc78f78d6c0 R15: 00007fc78f786000
[   81.943158]  ? iov_iter_revert+0x693/0x8c0
[   81.943161]  iov_iter_revert+0x693/0x8c0
[   81.943163] ---[ end trace ee32e92b96589675 ]---
[   81.943172]  ? dentry_needs_remove_privs.part.30+0x40/0x40
[   81.943181]  ? generic_write_checks+0x1d2/0x3d0
[   81.943185]  generic_file_direct_write+0x2ed/0x430
[   81.943195]  fuse_file_write_iter+0x5d8/0xb10 [fuse]
[   81.943209]  ? fuse_perform_write+0xed0/0xed0 [fuse]
[   81.943215]  ? kasan_unpoison_shadow+0x30/0x40
[   81.943221]  do_iter_readv_writev+0x3a6/0x700
[   81.943224]  ? no_seek_end_llseek_size+0x20/0x20
[   81.943228]  do_iter_write+0x12f/0x5f0
[   81.943233]  ? timerqueue_add+0x15e/0x280
[   81.943236]  vfs_writev+0x172/0x2d0
[   81.943240]  ? vfs_iter_write+0xc0/0xc0
[   81.943245]  ? hrtimer_start_range_ns+0x495/0x900
[   81.943248]  ? hrtimer_run_softirq+0x210/0x210
[   81.943252]  ? _raw_spin_lock_irq+0x7b/0xd0
[   81.943256]  ? _raw_write_unlock_irqrestore+0x50/0x50
[   81.943269]  ? perf_syscall_enter+0x136/0x8a0
[   81.943276]  ? perf_call_bpf_enter.isra.21+0x1a0/0x1a0
[   81.943283]  ? alarm_setitimer+0xa0/0x110
[   81.943287]  do_pwritev+0x151/0x200
[   81.943291]  ? __ia32_sys_writev+0xb0/0xb0
[   81.943296]  do_syscall_64+0x33/0x40
[   81.943301]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   81.943306] RIP: 0033:0x7fc78f09d78d
[   81.943312] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d cb 56 2c 00 f7 d8 64 89 01 48
[   81.943314] RSP: 002b:00007ffe01a9e168 EFLAGS: 00000246 ORIG_RAX: 0000000000000148
[   81.943318] RAX: ffffffffffffffda RBX: 0000000000000148 RCX: 00007fc78f09d78d
[   81.943320] RDX: 000000000000004b RSI: 0000000001ae7790 RDI: 00000000000000f2
[   81.943323] RBP: 0000000000000148 R08: 0000000075f74000 R09: 0000000000000001
[   81.943325] R10: 00830624a1148006 R11: 0000000000000246 R12: 0000000000000002
[   81.943328] R13: 00007fc78f6f3058 R14: 00007fc78f78d6c0 R15: 00007fc78f6f3000

[   81.943333] Allocated by task 0:
[   81.943334] (stack is not available)

[   81.943339] The buggy address belongs to the object at ffff88867e6a6000
                which belongs to the cache kmalloc-2k of size 2048
[   81.943342] The buggy address is located 2040 bytes to the right of
                2048-byte region [ffff88867e6a6000, ffff88867e6a6800)
[   81.943344] The buggy address belongs to the page:
[   81.943350] page:000000007cdca305 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x67e6a0
[   81.943354] head:000000007cdca305 order:3 compound_mapcount:0 compound_pincount:0
[   81.943358] flags: 0x17ffffc0010200(slab|head)
[   81.943365] raw: 0017ffffc0010200 dead000000000100 dead000000000122 ffff888107c4f0c0
[   81.943370] raw: 0000000000000000 0000000080080008 00000001ffffffff 0000000000000000
[   81.943372] page dumped because: kasan: bad access detected

[   81.943374] Memory state around the buggy address:
[   81.943379]  ffff88867e6a6e80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[   81.943382]  ffff88867e6a6f00: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[   81.943384] >ffff88867e6a6f80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[   81.943386]                                                                 ^
[   81.943388]  ffff88867e6a7000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[   81.943400]  ffff88867e6a7080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[   81.943402] ==================================================================
[   81.943403] Disabling lock debugging due to kernel taint

> ---
> diff --git a/fs/fuse/file.c b/fs/fuse/file.c
> index 6611ef3269a8..92de6b9b06b0 100644
> --- a/fs/fuse/file.c
> +++ b/fs/fuse/file.c
> @@ -3095,7 +3095,7 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter
> *iter)
>  	loff_t pos = 0;
>  	struct inode *inode;
>  	loff_t i_size;
> -	size_t count = iov_iter_count(iter);
> +	size_t count = iov_iter_count(iter), shortened;
>  	loff_t offset = iocb->ki_pos;
>  	struct fuse_io_priv *io;
>  
> @@ -3111,7 +3111,8 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter
> *iter)
>  		if (offset >= i_size)
>  			return 0;
>  		iov_iter_truncate(iter, fuse_round_up(ff->fc, i_size - offset));
> -		count = iov_iter_count(iter);
> +		shortened = count - iov_iter_count(iter);
> +		count -= shortened;
>  	}
>  
>  	io = kmalloc(sizeof(struct fuse_io_priv), GFP_KERNEL);
> @@ -3177,6 +3178,7 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter
> *iter)
>  		else if (ret < 0 && offset + count > i_size)
>  			fuse_do_truncate(file);
>  	}
> +	iov_iter_reexpand(iter, iov_iter_count(iter) + shortened);
>  
>  	return ret;
>  }
> 


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: slab-out-of-bounds in iov_iter_revert()
  2020-09-17 14:10         ` Qian Cai
@ 2020-09-17 16:44           ` Al Viro
  2020-09-17 17:42             ` Qian Cai
  2020-09-17 18:45             ` Qian Cai
  0 siblings, 2 replies; 12+ messages in thread
From: Al Viro @ 2020-09-17 16:44 UTC (permalink / raw)
  To: Qian Cai; +Cc: torvalds, vgoyal, miklos, linux-fsdevel, linux-kernel

On Thu, Sep 17, 2020 at 10:10:27AM -0400, Qian Cai wrote:

> [   81.942909]  generic_file_read_iter+0x23b/0x4b0
> [   81.942918]  fuse_file_read_iter+0x280/0x4e0 [fuse]
> [   81.942931]  ? fuse_direct_IO+0xd30/0xd30 [fuse]
> [   81.942949]  ? _raw_spin_lock_irqsave+0x80/0xe0
> [   81.942957]  ? timerqueue_add+0x15e/0x280
> [   81.942960]  ? _raw_spin_lock_irqsave+0x80/0xe0
> [   81.942966]  new_sync_read+0x3b7/0x620
> [   81.942968]  ? __ia32_sys_llseek+0x2e0/0x2e0

Interesting...  Basic logics in there:
	->direct_IO() might consume more (on iov_iter_get_pages()
and friends) than it actually reads.  We want to revert the
excess.  Suppose by the time we call ->direct_IO() we had
N bytes already consumed and C bytes left.  We expect that
after ->direct_IO() returns K, we have C' bytes left, N + (C - C')
consumed and N + K out of those actually read.  So we revert by
C - K - C'.  You end up trying to revert beyond the beginning.

	Use of iov_iter_truncate() is problematic here, since it
changes the amount of data left without having consumed anything.
Basically, it changes the position of end, and the logics in the
caller expects that to remain unchanged.  iov_iter_reexpand() use
should restore the position of end.

	How much IO does it take to trigger that on your reproducer?

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: slab-out-of-bounds in iov_iter_revert()
  2020-09-17 16:44           ` Al Viro
@ 2020-09-17 17:42             ` Qian Cai
  2020-09-17 18:45               ` Al Viro
  2020-09-17 18:45             ` Qian Cai
  1 sibling, 1 reply; 12+ messages in thread
From: Qian Cai @ 2020-09-17 17:42 UTC (permalink / raw)
  To: Al Viro; +Cc: torvalds, vgoyal, miklos, linux-fsdevel, linux-kernel

On Thu, 2020-09-17 at 17:44 +0100, Al Viro wrote:
> On Thu, Sep 17, 2020 at 10:10:27AM -0400, Qian Cai wrote:
> 
> > [   81.942909]  generic_file_read_iter+0x23b/0x4b0
> > [   81.942918]  fuse_file_read_iter+0x280/0x4e0 [fuse]
> > [   81.942931]  ? fuse_direct_IO+0xd30/0xd30 [fuse]
> > [   81.942949]  ? _raw_spin_lock_irqsave+0x80/0xe0
> > [   81.942957]  ? timerqueue_add+0x15e/0x280
> > [   81.942960]  ? _raw_spin_lock_irqsave+0x80/0xe0
> > [   81.942966]  new_sync_read+0x3b7/0x620
> > [   81.942968]  ? __ia32_sys_llseek+0x2e0/0x2e0
> 
> Interesting...  Basic logics in there:
> 	->direct_IO() might consume more (on iov_iter_get_pages()
> and friends) than it actually reads.  We want to revert the
> excess.  Suppose by the time we call ->direct_IO() we had
> N bytes already consumed and C bytes left.  We expect that
> after ->direct_IO() returns K, we have C' bytes left, N + (C - C')
> consumed and N + K out of those actually read.  So we revert by
> C - K - C'.  You end up trying to revert beyond the beginning.
> 
> 	Use of iov_iter_truncate() is problematic here, since it
> changes the amount of data left without having consumed anything.
> Basically, it changes the position of end, and the logics in the
> caller expects that to remain unchanged.  iov_iter_reexpand() use
> should restore the position of end.
> 
> 	How much IO does it take to trigger that on your reproducer?

That is something I don't know for sure because it is always reproducible by
running the trinity fuzzer for a few seconds (32 threads). I did another run
below (still with your patch applied) and then tried to capture some logs here:

http://people.redhat.com/qcai/iov_iter_revert/

- virtiofsd.txt: fuse server side log until it triggered.
- trinity-post-mortem.log: AFAICT, it was the final syscall of each child.
- trinity-child9.log: the child actually triggered it.

The last syscall of child9 is:
pwritev2(fd=230, vec=0x293d3f0, vlen=1, pos_l=120, pos_h=0x200000, flags=0x3)

[   77.125816] BUG: KASAN: stack-out-of-bounds in iov_iter_revert+0x693/0x8c0
[   77.127079] Read of size 8 at addr ffff8886efd5fda8 by task trinity-c9/1593
[   77.128292] 
[   77.128581] CPU: 10 PID: 1593 Comm: trinity-c9 Not tainted 5.9.0-rc5-iter+ #2
[   77.129798] Hardware name: Red Hat KVM, BIOS 1.14.0-1.module+el8.3.0+7638+07cf13d2 04/01/2014
[   77.131175] Call Trace:
[   77.131934]  dump_stack+0x7c/0xb0
[   77.132500]  ? iov_iter_revert+0x693/0x8c0
[   77.133254]  print_address_description.constprop.7+0x1e/0x230
[   77.134287]  ? kmsg_dump_rewind_nolock+0xd9/0xd9
[   77.135103]  ? _raw_write_lock_irqsave+0xe0/0xe0
[   77.135933]  ? iov_iter_revert+0x693/0x8c0
[   77.136651]  ? iov_iter_revert+0x693/0x8c0
[   77.137345]  kasan_report.cold.9+0x37/0x7c
[   77.138033]  ? iov_iter_revert+0x693/0x8c0
[   77.138768]  iov_iter_revert+0x693/0x8c0
[   77.139433]  ? dentry_needs_remove_privs.part.30+0x40/0x40
[   77.140684]  ? generic_write_checks+0x1d2/0x3d0
[   77.141477]  generic_file_direct_write+0x2ed/0x430
[   77.142601]  fuse_file_write_iter+0x5d8/0xb10 [fuse]
[   77.143857]  ? fuse_perform_write+0xed0/0xed0 [fuse]
[   77.145062]  do_iter_readv_writev+0x3a6/0x700
[   77.146071]  ? no_seek_end_llseek_size+0x20/0x20
[   77.146568] kexec-bzImage64: File is too short to be a bzImage
[   77.147277]  do_iter_write+0x12f/0x5f0
[   77.149501]  ? timerqueue_add+0x15e/0x280
[   77.150187]  vfs_writev+0x172/0x2d0
[   77.150837]  ? vfs_iter_write+0xc0/0xc0
[   77.151493]  ? hrtimer_start_range_ns+0x495/0x900
[   77.152316]  ? hrtimer_run_softirq+0x210/0x210
[   77.153087]  ? _raw_spin_lock_irq+0x7b/0xd0
[   77.153842]  ? _raw_write_unlock_irqrestore+0x50/0x50
[   77.154733]  ? do_setitimer+0x2e5/0x590
[   77.155393]  ? alarm_setitimer+0xa0/0x110
[   77.156059]  do_pwritev+0x151/0x200
[   77.156657]  ? __ia32_sys_writev+0xb0/0xb0
[   77.157336]  do_syscall_64+0x33/0x40
[   77.157928]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   77.158762] RIP: 0033:0x7fb8beed278d
[   77.159316] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d cb 56 2c 00 f7 d8 64 89 08
[   77.162391] RSP: 002b:00007ffcc8dcabc8 EFLAGS: 00000246 ORIG_RAX: 0000000000000148
[   77.163672] RAX: ffffffffffffffda RBX: 0000000000000148 RCX: 00007fb8beed278d
[   77.164921] RDX: 0000000000000001 RSI: 000000000293d3f0 RDI: 00000000000000e6
[   77.166115] RBP: 0000000000000148 R08: 0000000000200000 R09: 0000000000000003
[   77.167315] R10: 0000000000000078 R11: 0000000000000246 R12: 0000000000000002
[   77.168494] R13: 00007fb8bf57c058 R14: 00007fb8bf5c26c0 R15: 00007fb8bf57c000
[   77.169656] 
[   77.169914] The buggy address belongs to the page:
[   77.170723] page:000000006280f5ba refcount:0 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x6efd5f
[   77.172271] flags: 0x17ffffc0000000()
[   77.172906] raw: 0017ffffc0000000 0000000000000000 ffffea001bb87308 0000000000000000
[   77.174299] raw: 0000000000000000 0000000000000000 00000000ffffffff 0000000000000000
[   77.175651] page dumped because: kasan: bad access detected
[   77.176583] 
[   77.176848] addr ffff8886efd5fda8 is located in stack of task trinity-c9/1593 at offset 184 in frame:
[   77.178352]  vfs_writev+0x0/0x2d0
[   77.178924] 
[   77.179158] this frame has 3 objects:
[   77.179823]  [32, 40) 'iov'
[   77.179824]  [96, 136) 'iter'
[   77.180269]  [192, 320) 'iovstack'
[   77.180788] 
[   77.181768] Memory state around the buggy address:
[   77.182927]  ffff8886efd5fc80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1
[   77.184704]  ffff8886efd5fd00: f1 f1 00 f2 f2 f2 f2 f2 f2 f2 00 00 00 00 00 f2
[   77.186457] >ffff8886efd5fd80: f2 f2 f2 f2 f2 f2 00 00 00 00 00 00 00 00 00 00
[   77.188214]                                   ^
[   77.189388]  ffff8886efd5fe00: 00 00 00 00 00 00 f3 f3 f3 f3 00 00 00 00 00 00
[   77.191019]  ffff8886efd5fe80: 00 00 00 00 00 00 f1 f1 f1 f1 00 f2 f2 f2 00 00
[   77.192846] ==================================================================
[   77.194428] Disabling lock debugging due to kernel taint


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: slab-out-of-bounds in iov_iter_revert()
  2020-09-17 16:44           ` Al Viro
  2020-09-17 17:42             ` Qian Cai
@ 2020-09-17 18:45             ` Qian Cai
  1 sibling, 0 replies; 12+ messages in thread
From: Qian Cai @ 2020-09-17 18:45 UTC (permalink / raw)
  To: Al Viro; +Cc: torvalds, vgoyal, miklos, linux-fsdevel, linux-kernel

On Thu, 2020-09-17 at 17:44 +0100, Al Viro wrote:
> On Thu, Sep 17, 2020 at 10:10:27AM -0400, Qian Cai wrote:
> 
> > [   81.942909]  generic_file_read_iter+0x23b/0x4b0
> > [   81.942918]  fuse_file_read_iter+0x280/0x4e0 [fuse]
> > [   81.942931]  ? fuse_direct_IO+0xd30/0xd30 [fuse]
> > [   81.942949]  ? _raw_spin_lock_irqsave+0x80/0xe0
> > [   81.942957]  ? timerqueue_add+0x15e/0x280
> > [   81.942960]  ? _raw_spin_lock_irqsave+0x80/0xe0
> > [   81.942966]  new_sync_read+0x3b7/0x620
> > [   81.942968]  ? __ia32_sys_llseek+0x2e0/0x2e0
> 
> Interesting...  Basic logics in there:
> 	->direct_IO() might consume more (on iov_iter_get_pages()
> and friends) than it actually reads.  We want to revert the
> excess.  Suppose by the time we call ->direct_IO() we had
> N bytes already consumed and C bytes left.  We expect that
> after ->direct_IO() returns K, we have C' bytes left, N + (C - C')
> consumed and N + K out of those actually read.  So we revert by
> C - K - C'.  You end up trying to revert beyond the beginning.
> 
> 	Use of iov_iter_truncate() is problematic here, since it
> changes the amount of data left without having consumed anything.
> Basically, it changes the position of end, and the logics in the
> caller expects that to remain unchanged.  iov_iter_reexpand() use
> should restore the position of end.
> 
> 	How much IO does it take to trigger that on your reproducer?

I can even reproduce this with a single child of the trinity:

https://people.redhat.com/qcai/iov_iter_revert/single/

[   77.841021] BUG: KASAN: stack-out-of-bounds in iov_iter_revert+0x693/0x8c0
[   77.842055] Read of size 8 at addr ffff8886efe47d98 by task trinity-c0/1449


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: slab-out-of-bounds in iov_iter_revert()
  2020-09-17 17:42             ` Qian Cai
@ 2020-09-17 18:45               ` Al Viro
  2020-09-17 20:16                 ` Qian Cai
  0 siblings, 1 reply; 12+ messages in thread
From: Al Viro @ 2020-09-17 18:45 UTC (permalink / raw)
  To: Qian Cai; +Cc: torvalds, vgoyal, miklos, linux-fsdevel, linux-kernel

On Thu, Sep 17, 2020 at 01:42:44PM -0400, Qian Cai wrote:
> > 
> > 	How much IO does it take to trigger that on your reproducer?
> 
> That is something I don't know for sure because it is always reproducible by
> running the trinity fuzzer for a few seconds (32 threads). I did another run
> below (still with your patch applied) and then tried to capture some logs here:
> 
> http://people.redhat.com/qcai/iov_iter_revert/

FWIW, there were several bugs in that patch:
	* 'shortened' possibly left uninitialized
	* possible error returns with reexpand not done

Could you try this instead?

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
---
diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index 6611ef3269a8..43c165e796da 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -3091,11 +3091,10 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 	ssize_t ret = 0;
 	struct file *file = iocb->ki_filp;
 	struct fuse_file *ff = file->private_data;
-	bool async_dio = ff->fc->async_dio;
 	loff_t pos = 0;
 	struct inode *inode;
 	loff_t i_size;
-	size_t count = iov_iter_count(iter);
+	size_t count = iov_iter_count(iter), shortened = 0;
 	loff_t offset = iocb->ki_pos;
 	struct fuse_io_priv *io;
 
@@ -3103,17 +3102,9 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 	inode = file->f_mapping->host;
 	i_size = i_size_read(inode);
 
-	if ((iov_iter_rw(iter) == READ) && (offset > i_size))
+	if ((iov_iter_rw(iter) == READ) && (offset >= i_size))
 		return 0;
 
-	/* optimization for short read */
-	if (async_dio && iov_iter_rw(iter) != WRITE && offset + count > i_size) {
-		if (offset >= i_size)
-			return 0;
-		iov_iter_truncate(iter, fuse_round_up(ff->fc, i_size - offset));
-		count = iov_iter_count(iter);
-	}
-
 	io = kmalloc(sizeof(struct fuse_io_priv), GFP_KERNEL);
 	if (!io)
 		return -ENOMEM;
@@ -3129,15 +3120,22 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 	 * By default, we want to optimize all I/Os with async request
 	 * submission to the client filesystem if supported.
 	 */
-	io->async = async_dio;
+	io->async = ff->fc->async_dio;
 	io->iocb = iocb;
 	io->blocking = is_sync_kiocb(iocb);
 
+	/* optimization for short read */
+	if (io->async && !io->write && offset + count > i_size) {
+		iov_iter_truncate(iter, fuse_round_up(ff->fc, i_size - offset));
+		shortened = count - iov_iter_count(iter);
+		count -= shortened;
+	}
+
 	/*
 	 * We cannot asynchronously extend the size of a file.
 	 * In such case the aio will behave exactly like sync io.
 	 */
-	if ((offset + count > i_size) && iov_iter_rw(iter) == WRITE)
+	if ((offset + count > i_size) && io->write)
 		io->blocking = true;
 
 	if (io->async && io->blocking) {
@@ -3155,6 +3153,7 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 	} else {
 		ret = __fuse_direct_read(io, iter, &pos);
 	}
+	iov_iter_reexpand(iter, iov_iter_count(iter) + shortened);
 
 	if (io->async) {
 		bool blocking = io->blocking;

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: slab-out-of-bounds in iov_iter_revert()
  2020-09-17 18:45               ` Al Viro
@ 2020-09-17 20:16                 ` Qian Cai
  0 siblings, 0 replies; 12+ messages in thread
From: Qian Cai @ 2020-09-17 20:16 UTC (permalink / raw)
  To: Al Viro; +Cc: torvalds, vgoyal, miklos, linux-fsdevel, linux-kernel

On Thu, 2020-09-17 at 19:45 +0100, Al Viro wrote:
> On Thu, Sep 17, 2020 at 01:42:44PM -0400, Qian Cai wrote:
> > > 	How much IO does it take to trigger that on your reproducer?
> > 
> > That is something I don't know for sure because it is always reproducible by
> > running the trinity fuzzer for a few seconds (32 threads). I did another run
> > below (still with your patch applied) and then tried to capture some logs
> > here:
> > 
> > http://people.redhat.com/qcai/iov_iter_revert/
> 
> FWIW, there were several bugs in that patch:
> 	* 'shortened' possibly left uninitialized
> 	* possible error returns with reexpand not done
> 
> Could you try this instead?

This works fine. Thanks for taking care of this, Al.

> 
> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
> ---
> diff --git a/fs/fuse/file.c b/fs/fuse/file.c
> index 6611ef3269a8..43c165e796da 100644
> --- a/fs/fuse/file.c
> +++ b/fs/fuse/file.c
> @@ -3091,11 +3091,10 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter
> *iter)
>  	ssize_t ret = 0;
>  	struct file *file = iocb->ki_filp;
>  	struct fuse_file *ff = file->private_data;
> -	bool async_dio = ff->fc->async_dio;
>  	loff_t pos = 0;
>  	struct inode *inode;
>  	loff_t i_size;
> -	size_t count = iov_iter_count(iter);
> +	size_t count = iov_iter_count(iter), shortened = 0;
>  	loff_t offset = iocb->ki_pos;
>  	struct fuse_io_priv *io;
>  
> @@ -3103,17 +3102,9 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter
> *iter)
>  	inode = file->f_mapping->host;
>  	i_size = i_size_read(inode);
>  
> -	if ((iov_iter_rw(iter) == READ) && (offset > i_size))
> +	if ((iov_iter_rw(iter) == READ) && (offset >= i_size))
>  		return 0;
>  
> -	/* optimization for short read */
> -	if (async_dio && iov_iter_rw(iter) != WRITE && offset + count > i_size)
> {
> -		if (offset >= i_size)
> -			return 0;
> -		iov_iter_truncate(iter, fuse_round_up(ff->fc, i_size - offset));
> -		count = iov_iter_count(iter);
> -	}
> -
>  	io = kmalloc(sizeof(struct fuse_io_priv), GFP_KERNEL);
>  	if (!io)
>  		return -ENOMEM;
> @@ -3129,15 +3120,22 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter
> *iter)
>  	 * By default, we want to optimize all I/Os with async request
>  	 * submission to the client filesystem if supported.
>  	 */
> -	io->async = async_dio;
> +	io->async = ff->fc->async_dio;
>  	io->iocb = iocb;
>  	io->blocking = is_sync_kiocb(iocb);
>  
> +	/* optimization for short read */
> +	if (io->async && !io->write && offset + count > i_size) {
> +		iov_iter_truncate(iter, fuse_round_up(ff->fc, i_size - offset));
> +		shortened = count - iov_iter_count(iter);
> +		count -= shortened;
> +	}
> +
>  	/*
>  	 * We cannot asynchronously extend the size of a file.
>  	 * In such case the aio will behave exactly like sync io.
>  	 */
> -	if ((offset + count > i_size) && iov_iter_rw(iter) == WRITE)
> +	if ((offset + count > i_size) && io->write)
>  		io->blocking = true;
>  
>  	if (io->async && io->blocking) {
> @@ -3155,6 +3153,7 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter
> *iter)
>  	} else {
>  		ret = __fuse_direct_read(io, iter, &pos);
>  	}
> +	iov_iter_reexpand(iter, iov_iter_count(iter) + shortened);
>  
>  	if (io->async) {
>  		bool blocking = io->blocking;
> 


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: slab-out-of-bounds in iov_iter_revert()
@ 2020-09-17 13:46 kernel test robot
  0 siblings, 0 replies; 12+ messages in thread
From: kernel test robot @ 2020-09-17 13:46 UTC (permalink / raw)
  To: kbuild

[-- Attachment #1: Type: text/plain, Size: 10041 bytes --]

CC: kbuild-all(a)lists.01.org
In-Reply-To: <20200917020440.GQ3421308@ZenIV.linux.org.uk>
References: <20200917020440.GQ3421308@ZenIV.linux.org.uk>
TO: Al Viro <viro@zeniv.linux.org.uk>
TO: Qian Cai <cai@redhat.com>
CC: torvalds(a)linux-foundation.org
CC: vgoyal(a)redhat.com
CC: miklos(a)szeredi.hu
CC: linux-fsdevel(a)vger.kernel.org
CC: linux-kernel(a)vger.kernel.org

Hi Al,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on fuse/for-next]
[also build test WARNING on linux/master linus/master v5.9-rc5 next-20200917]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Al-Viro/Re-slab-out-of-bounds-in-iov_iter_revert/20200917-100520
base:   https://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git for-next
:::::: branch date: 12 hours ago
:::::: commit date: 12 hours ago
config: x86_64-randconfig-m001-20200917 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>

smatch warnings:
fs/fuse/file.c:3205 fuse_direct_IO() error: uninitialized symbol 'shortened'.

# https://github.com/0day-ci/linux/commit/cf78ce1d71bfd0c3c06adfd5bba1664c61e36bbd
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Al-Viro/Re-slab-out-of-bounds-in-iov_iter_revert/20200917-100520
git checkout cf78ce1d71bfd0c3c06adfd5bba1664c61e36bbd
vim +/shortened +3205 fs/fuse/file.c

e5c5f05dca0cf90 Maxim Patlasov        2013-05-30  3110  
4273b793ec68753 Anand Avati           2012-02-17  3111  static ssize_t
c8b8e32d700fe94 Christoph Hellwig     2016-04-07  3112  fuse_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
4273b793ec68753 Anand Avati           2012-02-17  3113  {
9d5722b7777e64d Christoph Hellwig     2015-02-02  3114  	DECLARE_COMPLETION_ONSTACK(wait);
4273b793ec68753 Anand Avati           2012-02-17  3115  	ssize_t ret = 0;
60b9df7a54804a9 Miklos Szeredi        2013-05-01  3116  	struct file *file = iocb->ki_filp;
60b9df7a54804a9 Miklos Szeredi        2013-05-01  3117  	struct fuse_file *ff = file->private_data;
e5c5f05dca0cf90 Maxim Patlasov        2013-05-30  3118  	bool async_dio = ff->fc->async_dio;
4273b793ec68753 Anand Avati           2012-02-17  3119  	loff_t pos = 0;
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3120  	struct inode *inode;
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3121  	loff_t i_size;
cf78ce1d71bfd0c Al Viro               2020-09-17  3122  	size_t count = iov_iter_count(iter), shortened;
c8b8e32d700fe94 Christoph Hellwig     2016-04-07  3123  	loff_t offset = iocb->ki_pos;
36cf66ed9f871fc Maxim Patlasov        2012-12-14  3124  	struct fuse_io_priv *io;
4273b793ec68753 Anand Avati           2012-02-17  3125  
4273b793ec68753 Anand Avati           2012-02-17  3126  	pos = offset;
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3127  	inode = file->f_mapping->host;
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3128  	i_size = i_size_read(inode);
4273b793ec68753 Anand Avati           2012-02-17  3129  
6f67376318abea5 Omar Sandoval         2015-03-16  3130  	if ((iov_iter_rw(iter) == READ) && (offset > i_size))
9fe55eea7e4b444 Steven Whitehouse     2014-01-24  3131  		return 0;
9fe55eea7e4b444 Steven Whitehouse     2014-01-24  3132  
439ee5f0c5080d4 Maxim Patlasov        2012-12-14  3133  	/* optimization for short read */
6f67376318abea5 Omar Sandoval         2015-03-16  3134  	if (async_dio && iov_iter_rw(iter) != WRITE && offset + count > i_size) {
439ee5f0c5080d4 Maxim Patlasov        2012-12-14  3135  		if (offset >= i_size)
439ee5f0c5080d4 Maxim Patlasov        2012-12-14  3136  			return 0;
5da784cce4308ae Constantine Shulyupin 2018-09-06  3137  		iov_iter_truncate(iter, fuse_round_up(ff->fc, i_size - offset));
cf78ce1d71bfd0c Al Viro               2020-09-17  3138  		shortened = count - iov_iter_count(iter);
cf78ce1d71bfd0c Al Viro               2020-09-17  3139  		count -= shortened;
439ee5f0c5080d4 Maxim Patlasov        2012-12-14  3140  	}
439ee5f0c5080d4 Maxim Patlasov        2012-12-14  3141  
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3142  	io = kmalloc(sizeof(struct fuse_io_priv), GFP_KERNEL);
36cf66ed9f871fc Maxim Patlasov        2012-12-14  3143  	if (!io)
36cf66ed9f871fc Maxim Patlasov        2012-12-14  3144  		return -ENOMEM;
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3145  	spin_lock_init(&io->lock);
744742d692e37ad Seth Forshee          2016-03-11  3146  	kref_init(&io->refcnt);
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3147  	io->reqs = 1;
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3148  	io->bytes = -1;
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3149  	io->size = 0;
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3150  	io->offset = offset;
6f67376318abea5 Omar Sandoval         2015-03-16  3151  	io->write = (iov_iter_rw(iter) == WRITE);
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3152  	io->err = 0;
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3153  	/*
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3154  	 * By default, we want to optimize all I/Os with async request
60b9df7a54804a9 Miklos Szeredi        2013-05-01  3155  	 * submission to the client filesystem if supported.
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3156  	 */
e5c5f05dca0cf90 Maxim Patlasov        2013-05-30  3157  	io->async = async_dio;
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3158  	io->iocb = iocb;
7879c4e58b7c884 Ashish Sangwan        2016-04-07  3159  	io->blocking = is_sync_kiocb(iocb);
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3160  
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3161  	/*
7879c4e58b7c884 Ashish Sangwan        2016-04-07  3162  	 * We cannot asynchronously extend the size of a file.
7879c4e58b7c884 Ashish Sangwan        2016-04-07  3163  	 * In such case the aio will behave exactly like sync io.
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3164  	 */
7879c4e58b7c884 Ashish Sangwan        2016-04-07  3165  	if ((offset + count > i_size) && iov_iter_rw(iter) == WRITE)
7879c4e58b7c884 Ashish Sangwan        2016-04-07  3166  		io->blocking = true;
4273b793ec68753 Anand Avati           2012-02-17  3167  
7879c4e58b7c884 Ashish Sangwan        2016-04-07  3168  	if (io->async && io->blocking) {
744742d692e37ad Seth Forshee          2016-03-11  3169  		/*
744742d692e37ad Seth Forshee          2016-03-11  3170  		 * Additional reference to keep io around after
744742d692e37ad Seth Forshee          2016-03-11  3171  		 * calling fuse_aio_complete()
744742d692e37ad Seth Forshee          2016-03-11  3172  		 */
744742d692e37ad Seth Forshee          2016-03-11  3173  		kref_get(&io->refcnt);
9d5722b7777e64d Christoph Hellwig     2015-02-02  3174  		io->done = &wait;
744742d692e37ad Seth Forshee          2016-03-11  3175  	}
9d5722b7777e64d Christoph Hellwig     2015-02-02  3176  
6f67376318abea5 Omar Sandoval         2015-03-16  3177  	if (iov_iter_rw(iter) == WRITE) {
812408fb51ef580 Al Viro               2015-03-30  3178  		ret = fuse_direct_io(io, iter, &pos, FUSE_DIO_WRITE);
812408fb51ef580 Al Viro               2015-03-30  3179  		fuse_invalidate_attr(inode);
812408fb51ef580 Al Viro               2015-03-30  3180  	} else {
d22a943f44c79c9 Al Viro               2014-03-16  3181  		ret = __fuse_direct_read(io, iter, &pos);
812408fb51ef580 Al Viro               2015-03-30  3182  	}
36cf66ed9f871fc Maxim Patlasov        2012-12-14  3183  
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3184  	if (io->async) {
ebacb8127359955 Lukas Czerner         2018-11-09  3185  		bool blocking = io->blocking;
ebacb8127359955 Lukas Czerner         2018-11-09  3186  
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3187  		fuse_aio_complete(io, ret < 0 ? ret : 0, -1);
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3188  
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3189  		/* we have a non-extending, async request, so return */
ebacb8127359955 Lukas Czerner         2018-11-09  3190  		if (!blocking)
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3191  			return -EIOCBQUEUED;
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3192  
9d5722b7777e64d Christoph Hellwig     2015-02-02  3193  		wait_for_completion(&wait);
9d5722b7777e64d Christoph Hellwig     2015-02-02  3194  		ret = fuse_get_res_by_io(io);
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3195  	}
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3196  
744742d692e37ad Seth Forshee          2016-03-11  3197  	kref_put(&io->refcnt, fuse_io_release);
9d5722b7777e64d Christoph Hellwig     2015-02-02  3198  
6f67376318abea5 Omar Sandoval         2015-03-16  3199  	if (iov_iter_rw(iter) == WRITE) {
efb9fa9e911b23c Maxim Patlasov        2012-12-18  3200  		if (ret > 0)
bcba24ccdc82f74 Maxim Patlasov        2012-12-14  3201  			fuse_write_update_size(inode, pos);
efb9fa9e911b23c Maxim Patlasov        2012-12-18  3202  		else if (ret < 0 && offset + count > i_size)
efb9fa9e911b23c Maxim Patlasov        2012-12-18  3203  			fuse_do_truncate(file);
efb9fa9e911b23c Maxim Patlasov        2012-12-18  3204  	}
cf78ce1d71bfd0c Al Viro               2020-09-17 @3205  	if (shortened)
cf78ce1d71bfd0c Al Viro               2020-09-17  3206  		iov_iter_reexpand(iter, shortened);
4273b793ec68753 Anand Avati           2012-02-17  3207  
4273b793ec68753 Anand Avati           2012-02-17  3208  	return ret;
4273b793ec68753 Anand Avati           2012-02-17  3209  }
4273b793ec68753 Anand Avati           2012-02-17  3210  

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 30052 bytes --]

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2020-09-17 20:17 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-11 21:59 slab-out-of-bounds in iov_iter_revert() Qian Cai
2020-09-11 23:55 ` Al Viro
2020-09-16 21:09   ` Qian Cai
2020-09-17  2:04     ` Al Viro
2020-09-17  2:14       ` Al Viro
2020-09-17 14:10         ` Qian Cai
2020-09-17 16:44           ` Al Viro
2020-09-17 17:42             ` Qian Cai
2020-09-17 18:45               ` Al Viro
2020-09-17 20:16                 ` Qian Cai
2020-09-17 18:45             ` Qian Cai
2020-09-17 13:46 kernel test robot

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.