Re: [PATCH v7] fs: Fix page cache inconsistency when mixing buffered and AIO DIO

[Lukas Czerner <lczerner@redhat.com>]

On Tue, Oct 10, 2017 at 04:34:45PM +0200, David Sterba wrote:
> On Tue, Aug 15, 2017 at 03:28:54PM +0200, Lukas Czerner wrote:
> > +	/*
> > +	 * Try again to invalidate clean pages which might have been cached by
> > +	 * non-direct readahead, or faulted in by get_user_pages() if the source
> > +	 * of the write was an mmap'ed region of the file we're writing.  Either
> > +	 * one is a pretty crazy thing to do, so we don't support it 100%.  If
> > +	 * this invalidation fails, tough, the write still worked...
> > +	 */
> > +	if (ret > 0 && dio->op == REQ_OP_WRITE &&
> > +	    dio->inode->i_mapping->nrpages) {
> > +		err = invalidate_inode_pages2_range(dio->inode->i_mapping,
> > +					offset >> PAGE_SHIFT,
> > +					(offset + ret - 1) >> PAGE_SHIFT);
> > +		WARN_ON_ONCE(err);
> 
> fstests/btrfs/062 reports this:
> 
> [ 6235.547298] ------------[ cut here ]------------
> [ 6235.552098] WARNING: CPU: 7 PID: 24321 at fs/direct-io.c:274 dio_complete+0x16f/0x1f0
> [ 6235.560858] Modules linked in: dm_flakey loop rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace sunrpc fscache af_packet br_netfilter bridge stp llc iscsi_ibft iscsi_boot_sysfs btrfs xor zstd_decompress zstd_compress i2c_algo_bit drm_kms_helper xxhash zlib_deflate raid6_pq syscopyarea sysfillrect sysimgblt fb_sys_fops ttm tg3 drm dm_mod dax ptp kvm_amd pps_core kvm libphy tpm_infineon mptctl shpchp k10temp tpm_tis tpm_tis_core button i2c_piix4 tpm pcspkr irqbypass acpi_cpufreq ext4 mbcache jbd2 sr_mod cdrom ohci_pci ehci_pci ohci_hcd mptsas ehci_hcd scsi_transport_sas ata_generic mptscsih serio_raw mptbase usbcore sata_svw pata_serverworks sg scsi_dh_rdac scsi_dh_emc scsi_dh_alua
> [ 6235.560942] CPU: 7 PID: 24321 Comm: kworker/7:1 Not tainted 4.14.0-rc4-1.ge195904-vanilla+ #71
> [ 6235.560944] Hardware name: empty empty/S3993, BIOS PAQEX0-3 02/24/2008
> [ 6235.560950] Workqueue: dio/sdb6 dio_aio_complete_work
> [ 6235.560953] task: ffff894fe0bd8300 task.stack: ffffb45742f7c000
> [ 6235.560957] RIP: 0010:dio_complete+0x16f/0x1f0
> [ 6235.560959] RSP: 0018:ffffb45742f7fde8 EFLAGS: 00010286
> [ 6235.560968] RAX: 00000000fffffff0 RBX: ffff894fd1e3a680 RCX: ffff894fe0bd8300
> [ 6235.560970] RDX: 0000000000000000 RSI: 00000000000002b4 RDI: ffffffffaba438e9
> [ 6235.560971] RBP: ffffb45742f7fe10 R08: 0000000000000000 R09: 0000000000000025
> [ 6235.560973] R10: 0000000000000000 R11: 0000000000000000 R12: 000000000001e000
> [ 6235.560974] R13: 000000000001e000 R14: 0000000000007000 R15: 0000000000000001
> [ 6235.560977] FS:  0000000000000000(0000) GS:ffff894fefdc0000(0000) knlGS:0000000000000000
> [ 6235.560978] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 6235.560980] CR2: 00007fe1e1dfb610 CR3: 0000000213ee7000 CR4: 00000000000006e0
> [ 6235.560982] Call Trace:
> [ 6235.561075]  dio_aio_complete_work+0x1c/0x20
> [ 6235.561082]  process_one_work+0x1d8/0x620
> [ 6235.561085]  ? process_one_work+0x14b/0x620
> [ 6235.561092]  worker_thread+0x4d/0x3c0
> [ 6235.561097]  ? trace_hardirqs_on+0xd/0x10
> [ 6235.561105]  kthread+0x152/0x190
> [ 6235.561107]  ? process_one_work+0x620/0x620
> [ 6235.561111]  ? kthread_create_on_node+0x40/0x40
> [ 6235.561116]  ? do_syscall_64+0x69/0x180
> [ 6235.561122]  ret_from_fork+0x2a/0x40
> [ 6235.561131] Code: 48 83 bf 00 01 00 00 00 0f 84 37 ff ff ff 4b 8d 54 34 ff 4c 89 f6 48 c1 fe 0c 48 c1 fa 0c e8 49 82 f2 ff 85 c0 0f 84 1a ff ff ff <0f> ff e9 13 ff ff ff 48 81 c7 e0 00 00 00 be 09 00 00 00 e8 79
> [ 6235.561179] ---[ end trace ba80cd81f19cb389 ]---
> 
> I've added Chris and Bo to CC if they have more to say about the specifics of
> dio and buffered writes as implemented in btrfs.

There are several places where we warn on invalidation failing. The
question is why invalidation failed on btrfs in this test.

-Lukas