Re: [PATCH 0/2 RFC v2] fs: Hole punch vs page cache filling races

[Dave Chinner <david@fromorbit.com>]

On Mon, Feb 08, 2021 at 05:39:16PM +0100, Jan Kara wrote:
> Hello,
> 
> Amir has reported [1] a that ext4 has a potential issues when reads can race
> with hole punching possibly exposing stale data from freed blocks or even
> corrupting filesystem when stale mapping data gets used for writeout. The
> problem is that during hole punching, new page cache pages can get instantiated
> and block mapping from the looked up in a punched range after
> truncate_inode_pages() has run but before the filesystem removes blocks from
> the file. In principle any filesystem implementing hole punching thus needs to
> implement a mechanism to block instantiating page cache pages during hole
> punching to avoid this race. This is further complicated by the fact that there
> are multiple places that can instantiate pages in page cache.  We can have
> regular read(2) or page fault doing this but fadvise(2) or madvise(2) can also
> result in reading in page cache pages through force_page_cache_readahead().
> 
> There are couple of ways how to fix this. First way (currently implemented by
> XFS) is to protect read(2) and *advise(2) calls with i_rwsem so that they are
> serialized with hole punching. This is easy to do but as a result all reads
> would then be serialized with writes and thus mixed read-write workloads suffer
> heavily on ext4. Thus this series introduces inode->i_mapping_sem and uses it
> when creating new pages in the page cache and looking up their corresponding
> block mapping. We also replace EXT4_I(inode)->i_mmap_sem with this new rwsem
> which provides necessary serialization with hole punching for ext4. If this
> approach looks viable, I'll convert also other equivalent fs locks to use this
> new VFS semaphore instead - in particular XFS' XFS_MMAPLOCK, f2fs's i_mmap_sem,
> fuse's i_mmap_sem and maybe others as well.

So a page cache read needs to take this lock.

What happens if a hole punch range is not block aligned and needs to
zero part of a block that is not in cache? i.e. we do this:

fallocate(punch_hole)
down_write(i_mapping_sem)
invalidate whole cached pages within the punched range
zero sub-block edges of range
punch extents out extents
up_write(i_mapping_sem)

The question that comes to mind for me is about the zeroing of the
edges of the range. If those pages are not in cache, we have to read
them in, and that goes through the page cache, which according to
the rules you mention above should be taking
down_read(i_mapping_sem)....

Of course, if we are now requiring different locking for page cache
instantiation done by read() call patchs vs those done by, say,
iomap_zero_range(), then this seems like it is opening up a
can of worms with some interfaces requiring the caller to hold
i_mapping_sem and others taking it internally so the caller must not
hold it....

Can you spell out the way this lock is supposed to nest amongst
other locks, and where and how it is expected to be taken, what the
rules are for doing atomic RMW operations through the page cache
while we have IO and page faults locked out, etc?

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com