* [LSF/MM TOPIC] sharing pages between mappings @ 2017-01-11 10:29 Miklos Szeredi 2017-01-11 11:51 ` [Lsf-pc] " Jan Kara 2017-01-11 20:35 ` Andreas Dilger 0 siblings, 2 replies; 6+ messages in thread From: Miklos Szeredi @ 2017-01-11 10:29 UTC (permalink / raw) To: linux-mm, linux-fsdevel, linux-btrfs, lsf-pc I know there's work on this for xfs, but could this be done in generic mm code? What are the obstacles? page->mapping and page->index are the obvious ones. If that's too difficult is it maybe enough to share mappings between files while they are completely identical and clone the mapping when necessary? All COW filesystems would benefit, as well as layered ones: lots of fuse fs, and in some cases overlayfs too. Related: what can DAX do in the presence of cloned block? Thanks, Miklos -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] sharing pages between mappings 2017-01-11 10:29 [LSF/MM TOPIC] sharing pages between mappings Miklos Szeredi @ 2017-01-11 11:51 ` Jan Kara 2017-01-11 14:13 ` Miklos Szeredi 2017-01-11 18:05 ` Darrick J. Wong 2017-01-11 20:35 ` Andreas Dilger 1 sibling, 2 replies; 6+ messages in thread From: Jan Kara @ 2017-01-11 11:51 UTC (permalink / raw) To: Miklos Szeredi; +Cc: linux-mm, linux-fsdevel, linux-btrfs, lsf-pc On Wed 11-01-17 11:29:28, Miklos Szeredi wrote: > I know there's work on this for xfs, but could this be done in generic mm > code? > > What are the obstacles? page->mapping and page->index are the obvious > ones. Yes, these two are the main that come to my mind. Also you'd need to somehow share the mapping->i_mmap tree so that unmap_mapping_range() works. > If that's too difficult is it maybe enough to share mappings between > files while they are completely identical and clone the mapping when > necessary? Well, but how would the page->mapping->host indirection work? Even if you have identical contents of the mappings, you still need to be aware there are several inodes behind them and you need to pick the right one somehow... > All COW filesystems would benefit, as well as layered ones: lots of > fuse fs, and in some cases overlayfs too. > > Related: what can DAX do in the presence of cloned block? For DAX handling a block COW should be doable if that is what you are asking about. Handling of blocks that can be written to while they are shared will be rather difficult (you have problems with keeping dirty bits in the radix tree consistent if nothing else). Honza -- Jan Kara <jack@suse.com> SUSE Labs, CR -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] sharing pages between mappings 2017-01-11 11:51 ` [Lsf-pc] " Jan Kara @ 2017-01-11 14:13 ` Miklos Szeredi 2017-01-17 15:46 ` Jan Kara 2017-01-11 18:05 ` Darrick J. Wong 1 sibling, 1 reply; 6+ messages in thread From: Miklos Szeredi @ 2017-01-11 14:13 UTC (permalink / raw) To: Jan Kara; +Cc: linux-mm, linux-fsdevel, linux-btrfs, lsf-pc On Wed, Jan 11, 2017 at 12:51 PM, Jan Kara <jack@suse.cz> wrote: > On Wed 11-01-17 11:29:28, Miklos Szeredi wrote: >> I know there's work on this for xfs, but could this be done in generic mm >> code? >> >> What are the obstacles? page->mapping and page->index are the obvious >> ones. > > Yes, these two are the main that come to my mind. Also you'd need to > somehow share the mapping->i_mmap tree so that unmap_mapping_range() works. > >> If that's too difficult is it maybe enough to share mappings between >> files while they are completely identical and clone the mapping when >> necessary? > > Well, but how would the page->mapping->host indirection work? Even if you > have identical contents of the mappings, you still need to be aware there > are several inodes behind them and you need to pick the right one > somehow... When do we actually need page->mapping->host? The only place where it's not available is page writeback. Then we can know that the original page was already cow-ed and after being cowed, the page belong only to a single inode. What then happens if the newly written data is cloned before being written back? We can either write back the page during the clone, so that only clean pages are ever shared. Or we can let dirty pages be shared between inodes. In that latter case the question is: do we care about which inode we use for writing back the data? Is the inode needed at all? I don't know enough about filesystem internals to see clearly what happens in such a situation. >> All COW filesystems would benefit, as well as layered ones: lots of >> fuse fs, and in some cases overlayfs too. >> >> Related: what can DAX do in the presence of cloned block? > > For DAX handling a block COW should be doable if that is what you are > asking about. Handling of blocks that can be written to while they are > shared will be rather difficult (you have problems with keeping dirty bits > in the radix tree consistent if nothing else). What happens if you do: - clone_file_range(A, off1, B, off2, len); - mmap both A and B using DAX. The mapping will contain the same struct page for two different mappings, no? Thanks, Miklos -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] sharing pages between mappings 2017-01-11 14:13 ` Miklos Szeredi @ 2017-01-17 15:46 ` Jan Kara 0 siblings, 0 replies; 6+ messages in thread From: Jan Kara @ 2017-01-17 15:46 UTC (permalink / raw) To: Miklos Szeredi; +Cc: Jan Kara, linux-mm, linux-fsdevel, linux-btrfs, lsf-pc On Wed 11-01-17 15:13:19, Miklos Szeredi wrote: > On Wed, Jan 11, 2017 at 12:51 PM, Jan Kara <jack@suse.cz> wrote: > > On Wed 11-01-17 11:29:28, Miklos Szeredi wrote: > >> I know there's work on this for xfs, but could this be done in generic mm > >> code? > >> > >> What are the obstacles? page->mapping and page->index are the obvious > >> ones. > > > > Yes, these two are the main that come to my mind. Also you'd need to > > somehow share the mapping->i_mmap tree so that unmap_mapping_range() works. > > > >> If that's too difficult is it maybe enough to share mappings between > >> files while they are completely identical and clone the mapping when > >> necessary? > > > > Well, but how would the page->mapping->host indirection work? Even if you > > have identical contents of the mappings, you still need to be aware there > > are several inodes behind them and you need to pick the right one > > somehow... > > When do we actually need page->mapping->host? The only place where > it's not available is page writeback. Then we can know that the > original page was already cow-ed and after being cowed, the page > belong only to a single inode. Yeah, in principle the information may exist, however propagating it to all appropriate place may be a mess. > What then happens if the newly written data is cloned before being > written back? We can either write back the page during the clone, so > that only clean pages are ever shared. Or we can let dirty pages be > shared between inodes. The former is what I'd suggest for sanity... I.e. share only read-only pages. > In that latter case the question is: do we > care about which inode we use for writing back the data? Is the inode > needed at all? I don't know enough about filesystem internals to see > clearly what happens in such a situation. > > >> All COW filesystems would benefit, as well as layered ones: lots of > >> fuse fs, and in some cases overlayfs too. > >> > >> Related: what can DAX do in the presence of cloned block? > > > > For DAX handling a block COW should be doable if that is what you are > > asking about. Handling of blocks that can be written to while they are > > shared will be rather difficult (you have problems with keeping dirty bits > > in the radix tree consistent if nothing else). > > What happens if you do: > > - clone_file_range(A, off1, B, off2, len); > > - mmap both A and B using DAX. > > The mapping will contain the same struct page for two different mappings, no? Not the same struct page, as DAX does not have pages with struct page. However the same pfn will be underlying off1 of A and off2 of B. And for reads this is just fine. Once you want to write, you have to make sure you COW before you start modifying the data or you'll get data corruption (we synchronize operations using the exceptional entries in mapping->page_tree in DAX and these are separate for A and B). Honza -- Jan Kara <jack@suse.com> SUSE Labs, CR -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] sharing pages between mappings 2017-01-11 11:51 ` [Lsf-pc] " Jan Kara 2017-01-11 14:13 ` Miklos Szeredi @ 2017-01-11 18:05 ` Darrick J. Wong 1 sibling, 0 replies; 6+ messages in thread From: Darrick J. Wong @ 2017-01-11 18:05 UTC (permalink / raw) To: Jan Kara; +Cc: Miklos Szeredi, linux-mm, linux-fsdevel, linux-btrfs, lsf-pc On Wed, Jan 11, 2017 at 12:51:43PM +0100, Jan Kara wrote: > On Wed 11-01-17 11:29:28, Miklos Szeredi wrote: > > I know there's work on this for xfs, but could this be done in generic mm > > code? > > > > What are the obstacles? page->mapping and page->index are the obvious > > ones. > > Yes, these two are the main that come to my mind. Also you'd need to > somehow share the mapping->i_mmap tree so that unmap_mapping_range() works. > > > If that's too difficult is it maybe enough to share mappings between > > files while they are completely identical and clone the mapping when > > necessary? > > Well, but how would the page->mapping->host indirection work? Even if you > have identical contents of the mappings, you still need to be aware there > are several inodes behind them and you need to pick the right one > somehow... > > > All COW filesystems would benefit, as well as layered ones: lots of > > fuse fs, and in some cases overlayfs too. > > > > Related: what can DAX do in the presence of cloned block? > > For DAX handling a block COW should be doable if that is what you are > asking about. Handling of blocks that can be written to while they are > shared will be rather difficult (you have problems with keeping dirty bits > in the radix tree consistent if nothing else). I'm also interested in this topic, though I haven't gotten any further than a hand-wavy notion of handling cow by allocating new blocks, memcpy the contents to the new blocks (how?), then update the mappings to point to the new blocks (how?). It looks a lot easier now with the iomap stuff, but that's as far as I got. :) (IOWs it basically took all the time since the last LSF to get reflink polished enough to handle regular files reasonably well.) --D > > Honza > -- > Jan Kara <jack@suse.com> > SUSE Labs, CR > -- > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [LSF/MM TOPIC] sharing pages between mappings 2017-01-11 10:29 [LSF/MM TOPIC] sharing pages between mappings Miklos Szeredi 2017-01-11 11:51 ` [Lsf-pc] " Jan Kara @ 2017-01-11 20:35 ` Andreas Dilger 1 sibling, 0 replies; 6+ messages in thread From: Andreas Dilger @ 2017-01-11 20:35 UTC (permalink / raw) To: Miklos Szeredi; +Cc: linux-mm, linux-fsdevel, linux-btrfs, lsf-pc [-- Attachment #1: Type: text/plain, Size: 960 bytes --] On Jan 11, 2017, at 3:29 AM, Miklos Szeredi <miklos@szeredi.hu> wrote: > > I know there's work on this for xfs, but could this be done in generic mm code? > > What are the obstacles? page->mapping and page->index are the obvious ones. > > If that's too difficult is it maybe enough to share mappings between > files while they are completely identical and clone the mapping when > necessary? > > All COW filesystems would benefit, as well as layered ones: lots of > fuse fs, and in some cases overlayfs too. For layered filesystems it would also be useful to have an API to move pages between mappings easily. > Related: what can DAX do in the presence of cloned block? > > Thanks, > Miklos > -- > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html Cheers, Andreas [-- Attachment #2: Message signed with OpenPGP using GPGMail --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-01-17 15:46 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-01-11 10:29 [LSF/MM TOPIC] sharing pages between mappings Miklos Szeredi 2017-01-11 11:51 ` [Lsf-pc] " Jan Kara 2017-01-11 14:13 ` Miklos Szeredi 2017-01-17 15:46 ` Jan Kara 2017-01-11 18:05 ` Darrick J. Wong 2017-01-11 20:35 ` Andreas Dilger
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).