[LSF/MM TOPIC] sharing pages between mappings

[LSF/MM TOPIC] sharing pages between mappings

[Miklos Szeredi]

I know there's work on this for xfs, but could this be done in generic mm code?

What are the obstacles?  page->mapping and page->index are the obvious ones.

If that's too difficult is it maybe enough to share mappings between
files while they are completely identical and clone the mapping when
necessary?

All COW filesystems would benefit, as well as layered ones: lots of
fuse fs, and in some cases overlayfs too.

Related:  what can DAX do in the presence of cloned block?

Thanks,
Miklos

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] sharing pages between mappings

[Jan Kara]

On Wed 11-01-17 11:29:28, Miklos Szeredi wrote:
> I know there's work on this for xfs, but could this be done in generic mm
> code?
> 
> What are the obstacles?  page->mapping and page->index are the obvious
> ones.

Yes, these two are the main that come to my mind. Also you'd need to
somehow share the mapping->i_mmap tree so that unmap_mapping_range() works.

> If that's too difficult is it maybe enough to share mappings between
> files while they are completely identical and clone the mapping when
> necessary?

Well, but how would the page->mapping->host indirection work? Even if you
have identical contents of the mappings, you still need to be aware there
are several inodes behind them and you need to pick the right one
somehow...

> All COW filesystems would benefit, as well as layered ones: lots of
> fuse fs, and in some cases overlayfs too.
> 
> Related:  what can DAX do in the presence of cloned block?

For DAX handling a block COW should be doable if that is what you are
asking about. Handling of blocks that can be written to while they are
shared will be rather difficult (you have problems with keeping dirty bits
in the radix tree consistent if nothing else).

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] sharing pages between mappings

[Miklos Szeredi]

On Wed, Jan 11, 2017 at 12:51 PM, Jan Kara <jack@suse.cz> wrote:
> On Wed 11-01-17 11:29:28, Miklos Szeredi wrote:
>> I know there's work on this for xfs, but could this be done in generic mm
>> code?
>>
>> What are the obstacles?  page->mapping and page->index are the obvious
>> ones.
>
> Yes, these two are the main that come to my mind. Also you'd need to
> somehow share the mapping->i_mmap tree so that unmap_mapping_range() works.
>
>> If that's too difficult is it maybe enough to share mappings between
>> files while they are completely identical and clone the mapping when
>> necessary?
>
> Well, but how would the page->mapping->host indirection work? Even if you
> have identical contents of the mappings, you still need to be aware there
> are several inodes behind them and you need to pick the right one
> somehow...

When do we actually need page->mapping->host?  The only place where
it's not available is page writeback.  Then we can know that the
original page was already cow-ed and after being cowed, the page
belong only to a single inode.

What then happens if the newly written data is cloned before being
written back?   We can either write back the page during the clone, so
that only clean pages are ever shared.  Or we can let dirty pages be
shared between inodes.  In that latter case the question is: do we
care about which inode we use for writing back the data?  Is the inode
needed at all?  I don't know enough about filesystem internals to see
clearly what happens in such a situation.

>> All COW filesystems would benefit, as well as layered ones: lots of
>> fuse fs, and in some cases overlayfs too.
>>
>> Related:  what can DAX do in the presence of cloned block?
>
> For DAX handling a block COW should be doable if that is what you are
> asking about. Handling of blocks that can be written to while they are
> shared will be rather difficult (you have problems with keeping dirty bits
> in the radix tree consistent if nothing else).

What happens if you do:

- clone_file_range(A, off1, B, off2, len);

- mmap both A and B using DAX.

The mapping will contain the same struct page for two different mappings, no?

Thanks,
Miklos

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] sharing pages between mappings

[Darrick J. Wong]

On Wed, Jan 11, 2017 at 12:51:43PM +0100, Jan Kara wrote:
> On Wed 11-01-17 11:29:28, Miklos Szeredi wrote:
> > I know there's work on this for xfs, but could this be done in generic mm
> > code?
> > 
> > What are the obstacles?  page->mapping and page->index are the obvious
> > ones.
> 
> Yes, these two are the main that come to my mind. Also you'd need to
> somehow share the mapping->i_mmap tree so that unmap_mapping_range() works.
> 
> > If that's too difficult is it maybe enough to share mappings between
> > files while they are completely identical and clone the mapping when
> > necessary?
> 
> Well, but how would the page->mapping->host indirection work? Even if you
> have identical contents of the mappings, you still need to be aware there
> are several inodes behind them and you need to pick the right one
> somehow...
> 
> > All COW filesystems would benefit, as well as layered ones: lots of
> > fuse fs, and in some cases overlayfs too.
> > 
> > Related:  what can DAX do in the presence of cloned block?
> 
> For DAX handling a block COW should be doable if that is what you are
> asking about. Handling of blocks that can be written to while they are
> shared will be rather difficult (you have problems with keeping dirty bits
> in the radix tree consistent if nothing else).

I'm also interested in this topic, though I haven't gotten any further
than a hand-wavy notion of handling cow by allocating new blocks, memcpy
the contents to the new blocks (how?), then update the mappings to point
to the new blocks (how?).  It looks a lot easier now with the iomap
stuff, but that's as far as I got. :)

(IOWs it basically took all the time since the last LSF to get reflink
polished enough to handle regular files reasonably well.)

--D

> 
> 								Honza
> -- 
> Jan Kara <jack@suse.com>
> SUSE Labs, CR
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [LSF/MM TOPIC] sharing pages between mappings

[Andreas Dilger]

[-- Attachment #1: Type: text/plain, Size: 960 bytes --]

On Jan 11, 2017, at 3:29 AM, Miklos Szeredi <miklos@szeredi.hu> wrote:
> 
> I know there's work on this for xfs, but could this be done in generic mm code?
> 
> What are the obstacles?  page->mapping and page->index are the obvious ones.
> 
> If that's too difficult is it maybe enough to share mappings between
> files while they are completely identical and clone the mapping when
> necessary?
> 
> All COW filesystems would benefit, as well as layered ones: lots of
> fuse fs, and in some cases overlayfs too.

For layered filesystems it would also be useful to have an API to move
pages between mappings easily.

> Related:  what can DAX do in the presence of cloned block?
> 
> Thanks,
> Miklos
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


Cheers, Andreas






[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Lsf-pc] [LSF/MM TOPIC] sharing pages between mappings

[Jan Kara]

On Wed 11-01-17 15:13:19, Miklos Szeredi wrote:
> On Wed, Jan 11, 2017 at 12:51 PM, Jan Kara <jack@suse.cz> wrote:
> > On Wed 11-01-17 11:29:28, Miklos Szeredi wrote:
> >> I know there's work on this for xfs, but could this be done in generic mm
> >> code?
> >>
> >> What are the obstacles?  page->mapping and page->index are the obvious
> >> ones.
> >
> > Yes, these two are the main that come to my mind. Also you'd need to
> > somehow share the mapping->i_mmap tree so that unmap_mapping_range() works.
> >
> >> If that's too difficult is it maybe enough to share mappings between
> >> files while they are completely identical and clone the mapping when
> >> necessary?
> >
> > Well, but how would the page->mapping->host indirection work? Even if you
> > have identical contents of the mappings, you still need to be aware there
> > are several inodes behind them and you need to pick the right one
> > somehow...
> 
> When do we actually need page->mapping->host?  The only place where
> it's not available is page writeback.  Then we can know that the
> original page was already cow-ed and after being cowed, the page
> belong only to a single inode.

Yeah, in principle the information may exist, however propagating it to all
appropriate place may be a mess.

> What then happens if the newly written data is cloned before being
> written back?   We can either write back the page during the clone, so
> that only clean pages are ever shared.  Or we can let dirty pages be
> shared between inodes.

The former is what I'd suggest for sanity... I.e. share only read-only
pages.

> In that latter case the question is: do we
> care about which inode we use for writing back the data?  Is the inode
> needed at all?  I don't know enough about filesystem internals to see
> clearly what happens in such a situation.
> 
> >> All COW filesystems would benefit, as well as layered ones: lots of
> >> fuse fs, and in some cases overlayfs too.
> >>
> >> Related:  what can DAX do in the presence of cloned block?
> >
> > For DAX handling a block COW should be doable if that is what you are
> > asking about. Handling of blocks that can be written to while they are
> > shared will be rather difficult (you have problems with keeping dirty bits
> > in the radix tree consistent if nothing else).
> 
> What happens if you do:
> 
> - clone_file_range(A, off1, B, off2, len);
> 
> - mmap both A and B using DAX.
> 
> The mapping will contain the same struct page for two different mappings, no?

Not the same struct page, as DAX does not have pages with struct page.
However the same pfn will be underlying off1 of A and off2 of B. And for
reads this is just fine. Once you want to write, you have to make sure you
COW before you start modifying the data or you'll get data corruption (we
synchronize operations using the exceptional entries in mapping->page_tree
in DAX and these are separate for A and B).

									Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>