* [LSF/MM TOPIC] The end of the DAX experiment @ 2019-02-06 21:12 Dan Williams 2019-02-07 21:51 ` Kani, Toshi ` (2 more replies) 0 siblings, 3 replies; 16+ messages in thread From: Dan Williams @ 2019-02-06 21:12 UTC (permalink / raw) To: lsf-pc Cc: linux-nvdimm, linux-xfs, Linux Kernel Mailing List, linux-fsdevel, linux-ext4 Before people get too excited this isn't a proposal to kill DAX. The topic proposal is a discussion to resolve lingering open questions that currently motivate ext4 and xfs to scream "EXPERIMENTAL" when the current DAX facilities are enabled. The are 2 primary concerns to resolve. Enumerate the remaining features/fixes, and identify a path to implement it all without regressing any existing application use cases. An enumeration of remaining projects follows, please expand this list if I missed something: * "DAX" has no specific meaning by itself, users have 2 use cases for "DAX" capabilities: userspace cache management via MAP_SYNC, and page cache avoidance where the latter aspect of DAX has no current api to discover / use it. The project is to supplement MAP_SYNC with a MAP_DIRECT facility and MADV_SYNC / MADV_DIRECT to indicate the same dynamically via madvise. Similar to O_DIRECT, MAP_DIRECT would be an application hint to avoid / minimiize page cache usage, but no strict guarantee like what MAP_SYNC provides. * Resolve all "if (dax) goto fail;" patterns in the kernel. Outside of longterm-GUP (a topic in its own right) the projects here are XFS-reflink and XFS-realtime-device support. DAX+reflink effectively requires a given physical page to be mapped into two different inodes at different (page->index) offsets. The challenge is to support DAX-reflink without violating any existing application visible semantics, the operating assumption / strawman to debate is that experimental status is not blanket permission to go change existing semantics in backwards incompatible ways. * Deprecate, but not remove, the DAX mount option. Too many flows depend on the option so it will never go away, but the facility is too coarse. Provide an option to enable MAP_SYNC and more-likely-to-do-something-useful-MAP_DIRECT on a per-directory basis. The current proposal is to allow this property to only be toggled while the directory is empty to avoid the complications of racing page invalidation with new DAX mappings. Secondary projects, i.e. important but I would submit are not in the critical path to removing the "experimental" designation: * Filesystem-integrated badblock management. Hook up the media error notifications from libnvdimm to the filesystem to allow for operations like "list files with media errors" and "enumerate bad file offsets on a granulatiy smaller than a page". Another consideration along these lines is to integrate machine-check-handling and dynamic error notification into a filesystem interface. I've heard complaints that the sigaction() based mechanism to receive BUS_MCEERR_* information, while sufficient for the "System RAM" use case, is not precise enough for the "Persistent Memory / DAX" use case where errors are repairable and sub-page error information is useful. * Userfaultfd for file-backed mappings and DAX Ideally all the usual DAX, persistent memory, and GUP suspects could be in the room to discuss this: * Jan Kara * Dave Chinner * Christoph Hellwig * Jeff Moyer * Johannes Thumshirn * Matthew Wilcox * John Hubbard * Jérôme Glisse * MM folks for the reflink vs 'struct page' vs Xarray considerations ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LSF/MM TOPIC] The end of the DAX experiment 2019-02-06 21:12 [LSF/MM TOPIC] The end of the DAX experiment Dan Williams @ 2019-02-07 21:51 ` Kani, Toshi 2019-02-08 9:02 ` Johannes Thumshirn 2019-02-14 13:46 ` [Lsf-pc] " Michal Hocko 2019-02-17 20:29 ` Ric Wheeler 2 siblings, 1 reply; 16+ messages in thread From: Kani, Toshi @ 2019-02-07 21:51 UTC (permalink / raw) To: dan.j.williams, lsf-pc Cc: linux-ext4, linux-xfs, linux-nvdimm, linux-kernel, linux-fsdevel On Wed, 2019-02-06 at 13:12 -0800, Dan Williams wrote: > Before people get too excited this isn't a proposal to kill DAX. The > topic proposal is a discussion to resolve lingering open questions > that currently motivate ext4 and xfs to scream "EXPERIMENTAL" when the > current DAX facilities are enabled. The are 2 primary concerns to > resolve. Enumerate the remaining features/fixes, and identify a path > to implement it all without regressing any existing application use > cases. > > An enumeration of remaining projects follows, please expand this list > if I missed something: > > * "DAX" has no specific meaning by itself, users have 2 use cases for > "DAX" capabilities: userspace cache management via MAP_SYNC, and page > cache avoidance where the latter aspect of DAX has no current api to > discover / use it. The project is to supplement MAP_SYNC with a > MAP_DIRECT facility and MADV_SYNC / MADV_DIRECT to indicate the same > dynamically via madvise. Similar to O_DIRECT, MAP_DIRECT would be an > application hint to avoid / minimiize page cache usage, but no strict > guarantee like what MAP_SYNC provides. Agreed that basic dax programming model needs to be settled. > * Resolve all "if (dax) goto fail;" patterns in the kernel. Outside of > longterm-GUP (a topic in its own right) the projects here are > XFS-reflink and XFS-realtime-device support. DAX+reflink effectively > requires a given physical page to be mapped into two different inodes > at different (page->index) offsets. The challenge is to support > DAX-reflink without violating any existing application visible > semantics, the operating assumption / strawman to debate is that > experimental status is not blanket permission to go change existing > semantics in backwards incompatible ways. I do not think "if (dax) goto fail;" is actually bad. It helps users to immediately realize that dax may not be enabled and they have a setup issue. I also think that FS-specific enhancements like DAX-reflink=1 support should be managed separately and should be possible after EXPERIMENTAL is removed from 'mount -o dax'. That is, removing EXPERIMENTAL for DAX-reflink=0 should not require support of DAX- reflink=1. I've also had multiple users complained about 'mount -o dax' succeeded by falling back to non-dax despite of their intent. Such users wasted many hours without knowing their setup error / current restrictions. The next think they always ask is how to check if dax is enabled in such case, and they are not happy about limited interfaces (ex. look for mount entry in /proc/mounts), either. > * Deprecate, but not remove, the DAX mount option. Too many flows > depend on the option so it will never go away, but the facility is too > coarse. Provide an option to enable MAP_SYNC and > more-likely-to-do-something-useful-MAP_DIRECT on a per-directory > basis. The current proposal is to allow this property to only be > toggled while the directory is empty to avoid the complications of > racing page invalidation with new DAX mappings. Same as above. Such enhancement should be possible after EXPERIMENTAL is removed from 'mount -o dax'. IOW, a separate EXPERIMENTAL message can be shown when user requests per-directory dax. Thanks, -Toshi ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LSF/MM TOPIC] The end of the DAX experiment 2019-02-07 21:51 ` Kani, Toshi @ 2019-02-08 9:02 ` Johannes Thumshirn 0 siblings, 0 replies; 16+ messages in thread From: Johannes Thumshirn @ 2019-02-08 9:02 UTC (permalink / raw) To: Kani, Toshi Cc: dan.j.williams, lsf-pc, linux-xfs, linux-fsdevel, linux-ext4, linux-kernel, linux-nvdimm On 07/02/2019 22:51, Kani, Toshi wrote: > I've also had multiple users complained about 'mount -o dax' succeeded > by falling back to non-dax despite of their intent. Such users wasted > many hours without knowing their setup error / current restrictions. > The next think they always ask is how to check if dax is enabled in such > case, and they are not happy about limited interfaces (ex. look for > mount entry in /proc/mounts), either. Exactly, there's a need for admins and user-space applications to check if they can use DAX or not. See also [1] for reference. > Same as above. Such enhancement should be possible after EXPERIMENTAL > is removed from 'mount -o dax'. IOW, a separate EXPERIMENTAL message > can be shown when user requests per-directory dax. Just as a data point here, in our Kernels we've removed the EXPERIMENTAL warning for DAX on XFS (but not EXT4) and we are supporting it for some limited use-cases. [1] https://lore.kernel.org/linux-fsdevel/20181002100531.GC4135@quack2.suse.cz/ Byte, Johannes -- Johannes Thumshirn SUSE Labs Filesystems jthumshirn@suse.de +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850 ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] The end of the DAX experiment 2019-02-06 21:12 [LSF/MM TOPIC] The end of the DAX experiment Dan Williams 2019-02-07 21:51 ` Kani, Toshi @ 2019-02-14 13:46 ` Michal Hocko 2019-02-14 18:25 ` Dan Williams 2019-02-17 20:29 ` Ric Wheeler 2 siblings, 1 reply; 16+ messages in thread From: Michal Hocko @ 2019-02-14 13:46 UTC (permalink / raw) To: Dan Williams Cc: lsf-pc, linux-xfs, linux-fsdevel, linux-ext4, Linux Kernel Mailing List, linux-nvdimm On Wed 06-02-19 13:12:59, Dan Williams wrote: [...] > * Userfaultfd for file-backed mappings and DAX I assume that other topics are meant to be FS track but this one is MM, right? -- Michal Hocko SUSE Labs ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] The end of the DAX experiment 2019-02-14 13:46 ` [Lsf-pc] " Michal Hocko @ 2019-02-14 18:25 ` Dan Williams 2019-02-14 19:10 ` Jerome Glisse 2019-12-09 16:20 ` Boaz Harrosh 0 siblings, 2 replies; 16+ messages in thread From: Dan Williams @ 2019-02-14 18:25 UTC (permalink / raw) To: Michal Hocko Cc: lsf-pc, linux-xfs, linux-fsdevel, linux-ext4, Linux Kernel Mailing List, linux-nvdimm On Thu, Feb 14, 2019 at 5:46 AM Michal Hocko <mhocko@kernel.org> wrote: > > On Wed 06-02-19 13:12:59, Dan Williams wrote: > [...] > > * Userfaultfd for file-backed mappings and DAX > > I assume that other topics are meant to be FS track but this one is MM, > right? Yes, but I think it is the lowest priority of all the noted sub-topics in this proposal. The DAX-reflink discussion, where a given physical-page may need to be mapped into multiple inodes at different offsets, might be more fruitful to have as a joint discussion with MM. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] The end of the DAX experiment 2019-02-14 18:25 ` Dan Williams @ 2019-02-14 19:10 ` Jerome Glisse 2019-02-14 19:31 ` Dan Williams 2019-12-09 16:20 ` Boaz Harrosh 1 sibling, 1 reply; 16+ messages in thread From: Jerome Glisse @ 2019-02-14 19:10 UTC (permalink / raw) To: Dan Williams Cc: Michal Hocko, lsf-pc, linux-xfs, linux-fsdevel, linux-ext4, Linux Kernel Mailing List, linux-nvdimm On Thu, Feb 14, 2019 at 10:25:07AM -0800, Dan Williams wrote: > On Thu, Feb 14, 2019 at 5:46 AM Michal Hocko <mhocko@kernel.org> wrote: > > > > On Wed 06-02-19 13:12:59, Dan Williams wrote: > > [...] > > > * Userfaultfd for file-backed mappings and DAX > > > > I assume that other topics are meant to be FS track but this one is MM, > > right? > > Yes, but I think it is the lowest priority of all the noted sub-topics > in this proposal. The DAX-reflink discussion, where a given > physical-page may need to be mapped into multiple inodes at different > offsets, might be more fruitful to have as a joint discussion with MM. Note that my generic page write protection work can be use for that ie having a single page correspond to multiple different mapping with also different offset within each mapping. While in my patchset i only solve the mapping aliasing issue, the index can be solve in much the same way because same thinking apply. Namely that when you work on a file you know the mapping and file offset and thus the index and when you work on the vma you know the mapping and offset within the vma which translate to offset within the file. They are only few places that do not have the informations available and those do not care about it. I am just again working on my struct page mapping patchset as well as the generic page write protection that sits on top. I hope to be able to post the v2 in couple weeks. You can always look at my posting last year to see more details. Cheers, Jérôme ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] The end of the DAX experiment 2019-02-14 19:10 ` Jerome Glisse @ 2019-02-14 19:31 ` Dan Williams 2019-02-14 20:08 ` Matthew Wilcox 0 siblings, 1 reply; 16+ messages in thread From: Dan Williams @ 2019-02-14 19:31 UTC (permalink / raw) To: Jerome Glisse Cc: Michal Hocko, lsf-pc, linux-xfs, linux-fsdevel, linux-ext4, Linux Kernel Mailing List, linux-nvdimm On Thu, Feb 14, 2019 at 11:10 AM Jerome Glisse <jglisse@redhat.com> wrote: > > On Thu, Feb 14, 2019 at 10:25:07AM -0800, Dan Williams wrote: > > On Thu, Feb 14, 2019 at 5:46 AM Michal Hocko <mhocko@kernel.org> wrote: > > > > > > On Wed 06-02-19 13:12:59, Dan Williams wrote: > > > [...] > > > > * Userfaultfd for file-backed mappings and DAX > > > > > > I assume that other topics are meant to be FS track but this one is MM, > > > right? > > > > Yes, but I think it is the lowest priority of all the noted sub-topics > > in this proposal. The DAX-reflink discussion, where a given > > physical-page may need to be mapped into multiple inodes at different > > offsets, might be more fruitful to have as a joint discussion with MM. > > Note that my generic page write protection work can be use for that ie > having a single page correspond to multiple different mapping with also > different offset within each mapping. While in my patchset i only solve > the mapping aliasing issue, the index can be solve in much the same way > because same thinking apply. Namely that when you work on a file you > know the mapping and file offset and thus the index and when you work on > the vma you know the mapping and offset within the vma which translate > to offset within the file. They are only few places that do not have the > informations available and those do not care about it. > > I am just again working on my struct page mapping patchset as well as > the generic page write protection that sits on top. I hope to be able > to post the v2 in couple weeks. You can always look at my posting last > year to see more details. Yes, I have that in mind as one of the contenders. However, it's not clear to me that its a suitable fit for filesystem-reflink. Others have floated the 'page proxy' idea, so it would be good to discuss the merits of the general approaches. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] The end of the DAX experiment 2019-02-14 19:31 ` Dan Williams @ 2019-02-14 20:08 ` Matthew Wilcox 2019-02-14 20:20 ` Dan Williams 0 siblings, 1 reply; 16+ messages in thread From: Matthew Wilcox @ 2019-02-14 20:08 UTC (permalink / raw) To: Dan Williams Cc: Jerome Glisse, Michal Hocko, lsf-pc, linux-xfs, linux-fsdevel, linux-ext4, Linux Kernel Mailing List, linux-nvdimm On Thu, Feb 14, 2019 at 11:31:24AM -0800, Dan Williams wrote: > On Thu, Feb 14, 2019 at 11:10 AM Jerome Glisse <jglisse@redhat.com> wrote: > > I am just again working on my struct page mapping patchset as well as > > the generic page write protection that sits on top. I hope to be able > > to post the v2 in couple weeks. You can always look at my posting last > > year to see more details. > > Yes, I have that in mind as one of the contenders. However, it's not > clear to me that its a suitable fit for filesystem-reflink. Others > have floated the 'page proxy' idea, so it would be good to discuss the > merits of the general approaches. ... and my preferred option of putting pfn entries in the page cache. Or is that what you meant by "page proxy"? ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] The end of the DAX experiment 2019-02-14 20:08 ` Matthew Wilcox @ 2019-02-14 20:20 ` Dan Williams 2019-02-14 20:34 ` Jerome Glisse ` (2 more replies) 0 siblings, 3 replies; 16+ messages in thread From: Dan Williams @ 2019-02-14 20:20 UTC (permalink / raw) To: Matthew Wilcox Cc: Jerome Glisse, Michal Hocko, lsf-pc, linux-xfs, linux-fsdevel, linux-ext4, Linux Kernel Mailing List, linux-nvdimm On Thu, Feb 14, 2019 at 12:09 PM Matthew Wilcox <willy@infradead.org> wrote: > > On Thu, Feb 14, 2019 at 11:31:24AM -0800, Dan Williams wrote: > > On Thu, Feb 14, 2019 at 11:10 AM Jerome Glisse <jglisse@redhat.com> wrote: > > > I am just again working on my struct page mapping patchset as well as > > > the generic page write protection that sits on top. I hope to be able > > > to post the v2 in couple weeks. You can always look at my posting last > > > year to see more details. > > > > Yes, I have that in mind as one of the contenders. However, it's not > > clear to me that its a suitable fit for filesystem-reflink. Others > > have floated the 'page proxy' idea, so it would be good to discuss the > > merits of the general approaches. > > ... and my preferred option of putting pfn entries in the page cache. Another option to include the discussion. > Or is that what you meant by "page proxy"? Page proxy would be an object that a filesystem could allocate to point back to a single physical 'struct page *'. The proxy would contain an override for page->index. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] The end of the DAX experiment 2019-02-14 20:20 ` Dan Williams @ 2019-02-14 20:34 ` Jerome Glisse 2019-02-16 5:37 ` Dave Chinner 2019-06-05 16:22 ` Goldwyn Rodrigues 2 siblings, 0 replies; 16+ messages in thread From: Jerome Glisse @ 2019-02-14 20:34 UTC (permalink / raw) To: Dan Williams Cc: Matthew Wilcox, Michal Hocko, lsf-pc, linux-xfs, linux-fsdevel, linux-ext4, Linux Kernel Mailing List, linux-nvdimm On Thu, Feb 14, 2019 at 12:20:11PM -0800, Dan Williams wrote: > On Thu, Feb 14, 2019 at 12:09 PM Matthew Wilcox <willy@infradead.org> wrote: > > > > On Thu, Feb 14, 2019 at 11:31:24AM -0800, Dan Williams wrote: > > > On Thu, Feb 14, 2019 at 11:10 AM Jerome Glisse <jglisse@redhat.com> wrote: > > > > I am just again working on my struct page mapping patchset as well as > > > > the generic page write protection that sits on top. I hope to be able > > > > to post the v2 in couple weeks. You can always look at my posting last > > > > year to see more details. > > > > > > Yes, I have that in mind as one of the contenders. However, it's not > > > clear to me that its a suitable fit for filesystem-reflink. Others > > > have floated the 'page proxy' idea, so it would be good to discuss the > > > merits of the general approaches. > > > > ... and my preferred option of putting pfn entries in the page cache. > > Another option to include the discussion. > > > Or is that what you meant by "page proxy"? > > Page proxy would be an object that a filesystem could allocate to > point back to a single physical 'struct page *'. The proxy would > contain an override for page->index. Note that generic page write protection has such object, kind of like stable_node in KSM. You overwritte page->mapping to point to this generic struct which has a pointer to set of callback so that whatever is protecting the page can offer API to break protection (break sharing here). So instead of having struct proxy_page -> struct page you would have the reverse: struct page -> struct proxy and so you do not have to change much in all the file system beside removing the reliance on page->mapping which is what most of my patches are about. Cheers, Jérôme ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] The end of the DAX experiment 2019-02-14 20:20 ` Dan Williams 2019-02-14 20:34 ` Jerome Glisse @ 2019-02-16 5:37 ` Dave Chinner 2019-06-05 16:22 ` Goldwyn Rodrigues 2 siblings, 0 replies; 16+ messages in thread From: Dave Chinner @ 2019-02-16 5:37 UTC (permalink / raw) To: Dan Williams Cc: Matthew Wilcox, Jerome Glisse, Michal Hocko, lsf-pc, linux-xfs, linux-fsdevel, linux-ext4, Linux Kernel Mailing List, linux-nvdimm On Thu, Feb 14, 2019 at 12:20:11PM -0800, Dan Williams wrote: > On Thu, Feb 14, 2019 at 12:09 PM Matthew Wilcox <willy@infradead.org> wrote: > > > > On Thu, Feb 14, 2019 at 11:31:24AM -0800, Dan Williams wrote: > > > On Thu, Feb 14, 2019 at 11:10 AM Jerome Glisse <jglisse@redhat.com> wrote: > > > > I am just again working on my struct page mapping patchset as well as > > > > the generic page write protection that sits on top. I hope to be able > > > > to post the v2 in couple weeks. You can always look at my posting last > > > > year to see more details. > > > > > > Yes, I have that in mind as one of the contenders. However, it's not > > > clear to me that its a suitable fit for filesystem-reflink. Others > > > have floated the 'page proxy' idea, so it would be good to discuss the > > > merits of the general approaches. > > > > ... and my preferred option of putting pfn entries in the page cache. > > Another option to include the discussion. > > > Or is that what you meant by "page proxy"? > > Page proxy would be an object that a filesystem could allocate to > point back to a single physical 'struct page *'. The proxy would > contain an override for page->index. Needs to override page->mapping, potentially the page flags, what happens when someone takes a gup reference to the proxy structure, etc? Besides, didn't we discuss all this last year? how about we start with a summary of all the options considered last year, the pros and cons, etc, and then go from there? Regurgitating everything we've already talked about last year doesn't seem particularly useful to me... Cheers, Dave. -- Dave Chinner david@fromorbit.com ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] The end of the DAX experiment 2019-02-14 20:20 ` Dan Williams 2019-02-14 20:34 ` Jerome Glisse 2019-02-16 5:37 ` Dave Chinner @ 2019-06-05 16:22 ` Goldwyn Rodrigues 2019-06-07 6:00 ` Dan Williams 2 siblings, 1 reply; 16+ messages in thread From: Goldwyn Rodrigues @ 2019-06-05 16:22 UTC (permalink / raw) To: Dan Williams Cc: Matthew Wilcox, Jerome Glisse, Michal Hocko, lsf-pc, linux-xfs, linux-fsdevel, linux-ext4, Linux Kernel Mailing List, linux-nvdimm Hi Dan/Jerome, On 12:20 14/02, Dan Williams wrote: > On Thu, Feb 14, 2019 at 12:09 PM Matthew Wilcox <willy@infradead.org> wrote: > > > > On Thu, Feb 14, 2019 at 11:31:24AM -0800, Dan Williams wrote: > > > On Thu, Feb 14, 2019 at 11:10 AM Jerome Glisse <jglisse@redhat.com> wrote: > > > > I am just again working on my struct page mapping patchset as well as > > > > the generic page write protection that sits on top. I hope to be able > > > > to post the v2 in couple weeks. You can always look at my posting last > > > > year to see more details. > > > > > > Yes, I have that in mind as one of the contenders. However, it's not > > > clear to me that its a suitable fit for filesystem-reflink. Others > > > have floated the 'page proxy' idea, so it would be good to discuss the > > > merits of the general approaches. > > > > ... and my preferred option of putting pfn entries in the page cache. > > Another option to include the discussion. > > > Or is that what you meant by "page proxy"? > > Page proxy would be an object that a filesystem could allocate to > point back to a single physical 'struct page *'. The proxy would > contain an override for page->index. Was there any outcome on this and its implementation? I am specifically interested in this for DAX support on btrfs/CoW: The TODO comment on top of dax_associate_entry() :) If there are patches/git tree I could use to base my patches on, it would be nice. -- Goldwyn ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] The end of the DAX experiment 2019-06-05 16:22 ` Goldwyn Rodrigues @ 2019-06-07 6:00 ` Dan Williams 0 siblings, 0 replies; 16+ messages in thread From: Dan Williams @ 2019-06-07 6:00 UTC (permalink / raw) To: Goldwyn Rodrigues Cc: Matthew Wilcox, Jerome Glisse, Michal Hocko, lsf-pc, linux-xfs, linux-fsdevel, linux-ext4, Linux Kernel Mailing List, linux-nvdimm On Wed, Jun 5, 2019 at 9:22 AM Goldwyn Rodrigues <rgoldwyn@suse.de> wrote: > > Hi Dan/Jerome, > > On 12:20 14/02, Dan Williams wrote: > > On Thu, Feb 14, 2019 at 12:09 PM Matthew Wilcox <willy@infradead.org> wrote: > > > > > > On Thu, Feb 14, 2019 at 11:31:24AM -0800, Dan Williams wrote: > > > > On Thu, Feb 14, 2019 at 11:10 AM Jerome Glisse <jglisse@redhat.com> wrote: > > > > > I am just again working on my struct page mapping patchset as well as > > > > > the generic page write protection that sits on top. I hope to be able > > > > > to post the v2 in couple weeks. You can always look at my posting last > > > > > year to see more details. > > > > > > > > Yes, I have that in mind as one of the contenders. However, it's not > > > > clear to me that its a suitable fit for filesystem-reflink. Others > > > > have floated the 'page proxy' idea, so it would be good to discuss the > > > > merits of the general approaches. > > > > > > ... and my preferred option of putting pfn entries in the page cache. > > > > Another option to include the discussion. > > > > > Or is that what you meant by "page proxy"? > > > > Page proxy would be an object that a filesystem could allocate to > > point back to a single physical 'struct page *'. The proxy would > > contain an override for page->index. > > Was there any outcome on this and its implementation? I am specifically > interested in this for DAX support on btrfs/CoW: The TODO comment on > top of dax_associate_entry() :) > > If there are patches/git tree I could use to base my patches on, it would > be nice. Half joking, but I was hoping that by the time I had circled back to finally reviewing the brtfs dax patches that a solution to this problem would be waiting. We spent more time on other DAX topics. I recall that Jerome and I were mutually skeptical of each others approaches at first glance, but we did not get into the details. So, I think it will be a matter of getting the idea coded up. I'm finally coming out from under a pile of mm work so I should have some time this cycle to push the page proxy idea forward. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Lsf-pc] [LSF/MM TOPIC] The end of the DAX experiment 2019-02-14 18:25 ` Dan Williams 2019-02-14 19:10 ` Jerome Glisse @ 2019-12-09 16:20 ` Boaz Harrosh 1 sibling, 0 replies; 16+ messages in thread From: Boaz Harrosh @ 2019-12-09 16:20 UTC (permalink / raw) To: Dan Williams, Michal Hocko Cc: lsf-pc, linux-xfs, linux-fsdevel, linux-ext4, Linux Kernel Mailing List, linux-nvdimm On 14/02/2019 20:25, Dan Williams wrote: > On Thu, Feb 14, 2019 at 5:46 AM Michal Hocko <mhocko@kernel.org> wrote: >> >> On Wed 06-02-19 13:12:59, Dan Williams wrote: >> [...] >>> * Userfaultfd for file-backed mappings and DAX >> >> I assume that other topics are meant to be FS track but this one is MM, >> right? > > Yes, but I think it is the lowest priority of all the noted sub-topics > in this proposal. The DAX-reflink discussion, where a given > physical-page may need to be mapped into multiple inodes at different > offsets, might be more fruitful to have as a joint discussion with MM. > This topic is very interesting to me. In current ZUFS implementation we support this option for a long time. IE: Map same pte_t into different indexes of the same file-mappings as well as in vma(s) of different files, at different indexes. Including invalidation of mapping of a pwrite into such a shared page. (A write to a shared block will allocate a new block for writing) This effort off-course involves the participation of the FileSystem to give a list of files and indexes for map_unmapping(). I can explain if you want how we did this. Cheers Boaz ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LSF/MM TOPIC] The end of the DAX experiment 2019-02-06 21:12 [LSF/MM TOPIC] The end of the DAX experiment Dan Williams 2019-02-07 21:51 ` Kani, Toshi 2019-02-14 13:46 ` [Lsf-pc] " Michal Hocko @ 2019-02-17 20:29 ` Ric Wheeler 2019-02-18 18:17 ` Dan Williams 2 siblings, 1 reply; 16+ messages in thread From: Ric Wheeler @ 2019-02-17 20:29 UTC (permalink / raw) To: Dan Williams, lsf-pc Cc: linux-nvdimm, linux-xfs, Linux Kernel Mailing List, linux-fsdevel, linux-ext4 On 2/6/19 4:12 PM, Dan Williams wrote: > Before people get too excited this isn't a proposal to kill DAX. The > topic proposal is a discussion to resolve lingering open questions > that currently motivate ext4 and xfs to scream "EXPERIMENTAL" when the > current DAX facilities are enabled. The are 2 primary concerns to > resolve. Enumerate the remaining features/fixes, and identify a path > to implement it all without regressing any existing application use > cases. > > An enumeration of remaining projects follows, please expand this list > if I missed something: > > * "DAX" has no specific meaning by itself, users have 2 use cases for > "DAX" capabilities: userspace cache management via MAP_SYNC, and page > cache avoidance where the latter aspect of DAX has no current api to > discover / use it. The project is to supplement MAP_SYNC with a > MAP_DIRECT facility and MADV_SYNC / MADV_DIRECT to indicate the same > dynamically via madvise. Similar to O_DIRECT, MAP_DIRECT would be an > application hint to avoid / minimiize page cache usage, but no strict > guarantee like what MAP_SYNC provides. Sounds like a great topic to me. Having just gone through a new round of USENIX paper reviews, it is interesting to see how many academic systems are being pitched in this space (and most of them don't mention the kernel based xfs/ext4 with dax). Regards, Ric > > * Resolve all "if (dax) goto fail;" patterns in the kernel. Outside of > longterm-GUP (a topic in its own right) the projects here are > XFS-reflink and XFS-realtime-device support. DAX+reflink effectively > requires a given physical page to be mapped into two different inodes > at different (page->index) offsets. The challenge is to support > DAX-reflink without violating any existing application visible > semantics, the operating assumption / strawman to debate is that > experimental status is not blanket permission to go change existing > semantics in backwards incompatible ways. > > * Deprecate, but not remove, the DAX mount option. Too many flows > depend on the option so it will never go away, but the facility is too > coarse. Provide an option to enable MAP_SYNC and > more-likely-to-do-something-useful-MAP_DIRECT on a per-directory > basis. The current proposal is to allow this property to only be > toggled while the directory is empty to avoid the complications of > racing page invalidation with new DAX mappings. > > > Secondary projects, i.e. important but I would submit are not in the > critical path to removing the "experimental" designation: > * Filesystem-integrated badblock management. Hook up the media error > notifications from libnvdimm to the filesystem to allow for operations > like "list files with media errors" and "enumerate bad file offsets on > a granulatiy smaller than a page". Another consideration along these > lines is to integrate machine-check-handling and dynamic error > notification into a filesystem interface. I've heard complaints that > the sigaction() based mechanism to receive BUS_MCEERR_* information, > while sufficient for the "System RAM" use case, is not precise enough > for the "Persistent Memory / DAX" use case where errors are repairable > and sub-page error information is useful. > > * Userfaultfd for file-backed mappings and DAX > > > Ideally all the usual DAX, persistent memory, and GUP suspects could > be in the room to discuss this: > * Jan Kara > * Dave Chinner > * Christoph Hellwig > * Jeff Moyer > * Johannes Thumshirn > * Matthew Wilcox > * John Hubbard > * Jérôme Glisse > * MM folks for the reflink vs 'struct page' vs Xarray considerations ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LSF/MM TOPIC] The end of the DAX experiment 2019-02-17 20:29 ` Ric Wheeler @ 2019-02-18 18:17 ` Dan Williams 0 siblings, 0 replies; 16+ messages in thread From: Dan Williams @ 2019-02-18 18:17 UTC (permalink / raw) To: Ric Wheeler Cc: lsf-pc, linux-nvdimm, linux-xfs, Linux Kernel Mailing List, linux-fsdevel, linux-ext4 On Sun, Feb 17, 2019 at 12:29 PM Ric Wheeler <ricwheeler@gmail.com> wrote: > > On 2/6/19 4:12 PM, Dan Williams wrote: > > Before people get too excited this isn't a proposal to kill DAX. The > > topic proposal is a discussion to resolve lingering open questions > > that currently motivate ext4 and xfs to scream "EXPERIMENTAL" when the > > current DAX facilities are enabled. The are 2 primary concerns to > > resolve. Enumerate the remaining features/fixes, and identify a path > > to implement it all without regressing any existing application use > > cases. > > > > An enumeration of remaining projects follows, please expand this list > > if I missed something: > > > > * "DAX" has no specific meaning by itself, users have 2 use cases for > > "DAX" capabilities: userspace cache management via MAP_SYNC, and page > > cache avoidance where the latter aspect of DAX has no current api to > > discover / use it. The project is to supplement MAP_SYNC with a > > MAP_DIRECT facility and MADV_SYNC / MADV_DIRECT to indicate the same > > dynamically via madvise. Similar to O_DIRECT, MAP_DIRECT would be an > > application hint to avoid / minimiize page cache usage, but no strict > > guarantee like what MAP_SYNC provides. > > > Sounds like a great topic to me. Having just gone through a new round of USENIX > paper reviews, it is interesting to see how many academic systems are being > pitched in this space (and most of them don't mention the kernel based xfs/ext4 > with dax). Makes sense, the current fsdax facility is chasing feature deficiencies relative to device-dax (longterm pin support) and nova (dax techniques for fs-metadata). ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2019-12-09 16:20 UTC | newest] Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-02-06 21:12 [LSF/MM TOPIC] The end of the DAX experiment Dan Williams 2019-02-07 21:51 ` Kani, Toshi 2019-02-08 9:02 ` Johannes Thumshirn 2019-02-14 13:46 ` [Lsf-pc] " Michal Hocko 2019-02-14 18:25 ` Dan Williams 2019-02-14 19:10 ` Jerome Glisse 2019-02-14 19:31 ` Dan Williams 2019-02-14 20:08 ` Matthew Wilcox 2019-02-14 20:20 ` Dan Williams 2019-02-14 20:34 ` Jerome Glisse 2019-02-16 5:37 ` Dave Chinner 2019-06-05 16:22 ` Goldwyn Rodrigues 2019-06-07 6:00 ` Dan Williams 2019-12-09 16:20 ` Boaz Harrosh 2019-02-17 20:29 ` Ric Wheeler 2019-02-18 18:17 ` Dan Williams
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).