[PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Toshi Kani]

When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
size.  This feature relies on both mmap virtual address and FS
block (i.e. physical address) to be aligned by the pmd page size.
Users can use mkfs options to specify FS to align block allocations.
However, aligning mmap address requires code changes to existing
applications for providing a pmd-aligned address to mmap().

For instance, fio with "ioengine=mmap" performs I/Os with mmap() [1].
It calls mmap() with a NULL address, which needs to be changed to
provide a pmd-aligned address for testing with DAX pmd mappings.
Changing all applications that call mmap() with NULL is undesirable.

This patch-set extends filesystems to align an mmap address for
a DAX file so that unmodified applications can use DAX pmd mappings.

[1]: https://github.com/axboe/fio/blob/master/engines/mmap.c

RESEND:
 - Rebased to 4.8.0-rc4, and drop blk as BLK_DEV_DAX was removed.

v4:
 - Use loff_t for offset and cast before shift (Jan Kara)
 - Remove redundant paranthesis (Jan Kara)
 - Allow integration with huge page cache support (Matthew Wilcox)
 - Prepare for PUD mapping support (Mike Kravetz, Matthew Wilcox)

v3:
 - Check overflow condition to offset + length. (Matthew Wilcox)
 - Remove indent by using gotos. (Matthew Wilcox)
 - Define dax_get_unmapped_area to NULL when CONFIG_FS_DAX is unset.
   (Matthew Wilcox)
 - Squash all filesystem patches together. (Matthew Wilcox)

v2:
 - Change filesystems to provide their get_unmapped_area().
   (Matthew Wilcox)
 - Add more description about the benefit. (Matthew Wilcox)

---
Toshi Kani (2):
 1/2 thp, dax: add thp_get_unmapped_area for pmd mappings
 2/2 ext2/4, xfs, blk: call thp_get_unmapped_area() for pmd mappings

---
 fs/ext2/file.c          |  1 +
 fs/ext4/file.c          |  1 +
 fs/xfs/xfs_file.c       |  1 +
 include/linux/huge_mm.h |  7 +++++++
 mm/huge_memory.c        | 43 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 53 insertions(+)
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

[PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Toshi Kani]

When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
size.  This feature relies on both mmap virtual address and FS
block (i.e. physical address) to be aligned by the pmd page size.
Users can use mkfs options to specify FS to align block allocations.
However, aligning mmap address requires code changes to existing
applications for providing a pmd-aligned address to mmap().

For instance, fio with "ioengine=mmap" performs I/Os with mmap() [1].
It calls mmap() with a NULL address, which needs to be changed to
provide a pmd-aligned address for testing with DAX pmd mappings.
Changing all applications that call mmap() with NULL is undesirable.

This patch-set extends filesystems to align an mmap address for
a DAX file so that unmodified applications can use DAX pmd mappings.

[1]: https://github.com/axboe/fio/blob/master/engines/mmap.c

RESEND:
 - Rebased to 4.8.0-rc4, and drop blk as BLK_DEV_DAX was removed.

v4:
 - Use loff_t for offset and cast before shift (Jan Kara)
 - Remove redundant paranthesis (Jan Kara)
 - Allow integration with huge page cache support (Matthew Wilcox)
 - Prepare for PUD mapping support (Mike Kravetz, Matthew Wilcox)

v3:
 - Check overflow condition to offset + length. (Matthew Wilcox)
 - Remove indent by using gotos. (Matthew Wilcox)
 - Define dax_get_unmapped_area to NULL when CONFIG_FS_DAX is unset.
   (Matthew Wilcox)
 - Squash all filesystem patches together. (Matthew Wilcox)

v2:
 - Change filesystems to provide their get_unmapped_area().
   (Matthew Wilcox)
 - Add more description about the benefit. (Matthew Wilcox)

---
Toshi Kani (2):
 1/2 thp, dax: add thp_get_unmapped_area for pmd mappings
 2/2 ext2/4, xfs, blk: call thp_get_unmapped_area() for pmd mappings

---
 fs/ext2/file.c          |  1 +
 fs/ext4/file.c          |  1 +
 fs/xfs/xfs_file.c       |  1 +
 include/linux/huge_mm.h |  7 +++++++
 mm/huge_memory.c        | 43 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 53 insertions(+)

[PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Toshi Kani]

When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
size.  This feature relies on both mmap virtual address and FS
block (i.e. physical address) to be aligned by the pmd page size.
Users can use mkfs options to specify FS to align block allocations.
However, aligning mmap address requires code changes to existing
applications for providing a pmd-aligned address to mmap().

For instance, fio with "ioengine=mmap" performs I/Os with mmap() [1].
It calls mmap() with a NULL address, which needs to be changed to
provide a pmd-aligned address for testing with DAX pmd mappings.
Changing all applications that call mmap() with NULL is undesirable.

This patch-set extends filesystems to align an mmap address for
a DAX file so that unmodified applications can use DAX pmd mappings.

[1]: https://github.com/axboe/fio/blob/master/engines/mmap.c

RESEND:
 - Rebased to 4.8.0-rc4, and drop blk as BLK_DEV_DAX was removed.

v4:
 - Use loff_t for offset and cast before shift (Jan Kara)
 - Remove redundant paranthesis (Jan Kara)
 - Allow integration with huge page cache support (Matthew Wilcox)
 - Prepare for PUD mapping support (Mike Kravetz, Matthew Wilcox)

v3:
 - Check overflow condition to offset + length. (Matthew Wilcox)
 - Remove indent by using gotos. (Matthew Wilcox)
 - Define dax_get_unmapped_area to NULL when CONFIG_FS_DAX is unset.
   (Matthew Wilcox)
 - Squash all filesystem patches together. (Matthew Wilcox)

v2:
 - Change filesystems to provide their get_unmapped_area().
   (Matthew Wilcox)
 - Add more description about the benefit. (Matthew Wilcox)

---
Toshi Kani (2):
 1/2 thp, dax: add thp_get_unmapped_area for pmd mappings
 2/2 ext2/4, xfs, blk: call thp_get_unmapped_area() for pmd mappings

---
 fs/ext2/file.c          |  1 +
 fs/ext4/file.c          |  1 +
 fs/xfs/xfs_file.c       |  1 +
 include/linux/huge_mm.h |  7 +++++++
 mm/huge_memory.c        | 43 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 53 insertions(+)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v4 RESEND 1/2] thp, dax: add thp_get_unmapped_area for pmd mappings

[Toshi Kani]

When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
size.  This feature relies on both mmap virtual address and FS
block (i.e. physical address) to be aligned by the pmd page size.
Users can use mkfs options to specify FS to align block allocations.
However, aligning mmap address requires code changes to existing
applications for providing a pmd-aligned address to mmap().

For instance, fio with "ioengine=mmap" performs I/Os with mmap() [1].
It calls mmap() with a NULL address, which needs to be changed to
provide a pmd-aligned address for testing with DAX pmd mappings.
Changing all applications that call mmap() with NULL is undesirable.

Add thp_get_unmapped_area(), which can be called by filesystem's
get_unmapped_area to align an mmap address by the pmd size for
a DAX file.  It calls the default handler, mm->get_unmapped_area(),
to find a range and then aligns it for a DAX file.

The patch is based on Matthew Wilcox's change that allows adding
support of the pud page size easily.

[1]: https://github.com/axboe/fio/blob/master/engines/mmap.c
Signed-off-by: Toshi Kani <toshi.kani@hpe.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Matthew Wilcox <mawilcox@microsoft.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Andreas Dilger <adilger.kernel@dilger.ca>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
---
 include/linux/huge_mm.h |    7 +++++++
 mm/huge_memory.c        |   43 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+)

diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
index 6f14de4..4fca526 100644
--- a/include/linux/huge_mm.h
+++ b/include/linux/huge_mm.h
@@ -87,6 +87,10 @@ extern bool is_vma_temporary_stack(struct vm_area_struct *vma);
 
 extern unsigned long transparent_hugepage_flags;
 
+extern unsigned long thp_get_unmapped_area(struct file *filp,
+		unsigned long addr, unsigned long len, unsigned long pgoff,
+		unsigned long flags);
+
 extern void prep_transhuge_page(struct page *page);
 extern void free_transhuge_page(struct page *page);
 
@@ -169,6 +173,9 @@ void put_huge_zero_page(void);
 static inline void prep_transhuge_page(struct page *page) {}
 
 #define transparent_hugepage_flags 0UL
+
+#define thp_get_unmapped_area	NULL
+
 static inline int
 split_huge_page_to_list(struct page *page, struct list_head *list)
 {
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 2db2112..883f0ee 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -469,6 +469,49 @@ void prep_transhuge_page(struct page *page)
 	set_compound_page_dtor(page, TRANSHUGE_PAGE_DTOR);
 }
 
+unsigned long __thp_get_unmapped_area(struct file *filp, unsigned long len,
+		loff_t off, unsigned long flags, unsigned long size)
+{
+	unsigned long addr;
+	loff_t off_end = off + len;
+	loff_t off_align = round_up(off, size);
+	unsigned long len_pad;
+
+	if (off_end <= off_align || (off_end - off_align) < size)
+		return 0;
+
+	len_pad = len + size;
+	if (len_pad < len || (off + len_pad) < off)
+		return 0;
+
+	addr = current->mm->get_unmapped_area(filp, 0, len_pad,
+					      off >> PAGE_SHIFT, flags);
+	if (IS_ERR_VALUE(addr))
+		return 0;
+
+	addr += (off - addr) & (size - 1);
+	return addr;
+}
+
+unsigned long thp_get_unmapped_area(struct file *filp, unsigned long addr,
+		unsigned long len, unsigned long pgoff, unsigned long flags)
+{
+	loff_t off = (loff_t)pgoff << PAGE_SHIFT;
+
+	if (addr)
+		goto out;
+	if (!IS_DAX(filp->f_mapping->host) || !IS_ENABLED(CONFIG_FS_DAX_PMD))
+		goto out;
+
+	addr = __thp_get_unmapped_area(filp, len, off, flags, PMD_SIZE);
+	if (addr)
+		return addr;
+
+ out:
+	return current->mm->get_unmapped_area(filp, addr, len, pgoff, flags);
+}
+EXPORT_SYMBOL_GPL(thp_get_unmapped_area);
+
 static int __do_huge_pmd_anonymous_page(struct fault_env *fe, struct page *page,
 		gfp_t gfp)
 {
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

[PATCH v4 RESEND 1/2] thp, dax: add thp_get_unmapped_area for pmd mappings

[Toshi Kani]

When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
size.  This feature relies on both mmap virtual address and FS
block (i.e. physical address) to be aligned by the pmd page size.
Users can use mkfs options to specify FS to align block allocations.
However, aligning mmap address requires code changes to existing
applications for providing a pmd-aligned address to mmap().

For instance, fio with "ioengine=mmap" performs I/Os with mmap() [1].
It calls mmap() with a NULL address, which needs to be changed to
provide a pmd-aligned address for testing with DAX pmd mappings.
Changing all applications that call mmap() with NULL is undesirable.

Add thp_get_unmapped_area(), which can be called by filesystem's
get_unmapped_area to align an mmap address by the pmd size for
a DAX file.  It calls the default handler, mm->get_unmapped_area(),
to find a range and then aligns it for a DAX file.

The patch is based on Matthew Wilcox's change that allows adding
support of the pud page size easily.

[1]: https://github.com/axboe/fio/blob/master/engines/mmap.c
Signed-off-by: Toshi Kani <toshi.kani@hpe.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Matthew Wilcox <mawilcox@microsoft.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Andreas Dilger <adilger.kernel@dilger.ca>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
---
 include/linux/huge_mm.h |    7 +++++++
 mm/huge_memory.c        |   43 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+)

diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
index 6f14de4..4fca526 100644
--- a/include/linux/huge_mm.h
+++ b/include/linux/huge_mm.h
@@ -87,6 +87,10 @@ extern bool is_vma_temporary_stack(struct vm_area_struct *vma);
 
 extern unsigned long transparent_hugepage_flags;
 
+extern unsigned long thp_get_unmapped_area(struct file *filp,
+		unsigned long addr, unsigned long len, unsigned long pgoff,
+		unsigned long flags);
+
 extern void prep_transhuge_page(struct page *page);
 extern void free_transhuge_page(struct page *page);
 
@@ -169,6 +173,9 @@ void put_huge_zero_page(void);
 static inline void prep_transhuge_page(struct page *page) {}
 
 #define transparent_hugepage_flags 0UL
+
+#define thp_get_unmapped_area	NULL
+
 static inline int
 split_huge_page_to_list(struct page *page, struct list_head *list)
 {
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 2db2112..883f0ee 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -469,6 +469,49 @@ void prep_transhuge_page(struct page *page)
 	set_compound_page_dtor(page, TRANSHUGE_PAGE_DTOR);
 }
 
+unsigned long __thp_get_unmapped_area(struct file *filp, unsigned long len,
+		loff_t off, unsigned long flags, unsigned long size)
+{
+	unsigned long addr;
+	loff_t off_end = off + len;
+	loff_t off_align = round_up(off, size);
+	unsigned long len_pad;
+
+	if (off_end <= off_align || (off_end - off_align) < size)
+		return 0;
+
+	len_pad = len + size;
+	if (len_pad < len || (off + len_pad) < off)
+		return 0;
+
+	addr = current->mm->get_unmapped_area(filp, 0, len_pad,
+					      off >> PAGE_SHIFT, flags);
+	if (IS_ERR_VALUE(addr))
+		return 0;
+
+	addr += (off - addr) & (size - 1);
+	return addr;
+}
+
+unsigned long thp_get_unmapped_area(struct file *filp, unsigned long addr,
+		unsigned long len, unsigned long pgoff, unsigned long flags)
+{
+	loff_t off = (loff_t)pgoff << PAGE_SHIFT;
+
+	if (addr)
+		goto out;
+	if (!IS_DAX(filp->f_mapping->host) || !IS_ENABLED(CONFIG_FS_DAX_PMD))
+		goto out;
+
+	addr = __thp_get_unmapped_area(filp, len, off, flags, PMD_SIZE);
+	if (addr)
+		return addr;
+
+ out:
+	return current->mm->get_unmapped_area(filp, addr, len, pgoff, flags);
+}
+EXPORT_SYMBOL_GPL(thp_get_unmapped_area);
+
 static int __do_huge_pmd_anonymous_page(struct fault_env *fe, struct page *page,
 		gfp_t gfp)
 {

[PATCH v4 RESEND 1/2] thp, dax: add thp_get_unmapped_area for pmd mappings

[Toshi Kani]

When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
size.  This feature relies on both mmap virtual address and FS
block (i.e. physical address) to be aligned by the pmd page size.
Users can use mkfs options to specify FS to align block allocations.
However, aligning mmap address requires code changes to existing
applications for providing a pmd-aligned address to mmap().

For instance, fio with "ioengine=mmap" performs I/Os with mmap() [1].
It calls mmap() with a NULL address, which needs to be changed to
provide a pmd-aligned address for testing with DAX pmd mappings.
Changing all applications that call mmap() with NULL is undesirable.

Add thp_get_unmapped_area(), which can be called by filesystem's
get_unmapped_area to align an mmap address by the pmd size for
a DAX file.  It calls the default handler, mm->get_unmapped_area(),
to find a range and then aligns it for a DAX file.

The patch is based on Matthew Wilcox's change that allows adding
support of the pud page size easily.

[1]: https://github.com/axboe/fio/blob/master/engines/mmap.c
Signed-off-by: Toshi Kani <toshi.kani@hpe.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Matthew Wilcox <mawilcox@microsoft.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Andreas Dilger <adilger.kernel@dilger.ca>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
---
 include/linux/huge_mm.h |    7 +++++++
 mm/huge_memory.c        |   43 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+)

diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
index 6f14de4..4fca526 100644
--- a/include/linux/huge_mm.h
+++ b/include/linux/huge_mm.h
@@ -87,6 +87,10 @@ extern bool is_vma_temporary_stack(struct vm_area_struct *vma);
 
 extern unsigned long transparent_hugepage_flags;
 
+extern unsigned long thp_get_unmapped_area(struct file *filp,
+		unsigned long addr, unsigned long len, unsigned long pgoff,
+		unsigned long flags);
+
 extern void prep_transhuge_page(struct page *page);
 extern void free_transhuge_page(struct page *page);
 
@@ -169,6 +173,9 @@ void put_huge_zero_page(void);
 static inline void prep_transhuge_page(struct page *page) {}
 
 #define transparent_hugepage_flags 0UL
+
+#define thp_get_unmapped_area	NULL
+
 static inline int
 split_huge_page_to_list(struct page *page, struct list_head *list)
 {
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 2db2112..883f0ee 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -469,6 +469,49 @@ void prep_transhuge_page(struct page *page)
 	set_compound_page_dtor(page, TRANSHUGE_PAGE_DTOR);
 }
 
+unsigned long __thp_get_unmapped_area(struct file *filp, unsigned long len,
+		loff_t off, unsigned long flags, unsigned long size)
+{
+	unsigned long addr;
+	loff_t off_end = off + len;
+	loff_t off_align = round_up(off, size);
+	unsigned long len_pad;
+
+	if (off_end <= off_align || (off_end - off_align) < size)
+		return 0;
+
+	len_pad = len + size;
+	if (len_pad < len || (off + len_pad) < off)
+		return 0;
+
+	addr = current->mm->get_unmapped_area(filp, 0, len_pad,
+					      off >> PAGE_SHIFT, flags);
+	if (IS_ERR_VALUE(addr))
+		return 0;
+
+	addr += (off - addr) & (size - 1);
+	return addr;
+}
+
+unsigned long thp_get_unmapped_area(struct file *filp, unsigned long addr,
+		unsigned long len, unsigned long pgoff, unsigned long flags)
+{
+	loff_t off = (loff_t)pgoff << PAGE_SHIFT;
+
+	if (addr)
+		goto out;
+	if (!IS_DAX(filp->f_mapping->host) || !IS_ENABLED(CONFIG_FS_DAX_PMD))
+		goto out;
+
+	addr = __thp_get_unmapped_area(filp, len, off, flags, PMD_SIZE);
+	if (addr)
+		return addr;
+
+ out:
+	return current->mm->get_unmapped_area(filp, addr, len, pgoff, flags);
+}
+EXPORT_SYMBOL_GPL(thp_get_unmapped_area);
+
 static int __do_huge_pmd_anonymous_page(struct fault_env *fe, struct page *page,
 		gfp_t gfp)
 {

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v4 RESEND 2/2] ext2/4, xfs: call thp_get_unmapped_area() for pmd mappings

[Toshi Kani]

To support DAX pmd mappings with unmodified applications,
filesystems need to align an mmap address by the pmd size.

Call thp_get_unmapped_area() from f_op->get_unmapped_area.

Note, there is no change in behavior for a non-DAX file.

Signed-off-by: Toshi Kani <toshi.kani@hpe.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Matthew Wilcox <mawilcox@microsoft.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Andreas Dilger <adilger.kernel@dilger.ca>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
---
 fs/ext2/file.c    |    1 +
 fs/ext4/file.c    |    1 +
 fs/xfs/xfs_file.c |    1 +
 3 files changed, 3 insertions(+)

diff --git a/fs/ext2/file.c b/fs/ext2/file.c
index 5efeefe..d86831c 100644
--- a/fs/ext2/file.c
+++ b/fs/ext2/file.c
@@ -172,6 +172,7 @@ const struct file_operations ext2_file_operations = {
 	.open		= dquot_file_open,
 	.release	= ext2_release_file,
 	.fsync		= ext2_fsync,
+	.get_unmapped_area = thp_get_unmapped_area,
 	.splice_read	= generic_file_splice_read,
 	.splice_write	= iter_file_splice_write,
 };
diff --git a/fs/ext4/file.c b/fs/ext4/file.c
index 261ac37..28f542b 100644
--- a/fs/ext4/file.c
+++ b/fs/ext4/file.c
@@ -703,6 +703,7 @@ const struct file_operations ext4_file_operations = {
 	.open		= ext4_file_open,
 	.release	= ext4_release_file,
 	.fsync		= ext4_sync_file,
+	.get_unmapped_area = thp_get_unmapped_area,
 	.splice_read	= generic_file_splice_read,
 	.splice_write	= iter_file_splice_write,
 	.fallocate	= ext4_fallocate,
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index e612a02..7f71bb8 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -1662,6 +1662,7 @@ const struct file_operations xfs_file_operations = {
 	.open		= xfs_file_open,
 	.release	= xfs_file_release,
 	.fsync		= xfs_file_fsync,
+	.get_unmapped_area = thp_get_unmapped_area,
 	.fallocate	= xfs_file_fallocate,
 };
 
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

[PATCH v4 RESEND 2/2] ext2/4, xfs: call thp_get_unmapped_area() for pmd mappings

[Toshi Kani]

To support DAX pmd mappings with unmodified applications,
filesystems need to align an mmap address by the pmd size.

Call thp_get_unmapped_area() from f_op->get_unmapped_area.

Note, there is no change in behavior for a non-DAX file.

Signed-off-by: Toshi Kani <toshi.kani@hpe.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Matthew Wilcox <mawilcox@microsoft.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Andreas Dilger <adilger.kernel@dilger.ca>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
---
 fs/ext2/file.c    |    1 +
 fs/ext4/file.c    |    1 +
 fs/xfs/xfs_file.c |    1 +
 3 files changed, 3 insertions(+)

diff --git a/fs/ext2/file.c b/fs/ext2/file.c
index 5efeefe..d86831c 100644
--- a/fs/ext2/file.c
+++ b/fs/ext2/file.c
@@ -172,6 +172,7 @@ const struct file_operations ext2_file_operations = {
 	.open		= dquot_file_open,
 	.release	= ext2_release_file,
 	.fsync		= ext2_fsync,
+	.get_unmapped_area = thp_get_unmapped_area,
 	.splice_read	= generic_file_splice_read,
 	.splice_write	= iter_file_splice_write,
 };
diff --git a/fs/ext4/file.c b/fs/ext4/file.c
index 261ac37..28f542b 100644
--- a/fs/ext4/file.c
+++ b/fs/ext4/file.c
@@ -703,6 +703,7 @@ const struct file_operations ext4_file_operations = {
 	.open		= ext4_file_open,
 	.release	= ext4_release_file,
 	.fsync		= ext4_sync_file,
+	.get_unmapped_area = thp_get_unmapped_area,
 	.splice_read	= generic_file_splice_read,
 	.splice_write	= iter_file_splice_write,
 	.fallocate	= ext4_fallocate,
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index e612a02..7f71bb8 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -1662,6 +1662,7 @@ const struct file_operations xfs_file_operations = {
 	.open		= xfs_file_open,
 	.release	= xfs_file_release,
 	.fsync		= xfs_file_fsync,
+	.get_unmapped_area = thp_get_unmapped_area,
 	.fallocate	= xfs_file_fallocate,
 };
 

[PATCH v4 RESEND 2/2] ext2/4, xfs: call thp_get_unmapped_area() for pmd mappings

[Toshi Kani]

To support DAX pmd mappings with unmodified applications,
filesystems need to align an mmap address by the pmd size.

Call thp_get_unmapped_area() from f_op->get_unmapped_area.

Note, there is no change in behavior for a non-DAX file.

Signed-off-by: Toshi Kani <toshi.kani@hpe.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Matthew Wilcox <mawilcox@microsoft.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Andreas Dilger <adilger.kernel@dilger.ca>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
---
 fs/ext2/file.c    |    1 +
 fs/ext4/file.c    |    1 +
 fs/xfs/xfs_file.c |    1 +
 3 files changed, 3 insertions(+)

diff --git a/fs/ext2/file.c b/fs/ext2/file.c
index 5efeefe..d86831c 100644
--- a/fs/ext2/file.c
+++ b/fs/ext2/file.c
@@ -172,6 +172,7 @@ const struct file_operations ext2_file_operations = {
 	.open		= dquot_file_open,
 	.release	= ext2_release_file,
 	.fsync		= ext2_fsync,
+	.get_unmapped_area = thp_get_unmapped_area,
 	.splice_read	= generic_file_splice_read,
 	.splice_write	= iter_file_splice_write,
 };
diff --git a/fs/ext4/file.c b/fs/ext4/file.c
index 261ac37..28f542b 100644
--- a/fs/ext4/file.c
+++ b/fs/ext4/file.c
@@ -703,6 +703,7 @@ const struct file_operations ext4_file_operations = {
 	.open		= ext4_file_open,
 	.release	= ext4_release_file,
 	.fsync		= ext4_sync_file,
+	.get_unmapped_area = thp_get_unmapped_area,
 	.splice_read	= generic_file_splice_read,
 	.splice_write	= iter_file_splice_write,
 	.fallocate	= ext4_fallocate,
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index e612a02..7f71bb8 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -1662,6 +1662,7 @@ const struct file_operations xfs_file_operations = {
 	.open		= xfs_file_open,
 	.release	= xfs_file_release,
 	.fsync		= xfs_file_fsync,
+	.get_unmapped_area = thp_get_unmapped_area,
 	.fallocate	= xfs_file_fallocate,
 };
 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v4 RESEND 1/2] thp, dax: add thp_get_unmapped_area for pmd mappings

[Dan Williams]

On Mon, Aug 29, 2016 at 12:11 PM, Toshi Kani <toshi.kani@hpe.com> wrote:
> When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> size.  This feature relies on both mmap virtual address and FS
> block (i.e. physical address) to be aligned by the pmd page size.
> Users can use mkfs options to specify FS to align block allocations.
> However, aligning mmap address requires code changes to existing
> applications for providing a pmd-aligned address to mmap().
>
> For instance, fio with "ioengine=mmap" performs I/Os with mmap() [1].
> It calls mmap() with a NULL address, which needs to be changed to
> provide a pmd-aligned address for testing with DAX pmd mappings.
> Changing all applications that call mmap() with NULL is undesirable.
>
> Add thp_get_unmapped_area(), which can be called by filesystem's
> get_unmapped_area to align an mmap address by the pmd size for
> a DAX file.  It calls the default handler, mm->get_unmapped_area(),
> to find a range and then aligns it for a DAX file.
>
> The patch is based on Matthew Wilcox's change that allows adding
> support of the pud page size easily.
>
> [1]: https://github.com/axboe/fio/blob/master/engines/mmap.c
> Signed-off-by: Toshi Kani <toshi.kani@hpe.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: Matthew Wilcox <mawilcox@microsoft.com>
> Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Cc: Dave Chinner <david@fromorbit.com>
> Cc: Jan Kara <jack@suse.cz>
> Cc: Theodore Ts'o <tytso@mit.edu>
> Cc: Andreas Dilger <adilger.kernel@dilger.ca>
> Cc: Mike Kravetz <mike.kravetz@oracle.com>
> ---

Reviewed-by: Dan Williams <dan.j.williams@intel.com>

...with one minor nit:


>  include/linux/huge_mm.h |    7 +++++++
>  mm/huge_memory.c        |   43 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 50 insertions(+)
>
> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> index 6f14de4..4fca526 100644
> --- a/include/linux/huge_mm.h
> +++ b/include/linux/huge_mm.h
> @@ -87,6 +87,10 @@ extern bool is_vma_temporary_stack(struct vm_area_struct *vma);
>
>  extern unsigned long transparent_hugepage_flags;
>
> +extern unsigned long thp_get_unmapped_area(struct file *filp,
> +               unsigned long addr, unsigned long len, unsigned long pgoff,
> +               unsigned long flags);
> +
>  extern void prep_transhuge_page(struct page *page);
>  extern void free_transhuge_page(struct page *page);
>
> @@ -169,6 +173,9 @@ void put_huge_zero_page(void);
>  static inline void prep_transhuge_page(struct page *page) {}
>
>  #define transparent_hugepage_flags 0UL
> +
> +#define thp_get_unmapped_area  NULL

Lets make this:

static inline unsigned long thp_get_unmapped_area(struct file *filp,
               unsigned long addr, unsigned long len, unsigned long pgoff,
               unsigned long flags)
{
    return 0;
}

...to get some type checking in the CONFIG_TRANSPARENT_HUGEPAGE=n case.
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH v4 RESEND 1/2] thp, dax: add thp_get_unmapped_area for pmd mappings

[Dan Williams]

On Mon, Aug 29, 2016 at 12:11 PM, Toshi Kani <toshi.kani@hpe.com> wrote:
> When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> size.  This feature relies on both mmap virtual address and FS
> block (i.e. physical address) to be aligned by the pmd page size.
> Users can use mkfs options to specify FS to align block allocations.
> However, aligning mmap address requires code changes to existing
> applications for providing a pmd-aligned address to mmap().
>
> For instance, fio with "ioengine=mmap" performs I/Os with mmap() [1].
> It calls mmap() with a NULL address, which needs to be changed to
> provide a pmd-aligned address for testing with DAX pmd mappings.
> Changing all applications that call mmap() with NULL is undesirable.
>
> Add thp_get_unmapped_area(), which can be called by filesystem's
> get_unmapped_area to align an mmap address by the pmd size for
> a DAX file.  It calls the default handler, mm->get_unmapped_area(),
> to find a range and then aligns it for a DAX file.
>
> The patch is based on Matthew Wilcox's change that allows adding
> support of the pud page size easily.
>
> [1]: https://github.com/axboe/fio/blob/master/engines/mmap.c
> Signed-off-by: Toshi Kani <toshi.kani@hpe.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: Matthew Wilcox <mawilcox@microsoft.com>
> Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Cc: Dave Chinner <david@fromorbit.com>
> Cc: Jan Kara <jack@suse.cz>
> Cc: Theodore Ts'o <tytso@mit.edu>
> Cc: Andreas Dilger <adilger.kernel@dilger.ca>
> Cc: Mike Kravetz <mike.kravetz@oracle.com>
> ---

Reviewed-by: Dan Williams <dan.j.williams@intel.com>

...with one minor nit:


>  include/linux/huge_mm.h |    7 +++++++
>  mm/huge_memory.c        |   43 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 50 insertions(+)
>
> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> index 6f14de4..4fca526 100644
> --- a/include/linux/huge_mm.h
> +++ b/include/linux/huge_mm.h
> @@ -87,6 +87,10 @@ extern bool is_vma_temporary_stack(struct vm_area_struct *vma);
>
>  extern unsigned long transparent_hugepage_flags;
>
> +extern unsigned long thp_get_unmapped_area(struct file *filp,
> +               unsigned long addr, unsigned long len, unsigned long pgoff,
> +               unsigned long flags);
> +
>  extern void prep_transhuge_page(struct page *page);
>  extern void free_transhuge_page(struct page *page);
>
> @@ -169,6 +173,9 @@ void put_huge_zero_page(void);
>  static inline void prep_transhuge_page(struct page *page) {}
>
>  #define transparent_hugepage_flags 0UL
> +
> +#define thp_get_unmapped_area  NULL

Lets make this:

static inline unsigned long thp_get_unmapped_area(struct file *filp,
               unsigned long addr, unsigned long len, unsigned long pgoff,
               unsigned long flags)
{
    return 0;
}

...to get some type checking in the CONFIG_TRANSPARENT_HUGEPAGE=n case.

Re: [PATCH v4 RESEND 1/2] thp, dax: add thp_get_unmapped_area for pmd mappings

[Dan Williams]

On Mon, Aug 29, 2016 at 12:11 PM, Toshi Kani <toshi.kani@hpe.com> wrote:
> When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> size.  This feature relies on both mmap virtual address and FS
> block (i.e. physical address) to be aligned by the pmd page size.
> Users can use mkfs options to specify FS to align block allocations.
> However, aligning mmap address requires code changes to existing
> applications for providing a pmd-aligned address to mmap().
>
> For instance, fio with "ioengine=mmap" performs I/Os with mmap() [1].
> It calls mmap() with a NULL address, which needs to be changed to
> provide a pmd-aligned address for testing with DAX pmd mappings.
> Changing all applications that call mmap() with NULL is undesirable.
>
> Add thp_get_unmapped_area(), which can be called by filesystem's
> get_unmapped_area to align an mmap address by the pmd size for
> a DAX file.  It calls the default handler, mm->get_unmapped_area(),
> to find a range and then aligns it for a DAX file.
>
> The patch is based on Matthew Wilcox's change that allows adding
> support of the pud page size easily.
>
> [1]: https://github.com/axboe/fio/blob/master/engines/mmap.c
> Signed-off-by: Toshi Kani <toshi.kani@hpe.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: Matthew Wilcox <mawilcox@microsoft.com>
> Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Cc: Dave Chinner <david@fromorbit.com>
> Cc: Jan Kara <jack@suse.cz>
> Cc: Theodore Ts'o <tytso@mit.edu>
> Cc: Andreas Dilger <adilger.kernel@dilger.ca>
> Cc: Mike Kravetz <mike.kravetz@oracle.com>
> ---

Reviewed-by: Dan Williams <dan.j.williams@intel.com>

...with one minor nit:


>  include/linux/huge_mm.h |    7 +++++++
>  mm/huge_memory.c        |   43 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 50 insertions(+)
>
> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> index 6f14de4..4fca526 100644
> --- a/include/linux/huge_mm.h
> +++ b/include/linux/huge_mm.h
> @@ -87,6 +87,10 @@ extern bool is_vma_temporary_stack(struct vm_area_struct *vma);
>
>  extern unsigned long transparent_hugepage_flags;
>
> +extern unsigned long thp_get_unmapped_area(struct file *filp,
> +               unsigned long addr, unsigned long len, unsigned long pgoff,
> +               unsigned long flags);
> +
>  extern void prep_transhuge_page(struct page *page);
>  extern void free_transhuge_page(struct page *page);
>
> @@ -169,6 +173,9 @@ void put_huge_zero_page(void);
>  static inline void prep_transhuge_page(struct page *page) {}
>
>  #define transparent_hugepage_flags 0UL
> +
> +#define thp_get_unmapped_area  NULL

Lets make this:

static inline unsigned long thp_get_unmapped_area(struct file *filp,
               unsigned long addr, unsigned long len, unsigned long pgoff,
               unsigned long flags)
{
    return 0;
}

...to get some type checking in the CONFIG_TRANSPARENT_HUGEPAGE=n case.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v4 RESEND 1/2] thp, dax: add thp_get_unmapped_area for pmd mappings

[Kani, Toshimitsu]

On Mon, 2016-08-29 at 12:34 -0700, Dan Williams wrote:
> On Mon, Aug 29, 2016 at 12:11 PM, Toshi Kani <toshi.kani@hpe.com>
> wrote:
> > 
> > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> > size.  This feature relies on both mmap virtual address and FS
> > block (i.e. physical address) to be aligned by the pmd page size.
> > Users can use mkfs options to specify FS to align block
> > allocations. However, aligning mmap address requires code changes
> > to existing applications for providing a pmd-aligned address to
> > mmap().
> > 
> > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
> > [1]. It calls mmap() with a NULL address, which needs to be changed
> > to provide a pmd-aligned address for testing with DAX pmd mappings.
> > Changing all applications that call mmap() with NULL is
> > undesirable.
> > 
> > Add thp_get_unmapped_area(), which can be called by filesystem's
> > get_unmapped_area to align an mmap address by the pmd size for
> > a DAX file.  It calls the default handler, mm->get_unmapped_area(),
> > to find a range and then aligns it for a DAX file.
> > 
> > The patch is based on Matthew Wilcox's change that allows adding
> > support of the pud page size easily.
 :
> 
> Reviewed-by: Dan Williams <dan.j.williams@intel.com>

Great!

> ...with one minor nit:
> 
> 
> > 
> >  include/linux/huge_mm.h |    7 +++++++
> >  mm/huge_memory.c        |   43
> > +++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 50 insertions(+)
> > 
> > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> > index 6f14de4..4fca526 100644
> > --- a/include/linux/huge_mm.h
> > +++ b/include/linux/huge_mm.h
> > @@ -87,6 +87,10 @@ extern bool is_vma_temporary_stack(struct
> > vm_area_struct *vma);
> > 
> >  extern unsigned long transparent_hugepage_flags;
> > 
> > +extern unsigned long thp_get_unmapped_area(struct file *filp,
> > +               unsigned long addr, unsigned long len, unsigned
> > long pgoff,
> > +               unsigned long flags);
> > +
> >  extern void prep_transhuge_page(struct page *page);
> >  extern void free_transhuge_page(struct page *page);
> > 
> > @@ -169,6 +173,9 @@ void put_huge_zero_page(void);
> >  static inline void prep_transhuge_page(struct page *page) {}
> > 
> >  #define transparent_hugepage_flags 0UL
> > +
> > +#define thp_get_unmapped_area  NULL
> 
> Lets make this:
> 
> static inline unsigned long thp_get_unmapped_area(struct file *filp,
>                unsigned long addr, unsigned long len, unsigned long
> pgoff,
>                unsigned long flags)
> {
>     return 0;
> }
> 
> ...to get some type checking in the CONFIG_TRANSPARENT_HUGEPAGE=n
> case.
> 

Per get_unmapped_area() in mm/mmap.c, I think we need to set it to NULL
when we do not override current->mm->get_unmapped_area.

Thanks!
-Toshi

_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH v4 RESEND 1/2] thp, dax: add thp_get_unmapped_area for pmd mappings

[Kani, Toshimitsu]

On Mon, 2016-08-29 at 12:34 -0700, Dan Williams wrote:
> On Mon, Aug 29, 2016 at 12:11 PM, Toshi Kani <toshi.kani@hpe.com>
> wrote:
> > 
> > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> > size.  This feature relies on both mmap virtual address and FS
> > block (i.e. physical address) to be aligned by the pmd page size.
> > Users can use mkfs options to specify FS to align block
> > allocations. However, aligning mmap address requires code changes
> > to existing applications for providing a pmd-aligned address to
> > mmap().
> > 
> > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
> > [1]. It calls mmap() with a NULL address, which needs to be changed
> > to provide a pmd-aligned address for testing with DAX pmd mappings.
> > Changing all applications that call mmap() with NULL is
> > undesirable.
> > 
> > Add thp_get_unmapped_area(), which can be called by filesystem's
> > get_unmapped_area to align an mmap address by the pmd size for
> > a DAX file.  It calls the default handler, mm->get_unmapped_area(),
> > to find a range and then aligns it for a DAX file.
> > 
> > The patch is based on Matthew Wilcox's change that allows adding
> > support of the pud page size easily.
 :
> 
> Reviewed-by: Dan Williams <dan.j.williams@intel.com>

Great!

> ...with one minor nit:
> 
> 
> > 
> >  include/linux/huge_mm.h |    7 +++++++
> >  mm/huge_memory.c        |   43
> > +++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 50 insertions(+)
> > 
> > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> > index 6f14de4..4fca526 100644
> > --- a/include/linux/huge_mm.h
> > +++ b/include/linux/huge_mm.h
> > @@ -87,6 +87,10 @@ extern bool is_vma_temporary_stack(struct
> > vm_area_struct *vma);
> > 
> >  extern unsigned long transparent_hugepage_flags;
> > 
> > +extern unsigned long thp_get_unmapped_area(struct file *filp,
> > +               unsigned long addr, unsigned long len, unsigned
> > long pgoff,
> > +               unsigned long flags);
> > +
> >  extern void prep_transhuge_page(struct page *page);
> >  extern void free_transhuge_page(struct page *page);
> > 
> > @@ -169,6 +173,9 @@ void put_huge_zero_page(void);
> >  static inline void prep_transhuge_page(struct page *page) {}
> > 
> >  #define transparent_hugepage_flags 0UL
> > +
> > +#define thp_get_unmapped_area  NULL
> 
> Lets make this:
> 
> static inline unsigned long thp_get_unmapped_area(struct file *filp,
>                unsigned long addr, unsigned long len, unsigned long
> pgoff,
>                unsigned long flags)
> {
>     return 0;
> }
> 
> ...to get some type checking in the CONFIG_TRANSPARENT_HUGEPAGE=n
> case.
> 

Per get_unmapped_area() in mm/mmap.c, I think we need to set it to NULL
when we do not override current->mm->get_unmapped_area.

Thanks!
-Toshi

Re: [PATCH v4 RESEND 1/2] thp, dax: add thp_get_unmapped_area for pmd mappings

[Kani, Toshimitsu]

On Mon, 2016-08-29 at 12:34 -0700, Dan Williams wrote:
> On Mon, Aug 29, 2016 at 12:11 PM, Toshi Kani <toshi.kani@hpe.com>
> wrote:
> > 
> > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> > size.  This feature relies on both mmap virtual address and FS
> > block (i.e. physical address) to be aligned by the pmd page size.
> > Users can use mkfs options to specify FS to align block
> > allocations. However, aligning mmap address requires code changes
> > to existing applications for providing a pmd-aligned address to
> > mmap().
> > 
> > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
> > [1]. It calls mmap() with a NULL address, which needs to be changed
> > to provide a pmd-aligned address for testing with DAX pmd mappings.
> > Changing all applications that call mmap() with NULL is
> > undesirable.
> > 
> > Add thp_get_unmapped_area(), which can be called by filesystem's
> > get_unmapped_area to align an mmap address by the pmd size for
> > a DAX file.  It calls the default handler, mm->get_unmapped_area(),
> > to find a range and then aligns it for a DAX file.
> > 
> > The patch is based on Matthew Wilcox's change that allows adding
> > support of the pud page size easily.
 :
> 
> Reviewed-by: Dan Williams <dan.j.williams@intel.com>

Great!

> ...with one minor nit:
> 
> 
> > 
> >  include/linux/huge_mm.h |    7 +++++++
> >  mm/huge_memory.c        |   43
> > +++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 50 insertions(+)
> > 
> > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> > index 6f14de4..4fca526 100644
> > --- a/include/linux/huge_mm.h
> > +++ b/include/linux/huge_mm.h
> > @@ -87,6 +87,10 @@ extern bool is_vma_temporary_stack(struct
> > vm_area_struct *vma);
> > 
> >  extern unsigned long transparent_hugepage_flags;
> > 
> > +extern unsigned long thp_get_unmapped_area(struct file *filp,
> > +               unsigned long addr, unsigned long len, unsigned
> > long pgoff,
> > +               unsigned long flags);
> > +
> >  extern void prep_transhuge_page(struct page *page);
> >  extern void free_transhuge_page(struct page *page);
> > 
> > @@ -169,6 +173,9 @@ void put_huge_zero_page(void);
> >  static inline void prep_transhuge_page(struct page *page) {}
> > 
> >  #define transparent_hugepage_flags 0UL
> > +
> > +#define thp_get_unmapped_area  NULL
> 
> Lets make this:
> 
> static inline unsigned long thp_get_unmapped_area(struct file *filp,
>                unsigned long addr, unsigned long len, unsigned long
> pgoff,
>                unsigned long flags)
> {
>     return 0;
> }
> 
> ...to get some type checking in the CONFIG_TRANSPARENT_HUGEPAGE=n
> case.
> 

Per get_unmapped_area() in mm/mmap.c, I think we need to set it to NULL
when we do not override current->mm->get_unmapped_area.

Thanks!
-Toshi

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kirill A. Shutemov]

On Mon, Aug 29, 2016 at 01:11:19PM -0600, Toshi Kani wrote:
> When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> size.  This feature relies on both mmap virtual address and FS
> block (i.e. physical address) to be aligned by the pmd page size.
> Users can use mkfs options to specify FS to align block allocations.
> However, aligning mmap address requires code changes to existing
> applications for providing a pmd-aligned address to mmap().
> 
> For instance, fio with "ioengine=mmap" performs I/Os with mmap() [1].
> It calls mmap() with a NULL address, which needs to be changed to
> provide a pmd-aligned address for testing with DAX pmd mappings.
> Changing all applications that call mmap() with NULL is undesirable.
> 
> This patch-set extends filesystems to align an mmap address for
> a DAX file so that unmodified applications can use DAX pmd mappings.

+Hugh

Can we get it used for shmem/tmpfs too?
I don't think we should duplicate essentially the same functionality in
multiple places.

-- 
 Kirill A. Shutemov
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kirill A. Shutemov]

On Mon, Aug 29, 2016 at 01:11:19PM -0600, Toshi Kani wrote:
> When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> size.  This feature relies on both mmap virtual address and FS
> block (i.e. physical address) to be aligned by the pmd page size.
> Users can use mkfs options to specify FS to align block allocations.
> However, aligning mmap address requires code changes to existing
> applications for providing a pmd-aligned address to mmap().
> 
> For instance, fio with "ioengine=mmap" performs I/Os with mmap() [1].
> It calls mmap() with a NULL address, which needs to be changed to
> provide a pmd-aligned address for testing with DAX pmd mappings.
> Changing all applications that call mmap() with NULL is undesirable.
> 
> This patch-set extends filesystems to align an mmap address for
> a DAX file so that unmodified applications can use DAX pmd mappings.

+Hugh

Can we get it used for shmem/tmpfs too?
I don't think we should duplicate essentially the same functionality in
multiple places.

-- 
 Kirill A. Shutemov

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kirill A. Shutemov]

On Mon, Aug 29, 2016 at 01:11:19PM -0600, Toshi Kani wrote:
> When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> size.  This feature relies on both mmap virtual address and FS
> block (i.e. physical address) to be aligned by the pmd page size.
> Users can use mkfs options to specify FS to align block allocations.
> However, aligning mmap address requires code changes to existing
> applications for providing a pmd-aligned address to mmap().
> 
> For instance, fio with "ioengine=mmap" performs I/Os with mmap() [1].
> It calls mmap() with a NULL address, which needs to be changed to
> provide a pmd-aligned address for testing with DAX pmd mappings.
> Changing all applications that call mmap() with NULL is undesirable.
> 
> This patch-set extends filesystems to align an mmap address for
> a DAX file so that unmodified applications can use DAX pmd mappings.

+Hugh

Can we get it used for shmem/tmpfs too?
I don't think we should duplicate essentially the same functionality in
multiple places.

-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v4 RESEND 1/2] thp, dax: add thp_get_unmapped_area for pmd mappings

[Dan Williams]

On Mon, Aug 29, 2016 at 1:44 PM, Kani, Toshimitsu <toshi.kani@hpe.com> wrote:
> On Mon, 2016-08-29 at 12:34 -0700, Dan Williams wrote:
>> On Mon, Aug 29, 2016 at 12:11 PM, Toshi Kani <toshi.kani@hpe.com>
>> wrote:
>> >
>> > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
>> > size.  This feature relies on both mmap virtual address and FS
>> > block (i.e. physical address) to be aligned by the pmd page size.
>> > Users can use mkfs options to specify FS to align block
>> > allocations. However, aligning mmap address requires code changes
>> > to existing applications for providing a pmd-aligned address to
>> > mmap().
>> >
>> > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
>> > [1]. It calls mmap() with a NULL address, which needs to be changed
>> > to provide a pmd-aligned address for testing with DAX pmd mappings.
>> > Changing all applications that call mmap() with NULL is
>> > undesirable.
>> >
>> > Add thp_get_unmapped_area(), which can be called by filesystem's
>> > get_unmapped_area to align an mmap address by the pmd size for
>> > a DAX file.  It calls the default handler, mm->get_unmapped_area(),
>> > to find a range and then aligns it for a DAX file.
>> >
>> > The patch is based on Matthew Wilcox's change that allows adding
>> > support of the pud page size easily.
>  :
>>
>> Reviewed-by: Dan Williams <dan.j.williams@intel.com>
>
> Great!
>
>> ...with one minor nit:
>>
>>
>> >
>> >  include/linux/huge_mm.h |    7 +++++++
>> >  mm/huge_memory.c        |   43
>> > +++++++++++++++++++++++++++++++++++++++++++
>> >  2 files changed, 50 insertions(+)
>> >
>> > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
>> > index 6f14de4..4fca526 100644
>> > --- a/include/linux/huge_mm.h
>> > +++ b/include/linux/huge_mm.h
>> > @@ -87,6 +87,10 @@ extern bool is_vma_temporary_stack(struct
>> > vm_area_struct *vma);
>> >
>> >  extern unsigned long transparent_hugepage_flags;
>> >
>> > +extern unsigned long thp_get_unmapped_area(struct file *filp,
>> > +               unsigned long addr, unsigned long len, unsigned
>> > long pgoff,
>> > +               unsigned long flags);
>> > +
>> >  extern void prep_transhuge_page(struct page *page);
>> >  extern void free_transhuge_page(struct page *page);
>> >
>> > @@ -169,6 +173,9 @@ void put_huge_zero_page(void);
>> >  static inline void prep_transhuge_page(struct page *page) {}
>> >
>> >  #define transparent_hugepage_flags 0UL
>> > +
>> > +#define thp_get_unmapped_area  NULL
>>
>> Lets make this:
>>
>> static inline unsigned long thp_get_unmapped_area(struct file *filp,
>>                unsigned long addr, unsigned long len, unsigned long
>> pgoff,
>>                unsigned long flags)
>> {
>>     return 0;
>> }
>>
>> ...to get some type checking in the CONFIG_TRANSPARENT_HUGEPAGE=n
>> case.
>>
>
> Per get_unmapped_area() in mm/mmap.c, I think we need to set it to NULL
> when we do not override current->mm->get_unmapped_area.

Ah, ok.
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH v4 RESEND 1/2] thp, dax: add thp_get_unmapped_area for pmd mappings

[Dan Williams]

On Mon, Aug 29, 2016 at 1:44 PM, Kani, Toshimitsu <toshi.kani@hpe.com> wrote:
> On Mon, 2016-08-29 at 12:34 -0700, Dan Williams wrote:
>> On Mon, Aug 29, 2016 at 12:11 PM, Toshi Kani <toshi.kani@hpe.com>
>> wrote:
>> >
>> > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
>> > size.  This feature relies on both mmap virtual address and FS
>> > block (i.e. physical address) to be aligned by the pmd page size.
>> > Users can use mkfs options to specify FS to align block
>> > allocations. However, aligning mmap address requires code changes
>> > to existing applications for providing a pmd-aligned address to
>> > mmap().
>> >
>> > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
>> > [1]. It calls mmap() with a NULL address, which needs to be changed
>> > to provide a pmd-aligned address for testing with DAX pmd mappings.
>> > Changing all applications that call mmap() with NULL is
>> > undesirable.
>> >
>> > Add thp_get_unmapped_area(), which can be called by filesystem's
>> > get_unmapped_area to align an mmap address by the pmd size for
>> > a DAX file.  It calls the default handler, mm->get_unmapped_area(),
>> > to find a range and then aligns it for a DAX file.
>> >
>> > The patch is based on Matthew Wilcox's change that allows adding
>> > support of the pud page size easily.
>  :
>>
>> Reviewed-by: Dan Williams <dan.j.williams@intel.com>
>
> Great!
>
>> ...with one minor nit:
>>
>>
>> >
>> >  include/linux/huge_mm.h |    7 +++++++
>> >  mm/huge_memory.c        |   43
>> > +++++++++++++++++++++++++++++++++++++++++++
>> >  2 files changed, 50 insertions(+)
>> >
>> > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
>> > index 6f14de4..4fca526 100644
>> > --- a/include/linux/huge_mm.h
>> > +++ b/include/linux/huge_mm.h
>> > @@ -87,6 +87,10 @@ extern bool is_vma_temporary_stack(struct
>> > vm_area_struct *vma);
>> >
>> >  extern unsigned long transparent_hugepage_flags;
>> >
>> > +extern unsigned long thp_get_unmapped_area(struct file *filp,
>> > +               unsigned long addr, unsigned long len, unsigned
>> > long pgoff,
>> > +               unsigned long flags);
>> > +
>> >  extern void prep_transhuge_page(struct page *page);
>> >  extern void free_transhuge_page(struct page *page);
>> >
>> > @@ -169,6 +173,9 @@ void put_huge_zero_page(void);
>> >  static inline void prep_transhuge_page(struct page *page) {}
>> >
>> >  #define transparent_hugepage_flags 0UL
>> > +
>> > +#define thp_get_unmapped_area  NULL
>>
>> Lets make this:
>>
>> static inline unsigned long thp_get_unmapped_area(struct file *filp,
>>                unsigned long addr, unsigned long len, unsigned long
>> pgoff,
>>                unsigned long flags)
>> {
>>     return 0;
>> }
>>
>> ...to get some type checking in the CONFIG_TRANSPARENT_HUGEPAGE=n
>> case.
>>
>
> Per get_unmapped_area() in mm/mmap.c, I think we need to set it to NULL
> when we do not override current->mm->get_unmapped_area.

Ah, ok.

Re: [PATCH v4 RESEND 1/2] thp, dax: add thp_get_unmapped_area for pmd mappings

[Dan Williams]

On Mon, Aug 29, 2016 at 1:44 PM, Kani, Toshimitsu <toshi.kani@hpe.com> wrote:
> On Mon, 2016-08-29 at 12:34 -0700, Dan Williams wrote:
>> On Mon, Aug 29, 2016 at 12:11 PM, Toshi Kani <toshi.kani@hpe.com>
>> wrote:
>> >
>> > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
>> > size.  This feature relies on both mmap virtual address and FS
>> > block (i.e. physical address) to be aligned by the pmd page size.
>> > Users can use mkfs options to specify FS to align block
>> > allocations. However, aligning mmap address requires code changes
>> > to existing applications for providing a pmd-aligned address to
>> > mmap().
>> >
>> > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
>> > [1]. It calls mmap() with a NULL address, which needs to be changed
>> > to provide a pmd-aligned address for testing with DAX pmd mappings.
>> > Changing all applications that call mmap() with NULL is
>> > undesirable.
>> >
>> > Add thp_get_unmapped_area(), which can be called by filesystem's
>> > get_unmapped_area to align an mmap address by the pmd size for
>> > a DAX file.  It calls the default handler, mm->get_unmapped_area(),
>> > to find a range and then aligns it for a DAX file.
>> >
>> > The patch is based on Matthew Wilcox's change that allows adding
>> > support of the pud page size easily.
>  :
>>
>> Reviewed-by: Dan Williams <dan.j.williams@intel.com>
>
> Great!
>
>> ...with one minor nit:
>>
>>
>> >
>> >  include/linux/huge_mm.h |    7 +++++++
>> >  mm/huge_memory.c        |   43
>> > +++++++++++++++++++++++++++++++++++++++++++
>> >  2 files changed, 50 insertions(+)
>> >
>> > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
>> > index 6f14de4..4fca526 100644
>> > --- a/include/linux/huge_mm.h
>> > +++ b/include/linux/huge_mm.h
>> > @@ -87,6 +87,10 @@ extern bool is_vma_temporary_stack(struct
>> > vm_area_struct *vma);
>> >
>> >  extern unsigned long transparent_hugepage_flags;
>> >
>> > +extern unsigned long thp_get_unmapped_area(struct file *filp,
>> > +               unsigned long addr, unsigned long len, unsigned
>> > long pgoff,
>> > +               unsigned long flags);
>> > +
>> >  extern void prep_transhuge_page(struct page *page);
>> >  extern void free_transhuge_page(struct page *page);
>> >
>> > @@ -169,6 +173,9 @@ void put_huge_zero_page(void);
>> >  static inline void prep_transhuge_page(struct page *page) {}
>> >
>> >  #define transparent_hugepage_flags 0UL
>> > +
>> > +#define thp_get_unmapped_area  NULL
>>
>> Lets make this:
>>
>> static inline unsigned long thp_get_unmapped_area(struct file *filp,
>>                unsigned long addr, unsigned long len, unsigned long
>> pgoff,
>>                unsigned long flags)
>> {
>>     return 0;
>> }
>>
>> ...to get some type checking in the CONFIG_TRANSPARENT_HUGEPAGE=n
>> case.
>>
>
> Per get_unmapped_area() in mm/mmap.c, I think we need to set it to NULL
> when we do not override current->mm->get_unmapped_area.

Ah, ok.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

On Mon, 2016-08-29 at 23:48 +0300, Kirill A. Shutemov wrote:
> On Mon, Aug 29, 2016 at 01:11:19PM -0600, Toshi Kani wrote:
> > 
> > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> > size.  This feature relies on both mmap virtual address and FS
> > block (i.e. physical address) to be aligned by the pmd page size.
> > Users can use mkfs options to specify FS to align block
> > allocations. However, aligning mmap address requires code changes
> > to existing applications for providing a pmd-aligned address to
> > mmap().
> > 
> > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
> > [1]. It calls mmap() with a NULL address, which needs to be changed
> > to provide a pmd-aligned address for testing with DAX pmd mappings.
> > Changing all applications that call mmap() with NULL is
> > undesirable.
> > 
> > This patch-set extends filesystems to align an mmap address for
> > a DAX file so that unmodified applications can use DAX pmd
> > mappings.
> 
> +Hugh
> 
> Can we get it used for shmem/tmpfs too?
> I don't think we should duplicate essentially the same functionality
> in multiple places.

Here is my brief analysis when I had looked at the Hugh's patch last
time (before shmem_get_unmapped_area() was accepted).
https://patchwork.kernel.org/patch/8916741/

Besides some differences in the logic, ex. shmem_get_unmapped_area()
always calls current->mm->get_unmapped_area twice, yes, they basically
provide the same functionality.

I think one issue is that shmem_get_unmapped_area() checks with its
static flag 'shmem_huge', and additinally deals with SHMEM_HUGE_DENY
and SHMEM_HUGE_FORCE cases.  It also handles non-file case for
!SHMEM_HUGE_FORCE.

Thanks,
-Toshi



_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

On Mon, 2016-08-29 at 23:48 +0300, Kirill A. Shutemov wrote:
> On Mon, Aug 29, 2016 at 01:11:19PM -0600, Toshi Kani wrote:
> > 
> > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> > size.  This feature relies on both mmap virtual address and FS
> > block (i.e. physical address) to be aligned by the pmd page size.
> > Users can use mkfs options to specify FS to align block
> > allocations. However, aligning mmap address requires code changes
> > to existing applications for providing a pmd-aligned address to
> > mmap().
> > 
> > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
> > [1]. It calls mmap() with a NULL address, which needs to be changed
> > to provide a pmd-aligned address for testing with DAX pmd mappings.
> > Changing all applications that call mmap() with NULL is
> > undesirable.
> > 
> > This patch-set extends filesystems to align an mmap address for
> > a DAX file so that unmodified applications can use DAX pmd
> > mappings.
> 
> +Hugh
> 
> Can we get it used for shmem/tmpfs too?
> I don't think we should duplicate essentially the same functionality
> in multiple places.

Here is my brief analysis when I had looked at the Hugh's patch last
time (before shmem_get_unmapped_area() was accepted).
https://patchwork.kernel.org/patch/8916741/

Besides some differences in the logic, ex. shmem_get_unmapped_area()
always calls current->mm->get_unmapped_area twice, yes, they basically
provide the same functionality.

I think one issue is that shmem_get_unmapped_area() checks with its
static flag 'shmem_huge', and additinally deals with SHMEM_HUGE_DENY
and SHMEM_HUGE_FORCE cases.  It also handles non-file case for
!SHMEM_HUGE_FORCE.

Thanks,
-Toshi

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1869 bytes --]

On Mon, 2016-08-29 at 23:48 +0300, Kirill A. Shutemov wrote:
> On Mon, Aug 29, 2016 at 01:11:19PM -0600, Toshi Kani wrote:
> > 
> > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> > size.  This feature relies on both mmap virtual address and FS
> > block (i.e. physical address) to be aligned by the pmd page size.
> > Users can use mkfs options to specify FS to align block
> > allocations. However, aligning mmap address requires code changes
> > to existing applications for providing a pmd-aligned address to
> > mmap().
> > 
> > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
> > [1]. It calls mmap() with a NULL address, which needs to be changed
> > to provide a pmd-aligned address for testing with DAX pmd mappings.
> > Changing all applications that call mmap() with NULL is
> > undesirable.
> > 
> > This patch-set extends filesystems to align an mmap address for
> > a DAX file so that unmodified applications can use DAX pmd
> > mappings.
> 
> +Hugh
> 
> Can we get it used for shmem/tmpfs too?
> I don't think we should duplicate essentially the same functionality
> in multiple places.

Here is my brief analysis when I had looked at the Hugh's patch last
time (before shmem_get_unmapped_area() was accepted).
https://patchwork.kernel.org/patch/8916741/

Besides some differences in the logic, ex. shmem_get_unmapped_area()
always calls current->mm->get_unmapped_area twice, yes, they basically
provide the same functionality.

I think one issue is that shmem_get_unmapped_area() checks with its
static flag 'shmem_huge', and additinally deals with SHMEM_HUGE_DENY
and SHMEM_HUGE_FORCE cases.  It also handles non-file case for
!SHMEM_HUGE_FORCE.

Thanks,
-Toshi



N‹§²æìr¸›zǧu©ž²Æ {\b­†éì¹»\x1c®&Þ–)îÆi¢žØ^n‡r¶‰šŽŠÝ¢j$½§$¢¸\x05¢¹¨­è§~Š'.)îÄÃ,yèm¶ŸÿÃ\f%Š{±šj+ƒðèž×¦j)Z†·Ÿ

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

On Mon, 2016-08-29 at 15:31 -0600, Kani, Toshimitsu wrote:
> On Mon, 2016-08-29 at 23:48 +0300, Kirill A. Shutemov wrote:
> > 
> > On Mon, Aug 29, 2016 at 01:11:19PM -0600, Toshi Kani wrote:
> > > 
> > > 
> > > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> > > size.  This feature relies on both mmap virtual address and FS
> > > block (i.e. physical address) to be aligned by the pmd page size.
> > > Users can use mkfs options to specify FS to align block
> > > allocations. However, aligning mmap address requires code changes
> > > to existing applications for providing a pmd-aligned address to
> > > mmap().
> > > 
> > > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
> > > [1]. It calls mmap() with a NULL address, which needs to be
> > > changed to provide a pmd-aligned address for testing with DAX pmd
> > > mappings. Changing all applications that call mmap() with NULL is
> > > undesirable.
> > > 
> > > This patch-set extends filesystems to align an mmap address for
> > > a DAX file so that unmodified applications can use DAX pmd
> > > mappings.
> > 
> > +Hugh
> > 
> > Can we get it used for shmem/tmpfs too?
> > I don't think we should duplicate essentially the same
> > functionality in multiple places.
> 
> Here is my brief analysis when I had looked at the Hugh's patch last
> time (before shmem_get_unmapped_area() was accepted).
> https://patchwork.kernel.org/patch/8916741/
> 
> Besides some differences in the logic, ex. shmem_get_unmapped_area()
> always calls current->mm->get_unmapped_area twice, yes, they
> basically provide the same functionality.
> 
> I think one issue is that shmem_get_unmapped_area() checks with its
> static flag 'shmem_huge', and additinally deals with SHMEM_HUGE_DENY
> and SHMEM_HUGE_FORCE cases.  It also handles non-file case for
> !SHMEM_HUGE_FORCE.

Looking further, these shmem_huge handlings only check pre-conditions.
 So, we should be able to make shmem_get_unmapped_area() as a wrapper,
which checks such shmem-specific conitions, and then
call __thp_get_unmapped_area() for the actual work.  All DAX-specific
checks are performed in thp_get_unmapped_area() as well.  We can make
 __thp_get_unmapped_area() as a common function.

I'd prefer to make such change as a separate item, but I can include it
to this patch series if needed. 

Thanks,
-Toshi
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

On Mon, 2016-08-29 at 15:31 -0600, Kani, Toshimitsu wrote:
> On Mon, 2016-08-29 at 23:48 +0300, Kirill A. Shutemov wrote:
> > 
> > On Mon, Aug 29, 2016 at 01:11:19PM -0600, Toshi Kani wrote:
> > > 
> > > 
> > > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> > > size.  This feature relies on both mmap virtual address and FS
> > > block (i.e. physical address) to be aligned by the pmd page size.
> > > Users can use mkfs options to specify FS to align block
> > > allocations. However, aligning mmap address requires code changes
> > > to existing applications for providing a pmd-aligned address to
> > > mmap().
> > > 
> > > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
> > > [1]. It calls mmap() with a NULL address, which needs to be
> > > changed to provide a pmd-aligned address for testing with DAX pmd
> > > mappings. Changing all applications that call mmap() with NULL is
> > > undesirable.
> > > 
> > > This patch-set extends filesystems to align an mmap address for
> > > a DAX file so that unmodified applications can use DAX pmd
> > > mappings.
> > 
> > +Hugh
> > 
> > Can we get it used for shmem/tmpfs too?
> > I don't think we should duplicate essentially the same
> > functionality in multiple places.
> 
> Here is my brief analysis when I had looked at the Hugh's patch last
> time (before shmem_get_unmapped_area() was accepted).
> https://patchwork.kernel.org/patch/8916741/
> 
> Besides some differences in the logic, ex. shmem_get_unmapped_area()
> always calls current->mm->get_unmapped_area twice, yes, they
> basically provide the same functionality.
> 
> I think one issue is that shmem_get_unmapped_area() checks with its
> static flag 'shmem_huge', and additinally deals with SHMEM_HUGE_DENY
> and SHMEM_HUGE_FORCE cases.  It also handles non-file case for
> !SHMEM_HUGE_FORCE.

Looking further, these shmem_huge handlings only check pre-conditions.
 So, we should be able to make shmem_get_unmapped_area() as a wrapper,
which checks such shmem-specific conitions, and then
call __thp_get_unmapped_area() for the actual work.  All DAX-specific
checks are performed in thp_get_unmapped_area() as well.  We can make
 __thp_get_unmapped_area() as a common function.

I'd prefer to make such change as a separate item, but I can include it
to this patch series if needed. 

Thanks,
-Toshi

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

On Mon, 2016-08-29 at 15:31 -0600, Kani, Toshimitsu wrote:
> On Mon, 2016-08-29 at 23:48 +0300, Kirill A. Shutemov wrote:
> > 
> > On Mon, Aug 29, 2016 at 01:11:19PM -0600, Toshi Kani wrote:
> > > 
> > > 
> > > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> > > size.  This feature relies on both mmap virtual address and FS
> > > block (i.e. physical address) to be aligned by the pmd page size.
> > > Users can use mkfs options to specify FS to align block
> > > allocations. However, aligning mmap address requires code changes
> > > to existing applications for providing a pmd-aligned address to
> > > mmap().
> > > 
> > > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
> > > [1]. It calls mmap() with a NULL address, which needs to be
> > > changed to provide a pmd-aligned address for testing with DAX pmd
> > > mappings. Changing all applications that call mmap() with NULL is
> > > undesirable.
> > > 
> > > This patch-set extends filesystems to align an mmap address for
> > > a DAX file so that unmodified applications can use DAX pmd
> > > mappings.
> > 
> > +Hugh
> > 
> > Can we get it used for shmem/tmpfs too?
> > I don't think we should duplicate essentially the same
> > functionality in multiple places.
> 
> Here is my brief analysis when I had looked at the Hugh's patch last
> time (before shmem_get_unmapped_area() was accepted).
> https://patchwork.kernel.org/patch/8916741/
> 
> Besides some differences in the logic, ex. shmem_get_unmapped_area()
> always calls current->mm->get_unmapped_area twice, yes, they
> basically provide the same functionality.
> 
> I think one issue is that shmem_get_unmapped_area() checks with its
> static flag 'shmem_huge', and additinally deals with SHMEM_HUGE_DENY
> and SHMEM_HUGE_FORCE cases.  It also handles non-file case for
> !SHMEM_HUGE_FORCE.

Looking further, these shmem_huge handlings only check pre-conditions.
 So, we should be able to make shmem_get_unmapped_area() as a wrapper,
which checks such shmem-specific conitions, and then
call __thp_get_unmapped_area() for the actual work.  All DAX-specific
checks are performed in thp_get_unmapped_area() as well.  We can make
 __thp_get_unmapped_area() as a common function.

I'd prefer to make such change as a separate item, but I can include it
to this patch series if needed. 

Thanks,
-Toshi

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kirill A. Shutemov]

On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu wrote:
> On Mon, 2016-08-29 at 15:31 -0600, Kani, Toshimitsu wrote:
> > On Mon, 2016-08-29 at 23:48 +0300, Kirill A. Shutemov wrote:
> > > 
> > > On Mon, Aug 29, 2016 at 01:11:19PM -0600, Toshi Kani wrote:
> > > > 
> > > > 
> > > > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> > > > size.��This feature relies on both mmap virtual address and FS
> > > > block (i.e. physical address) to be aligned by the pmd page size.
> > > > Users can use mkfs options to specify FS to align block
> > > > allocations. However, aligning mmap address requires code changes
> > > > to existing applications for providing a pmd-aligned address to
> > > > mmap().
> > > > 
> > > > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
> > > > [1]. It calls mmap() with a NULL address, which needs to be
> > > > changed to provide a pmd-aligned address for testing with DAX pmd
> > > > mappings. Changing all applications that call mmap() with NULL is
> > > > undesirable.
> > > > 
> > > > This patch-set extends filesystems to align an mmap address for
> > > > a DAX file so that unmodified applications can use DAX pmd
> > > > mappings.
> > > 
> > > +Hugh
> > > 
> > > Can we get it used for shmem/tmpfs too?
> > > I don't think we should duplicate essentially the same
> > > functionality in multiple places.
> > 
> > Here is my brief analysis when I had looked at the Hugh's patch last
> > time (before�shmem_get_unmapped_area() was accepted).
> > https://patchwork.kernel.org/patch/8916741/
> > 
> > Besides some differences in the logic, ex. shmem_get_unmapped_area()
> > always calls�current->mm->get_unmapped_area twice, yes, they
> > basically provide the same functionality.
> > 
> > I think one issue is that shmem_get_unmapped_area() checks with its
> > static flag 'shmem_huge', and additinally deals with SHMEM_HUGE_DENY
> > and�SHMEM_HUGE_FORCE cases. �It also handles non-file case for
> > !SHMEM_HUGE_FORCE.
> 
> Looking further, these shmem_huge handlings only check pre-conditions.
> �So, we should be able to make shmem_get_unmapped_area() as a wrapper,
> which checks such shmem-specific conitions, and then
> call�__thp_get_unmapped_area() for the actual work. �All DAX-specific
> checks are performed in thp_get_unmapped_area() as well. �We can make
> �__thp_get_unmapped_area() as a common function.
> 
> I'd prefer to make such change as a separate item,

Do you have plan to submit such change?

-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kirill A. Shutemov]

On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu wrote:
> On Mon, 2016-08-29 at 15:31 -0600, Kani, Toshimitsu wrote:
> > On Mon, 2016-08-29 at 23:48 +0300, Kirill A. Shutemov wrote:
> > > 
> > > On Mon, Aug 29, 2016 at 01:11:19PM -0600, Toshi Kani wrote:
> > > > 
> > > > 
> > > > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> > > > size.  This feature relies on both mmap virtual address and FS
> > > > block (i.e. physical address) to be aligned by the pmd page size.
> > > > Users can use mkfs options to specify FS to align block
> > > > allocations. However, aligning mmap address requires code changes
> > > > to existing applications for providing a pmd-aligned address to
> > > > mmap().
> > > > 
> > > > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
> > > > [1]. It calls mmap() with a NULL address, which needs to be
> > > > changed to provide a pmd-aligned address for testing with DAX pmd
> > > > mappings. Changing all applications that call mmap() with NULL is
> > > > undesirable.
> > > > 
> > > > This patch-set extends filesystems to align an mmap address for
> > > > a DAX file so that unmodified applications can use DAX pmd
> > > > mappings.
> > > 
> > > +Hugh
> > > 
> > > Can we get it used for shmem/tmpfs too?
> > > I don't think we should duplicate essentially the same
> > > functionality in multiple places.
> > 
> > Here is my brief analysis when I had looked at the Hugh's patch last
> > time (before shmem_get_unmapped_area() was accepted).
> > https://patchwork.kernel.org/patch/8916741/
> > 
> > Besides some differences in the logic, ex. shmem_get_unmapped_area()
> > always calls current->mm->get_unmapped_area twice, yes, they
> > basically provide the same functionality.
> > 
> > I think one issue is that shmem_get_unmapped_area() checks with its
> > static flag 'shmem_huge', and additinally deals with SHMEM_HUGE_DENY
> > and SHMEM_HUGE_FORCE cases.  It also handles non-file case for
> > !SHMEM_HUGE_FORCE.
> 
> Looking further, these shmem_huge handlings only check pre-conditions.
>  So, we should be able to make shmem_get_unmapped_area() as a wrapper,
> which checks such shmem-specific conitions, and then
> call __thp_get_unmapped_area() for the actual work.  All DAX-specific
> checks are performed in thp_get_unmapped_area() as well.  We can make
>  __thp_get_unmapped_area() as a common function.
> 
> I'd prefer to make such change as a separate item,

Do you have plan to submit such change?

-- 
 Kirill A. Shutemov

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kirill A. Shutemov]

On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu wrote:
> On Mon, 2016-08-29 at 15:31 -0600, Kani, Toshimitsu wrote:
> > On Mon, 2016-08-29 at 23:48 +0300, Kirill A. Shutemov wrote:
> > > 
> > > On Mon, Aug 29, 2016 at 01:11:19PM -0600, Toshi Kani wrote:
> > > > 
> > > > 
> > > > When CONFIG_FS_DAX_PMD is set, DAX supports mmap() using pmd page
> > > > size.  This feature relies on both mmap virtual address and FS
> > > > block (i.e. physical address) to be aligned by the pmd page size.
> > > > Users can use mkfs options to specify FS to align block
> > > > allocations. However, aligning mmap address requires code changes
> > > > to existing applications for providing a pmd-aligned address to
> > > > mmap().
> > > > 
> > > > For instance, fio with "ioengine=mmap" performs I/Os with mmap()
> > > > [1]. It calls mmap() with a NULL address, which needs to be
> > > > changed to provide a pmd-aligned address for testing with DAX pmd
> > > > mappings. Changing all applications that call mmap() with NULL is
> > > > undesirable.
> > > > 
> > > > This patch-set extends filesystems to align an mmap address for
> > > > a DAX file so that unmodified applications can use DAX pmd
> > > > mappings.
> > > 
> > > +Hugh
> > > 
> > > Can we get it used for shmem/tmpfs too?
> > > I don't think we should duplicate essentially the same
> > > functionality in multiple places.
> > 
> > Here is my brief analysis when I had looked at the Hugh's patch last
> > time (before shmem_get_unmapped_area() was accepted).
> > https://patchwork.kernel.org/patch/8916741/
> > 
> > Besides some differences in the logic, ex. shmem_get_unmapped_area()
> > always calls current->mm->get_unmapped_area twice, yes, they
> > basically provide the same functionality.
> > 
> > I think one issue is that shmem_get_unmapped_area() checks with its
> > static flag 'shmem_huge', and additinally deals with SHMEM_HUGE_DENY
> > and SHMEM_HUGE_FORCE cases.  It also handles non-file case for
> > !SHMEM_HUGE_FORCE.
> 
> Looking further, these shmem_huge handlings only check pre-conditions.
>  So, we should be able to make shmem_get_unmapped_area() as a wrapper,
> which checks such shmem-specific conitions, and then
> call __thp_get_unmapped_area() for the actual work.  All DAX-specific
> checks are performed in thp_get_unmapped_area() as well.  We can make
>  __thp_get_unmapped_area() as a common function.
> 
> I'd prefer to make such change as a separate item,

Do you have plan to submit such change?

-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu wrote:
> > 
> > On Mon, 2016-08-29 at 15:31 -0600, Kani, Toshimitsu wrote:
> > > 
> > > On Mon, 2016-08-29 at 23:48 +0300, Kirill A. Shutemov wrote:
> > > > 
 :
> > Looking further, these shmem_huge handlings only check pre-
> > conditions.  So, we should 
> > be able to make shmem_get_unmapped_area() as a wrapper, which
> > checks such shmem-specific conitions, and
> > then call __thp_get_unmapped_area() for the actual work.  All DAX-
> > specific checks are performed in thp_get_unmapped_area() as well.
> >  We can make  __thp_get_unmapped_area() as a common function.
> > 
> > I'd prefer to make such change as a separate item,
> 
> Do you have plan to submit such change?

Yes, I will submit the change once I finish testing.

Thanks,
-Toshi
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu wrote:
> > 
> > On Mon, 2016-08-29 at 15:31 -0600, Kani, Toshimitsu wrote:
> > > 
> > > On Mon, 2016-08-29 at 23:48 +0300, Kirill A. Shutemov wrote:
> > > > 
 :
> > Looking further, these shmem_huge handlings only check pre-
> > conditions.  So, we should 
> > be able to make shmem_get_unmapped_area() as a wrapper, which
> > checks such shmem-specific conitions, and
> > then call __thp_get_unmapped_area() for the actual work.  All DAX-
> > specific checks are performed in thp_get_unmapped_area() as well.
> >  We can make  __thp_get_unmapped_area() as a common function.
> > 
> > I'd prefer to make such change as a separate item,
> 
> Do you have plan to submit such change?

Yes, I will submit the change once I finish testing.

Thanks,
-Toshi

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1005 bytes --]

On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu wrote:
> > 
> > On Mon, 2016-08-29 at 15:31 -0600, Kani, Toshimitsu wrote:
> > > 
> > > On Mon, 2016-08-29 at 23:48 +0300, Kirill A. Shutemov wrote:
> > > > 
 :
> > Looking further, these shmem_huge handlings only check pre-
> > conditions.  So, we should 
> > be able to make shmem_get_unmapped_area() as a wrapper, which
> > checks such shmem-specific conitions, and
> > then call __thp_get_unmapped_area() for the actual work.  All DAX-
> > specific checks are performed in thp_get_unmapped_area() as well.
> >  We can make  __thp_get_unmapped_area() as a common function.
> > 
> > I'd prefer to make such change as a separate item,
> 
> Do you have plan to submit such change?

Yes, I will submit the change once I finish testing.

Thanks,
-ToshiN‹§²æìr¸›zǧu©ž²Æ {\b­†éì¹»\x1c®&Þ–)îÆi¢žØ^n‡r¶‰šŽŠÝ¢j$½§$¢¸\x05¢¹¨­è§~Š'.)îÄÃ,yèm¶ŸÿÃ\f%Š{±šj+ƒðèž×¦j)Z†·Ÿ

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

On Thu, 2016-09-08 at 07:48 -0600, Kani, Toshimitsu wrote:
> On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> > 
> > On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu wrote:
 :
> > > 
> > > Looking further, these shmem_huge handlings only check pre-
> > > conditions.  So, we should be able to make shmem_get_unmapped_are
> > > a() as a wrapper, which checks such shmem-specific conitions, and
> > > then call __thp_get_unmapped_area() for the actual work.  All
> > > DAX-specific checks are performed in thp_get_unmapped_area() as
> > > well.  We can make  __thp_get_unmapped_area() as a common
> > > function.
> > > 
> > > I'd prefer to make such change as a separate item,
> > 
> > Do you have plan to submit such change?
> 
> Yes, I will submit the change once I finish testing.

I found a bug in the current code, and need some clarification.  The
if-statement below is reverted.

===
diff --git a/mm/shmem.c b/mm/shmem.c
index fd8b2b5..aec5b49 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -1980,7 +1980,7 @@ unsigned long shmem_get_unmapped_area(struct file
*file,
                                return addr;
                        sb = shm_mnt->mnt_sb;
                }
-               if (SHMEM_SB(sb)->huge != SHMEM_HUGE_NEVER)
+               if (SHMEM_SB(sb)->huge == SHMEM_HUGE_NEVER)
                        return addr;
        }
===

Because of this bug, mounting tmpfs with "huge=never" enables huge page
mappings, and "huge=always" or others disables it...

The above simple change will change the default behavior, though.  When
"huge=" option is not specified, SHMEM_SB(sb)->huge is set to zero,
which is SHMEM_HUGE_NEVER.  Therefore, huge page mappings are enabled
by default because of this bug.

What's the intended default behavior of this feature?

Thanks,
-Toshi
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

On Thu, 2016-09-08 at 07:48 -0600, Kani, Toshimitsu wrote:
> On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> > 
> > On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu wrote:
 :
> > > 
> > > Looking further, these shmem_huge handlings only check pre-
> > > conditions.  So, we should be able to make shmem_get_unmapped_are
> > > a() as a wrapper, which checks such shmem-specific conitions, and
> > > then call __thp_get_unmapped_area() for the actual work.  All
> > > DAX-specific checks are performed in thp_get_unmapped_area() as
> > > well.  We can make  __thp_get_unmapped_area() as a common
> > > function.
> > > 
> > > I'd prefer to make such change as a separate item,
> > 
> > Do you have plan to submit such change?
> 
> Yes, I will submit the change once I finish testing.

I found a bug in the current code, and need some clarification.  The
if-statement below is reverted.

===
diff --git a/mm/shmem.c b/mm/shmem.c
index fd8b2b5..aec5b49 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -1980,7 +1980,7 @@ unsigned long shmem_get_unmapped_area(struct file
*file,
                                return addr;
                        sb = shm_mnt->mnt_sb;
                }
-               if (SHMEM_SB(sb)->huge != SHMEM_HUGE_NEVER)
+               if (SHMEM_SB(sb)->huge == SHMEM_HUGE_NEVER)
                        return addr;
        }
===

Because of this bug, mounting tmpfs with "huge=never" enables huge page
mappings, and "huge=always" or others disables it...

The above simple change will change the default behavior, though.  When
"huge=" option is not specified, SHMEM_SB(sb)->huge is set to zero,
which is SHMEM_HUGE_NEVER.  Therefore, huge page mappings are enabled
by default because of this bug.

What's the intended default behavior of this feature?

Thanks,
-Toshi

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2115 bytes --]

On Thu, 2016-09-08 at 07:48 -0600, Kani, Toshimitsu wrote:
> On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> > 
> > On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu wrote:
 :
> > > 
> > > Looking further, these shmem_huge handlings only check pre-
> > > conditions.  So, we should be able to make shmem_get_unmapped_are
> > > a() as a wrapper, which checks such shmem-specific conitions, and
> > > then call __thp_get_unmapped_area() for the actual work.  All
> > > DAX-specific checks are performed in thp_get_unmapped_area() as
> > > well.  We can make  __thp_get_unmapped_area() as a common
> > > function.
> > > 
> > > I'd prefer to make such change as a separate item,
> > 
> > Do you have plan to submit such change?
> 
> Yes, I will submit the change once I finish testing.

I found a bug in the current code, and need some clarification.  The
if-statement below is reverted.

===
diff --git a/mm/shmem.c b/mm/shmem.c
index fd8b2b5..aec5b49 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -1980,7 +1980,7 @@ unsigned long shmem_get_unmapped_area(struct file
*file,
                                return addr;
                        sb = shm_mnt->mnt_sb;
                }
-               if (SHMEM_SB(sb)->huge != SHMEM_HUGE_NEVER)
+               if (SHMEM_SB(sb)->huge == SHMEM_HUGE_NEVER)
                        return addr;
        }
===

Because of this bug, mounting tmpfs with "huge=never" enables huge page
mappings, and "huge=always" or others disables it...

The above simple change will change the default behavior, though.  When
"huge=" option is not specified, SHMEM_SB(sb)->huge is set to zero,
which is SHMEM_HUGE_NEVER.  Therefore, huge page mappings are enabled
by default because of this bug.

What's the intended default behavior of this feature?

Thanks,
-Toshi
N‹§²æìr¸›zǧu©ž²Æ {\b­†éì¹»\x1c®&Þ–)îÆi¢žØ^n‡r¶‰šŽŠÝ¢j$½§$¢¸\x05¢¹¨­è§~Š'.)îÄÃ,yèm¶ŸÿÃ\f%Š{±šj+ƒðèž×¦j)Z†·Ÿ

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kirill A. Shutemov]

On Thu, Sep 08, 2016 at 11:21:46PM +0000, Kani, Toshimitsu wrote:
> On Thu, 2016-09-08 at 07:48 -0600, Kani, Toshimitsu wrote:
> > On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> > > 
> > > On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu wrote:
> �:
> > > > 
> > > > Looking further, these shmem_huge handlings only check pre-
> > > > conditions.��So,�we�should�be�able�to�make�shmem_get_unmapped_are
> > > > a() as a wrapper, which checks such shmem-specific conitions, and
> > > > then�call�__thp_get_unmapped_area() for the actual work. �All
> > > > DAX-specific checks are performed in thp_get_unmapped_area() as
> > > > well. �We can make �__thp_get_unmapped_area() as a common
> > > > function.
> > > > 
> > > > I'd prefer to make such change as a separate item,
> > > 
> > > Do you have plan to submit such change?
> > 
> > Yes, I will submit the change once I finish testing.
> 
> I found a bug in the current code, and need some clarification. �The
> if-statement below is reverted.

<two-hands-facepalm>

Yeah. It was repored by Hillf[1]. The fixup got lost. :(

Could you post a proper patch with the fix?

I would be nice to credit Hillf there too.

[1] http://lkml.kernel.org/r/054f01d1c86f$2994d5c0$7cbe8140$@alibaba-inc.com

-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kirill A. Shutemov]

On Thu, Sep 08, 2016 at 11:21:46PM +0000, Kani, Toshimitsu wrote:
> On Thu, 2016-09-08 at 07:48 -0600, Kani, Toshimitsu wrote:
> > On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> > > 
> > > On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu wrote:
>  :
> > > > 
> > > > Looking further, these shmem_huge handlings only check pre-
> > > > conditions.  So, we should be able to make shmem_get_unmapped_are
> > > > a() as a wrapper, which checks such shmem-specific conitions, and
> > > > then call __thp_get_unmapped_area() for the actual work.  All
> > > > DAX-specific checks are performed in thp_get_unmapped_area() as
> > > > well.  We can make  __thp_get_unmapped_area() as a common
> > > > function.
> > > > 
> > > > I'd prefer to make such change as a separate item,
> > > 
> > > Do you have plan to submit such change?
> > 
> > Yes, I will submit the change once I finish testing.
> 
> I found a bug in the current code, and need some clarification.  The
> if-statement below is reverted.

<two-hands-facepalm>

Yeah. It was repored by Hillf[1]. The fixup got lost. :(

Could you post a proper patch with the fix?

I would be nice to credit Hillf there too.

[1] http://lkml.kernel.org/r/054f01d1c86f$2994d5c0$7cbe8140$@alibaba-inc.com

-- 
 Kirill A. Shutemov

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kirill A. Shutemov]

On Thu, Sep 08, 2016 at 11:21:46PM +0000, Kani, Toshimitsu wrote:
> On Thu, 2016-09-08 at 07:48 -0600, Kani, Toshimitsu wrote:
> > On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> > > 
> > > On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu wrote:
>  :
> > > > 
> > > > Looking further, these shmem_huge handlings only check pre-
> > > > conditions.  So, we should be able to make shmem_get_unmapped_are
> > > > a() as a wrapper, which checks such shmem-specific conitions, and
> > > > then call __thp_get_unmapped_area() for the actual work.  All
> > > > DAX-specific checks are performed in thp_get_unmapped_area() as
> > > > well.  We can make  __thp_get_unmapped_area() as a common
> > > > function.
> > > > 
> > > > I'd prefer to make such change as a separate item,
> > > 
> > > Do you have plan to submit such change?
> > 
> > Yes, I will submit the change once I finish testing.
> 
> I found a bug in the current code, and need some clarification.  The
> if-statement below is reverted.

<two-hands-facepalm>

Yeah. It was repored by Hillf[1]. The fixup got lost. :(

Could you post a proper patch with the fix?

I would be nice to credit Hillf there too.

[1] http://lkml.kernel.org/r/054f01d1c86f$2994d5c0$7cbe8140$@alibaba-inc.com

-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

On Fri, 2016-09-09 at 15:36 +0300, Kirill A. Shutemov wrote:
> On Thu, Sep 08, 2016 at 11:21:46PM +0000, Kani, Toshimitsu wrote:
> > 
> > On Thu, 2016-09-08 at 07:48 -0600, Kani, Toshimitsu wrote:
> > > 
> > > On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> > > > 
> > > > On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu
> > > > wrote:
> >  :
> > > > > Looking further, these shmem_huge handlings only check pre-
> > > > > conditions.  So, we should be able to make shmem_get_unmapped
> > > > > _are a() as a wrapper, which checks such shmem-specific
> > > > > conitions, and then call __thp_get_unmapped_area() for the
> > > > > actual work.  All DAX-specific checks are performed in
> > > > > thp_get_unmapped_area() as well.  We can make
> > > > >  __thp_get_unmapped_area() as a common
> > > > > function.
> > > > > 
> > > > > I'd prefer to make such change as a separate item,
> > > > 
> > > > Do you have plan to submit such change?
> > > 
> > > Yes, I will submit the change once I finish testing.
> > 
> > I found a bug in the current code, and need some clarification.
> >  The if-statement below is reverted.
> 
> <two-hands-facepalm>
> 
> Yeah. It was repored by Hillf[1]. The fixup got lost. :(
> 
> Could you post a proper patch with the fix?
>
> I would be nice to credit Hillf there too.
> 
> [1] http://lkml.kernel.org/r/054f01d1c86f$2994d5c0$7cbe8140$@alibaba-
> inc.com

Yes, I will submit the fix as well.

I will not change the default value of sbinfo->huge in this fix.  So,
user will have to specify "huge=" option to enable huge page mappings.
 If this is not desireable, we will need a separate patch.

Thanks,
-Toshi

_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

On Fri, 2016-09-09 at 15:36 +0300, Kirill A. Shutemov wrote:
> On Thu, Sep 08, 2016 at 11:21:46PM +0000, Kani, Toshimitsu wrote:
> > 
> > On Thu, 2016-09-08 at 07:48 -0600, Kani, Toshimitsu wrote:
> > > 
> > > On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> > > > 
> > > > On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu
> > > > wrote:
> >  :
> > > > > Looking further, these shmem_huge handlings only check pre-
> > > > > conditions.  So, we should be able to make shmem_get_unmapped
> > > > > _are a() as a wrapper, which checks such shmem-specific
> > > > > conitions, and then call __thp_get_unmapped_area() for the
> > > > > actual work.  All DAX-specific checks are performed in
> > > > > thp_get_unmapped_area() as well.  We can make
> > > > >  __thp_get_unmapped_area() as a common
> > > > > function.
> > > > > 
> > > > > I'd prefer to make such change as a separate item,
> > > > 
> > > > Do you have plan to submit such change?
> > > 
> > > Yes, I will submit the change once I finish testing.
> > 
> > I found a bug in the current code, and need some clarification.
> >  The if-statement below is reverted.
> 
> <two-hands-facepalm>
> 
> Yeah. It was repored by Hillf[1]. The fixup got lost. :(
> 
> Could you post a proper patch with the fix?
>
> I would be nice to credit Hillf there too.
> 
> [1] http://lkml.kernel.org/r/054f01d1c86f$2994d5c0$7cbe8140$@alibaba-
> inc.com

Yes, I will submit the fix as well.

I will not change the default value of sbinfo->huge in this fix.  So,
user will have to specify "huge=" option to enable huge page mappings.
 If this is not desireable, we will need a separate patch.

Thanks,
-Toshi

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kani, Toshimitsu]

On Fri, 2016-09-09 at 15:36 +0300, Kirill A. Shutemov wrote:
> On Thu, Sep 08, 2016 at 11:21:46PM +0000, Kani, Toshimitsu wrote:
> > 
> > On Thu, 2016-09-08 at 07:48 -0600, Kani, Toshimitsu wrote:
> > > 
> > > On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> > > > 
> > > > On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu
> > > > wrote:
> >  :
> > > > > Looking further, these shmem_huge handlings only check pre-
> > > > > conditions.  So, we should be able to make shmem_get_unmapped
> > > > > _are a() as a wrapper, which checks such shmem-specific
> > > > > conitions, and then call __thp_get_unmapped_area() for the
> > > > > actual work.  All DAX-specific checks are performed in
> > > > > thp_get_unmapped_area() as well.  We can make
> > > > >  __thp_get_unmapped_area() as a common
> > > > > function.
> > > > > 
> > > > > I'd prefer to make such change as a separate item,
> > > > 
> > > > Do you have plan to submit such change?
> > > 
> > > Yes, I will submit the change once I finish testing.
> > 
> > I found a bug in the current code, and need some clarification.
> >  The if-statement below is reverted.
> 
> <two-hands-facepalm>
> 
> Yeah. It was repored by Hillf[1]. The fixup got lost. :(
> 
> Could you post a proper patch with the fix?
>
> I would be nice to credit Hillf there too.
> 
> [1] http://lkml.kernel.org/r/054f01d1c86f$2994d5c0$7cbe8140$@alibaba-
> inc.com

Yes, I will submit the fix as well.

I will not change the default value of sbinfo->huge in this fix.  So,
user will have to specify "huge=" option to enable huge page mappings.
 If this is not desireable, we will need a separate patch.

Thanks,
-Toshi

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kirill A. Shutemov]

On Fri, Sep 09, 2016 at 05:21:40PM +0000, Kani, Toshimitsu wrote:
> On Fri, 2016-09-09 at 15:36 +0300, Kirill A. Shutemov wrote:
> > On Thu, Sep 08, 2016 at 11:21:46PM +0000, Kani, Toshimitsu wrote:
> > > 
> > > On Thu, 2016-09-08 at 07:48 -0600, Kani, Toshimitsu wrote:
> > > > 
> > > > On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> > > > > 
> > > > > On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu
> > > > > wrote:
> > >  :
> > > > > > Looking further, these shmem_huge handlings only check pre-
> > > > > > conditions.  So, we should be able to make shmem_get_unmapped
> > > > > > _are a() as a wrapper, which checks such shmem-specific
> > > > > > conitions, and then call __thp_get_unmapped_area() for the
> > > > > > actual work.  All DAX-specific checks are performed in
> > > > > > thp_get_unmapped_area() as well.  We can make
> > > > > >  __thp_get_unmapped_area() as a common
> > > > > > function.
> > > > > > 
> > > > > > I'd prefer to make such change as a separate item,
> > > > > 
> > > > > Do you have plan to submit such change?
> > > > 
> > > > Yes, I will submit the change once I finish testing.
> > > 
> > > I found a bug in the current code, and need some clarification.
> > >  The if-statement below is reverted.
> > 
> > <two-hands-facepalm>
> > 
> > Yeah. It was repored by Hillf[1]. The fixup got lost. :(
> > 
> > Could you post a proper patch with the fix?
> >
> > I would be nice to credit Hillf there too.
> > 
> > [1] http://lkml.kernel.org/r/054f01d1c86f$2994d5c0$7cbe8140$@alibaba-
> > inc.com
> 
> Yes, I will submit the fix as well.
> 
> I will not change the default value of sbinfo->huge in this fix.  So,
> user will have to specify "huge=" option to enable huge page mappings.
>  If this is not desireable, we will need a separate patch.

That's okay. I only screwed up part wich make vma aligned. Allocation of
huge page happens under right condition.

-- 
 Kirill A. Shutemov
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kirill A. Shutemov]

On Fri, Sep 09, 2016 at 05:21:40PM +0000, Kani, Toshimitsu wrote:
> On Fri, 2016-09-09 at 15:36 +0300, Kirill A. Shutemov wrote:
> > On Thu, Sep 08, 2016 at 11:21:46PM +0000, Kani, Toshimitsu wrote:
> > > 
> > > On Thu, 2016-09-08 at 07:48 -0600, Kani, Toshimitsu wrote:
> > > > 
> > > > On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> > > > > 
> > > > > On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu
> > > > > wrote:
> > >  :
> > > > > > Looking further, these shmem_huge handlings only check pre-
> > > > > > conditions.  So, we should be able to make shmem_get_unmapped
> > > > > > _are a() as a wrapper, which checks such shmem-specific
> > > > > > conitions, and then call __thp_get_unmapped_area() for the
> > > > > > actual work.  All DAX-specific checks are performed in
> > > > > > thp_get_unmapped_area() as well.  We can make
> > > > > >  __thp_get_unmapped_area() as a common
> > > > > > function.
> > > > > > 
> > > > > > I'd prefer to make such change as a separate item,
> > > > > 
> > > > > Do you have plan to submit such change?
> > > > 
> > > > Yes, I will submit the change once I finish testing.
> > > 
> > > I found a bug in the current code, and need some clarification.
> > >  The if-statement below is reverted.
> > 
> > <two-hands-facepalm>
> > 
> > Yeah. It was repored by Hillf[1]. The fixup got lost. :(
> > 
> > Could you post a proper patch with the fix?
> >
> > I would be nice to credit Hillf there too.
> > 
> > [1] http://lkml.kernel.org/r/054f01d1c86f$2994d5c0$7cbe8140$@alibaba-
> > inc.com
> 
> Yes, I will submit the fix as well.
> 
> I will not change the default value of sbinfo->huge in this fix.  So,
> user will have to specify "huge=" option to enable huge page mappings.
>  If this is not desireable, we will need a separate patch.

That's okay. I only screwed up part wich make vma aligned. Allocation of
huge page happens under right condition.

-- 
 Kirill A. Shutemov

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kirill A. Shutemov]

On Fri, Sep 09, 2016 at 05:21:40PM +0000, Kani, Toshimitsu wrote:
> On Fri, 2016-09-09 at 15:36 +0300, Kirill A. Shutemov wrote:
> > On Thu, Sep 08, 2016 at 11:21:46PM +0000, Kani, Toshimitsu wrote:
> > > 
> > > On Thu, 2016-09-08 at 07:48 -0600, Kani, Toshimitsu wrote:
> > > > 
> > > > On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> > > > > 
> > > > > On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu
> > > > > wrote:
> > > �:
> > > > > > Looking further, these shmem_huge handlings only check pre-
> > > > > > conditions.��So,�we�should�be�able�to�make�shmem_get_unmapped
> > > > > > _are a() as a wrapper, which checks such shmem-specific
> > > > > > conitions, and then�call�__thp_get_unmapped_area() for the
> > > > > > actual work. �All DAX-specific checks are performed in
> > > > > > thp_get_unmapped_area() as well. �We can make
> > > > > > �__thp_get_unmapped_area() as a common
> > > > > > function.
> > > > > > 
> > > > > > I'd prefer to make such change as a separate item,
> > > > > 
> > > > > Do you have plan to submit such change?
> > > > 
> > > > Yes, I will submit the change once I finish testing.
> > > 
> > > I found a bug in the current code, and need some clarification.
> > > �The if-statement below is reverted.
> > 
> > <two-hands-facepalm>
> > 
> > Yeah. It was repored by Hillf[1]. The fixup got lost. :(
> > 
> > Could you post a proper patch with the fix?
> >
> > I would be nice to credit Hillf there too.
> > 
> > [1] http://lkml.kernel.org/r/054f01d1c86f$2994d5c0$7cbe8140$@alibaba-
> > inc.com
> 
> Yes, I will submit the fix as well.
> 
> I will not change the default value of sbinfo->huge in this fix. �So,
> user will have to specify "huge=" option to enable huge page mappings.
> �If this is not desireable, we will need a separate patch.

That's okay. I only screwed up part wich make vma aligned. Allocation of
huge page happens under right condition.

-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v4 RESEND 0/2] Align mmap address for DAX pmd mappings

[Kirill A. Shutemov]

On Fri, Sep 09, 2016 at 05:21:40PM +0000, Kani, Toshimitsu wrote:
> On Fri, 2016-09-09 at 15:36 +0300, Kirill A. Shutemov wrote:
> > On Thu, Sep 08, 2016 at 11:21:46PM +0000, Kani, Toshimitsu wrote:
> > > 
> > > On Thu, 2016-09-08 at 07:48 -0600, Kani, Toshimitsu wrote:
> > > > 
> > > > On Thu, 2016-09-08 at 13:57 +0300, Kirill A. Shutemov wrote:
> > > > > 
> > > > > On Mon, Aug 29, 2016 at 10:00:43PM +0000, Kani, Toshimitsu
> > > > > wrote:
> > >  :
> > > > > > Looking further, these shmem_huge handlings only check pre-
> > > > > > conditions.  So, we should be able to make shmem_get_unmapped
> > > > > > _are a() as a wrapper, which checks such shmem-specific
> > > > > > conitions, and then call __thp_get_unmapped_area() for the
> > > > > > actual work.  All DAX-specific checks are performed in
> > > > > > thp_get_unmapped_area() as well.  We can make
> > > > > >  __thp_get_unmapped_area() as a common
> > > > > > function.
> > > > > > 
> > > > > > I'd prefer to make such change as a separate item,
> > > > > 
> > > > > Do you have plan to submit such change?
> > > > 
> > > > Yes, I will submit the change once I finish testing.
> > > 
> > > I found a bug in the current code, and need some clarification.
> > >  The if-statement below is reverted.
> > 
> > <two-hands-facepalm>
> > 
> > Yeah. It was repored by Hillf[1]. The fixup got lost. :(
> > 
> > Could you post a proper patch with the fix?
> >
> > I would be nice to credit Hillf there too.
> > 
> > [1] http://lkml.kernel.org/r/054f01d1c86f$2994d5c0$7cbe8140$@alibaba-
> > inc.com
> 
> Yes, I will submit the fix as well.
> 
> I will not change the default value of sbinfo->huge in this fix.  So,
> user will have to specify "huge=" option to enable huge page mappings.
>  If this is not desireable, we will need a separate patch.

That's okay. I only screwed up part wich make vma aligned. Allocation of
huge page happens under right condition.

-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>