UBIFS and page migration (take 2)

UBIFS and page migration (take 2)

[Richard Weinberger]

During page migrations UBIFS gets confused. We triggered this by using CMA
on two different targets.
It turned out that fallback_migrate_page() is not suitable for UBIFS as it
does not copy the PagePrivate flag.
UBIFS is using this flag among with PageChecked to account free space.
One possible solution is implementing a ->migratepage() function in UBIFS
which does more or less the same as fallback_migrate_page() but also
copies PagePrivate. I'm not at all sure whether this is they way to go.
IMHO either page migration should not happen if ->migratepage() is not implement
or fallback_migrate_page() has to work for all filesystems.

Comments? Flames? :-)

Thanks,
//richard

[PATCH 1/2] mm: Export migrate_page_move_mapping and
[PATCH 2/2] UBIFS: Implement ->migratepage()

[PATCH 1/2] mm: Export migrate_page_move_mapping and migrate_page_copy

[Richard Weinberger]

Export these symbols such that UBIFS can implement
->migratepage.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 mm/migrate.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mm/migrate.c b/mm/migrate.c
index 6c822a7..6bc1035 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -431,6 +431,7 @@ int migrate_page_move_mapping(struct address_space *mapping,
 
 	return MIGRATEPAGE_SUCCESS;
 }
+EXPORT_SYMBOL(migrate_page_move_mapping);
 
 /*
  * The expected number of remaining references is the same as that
@@ -586,6 +587,7 @@ void migrate_page_copy(struct page *newpage, struct page *page)
 
 	mem_cgroup_migrate(page, newpage);
 }
+EXPORT_SYMBOL(migrate_page_copy);
 
 /************************************************************
  *                    Migration functions
-- 
1.8.4.5

[PATCH 2/2] UBIFS: Implement ->migratepage()

[Richard Weinberger]

From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>

During page migrations UBIFS might get confused
and the following assert triggers:
UBIFS assert failed in ubifs_set_page_dirty at 1451 (pid 436)

UBIFS is using PagePrivate() which can have different meanings across
filesystems. Therefore the generic page migration code cannot handle this
case correctly.
We have to implement our own migration function which basically does a
plain copy but also duplicates the page private flag.
UBIFS is not a block device filesystem and cannot use buffer_migrate_page().

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
[rw: Massaged changelog]
Signed-off-by: Richard Weinberger <richard@nod.at>

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 fs/ubifs/file.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/fs/ubifs/file.c b/fs/ubifs/file.c
index 065c88f..5eea5f5 100644
--- a/fs/ubifs/file.c
+++ b/fs/ubifs/file.c
@@ -52,6 +52,7 @@
 #include "ubifs.h"
 #include <linux/mount.h>
 #include <linux/slab.h>
+#include <linux/migrate.h>
 
 static int read_block(struct inode *inode, void *addr, unsigned int block,
 		      struct ubifs_data_node *dn)
@@ -1452,6 +1453,24 @@ static int ubifs_set_page_dirty(struct page *page)
 	return ret;
 }
 
+static int ubifs_migrate_page(struct address_space *mapping,
+		struct page *newpage, struct page *page, enum migrate_mode mode)
+{
+	int rc;
+
+	rc = migrate_page_move_mapping(mapping, newpage, page, NULL, mode, 0);
+	if (rc != MIGRATEPAGE_SUCCESS)
+		return rc;
+
+	if (PagePrivate(page)) {
+		ClearPagePrivate(page);
+		SetPagePrivate(newpage);
+	}
+
+	migrate_page_copy(newpage, page);
+	return MIGRATEPAGE_SUCCESS;
+}
+
 static int ubifs_releasepage(struct page *page, gfp_t unused_gfp_flags)
 {
 	/*
@@ -1591,6 +1610,7 @@ const struct address_space_operations ubifs_file_address_operations = {
 	.write_end      = ubifs_write_end,
 	.invalidatepage = ubifs_invalidatepage,
 	.set_page_dirty = ubifs_set_page_dirty,
+	.migratepage	= ubifs_migrate_page,
 	.releasepage    = ubifs_releasepage,
 };
 
-- 
1.8.4.5

Re: UBIFS and page migration (take 2)

[Vlastimil Babka]

On 03/31/2016 11:58 PM, Richard Weinberger wrote:
> During page migrations UBIFS gets confused. We triggered this by using CMA
> on two different targets.
> It turned out that fallback_migrate_page() is not suitable for UBIFS as it
> does not copy the PagePrivate flag.
> UBIFS is using this flag among with PageChecked to account free space.
> One possible solution is implementing a ->migratepage() function in UBIFS
> which does more or less the same as fallback_migrate_page() but also
> copies PagePrivate. I'm not at all sure whether this is they way to go.
> IMHO either page migration should not happen if ->migratepage() is not implement
> or fallback_migrate_page() has to work for all filesystems.

Yes, we could document more thoroughly the expectations of 
fallback_migrate_page() and audit the existing users, but still relying on every 
new address_space_operations instance to verify them isn't without risk. And I 
doubt there can be a default fallback that's guaranteed safe for all filesystems.

> Comments? Flames? :-)
>
> Thanks,
> //richard
>
> [PATCH 1/2] mm: Export migrate_page_move_mapping and
> [PATCH 2/2] UBIFS: Implement ->migratepage()
>

Re: [PATCH 2/2] UBIFS: Implement ->migratepage()

[Vlastimil Babka]

On 03/31/2016 11:58 PM, Richard Weinberger wrote:
> From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
>
> During page migrations UBIFS might get confused
> and the following assert triggers:
> UBIFS assert failed in ubifs_set_page_dirty at 1451 (pid 436)

It would be useful to have the full trace in changelog.

> UBIFS is using PagePrivate() which can have different meanings across
> filesystems. Therefore the generic page migration code cannot handle this
> case correctly.
> We have to implement our own migration function which basically does a
> plain copy but also duplicates the page private flag.
> UBIFS is not a block device filesystem and cannot use buffer_migrate_page().
>
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> [rw: Massaged changelog]
> Signed-off-by: Richard Weinberger <richard@nod.at>

Stable?

> Signed-off-by: Richard Weinberger <richard@nod.at>
> ---
>   fs/ubifs/file.c | 20 ++++++++++++++++++++
>   1 file changed, 20 insertions(+)
>
> diff --git a/fs/ubifs/file.c b/fs/ubifs/file.c
> index 065c88f..5eea5f5 100644
> --- a/fs/ubifs/file.c
> +++ b/fs/ubifs/file.c
> @@ -52,6 +52,7 @@
>   #include "ubifs.h"
>   #include <linux/mount.h>
>   #include <linux/slab.h>
> +#include <linux/migrate.h>
>
>   static int read_block(struct inode *inode, void *addr, unsigned int block,
>   		      struct ubifs_data_node *dn)
> @@ -1452,6 +1453,24 @@ static int ubifs_set_page_dirty(struct page *page)
>   	return ret;
>   }
>
> +static int ubifs_migrate_page(struct address_space *mapping,
> +		struct page *newpage, struct page *page, enum migrate_mode mode)
> +{
> +	int rc;
> +
> +	rc = migrate_page_move_mapping(mapping, newpage, page, NULL, mode, 0);
> +	if (rc != MIGRATEPAGE_SUCCESS)
> +		return rc;
> +
> +	if (PagePrivate(page)) {
> +		ClearPagePrivate(page);
> +		SetPagePrivate(newpage);
> +	}
> +
> +	migrate_page_copy(newpage, page);
> +	return MIGRATEPAGE_SUCCESS;
> +}
> +
>   static int ubifs_releasepage(struct page *page, gfp_t unused_gfp_flags)
>   {
>   	/*
> @@ -1591,6 +1610,7 @@ const struct address_space_operations ubifs_file_address_operations = {
>   	.write_end      = ubifs_write_end,
>   	.invalidatepage = ubifs_invalidatepage,
>   	.set_page_dirty = ubifs_set_page_dirty,
> +	.migratepage	= ubifs_migrate_page,
>   	.releasepage    = ubifs_releasepage,
>   };
>
>

Re: [PATCH 2/2] UBIFS: Implement ->migratepage()

[Richard Weinberger]

Am 01.04.2016 um 12:14 schrieb Vlastimil Babka:
> On 03/31/2016 11:58 PM, Richard Weinberger wrote:
>> From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
>>
>> During page migrations UBIFS might get confused
>> and the following assert triggers:
>> UBIFS assert failed in ubifs_set_page_dirty at 1451 (pid 436)
> 
> It would be useful to have the full trace in changelog.

Oh. Yes.

>> UBIFS is using PagePrivate() which can have different meanings across
>> filesystems. Therefore the generic page migration code cannot handle this
>> case correctly.
>> We have to implement our own migration function which basically does a
>> plain copy but also duplicates the page private flag.
>> UBIFS is not a block device filesystem and cannot use buffer_migrate_page().
>>
>> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>> [rw: Massaged changelog]
>> Signed-off-by: Richard Weinberger <richard@nod.at>
> 
> Stable?

Yep. But first I'd like to clarify if this approach is really the way to go.
It is also not clear to me whether this issue was always the case or if
a recently introduced change in MM uncovered it...
Blindly applying to all stable versions is risky.

Thanks,
//richard

Re: [PATCH 2/2] UBIFS: Implement ->migratepage()

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 1547 bytes --]

Hi Kirill,

[auto build test ERROR on v4.6-rc1]
[also build test ERROR on next-20160401]
[if your patch is applied to the wrong git tree, please drop us a note to help improving the system]

url:    https://github.com/0day-ci/linux/commits/Richard-Weinberger/mm-Export-migrate_page_move_mapping-and-migrate_page_copy/20160401-060041
config: xtensa-audio_kc705_defconfig (attached as .config)
reproduce:
        wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=xtensa 

All errors (new ones prefixed by >>):

   fs/ubifs/file.c: In function 'ubifs_migrate_page':
>> fs/ubifs/file.c:1461:2: error: implicit declaration of function 'migrate_page_move_mapping' [-Werror=implicit-function-declaration]
     rc = migrate_page_move_mapping(mapping, newpage, page, NULL, mode, 0);
     ^
   cc1: some warnings being treated as errors

vim +/migrate_page_move_mapping +1461 fs/ubifs/file.c

  1455	
  1456	static int ubifs_migrate_page(struct address_space *mapping,
  1457			struct page *newpage, struct page *page, enum migrate_mode mode)
  1458	{
  1459		int rc;
  1460	
> 1461		rc = migrate_page_move_mapping(mapping, newpage, page, NULL, mode, 0);
  1462		if (rc != MIGRATEPAGE_SUCCESS)
  1463			return rc;
  1464	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 12169 bytes --]