From: Anshuman Khandual <khandual@linux.vnet.ibm.com>
To: Minchan Kim <minchan@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
Rik van Riel <riel@redhat.com>, Vlastimil Babka <vbabka@suse.cz>,
Joonsoo Kim <iamjoonsoo.kim@lge.com>,
Mel Gorman <mgorman@suse.de>, Hugh Dickins <hughd@google.com>,
Rafael Aquini <aquini@redhat.com>,
virtualization@lists.linux-foundation.org,
Jonathan Corbet <corbet@lwn.net>,
John Einar Reitan <john.reitan@foss.arm.com>,
dri-devel@lists.freedesktop.org,
Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
Gioh Kim <gi-oh.kim@profitbricks.com>
Subject: Re: [PATCH v6v3 02/12] mm: migrate: support non-lru movable page migration
Date: Thu, 30 Jun 2016 11:26:45 +0530 [thread overview]
Message-ID: <5774B49D.6080000@linux.vnet.ibm.com> (raw)
In-Reply-To: <20160628063912.GA25560@bbox>
On 06/28/2016 12:09 PM, Minchan Kim wrote:
> On Mon, Jun 27, 2016 at 11:21:01AM +0530, Anshuman Khandual wrote:
>> On 06/16/2016 11:07 AM, Minchan Kim wrote:
>>> On Thu, Jun 16, 2016 at 09:12:07AM +0530, Anshuman Khandual wrote:
>>>> On 06/16/2016 05:56 AM, Minchan Kim wrote:
>>>>> On Wed, Jun 15, 2016 at 12:15:04PM +0530, Anshuman Khandual wrote:
>>>>>> On 06/15/2016 08:02 AM, Minchan Kim wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> On Mon, Jun 13, 2016 at 03:08:19PM +0530, Anshuman Khandual wrote:
>>>>>>>>> On 05/31/2016 05:31 AM, Minchan Kim wrote:
>>>>>>>>>>> @@ -791,6 +921,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
>>>>>>>>>>> int rc = -EAGAIN;
>>>>>>>>>>> int page_was_mapped = 0;
>>>>>>>>>>> struct anon_vma *anon_vma = NULL;
>>>>>>>>>>> + bool is_lru = !__PageMovable(page);
>>>>>>>>>>>
>>>>>>>>>>> if (!trylock_page(page)) {
>>>>>>>>>>> if (!force || mode == MIGRATE_ASYNC)
>>>>>>>>>>> @@ -871,6 +1002,11 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
>>>>>>>>>>> goto out_unlock_both;
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> + if (unlikely(!is_lru)) {
>>>>>>>>>>> + rc = move_to_new_page(newpage, page, mode);
>>>>>>>>>>> + goto out_unlock_both;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>
>>>>>>>>> Hello Minchan,
>>>>>>>>>
>>>>>>>>> I might be missing something here but does this implementation support the
>>>>>>>>> scenario where these non LRU pages owned by the driver mapped as PTE into
>>>>>>>>> process page table ? Because the "goto out_unlock_both" statement above
>>>>>>>>> skips all the PTE unmap, putting a migration PTE and removing the migration
>>>>>>>>> PTE steps.
>>>>>>> You're right. Unfortunately, it doesn't support right now but surely,
>>>>>>> it's my TODO after landing this work.
>>>>>>>
>>>>>>> Could you share your usecase?
>>>>>>
>>>>>> Sure.
>>>>>
>>>>> Thanks a lot!
>>>>>
>>>>>>
>>>>>> My driver has privately managed non LRU pages which gets mapped into user space
>>>>>> process page table through f_ops->mmap() and vmops->fault() which then updates
>>>>>> the file RMAP (page->mapping->i_mmap) through page_add_file_rmap(page). One thing
>>>>>
>>>>> Hmm, page_add_file_rmap is not exported function. How does your driver can use it?
>>>>
>>>> Its not using the function directly, I just re-iterated the sequence of functions
>>>> above. (do_set_pte -> page_add_file_rmap) gets called after we grab the page from
>>>> driver through (__do_fault->vma->vm_ops->fault()).
>>>>
>>>>> Do you use vm_insert_pfn?
>>>>> What type your vma is? VM_PFNMMAP or VM_MIXEDMAP?
>>>>
>>>> I dont use vm_insert_pfn(). Here is the sequence of events how the user space
>>>> VMA gets the non LRU pages from the driver.
>>>>
>>>> - Driver registers a character device with 'struct file_operations' binding
>>>> - Then the 'fops->mmap()' just binds the incoming 'struct vma' with a 'struct
>>>> vm_operations_struct' which provides the 'vmops->fault()' routine which
>>>> basically traps all page faults on the VMA and provides one page at a time
>>>> through a driver specific allocation routine which hands over non LRU pages
>>>>
>>>> The VMA is not anything special as such. Its what we get when we try to do a
>>>> simple mmap() on a file descriptor pointing to a character device. I can
>>>> figure out all the VM_* flags it holds after creation.
>>>>
>>>>>
>>>>> I want to make dummy driver to simulate your case.
>>>>
>>>> Sure. I hope the above mentioned steps will help you but in case you need more
>>>> information, please do let me know.
>>>
>>> I got understood now. :)
>>> I will test it with dummy driver and will Cc'ed when I send a patch.
>>
>> Hello Minchan,
>>
>> Do you have any updates on this ? The V7 of the series still has this limitation.
>> Did you get a chance to test the driver out ? I am still concerned about how to
>> handle the struct address_space override problem within the struct page.
>
> Hi Anshuman,
>
> Slow but I am working on that. :) However, as I said, I want to do it
I really appreciate. Was just curious about the problem and any potential
solution we can look into.
> after soft landing of current non-lru-no-mapped page migration to solve
> current real field issues.
yeah it makes sense.
>
> About the overriding problem of non-lru-mapped-page, I implemented dummy
> driver as miscellaneous device and in test_mmap(file_operations.mmap),
> I changed a_ops with my address_space_operations.
>
> int test_mmap(struct file *filp, struct vm_area_struct *vma)
> {
> filp->f_mapping->a_ops = &test_aops;
> vma->vm_ops = &test_vm_ops;
> vma->vm_private_data = filp->private_data;
> return 0;
> }
>
Okay.
> test_aops should have *set_page_dirty* overriding.
>
> static int test_set_pag_dirty(struct page *page)
> {
> if (!PageDirty(page))
> SetPageDirty*page);
> return 0;
> }
>
> Otherwise, it goes BUG_ON during radix tree operation because
> currently try_to_unmap is designed for file-lru pages which lives
> in page cache so it propagates page table dirty bit to PG_dirty flag
> of struct page by set_page_dirty. And set_page_dirty want to mark
> dirty tag in radix tree node but it's character driver so the page
> cache doesn't have it. That's why we encounter BUG_ON in radix tree
> operation. Anyway, to test, I implemented set_page_dirty in my dummy
> driver.
Okay and the above test_set_page_dirty() example is sufficient ?
>
> With only that, it doesn't work because I need to modify migrate.c to
> work non-lru-mapped-page and changing PG_isolated flag which is
> override of PG_reclaim which is cleared in set_page_dirty.
Got it, so what changes you did ? Implemented PG_isolated differently
not by overriding PG_reclaim or something else ? Yes set_page_dirty
indeed clears the PG_reclaim flag.
>
> With that, it seems to work. But I'm not saying it's right model now
So the mapped pages migration was successful ? Even after overloading
filp->f_mapping->a_ops = &test_aops, we still have the RMAP information
intact with filp->f_mappinp pointed interval tree. But would really like
to see the code changes.
> for device drivers. In runtime, replacing filp->f_mapping->a_ops with
> custom a_ops of own driver seems to be hacky to me.
Yeah I thought so.
> So, I'm considering now new pseudo fs "movable_inode" which will
> support
>
> struct file *movable_inode_getfile(const char *name,
> const struct file_operations *fop,
> const struct address_space_operations *a_ops)
> {
> struct path path;
> struct qstr this;
> struct inode *inode;
> struct super_block *sb;
>
> this.name = name;
> this.len = strlen(name);
> this.hash = 0;
> sb = movable_mnt.mnt_sb;
> patch.denty = d_alloc_pseudo(movable_inode_mnt->mnt_sb, &this);
> patch.mnt = mntget(movable_inode_mnt);
>
> inode = new_inode(sb);
> ..
> ..
> inode->i_mapping->a_ops = a_ops;
> d_instantiate(path.dentry, inode);
>
> return alloc_file(&path, FMODE_WRITE | FMODE_READ, f_op);
> }
>
> And in our driver, we can change vma->vm_file with new one.
>
> int test_mmap(struct file *filp, struct vm_area_structd *vma)
> {
> struct file *newfile = movable_inode_getfile("[test"],
> filep->f_op, &test_aops);
> vma->vm_file = newfile;
> ..
> ..
> }
>
> When I read mmap_region in mm/mmap.c, it's reasonable usecase
> which dirver's mmap changes vma->vm_file with own file.
I will look into these details.
> Anyway, it needs many subtle changes in mm/vfs/driver side so
> need to review from each maintainers related subsystem so I
> want to not be hurry.
Sure, makes sense. Mean while it will be really great if you could share
your code changes as described above, so that I can try them out.
next prev parent reply other threads:[~2016-06-30 5:57 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-05-20 14:23 [PATCH v6 00/12] Support non-lru page migration Minchan Kim
2016-05-20 14:23 ` [PATCH v6 01/12] mm: use put_page to free page instead of putback_lru_page Minchan Kim
2016-05-20 14:23 ` [PATCH v6 02/12] mm: migrate: support non-lru movable page migration Minchan Kim
2016-05-27 14:26 ` Vlastimil Babka
2016-05-30 1:33 ` Minchan Kim
2016-05-30 9:01 ` Vlastimil Babka
2016-05-30 1:39 ` PATCH v6v2 " Minchan Kim
2016-05-30 9:36 ` Vlastimil Babka
2016-05-30 16:25 ` Minchan Kim
2016-05-31 7:51 ` Vlastimil Babka
2016-05-31 0:01 ` [PATCH v6v3 " Minchan Kim
2016-05-31 7:52 ` Vlastimil Babka
2016-05-31 23:05 ` Minchan Kim
2016-06-13 9:38 ` Anshuman Khandual
2016-06-15 2:32 ` Minchan Kim
2016-06-15 6:45 ` Anshuman Khandual
2016-06-16 0:26 ` Minchan Kim
2016-06-16 3:42 ` Anshuman Khandual
2016-06-16 5:37 ` Minchan Kim
2016-06-27 5:51 ` Anshuman Khandual
2016-06-28 6:39 ` Minchan Kim
2016-06-30 5:56 ` Anshuman Khandual [this message]
2016-06-30 6:18 ` Minchan Kim
2016-05-20 14:23 ` [PATCH v6 03/12] mm: balloon: use general non-lru movable page feature Minchan Kim
2016-05-30 12:16 ` Vlastimil Babka
2016-05-20 14:23 ` [PATCH v6 04/12] zsmalloc: keep max_object in size_class Minchan Kim
2016-05-20 14:23 ` [PATCH v6 05/12] zsmalloc: use bit_spin_lock Minchan Kim
2016-05-20 14:23 ` [PATCH v6 06/12] zsmalloc: use accessor Minchan Kim
2016-05-20 14:23 ` [PATCH v6 07/12] zsmalloc: factor page chain functionality out Minchan Kim
2016-05-20 14:23 ` [PATCH v6 08/12] zsmalloc: introduce zspage structure Minchan Kim
2016-05-20 14:23 ` [PATCH v6 09/12] zsmalloc: separate free_zspage from putback_zspage Minchan Kim
2016-05-20 14:23 ` [PATCH v6 10/12] zsmalloc: use freeobj for index Minchan Kim
2016-05-20 14:23 ` [PATCH v6 11/12] zsmalloc: page migration support Minchan Kim
2016-05-24 5:28 ` Sergey Senozhatsky
2016-05-24 6:28 ` Minchan Kim
2016-05-24 8:05 ` Sergey Senozhatsky
2016-05-24 8:17 ` Minchan Kim
2016-05-25 5:14 ` Minchan Kim
2016-05-25 15:23 ` Sergey Senozhatsky
2016-05-26 0:32 ` Minchan Kim
2016-05-26 0:59 ` Sergey Senozhatsky
2016-05-26 4:37 ` Minchan Kim
2016-05-26 21:50 ` [PATCH v6r2 " Minchan Kim
2016-05-20 14:23 ` [PATCH v6 12/12] zram: use __GFP_MOVABLE for memory allocation Minchan Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5774B49D.6080000@linux.vnet.ibm.com \
--to=khandual@linux.vnet.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=aquini@redhat.com \
--cc=corbet@lwn.net \
--cc=dri-devel@lists.freedesktop.org \
--cc=gi-oh.kim@profitbricks.com \
--cc=hughd@google.com \
--cc=iamjoonsoo.kim@lge.com \
--cc=john.reitan@foss.arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=minchan@kernel.org \
--cc=riel@redhat.com \
--cc=sergey.senozhatsky@gmail.com \
--cc=vbabka@suse.cz \
--cc=virtualization@lists.linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).