From: Gioh Kim <gioh.kim@lge.com> To: jlayton@poochiereds.net, bfields@fieldses.org, vbabka@suse.cz, iamjoonsoo.kim@lge.com, viro@zeniv.linux.org.uk, mst@redhat.com, koct9i@gmail.com, minchan@kernel.org, aquini@redhat.com, linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, linux-api@vger.kernel.org, linux-mm@kvack.org Cc: gunho.lee@lge.com, akpm@linux-foundation.org, Gioh Kim <gurugio@hanmail.net> Subject: [RFCv3 0/5] enable migration of driver pages Date: Tue, 7 Jul 2015 13:36:20 +0900 [thread overview] Message-ID: <1436243785-24105-1-git-send-email-gioh.kim@lge.com> (raw) From: Gioh Kim <gurugio@hanmail.net> Hello, This series try to enable migration of non-LRU pages, such as driver's page. My ARM-based platform occured severe fragmentation problem after long-term (several days) test. Sometimes even order-3 page allocation failed. It has memory size 512MB ~ 1024MB. 30% ~ 40% memory is consumed for graphic processing and 20~30 memory is reserved for zram. I found that many pages of GPU driver and zram are non-movable pages. So I reported Minchan Kim, the maintainer of zram, and he made the internal compaction logic of zram. And I made the internal compaction of GPU driver. They reduced some fragmentation but they are not enough effective. They are activated by its own interface, /sys, so they are not cooperative with kernel compaction. If there is too much fragmentation and kernel starts to compaction, zram and GPU driver cannot work with the kernel compaction. This patch set combines 5 patches. 1. patch 1/5: get inode from anon_inodes This patch adds new interface to create inode from anon_inodes. 2. patch 2/5: framework to isolate/migrate/putback page Add isolatepage, putbackpage into address_space_operations and wrapper function to call them 3. patch 3/5: apply the framework into balloon driver The balloon driver is applied into the framework. It gets a inode from anon_inodes and register operations in the inode. Any other drivers can register operations via inode like this to migrate it's pages. 4. patch 4/5: compaction/migration call the generic interfaces Compaction and migration pages call the generic interfaces of the framework, instead of calling balloon migration directly. 5. patch 5/5: remove direct calling of migration of driver pages Non-lru pages are migrated with lru pages by move_to_new_page(). This patch set is tested: - turn on Ubuntu 14.04 with 1G memory on qemu. - do kernel building - after several seconds check more than 512MB is used with free command - command "balloon 512" in qemu monitor - check hundreds MB of pages are migrated My thanks to Konstantin Khlebnikov for his reviews of the v2 patch set. Most of the changes were based on his feedback. Changes since v2: - change the name of page type from migratable page into mobile page - get and lock page to isolate page - add wrapper interfaces for page->mapping->a_ops->isolate/putback - leave balloon pages marked as balloon This patch-set is based on v4.1 Gioh Kim (5): fs/anon_inodes: new interface to create new inode mm/compaction: enable mobile-page migration mm/balloon: apply mobile page migratable into balloon mm/compaction: call generic migration callbacks mm: remove direct calling of migration drivers/virtio/virtio_balloon.c | 3 ++ fs/anon_inodes.c | 6 +++ fs/proc/page.c | 3 ++ include/linux/anon_inodes.h | 1 + include/linux/balloon_compaction.h | 15 +++++-- include/linux/compaction.h | 76 ++++++++++++++++++++++++++++++++++ include/linux/fs.h | 2 + include/linux/page-flags.h | 19 +++++++++ include/uapi/linux/kernel-page-flags.h | 1 + mm/balloon_compaction.c | 71 ++++++++++--------------------- mm/compaction.c | 8 ++-- mm/migrate.c | 24 +++-------- 12 files changed, 154 insertions(+), 75 deletions(-) -- 2.1.4
WARNING: multiple messages have this Message-ID (diff)
From: Gioh Kim <gioh.kim@lge.com> To: jlayton@poochiereds.net, bfields@fieldses.org, vbabka@suse.cz, iamjoonsoo.kim@lge.com, viro@zeniv.linux.org.uk, mst@redhat.com, koct9i@gmail.com, minchan@kernel.org, aquini@redhat.com, linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, linux-api@vger.kernel.org, linux-mm@kvack.org Cc: gunho.lee@lge.com, akpm@linux-foundation.org, Gioh Kim <gurugio@hanmail.net> Subject: [RFCv3 0/5] enable migration of driver pages Date: Tue, 7 Jul 2015 13:36:20 +0900 [thread overview] Message-ID: <1436243785-24105-1-git-send-email-gioh.kim@lge.com> (raw) From: Gioh Kim <gurugio@hanmail.net> Hello, This series try to enable migration of non-LRU pages, such as driver's page. My ARM-based platform occured severe fragmentation problem after long-term (several days) test. Sometimes even order-3 page allocation failed. It has memory size 512MB ~ 1024MB. 30% ~ 40% memory is consumed for graphic processing and 20~30 memory is reserved for zram. I found that many pages of GPU driver and zram are non-movable pages. So I reported Minchan Kim, the maintainer of zram, and he made the internal compaction logic of zram. And I made the internal compaction of GPU driver. They reduced some fragmentation but they are not enough effective. They are activated by its own interface, /sys, so they are not cooperative with kernel compaction. If there is too much fragmentation and kernel starts to compaction, zram and GPU driver cannot work with the kernel compaction. This patch set combines 5 patches. 1. patch 1/5: get inode from anon_inodes This patch adds new interface to create inode from anon_inodes. 2. patch 2/5: framework to isolate/migrate/putback page Add isolatepage, putbackpage into address_space_operations and wrapper function to call them 3. patch 3/5: apply the framework into balloon driver The balloon driver is applied into the framework. It gets a inode from anon_inodes and register operations in the inode. Any other drivers can register operations via inode like this to migrate it's pages. 4. patch 4/5: compaction/migration call the generic interfaces Compaction and migration pages call the generic interfaces of the framework, instead of calling balloon migration directly. 5. patch 5/5: remove direct calling of migration of driver pages Non-lru pages are migrated with lru pages by move_to_new_page(). This patch set is tested: - turn on Ubuntu 14.04 with 1G memory on qemu. - do kernel building - after several seconds check more than 512MB is used with free command - command "balloon 512" in qemu monitor - check hundreds MB of pages are migrated My thanks to Konstantin Khlebnikov for his reviews of the v2 patch set. Most of the changes were based on his feedback. Changes since v2: - change the name of page type from migratable page into mobile page - get and lock page to isolate page - add wrapper interfaces for page->mapping->a_ops->isolate/putback - leave balloon pages marked as balloon This patch-set is based on v4.1 Gioh Kim (5): fs/anon_inodes: new interface to create new inode mm/compaction: enable mobile-page migration mm/balloon: apply mobile page migratable into balloon mm/compaction: call generic migration callbacks mm: remove direct calling of migration drivers/virtio/virtio_balloon.c | 3 ++ fs/anon_inodes.c | 6 +++ fs/proc/page.c | 3 ++ include/linux/anon_inodes.h | 1 + include/linux/balloon_compaction.h | 15 +++++-- include/linux/compaction.h | 76 ++++++++++++++++++++++++++++++++++ include/linux/fs.h | 2 + include/linux/page-flags.h | 19 +++++++++ include/uapi/linux/kernel-page-flags.h | 1 + mm/balloon_compaction.c | 71 ++++++++++--------------------- mm/compaction.c | 8 ++-- mm/migrate.c | 24 +++-------- 12 files changed, 154 insertions(+), 75 deletions(-) -- 2.1.4 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next reply other threads:[~2015-07-07 4:34 UTC|newest] Thread overview: 82+ messages / expand[flat|nested] mbox.gz Atom feed top 2015-07-07 4:36 Gioh Kim [this message] 2015-07-07 4:36 ` [RFCv3 0/5] enable migration of driver pages Gioh Kim 2015-07-07 4:36 ` [RFCv3 1/5] fs/anon_inodes: new interface to create new inode Gioh Kim 2015-07-07 4:36 ` Gioh Kim 2015-07-07 4:36 ` [RFCv3 2/5] mm/compaction: enable mobile-page migration Gioh Kim 2015-07-07 4:36 ` Gioh Kim 2015-07-10 13:07 ` Konstantin Khlebnikov 2015-07-10 13:07 ` Konstantin Khlebnikov 2015-07-13 8:45 ` Gioh Kim 2015-07-13 8:45 ` Gioh Kim 2015-07-13 8:45 ` Gioh Kim 2015-07-10 13:07 ` Konstantin Khlebnikov 2015-07-10 13:41 ` Konstantin Khlebnikov 2015-07-10 13:41 ` Konstantin Khlebnikov 2015-07-10 13:41 ` Konstantin Khlebnikov 2015-07-07 4:36 ` Gioh Kim 2015-07-07 4:36 ` [RFCv3 3/5] mm/balloon: apply mobile page migratable into balloon Gioh Kim 2015-07-07 4:36 ` Gioh Kim 2015-07-07 4:36 ` Gioh Kim 2015-07-09 8:07 ` Michael S. Tsirkin 2015-07-09 8:07 ` Michael S. Tsirkin 2015-07-09 8:07 ` Michael S. Tsirkin 2015-07-09 8:07 ` Michael S. Tsirkin 2015-07-10 0:21 ` Gioh Kim 2015-07-10 0:21 ` Gioh Kim 2015-07-10 0:21 ` Gioh Kim 2015-07-10 13:29 ` Konstantin Khlebnikov 2015-07-10 13:29 ` Konstantin Khlebnikov 2015-07-10 13:29 ` Konstantin Khlebnikov 2015-07-10 13:29 ` Konstantin Khlebnikov 2015-07-07 4:36 ` [RFCv3 4/5] mm: call generic migration callbacks Gioh Kim 2015-07-07 4:36 ` Gioh Kim 2015-07-07 4:36 ` Gioh Kim 2015-07-07 4:36 ` [RFCv3 5/5] mm: remove direct calling of migration Gioh Kim 2015-07-07 4:36 ` Gioh Kim 2015-07-07 4:36 ` Gioh Kim 2015-07-07 22:37 ` [RFCv3 0/5] enable migration of driver pages Andrew Morton 2015-07-07 22:37 ` Andrew Morton 2015-07-07 22:37 ` Andrew Morton 2015-07-08 0:02 ` Gioh Kim 2015-07-08 0:02 ` Gioh Kim 2015-07-08 0:02 ` Gioh Kim 2015-07-08 0:07 ` Andrew Morton 2015-07-08 0:07 ` Andrew Morton 2015-07-08 0:07 ` Andrew Morton 2015-07-08 0:19 ` Gioh Kim 2015-07-08 0:19 ` Gioh Kim 2015-07-08 0:19 ` Gioh Kim 2015-07-08 0:35 ` Minchan Kim 2015-07-08 0:35 ` Minchan Kim 2015-07-08 0:35 ` Minchan Kim 2015-07-08 22:47 ` Dave Airlie 2015-07-08 22:47 ` Dave Airlie 2015-07-08 22:47 ` Dave Airlie 2015-07-08 22:47 ` Dave Airlie 2015-07-08 23:55 ` Gioh Kim 2015-07-08 23:55 ` Gioh Kim 2015-07-08 23:55 ` Gioh Kim 2015-07-09 13:08 ` Daniel Vetter 2015-07-09 13:08 ` Daniel Vetter 2015-07-09 13:08 ` Daniel Vetter 2015-07-09 13:08 ` Daniel Vetter 2015-07-09 13:33 ` Ville Syrjälä 2015-07-09 13:33 ` Ville Syrjälä 2015-07-09 13:33 ` Ville Syrjälä 2015-07-09 14:02 ` Kirill A. Shutemov 2015-07-09 14:02 ` Kirill A. Shutemov 2015-07-09 14:02 ` Kirill A. Shutemov 2015-07-09 14:02 ` Kirill A. Shutemov 2015-07-09 13:33 ` Ville Syrjälä 2015-07-10 0:02 ` Gioh Kim 2015-07-10 0:02 ` Gioh Kim 2015-07-10 0:02 ` Gioh Kim 2015-07-10 0:02 ` Gioh Kim 2015-07-08 23:55 ` Gioh Kim 2015-07-08 0:07 ` Andrew Morton 2015-07-08 0:02 ` Gioh Kim 2015-07-09 1:00 ` Rafael Aquini 2015-07-09 1:00 ` Rafael Aquini 2015-07-09 1:00 ` Rafael Aquini 2015-07-09 1:00 ` Rafael Aquini -- strict thread matches above, loose matches on Subject: below -- 2015-07-07 4:36 Gioh Kim
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1436243785-24105-1-git-send-email-gioh.kim@lge.com \ --to=gioh.kim@lge.com \ --cc=akpm@linux-foundation.org \ --cc=aquini@redhat.com \ --cc=bfields@fieldses.org \ --cc=gunho.lee@lge.com \ --cc=gurugio@hanmail.net \ --cc=iamjoonsoo.kim@lge.com \ --cc=jlayton@poochiereds.net \ --cc=koct9i@gmail.com \ --cc=linux-api@vger.kernel.org \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=minchan@kernel.org \ --cc=mst@redhat.com \ --cc=vbabka@suse.cz \ --cc=viro@zeniv.linux.org.uk \ --cc=virtualization@lists.linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.