All of lore.kernel.org
 help / color / mirror / Atom feed
From: Liang Li <liang.z.li@intel.com>
To: mst@redhat.com
Cc: linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org, linux-mm@kvack.org,
	virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org,
	qemu-devel@nongnu.org, quintela@redhat.com, dgilbert@redhat.com,
	dave.hansen@intel.com, pbonzini@redhat.com,
	cornelia.huck@de.ibm.com, amit.shah@redhat.com,
	Liang Li <liang.z.li@intel.com>
Subject: [RESEND PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration
Date: Fri, 21 Oct 2016 14:24:33 +0800	[thread overview]
Message-ID: <1477031080-12616-1-git-send-email-liang.z.li@intel.com> (raw)

This patch set contains two parts of changes to the virtio-balloon.
 
One is the change for speeding up the inflating & deflating process,
the main idea of this optimization is to use bitmap to send the page
information to host instead of the PFNs, to reduce the overhead of
virtio data transmission, address translation and madvise(). This can
help to improve the performance by about 85%.
 
Another change is for speeding up live migration. By skipping process
guest's free pages in the first round of data copy, to reduce needless
data processing, this can help to save quite a lot of CPU cycles and
network bandwidth. We put guest's free page information in bitmap and
send it to host with the virt queue of virtio-balloon. For an idle 8GB
guest, this can help to shorten the total live migration time from 2Sec
to about 500ms in the 10Gbps network environment.
 
Dave Hansen suggested a new scheme to encode the data structure,
because of additional complexity, it's not implemented in v3.
 
Changes from v2 to v3:
    * Change the name of 'free page' to 'unused page'.
    * Use the scatter & gather bitmap instead of a 1MB page bitmap.
    * Fix overwriting the page bitmap after kicking.
    * Some of MST's comments for v2.
 
Changes from v1 to v2:
    * Abandon the patch for dropping page cache.
    * Put some structures to uapi head file.
    * Use a new way to determine the page bitmap size.
    * Use a unified way to send the free page information with the bitmap
    * Address the issues referred in MST's comments

Liang Li (7):
  virtio-balloon: rework deflate to add page to a list
  virtio-balloon: define new feature bit and page bitmap head
  mm: add a function to get the max pfn
  virtio-balloon: speed up inflate/deflate process
  mm: add the related functions to get unused page
  virtio-balloon: define feature bit and head for misc virt queue
  virtio-balloon: tell host vm's unused page info

 drivers/virtio/virtio_balloon.c     | 390 ++++++++++++++++++++++++++++++++----
 include/linux/mm.h                  |   3 +
 include/uapi/linux/virtio_balloon.h |  41 ++++
 mm/page_alloc.c                     |  94 +++++++++
 4 files changed, 485 insertions(+), 43 deletions(-)

-- 
1.8.3.1

WARNING: multiple messages have this Message-ID (diff)
From: Liang Li <liang.z.li@intel.com>
To: mst@redhat.com
Cc: virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org,
	amit.shah@redhat.com, Liang Li <liang.z.li@intel.com>,
	dave.hansen@intel.com, qemu-devel@nongnu.org,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	pbonzini@redhat.com, virtualization@lists.linux-foundation.org,
	dgilbert@redhat.com
Subject: [RESEND PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration
Date: Fri, 21 Oct 2016 14:24:33 +0800	[thread overview]
Message-ID: <1477031080-12616-1-git-send-email-liang.z.li@intel.com> (raw)

This patch set contains two parts of changes to the virtio-balloon.
 
One is the change for speeding up the inflating & deflating process,
the main idea of this optimization is to use bitmap to send the page
information to host instead of the PFNs, to reduce the overhead of
virtio data transmission, address translation and madvise(). This can
help to improve the performance by about 85%.
 
Another change is for speeding up live migration. By skipping process
guest's free pages in the first round of data copy, to reduce needless
data processing, this can help to save quite a lot of CPU cycles and
network bandwidth. We put guest's free page information in bitmap and
send it to host with the virt queue of virtio-balloon. For an idle 8GB
guest, this can help to shorten the total live migration time from 2Sec
to about 500ms in the 10Gbps network environment.
 
Dave Hansen suggested a new scheme to encode the data structure,
because of additional complexity, it's not implemented in v3.
 
Changes from v2 to v3:
    * Change the name of 'free page' to 'unused page'.
    * Use the scatter & gather bitmap instead of a 1MB page bitmap.
    * Fix overwriting the page bitmap after kicking.
    * Some of MST's comments for v2.
 
Changes from v1 to v2:
    * Abandon the patch for dropping page cache.
    * Put some structures to uapi head file.
    * Use a new way to determine the page bitmap size.
    * Use a unified way to send the free page information with the bitmap
    * Address the issues referred in MST's comments

Liang Li (7):
  virtio-balloon: rework deflate to add page to a list
  virtio-balloon: define new feature bit and page bitmap head
  mm: add a function to get the max pfn
  virtio-balloon: speed up inflate/deflate process
  mm: add the related functions to get unused page
  virtio-balloon: define feature bit and head for misc virt queue
  virtio-balloon: tell host vm's unused page info

 drivers/virtio/virtio_balloon.c     | 390 ++++++++++++++++++++++++++++++++----
 include/linux/mm.h                  |   3 +
 include/uapi/linux/virtio_balloon.h |  41 ++++
 mm/page_alloc.c                     |  94 +++++++++
 4 files changed, 485 insertions(+), 43 deletions(-)

-- 
1.8.3.1

WARNING: multiple messages have this Message-ID (diff)
From: Liang Li <liang.z.li@intel.com>
To: mst@redhat.com
Cc: linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org, linux-mm@kvack.org,
	virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org,
	qemu-devel@nongnu.org, quintela@redhat.com, dgilbert@redhat.com,
	dave.hansen@intel.com, pbonzini@redhat.com,
	cornelia.huck@de.ibm.com, amit.shah@redhat.com,
	Liang Li <liang.z.li@intel.com>
Subject: [RESEND PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration
Date: Fri, 21 Oct 2016 14:24:33 +0800	[thread overview]
Message-ID: <1477031080-12616-1-git-send-email-liang.z.li@intel.com> (raw)

This patch set contains two parts of changes to the virtio-balloon.
 
One is the change for speeding up the inflating & deflating process,
the main idea of this optimization is to use bitmap to send the page
information to host instead of the PFNs, to reduce the overhead of
virtio data transmission, address translation and madvise(). This can
help to improve the performance by about 85%.
 
Another change is for speeding up live migration. By skipping process
guest's free pages in the first round of data copy, to reduce needless
data processing, this can help to save quite a lot of CPU cycles and
network bandwidth. We put guest's free page information in bitmap and
send it to host with the virt queue of virtio-balloon. For an idle 8GB
guest, this can help to shorten the total live migration time from 2Sec
to about 500ms in the 10Gbps network environment.
 
Dave Hansen suggested a new scheme to encode the data structure,
because of additional complexity, it's not implemented in v3.
 
Changes from v2 to v3:
    * Change the name of 'free page' to 'unused page'.
    * Use the scatter & gather bitmap instead of a 1MB page bitmap.
    * Fix overwriting the page bitmap after kicking.
    * Some of MST's comments for v2.
 
Changes from v1 to v2:
    * Abandon the patch for dropping page cache.
    * Put some structures to uapi head file.
    * Use a new way to determine the page bitmap size.
    * Use a unified way to send the free page information with the bitmap
    * Address the issues referred in MST's comments

Liang Li (7):
  virtio-balloon: rework deflate to add page to a list
  virtio-balloon: define new feature bit and page bitmap head
  mm: add a function to get the max pfn
  virtio-balloon: speed up inflate/deflate process
  mm: add the related functions to get unused page
  virtio-balloon: define feature bit and head for misc virt queue
  virtio-balloon: tell host vm's unused page info

 drivers/virtio/virtio_balloon.c     | 390 ++++++++++++++++++++++++++++++++----
 include/linux/mm.h                  |   3 +
 include/uapi/linux/virtio_balloon.h |  41 ++++
 mm/page_alloc.c                     |  94 +++++++++
 4 files changed, 485 insertions(+), 43 deletions(-)

-- 
1.8.3.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Liang Li <liang.z.li@intel.com>
To: mst@redhat.com
Cc: linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org, linux-mm@kvack.org,
	virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org,
	qemu-devel@nongnu.org, quintela@redhat.com, dgilbert@redhat.com,
	dave.hansen@intel.com, pbonzini@redhat.com,
	cornelia.huck@de.ibm.com, amit.shah@redhat.com,
	Liang Li <liang.z.li@intel.com>
Subject: [Qemu-devel] [RESEND PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration
Date: Fri, 21 Oct 2016 14:24:33 +0800	[thread overview]
Message-ID: <1477031080-12616-1-git-send-email-liang.z.li@intel.com> (raw)

This patch set contains two parts of changes to the virtio-balloon.
 
One is the change for speeding up the inflating & deflating process,
the main idea of this optimization is to use bitmap to send the page
information to host instead of the PFNs, to reduce the overhead of
virtio data transmission, address translation and madvise(). This can
help to improve the performance by about 85%.
 
Another change is for speeding up live migration. By skipping process
guest's free pages in the first round of data copy, to reduce needless
data processing, this can help to save quite a lot of CPU cycles and
network bandwidth. We put guest's free page information in bitmap and
send it to host with the virt queue of virtio-balloon. For an idle 8GB
guest, this can help to shorten the total live migration time from 2Sec
to about 500ms in the 10Gbps network environment.
 
Dave Hansen suggested a new scheme to encode the data structure,
because of additional complexity, it's not implemented in v3.
 
Changes from v2 to v3:
    * Change the name of 'free page' to 'unused page'.
    * Use the scatter & gather bitmap instead of a 1MB page bitmap.
    * Fix overwriting the page bitmap after kicking.
    * Some of MST's comments for v2.
 
Changes from v1 to v2:
    * Abandon the patch for dropping page cache.
    * Put some structures to uapi head file.
    * Use a new way to determine the page bitmap size.
    * Use a unified way to send the free page information with the bitmap
    * Address the issues referred in MST's comments

Liang Li (7):
  virtio-balloon: rework deflate to add page to a list
  virtio-balloon: define new feature bit and page bitmap head
  mm: add a function to get the max pfn
  virtio-balloon: speed up inflate/deflate process
  mm: add the related functions to get unused page
  virtio-balloon: define feature bit and head for misc virt queue
  virtio-balloon: tell host vm's unused page info

 drivers/virtio/virtio_balloon.c     | 390 ++++++++++++++++++++++++++++++++----
 include/linux/mm.h                  |   3 +
 include/uapi/linux/virtio_balloon.h |  41 ++++
 mm/page_alloc.c                     |  94 +++++++++
 4 files changed, 485 insertions(+), 43 deletions(-)

-- 
1.8.3.1

             reply	other threads:[~2016-10-21  6:37 UTC|newest]

Thread overview: 117+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-10-21  6:24 Liang Li [this message]
2016-10-21  6:24 ` [Qemu-devel] [RESEND PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration Liang Li
2016-10-21  6:24 ` Liang Li
2016-10-21  6:24 ` Liang Li
2016-10-21  6:24 ` [RESEND PATCH v3 kernel 1/7] virtio-balloon: rework deflate to add page to a list Liang Li
2016-10-21  6:24   ` [Qemu-devel] " Liang Li
2016-10-21  6:24   ` Liang Li
2016-10-24 16:46   ` Dave Hansen
2016-10-24 16:46   ` Dave Hansen
2016-10-24 16:46     ` [Qemu-devel] " Dave Hansen
2016-10-24 16:46     ` Dave Hansen
2016-10-25  1:14     ` Li, Liang Z
2016-10-25  1:14       ` [Qemu-devel] " Li, Liang Z
2016-10-25  1:14       ` Li, Liang Z
2016-10-25  1:14       ` Li, Liang Z
2016-10-21  6:24 ` Liang Li
2016-10-21  6:24 ` [RESEND PATCH v3 kernel 2/7] virtio-balloon: define new feature bit and page bitmap head Liang Li
2016-10-21  6:24   ` [Qemu-devel] " Liang Li
2016-10-21  6:24   ` Liang Li
2016-10-24 16:51   ` Dave Hansen
2016-10-24 16:51     ` [Qemu-devel] " Dave Hansen
2016-10-24 16:51     ` Dave Hansen
2016-10-24 16:51     ` Dave Hansen
2016-10-25  1:21     ` Li, Liang Z
2016-10-25  1:21       ` [Qemu-devel] " Li, Liang Z
2016-10-25  1:21       ` Li, Liang Z
2016-10-25  1:21       ` Li, Liang Z
2016-10-26 15:35   ` Michael S. Tsirkin
2016-10-26 15:35     ` [Qemu-devel] " Michael S. Tsirkin
2016-10-26 15:35     ` Michael S. Tsirkin
2016-10-26 15:35     ` Michael S. Tsirkin
2016-10-21  6:24 ` Liang Li
2016-10-21  6:24 ` [RESEND PATCH v3 kernel 3/7] mm: add a function to get the max pfn Liang Li
2016-10-21  6:24   ` [Qemu-devel] " Liang Li
2016-10-21  6:24   ` Liang Li
2016-10-24 16:53   ` Dave Hansen
2016-10-24 16:53     ` [Qemu-devel] " Dave Hansen
2016-10-24 16:53     ` Dave Hansen
2016-10-25  1:24     ` Li, Liang Z
2016-10-25  1:24     ` Li, Liang Z
2016-10-25  1:24       ` [Qemu-devel] " Li, Liang Z
2016-10-25  1:24       ` Li, Liang Z
2016-10-25  1:24       ` Li, Liang Z
2016-10-24 16:53   ` Dave Hansen
2016-10-21  6:24 ` Liang Li
2016-10-21  6:24 ` [RESEND PATCH v3 kernel 4/7] virtio-balloon: speed up inflate/deflate process Liang Li
2016-10-21  6:24 ` Liang Li
2016-10-21  6:24   ` [Qemu-devel] " Liang Li
2016-10-21  6:24   ` Liang Li
2016-10-25  6:36   ` Michael S. Tsirkin
2016-10-25  6:36     ` [Qemu-devel] " Michael S. Tsirkin
2016-10-25  6:36     ` Michael S. Tsirkin
2016-10-25  6:36     ` Michael S. Tsirkin
2016-10-25  9:46     ` Li, Liang Z
2016-10-25  9:46       ` [Qemu-devel] " Li, Liang Z
2016-10-25  9:46       ` Li, Liang Z
2016-10-25  9:46     ` Li, Liang Z
2016-10-21  6:24 ` [RESEND PATCH v3 kernel 5/7] mm: add the related functions to get unused page Liang Li
2016-10-21  6:24   ` [Qemu-devel] " Liang Li
2016-10-21  6:24   ` Liang Li
2016-10-21  6:24 ` Liang Li
2016-10-21  6:24 ` [RESEND PATCH v3 kernel 6/7] virtio-balloon: define feature bit and head for misc virt queue Liang Li
2016-10-21  6:24 ` Liang Li
2016-10-21  6:24   ` [Qemu-devel] " Liang Li
2016-10-21  6:24   ` Liang Li
2016-10-27 18:29   ` Michael S. Tsirkin
2016-10-27 18:29     ` [Qemu-devel] " Michael S. Tsirkin
2016-10-27 18:29     ` Michael S. Tsirkin
2016-10-27 18:29   ` Michael S. Tsirkin
2016-10-21  6:24 ` [RESEND PATCH v3 kernel 7/7] virtio-balloon: tell host vm's unused page info Liang Li
2016-10-21  6:24 ` Liang Li
2016-10-21  6:24   ` [Qemu-devel] " Liang Li
2016-10-21  6:24   ` Liang Li
2016-10-21 17:25 ` [RESEND PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration Dave Hansen
2016-10-21 17:25   ` [Qemu-devel] " Dave Hansen
2016-10-21 17:25   ` Dave Hansen
2016-10-21 17:25   ` Dave Hansen
2016-10-21 19:44   ` Michael S. Tsirkin
2016-10-21 19:44   ` Michael S. Tsirkin
2016-10-21 19:44     ` [Qemu-devel] " Michael S. Tsirkin
2016-10-21 19:44     ` Michael S. Tsirkin
2016-10-23 11:29     ` Li, Liang Z
2016-10-23 11:29     ` Li, Liang Z
2016-10-23 11:29       ` [Qemu-devel] " Li, Liang Z
2016-10-23 11:29       ` Li, Liang Z
2016-10-25  3:52       ` Michael S. Tsirkin
2016-10-25  3:52       ` Michael S. Tsirkin
2016-10-25  3:52         ` [Qemu-devel] " Michael S. Tsirkin
2016-10-25  3:52         ` Michael S. Tsirkin
2016-10-25  3:52         ` Michael S. Tsirkin
2016-10-26 10:06   ` Li, Liang Z
2016-10-26 10:06     ` [Qemu-devel] " Li, Liang Z
2016-10-26 10:06     ` Li, Liang Z
2016-10-26 10:06     ` Li, Liang Z
2016-10-26 10:13     ` Li, Liang Z
2016-10-26 10:13     ` Li, Liang Z
2016-10-26 10:13       ` [Qemu-devel] " Li, Liang Z
2016-10-26 10:13       ` Li, Liang Z
2016-10-26 10:13       ` Li, Liang Z
2016-10-26 18:15       ` Dave Hansen
2016-10-26 18:15         ` [Qemu-devel] " Dave Hansen
2016-10-26 18:15         ` Dave Hansen
2016-10-26 18:15         ` Dave Hansen
2016-10-27  0:36         ` Li, Liang Z
2016-10-27  0:36         ` Li, Liang Z
2016-10-27  0:36           ` [Qemu-devel] " Li, Liang Z
2016-10-27  0:36           ` Li, Liang Z
2016-10-27  0:36           ` Li, Liang Z
2016-10-26 18:11     ` Dave Hansen
2016-10-26 18:11       ` [Qemu-devel] " Dave Hansen
2016-10-26 18:11       ` Dave Hansen
2016-10-27  0:51       ` Li, Liang Z
2016-10-27  0:51         ` [Qemu-devel] " Li, Liang Z
2016-10-27  0:51         ` Li, Liang Z
2016-10-27  0:51         ` Li, Liang Z
2016-10-26 18:11     ` Dave Hansen
2016-10-26 10:06   ` Li, Liang Z

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1477031080-12616-1-git-send-email-liang.z.li@intel.com \
    --to=liang.z.li@intel.com \
    --cc=amit.shah@redhat.com \
    --cc=cornelia.huck@de.ibm.com \
    --cc=dave.hansen@intel.com \
    --cc=dgilbert@redhat.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mst@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=quintela@redhat.com \
    --cc=virtio-dev@lists.oasis-open.org \
    --cc=virtualization@lists.linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.