From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Dr. David Alan Gilbert" Subject: Re: [RFC Design Doc]Speed up live migration by skipping free pages Date: Thu, 24 Mar 2016 09:00:05 +0000 Message-ID: <20160324090004.GA2230@work-vm> References: <1458632629-4649-1-git-send-email-liang.z.li@intel.com> <20160322190530.GI2216@work-vm> <20160324012424.GB14956@linux-gk3p> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: "Li, Liang Z" , "qemu-devel@nongnu.org" , "kvm@vger.kernel.org" , "linux-kernel@vger.kenel.org" , "pbonzini@redhat.com" , "rth@twiddle.net" , "ehabkost@redhat.com" , "mst@redhat.com" , "amit.shah@redhat.com" , "quintela@redhat.com" , "mohan_parthasarathy@hpe.com" , "jitendra.kolhe@hpe.com" , "simhan@hpe.com" , "rkagan@virtuozzo.com" , "riel@redhat.com" To: Wei Yang Return-path: Received: from mx1.redhat.com ([209.132.183.28]:39672 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751245AbcCXJAN convert rfc822-to-8bit (ORCPT ); Thu, 24 Mar 2016 05:00:13 -0400 Content-Disposition: inline In-Reply-To: <20160324012424.GB14956@linux-gk3p> Sender: kvm-owner@vger.kernel.org List-ID: * Wei Yang (richard.weiyang@huawei.com) wrote: > On Wed, Mar 23, 2016 at 06:48:22AM +0000, Li, Liang Z wrote: > [...] > >> > 8. Pseudo code > >> > Dirty page logging should be enabled before getting the free pag= e > >> > information from guest, this is important because during the pro= cess > >> > of getting free pages, some free pages may be used and written b= y the > >> > guest, dirty page logging can trace these pages. The pseudo code= =C2=A0is > >> > like below: > >> > > >> > ----------------------------------------------- > >> > MigrationState *s =3D migrate_get_current(); > >> > ... > >> > > >> > memory_global_dirty_log_start(); > >> > > >> > if (get_guest_mem_info(&info)) { > >> > while (!get_free_page_bmap(free_page_bitmap, drop_page_= cache) > >> && > >> > s->state !=3D MIGRATION_STATUS_CANCELLING) { > >> > usleep(1000) // sleep for 1 ms > >> > } > >> > > >> > tighten_free_page_bmap =3D > >> tighten_guest_free_pages(free_page_bitmap); > >> > filter_out_guest_free_pages(tighten_free_page_bmap); > >> > } > >>=20 > >> Given the typical speed of networks; it wouldn't do too much harm = to start > >> sending assuming all pages are dirty and then when the guest final= ly gets > >> around to finishing the bitmap then update, so it's asynchronous -= and then if > >> the guest never responds we don't really care. > > > >Indeed, thanks! > > >=20 > This is interesting. By doing so, the threshold I mentioned in anothe= r mail is > not necessary, since we can do it in parallel. Actually I just realised it's a little more complex; we can't sync the = dirty bitmap again from the guest until after we've received the guests 'free= ' bitmap; that's because we wouldn't know if a 'dirty' page reflected that a page= declared as 'free' had now been reused - so there is still an ordering there. Dave >=20 > >Liang > >>=20 > >> Dave > >>=20 > >> > > >> > migration_bitmap_sync(); > >> > ... > >> > > >> > ----------------------------------------------- > >> > > >> > > >> > -- > >> > 1.9.1 > >> > > >> -- > >> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > >N=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BDr=EF=BF=BD=EF=BF=BDy=EF= =BF=BD=EF=BF=BD=EF=BF=BDb=EF=BF=BDX=EF=BF=BD=EF=BF=BD=C7=A7v=EF=BF=BD^=EF= =BF=BD)=DE=BA{.n=EF=BF=BD+=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BDh=EF=BF=BD= =EF=BF=BD=17=EF=BF=BD=EF=BF=BD=DC=A8}=EF=BF=BD=EF=BF=BD=EF=BF=BD=C6=A0z= =EF=BF=BD&j:+v=EF=BF=BD=EF=BF=BD=EF=BF=BD=07=EF=BF=BD=EF=BF=BD=EF=BF=BD= =EF=BF=BDzZ+=EF=BF=BD=EF=BF=BD+zf=EF=BF=BD=EF=BF=BD=EF=BF=BDh=EF=BF=BD=EF= =BF=BD=EF=BF=BD~=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BDi=EF=BF=BD=EF=BF=BD= =EF=BF=BDz=EF=BF=BD=1E=EF=BF=BDw=EF=BF=BD=EF=BF=BD=EF=BF=BD?=EF=BF=BD=EF= =BF=BD=EF=BF=BD=EF=BF=BD&=EF=BF=BD)=DF=A2=1Bf > --=20 > Richard Yang\nHelp you, Help me -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57545) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aj17q-0001vJ-Lg for qemu-devel@nongnu.org; Thu, 24 Mar 2016 05:00:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aj17n-0008HR-Fi for qemu-devel@nongnu.org; Thu, 24 Mar 2016 05:00:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45893) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aj17n-0008HA-7y for qemu-devel@nongnu.org; Thu, 24 Mar 2016 05:00:15 -0400 Date: Thu, 24 Mar 2016 09:00:05 +0000 From: "Dr. David Alan Gilbert" Message-ID: <20160324090004.GA2230@work-vm> References: <1458632629-4649-1-git-send-email-liang.z.li@intel.com> <20160322190530.GI2216@work-vm> <20160324012424.GB14956@linux-gk3p> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <20160324012424.GB14956@linux-gk3p> Subject: Re: [Qemu-devel] [RFC Design Doc]Speed up live migration by skipping free pages List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Wei Yang Cc: "rkagan@virtuozzo.com" , "linux-kernel@vger.kenel.org" , "ehabkost@redhat.com" , "kvm@vger.kernel.org" , "mst@redhat.com" , "simhan@hpe.com" , "Li, Liang Z" , "qemu-devel@nongnu.org" , "jitendra.kolhe@hpe.com" , "quintela@redhat.com" , "mohan_parthasarathy@hpe.com" , "amit.shah@redhat.com" , "pbonzini@redhat.com" , "rth@twiddle.net" * Wei Yang (richard.weiyang@huawei.com) wrote: > On Wed, Mar 23, 2016 at 06:48:22AM +0000, Li, Liang Z wrote: > [...] > >> > 8. Pseudo code > >> > Dirty page logging should be enabled before getting the free page > >> > information from guest, this is important because during the process > >> > of getting free pages, some free pages may be used and written by the > >> > guest, dirty page logging can trace these pages. The pseudo code=C2= =A0is > >> > like below: > >> > > >> > ----------------------------------------------- > >> > MigrationState *s =3D migrate_get_current(); > >> > ... > >> > > >> > memory_global_dirty_log_start(); > >> > > >> > if (get_guest_mem_info(&info)) { > >> > while (!get_free_page_bmap(free_page_bitmap, drop_page_cach= e) > >> && > >> > s->state !=3D MIGRATION_STATUS_CANCELLING) { > >> > usleep(1000) // sleep for 1 ms > >> > } > >> > > >> > tighten_free_page_bmap =3D > >> tighten_guest_free_pages(free_page_bitmap); > >> > filter_out_guest_free_pages(tighten_free_page_bmap); > >> > } > >>=20 > >> Given the typical speed of networks; it wouldn't do too much harm to s= tart > >> sending assuming all pages are dirty and then when the guest finally g= ets > >> around to finishing the bitmap then update, so it's asynchronous - and= then if > >> the guest never responds we don't really care. > > > >Indeed, thanks! > > >=20 > This is interesting. By doing so, the threshold I mentioned in another ma= il is > not necessary, since we can do it in parallel. Actually I just realised it's a little more complex; we can't sync the dirty bitmap again from the guest until after we've received the guests 'free' bi= tmap; that's because we wouldn't know if a 'dirty' page reflected that a page dec= lared as 'free' had now been reused - so there is still an ordering there. Dave >=20 > >Liang > >>=20 > >> Dave > >>=20 > >> > > >> > migration_bitmap_sync(); > >> > ... > >> > > >> > ----------------------------------------------- > >> > > >> > > >> > -- > >> > 1.9.1 > >> > > >> -- > >> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > >N=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BDr=EF=BF=BD=EF=BF=BDy=EF=BF= =BD=EF=BF=BD=EF=BF=BDb=EF=BF=BDX=EF=BF=BD=EF=BF=BD=C7=A7v=EF=BF=BD^=EF=BF= =BD)=DE=BA{.n=EF=BF=BD+=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BDh=EF=BF=BD=EF=BF= =BD=17=EF=BF=BD=EF=BF=BD=DC=A8}=EF=BF=BD=EF=BF=BD=EF=BF=BD=C6=A0z=EF=BF=BD&= j:+v=EF=BF=BD=EF=BF=BD=EF=BF=BD=07=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BDzZ+= =EF=BF=BD=EF=BF=BD+zf=EF=BF=BD=EF=BF=BD=EF=BF=BDh=EF=BF=BD=EF=BF=BD=EF=BF= =BD~=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BDi=EF=BF=BD=EF=BF=BD=EF=BF=BDz=EF=BF= =BD=1E=EF=BF=BDw=EF=BF=BD=EF=BF=BD=EF=BF=BD?=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF= =BF=BD&=EF=BF=BD)=DF=A2=1Bf > --=20 > Richard Yang\nHelp you, Help me -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK