[Qemu-devel] [PATCH 0/7] migration: Optimization the xbzrle and fix two corruption issues

[Qemu-devel] [PATCH 0/7] migration: Optimization the xbzrle and fix two corruption issues

[Gonglei (Arei)]

a. Optimization the xbzrle remarkable decrease the cache misses.
    The efficiency of compress increases more than fifty times.
    Before the patch set, the cache almost totally miss when the 
    number of cache item less than the dirty page number. Now the
    hot pages in the cache will not be replaced by other pages. 

b. Reducing the data copy

c. Fix two corruption issues.

ChenLiang (7):
  Fix one XBZRLE corruption issues
  migration: Add counters of updating the dirty bitmap.
  XBZRLE: optimize XBZRLE to decrease the cache missing
  XBZRLE: rebuild the cache_is_cached function
  migration: Fix the migrate auto converge process
  migraion: optimiztion xbzrle by reducing data copy
  migraion: delete death code

 arch_init.c                    | 94 ++++++++++++++++++------------------------
 include/migration/page_cache.h |  8 +++-
 page_cache.c                   | 36 +++++++++-------
 3 files changed, 69 insertions(+), 69 deletions(-)


Best regards,
-Gonglei

Re: [Qemu-devel] [PATCH 0/7] migration: Optimization the xbzrle and fix two corruption issues

[Dr. David Alan Gilbert]

* Gonglei (Arei) (arei.gonglei@huawei.com) wrote:

Hi,

> a. Optimization the xbzrle remarkable decrease the cache misses.
>     The efficiency of compress increases more than fifty times.
>     Before the patch set, the cache almost totally miss when the 
>     number of cache item less than the dirty page number. Now the
>     hot pages in the cache will not be replaced by other pages. 

Nice, what do you use as your performance test case for xbzrle?

Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 0/7] migration: Optimization the xbzrle and fix two corruption issues

[Gonglei]

On 2014/2/28 17:19, Dr. David Alan Gilbert wrote:

> * Gonglei (Arei) (arei.gonglei@huawei.com) wrote:
> 
> Hi,
> 
>> a. Optimization the xbzrle remarkable decrease the cache misses.
>>     The efficiency of compress increases more than fifty times.
>>     Before the patch set, the cache almost totally miss when the 
>>     number of cache item less than the dirty page number. Now the
>>     hot pages in the cache will not be replaced by other pages. 
> 
> Nice, what do you use as your performance test case for xbzrle?


The VM we used with 25G memory and 1Gbit nic. We run a test procedure
in the vm, which as this:

#include <stdio.h>
#include <stdlib.h>
#define PAGE_SIZE 4096
void main(void)
{
    char *p,*p1;
    long i,j,z;

    p = (char*)calloc(8*1024,1024*1024);
    if( p == NULL ){
        printf("fail to calloc \n");
        exit(1);
    }
    for(;;){
        p1 = p;
        for( i = 0 ; i < 8*1024 ; i++ ){

            for( j = 0 ; j < 1024*1024 ; j+=PAGE_SIZE ){
               *p1 = 0x55;
               p1+=PAGE_SIZE;
            }
        }
    }
}

finally,
the results of without enable xbzrle: 115MB/sec

using xbzrle without optimization (the size of cache 2G): 116MB/sec

using xbzrle with our optimization (the size of cache 2G): 150MB/sec

> 
> Dave
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK


Best regards,
-Gonglei

Re: [Qemu-devel] [PATCH 0/7] migration: Optimization the xbzrle and fix two corruption issues

[Dr. David Alan Gilbert]

* Gonglei (arei.gonglei@huawei.com) wrote:
> On 2014/2/28 17:19, Dr. David Alan Gilbert wrote:
> 
> > * Gonglei (Arei) (arei.gonglei@huawei.com) wrote:
> > 
> > Hi,
> > 
> >> a. Optimization the xbzrle remarkable decrease the cache misses.
> >>     The efficiency of compress increases more than fifty times.
> >>     Before the patch set, the cache almost totally miss when the 
> >>     number of cache item less than the dirty page number. Now the
> >>     hot pages in the cache will not be replaced by other pages. 
> > 
> > Nice, what do you use as your performance test case for xbzrle?
> 
> 
> The VM we used with 25G memory and 1Gbit nic. We run a test procedure
> in the vm, which as this:
> 
> #include <stdio.h>
> #include <stdlib.h>
> #define PAGE_SIZE 4096
> void main(void)
> {
>     char *p,*p1;
>     long i,j,z;
> 
>     p = (char*)calloc(8*1024,1024*1024);
>     if( p == NULL ){
>         printf("fail to calloc \n");
>         exit(1);
>     }
>     for(;;){
>         p1 = p;
>         for( i = 0 ; i < 8*1024 ; i++ ){
> 
>             for( j = 0 ; j < 1024*1024 ; j+=PAGE_SIZE ){
>                *p1 = 0x55;
>                p1+=PAGE_SIZE;
>             }
>         }
>     }
> }
> 
> finally,
> the results of without enable xbzrle: 115MB/sec
> 
> using xbzrle without optimization (the size of cache 2G): 116MB/sec
> 
> using xbzrle with our optimization (the size of cache 2G): 150MB/sec

Hmm yes, it's not a very realistic test is it.
Having said that, I've not managed to find a reallistic test people
can agree on; I was hoping you had one!

You're listing the differences in in MB/sec - what about total
time to migrate?

However, the other question is why your optimisation works well
with that test; is it just the CPU overhead that it's reducing
because it's not bothering to copy lots of stuff into the cache?
If that's all the guest is running, I can't see that it would
actually XBZRLE much - maybe just OS pages.
What do the 'info migrate' stats look like with/without your
optimisation - I'm interested in how many xbzrle pages are sent?

Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 0/7] migration: Optimization the xbzrle and fix two corruption issues

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1240 bytes --]

On 02/27/2014 09:05 PM, Gonglei (Arei) wrote:
> a. Optimization the xbzrle remarkable decrease the cache misses.
>     The efficiency of compress increases more than fifty times.
>     Before the patch set, the cache almost totally miss when the 
>     number of cache item less than the dirty page number. Now the
>     hot pages in the cache will not be replaced by other pages. 
> 
> b. Reducing the data copy
> 
> c. Fix two corruption issues.
> 
> ChenLiang (7):
>   Fix one XBZRLE corruption issues
>   migration: Add counters of updating the dirty bitmap.
>   XBZRLE: optimize XBZRLE to decrease the cache missing
>   XBZRLE: rebuild the cache_is_cached function
>   migration: Fix the migrate auto converge process
>   migraion: optimiztion xbzrle by reducing data copy
>   migraion: delete death code

Your series did not come through properly threaded, which makes it
harder to review.  All of the patches are lacking an In-Reply-To: header
pointing back to this cover letter.  'git send-email' is able to send
threaded series; please investigate why it didn't work for you this time
around.


-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 0/7] migration: Optimization the xbzrle and fix two corruption issues

[陈梁]

> * Gonglei (arei.gonglei@huawei.com) wrote:
>> On 2014/2/28 17:19, Dr. David Alan Gilbert wrote:
>> 
>>> * Gonglei (Arei) (arei.gonglei@huawei.com) wrote:
>>> 
>>> Hi,
>>> 
>>>> a. Optimization the xbzrle remarkable decrease the cache misses.
>>>>    The efficiency of compress increases more than fifty times.
>>>>    Before the patch set, the cache almost totally miss when the 
>>>>    number of cache item less than the dirty page number. Now the
>>>>    hot pages in the cache will not be replaced by other pages. 
>>> 
>>> Nice, what do you use as your performance test case for xbzrle?
>> 
>> 
>> The VM we used with 25G memory and 1Gbit nic. We run a test procedure
>> in the vm, which as this:
>> 
>> #include <stdio.h>
>> #include <stdlib.h>
>> #define PAGE_SIZE 4096
>> void main(void)
>> {
>>    char *p,*p1;
>>    long i,j,z;
>> 
>>    p = (char*)calloc(8*1024,1024*1024);
>>    if( p == NULL ){
>>        printf("fail to calloc \n");
>>        exit(1);
>>    }
>>    for(;;){
>>        p1 = p;
>>        for( i = 0 ; i < 8*1024 ; i++ ){
>> 
>>            for( j = 0 ; j < 1024*1024 ; j+=PAGE_SIZE ){
>>               *p1 = 0x55;
>>               p1+=PAGE_SIZE;
>>            }
>>        }
>>    }
>> }
>> 
>> finally,
>> the results of without enable xbzrle: 115MB/sec
>> 
>> using xbzrle without optimization (the size of cache 2G): 116MB/sec
>> 
>> using xbzrle with our optimization (the size of cache 2G): 150MB/sec
> 
> Hmm yes, it's not a very realistic test is it.
> Having said that, I've not managed to find a reallistic test people
> can agree on; I was hoping you had one!
> 
> You're listing the differences in in MB/sec - what about total
> time to migrate?
> 
> However, the other question is why your optimisation works well
> with that test; is it just the CPU overhead that it's reducing
> because it's not bothering to copy lots of stuff into the cache?
> If that's all the guest is running, I can't see that it would
> actually XBZRLE much - maybe just OS pages.
> What do the 'info migrate' stats look like with/without your
> optimisation - I'm interested in how many xbzrle pages are sent?
> 
> Dave
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

I am in home now, can not do some other tests. BTW, we know 
the limit of migration speed is the bandwidth of nic and the speed
of cpu. The sets of patches just resolve them. When the size of the
cache is small(2GB), the algorithm of the old code will be cache miss
totally, the bandwidth of nic is the limit. If the size of the cache is big
enough(8GB), the result is 2.4GB/sec ,the usage of cpu is 100%. 
After reducing the data copy, the result is 3.1GB/sec.

Re: [Qemu-devel] [PATCH 0/7] migration: Optimization the xbzrle and fix two corruption issues

[陈梁]

> On 02/27/2014 09:05 PM, Gonglei (Arei) wrote:
>> a. Optimization the xbzrle remarkable decrease the cache misses.
>>    The efficiency of compress increases more than fifty times.
>>    Before the patch set, the cache almost totally miss when the 
>>    number of cache item less than the dirty page number. Now the
>>    hot pages in the cache will not be replaced by other pages. 
>> 
>> b. Reducing the data copy
>> 
>> c. Fix two corruption issues.
>> 
>> ChenLiang (7):
>>  Fix one XBZRLE corruption issues
>>  migration: Add counters of updating the dirty bitmap.
>>  XBZRLE: optimize XBZRLE to decrease the cache missing
>>  XBZRLE: rebuild the cache_is_cached function
>>  migration: Fix the migrate auto converge process
>>  migraion: optimiztion xbzrle by reducing data copy
>>  migraion: delete death code
> 
> Your series did not come through properly threaded, which makes it
> harder to review.  All of the patches are lacking an In-Reply-To: header
> pointing back to this cover letter.  'git send-email' is able to send
> threaded series; please investigate why it didn't work for you this time
> around.
> 
> -- 
> Eric Blake   eblake redhat com    +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
Thanks, I will fix it next time.