From: "Chunguang Li" <lichunguang@hust.edu.cn>
To: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Cc: Amit Shah <amit.shah@redhat.com>,
qemu-devel@nongnu.org, pbonzini@redhat.com, stefanha@redhat.com,
quintela@redhat.com
Subject: Re: [Qemu-devel] Migration dirty bitmap: should only mark pages as dirty after they have been sent
Date: Tue, 8 Nov 2016 21:40:03 +0800 (GMT+08:00) [thread overview]
Message-ID: <1d21a67.a9f8.158442ac47f.Coremail.lichunguang@hust.edu.cn> (raw)
In-Reply-To: <20161108110552.GD2042@work-vm>
> -----Original Messages-----
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> Sent Time: Tuesday, November 8, 2016
> To: "Chunguang Li" <lichunguang@hust.edu.cn>
> Cc: "Amit Shah" <amit.shah@redhat.com>, qemu-devel@nongnu.org, pbonzini@redhat.com, stefanha@redhat.com, quintela@redhat.com
> Subject: Re: [Qemu-devel] Migration dirty bitmap: should only mark pages as dirty after they have been sent
>
> * Chunguang Li (lichunguang@hust.edu.cn) wrote:
> >
> >
> >
> > > -----Original Messages-----
> > > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > > Sent Time: Friday, October 14, 2016
> > > To: "Chunguang Li" <lichunguang@hust.edu.cn>
> > > Cc: "Amit Shah" <amit.shah@redhat.com>, qemu-devel@nongnu.org, pbonzini@redhat.com, stefanha@redhat.com, quintela@redhat.com
> > > Subject: Re: [Qemu-devel] Migration dirty bitmap: should only mark pages as dirty after they have been sent
> > >
> > > * Chunguang Li (lichunguang@hust.edu.cn) wrote:
> > > >
> > > >
> > > >
> > > > > -----原始邮件-----
> > > > > 发件人: "Amit Shah" <amit.shah@redhat.com>
> > > > > 发送时间: 2016年9月30日 星期五
> > > > > 收件人: "Chunguang Li" <lichunguang@hust.edu.cn>
> > > > > 抄送: "Dr. David Alan Gilbert" <dgilbert@redhat.com>, qemu-devel@nongnu.org, pbonzini@redhat.com, stefanha@redhat.com, quintela@redhat.com
> > > > > 主题: Re: Re: [Qemu-devel] Migration dirty bitmap: should only mark pages as dirty after they have been sent
> > > > >
> > > > > On (Mon) 26 Sep 2016 [22:55:01], Chunguang Li wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > > -----原始邮件-----
> > > > > > > 发件人: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > > > > > > 发送时间: 2016年9月26日 星期一
> > > > > > > 收件人: "Chunguang Li" <lichunguang@hust.edu.cn>
> > > > > > > 抄送: qemu-devel@nongnu.org, amit.shah@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, quintela@redhat.com
> > > > > > > 主题: Re: [Qemu-devel] Migration dirty bitmap: should only mark pages as dirty after they have been sent
> > > > > > >
> > > > > > > * Chunguang Li (lichunguang@hust.edu.cn) wrote:
> > > > > > > > Hi all!
> > > > > > > > I have some confusion about the dirty bitmap during migration. I have digged into the code. I figure out that every now and then during migration, the dirty bitmap will be grabbed from the kernel space through ioctl(KVM_GET_DIRTY_LOG), and then be used to update qemu's dirty bitmap. However I think this mechanism leads to resendness of some NON-dirty pages.
> > > > > > > >
> > > > > > > > Take the first iteration of precopy for instance, during which all the pages will be sent. Before that during the migration setup, the ioctl(KVM_GET_DIRTY_LOG) is called once, so the kernel begins to produce the dirty bitmap from this moment. When the pages "that haven't been sent" are written, the kernel space marks them as dirty. However I don't think this is correct, because these pages will be sent during this and the next iterations with the same content (if they are not written again after they are sent). It only makes sense to mark the pages which have already been sent during one iteration as dirty when they are written.
> > > > > > > >
> > > > > > > >
> > > > > > > > Am I right about this consideration? If I am right, is there some advice to improve this?
> > > > > > >
> > > > > > > I think you're right that this can happen; to clarify I think the
> > > > > > > case you're talking about is:
> > > > > > >
> > > > > > > Iteration 1
> > > > > > > sync bitmap
> > > > > > > start sending pages
> > > > > > > page 'n' is modified - but hasn't been sent yet
> > > > > > > page 'n' gets sent
> > > > > > > Iteration 2
> > > > > > > sync bitmap
> > > > > > > 'page n is shown as modified'
> > > > > > > send page 'n' again
> > > > > > >
> > > > > >
> > > > > > Yes，this is right the case I am talking about.
> > > > > >
> > > > > > > So you're right that is wasteful; I guess it's more wasteful
> > > > > > > on big VMs with slow networks where the length of each iteration
> > > > > > > is large.
> > > > > >
> > > > > > I think this is "very" wasteful. Assume the workload writes the pages dirty randomly within the guest address space, and the transfer speed is constant. Intuitively, I think nearly half of the dirty pages produced in Iteration 1 is not really dirty. This means the time of Iteration 2 is double of that to send only really dirty pages.
> > > > >
> > > > > It makes sense, can you get some perf numbers to show what kinds of
> > > > > workloads get impacted the most? That would also help us to figure
> > > > > out what kinds of speed improvements we can expect.
> > > > >
> > > > >
> > > > > Amit
> > > >
> > > > I have picked up 6 workloads and got the following statistics numbers
> > > > of every iteration (except the last stop-copy one) during precopy.
> > > > These numbers are obtained with the basic precopy migration, without
> > > > the capabilities like xbzrle or compression, etc. The network for the
> > > > migration is exclusive, with a separate network for the workloads.
> > > > They are both gigabit ethernet. I use qemu-2.5.1.
> > > >
> > > > Three (booting, idle, web server) of them converged to the stop-copy phase,
> > > > with the given bandwidth and default downtime (300ms), while the other
> > > > three (kernel compilation, zeusmp, memcached) did not.
> > > >
> > > > One page is "not-really-dirty", if it is written first and is sent later
> > > > (and not written again after that) during one iteration. I guess this
> > > > would not happen so often during the other iterations as during the 1st
> > > > iteration. Because all the pages of the VM are sent to the dest node during
> > > > the 1st iteration, while during the others, only part of the pages are sent.
> > > > So I think the "not-really-dirty" pages should be produced mainly during
> > > > the 1st iteration , and maybe very little during the other iterations.
> > > >
> > > > If we could avoid resending the "not-really-dirty" pages, intuitively, I
> > > > think the time spent on Iteration 2 would be halved. This is a chain reaction,
> > > > because the dirty pages produced during Iteration 2 is halved, which incurs
> > > > that the time spent on Iteration 3 is halved, then Iteration 4, 5...
> > >
> > > Yes; these numbers don't show how many of them are false dirty though.
> > >
> > > One problem is thinking about pages that have been redirtied, if the page is dirtied
> > > after the sync but before the network write then it's the false-dirty that
> > > you're describing.
> > >
> > > However, if the page is being written a few times, and so it would have been written
> > > after the network write then it isn't a false-dirty.
> > >
> > > You might be able to figure that out with some kernel tracing of when the dirtying
> > > happens, but it might be easier to write the fix!
> > >
> > > Dave
> >
> > Hi, I have made some new progress now.
> >
> > To tell how many false dirty pages there are exactly in each iteration, I malloc a
> > buffer in memory as big as the size of the whole VM memory. When a page is
> > transferred to the dest node, it is copied to the buffer; During the next iteration,
> > if one page is transferred, it is compared to the old one in the buffer, and the
> > old one will be replaced for next comparison if it is really dirty. Thus, we are now
> > able to get the exact number of false dirty pages.
> >
> > This time, I use 15 workloads to get the statistic number. They are:
> >
> > 1. 11 benchmarks picked up from cpu2006 benchmark suit. They are all scientific
> > computing workloads like Quantum Chromodynamics, Fluid Dynamics, etc. I pick
> > up these 11 benchmarks because compared to others, they have bigger memory
> > occupation and higher memory dirty rate. Thus most of them could not converge
> > to stop-and-copy using the default migration speed (32MB/s).
> > 2. kernel compilation
> > 3. idle VM
> > 4. Apache web server which serves static content
> >
> > (the above workloads are all running in VM with 1 vcpu and 1GB memory, and the
> > migration speed is the default 32MB/s)
> >
> > 5. Memcached. The VM has 6 cpu cores and 6GB memory, and 4GB are used as the cache.
> > After filling up the 4GB cache, a client writes the cache at a constant speed
> > during migration. This time, migration speed has no limit, and is up to the
> > capability of 1Gbps Ethernet.
> >
> > Summarize the results first: (and you can read the precise number below)
> >
> > 1. 4 of these 15 workloads have a big proportion (>60%, even >80% during some iterations)
> > of false dirty pages out of all the dirty pages since iteration 2 (and the big
> > proportion lasts during the following iterations). They are cpu2006.zeusmp,
> > cpu2006.bzip2, cpu2006.mcf, and memcached.
> > 2. 2 workloads (idle, webserver) spend most of the migration time on iteration 1, even
> > though the proportion of false dirty pages is big since iteration 2, the space to
> > optimize is small.
> > 3. 1 workload (kernel compilation) only have a big proportion during iteration 2, not
> > in the other iterations.
> > 4. 8 workloads (the other 8 benchmarks of cpu2006) have little proportion of false
> > dirty pages since iteration 2. So the spaces to optimize for them are small.
> >
> > Now I want to talk a little more about the reasons why false dirty pages are produced.
> > The first reason is what we have discussed before---the mechanism to track the dirty
> > pages.
> > And then I come up with another reason. Here is the situation: a write operation to one
> > memory page happens, but it doesn't change any content of the page. So it's "write but
> > not dirty", and kernel still marks it as dirty. One guy in our lab has done some experiments
> > to figure out the proportion of "write but not dirty" operations, and he uses the cpu2006
> > benchmark suit. According to his results, general workloads has a little proportion (<10%)
> > of "write but not dirty" out of all the write operations, while few workloads have higher
> > proportion (one even as high as 50%). Now we are not sure why "write but not dirty" would
> > happen, it just happened.
>
> I think there are a few different reasons I can think of:
> a) You have a flag or mutex that's set and cleared; so it gets set (marked
> dirty) and cleared around some operation. By the time we come to migrate
> it then it's back to cleared again.
> Similarly with other temporary data structures.
> b) Some system operation causes the page to be moved - e.g. swap or the kernel
> reorganising memory.
Sorry, I don't quite understand reason (b). Take swap as example, do you mean a page
is swapped out and swapped in to the old address again, so the content remains unchanged?
>
> However, it's a shame I don't think you can tell in your experiment which of the
> two cases we're hitting? I'd like to know if it's worth working on
> making the page sync mechanism better or if it's nore important to deal
> with the second reason you show.
Yes, you are right, it's hard to tell which case we're hitting (including the cases you
think of). However, as I use the SHA1 method, now I don't have to tell them. Because it
just handle all the cases we have thought of.
>
> > So these two reasons contribute to the false dirty pages. To optimize, I compute and store
> > the SHA1 hash before transferring each page. Next time, if one page needs retransmission, its
> > SHA1 hash is computed again, and compared to the old hash. If the hash is the same, it's a
> > false dirty page, and we just skip this page; Otherwise, the page is transferred, and the new
> > hash replaces the old one for next comparison.
> > The reason to use SHA1 hash but not byte-by-byte comparison is the memory overheads. One SHA1
> > hash is 20 bytes. So we need extra 20/4096 (<1/200) memory space of the whole VM memory, which
> > is relatively small.
> > As far as I know, SHA1 hash is widely used in the scenes of deduplication for backup systems.
> > They have proven that the probability of hash collision is far smaller than disk hardware fault,
> > so it's secure hash, that is, if the hashes of two chunks are the same, the content must be the
> > same. So I think the SHA1 hash could replace byte-to-byte comparison in the VM memory scenery.
>
> There was a proposal ( https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg05331.html )
> to do a migration system where
> a copy of the migration RAM is stored on disc on the destination for cases where similar VMs
> are migrated, and it used a checksum for each page to find the matching page
> in the cache; that originally used a smaller hash, I think in the end they used a SHA-256.
> (Hash based checks still make me nervous for intentional collisions but that's probably
> me being paranoid?)
Em... I don't know if most people would accept the hash based checks.
Maybe it needs some more mathematical proving like they have done in the
field of deduplication for backup systems.
>
> > Then I do the same migration experiments using the SHA1 hash. For the 4 workloads which have
> > big proportions of false dirty pages, the improvement is remarkable. Without optimization,
> > they either can not converge to stop-and-copy, or take a very long time to complete. With the
> > SHA1 hash method, all of them now complete in a relatively short time.
> > For the reason I have talked above, the other workloads don't get notable improvements from the
> > optimization. So below, I only show the exact number after optimization for the 4 workloads with
> > remarkable improvements.
> >
> > Any comments or suggestions?
>
> You might be able to save some of the CPU time; we've
> got a test that checks if a page is all-zero; if you're doing
> the SHA calculation you could avoid doing the all-zero check
> and replace it by comparing hte output of the SHA.
Yes, this is one way. However now I'm doing the opposite. I first
calculate the SHA1 of the all-zero page and remember that. Then next time,
if I recognize an all-zero page after the check, I just store the
SHA1 I have got earlier, avoiding calculating the SHA1 of the all-zero
page again. I think this is better, because I think the current implementation
to check all-zero pages is faster than calculating SHA1.
Thanks,
Chunguang
>
> >
> > Below is the experiments data:
> > (
> > "dup" means zero page, this kind of pages takes very little migration time and network
> > resources, so they are always not regard as dirty pages in my numbers;
> > "rd" means really dirty pages;
> > "fd" means false dirty pages;
> > The numbers refer to the quantities of pages.
> > )
> >
> > ------------------The 4 workloads with remarkable improvements (both the results of original precopy and with optimization are shown)-------------------
> >
> > 1. memcached
> >
> > ----- original pre-copy (can not converge): -----
> > Iteration 1, duration: 42111 ms , transferred pages: 1568788 (dup: 416239, rd: 1152549, fd: 0) , new dirty pages: 499015 , remaining dirty pages: 507397
> > Iteration 2, duration: 17208 ms , transferred pages: 498946 (dup: 5456, rd: 160206, fd: 333284) , new dirty pages: 261237 , remaining dirty pages: 269688
> > Iteration 3, duration: 9134 ms , transferred pages: 262377 (dup: 519, rd: 111900, fd: 149958) , new dirty pages: 170281 , remaining dirty pages: 177592
> > Iteration 4, duration: 5920 ms , transferred pages: 169966 (dup: 87, rd: 82487, fd: 87392) , new dirty pages: 121154 , remaining dirty pages: 128780
> > Iteration 5, duration: 4239 ms , transferred pages: 121551 (dup: 81, rd: 64120, fd: 57350) , new dirty pages: 100976 , remaining dirty pages: 108205
> > Iteration 6, duration: 3495 ms , transferred pages: 100353 (dup: 90, rd: 56021, fd: 44242) , new dirty pages: 74547 , remaining dirty pages: 82399
> > Iteration 7, duration: 2583 ms , transferred pages: 74160 (dup: 56, rd: 38016, fd: 36088) , new dirty pages: 58209 , remaining dirty pages: 66448
> > Iteration 8, duration: 2039 ms , transferred pages: 58534 (dup: 81, rd: 26885, fd: 31568) , new dirty pages: 43511 , remaining dirty pages: 51425
> > Iteration 9, duration: 1513 ms , transferred pages: 43484 (dup: 55, rd: 26641, fd: 16788) , new dirty pages: 43722 , remaining dirty pages: 51663
> > Iteration 10, duration: 1521 ms , transferred pages: 43676 (dup: 62, rd: 26463, fd: 17151) , new dirty pages: 35347 , remaining dirty pages: 43334
> > Iteration 11, duration: 1230 ms , transferred pages: 35287 (dup: 0, rd: 21293, fd: 13994) , new dirty pages: 28851 , remaining dirty pages: 36898
> > Iteration 12, duration: 1031 ms , transferred pages: 29651 (dup: 82, rd: 18143, fd: 11426) , new dirty pages: 27062 , remaining dirty pages: 34309
> > Iteration 13, duration: 917 ms , transferred pages: 26385 (dup: 56, rd: 14149, fd: 12180) , new dirty pages: 22723 , remaining dirty pages: 30647
> > Iteration 14, duration: 762 ms , transferred pages: 21902 (dup: 55, rd: 16355, fd: 5492) , new dirty pages: 18208 , remaining dirty pages: 26953
> > Iteration 15, duration: 650 ms , transferred pages: 18636 (dup: 0, rd: 11943, fd: 6693) , new dirty pages: 16085 , remaining dirty pages: 24402
> > Iteration 16, duration: 554 ms , transferred pages: 15946 (dup: 56, rd: 9527, fd: 6363) , new dirty pages: 14766 , remaining dirty pages: 23222
> > Iteration 17, duration: 538 ms , transferred pages: 15434 (dup: 0, rd: 9779, fd: 5655) , new dirty pages: 13381 , remaining dirty pages: 21169
> > Iteration 18, duration: 487 ms , transferred pages: 14089 (dup: 81, rd: 7737, fd: 6271) , new dirty pages: 13325 , remaining dirty pages: 20405
> > Iteration 19, duration: 428 ms , transferred pages: 12232 (dup: 0, rd: 8488, fd: 3744) , new dirty pages: 10274 , remaining dirty pages: 18447
> > Iteration 20, duration: 377 ms , transferred pages: 10887 (dup: 56, rd: 6362, fd: 4469) , new dirty pages: 9708 , remaining dirty pages: 17268
> > Iteration 21, duration: 320 ms , transferred pages: 9222 (dup: 0, rd: 5789, fd: 3433) , new dirty pages: 8015 , remaining dirty pages: 16061
> > Iteration 22, duration: 268 ms , transferred pages: 7621 (dup: 0, rd: 6204, fd: 1417) , new dirty pages: 7227 , remaining dirty pages: 15667
> > Iteration 23, duration: 269 ms , transferred pages: 7813 (dup: 56, rd: 4410, fd: 3347) , new dirty pages: 7591 , remaining dirty pages: 15445
> > Iteration 24, duration: 271 ms , transferred pages: 7749 (dup: 0, rd: 4565, fd: 3184) , new dirty pages: 15126 , remaining dirty pages: 22822
> > Iteration 25, duration: 549 ms , transferred pages: 15818 (dup: 60, rd: 10545, fd: 5213) , new dirty pages: 14559 , remaining dirty pages: 21563
> > Iteration 26, duration: 499 ms , transferred pages: 14281 (dup: 3, rd: 8760, fd: 5518) , new dirty pages: 11947 , remaining dirty pages: 19229
> > Iteration 27, duration: 376 ms , transferred pages: 10823 (dup: 25, rd: 6550, fd: 4248) , new dirty pages: 8561 , remaining dirty pages: 16967
> > Iteration 28, duration: 324 ms , transferred pages: 9350 (dup: 31, rd: 5292, fd: 4027) , new dirty pages: 8655 , remaining dirty pages: 16272
> > Iteration 29, duration: 274 ms , transferred pages: 7813 (dup: 0, rd: 6088, fd: 1725) , new dirty pages: 6300 , remaining dirty pages: 14759
> > Iteration 30, duration: 218 ms , transferred pages: 6340 (dup: 45, rd: 3196, fd: 3099) , new dirty pages: 5143 , remaining dirty pages: 13562
> >
> > ----- after optimization: -----
> > Iteration 1, duration: 40664 ms , transferred pages: 1569037 (dup: 405940, rd: 1163097) , new dirty pages: 506846 , remaining dirty pages: 514979
> > Iteration 2, duration: 8032 ms , transferred pages: 161130 (dup: 4007, rd: 157123) , new dirty pages: 153479 , remaining dirty pages: 153479
>
> Big difference.
>
> > Iteration 3, duration: 2620 ms , transferred pages: 65260 (dup: 20, rd: 65240) , new dirty pages: 64014 , remaining dirty pages: 67100
> > Iteration 4, duration: 1160 ms , transferred pages: 30227 (dup: 60, rd: 30167) , new dirty pages: 34031 , remaining dirty pages: 41414
> > Iteration 5, duration: 648 ms , transferred pages: 18700 (dup: 56, rd: 18644) , new dirty pages: 18375 , remaining dirty pages: 25536
> > Iteration 6, duration: 389 ms , transferred pages: 11399 (dup: 55, rd: 11344) , new dirty pages: 12536 , remaining dirty pages: 17516
> > Iteration 7, duration: 292 ms , transferred pages: 8197 (dup: 0, rd: 8197) , new dirty pages: 8387 , remaining dirty pages: 16802
> > Iteration 8, duration: 171 ms , transferred pages: 4931 (dup: 39, rd: 4892) , new dirty pages: 6182 , remaining dirty pages: 14060
> > Iteration 9, duration: 163 ms , transferred pages: 4355 (dup: 16, rd: 4339) , new dirty pages: 5530 , remaining dirty pages: 11973
> > Iteration 10, duration: 104 ms , transferred pages: 3266 (dup: 0, rd: 3266) , new dirty pages: 2893 , remaining dirty pages: 11014
> > Iteration 11, duration: 52 ms , transferred pages: 1153 (dup: 0, rd: 1153) , new dirty pages: 1586 , remaining dirty pages: 10516
> > Iteration 12, duration: 52 ms , transferred pages: 1921 (dup: 39, rd: 1882) , new dirty pages: 1619 , remaining dirty pages: 8842
> > Iteration 13, duration: 62 ms , transferred pages: 1537 (dup: 0, rd: 1537) , new dirty pages: 2052 , remaining dirty pages: 8871
> > Iteration 14, duration: 58 ms , transferred pages: 1665 (dup: 0, rd: 1665) , new dirty pages: 1947 , remaining dirty pages: 7989
> > Iteration 15, duration: 2 ms , transferred pages: 0 (dup: 0, rd: 0) , new dirty pages: 0 , remaining dirty pages: 7989
> > total time: 54693 milliseconds
>
> Very nice.
>
> Dave
>
> > 2. cpu2006.zeusmp
> >
> > ----- original pre-copy (can not converge): -----
> > Iteration 1, duration: 21112 ms , transferred pages: 266450 (dup: 93385, rd: 173065, fd: 0) , new dirty pages: 127866 , remaining dirty pages: 127866
> > Iteration 2, duration: 6192 ms , transferred pages: 125662 (dup: 75762, rd: 17389, fd: 32511) , new dirty pages: 131655 , remaining dirty pages: 133859
> > Iteration 3, duration: 6699 ms , transferred pages: 131937 (dup: 77298, rd: 20320, fd: 34319) , new dirty pages: 121027 , remaining dirty pages: 122949
> > Iteration 4, duration: 5999 ms , transferred pages: 122512 (dup: 73588, rd: 17236, fd: 31688) , new dirty pages: 122759 , remaining dirty pages: 123196
> > Iteration 5, duration: 5804 ms , transferred pages: 122717 (dup: 75436, rd: 19016, fd: 28265) , new dirty pages: 123697 , remaining dirty pages: 124176
> > Iteration 6, duration: 5698 ms , transferred pages: 123708 (dup: 77249, rd: 18022, fd: 28437) , new dirty pages: 121838 , remaining dirty pages: 122306
> > Iteration 7, duration: 5515 ms , transferred pages: 122306 (dup: 76727, rd: 14819, fd: 30760) , new dirty pages: 122382 , remaining dirty pages: 122382
> > Iteration 8, duration: 6086 ms , transferred pages: 120825 (dup: 71834, rd: 15987, fd: 33004) , new dirty pages: 121587 , remaining dirty pages: 123144
> > Iteration 9, duration: 5899 ms , transferred pages: 120964 (dup: 72860, rd: 18191, fd: 29913) , new dirty pages: 120391 , remaining dirty pages: 122571
> > Iteration 10, duration: 5801 ms , transferred pages: 121425 (dup: 74140, rd: 20722, fd: 26563) , new dirty pages: 122302 , remaining dirty pages: 123448
> > Iteration 11, duration: 5909 ms , transferred pages: 123448 (dup: 74735, rd: 19678, fd: 29035) , new dirty pages: 123258 , remaining dirty pages: 123258
> > Iteration 12, duration: 6293 ms , transferred pages: 121211 (dup: 70442, rd: 18128, fd: 32641) , new dirty pages: 123623 , remaining dirty pages: 125670
> > Iteration 13, duration: 6398 ms , transferred pages: 124897 (dup: 72701, rd: 21134, fd: 31062) , new dirty pages: 122355 , remaining dirty pages: 123128
> > Iteration 14, duration: 6301 ms , transferred pages: 121893 (dup: 70514, rd: 23470, fd: 27909) , new dirty pages: 120980 , remaining dirty pages: 122215
> > Iteration 15, duration: 6304 ms , transferred pages: 121389 (dup: 70005, rd: 21731, fd: 29653) , new dirty pages: 121628 , remaining dirty pages: 122454
> > Iteration 16, duration: 6398 ms , transferred pages: 122164 (dup: 69962, rd: 24376, fd: 27826) , new dirty pages: 122246 , remaining dirty pages: 122536
> > Iteration 17, duration: 6201 ms , transferred pages: 121548 (dup: 70984, rd: 23915, fd: 26649) , new dirty pages: 121460 , remaining dirty pages: 122448
> > Iteration 18, duration: 6401 ms , transferred pages: 122272 (dup: 70072, rd: 22261, fd: 29939) , new dirty pages: 123518 , remaining dirty pages: 123694
> > Iteration 19, duration: 7003 ms , transferred pages: 121873 (dup: 64754, rd: 27325, fd: 29794) , new dirty pages: 120568 , remaining dirty pages: 122389
> > Iteration 20, duration: 6400 ms , transferred pages: 121422 (dup: 69221, rd: 25300, fd: 26901) , new dirty pages: 121229 , remaining dirty pages: 122196
> > Iteration 21, duration: 6703 ms , transferred pages: 119895 (dup: 65232, rd: 25877, fd: 28786) , new dirty pages: 123284 , remaining dirty pages: 125585
> > Iteration 22, duration: 6902 ms , transferred pages: 123884 (dup: 67582, rd: 29020, fd: 27282) , new dirty pages: 122057 , remaining dirty pages: 123758
> > Iteration 23, duration: 6800 ms , transferred pages: 122010 (dup: 66529, rd: 30644, fd: 24837) , new dirty pages: 120916 , remaining dirty pages: 122664
> > Iteration 24, duration: 7202 ms , transferred pages: 121951 (dup: 63188, rd: 31105, fd: 27658) , new dirty pages: 122715 , remaining dirty pages: 123428
> > Iteration 25, duration: 7202 ms , transferred pages: 122919 (dup: 64161, rd: 32063, fd: 26695) , new dirty pages: 123180 , remaining dirty pages: 123689
> > Iteration 26, duration: 7404 ms , transferred pages: 123092 (dup: 62694, rd: 33459, fd: 26939) , new dirty pages: 122149 , remaining dirty pages: 122746
> > Iteration 27, duration: 7205 ms , transferred pages: 120427 (dup: 61664, rd: 34344, fd: 24419) , new dirty pages: 120299 , remaining dirty pages: 122618
> > Iteration 28, duration: 7100 ms , transferred pages: 121074 (dup: 63130, rd: 32403, fd: 25541) , new dirty pages: 122984 , remaining dirty pages: 124528
> > Iteration 29, duration: 7904 ms , transferred pages: 124060 (dup: 59564, rd: 35631, fd: 28865) , new dirty pages: 127080 , remaining dirty pages: 127548
> > Iteration 30, duration: 7906 ms , transferred pages: 127518 (dup: 63029, rd: 34416, fd: 30073) , new dirty pages: 125028 , remaining dirty pages: 125058
> >
> > ----- after optimization: -----
> > Iteration 1, duration: 21601 ms , transferred pages: 266450 (dup: 89731, rd: 176719) , new dirty pages: 139843 , remaining dirty pages: 139843
> > Iteration 2, duration: 1747 ms , transferred pages: 92077 (dup: 78364, rd: 13713) , new dirty pages: 90945 , remaining dirty pages: 90945
> > Iteration 3, duration: 1592 ms , transferred pages: 62253 (dup: 49435, rd: 12818) , new dirty pages: 76929 , remaining dirty pages: 76929
> > Iteration 4, duration: 992 ms , transferred pages: 44837 (dup: 37886, rd: 6951) , new dirty pages: 71331 , remaining dirty pages: 72916
> > Iteration 5, duration: 998 ms , transferred pages: 55229 (dup: 47150, rd: 8079) , new dirty pages: 21703 , remaining dirty pages: 23302
> > Iteration 6, duration: 211 ms , transferred pages: 20337 (dup: 18516, rd: 1821) , new dirty pages: 14500 , remaining dirty pages: 14500
> > Iteration 7, duration: 31 ms , transferred pages: 12933 (dup: 12627, rd: 306) , new dirty pages: 1520 , remaining dirty pages: 1520
> > Iteration 8, duration: 30 ms , transferred pages: 0 (dup: 0, rd: 0) , new dirty pages: 4 , remaining dirty pages: 1524
> > total time: 27225 milliseconds
> >
> > 3. cpu2006.bzip2
> >
> > ----- original pre-copy: -----
> > Iteration 1, duration: 18306 ms , transferred pages: 266450 (dup: 116569, rd: 149881, fd: 0) , new dirty pages: 106299 , remaining dirty pages: 106299
> > Iteration 2, duration: 10694 ms , transferred pages: 104611 (dup: 17550, rd: 10536, fd: 76525) , new dirty pages: 34394 , remaining dirty pages: 36082
> > Iteration 3, duration: 2998 ms , transferred pages: 34442 (dup: 9924, rd: 12254, fd: 12264) , new dirty pages: 6419 , remaining dirty pages: 8059
> > Iteration 4, duration: 699 ms , transferred pages: 5748 (dup: 22, rd: 2583, fd: 3143) , new dirty pages: 1226 , remaining dirty pages: 3537
> > Iteration 5, duration: 200 ms , transferred pages: 1636 (dup: 0, rd: 1194, fd: 442) , new dirty pages: 478 , remaining dirty pages: 2379
> > Iteration 6, duration: 1 ms , transferred pages: 0 (dup: 0, rd: 0, fd: 0) , new dirty pages: 0 , remaining dirty pages: 2379
> >
> > ----- after optimization: -----
> > Iteration 1, duration: 13995 ms , transferred pages: 266314 (dup: 152118, rd: 114196) , new dirty pages: 97009 , remaining dirty pages: 97145
> > Iteration 2, duration: 1215 ms , transferred pages: 33400 (dup: 26745, rd: 6655) , new dirty pages: 12866 , remaining dirty pages: 14017
> > Iteration 3, duration: 701 ms , transferred pages: 5774 (dup: 48, rd: 5726) , new dirty pages: 6342 , remaining dirty pages: 8761
> > Iteration 4, duration: 500 ms , transferred pages: 4111 (dup: 21, rd: 4090) , new dirty pages: 4311 , remaining dirty pages: 6485
> > Iteration 5, duration: 400 ms , transferred pages: 3273 (dup: 1, rd: 3272) , new dirty pages: 3034 , remaining dirty pages: 5431
> > Iteration 6, duration: 301 ms , transferred pages: 2454 (dup: 0, rd: 2454) , new dirty pages: 2094 , remaining dirty pages: 4472
> > Iteration 7, duration: 299 ms , transferred pages: 2454 (dup: 0, rd: 2454) , new dirty pages: 2066 , remaining dirty pages: 4082
> > Iteration 8, duration: 202 ms , transferred pages: 1636 (dup: 0, rd: 1636) , new dirty pages: 2881 , remaining dirty pages: 4648
> > Iteration 9, duration: 300 ms , transferred pages: 2454 (dup: 0, rd: 2454) , new dirty pages: 4775 , remaining dirty pages: 6778
> > Iteration 10, duration: 400 ms , transferred pages: 3281 (dup: 9, rd: 3272) , new dirty pages: 3757 , remaining dirty pages: 5576
> > Iteration 11, duration: 401 ms , transferred pages: 3279 (dup: 7, rd: 3272) , new dirty pages: 6980 , remaining dirty pages: 8906
> > Iteration 12, duration: 500 ms , transferred pages: 7118 (dup: 3035, rd: 4083) , new dirty pages: 10774 , remaining dirty pages: 11922
> > Iteration 13, duration: 116 ms , transferred pages: 11706 (dup: 10152, rd: 1554) , new dirty pages: 1326 , remaining dirty pages: 1326
> > Iteration 14, duration: 117 ms , transferred pages: 0 (dup: 0, rd: 0) , new dirty pages: 0 , remaining dirty pages: 1326
> > total time: 19479 milliseconds
> >
> > 4. cpu2006.mcf
> >
> > ----- original pre-copy: -----
> > Iteration 1, duration: 31711 ms , transferred pages: 266450 (dup: 6925, rd: 259525, fd: 0) , new dirty pages: 244403 , remaining dirty pages: 244403
> > Iteration 2, duration: 29603 ms , transferred pages: 242275 (dup: 377, rd: 224001, fd: 17897) , new dirty pages: 227335 , remaining dirty pages: 229463
> > Iteration 3, duration: 27806 ms , transferred pages: 227573 (dup: 169, rd: 65681, fd: 161723) , new dirty pages: 195593 , remaining dirty pages: 197483
> > Iteration 4, duration: 23907 ms , transferred pages: 195543 (dup: 41, rd: 39838, fd: 155664) , new dirty pages: 215066 , remaining dirty pages: 217006
> > Iteration 5, duration: 26305 ms , transferred pages: 215289 (dup: 155, rd: 33082, fd: 182052) , new dirty pages: 111098 , remaining dirty pages: 112815
> > Iteration 6, duration: 13502 ms , transferred pages: 110452 (dup: 22, rd: 26793, fd: 83637) , new dirty pages: 161054 , remaining dirty pages: 163417
> > Iteration 7, duration: 19705 ms , transferred pages: 161266 (dup: 120, rd: 33818, fd: 127328) , new dirty pages: 220562 , remaining dirty pages: 222713
> > Iteration 8, duration: 27003 ms , transferred pages: 220881 (dup: 21, rd: 215721, fd: 5139) , new dirty pages: 219787 , remaining dirty pages: 221619
> > Iteration 9, duration: 26802 ms , transferred pages: 219248 (dup: 24, rd: 84648, fd: 134576) , new dirty pages: 207959 , remaining dirty pages: 210330
> > Iteration 10, duration: 25411 ms , transferred pages: 207916 (dup: 144, rd: 35842, fd: 171930) , new dirty pages: 144442 , remaining dirty pages: 146856
> > Iteration 11, duration: 17714 ms , transferred pages: 144804 (dup: 18, rd: 25414, fd: 119372) , new dirty pages: 205127 , remaining dirty pages: 207179
> > Iteration 12, duration: 25112 ms , transferred pages: 205446 (dup: 128, rd: 23197, fd: 182121) , new dirty pages: 167319 , remaining dirty pages: 169052
> > Iteration 13, duration: 20411 ms , transferred pages: 166886 (dup: 14, rd: 21960, fd: 144912) , new dirty pages: 221592 , remaining dirty pages: 223758
> > Iteration 14, duration: 27126 ms , transferred pages: 221800 (dup: 122, rd: 42368, fd: 179310) , new dirty pages: 233630 , remaining dirty pages: 235588
> > Iteration 15, duration: 28517 ms , transferred pages: 233321 (dup: 191, rd: 222528, fd: 10602) , new dirty pages: 224282 , remaining dirty pages: 226549
> > Iteration 16, duration: 27422 ms , transferred pages: 224187 (dup: 55, rd: 45773, fd: 178359) , new dirty pages: 209815 , remaining dirty pages: 212177
> > Iteration 17, duration: 25723 ms , transferred pages: 210260 (dup: 34, rd: 79405, fd: 130821) , new dirty pages: 220297 , remaining dirty pages: 222214
> > Iteration 18, duration: 26920 ms , transferred pages: 220056 (dup: 14, rd: 214128, fd: 5914) , new dirty pages: 192015 , remaining dirty pages: 194173
> > Iteration 19, duration: 23520 ms , transferred pages: 192239 (dup: 9, rd: 25140, fd: 167090) , new dirty pages: 96450 , remaining dirty pages: 98384
> > Iteration 20, duration: 11805 ms , transferred pages: 96538 (dup: 14, rd: 7424, fd: 89100) , new dirty pages: 6978 , remaining dirty pages: 8824
> > Iteration 21, duration: 799 ms , transferred pages: 6545 (dup: 1, rd: 1802, fd: 4742) , new dirty pages: 138 , remaining dirty pages: 2417
> > Iteration 22, duration: 1 ms , transferred pages: 0 (dup: 0, rd: 0, fd: 0) , new dirty pages: 0 , remaining dirty pages: 2417
> >
> > ----- after optimization: -----
> > Iteration 1, duration: 31711 ms , transferred pages: 266450 (dup: 6831, rd: 259619) , new dirty pages: 240209 , remaining dirty pages: 240209
> > Iteration 2, duration: 6250 ms , transferred pages: 51244 (dup: 211, rd: 51033) , new dirty pages: 226651 , remaining dirty pages: 228571
> > Iteration 3, duration: 4395 ms , transferred pages: 36008 (dup: 80, rd: 35928) , new dirty pages: 110719 , remaining dirty pages: 111478
> > Iteration 4, duration: 3390 ms , transferred pages: 28068 (dup: 28, rd: 28040) , new dirty pages: 185172 , remaining dirty pages: 185172
> > Iteration 5, duration: 2986 ms , transferred pages: 23780 (dup: 45, rd: 23735) , new dirty pages: 64357 , remaining dirty pages: 66305
> > Iteration 6, duration: 2727 ms , transferred pages: 22800 (dup: 12, rd: 22788) , new dirty pages: 61675 , remaining dirty pages: 61675
> > Iteration 7, duration: 2372 ms , transferred pages: 18943 (dup: 13, rd: 18930) , new dirty pages: 55144 , remaining dirty pages: 55265
> > Iteration 8, duration: 2100 ms , transferred pages: 17189 (dup: 11, rd: 17178) , new dirty pages: 55244 , remaining dirty pages: 55668
> > Iteration 9, duration: 2003 ms , transferred pages: 16371 (dup: 11, rd: 16360) , new dirty pages: 107058 , remaining dirty pages: 108014
> > Iteration 10, duration: 2132 ms , transferred pages: 17825 (dup: 24, rd: 17801) , new dirty pages: 126214 , remaining dirty pages: 126214
> > Iteration 11, duration: 2229 ms , transferred pages: 18156 (dup: 22, rd: 18134) , new dirty pages: 65725 , remaining dirty pages: 65725
> > Iteration 12, duration: 2315 ms , transferred pages: 18651 (dup: 21, rd: 18630) , new dirty pages: 52575 , remaining dirty pages: 53903
> > Iteration 13, duration: 2147 ms , transferred pages: 17435 (dup: 16, rd: 17419) , new dirty pages: 46652 , remaining dirty pages: 47260
> > Iteration 14, duration: 2000 ms , transferred pages: 16371 (dup: 11, rd: 16360) , new dirty pages: 42721 , remaining dirty pages: 43266
> > Iteration 15, duration: 1901 ms , transferred pages: 15552 (dup: 10, rd: 15542) , new dirty pages: 38593 , remaining dirty pages: 40792
> > Iteration 16, duration: 1801 ms , transferred pages: 14735 (dup: 11, rd: 14724) , new dirty pages: 54252 , remaining dirty pages: 55639
> > Iteration 17, duration: 1708 ms , transferred pages: 13860 (dup: 2, rd: 13858) , new dirty pages: 72379 , remaining dirty pages: 74170
> > Iteration 18, duration: 1923 ms , transferred pages: 15442 (dup: 12, rd: 15430) , new dirty pages: 101911 , remaining dirty pages: 103547
> > Iteration 19, duration: 2311 ms , transferred pages: 18823 (dup: 9, rd: 18814) , new dirty pages: 80534 , remaining dirty pages: 82521
> > Iteration 20, duration: 2081 ms , transferred pages: 17156 (dup: 34, rd: 17122) , new dirty pages: 36054 , remaining dirty pages: 36054
> > Iteration 21, duration: 1665 ms , transferred pages: 13777 (dup: 10, rd: 13767) , new dirty pages: 29624 , remaining dirty pages: 29624
> > Iteration 22, duration: 1657 ms , transferred pages: 13290 (dup: 7, rd: 13283) , new dirty pages: 25949 , remaining dirty pages: 28265
> > Iteration 23, duration: 1599 ms , transferred pages: 13088 (dup: 0, rd: 13088) , new dirty pages: 22356 , remaining dirty pages: 24813
> > Iteration 24, duration: 1500 ms , transferred pages: 12280 (dup: 10, rd: 12270) , new dirty pages: 21181 , remaining dirty pages: 22608
> > Iteration 25, duration: 1400 ms , transferred pages: 11457 (dup: 5, rd: 11452) , new dirty pages: 18657 , remaining dirty pages: 20311
> > Iteration 26, duration: 1200 ms , transferred pages: 9822 (dup: 6, rd: 9816) , new dirty pages: 15690 , remaining dirty pages: 17294
> > Iteration 27, duration: 1201 ms , transferred pages: 9822 (dup: 6, rd: 9816) , new dirty pages: 14810 , remaining dirty pages: 15936
> > Iteration 28, duration: 1000 ms , transferred pages: 8183 (dup: 3, rd: 8180) , new dirty pages: 15387 , remaining dirty pages: 16423
> > Iteration 29, duration: 900 ms , transferred pages: 7372 (dup: 10, rd: 7362) , new dirty pages: 13303 , remaining dirty pages: 15292
> > Iteration 30, duration: 1000 ms , transferred pages: 8181 (dup: 1, rd: 8180) , new dirty pages: 17879 , remaining dirty pages: 18457
> > Iteration 31, duration: 951 ms , transferred pages: 8140 (dup: 9, rd: 8131) , new dirty pages: 21738 , remaining dirty pages: 23304
> > Iteration 32, duration: 946 ms , transferred pages: 6946 (dup: 1, rd: 6945) , new dirty pages: 15815 , remaining dirty pages: 15815
> > Iteration 33, duration: 747 ms , transferred pages: 6192 (dup: 0, rd: 6192) , new dirty pages: 6249 , remaining dirty pages: 7670
> > Iteration 34, duration: 501 ms , transferred pages: 4090 (dup: 0, rd: 4090) , new dirty pages: 6163 , remaining dirty pages: 8422
> > Iteration 35, duration: 600 ms , transferred pages: 4910 (dup: 2, rd: 4908) , new dirty pages: 3673 , remaining dirty pages: 5222
> > Iteration 36, duration: 300 ms , transferred pages: 2454 (dup: 0, rd: 2454) , new dirty pages: 2132 , remaining dirty pages: 4337
> > Iteration 37, duration: 200 ms , transferred pages: 1637 (dup: 1, rd: 1636) , new dirty pages: 544 , remaining dirty pages: 2251
> > Iteration 38, duration: 0 ms , transferred pages: 0 (dup: 0, rd: 0) , new dirty pages: 0 , remaining dirty pages: 2251
> > total time: 97919 milliseconds
> >
> > ------------------The other 11 workloads without notable improvements (only the result of original precopy is shown)-------------------
> >
> > 5. idle
> >
> > Iteration 1, duration: 14702 ms , transferred pages: 266450 (dup: 146393, rd: 120057, fd: 0) , new dirty pages: 14595 , remaining dirty pages: 14595
> > Iteration 2, duration: 1592 ms , transferred pages: 12412 (dup: 103, rd: 3280, fd: 9029) , new dirty pages: 218 , remaining dirty pages: 2401
> > Iteration 3, duration: 0 ms , transferred pages: 0 (dup: 0, rd: 0, fd: 0) , new dirty pages: 0 , remaining dirty pages: 2401
> >
> > 6. kernel compilation (can not converge)
> >
> > Iteration 1, duration: 20607 ms , transferred pages: 266450 (dup: 97552, rd: 168898, fd: 0) , new dirty pages: 19293 , remaining dirty pages: 19293
> > Iteration 2, duration: 2092 ms , transferred pages: 17176 (dup: 597, rd: 8625, fd: 7954) , new dirty pages: 8318 , remaining dirty pages: 10435
> > Iteration 3, duration: 1000 ms , transferred pages: 8484 (dup: 304, rd: 6256, fd: 1924) , new dirty pages: 8736 , remaining dirty pages: 10687
> > Iteration 4, duration: 1000 ms , transferred pages: 8435 (dup: 255, rd: 7089, fd: 1091) , new dirty pages: 7627 , remaining dirty pages: 9879
> > Iteration 5, duration: 900 ms , transferred pages: 7553 (dup: 191, rd: 5602, fd: 1760) , new dirty pages: 7287 , remaining dirty pages: 9613
> > Iteration 6, duration: 900 ms , transferred pages: 7620 (dup: 258, rd: 5761, fd: 1601) , new dirty pages: 8958 , remaining dirty pages: 10951
> > Iteration 7, duration: 1099 ms , transferred pages: 9309 (dup: 311, rd: 8051, fd: 947) , new dirty pages: 7189 , remaining dirty pages: 8831
> > Iteration 8, duration: 800 ms , transferred pages: 6832 (dup: 288, rd: 5717, fd: 827) , new dirty pages: 5782 , remaining dirty pages: 7781
> > Iteration 9, duration: 701 ms , transferred pages: 5875 (dup: 149, rd: 4005, fd: 1721) , new dirty pages: 4587 , remaining dirty pages: 6493
> > Iteration 10, duration: 500 ms , transferred pages: 4234 (dup: 144, rd: 3057, fd: 1033) , new dirty pages: 7352 , remaining dirty pages: 9611
> > Iteration 11, duration: 900 ms , transferred pages: 7759 (dup: 397, rd: 6563, fd: 799) , new dirty pages: 6686 , remaining dirty pages: 8538
> > Iteration 12, duration: 800 ms , transferred pages: 6808 (dup: 264, rd: 6017, fd: 527) , new dirty pages: 6871 , remaining dirty pages: 8601
> > Iteration 13, duration: 800 ms , transferred pages: 6775 (dup: 231, rd: 5722, fd: 822) , new dirty pages: 7540 , remaining dirty pages: 9366
> > Iteration 14, duration: 900 ms , transferred pages: 7507 (dup: 145, rd: 5900, fd: 1462) , new dirty pages: 7581 , remaining dirty pages: 9440
> > Iteration 15, duration: 900 ms , transferred pages: 7630 (dup: 268, rd: 6211, fd: 1151) , new dirty pages: 7268 , remaining dirty pages: 9078
> > Iteration 16, duration: 800 ms , transferred pages: 6759 (dup: 215, rd: 5763, fd: 781) , new dirty pages: 6861 , remaining dirty pages: 9180
> > Iteration 17, duration: 800 ms , transferred pages: 6838 (dup: 294, rd: 6037, fd: 507) , new dirty pages: 6196 , remaining dirty pages: 8538
> > Iteration 18, duration: 800 ms , transferred pages: 6852 (dup: 308, rd: 4905, fd: 1639) , new dirty pages: 5947 , remaining dirty pages: 7633
> > Iteration 19, duration: 700 ms , transferred pages: 5919 (dup: 193, rd: 4853, fd: 873) , new dirty pages: 5861 , remaining dirty pages: 7575
> > Iteration 20, duration: 600 ms , transferred pages: 5284 (dup: 376, rd: 4408, fd: 500) , new dirty pages: 5206 , remaining dirty pages: 7497
> > Iteration 21, duration: 600 ms , transferred pages: 5147 (dup: 239, rd: 4308, fd: 600) , new dirty pages: 5031 , remaining dirty pages: 7381
> > Iteration 22, duration: 599 ms , transferred pages: 5064 (dup: 156, rd: 4026, fd: 882) , new dirty pages: 5601 , remaining dirty pages: 7918
> > Iteration 23, duration: 702 ms , transferred pages: 5965 (dup: 239, rd: 5028, fd: 698) , new dirty pages: 6079 , remaining dirty pages: 8032
> > Iteration 24, duration: 700 ms , transferred pages: 6175 (dup: 449, rd: 5146, fd: 580) , new dirty pages: 10932 , remaining dirty pages: 12789
> > Iteration 25, duration: 1300 ms , transferred pages: 10936 (dup: 302, rd: 6205, fd: 4429) , new dirty pages: 8713 , remaining dirty pages: 10566
> > Iteration 26, duration: 1000 ms , transferred pages: 8282 (dup: 102, rd: 5662, fd: 2518) , new dirty pages: 5119 , remaining dirty pages: 7403
> > Iteration 27, duration: 600 ms , transferred pages: 5007 (dup: 99, rd: 4099, fd: 809) , new dirty pages: 2226 , remaining dirty pages: 4622
> > Iteration 28, duration: 300 ms , transferred pages: 2491 (dup: 37, rd: 1794, fd: 660) , new dirty pages: 6746 , remaining dirty pages: 8877
> > Iteration 29, duration: 800 ms , transferred pages: 6757 (dup: 213, rd: 5532, fd: 1012) , new dirty pages: 6070 , remaining dirty pages: 8190
> > Iteration 30, duration: 700 ms , transferred pages: 6052 (dup: 326, rd: 5107, fd: 619) , new dirty pages: 5177 , remaining dirty pages: 7315
> >
> > 7. web server
> >
> > Iteration 1, duration: 20902 ms , transferred pages: 266450 (dup: 95497, rd: 170953, fd: 0) , new dirty pages: 8528 , remaining dirty pages: 8528
> > Iteration 2, duration: 796 ms , transferred pages: 6472 (dup: 131, rd: 1885, fd: 4456) , new dirty pages: 650 , remaining dirty pages: 2706
> > Iteration 3, duration: 100 ms , transferred pages: 818 (dup: 0, rd: 383, fd: 435) , new dirty pages: 328 , remaining dirty pages: 2216
> > Iteration 4, duration: 0 ms , transferred pages: 0 (dup: 0, rd: 0, fd: 0) , new dirty pages: 0 , remaining dirty pages: 2216
> >
> >
> > 8. cpu2006.bwaves (can not converge)
> >
> > Iteration 1, duration: 31715 ms , transferred pages: 266450 (dup: 6766, rd: 259684, fd: 0) , new dirty pages: 242702 , remaining dirty pages: 242702
> > Iteration 2, duration: 29397 ms , transferred pages: 240508 (dup: 405, rd: 225588, fd: 14515) , new dirty pages: 230889 , remaining dirty pages: 233083
> > Iteration 3, duration: 28205 ms , transferred pages: 230858 (dup: 182, rd: 214596, fd: 16080) , new dirty pages: 226998 , remaining dirty pages: 229223
> > Iteration 4, duration: 27805 ms , transferred pages: 227574 (dup: 170, rd: 217045, fd: 10359) , new dirty pages: 227360 , remaining dirty pages: 229009
> > Iteration 5, duration: 27703 ms , transferred pages: 226786 (dup: 200, rd: 212130, fd: 14456) , new dirty pages: 225885 , remaining dirty pages: 228108
> > Iteration 6, duration: 27600 ms , transferred pages: 225923 (dup: 155, rd: 215503, fd: 10265) , new dirty pages: 223555 , remaining dirty pages: 225740
> > Iteration 7, duration: 27309 ms , transferred pages: 223574 (dup: 260, rd: 215641, fd: 7673) , new dirty pages: 231975 , remaining dirty pages: 234141
> > Iteration 8, duration: 28403 ms , transferred pages: 232397 (dup: 85, rd: 214086, fd: 18226) , new dirty pages: 222170 , remaining dirty pages: 223914
> > Iteration 9, duration: 27105 ms , transferred pages: 221809 (dup: 131, rd: 214988, fd: 6690) , new dirty pages: 230065 , remaining dirty pages: 232170
> > Iteration 10, duration: 28104 ms , transferred pages: 230201 (dup: 343, rd: 213531, fd: 16327) , new dirty pages: 227590 , remaining dirty pages: 229559
> > Iteration 11, duration: 27801 ms , transferred pages: 227717 (dup: 313, rd: 221408, fd: 5996) , new dirty pages: 228457 , remaining dirty pages: 230299
> > Iteration 12, duration: 27916 ms , transferred pages: 228560 (dup: 338, rd: 219660, fd: 8562) , new dirty pages: 238326 , remaining dirty pages: 240065
> >
> > 9. cpu2006.lbm (can not converge)
> > Iteration 1, duration: 31012 ms , transferred pages: 266450 (dup: 12253, rd: 254197, fd: 0) , new dirty pages: 108960 , remaining dirty pages: 108960
> > Iteration 2, duration: 13095 ms , transferred pages: 106522 (dup: 3, rd: 102045, fd: 4474) , new dirty pages: 129292 , remaining dirty pages: 131730
> > Iteration 3, duration: 15802 ms , transferred pages: 129688 (dup: 444, rd: 110860, fd: 18384) , new dirty pages: 116682 , remaining dirty pages: 118724
> > Iteration 4, duration: 14204 ms , transferred pages: 116316 (dup: 160, rd: 104951, fd: 11205) , new dirty pages: 107246 , remaining dirty pages: 109654
> > Iteration 5, duration: 13208 ms , transferred pages: 107977 (dup: 1, rd: 101834, fd: 6142) , new dirty pages: 105371 , remaining dirty pages: 107048
> > Iteration 6, duration: 12804 ms , transferred pages: 104705 (dup: 1, rd: 99629, fd: 5075) , new dirty pages: 103841 , remaining dirty pages: 106184
> > Iteration 7, duration: 12709 ms , transferred pages: 103891 (dup: 5, rd: 99212, fd: 4674) , new dirty pages: 106692 , remaining dirty pages: 108985
> > Iteration 8, duration: 13105 ms , transferred pages: 107169 (dup: 11, rd: 100125, fd: 7033) , new dirty pages: 103132 , remaining dirty pages: 104948
> > Iteration 9, duration: 12607 ms , transferred pages: 103068 (dup: 0, rd: 99460, fd: 3608) , new dirty pages: 102511 , remaining dirty pages: 104391
> > Iteration 10, duration: 12514 ms , transferred pages: 102250 (dup: 0, rd: 99094, fd: 3156) , new dirty pages: 102888 , remaining dirty pages: 105029
> >
> > 10. cpu2006.astar (can not converge)
> >
> > Iteration 1, duration: 28402 ms , transferred pages: 266450 (dup: 33770, rd: 232680, fd: 0) , new dirty pages: 62078 , remaining dirty pages: 62078
> > Iteration 2, duration: 7393 ms , transferred pages: 60107 (dup: 10, rd: 51722, fd: 8375) , new dirty pages: 48854 , remaining dirty pages: 50825
> > Iteration 3, duration: 6001 ms , transferred pages: 49094 (dup: 14, rd: 46540, fd: 2540) , new dirty pages: 48137 , remaining dirty pages: 49868
> > Iteration 4, duration: 5800 ms , transferred pages: 47444 (dup: 0, rd: 45389, fd: 2055) , new dirty pages: 49147 , remaining dirty pages: 51571
> > Iteration 5, duration: 6102 ms , transferred pages: 49912 (dup: 14, rd: 46216, fd: 3682) , new dirty pages: 55606 , remaining dirty pages: 57265
> > Iteration 6, duration: 6699 ms , transferred pages: 54949 (dup: 143, rd: 20745, fd: 34061) , new dirty pages: 9166 , remaining dirty pages: 11482
> > Iteration 7, duration: 1200 ms , transferred pages: 9830 (dup: 14, rd: 7011, fd: 2805) , new dirty pages: 8294 , remaining dirty pages: 9946
> > Iteration 8, duration: 1000 ms , transferred pages: 8194 (dup: 14, rd: 7178, fd: 1002) , new dirty pages: 5475 , remaining dirty pages: 7227
> > Iteration 9, duration: 600 ms , transferred pages: 4908 (dup: 0, rd: 3470, fd: 1438) , new dirty pages: 4175 , remaining dirty pages: 6494
> > Iteration 10, duration: 500 ms , transferred pages: 4090 (dup: 0, rd: 3856, fd: 234) , new dirty pages: 4095 , remaining dirty pages: 6499
> > Iteration 11, duration: 500 ms , transferred pages: 4090 (dup: 0, rd: 3313, fd: 777) , new dirty pages: 3371 , remaining dirty pages: 5780
> > Iteration 12, duration: 502 ms , transferred pages: 4090 (dup: 0, rd: 3823, fd: 267) , new dirty pages: 7518 , remaining dirty pages: 9208
> > Iteration 13, duration: 899 ms , transferred pages: 7376 (dup: 14, rd: 6028, fd: 1334) , new dirty pages: 3931 , remaining dirty pages: 5763
> > Iteration 14, duration: 500 ms , transferred pages: 4090 (dup: 0, rd: 4078, fd: 12) , new dirty pages: 4346 , remaining dirty pages: 6019
> > Iteration 15, duration: 502 ms , transferred pages: 4090 (dup: 0, rd: 3817, fd: 273) , new dirty pages: 3054 , remaining dirty pages: 4983
> > Iteration 16, duration: 400 ms , transferred pages: 3272 (dup: 0, rd: 3138, fd: 134) , new dirty pages: 3874 , remaining dirty pages: 5585
> > Iteration 17, duration: 399 ms , transferred pages: 3272 (dup: 0, rd: 3248, fd: 24) , new dirty pages: 5285 , remaining dirty pages: 7598
> > Iteration 18, duration: 701 ms , transferred pages: 5726 (dup: 0, rd: 4385, fd: 1341) , new dirty pages: 8903 , remaining dirty pages: 10775
> > Iteration 19, duration: 1101 ms , transferred pages: 9010 (dup: 12, rd: 5597, fd: 3401) , new dirty pages: 4199 , remaining dirty pages: 5964
> > Iteration 20, duration: 500 ms , transferred pages: 4090 (dup: 0, rd: 4078, fd: 12) , new dirty pages: 3829 , remaining dirty pages: 5703
> >
> > 11. cpu2006.xalancbmk (can not converge)
> >
> > Iteration 1, duration: 30407 ms , transferred pages: 266450 (dup: 17700, rd: 248750, fd: 0) , new dirty pages: 96169 , remaining dirty pages: 96169
> > Iteration 2, duration: 11495 ms , transferred pages: 94164 (dup: 205, rd: 67068, fd: 26891) , new dirty pages: 61766 , remaining dirty pages: 63771
> > Iteration 3, duration: 7501 ms , transferred pages: 61471 (dup: 121, rd: 53587, fd: 7763) , new dirty pages: 56569 , remaining dirty pages: 58869
> > Iteration 4, duration: 6902 ms , transferred pages: 56461 (dup: 19, rd: 50553, fd: 5889) , new dirty pages: 52181 , remaining dirty pages: 54589
> > Iteration 5, duration: 6402 ms , transferred pages: 52459 (dup: 107, rd: 46986, fd: 5366) , new dirty pages: 54051 , remaining dirty pages: 56181
> > Iteration 6, duration: 6601 ms , transferred pages: 54003 (dup: 15, rd: 47566, fd: 6422) , new dirty pages: 50844 , remaining dirty pages: 53022
> > Iteration 7, duration: 6202 ms , transferred pages: 50723 (dup: 7, rd: 47143, fd: 3573) , new dirty pages: 64880 , remaining dirty pages: 67179
> > Iteration 8, duration: 8001 ms , transferred pages: 65447 (dup: 7, rd: 61159, fd: 4281) , new dirty pages: 67854 , remaining dirty pages: 69586
> > Iteration 9, duration: 8202 ms , transferred pages: 67444 (dup: 368, rd: 56357, fd: 10719) , new dirty pages: 65178 , remaining dirty pages: 67320
> > Iteration 10, duration: 8000 ms , transferred pages: 65455 (dup: 15, rd: 60581, fd: 4859) , new dirty pages: 52421 , remaining dirty pages: 54286
> >
> > 12. cpu2006.milc (can not converge)
> >
> > Iteration 1, duration: 31410 ms , transferred pages: 266450 (dup: 9454, rd: 256996, fd: 0) , new dirty pages: 158860 , remaining dirty pages: 158860
> > Iteration 2, duration: 19193 ms , transferred pages: 157048 (dup: 150, rd: 96807, fd: 60091) , new dirty pages: 102238 , remaining dirty pages: 104050
> > Iteration 3, duration: 12504 ms , transferred pages: 102271 (dup: 21, rd: 95107, fd: 7143) , new dirty pages: 97944 , remaining dirty pages: 99723
> > Iteration 4, duration: 11905 ms , transferred pages: 97360 (dup: 18, rd: 93610, fd: 3732) , new dirty pages: 99150 , remaining dirty pages: 101513
> > Iteration 5, duration: 12105 ms , transferred pages: 99094 (dup: 116, rd: 94125, fd: 4853) , new dirty pages: 98589 , remaining dirty pages: 101008
> > Iteration 6, duration: 12101 ms , transferred pages: 98995 (dup: 17, rd: 94069, fd: 4909) , new dirty pages: 147403 , remaining dirty pages: 149416
> > Iteration 7, duration: 18001 ms , transferred pages: 147284 (dup: 44, rd: 135691, fd: 11549) , new dirty pages: 136445 , remaining dirty pages: 138577
> > Iteration 8, duration: 16702 ms , transferred pages: 136636 (dup: 30, rd: 130805, fd: 5801) , new dirty pages: 145481 , remaining dirty pages: 147422
> > Iteration 9, duration: 17800 ms , transferred pages: 145734 (dup: 130, rd: 133239, fd: 12365) , new dirty pages: 98032 , remaining dirty pages: 99720
> > Iteration 10, duration: 11902 ms , transferred pages: 97364 (dup: 22, rd: 93096, fd: 4246) , new dirty pages: 95391 , remaining dirty pages: 97747
> >
> > 13. cpu2006.cactusADM (can not converge)
> >
> > Iteration 1, duration: 23508 ms , transferred pages: 266450 (dup: 73568, rd: 192882, fd: 0) , new dirty pages: 123869 , remaining dirty pages: 123869
> > Iteration 2, duration: 13989 ms , transferred pages: 121594 (dup: 7874, rd: 81653, fd: 32067) , new dirty pages: 112960 , remaining dirty pages: 115235
> > Iteration 3, duration: 13605 ms , transferred pages: 113276 (dup: 2028, rd: 83783, fd: 27465) , new dirty pages: 112314 , remaining dirty pages: 114273
> > Iteration 4, duration: 13509 ms , transferred pages: 111935 (dup: 1505, rd: 83535, fd: 26895) , new dirty pages: 114078 , remaining dirty pages: 116416
> > Iteration 5, duration: 13810 ms , transferred pages: 114262 (dup: 1378, rd: 84039, fd: 28845) , new dirty pages: 112271 , remaining dirty pages: 114425
> > Iteration 6, duration: 13604 ms , transferred pages: 112664 (dup: 1416, rd: 84300, fd: 26948) , new dirty pages: 112903 , remaining dirty pages: 114664
> > Iteration 7, duration: 13604 ms , transferred pages: 112655 (dup: 1407, rd: 84027, fd: 27221) , new dirty pages: 110943 , remaining dirty pages: 112952
> > Iteration 8, duration: 13406 ms , transferred pages: 110720 (dup: 1108, rd: 84075, fd: 25537) , new dirty pages: 109321 , remaining dirty pages: 111553
> > Iteration 9, duration: 13306 ms , transferred pages: 109726 (dup: 932, rd: 83652, fd: 25142) , new dirty pages: 113446 , remaining dirty pages: 115273
> > Iteration 10, duration: 13705 ms , transferred pages: 113121 (dup: 1055, rd: 84671, fd: 27395) , new dirty pages: 108776 , remaining dirty pages: 110928
> >
> > 14. cpu2006.GmesFDTD (can not converge)
> >
> > Iteration 1, duration: 13303 ms , transferred pages: 266450 (dup: 157809, rd: 108641, fd: 0) , new dirty pages: 226802 , remaining dirty pages: 226802
> > Iteration 2, duration: 10797 ms , transferred pages: 226507 (dup: 138637, rd: 61818, fd: 26052) , new dirty pages: 200769 , remaining dirty pages: 201064
> > Iteration 3, duration: 8900 ms , transferred pages: 199717 (dup: 127187, rd: 69340, fd: 3190) , new dirty pages: 203436 , remaining dirty pages: 204783
> > Iteration 4, duration: 10904 ms , transferred pages: 204127 (dup: 115211, rd: 85767, fd: 3149) , new dirty pages: 198407 , remaining dirty pages: 199063
> > Iteration 5, duration: 12109 ms , transferred pages: 198206 (dup: 99435, rd: 96956, fd: 1815) , new dirty pages: 213719 , remaining dirty pages: 214576
> > Iteration 6, duration: 16307 ms , transferred pages: 213595 (dup: 80422, rd: 116885, fd: 16288) , new dirty pages: 199637 , remaining dirty pages: 200618
> > Iteration 7, duration: 16915 ms , transferred pages: 198289 (dup: 60169, rd: 134208, fd: 3912) , new dirty pages: 199343 , remaining dirty pages: 201672
> > Iteration 8, duration: 19518 ms , transferred pages: 200452 (dup: 41014, rd: 156083, fd: 3355) , new dirty pages: 222927 , remaining dirty pages: 224147
> >
> > 15. cpu2006.wrf (can not converge)
> >
> > Iteration 1, duration: 18499 ms , transferred pages: 266380 (dup: 115285, rd: 151095, fd: 0) , new dirty pages: 112322 , remaining dirty pages: 112392
> > Iteration 2, duration: 9802 ms , transferred pages: 110025 (dup: 29917, rd: 65782, fd: 14326) , new dirty pages: 88855 , remaining dirty pages: 91222
> > Iteration 3, duration: 8199 ms , transferred pages: 89761 (dup: 22728, rd: 57262, fd: 9771) , new dirty pages: 58431 , remaining dirty pages: 59892
> > Iteration 4, duration: 5603 ms , transferred pages: 58502 (dup: 12716, rd: 41809, fd: 3977) , new dirty pages: 80556 , remaining dirty pages: 81946
> > Iteration 5, duration: 7101 ms , transferred pages: 79778 (dup: 21738, rd: 50896, fd: 7144) , new dirty pages: 62592 , remaining dirty pages: 64760
> > Iteration 6, duration: 5702 ms , transferred pages: 63388 (dup: 16793, rd: 42726, fd: 3869) , new dirty pages: 80747 , remaining dirty pages: 82119
> > Iteration 7, duration: 7000 ms , transferred pages: 80868 (dup: 23652, rd: 52194, fd: 5022) , new dirty pages: 84593 , remaining dirty pages: 85844
> > Iteration 8, duration: 7099 ms , transferred pages: 83799 (dup: 25769, rd: 51772, fd: 6258) , new dirty pages: 67951 , remaining dirty pages: 69996
> > Iteration 9, duration: 6303 ms , transferred pages: 68478 (dup: 16979, rd: 36490, fd: 15009) , new dirty pages: 81181 , remaining dirty pages: 82699
> > Iteration 10, duration: 7000 ms , transferred pages: 80724 (dup: 23503, rd: 52826, fd: 4395) , new dirty pages: 47930 , remaining dirty pages: 49905
> >
> >
> >
> > >
> > > > So I think "booting" and "kernel compilation" should benefit a lot from this
> > > > improvement. The reason of "kernel compilation" would benefit is that some
> > > > iterations take around 600ms, and if they are halved into 300ms, then the precopy
> > > > may have the chance to step into stop and copy phase.
> > > >
> > > > On the other hand, "idle" and "web server" would not benefit a lot, because
> > > > most of the time are spent on the 1st iteration and little on the others.
> > > >
> > > > As to the "zeusmp" and "memcached", although the time spent on the other iterations
> > > > but the 1st one may be halved, they still could not converge to stop and copy
> > > > with the 300ms downtime.
> > > >
> > > > --------------------1 vcpu, 1 GB ram, default bandwidth (32MB/s):------------------
> > > >
> > > > 1. booting : begin to migrate when the VM is booting
> > > >
> > > > Iteration 1, duration: 6997 ms , transferred pages: 266450 (n: 57269, d: 209181 ) , new dirty pages: 56414 , remaining dirty pages: 56414
> > > > Iteration 2, duration: 6497 ms , transferred pages: 54008 (n: 52701, d: 1307 ) , new dirty pages: 48053 , remaining dirty pages: 50459
> > > > Iteration 3, duration: 5800 ms , transferred pages: 48232 (n: 47444, d: 788 ) , new dirty pages: 9129 , remaining dirty pages: 11356
> > > > Iteration 4, duration: 1100 ms , transferred pages: 9091 (n: 8998, d: 93 ) , new dirty pages: 165 , remaining dirty pages: 2430
> > > > Iteration 5, duration: 1 ms , transferred pages: 0 (n: 0, d: 0 ) , new dirty pages: 0 , remaining dirty pages: 2430
> > > > (note: When the workload does converge, the output of the last iteration is "fake". It just indicates that the precopy steps into stop-copy phase now.
> > > > "n" means "normal pages" and "d" means "duplicate (zero) pages".)
> > > >
> > > > 2. idle
> > > >
> > > > Iteration 1, duration: 14496 ms , transferred pages: 266450 (n: 118980, d: 147470 ) , new dirty pages: 17398 , remaining dirty pages: 17398
> > > > Iteration 2, duration: 1896 ms , transferred pages: 14953 (n: 14854, d: 99 ) , new dirty pages: 1849 , remaining dirty pages: 4294
> > > > Iteration 3, duration: 300 ms , transferred pages: 2454 (n: 2454, d: 0 ) , new dirty pages: 9 , remaining dirty pages: 1849
> > > > Iteration 4, duration: 1 ms , transferred pages: 0 (n: 0, d: 0 ) , new dirty pages: 0 , remaining dirty pages: 1849
> > > >
> > > > 3. kernel compilation (can not converge)
> > > >
> > > > Iteration 1, duration: 20700 ms , transferred pages: 266450 (n: 169778, d: 96672 ) , new dirty pages: 40067 , remaining dirty pages: 40067
> > > > Iteration 2, duration: 4696 ms , transferred pages: 38401 (n: 37787, d: 614 ) , new dirty pages: 8852 , remaining dirty pages: 10518
> > > > Iteration 3, duration: 1000 ms , transferred pages: 8642 (n: 8180, d: 462 ) , new dirty pages: 6331 , remaining dirty pages: 8207
> > > > Iteration 4, duration: 700 ms , transferred pages: 6110 (n: 5726, d: 384 ) , new dirty pages: 5242 , remaining dirty pages: 7339
> > > > Iteration 5, duration: 600 ms , transferred pages: 5007 (n: 4908, d: 99 ) , new dirty pages: 4868 , remaining dirty pages: 7200
> > > > Iteration 6, duration: 600 ms , transferred pages: 5226 (n: 4908, d: 318 ) , new dirty pages: 6142 , remaining dirty pages: 8116
> > > > Iteration 7, duration: 700 ms , transferred pages: 5985 (n: 5726, d: 259 ) , new dirty pages: 5902 , remaining dirty pages: 8033
> > > > Iteration 8, duration: 701 ms , transferred pages: 5893 (n: 5726, d: 167 ) , new dirty pages: 7502 , remaining dirty pages: 9642
> > > > Iteration 9, duration: 900 ms , transferred pages: 7623 (n: 7362, d: 261 ) , new dirty pages: 6408 , remaining dirty pages: 8427
> > > > Iteration 10, duration: 700 ms , transferred pages: 6008 (n: 5726, d: 282 ) , new dirty pages: 8312 , remaining dirty pages: 10731
> > > > Iteration 11, duration: 1000 ms , transferred pages: 8353 (n: 8180, d: 173 ) , new dirty pages: 6874 , remaining dirty pages: 9252
> > > > Iteration 12, duration: 899 ms , transferred pages: 7477 (n: 7362, d: 115 ) , new dirty pages: 5573 , remaining dirty pages: 7348
> > > > Iteration 13, duration: 601 ms , transferred pages: 5099 (n: 4908, d: 191 ) , new dirty pages: 7671 , remaining dirty pages: 9920
> > > > Iteration 14, duration: 900 ms , transferred pages: 7586 (n: 7362, d: 224 ) , new dirty pages: 7359 , remaining dirty pages: 9693
> > > > Iteration 15, duration: 900 ms , transferred pages: 7682 (n: 7362, d: 320 ) , new dirty pages: 7371 , remaining dirty pages: 9382
> > > >
> > > > 4. cpu2006.zeusmp (can not converge)
> > > >
> > > > Iteration 1, duration: 21603 ms , transferred pages: 266450 (n: 176660, d: 89790 ) , new dirty pages: 145625 , remaining dirty pages: 145625
> > > > Iteration 2, duration: 8696 ms , transferred pages: 144389 (n: 70862, d: 73527 ) , new dirty pages: 125124 , remaining dirty pages: 126360
> > > > Iteration 3, duration: 6301 ms , transferred pages: 124057 (n: 51379, d: 72678 ) , new dirty pages: 122528 , remaining dirty pages: 124831
> > > > Iteration 4, duration: 6400 ms , transferred pages: 124330 (n: 52196, d: 72134 ) , new dirty pages: 124267 , remaining dirty pages: 124768
> > > > Iteration 5, duration: 6703 ms , transferred pages: 124034 (n: 54656, d: 69378 ) , new dirty pages: 124151 , remaining dirty pages: 124885
> > > > Iteration 6, duration: 6703 ms , transferred pages: 124357 (n: 54658, d: 69699 ) , new dirty pages: 124106 , remaining dirty pages: 124634
> > > > Iteration 7, duration: 6602 ms , transferred pages: 124568 (n: 53838, d: 70730 ) , new dirty pages: 133828 , remaining dirty pages: 133894
> > > > Iteration 8, duration: 7600 ms , transferred pages: 133030 (n: 62021, d: 71009 ) , new dirty pages: 126612 , remaining dirty pages: 127476
> > > > Iteration 9, duration: 7299 ms , transferred pages: 126511 (n: 59569, d: 66942 ) , new dirty pages: 122727 , remaining dirty pages: 123692
> > > > Iteration 10, duration: 6609 ms , transferred pages: 123692 (n: 54539, d: 69153 ) , new dirty pages: 122727 , remaining dirty pages: 122727
> > > > Iteration 11, duration: 6995 ms , transferred pages: 120347 (n: 56423, d: 63924 ) , new dirty pages: 121430 , remaining dirty pages: 123810
> > > > Iteration 12, duration: 6703 ms , transferred pages: 123040 (n: 54657, d: 68383 ) , new dirty pages: 122043 , remaining dirty pages: 122813
> > > > Iteration 13, duration: 7006 ms , transferred pages: 122353 (n: 57121, d: 65232 ) , new dirty pages: 133869 , remaining dirty pages: 134329
> > > > Iteration 14, duration: 8209 ms , transferred pages: 132325 (n: 66932, d: 65393 ) , new dirty pages: 126914 , remaining dirty pages: 128918
> > > > Iteration 15, duration: 7802 ms , transferred pages: 126931 (n: 63671, d: 63260 ) , new dirty pages: 122351 , remaining dirty pages: 124338
> > > >
> > > > 5. web server : An apache web server. The client is configured with 50 concurrent connections.
> > > >
> > > > Iteration 1, duration: 30697 ms , transferred pages: 266450 (n: 251215, d: 15235 ) , new dirty pages: 30628 , remaining dirty pages: 30628
> > > > Iteration 2, duration: 3496 ms , transferred pages: 28859 (n: 28513, d: 346 ) , new dirty pages: 5805 , remaining dirty pages: 7574
> > > > Iteration 3, duration: 701 ms , transferred pages: 5746 (n: 5726, d: 20 ) , new dirty pages: 3433 , remaining dirty pages: 5261
> > > > Iteration 4, duration: 400 ms , transferred pages: 3281 (n: 3272, d: 9 ) , new dirty pages: 1539 , remaining dirty pages: 3519
> > > > Iteration 5, duration: 199 ms , transferred pages: 1653 (n: 1636, d: 17 ) , new dirty pages: 301 , remaining dirty pages: 2167
> > > > Iteration 6, duration: 1 ms , transferred pages: 0 (n: 0, d: 0 ) , new dirty pages: 0 , remaining dirty pages: 2167
> > > >
> > > > --------------------6 vcpu, 6 GB ram, max bandwidth (941.08 mbps):------------------
> > > >
> > > > 6. memcached : 4 GB cache, memaslap: all write, concurrency = 5 (can not converge)
> > > >
> > > > Iteration 1, duration: 42486 ms , transferred pages: 1568087 (n: 1216079, d: 352008 ) , new dirty pages: 571940 , remaining dirty pages: 581023
> > > > Iteration 2, duration: 19774 ms , transferred pages: 571700 (n: 567416, d: 4284 ) , new dirty pages: 331690 , remaining dirty pages: 341013
> > > > Iteration 3, duration: 11589 ms , transferred pages: 332187 (n: 332095, d: 92 ) , new dirty pages: 222725 , remaining dirty pages: 231551
> > > > Iteration 4, duration: 7790 ms , transferred pages: 223571 (n: 223499, d: 72 ) , new dirty pages: 157658 , remaining dirty pages: 165638
> > > > Iteration 5, duration: 5518 ms , transferred pages: 158056 (n: 157998, d: 58 ) , new dirty pages: 128130 , remaining dirty pages: 135712
> > > > Iteration 6, duration: 4442 ms , transferred pages: 127764 (n: 127701, d: 63 ) , new dirty pages: 104839 , remaining dirty pages: 112787
> > > > Iteration 7, duration: 3649 ms , transferred pages: 104581 (n: 104523, d: 58 ) , new dirty pages: 100736 , remaining dirty pages: 108942
> > > > Iteration 8, duration: 3532 ms , transferred pages: 101379 (n: 101315, d: 64 ) , new dirty pages: 87869 , remaining dirty pages: 95432
> > > > Iteration 9, duration: 3030 ms , transferred pages: 86841 (n: 86786, d: 55 ) , new dirty pages: 77505 , remaining dirty pages: 86096
> > > > Iteration 10, duration: 2709 ms , transferred pages: 77875 (n: 77814, d: 61 ) , new dirty pages: 77197 , remaining dirty pages: 85418
> > > > Iteration 11, duration: 2696 ms , transferred pages: 77107 (n: 77044, d: 63 ) , new dirty pages: 65010 , remaining dirty pages: 73321
> > > > Iteration 12, duration: 2308 ms , transferred pages: 66540 (n: 66484, d: 56 ) , new dirty pages: 64388 , remaining dirty pages: 71169
> > > > Iteration 13, duration: 2198 ms , transferred pages: 62953 (n: 62897, d: 56 ) , new dirty pages: 62773 , remaining dirty pages: 70989
> > > > Iteration 14, duration: 2214 ms , transferred pages: 63466 (n: 63411, d: 55 ) , new dirty pages: 67538 , remaining dirty pages: 75061
> > > > Iteration 15, duration: 2329 ms , transferred pages: 66924 (n: 66875, d: 49 ) , new dirty pages: 63580 , remaining dirty pages: 71717
> > > > Iteration 16, duration: 2252 ms , transferred pages: 64554 (n: 64539, d: 15 ) , new dirty pages: 63094 , remaining dirty pages: 70257
> > > > Iteration 17, duration: 2188 ms , transferred pages: 62697 (n: 62641, d: 56 ) , new dirty pages: 63016 , remaining dirty pages: 70576
> > > > Iteration 18, duration: 2171 ms , transferred pages: 62377 (n: 62322, d: 55 ) , new dirty pages: 56764 , remaining dirty pages: 64963
> > > > Iteration 19, duration: 2003 ms , transferred pages: 57382 (n: 57324, d: 58 ) , new dirty pages: 65307 , remaining dirty pages: 72888
> > > > Iteration 20, duration: 2240 ms , transferred pages: 64426 (n: 64364, d: 62 ) , new dirty pages: 61585 , remaining dirty pages: 70047
> > > >
> > > >
> > > > --
> > > > Chunguang Li, Ph.D. Candidate
> > > > Wuhan National Laboratory for Optoelectronics (WNLO)
> > > > Huazhong University of Science & Technology (HUST)
> > > > Wuhan, Hubei Prov., China
> > > >
> > > >
> > > >
> > > --
> > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> >
> >
> > --
> > Chunguang Li, Ph.D. Candidate
> > Wuhan National Laboratory for Optoelectronics (WNLO)
> > Huazhong University of Science & Technology (HUST)
> > Wuhan, Hubei Prov., China
> >
> >
> >
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
--
Chunguang Li, Ph.D. Candidate
Wuhan National Laboratory for Optoelectronics (WNLO)
Huazhong University of Science & Technology (HUST)
Wuhan, Hubei Prov., China
prev parent reply other threads:[~2016-11-08 13:32 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-09-25 8:22 [Qemu-devel] Migration dirty bitmap: should only mark pages as dirty after they have been sent Chunguang Li
2016-09-26 11:23 ` Dr. David Alan Gilbert
2016-09-26 14:55 ` Chunguang Li
2016-09-26 18:52 ` Dr. David Alan Gilbert
2016-09-27 12:28 ` Chunguang Li
2016-09-30 5:46 ` Amit Shah
2016-09-30 8:18 ` Chunguang Li
2016-10-08 7:55 ` Chunguang Li
2016-10-14 11:15 ` Dr. David Alan Gilbert
2016-11-03 8:25 ` Chunguang Li
2016-11-03 9:59 ` Li, Liang Z
2016-11-03 10:13 ` Li, Liang Z
2016-11-04 3:07 ` Chunguang Li
2016-11-04 4:50 ` Li, Liang Z
2016-11-04 7:03 ` Chunguang Li
2016-11-07 13:52 ` Chunguang Li
2016-11-07 14:17 ` Li, Liang Z
2016-11-08 5:27 ` Chunguang Li
2016-11-07 14:44 ` Li, Liang Z
2016-11-08 11:05 ` Dr. David Alan Gilbert
2016-11-08 13:40 ` Chunguang Li [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1d21a67.a9f8.158442ac47f.Coremail.lichunguang@hust.edu.cn \
--to=lichunguang@hust.edu.cn \
--cc=amit.shah@redhat.com \
--cc=dgilbert@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.