From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dor Laor Subject: Re: [KVM-AUTOTEST PATCH 1/2] Add KSM test Date: Wed, 30 Sep 2009 14:23:13 +0200 Message-ID: <4AC34DB1.2070205@redhat.com> References: <44701752.775351253870568755.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com> <1254239454.8179.11.camel@localhost.localdomain> Reply-To: dlaor@redhat.com Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Jiri Zupka , kvm@vger.kernel.org, autotest@test.kernel.org, Lukas Doktor To: Lucas Meneghel Rodrigues Return-path: Received: from mx1.redhat.com ([209.132.183.28]:4623 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752995AbZI3MXF (ORCPT ); Wed, 30 Sep 2009 08:23:05 -0400 In-Reply-To: <1254239454.8179.11.camel@localhost.localdomain> Sender: kvm-owner@vger.kernel.org List-ID: On 09/29/2009 05:50 PM, Lucas Meneghel Rodrigues wrote: > On Fri, 2009-09-25 at 05:22 -0400, Jiri Zupka wrote: >> ----- "Dor Laor" wrote: >> >>> On 09/16/2009 04:09 PM, Jiri Zupka wrote: >>>> >>>> ----- "Dor Laor" wrote: >>>> >>>>> On 09/15/2009 09:58 PM, Jiri Zupka wrote: >>>>>>> After a quick review I have the following questions: >>>>>>> 1. Why did you implement the guest tool in 'c' and not in >>> python? >>>>>>> Python is much simpler and you can share some code with the >>>>> server. >>>>>>> This 'test protocol' would also be easier to understand this >>>>> way. >>>>>> >>>>>> We need speed and the precise control of allocate memory in >>> pages. >>>>>> >>>>>>> 2. IMHO there is no need to use select, you can do blocking >>> read. >>>>>> >>>>>> We replace socket communication by interactive program >>> communication >>>>> via ssh/telnet >>>>>> >>>>>>> 3. Also you can use plain malloc without the more complex ( a >>> bit) >>>>> mmap. >>>>>> >>>>>> We need address exactly the memory pages. We can't allow shift of >>>>> the data in memory. >>>>> >>>>> You can use the tmpfs+dd idea instead of the specific program as I >>>>> detailed before. Maybe some other binary can be used. My intention >>> is >>>>> to >>>>> simplify the test/environment as much as possible. >>>>> >>>> >>>> We need compatibility with others system, like Windows etc.. >>>> We want to add support for others system in next version >>> >>> KSM is a host feature and should be agnostic to the guest. >>> Also I don't think your code will compile on windows... >> >> Yes, I think you have true. > > First of all, sorry, I am doing the best I can to review carefully all > the patch queue, and as KSM is a more involved feature that I am not > very familiar with, I need a bit more time to review it! > >> But because we need generate special data to pages in memory. >> We need use script on guest side of test. Because communication >> over ssh is to slow to transfer lot of GB of special data to guests. >> >> We can use optimized C program which is 10x and more faster than >> python script on native system. Heavy load of virtual guest can >> make some performance problem. > > About code compiling under windows, I guess making a native windows c or > c++ program is an option, I generally agree with your reasoning, this > case seems to be better covered with a c program. Will get into it in > more detail ASAP... > >> We can use tmpfs but with python script to generate special data. >> We can't use dd with random because we need test some special case. >> (change only last 96B of page etc.. ) >> >> >> What do you think about it? I think it can be done with some simple scripting and it will be fast enough and more importantly, easier to understand and to change in the future. Here is a short example for creating lots of identical pages that contain '0' apart for the last two bytes. If you'll run it in a single guest you should expect to save lots of memory. Then you can change the last bytes to random value and see the memory consumption grow: [Remember to cancel the guest swap to keep it in the guest ram] dd if=/dev/zero of=template count=1 bs=4094 echo '1' >> template cp template large_file for ((i=0;i<10;i++)) do dd if=large_file of=large_file conv=notrunc oflag=append > /dev/null 2>&1 ; done It creates a 4k*2^10 file with identical pages (since it's on tmpfs with no swap) Can you try it? It should be far simpler than the original option. Thanks, Dor