Re: CPU Limits on KVM?

Re: CPU Limits on KVM?

[Francisco Mazzeo]

Hello,

 I am a new user to KVM and was wondering if there was any way to
limit a VE from using up all the resources of the processor.


 Right now I have a Quad core 2.5Ghz, I have a KVM VE (running windows
server 2003) and assigned 4 CPUs to it. If I max out the load for that
VE, the entire host node load will be 100% which may crash it if I
hosted more than 1 single VE.

 OpenVZ has cpulimit command, does KVM have something similar or any
way that I can implement a limit on a single VE? Say I want to only
give a max of 500Mhz per core, to total 2Ghz to the VE.

Thanks
Francisco
www.navigatoris.net / www.serversoutlet.com

Re: CPU Limits on KVM?

[Brian Jackson]

There's CPU cgroups. It doesn't have exactly the ability you are after, but it 
is able to limit process(es) CPU usage. Maxing out CPU usage won't crash your 
server. The kernel will arbitrate sharing the CPU evenly among processes/VMs.

--Brian Jackson


On Thursday 02 April 2009 16:41:10 Francisco Mazzeo wrote:
> Hello,
>
>  I am a new user to KVM and was wondering if there was any way to
> limit a VE from using up all the resources of the processor.
>
>
>  Right now I have a Quad core 2.5Ghz, I have a KVM VE (running windows
> server 2003) and assigned 4 CPUs to it. If I max out the load for that
> VE, the entire host node load will be 100% which may crash it if I
> hosted more than 1 single VE.
>
>  OpenVZ has cpulimit command, does KVM have something similar or any
> way that I can implement a limit on a single VE? Say I want to only
> give a max of 500Mhz per core, to total 2Ghz to the VE.
>
> Thanks
> Francisco
> www.navigatoris.net / www.serversoutlet.com
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CPU Limits on KVM?

[Brian Jackson]

I haven't ever really used cgroups. I always figured a fair host scheduler is 
good enough to handle spreading load. So I don't know if it will fit exactly 
what you need. I don't think so. I also don't know of any other options. I 
will say, If I gave 4 VMs a single cpu each on a 4 core host, I would expect 
the host to be fully loaded. I wouldn't see any reason for the host not to be 
fully loaded. That is after all one of the key points of virtualization. 
Better utilization of hardware.



On Thursday 02 April 2009 17:33:07 Francisco Mazzeo wrote:
> Hello Brian,
>
>  Thanks for the reply. is there a wiki about cgroupds and how to set them
> up?
>
>  Also, I tried just for kicks to see what would happen if I create 4
> Virtual Windows machines, run prime95 (a tool that does iterations
> like superpi to stress test memory/cpu) on all of them and just assign
> them only ONE core to them.
>
>  The server node did not crash and you are right, however I was hoping
> for the server load to stay below 50% as I only gave it one single
> core to each KVM VE. Instead it seems like KVM let each VE get one
> slice of each of the 4 cores of my CPU, which did not accomplish what
> I wanted.
>
>  Is cgroupds the only choice available?
>
> -- Francisco
>
> On Thu, Apr 2, 2009 at 3:29 PM, Brian Jackson <iggy@theiggy.com> wrote:
> > There's CPU cgroups. It doesn't have exactly the ability you are after,
> > but it is able to limit process(es) CPU usage. Maxing out CPU usage won't
> > crash your server. The kernel will arbitrate sharing the CPU evenly among
> > processes/VMs.
> >
> > --Brian Jackson
> >
> > On Thursday 02 April 2009 16:41:10 Francisco Mazzeo wrote:
> >> Hello,
> >>
> >>  I am a new user to KVM and was wondering if there was any way to
> >> limit a VE from using up all the resources of the processor.
> >>
> >>
> >>  Right now I have a Quad core 2.5Ghz, I have a KVM VE (running windows
> >> server 2003) and assigned 4 CPUs to it. If I max out the load for that
> >> VE, the entire host node load will be 100% which may crash it if I
> >> hosted more than 1 single VE.
> >>
> >>  OpenVZ has cpulimit command, does KVM have something similar or any
> >> way that I can implement a limit on a single VE? Say I want to only
> >> give a max of 500Mhz per core, to total 2Ghz to the VE.
> >>
> >> Thanks
> >> Francisco
> >> www.navigatoris.net / www.serversoutlet.com
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe kvm" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CPU Limits on KVM?

[Oshrit Feder]

Hi Francisco, 
I've been trying to limit the cpu usage of a VM using cgroups - so I can 
share my experience with you. However, from monitoring the kvm process 
looks like it doesnt force a hard limit of cpu usage%. Hopefully, someone 
can light up the issue. 
I'm interested in limiting the io bandwidth of network and disk as well. 
(dm-ioband - any experience with that? other suggestions?)

1. mount -t cgroup none /dev/cgroup -o cpu,memory
This creates the cgroup file system api on /dev/cgroup. The top level 
hierarchy node contains all the processes by default. 
2. cd /dev/cgroup
3. mkdir VM1
4. mkdir VM2
5. echo {kvm1 pid} > vm1/tasks
6. echo {kvm2 pid} > vm2/tasks
Moves the vm's processes from the top node to a child node - to enable 
atomic control
7. edit VM1/cpu.shares and VM2/cpu.shares to change cpu proportion 
(default is 1024 shares)
I've used 128 for VM1 and 1024 for VM2. To my understanding, that should 
yield proportion of 128/(1024+128) < 15% of cpu limit for VM1.
Each of the VMs was allocated 2 vcpu, the host has 4 cores.(Been tring the 
same with 1 vcpu each, results werent different)

Monitoring results:
I used top, atop and system monitoring GUI on the host and on the guest 
for monitoring. Was able to load VM1 to use more than 50%...
 

(Monitoring screenshot at http%3A%2F%2Ftinyurl.com%2Fcsx3v7 ) On the left 
- guest monitoring, time graph shows that there are peaks of more than 
50%), on the right - host monitoring, kvm pid 32620 (VM1 pid) with 63% 
load)


Regarding the references you asked for. Those are the references I've 
found and worked with:
www.mjmwired.net%2Fkernel%2FDocumentation%2Fcgroups.txt%A0 
http%3A%2F%2Ftinyurl.com%2Fdcnoav  

- Oshrit
Oshritf@il.ibm.com



From:
Brian Jackson <iggy@theiggy.com>
To:
Francisco Mazzeo <francisco.mazzeo@gmail.com>, kvm@vger.kernel.org
Date:
03/04/2009 02:32
Subject:
Re: CPU Limits on KVM?



I haven't ever really used cgroups. I always figured a fair host scheduler 
is 
good enough to handle spreading load. So I don't know if it will fit 
exactly 
what you need. I don't think so. I also don't know of any other options. I 

will say, If I gave 4 VMs a single cpu each on a 4 core host, I would 
expect 
the host to be fully loaded. I wouldn't see any reason for the host not to 
be 
fully loaded. That is after all one of the key points of virtualization. 
Better utilization of hardware.



On Thursday 02 April 2009 17:33:07 Francisco Mazzeo wrote:
> Hello Brian,
>
>  Thanks for the reply. is there a wiki about cgroupds and how to set 
them
> up?
>
>  Also, I tried just for kicks to see what would happen if I create 4
> Virtual Windows machines, run prime95 (a tool that does iterations
> like superpi to stress test memory/cpu) on all of them and just assign
> them only ONE core to them.
>
>  The server node did not crash and you are right, however I was hoping
> for the server load to stay below 50% as I only gave it one single
> core to each KVM VE. Instead it seems like KVM let each VE get one
> slice of each of the 4 cores of my CPU, which did not accomplish what
> I wanted.
>
>  Is cgroupds the only choice available?
>
> -- Francisco
>
> On Thu, Apr 2, 2009 at 3:29 PM, Brian Jackson <iggy@theiggy.com> wrote:
> > There's CPU cgroups. It doesn't have exactly the ability you are 
after,
> > but it is able to limit process(es) CPU usage. Maxing out CPU usage 
won't
> > crash your server. The kernel will arbitrate sharing the CPU evenly 
among
> > processes/VMs.
> >
> > --Brian Jackson
> >
> > On Thursday 02 April 2009 16:41:10 Francisco Mazzeo wrote:
> >> Hello,
> >>
> >>  I am a new user to KVM and was wondering if there was any way to
> >> limit a VE from using up all the resources of the processor.
> >>
> >>
> >>  Right now I have a Quad core 2.5Ghz, I have a KVM VE (running 
windows
> >> server 2003) and assigned 4 CPUs to it. If I max out the load for 
that
> >> VE, the entire host node load will be 100% which may crash it if I
> >> hosted more than 1 single VE.
> >>
> >>  OpenVZ has cpulimit command, does KVM have something similar or any
> >> way that I can implement a limit on a single VE? Say I want to only
> >> give a max of 500Mhz per core, to total 2Ghz to the VE.
> >>
> >> Thanks
> >> Francisco
> >> www.navigatoris.net / www.serversoutlet.com
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe kvm" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CPU Limits on KVM?

[FinnTux]

> I'm interested in limiting the io bandwidth of network and disk as well.
> (dm-ioband - any experience with that? other suggestions?)
I tested/used normal linux traffic shaping for this (htb.init to be
exact). It works at least with bridges (-net tap...).

> I used top, atop and system monitoring GUI on the host and on the guest
> for monitoring. Was able to load VM1 to use more than 50%...

I think usage will go above the limit as long as there is free  CPU
cycles. How about trying this:
- two VMs, once core each
- bind your VM processes to certain core on host using taskset
- use cgroups you described
- run something that uses lots of CPU (cpuburn) on both VMs

Now you should see that lower priority VM gets only its share. And if
you stop CPU hog process in higher priority VM, lower priority will
get more (ie. no hard limit).

Re: CPU Limits on KVM?

[sudhir kumar]

cpu controller/group scheduler does not yet provide hard limit. In
case there are free resources available on your machine those will be
used by the processes running in the control groups in the ratio of
shares allocated to them. group scheduler provides proper fairness
only under contention.

You can try using cpusets to bind your VMs to a particular domain.
Though I have no hands on experience on doing that.

On Mon, Apr 6, 2009 at 4:24 PM, Oshrit Feder <OSHRITF@il.ibm.com> wrote:
> Hi Francisco,
> I've been trying to limit the cpu usage of a VM using cgroups - so I can
> share my experience with you. However, from monitoring the kvm process
> looks like it doesnt force a hard limit of cpu usage%. Hopefully, someone
> can light up the issue.
> I'm interested in limiting the io bandwidth of network and disk as well.
> (dm-ioband - any experience with that? other suggestions?)
>
> 1. mount -t cgroup none /dev/cgroup -o cpu,memory
> This creates the cgroup file system api on /dev/cgroup. The top level
> hierarchy node contains all the processes by default.
> 2. cd /dev/cgroup
> 3. mkdir VM1
> 4. mkdir VM2
> 5. echo {kvm1 pid} > vm1/tasks
> 6. echo {kvm2 pid} > vm2/tasks
> Moves the vm's processes from the top node to a child node - to enable
> atomic control
> 7. edit VM1/cpu.shares and VM2/cpu.shares to change cpu proportion
> (default is 1024 shares)
> I've used 128 for VM1 and 1024 for VM2. To my understanding, that should
> yield proportion of 128/(1024+128) < 15% of cpu limit for VM1.
> Each of the VMs was allocated 2 vcpu, the host has 4 cores.(Been tring the
> same with 1 vcpu each, results werent different)
>
> Monitoring results:
> I used top, atop and system monitoring GUI on the host and on the guest
> for monitoring. Was able to load VM1 to use more than 50%...
>
>
> (Monitoring screenshot at http%3A%2F%2Ftinyurl.com%2Fcsx3v7 ) On the left
> - guest monitoring, time graph shows that there are peaks of more than
> 50%), on the right - host monitoring, kvm pid 32620 (VM1 pid) with 63%
> load)
>
>
> Regarding the references you asked for. Those are the references I've
> found and worked with:
> www.mjmwired.net%2Fkernel%2FDocumentation%2Fcgroups.txt%A0
> http%3A%2F%2Ftinyurl.com%2Fdcnoav
>
> - Oshrit
> Oshritf@il.ibm.com
>
>
>
> From:
> Brian Jackson <iggy@theiggy.com>
> To:
> Francisco Mazzeo <francisco.mazzeo@gmail.com>, kvm@vger.kernel.org
> Date:
> 03/04/2009 02:32
> Subject:
> Re: CPU Limits on KVM?
>
>
>
> I haven't ever really used cgroups. I always figured a fair host scheduler
> is
> good enough to handle spreading load. So I don't know if it will fit
> exactly
> what you need. I don't think so. I also don't know of any other options. I
>
> will say, If I gave 4 VMs a single cpu each on a 4 core host, I would
> expect
> the host to be fully loaded. I wouldn't see any reason for the host not to
> be
> fully loaded. That is after all one of the key points of virtualization.
> Better utilization of hardware.
>
>
>
> On Thursday 02 April 2009 17:33:07 Francisco Mazzeo wrote:
>> Hello Brian,
>>
>>  Thanks for the reply. is there a wiki about cgroupds and how to set
> them
>> up?
>>
>>  Also, I tried just for kicks to see what would happen if I create 4
>> Virtual Windows machines, run prime95 (a tool that does iterations
>> like superpi to stress test memory/cpu) on all of them and just assign
>> them only ONE core to them.
>>
>>  The server node did not crash and you are right, however I was hoping
>> for the server load to stay below 50% as I only gave it one single
>> core to each KVM VE. Instead it seems like KVM let each VE get one
>> slice of each of the 4 cores of my CPU, which did not accomplish what
>> I wanted.
>>
>>  Is cgroupds the only choice available?
>>
>> -- Francisco
>>
>> On Thu, Apr 2, 2009 at 3:29 PM, Brian Jackson <iggy@theiggy.com> wrote:
>> > There's CPU cgroups. It doesn't have exactly the ability you are
> after,
>> > but it is able to limit process(es) CPU usage. Maxing out CPU usage
> won't
>> > crash your server. The kernel will arbitrate sharing the CPU evenly
> among
>> > processes/VMs.
>> >
>> > --Brian Jackson
>> >
>> > On Thursday 02 April 2009 16:41:10 Francisco Mazzeo wrote:
>> >> Hello,
>> >>
>> >>  I am a new user to KVM and was wondering if there was any way to
>> >> limit a VE from using up all the resources of the processor.
>> >>
>> >>
>> >>  Right now I have a Quad core 2.5Ghz, I have a KVM VE (running
> windows
>> >> server 2003) and assigned 4 CPUs to it. If I max out the load for
> that
>> >> VE, the entire host node load will be 100% which may crash it if I
>> >> hosted more than 1 single VE.
>> >>
>> >>  OpenVZ has cpulimit command, does KVM have something similar or any
>> >> way that I can implement a limit on a single VE? Say I want to only
>> >> give a max of 500Mhz per core, to total 2Ghz to the VE.
>> >>
>> >> Thanks
>> >> Francisco
>> >> www.navigatoris.net / www.serversoutlet.com
>> >> --
>> >> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> >> the body of a message to majordomo@vger.kernel.org
>> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Sudhir Kumar