Re: Question about the ability of credit scheduler to handle I/O and CPU intensive VMs

[Yuehai Xu <yuehaixu@gmail.com>]

On Tue, Oct 5, 2010 at 10:16 AM, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
> On Tue, Oct 5, 2010 at 3:52 AM, Yuehai Xu <yuehaixu@gmail.com> wrote:
>> However, I stop one of the CPU intensive program in a DomU while keep
>> the other running, the results are:
>> .....
>> <  0.327815345 |x d1v0> (dom: 1) --> (dom: 2) vruntime : 1542607)
>> <  0.327906620 -x d2v0> (dom: 2) --> (dom: 1) vruntime : 109521)
>> <  0.344349033 |x d1v0> (dom: 1) --> (dom: 2) vruntime : 19779544)
>> <  0.344377129 -x d2v0> (dom: 2) --> (dom: 1) vruntime : 33528)
>> <  0.344570662 |x d1v0> (dom: 1) --> (dom: 2) vruntime : 232540)
>> <  0.344643933 -x d2v0> (dom: 2) --> (dom: 1) vruntime : 87857)
>> <  0.345009170 |x d1v0> (dom: 1) --> (dom: 2) vruntime : 439081)
>> <  0.345034387 -x d2v0> (dom: 2) --> (dom: 1) vruntime : 30059)
>> <  0.369973183 -x d1v0> (dom: 1) --> (dom: 1) vruntime : 30000506)
>> <  0.392423279 |x d1v0> (dom: 1) --> (dom: 2) vruntime : 27006658)
>> ....
>>
>> Here I am gotten confusing, since my algorithm of scheduling is very
>> simple, every VM should have 30ms of PCPU, however, from the results,
>> the time for
>> each VCPU to have PCPU is quite unstable. I think somewhere, the
>> routine of schedule() should be invoked frequently, and from xentop,
>> the VM with CPU
>> intensive occupies PCPU almost at 97%.
>
> Idle VMs are never 100% idle; there are a lot of "maintenance" tasks
> still to be done.
>
> Your mostly-idle domain (domain 2) seems to be running for pretty
> short periods of time -- less than 100us.  That's pretty reasonable.
>
> The question to ask is, what happens when domain 2 wakes up -- is it
> put on the runqueue, waiting for the running domain to finish its
> timeslice?  Or is it run immediately?  Given the trace you give here,
> I would guess that it's run immediately.  If that's not what you
> expect, you need to figure out why it's doing that and make it do
> something else. :-)

Yes, you are right, when domain 2 is waken up, it will be inserted to
the head of the scheduling list, and then, run immediately. This is
what I expect. I am sorry that I don't express the question clearly.

You have said that the mostly-idle domain should be running for a
pretty short periods of time, however, according to my understanding,
it is the scheduler to decide how long a domain should run. Since this
time is a fixed value, that is 30ms in my scheduler. What I am puzzled
is that why the running time for this mostly-idle domain is so short.
Is it because this domain idle? Or this domain is actually put into
other lists instead of runnable? is there other places that affect how
to schedule the VMs except such as csched_credit.c?

I know in the kernel of Linux, when a process is stocked because of
I/O, it will be deleted from runnable queue, so that the scheduler of
CPU can select next runnable process immediately. However, I thought
this was different from the scheduling of XEN. Since the scheduler
didn't really know whether the VCPU was consuming PCPU, it just
provided a certain period of time to the VM. I might be wrong. If it
is true, even a most idle VM should always consumes as the same PCPU
time as the busy one  in my scheduler. But the result is opposite. The
idle VM consumes much less PCPU then the busy one. This should not be
determined by the scheduling itself, otherwise, the idle one should
also have 50% PCPU. Then, what mechanism cause this result?

I really appreciate your help.

Thanks,
Yuehai