[Qemu-devel] [Qestion] What status of memory stats feature

[Qemu-devel] [Qestion] What status of memory stats feature

[Ken'ichi Ohmichi]

Hi,

I tried to get the memory stats by using virDomainMemoryStats() of libvirt,
but it could not do it because of the following patch:

[PATCH 03/23] disable guest-provided stats on "info balloon" command
  2010/10/01 from Luiz Capitulino
  http://www.mail-archive.com/qemu-devel@nongnu.org/msg43024.html 

According to the related thread, the above patch avoids hanging user
monitor, and people hope the memory stats feature will be able in the
future. So I'd like to know the status of this feature.

Is there the patch for enabling the feature ?
If the patch exists, I'd like to try it.

What is the essential problem ?
Does some infinite loop happen ?
Unfortunately, I cannot understand the cause of hanging user monitor.


Thanks
Ken'ichi Ohmichi

Re: [Qemu-devel] [Qestion] What status of memory stats feature

[Luiz Capitulino]

On Wed, 15 Dec 2010 16:20:05 +0900
"Ken'ichi Ohmichi" <oomichi@mxs.nes.nec.co.jp> wrote:

> 
> Hi,
> 
> I tried to get the memory stats by using virDomainMemoryStats() of libvirt,
> but it could not do it because of the following patch:
> 
> [PATCH 03/23] disable guest-provided stats on "info balloon" command
>   2010/10/01 from Luiz Capitulino
>   http://www.mail-archive.com/qemu-devel@nongnu.org/msg43024.html 
> 
> According to the related thread, the above patch avoids hanging user
> monitor, and people hope the memory stats feature will be able in the
> future. So I'd like to know the status of this feature.
> 
> Is there the patch for enabling the feature ?
> If the patch exists, I'd like to try it.

It doesn't, afaik.

> What is the essential problem ?

There are two essential problems here.

The first one is that QMP lacks true asynchronous command support (it does
have some code for it, but it's incomplete).

The second problem is that we must not make an existing synchronous command
asynchronous, because this breaks the ABI.

> Does some infinite loop happen ?

No, the guest doesn't respond.

> Unfortunately, I cannot understand the cause of hanging user monitor.

The balloon command needs guest cooperation (ie. it talks to the guest),
if the guest doesn't respond the command will never return.

We don't have a precise ETA for async support, but if it depends only on
the new error framework work, then we could have it for 0.15. If it depends
on the full monitor redesign work, then I guess we'll two releases.

Re: [Qemu-devel] [Qestion] What status of memory stats feature

[Adam Litke]

On Wed, 2010-12-15 at 15:39 -0200, Luiz Capitulino wrote:
> On Wed, 15 Dec 2010 16:20:05 +0900
> "Ken'ichi Ohmichi" <oomichi@mxs.nes.nec.co.jp> wrote:
> 
> > 
> > Hi,
> > 
> > I tried to get the memory stats by using virDomainMemoryStats() of libvirt,
> > but it could not do it because of the following patch:
> > 
> > [PATCH 03/23] disable guest-provided stats on "info balloon" command
> >   2010/10/01 from Luiz Capitulino
> >   http://www.mail-archive.com/qemu-devel@nongnu.org/msg43024.html 
> > 
> > According to the related thread, the above patch avoids hanging user
> > monitor, and people hope the memory stats feature will be able in the
> > future. So I'd like to know the status of this feature.
> > 
> > Is there the patch for enabling the feature ?
> > If the patch exists, I'd like to try it.
> 
> It doesn't, afaik.

It depends on what you are looking to do.  If you just want to play
around and aren't concerned about lockups, You could undo the above
patch to re-enable the interface (although I cannot guarantee that even
this will work).  

> > What is the essential problem ?
> 
> There are two essential problems here.
> 
> The first one is that QMP lacks true asynchronous command support (it does
> have some code for it, but it's incomplete).

I was never completely clear on the problems with the QMP async support.

> The second problem is that we must not make an existing synchronous command
> asynchronous, because this breaks the ABI.

The memory stats interface has always advertised itself as an async
command so this one shouldn't matter.  Whether those semantics actually
ever worked is another issue.

> > Does some infinite loop happen ?
> 
> No, the guest doesn't respond.
> 
> > Unfortunately, I cannot understand the cause of hanging user monitor.
> 
> The balloon command needs guest cooperation (ie. it talks to the guest),
> if the guest doesn't respond the command will never return.

Isn't that only a problem for the user monitor?  For QMP you might just
never get a response to the stats request.  IIRC, the QMP monitor is
never 'suspended' in the same way that the user monitor is so I wouldn't
expect it to be susceptible to the same lockup issues.

> We don't have a precise ETA for async support, but if it depends only on
> the new error framework work, then we could have it for 0.15. If it depends
> on the full monitor redesign work, then I guess we'll two releases.

I can't provide a definitive answer to this unfortunately since I
haven't looked in awhile (nor am I privy to the details of the full
monitor redesign proposal.

-- 
Thanks,
Adam

Re: [Qemu-devel] [Qestion] What status of memory stats feature

[Luiz Capitulino]

On Wed, 15 Dec 2010 16:58:03 -0600
Adam Litke <agl@us.ibm.com> wrote:

> On Wed, 2010-12-15 at 15:39 -0200, Luiz Capitulino wrote:
> > On Wed, 15 Dec 2010 16:20:05 +0900
> > "Ken'ichi Ohmichi" <oomichi@mxs.nes.nec.co.jp> wrote:
> > 
> > > 
> > > Hi,
> > > 
> > > I tried to get the memory stats by using virDomainMemoryStats() of libvirt,
> > > but it could not do it because of the following patch:
> > > 
> > > [PATCH 03/23] disable guest-provided stats on "info balloon" command
> > >   2010/10/01 from Luiz Capitulino
> > >   http://www.mail-archive.com/qemu-devel@nongnu.org/msg43024.html 
> > > 
> > > According to the related thread, the above patch avoids hanging user
> > > monitor, and people hope the memory stats feature will be able in the
> > > future. So I'd like to know the status of this feature.
> > > 
> > > Is there the patch for enabling the feature ?
> > > If the patch exists, I'd like to try it.
> > 
> > It doesn't, afaik.
> 
> It depends on what you are looking to do.  If you just want to play
> around and aren't concerned about lockups, You could undo the above
> patch to re-enable the interface (although I cannot guarantee that even
> this will work).  
> 
> > > What is the essential problem ?
> > 
> > There are two essential problems here.
> > 
> > The first one is that QMP lacks true asynchronous command support (it does
> > have some code for it, but it's incomplete).
> 
> I was never completely clear on the problems with the QMP async support.
> 
> > The second problem is that we must not make an existing synchronous command
> > asynchronous, because this breaks the ABI.
> 
> The memory stats interface has always advertised itself as an async
> command so this one shouldn't matter.  Whether those semantics actually
> ever worked is another issue.

The query-balloon command used to be synchronous (or quite fast, or would
never lockup). We changed that.

> > > Does some infinite loop happen ?
> > 
> > No, the guest doesn't respond.
> > 
> > > Unfortunately, I cannot understand the cause of hanging user monitor.
> > 
> > The balloon command needs guest cooperation (ie. it talks to the guest),
> > if the guest doesn't respond the command will never return.
> 
> Isn't that only a problem for the user monitor?  For QMP you might just
> never get a response to the stats request.  IIRC, the QMP monitor is
> never 'suspended' in the same way that the user monitor is so I wouldn't
> expect it to be susceptible to the same lockup issues.

We need to have a way to cancel the request. This is not the cause of
this problem, though.

> > We don't have a precise ETA for async support, but if it depends only on
> > the new error framework work, then we could have it for 0.15. If it depends
> > on the full monitor redesign work, then I guess we'll two releases.
> 
> I can't provide a definitive answer to this unfortunately since I
> haven't looked in awhile (nor am I privy to the details of the full
> monitor redesign proposal.

Basically, we're splitting HMP and QMP. The end result will be that HMP
will use QMP as a C library. Not hard to do, but there's lots of code
churn involved.

Now, regarding async support, I already have a proposal, but I'll send it
only next year because I'll be in vacations for two weeks.