[Qemu-devel] hmp interface for kdump compressed format

[Qemu-devel] hmp interface for kdump compressed format

[Christian Borntraeger]

Qiao Nuohan,

is there a reason why you did not implemented the HMP part for that format
of kdump compressed format? After all this is a patch mostly for developers,
so a HMP interface might come handy. Do you already have some patch in 
preparation or know somebody doing it?

Thanks a lot

Christian

Re: [Qemu-devel] hmp interface for kdump compressed format

[Laszlo Ersek]

On 03/20/14 21:38, Christian Borntraeger wrote:
> Qiao Nuohan,
> 
> is there a reason why you did not implemented the HMP part for that format
> of kdump compressed format? After all this is a patch mostly for developers,
> so a HMP interface might come handy. Do you already have some patch in 
> preparation or know somebody doing it?

http://thread.gmane.org/gmane.comp.emulators.qemu/249283/focus=250059

Laszlo

Re: [Qemu-devel] hmp interface for kdump compressed format

[Laszlo Ersek]

Two additional points:

On 03/20/14 21:56, Laszlo Ersek wrote:
> On 03/20/14 21:38, Christian Borntraeger wrote:
>> Qiao Nuohan,
>>
>> is there a reason why you did not implemented the HMP part for that format
>> of kdump compressed format? After all this is a patch mostly for developers,
>> so a HMP interface might come handy. Do you already have some patch in 
>> preparation or know somebody doing it?
> 
> http://thread.gmane.org/gmane.comp.emulators.qemu/249283/focus=250059

- HMP interface could be implemented as a fully new command of course,

- the feature being targeted at developers strikes me as a completely
unexpected idea. The main goal in my understanding is to allow the
management layer (libvirt) to save a compressed vmcore when the guest
panicks, crashes, or the admin feels like it. For communication with
another program, QMP is actually preferable.

So I'm certainly not against anyone adding a HMP interface too, but it
cannot be an extension to the current HMP command (because it would
break existing HMP command lines unless the HMP parser were reworked
too), plus during review I saw no reason to stall the series even
longer, for a side feature that I perceived to be of low importance (and
I actually thought that Qiao Nuohan shared that notion).

Thanks
Laszlo

Re: [Qemu-devel] hmp interface for kdump compressed format

[Christian Borntraeger]

On 20/03/14 21:56, Laszlo Ersek wrote:
> On 03/20/14 21:38, Christian Borntraeger wrote:
>> Qiao Nuohan,
>>
>> is there a reason why you did not implemented the HMP part for that format
>> of kdump compressed format? After all this is a patch mostly for developers,
>> so a HMP interface might come handy. Do you already have some patch in 
>> preparation or know somebody doing it?
> 
> http://thread.gmane.org/gmane.comp.emulators.qemu/249283/focus=250059
> 
> Laszlo
> 
So in essence: Due to the limitations of the command line parser we cannot
add the format to the hmp command line.
So something like adding

dump_guest_memory_set_format <format>

would be the only possible solution with the hmp code as is. Correct?
(If adding such a command is nice or not is another question)

Christian

Re: [Qemu-devel] hmp interface for kdump compressed format

[Laszlo Ersek]

On 03/20/14 22:18, Christian Borntraeger wrote:
> On 20/03/14 21:56, Laszlo Ersek wrote:
>> On 03/20/14 21:38, Christian Borntraeger wrote:
>>> Qiao Nuohan,
>>>
>>> is there a reason why you did not implemented the HMP part for that format
>>> of kdump compressed format? After all this is a patch mostly for developers,
>>> so a HMP interface might come handy. Do you already have some patch in 
>>> preparation or know somebody doing it?
>>
>> http://thread.gmane.org/gmane.comp.emulators.qemu/249283/focus=250059
>>
>> Laszlo
>>
> So in essence: Due to the limitations of the command line parser we cannot
> add the format to the hmp command line.
> So something like adding
> 
> dump_guest_memory_set_format <format>
> 
> would be the only possible solution with the hmp code as is. Correct?

Yes, one possibility would be to make the dump command stateful (=
compression format), and to add a new command setting/getting that state.

Another option would be to leave the current command intact, and add an
independent command that wouldn't take "begin", "end", nor "paging", but
would take compression format.

> (If adding such a command is nice or not is another question)

Yep.

Thanks
Laszlo

Re: [Qemu-devel] hmp interface for kdump compressed format

[Paolo Bonzini]

Il 20/03/2014 22:28, Laszlo Ersek ha scritto:
> On 03/20/14 22:18, Christian Borntraeger wrote:
>> On 20/03/14 21:56, Laszlo Ersek wrote:
>>> On 03/20/14 21:38, Christian Borntraeger wrote:
>>>> Qiao Nuohan,
>>>>
>>>> is there a reason why you did not implemented the HMP part for that format
>>>> of kdump compressed format? After all this is a patch mostly for developers,
>>>> so a HMP interface might come handy. Do you already have some patch in
>>>> preparation or know somebody doing it?
>>>
>>> http://thread.gmane.org/gmane.comp.emulators.qemu/249283/focus=250059
>>>
>>> Laszlo
>>>
>> So in essence: Due to the limitations of the command line parser we cannot
>> add the format to the hmp command line.
>> So something like adding
>>
>> dump_guest_memory_set_format <format>
>>
>> would be the only possible solution with the hmp code as is. Correct?
>
> Yes, one possibility would be to make the dump command stateful (=
> compression format), and to add a new command setting/getting that state.
>
> Another option would be to leave the current command intact, and add an
> independent command that wouldn't take "begin", "end", nor "paging", but
> would take compression format.

Another possibility is to kill begin/length from the dump-guest-memory 
command.  HMP is not stable, so if that's useful we can do it.

Paolo

Re: [Qemu-devel] hmp interface for kdump compressed format

[qiaonuohan]

On 03/21/2014 05:12 AM, Laszlo Ersek wrote:
 > Two additional points:
 >
 > On 03/20/14 21:56, Laszlo Ersek wrote:
 >> On 03/20/14 21:38, Christian Borntraeger wrote:
 >>> Qiao Nuohan,
 >>>
 >>> is there a reason why you did not implemented the HMP part for that format
 >>> of kdump compressed format? After all this is a patch mostly for developers,
 >>> so a HMP interface might come handy. Do you already have some patch in
 >>> preparation or know somebody doing it?
 >>
 >> http://thread.gmane.org/gmane.comp.emulators.qemu/249283/focus=250059
 >
 > - HMP interface could be implemented as a fully new command of course,
 >
 > - the feature being targeted at developers strikes me as a completely
 > unexpected idea. The main goal in my understanding is to allow the
 > management layer (libvirt) to save a compressed vmcore when the guest
 > panicks, crashes, or the admin feels like it. For communication with
 > another program, QMP is actually preferable.
 >
 > So I'm certainly not against anyone adding a HMP interface too, but it
 > cannot be an extension to the current HMP command (because it would
 > break existing HMP command lines unless the HMP parser were reworked
 > too), plus during review I saw no reason to stall the series even
 > longer, for a side feature that I perceived to be of low importance (and
 > I actually thought that Qiao Nuohan shared that notion).

Yes, my main purpose is let "virsh dump --memory-only" can dump a small
core that can be used to analyze kernel. So HMP is not that important to me.


On 03/21/2014 05:51 AM, Paolo Bonzini wrote:
> Il 20/03/2014 22:28, Laszlo Ersek ha scritto:
>> On 03/20/14 22:18, Christian Borntraeger wrote:
>>> On 20/03/14 21:56, Laszlo Ersek wrote:
>>>> On 03/20/14 21:38, Christian Borntraeger wrote:
>>>>> Qiao Nuohan,
>>>>>
>>>>> is there a reason why you did not implemented the HMP part for that format
>>>>> of kdump compressed format? After all this is a patch mostly for developers,
>>>>> so a HMP interface might come handy. Do you already have some patch in
>>>>> preparation or know somebody doing it?
>>>>
>>>> http://thread.gmane.org/gmane.comp.emulators.qemu/249283/focus=250059
>>>>
>>>> Laszlo
>>>>
>>> So in essence: Due to the limitations of the command line parser we cannot
>>> add the format to the hmp command line.
>>> So something like adding
>>>
>>> dump_guest_memory_set_format <format>
>>>
>>> would be the only possible solution with the hmp code as is. Correct?
>>
>> Yes, one possibility would be to make the dump command stateful (=
>> compression format), and to add a new command setting/getting that state.
>>
>> Another option would be to leave the current command intact, and add an
>> independent command that wouldn't take "begin", "end", nor "paging", but
>> would take compression format.
>
> Another possibility is to kill begin/length from the dump-guest-memory command.
> HMP is not stable, so if that's useful we can do it.

AFAIK, kdump-compressed format can only be analyzed by crash-utility right
now. ELF is big but we still need it, then begin/length will help us save
time and space when only part of the memory is need.

IMHO, a new HMP command will be a good choice and it is more simple than
making dump format 'stateful'. I am not so clear about HMP, but I think
the new HMP command can be 'dump-guest-memory-with-format', and still
base on qmp_dump_guest_memory. If you guys like it, I will send a patch
that add this command.

>
> Paolo
>
>
>


-- 
Regards
Qiao Nuohan

Re: [Qemu-devel] hmp interface for kdump compressed format

[Christian Borntraeger]

On 21/03/14 10:31, qiaonuohan@cn.fujitsu.com wrote:
[....]

>>> Another option would be to leave the current command intact, and add an
>>> independent command that wouldn't take "begin", "end", nor "paging", but
>>> would take compression format.
>>
>> Another possibility is to kill begin/length from the dump-guest-memory command.
>> HMP is not stable, so if that's useful we can do it.
> 
> AFAIK, kdump-compressed format can only be analyzed by crash-utility right
> now. ELF is big but we still need it, then begin/length will help us save
> time and space when only part of the memory is need.
> 
> IMHO, a new HMP command will be a good choice and it is more simple than
> making dump format 'stateful'. I am not so clear about HMP, but I think
> the new HMP command can be 'dump-guest-memory-with-format', and still
> base on qmp_dump_guest_memory. If you guys like it, I will send a patch
> that add this command.

Yes, please do, if you find the time.
Maybe user "dump-guest-memory-format" or whatever comes up might be a better name, though.
Thanks a lot

Christian

Re: [Qemu-devel] hmp interface for kdump compressed format

[Markus Armbruster]

"qiaonuohan@cn.fujitsu.com" <qiaonuohan@cn.fujitsu.com> writes:

> On 03/21/2014 05:12 AM, Laszlo Ersek wrote:
>  > Two additional points:
>  >
>  > On 03/20/14 21:56, Laszlo Ersek wrote:
>  >> On 03/20/14 21:38, Christian Borntraeger wrote:
>  >>> Qiao Nuohan,
>  >>>
>  >>> is there a reason why you did not implemented the HMP part for that format
>  >>> of kdump compressed format? After all this is a patch mostly for
>  >>> developers,
>  >>> so a HMP interface might come handy. Do you already have some patch in
>  >>> preparation or know somebody doing it?
>  >>
>  >> http://thread.gmane.org/gmane.comp.emulators.qemu/249283/focus=250059
>  >
>  > - HMP interface could be implemented as a fully new command of course,
>  >
>  > - the feature being targeted at developers strikes me as a completely
>  > unexpected idea. The main goal in my understanding is to allow the
>  > management layer (libvirt) to save a compressed vmcore when the guest
>  > panicks, crashes, or the admin feels like it. For communication with
>  > another program, QMP is actually preferable.
>  >
>  > So I'm certainly not against anyone adding a HMP interface too, but it
>  > cannot be an extension to the current HMP command (because it would
>  > break existing HMP command lines unless the HMP parser were reworked
>  > too), plus during review I saw no reason to stall the series even
>  > longer, for a side feature that I perceived to be of low importance (and
>  > I actually thought that Qiao Nuohan shared that notion).
>
> Yes, my main purpose is let "virsh dump --memory-only" can dump a small
> core that can be used to analyze kernel. So HMP is not that important to me.
>
>
> On 03/21/2014 05:51 AM, Paolo Bonzini wrote:
>> Il 20/03/2014 22:28, Laszlo Ersek ha scritto:
>>> On 03/20/14 22:18, Christian Borntraeger wrote:
>>>> On 20/03/14 21:56, Laszlo Ersek wrote:
>>>>> On 03/20/14 21:38, Christian Borntraeger wrote:
>>>>>> Qiao Nuohan,
>>>>>>
>>>>>> is there a reason why you did not implemented the HMP part for that format
>>>>>> of kdump compressed format? After all this is a patch mostly for
>>>>>> developers,
>>>>>> so a HMP interface might come handy. Do you already have some patch in
>>>>>> preparation or know somebody doing it?
>>>>>
>>>>> http://thread.gmane.org/gmane.comp.emulators.qemu/249283/focus=250059
>>>>>
>>>>> Laszlo
>>>>>
>>>> So in essence: Due to the limitations of the command line parser we cannot
>>>> add the format to the hmp command line.

Flags to select a format could perhaps work.

>>>> So something like adding
>>>>
>>>> dump_guest_memory_set_format <format>
>>>>
>>>> would be the only possible solution with the hmp code as is. Correct?
>>>
>>> Yes, one possibility would be to make the dump command stateful (=
>>> compression format), and to add a new command setting/getting that state.
>>>
>>> Another option would be to leave the current command intact, and add an
>>> independent command that wouldn't take "begin", "end", nor "paging", but
>>> would take compression format.
>>
>> Another possibility is to kill begin/length from the
>> dump-guest-memory command.
>> HMP is not stable, so if that's useful we can do it.
>
> AFAIK, kdump-compressed format can only be analyzed by crash-utility right
> now. ELF is big but we still need it, then begin/length will help us save
> time and space when only part of the memory is need.
>
> IMHO, a new HMP command will be a good choice and it is more simple than
> making dump format 'stateful'. I am not so clear about HMP, but I think
> the new HMP command can be 'dump-guest-memory-with-format', and still
> base on qmp_dump_guest_memory. If you guys like it, I will send a patch
> that add this command.

Paolo encouraged you to *break* the existing HMP command instead of
adding a new one.  I'd like to second that.

Re: [Qemu-devel] hmp interface for kdump compressed format

[qiaonuohan]

On 03/27/2014 01:04 AM, Markus Armbruster wrote:
>>>>> So something like adding
>>>>> >>>>
>>>>> >>>>  dump_guest_memory_set_format<format>
>>>>> >>>>
>>>>> >>>>  would be the only possible solution with the hmp code as is. Correct?
>>>> >>>
>>>> >>>  Yes, one possibility would be to make the dump command stateful (=
>>>> >>>  compression format), and to add a new command setting/getting that state.
>>>> >>>
>>>> >>>  Another option would be to leave the current command intact, and add an
>>>> >>>  independent command that wouldn't take "begin", "end", nor "paging", but
>>>> >>>  would take compression format.
>>> >>
>>> >>  Another possibility is to kill begin/length from the
>>> >>  dump-guest-memory command.
>>> >>  HMP is not stable, so if that's useful we can do it.
>> >
>> >  AFAIK, kdump-compressed format can only be analyzed by crash-utility right
>> >  now. ELF is big but we still need it, then begin/length will help us save
>> >  time and space when only part of the memory is need.
>> >
>> >  IMHO, a new HMP command will be a good choice and it is more simple than
>> >  making dump format 'stateful'. I am not so clear about HMP, but I think
>> >  the new HMP command can be 'dump-guest-memory-with-format', and still
>> >  base on qmp_dump_guest_memory. If you guys like it, I will send a patch
>> >  that add this command.
> Paolo encouraged you to*break*  the existing HMP command instead of
> adding a new one.  I'd like to second that.
> .
>

Hello markus,

I have finish my patch as I stated in my last mail, but something is
wrong with my mail server, I have to wait a couple of days before
sending the patch. You said you prefer *breaking* the existing HMP
command. Since paging/begin/length is still usefull, would you please
give some reason and I will consider that.

-- 
Regards
Qiao Nuohan

Re: [Qemu-devel] hmp interface for kdump compressed format

[Markus Armbruster]

"qiaonuohan@cn.fujitsu.com" <qiaonuohan@cn.fujitsu.com> writes:

> On 03/27/2014 01:04 AM, Markus Armbruster wrote:
>>>>>> So something like adding
>>>>>> >>>>
>>>>>> >>>>  dump_guest_memory_set_format<format>
>>>>>> >>>>
>>>>>> >>>>  would be the only possible solution with the hmp code as
>>>>>> >>>> is. Correct?
>>>>> >>>
>>>>> >>>  Yes, one possibility would be to make the dump command stateful (=
>>>>> >>>  compression format), and to add a new command
>>>>> >>> setting/getting that state.
>>>>> >>>
>>>>> >>>  Another option would be to leave the current command intact,
>>>>> >>> and add an
>>>>> >>>  independent command that wouldn't take "begin", "end", nor
>>>>> >>> "paging", but
>>>>> >>>  would take compression format.
>>>> >>
>>>> >>  Another possibility is to kill begin/length from the
>>>> >>  dump-guest-memory command.
>>>> >>  HMP is not stable, so if that's useful we can do it.
>>> >
>>> >  AFAIK, kdump-compressed format can only be analyzed by crash-utility right
>>> >  now. ELF is big but we still need it, then begin/length will help us save
>>> >  time and space when only part of the memory is need.
>>> >
>>> >  IMHO, a new HMP command will be a good choice and it is more simple than
>>> >  making dump format 'stateful'. I am not so clear about HMP, but I think
>>> >  the new HMP command can be 'dump-guest-memory-with-format', and still
>>> >  base on qmp_dump_guest_memory. If you guys like it, I will send a patch
>>> >  that add this command.
>> Paolo encouraged you to*break*  the existing HMP command instead of
>> adding a new one.  I'd like to second that.
>> .
>>
>
> Hello markus,
>
> I have finish my patch as I stated in my last mail, but something is
> wrong with my mail server, I have to wait a couple of days before
> sending the patch. You said you prefer *breaking* the existing HMP
> command. Since paging/begin/length is still usefull, would you please
> give some reason and I will consider that.

I'm encouraging you to design a HMP command that gives you all you need.
Then call that dump-guest-memory, backward compatibility be damned.

Would

    dump-guest-memory [-p] filename [[format] begin length]

do?

An alternative could be two commands:

    dump-guest-memory filename [format [begin length]]
    dump-guest-memory-elf [-p] filename [begin length]

This bakes the restriction "paging, begin and length work only with
format 'elf'" into the interface.  Not sure that's a good idea, but I'm
leaving that to folks who actually know something about this dumping
business.

If you can find better command names, go right ahead.  You could pick
names that avoid incompatible change.  I care about avoiding
incompatible change to exotic HMP commands even less (a lot less, in
fact) than about naming of exotic HMP commands.

Re: [Qemu-devel] hmp interface for kdump compressed format

[qiaonuohan]

Hello guys,

I have sent the patch, please check here:

http://lists.nongnu.org/archive/html/qemu-devel/2014-04/msg00018.html

On 03/27/2014 04:38 PM, Markus Armbruster wrote:
> "qiaonuohan@cn.fujitsu.com"<qiaonuohan@cn.fujitsu.com>  writes:
>
>> On 03/27/2014 01:04 AM, Markus Armbruster wrote:
>>>>>>> So something like adding
>>>>>>>>>>>
>>>>>>>>>>>   dump_guest_memory_set_format<format>
>>>>>>>>>>>
>>>>>>>>>>>   would be the only possible solution with the hmp code as
>>>>>>>>>>> is. Correct?
>>>>>>>>>
>>>>>>>>>   Yes, one possibility would be to make the dump command stateful (=
>>>>>>>>>   compression format), and to add a new command
>>>>>>>>> setting/getting that state.
>>>>>>>>>
>>>>>>>>>   Another option would be to leave the current command intact,
>>>>>>>>> and add an
>>>>>>>>>   independent command that wouldn't take "begin", "end", nor
>>>>>>>>> "paging", but
>>>>>>>>>   would take compression format.
>>>>>>>
>>>>>>>   Another possibility is to kill begin/length from the
>>>>>>>   dump-guest-memory command.
>>>>>>>   HMP is not stable, so if that's useful we can do it.
>>>>>
>>>>>   AFAIK, kdump-compressed format can only be analyzed by crash-utility right
>>>>>   now. ELF is big but we still need it, then begin/length will help us save
>>>>>   time and space when only part of the memory is need.
>>>>>
>>>>>   IMHO, a new HMP command will be a good choice and it is more simple than
>>>>>   making dump format 'stateful'. I am not so clear about HMP, but I think
>>>>>   the new HMP command can be 'dump-guest-memory-with-format', and still
>>>>>   base on qmp_dump_guest_memory. If you guys like it, I will send a patch
>>>>>   that add this command.
>>> Paolo encouraged you to*break*  the existing HMP command instead of
>>> adding a new one.  I'd like to second that.
>>> .
>>>
>>
>> Hello markus,
>>
>> I have finish my patch as I stated in my last mail, but something is
>> wrong with my mail server, I have to wait a couple of days before
>> sending the patch. You said you prefer *breaking* the existing HMP
>> command. Since paging/begin/length is still usefull, would you please
>> give some reason and I will consider that.
>
> I'm encouraging you to design a HMP command that gives you all you need.
> Then call that dump-guest-memory, backward compatibility be damned.
>
> Would
>
>      dump-guest-memory [-p] filename [[format] begin length]
>
> do?
>
> An alternative could be two commands:
>
>      dump-guest-memory filename [format [begin length]]
>      dump-guest-memory-elf [-p] filename [begin length]
>
> This bakes the restriction "paging, begin and length work only with
> format 'elf'" into the interface.  Not sure that's a good idea, but I'm
> leaving that to folks who actually know something about this dumping
> business.
>
> If you can find better command names, go right ahead.  You could pick
> names that avoid incompatible change.  I care about avoiding
> incompatible change to exotic HMP commands even less (a lot less, in
> fact) than about naming of exotic HMP commands.
>
> .
>


-- 
Regards
Qiao Nuohan