All of lore.kernel.org
 help / color / mirror / Atom feed
* KVM call minutes April 3
@ 2012-04-03 14:43 ` Markus Armbruster
  0 siblings, 0 replies; 24+ messages in thread
From: Markus Armbruster @ 2012-04-03 14:43 UTC (permalink / raw)
  To: qemu-devel, KVM devel mailing list

I'm afraid my notes are rather rough...

* 1.1
  soft freeze apr 15th (less than two weeks)
  hard freeze may 1
  three months cycle for 1.2
  stable machine types only every few releases?  "pc-next"

* Maintainers, got distracted and my notes make no sense, sorry

* MSI injection to KVM irqchips from userspace devices models

* qemu-kvm tree: working towards upstream merge

  not much left, mostly device assignment

* Migration: vmstate and visitors, decoupling the wire format
  why not ASN.1

* qtest: test cases wanted

^ permalink raw reply	[flat|nested] 24+ messages in thread

* [Qemu-devel] KVM call minutes April 3
@ 2012-04-03 14:43 ` Markus Armbruster
  0 siblings, 0 replies; 24+ messages in thread
From: Markus Armbruster @ 2012-04-03 14:43 UTC (permalink / raw)
  To: qemu-devel, KVM devel mailing list

I'm afraid my notes are rather rough...

* 1.1
  soft freeze apr 15th (less than two weeks)
  hard freeze may 1
  three months cycle for 1.2
  stable machine types only every few releases?  "pc-next"

* Maintainers, got distracted and my notes make no sense, sorry

* MSI injection to KVM irqchips from userspace devices models

* qemu-kvm tree: working towards upstream merge

  not much left, mostly device assignment

* Migration: vmstate and visitors, decoupling the wire format
  why not ASN.1

* qtest: test cases wanted

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
  2012-04-03 14:43 ` [Qemu-devel] " Markus Armbruster
  (?)
@ 2012-04-03 20:43 ` Dor Laor
  2012-04-04  1:18     ` Michael Roth
  2012-04-04 11:48     ` Anthony Liguori
  -1 siblings, 2 replies; 24+ messages in thread
From: Dor Laor @ 2012-04-03 20:43 UTC (permalink / raw)
  To: Markus Armbruster; +Cc: qemu-devel, KVM devel mailing list

On 04/03/2012 05:43 PM, Markus Armbruster wrote:
> I'm afraid my notes are rather rough...
>
> * 1.1
>    soft freeze apr 15th (less than two weeks)
>    hard freeze may 1
>    three months cycle for 1.2
>    stable machine types only every few releases?  "pc-next"
>
> * Maintainers, got distracted and my notes make no sense, sorry
>
> * MSI injection to KVM irqchips from userspace devices models
>
> * qemu-kvm tree: working towards upstream merge
>
>    not much left, mostly device assignment
>
> * Migration: vmstate and visitors, decoupling the wire format
>    why not ASN.1

Curiosity kills me of waiting for next week's meeting to get the answer

>
> * qtest: test cases wanted
>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
  2012-04-03 20:43 ` Dor Laor
@ 2012-04-04  1:18     ` Michael Roth
  2012-04-04 11:48     ` Anthony Liguori
  1 sibling, 0 replies; 24+ messages in thread
From: Michael Roth @ 2012-04-04  1:18 UTC (permalink / raw)
  To: Dor Laor; +Cc: Markus Armbruster, qemu-devel, KVM devel mailing list

On Tue, Apr 03, 2012 at 11:43:57PM +0300, Dor Laor wrote:
> On 04/03/2012 05:43 PM, Markus Armbruster wrote:
> >I'm afraid my notes are rather rough...
> >
> >* 1.1
> >   soft freeze apr 15th (less than two weeks)
> >   hard freeze may 1
> >   three months cycle for 1.2
> >   stable machine types only every few releases?  "pc-next"
> >
> >* Maintainers, got distracted and my notes make no sense, sorry
> >
> >* MSI injection to KVM irqchips from userspace devices models
> >
> >* qemu-kvm tree: working towards upstream merge
> >
> >   not much left, mostly device assignment
> >
> >* Migration: vmstate and visitors, decoupling the wire format
> >   why not ASN.1
> 
> Curiosity kills me of waiting for next week's meeting to get the answer

I believe when this had come up in the past the plan was to use ASN.1
 for the wire protocol, but not to address the decoupling problem.

Theoretically it could handle both, but I believe that requires defining
device structures using ASN.1 definitions, which probably isn't suitable
for devices since it results in high level structures which require
special accessors (at least for the libraries I've looked at)

An IDL compiler that generates visitors based on a simple device code
annotations still seems to be the leading option.

Previously I'd jumped the gun a bit by piggy-backing off vmstate to get at
the protocol side, but that permanently baked QEMUFile markers into the
wire protocol which was the wrong approach.

Attacking the IDL/schema side first is the more rationale approach. From
there we can potentially generate ASN.1 BER/DER visitors for the protocol
side, or potentially even just vmstate bindings as a start. I've recently
started looking into the latter... it's completely feasible, the only
downside is it complicates the IDL due requiring support for a lot of
what are very much vmstate-specific items, but it should be possible to
do this in a manner where those annotations are self-contained and
ignorable if we opted to replace vmstate-style declarations.

> 
> >
> >* qtest: test cases wanted
> >
> 
> 

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
@ 2012-04-04  1:18     ` Michael Roth
  0 siblings, 0 replies; 24+ messages in thread
From: Michael Roth @ 2012-04-04  1:18 UTC (permalink / raw)
  To: Dor Laor; +Cc: Markus Armbruster, KVM devel mailing list, qemu-devel

On Tue, Apr 03, 2012 at 11:43:57PM +0300, Dor Laor wrote:
> On 04/03/2012 05:43 PM, Markus Armbruster wrote:
> >I'm afraid my notes are rather rough...
> >
> >* 1.1
> >   soft freeze apr 15th (less than two weeks)
> >   hard freeze may 1
> >   three months cycle for 1.2
> >   stable machine types only every few releases?  "pc-next"
> >
> >* Maintainers, got distracted and my notes make no sense, sorry
> >
> >* MSI injection to KVM irqchips from userspace devices models
> >
> >* qemu-kvm tree: working towards upstream merge
> >
> >   not much left, mostly device assignment
> >
> >* Migration: vmstate and visitors, decoupling the wire format
> >   why not ASN.1
> 
> Curiosity kills me of waiting for next week's meeting to get the answer

I believe when this had come up in the past the plan was to use ASN.1
 for the wire protocol, but not to address the decoupling problem.

Theoretically it could handle both, but I believe that requires defining
device structures using ASN.1 definitions, which probably isn't suitable
for devices since it results in high level structures which require
special accessors (at least for the libraries I've looked at)

An IDL compiler that generates visitors based on a simple device code
annotations still seems to be the leading option.

Previously I'd jumped the gun a bit by piggy-backing off vmstate to get at
the protocol side, but that permanently baked QEMUFile markers into the
wire protocol which was the wrong approach.

Attacking the IDL/schema side first is the more rationale approach. From
there we can potentially generate ASN.1 BER/DER visitors for the protocol
side, or potentially even just vmstate bindings as a start. I've recently
started looking into the latter... it's completely feasible, the only
downside is it complicates the IDL due requiring support for a lot of
what are very much vmstate-specific items, but it should be possible to
do this in a manner where those annotations are self-contained and
ignorable if we opted to replace vmstate-style declarations.

> 
> >
> >* qtest: test cases wanted
> >
> 
> 

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
  2012-04-04  1:18     ` Michael Roth
@ 2012-04-04  7:40       ` Paolo Bonzini
  -1 siblings, 0 replies; 24+ messages in thread
From: Paolo Bonzini @ 2012-04-04  7:40 UTC (permalink / raw)
  To: Michael Roth
  Cc: Dor Laor, Markus Armbruster, qemu-devel, KVM devel mailing list

Il 04/04/2012 03:18, Michael Roth ha scritto:
> Attacking the IDL/schema side first is the more rationale approach. From
> there we can potentially generate ASN.1 BER/DER visitors for the protocol
> side, or potentially even just vmstate bindings as a start. I've recently
> started looking into the latter... it's completely feasible, the only
> downside is it complicates the IDL due requiring support for a lot of
> what are very much vmstate-specific items, but it should be possible to
> do this in a manner where those annotations are self-contained and
> ignorable if we opted to replace vmstate-style declarations.

We can also keep the current vmstate descriptions, but access fields
from the automatically-generated visitors instead of struct fields.
This keeps the IDL simple.

Paolo

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
@ 2012-04-04  7:40       ` Paolo Bonzini
  0 siblings, 0 replies; 24+ messages in thread
From: Paolo Bonzini @ 2012-04-04  7:40 UTC (permalink / raw)
  To: Michael Roth
  Cc: Dor Laor, Markus Armbruster, KVM devel mailing list, qemu-devel

Il 04/04/2012 03:18, Michael Roth ha scritto:
> Attacking the IDL/schema side first is the more rationale approach. From
> there we can potentially generate ASN.1 BER/DER visitors for the protocol
> side, or potentially even just vmstate bindings as a start. I've recently
> started looking into the latter... it's completely feasible, the only
> downside is it complicates the IDL due requiring support for a lot of
> what are very much vmstate-specific items, but it should be possible to
> do this in a manner where those annotations are self-contained and
> ignorable if we opted to replace vmstate-style declarations.

We can also keep the current vmstate descriptions, but access fields
from the automatically-generated visitors instead of struct fields.
This keeps the IDL simple.

Paolo

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: KVM call minutes April 3
  2012-04-04  7:40       ` Paolo Bonzini
@ 2012-04-04 10:37         ` Michael Roth
  -1 siblings, 0 replies; 24+ messages in thread
From: Michael Roth @ 2012-04-04 10:37 UTC (permalink / raw)
  To: Paolo Bonzini
  Cc: Dor Laor, qemu-devel, KVM devel mailing list, Markus Armbruster

[-- Attachment #1: Type: text/plain, Size: 1547 bytes --]

On Apr 4, 2012 2:42 AM, "Paolo Bonzini" <pbonzini@redhat.com> wrote:
>
> Il 04/04/2012 03:18, Michael Roth ha scritto:
> > Attacking the IDL/schema side first is the more rationale approach. From
> > there we can potentially generate ASN.1 BER/DER visitors for the
protocol
> > side, or potentially even just vmstate bindings as a start. I've
recently
> > started looking into the latter... it's completely feasible, the only
> > downside is it complicates the IDL due requiring support for a lot of
> > what are very much vmstate-specific items, but it should be possible to
> > do this in a manner where those annotations are self-contained and
> > ignorable if we opted to replace vmstate-style declarations.
>
> We can also keep the current vmstate descriptions, but access fields
> from the automatically-generated visitors instead of struct fields.
> This keeps the IDL simple.

It may be worthwhile as an incremental step though, one nice thing about
automatically generated bindings is that with the QIDL Anthony prototyped a
while back we assume we serialize by default, so changes in annotated
structs automatically trigger changes in the generated bindings unless you
explicitly mark fields as immutable/derivable/etc, which we can tie into
the build or make check to automatically detect and bring attention to
changes in vmstate. This may be worth the effort if we adopt the proposed 4
year migration support cycle for pc-1.0, since that'll continue to rely on
vmstate even after we move on to an IDL and newer protocol.

>
> Paolo
>

[-- Attachment #2: Type: text/html, Size: 1807 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
@ 2012-04-04 10:37         ` Michael Roth
  0 siblings, 0 replies; 24+ messages in thread
From: Michael Roth @ 2012-04-04 10:37 UTC (permalink / raw)
  To: Paolo Bonzini
  Cc: Dor Laor, qemu-devel, KVM devel mailing list, Markus Armbruster

[-- Attachment #1: Type: text/plain, Size: 1547 bytes --]

On Apr 4, 2012 2:42 AM, "Paolo Bonzini" <pbonzini@redhat.com> wrote:
>
> Il 04/04/2012 03:18, Michael Roth ha scritto:
> > Attacking the IDL/schema side first is the more rationale approach. From
> > there we can potentially generate ASN.1 BER/DER visitors for the
protocol
> > side, or potentially even just vmstate bindings as a start. I've
recently
> > started looking into the latter... it's completely feasible, the only
> > downside is it complicates the IDL due requiring support for a lot of
> > what are very much vmstate-specific items, but it should be possible to
> > do this in a manner where those annotations are self-contained and
> > ignorable if we opted to replace vmstate-style declarations.
>
> We can also keep the current vmstate descriptions, but access fields
> from the automatically-generated visitors instead of struct fields.
> This keeps the IDL simple.

It may be worthwhile as an incremental step though, one nice thing about
automatically generated bindings is that with the QIDL Anthony prototyped a
while back we assume we serialize by default, so changes in annotated
structs automatically trigger changes in the generated bindings unless you
explicitly mark fields as immutable/derivable/etc, which we can tie into
the build or make check to automatically detect and bring attention to
changes in vmstate. This may be worth the effort if we adopt the proposed 4
year migration support cycle for pc-1.0, since that'll continue to rely on
vmstate even after we move on to an IDL and newer protocol.

>
> Paolo
>

[-- Attachment #2: Type: text/html, Size: 1807 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
  2012-04-04 10:37         ` [Qemu-devel] " Michael Roth
@ 2012-04-04 10:53           ` Dor Laor
  -1 siblings, 0 replies; 24+ messages in thread
From: Dor Laor @ 2012-04-04 10:53 UTC (permalink / raw)
  To: Michael Roth
  Cc: Paolo Bonzini, qemu-devel, Markus Armbruster, KVM devel mailing list

On 04/04/2012 01:37 PM, Michael Roth wrote:
>
> On Apr 4, 2012 2:42 AM, "Paolo Bonzini" <pbonzini@redhat.com
> <mailto:pbonzini@redhat.com>> wrote:
>  >
>  > Il 04/04/2012 03:18, Michael Roth ha scritto:
>  > > Attacking the IDL/schema side first is the more rationale approach.
> From
>  > > there we can potentially generate ASN.1 BER/DER visitors for the
> protocol
>  > > side, or potentially even just vmstate bindings as a start. I've
> recently
>  > > started looking into the latter... it's completely feasible, the only
>  > > downside is it complicates the IDL due requiring support for a lot of
>  > > what are very much vmstate-specific items, but it should be possible to
>  > > do this in a manner where those annotations are self-contained and
>  > > ignorable if we opted to replace vmstate-style declarations.
>  >
>  > We can also keep the current vmstate descriptions, but access fields
>  > from the automatically-generated visitors instead of struct fields.
>  > This keeps the IDL simple.
>
> It may be worthwhile as an incremental step though, one nice thing about
> automatically generated bindings is that with the QIDL Anthony
> prototyped a while back we assume we serialize by default, so changes in
> annotated structs automatically trigger changes in the generated
> bindings unless you explicitly mark fields as immutable/derivable/etc,
> which we can tie into the build or make check to automatically detect
> and bring attention to changes in vmstate. This may be worth the effort
> if we adopt the proposed 4 year migration support cycle for pc-1.0,
> since that'll continue to rely on vmstate even after we move on to an
> IDL and newer protocol.

Beyond ASL/IDL I like to be sure that we're not just translating one 
format to other representation but instead we introduce some new 
functionality like:
  - Ability to negotiate the protocol version
  - Bi-direction data exchange, the sender will send data as a function
    of the target release
    - Include the machine type too
  - Synchronize the entire qemu cmdline and don't relay on management to
    set it up.
    - Along the way, deal w/ hotplug events.

>
>  >
>  > Paolo
>  >
>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
@ 2012-04-04 10:53           ` Dor Laor
  0 siblings, 0 replies; 24+ messages in thread
From: Dor Laor @ 2012-04-04 10:53 UTC (permalink / raw)
  To: Michael Roth
  Cc: Paolo Bonzini, qemu-devel, KVM devel mailing list, Markus Armbruster

On 04/04/2012 01:37 PM, Michael Roth wrote:
>
> On Apr 4, 2012 2:42 AM, "Paolo Bonzini" <pbonzini@redhat.com
> <mailto:pbonzini@redhat.com>> wrote:
>  >
>  > Il 04/04/2012 03:18, Michael Roth ha scritto:
>  > > Attacking the IDL/schema side first is the more rationale approach.
> From
>  > > there we can potentially generate ASN.1 BER/DER visitors for the
> protocol
>  > > side, or potentially even just vmstate bindings as a start. I've
> recently
>  > > started looking into the latter... it's completely feasible, the only
>  > > downside is it complicates the IDL due requiring support for a lot of
>  > > what are very much vmstate-specific items, but it should be possible to
>  > > do this in a manner where those annotations are self-contained and
>  > > ignorable if we opted to replace vmstate-style declarations.
>  >
>  > We can also keep the current vmstate descriptions, but access fields
>  > from the automatically-generated visitors instead of struct fields.
>  > This keeps the IDL simple.
>
> It may be worthwhile as an incremental step though, one nice thing about
> automatically generated bindings is that with the QIDL Anthony
> prototyped a while back we assume we serialize by default, so changes in
> annotated structs automatically trigger changes in the generated
> bindings unless you explicitly mark fields as immutable/derivable/etc,
> which we can tie into the build or make check to automatically detect
> and bring attention to changes in vmstate. This may be worth the effort
> if we adopt the proposed 4 year migration support cycle for pc-1.0,
> since that'll continue to rely on vmstate even after we move on to an
> IDL and newer protocol.

Beyond ASL/IDL I like to be sure that we're not just translating one 
format to other representation but instead we introduce some new 
functionality like:
  - Ability to negotiate the protocol version
  - Bi-direction data exchange, the sender will send data as a function
    of the target release
    - Include the machine type too
  - Synchronize the entire qemu cmdline and don't relay on management to
    set it up.
    - Along the way, deal w/ hotplug events.

>
>  >
>  > Paolo
>  >
>

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
  2012-04-03 20:43 ` Dor Laor
@ 2012-04-04 11:48     ` Anthony Liguori
  2012-04-04 11:48     ` Anthony Liguori
  1 sibling, 0 replies; 24+ messages in thread
From: Anthony Liguori @ 2012-04-04 11:48 UTC (permalink / raw)
  To: dlaor; +Cc: Markus Armbruster, qemu-devel, KVM devel mailing list

On 04/03/2012 03:43 PM, Dor Laor wrote:
> On 04/03/2012 05:43 PM, Markus Armbruster wrote:
>> I'm afraid my notes are rather rough...
>>
>> * 1.1
>> soft freeze apr 15th (less than two weeks)
>> hard freeze may 1
>> three months cycle for 1.2
>> stable machine types only every few releases? "pc-next"
>>
>> * Maintainers, got distracted and my notes make no sense, sorry
>>
>> * MSI injection to KVM irqchips from userspace devices models
>>
>> * qemu-kvm tree: working towards upstream merge
>>
>> not much left, mostly device assignment
>>
>> * Migration: vmstate and visitors, decoupling the wire format
>> why not ASN.1
>
> Curiosity kills me of waiting for next week's meeting to get the answer

ASN.1 is an IDL format.  It's encoded in many ways including BER.  I think 
there's wide spread agreement that the next migration wire format should be 
encoded with BER which means it could be described via ASN.1 but I don't think 
we intend on using ASN.1 within the code base.

I don't think using ASN.1 to describe devices makes sense.  There really aren't 
very good Open Source ASN.1 compilers.  I also don't think the syntax is 
flexible enough to fully describe a device either.

Regards,

Anthony Liguori

>
>>
>> * qtest: test cases wanted
>>
>
>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
@ 2012-04-04 11:48     ` Anthony Liguori
  0 siblings, 0 replies; 24+ messages in thread
From: Anthony Liguori @ 2012-04-04 11:48 UTC (permalink / raw)
  To: dlaor; +Cc: Markus Armbruster, KVM devel mailing list, qemu-devel

On 04/03/2012 03:43 PM, Dor Laor wrote:
> On 04/03/2012 05:43 PM, Markus Armbruster wrote:
>> I'm afraid my notes are rather rough...
>>
>> * 1.1
>> soft freeze apr 15th (less than two weeks)
>> hard freeze may 1
>> three months cycle for 1.2
>> stable machine types only every few releases? "pc-next"
>>
>> * Maintainers, got distracted and my notes make no sense, sorry
>>
>> * MSI injection to KVM irqchips from userspace devices models
>>
>> * qemu-kvm tree: working towards upstream merge
>>
>> not much left, mostly device assignment
>>
>> * Migration: vmstate and visitors, decoupling the wire format
>> why not ASN.1
>
> Curiosity kills me of waiting for next week's meeting to get the answer

ASN.1 is an IDL format.  It's encoded in many ways including BER.  I think 
there's wide spread agreement that the next migration wire format should be 
encoded with BER which means it could be described via ASN.1 but I don't think 
we intend on using ASN.1 within the code base.

I don't think using ASN.1 to describe devices makes sense.  There really aren't 
very good Open Source ASN.1 compilers.  I also don't think the syntax is 
flexible enough to fully describe a device either.

Regards,

Anthony Liguori

>
>>
>> * qtest: test cases wanted
>>
>
>

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
  2012-04-04 10:53           ` Dor Laor
@ 2012-04-04 11:52             ` Anthony Liguori
  -1 siblings, 0 replies; 24+ messages in thread
From: Anthony Liguori @ 2012-04-04 11:52 UTC (permalink / raw)
  To: dlaor
  Cc: Michael Roth, Paolo Bonzini, qemu-devel, KVM devel mailing list,
	Markus Armbruster

On 04/04/2012 05:53 AM, Dor Laor wrote:
> On 04/04/2012 01:37 PM, Michael Roth wrote:
>>
>> On Apr 4, 2012 2:42 AM, "Paolo Bonzini" <pbonzini@redhat.com
>> <mailto:pbonzini@redhat.com>> wrote:
>> >
>> > Il 04/04/2012 03:18, Michael Roth ha scritto:
>> > > Attacking the IDL/schema side first is the more rationale approach.
>> From
>> > > there we can potentially generate ASN.1 BER/DER visitors for the
>> protocol
>> > > side, or potentially even just vmstate bindings as a start. I've
>> recently
>> > > started looking into the latter... it's completely feasible, the only
>> > > downside is it complicates the IDL due requiring support for a lot of
>> > > what are very much vmstate-specific items, but it should be possible to
>> > > do this in a manner where those annotations are self-contained and
>> > > ignorable if we opted to replace vmstate-style declarations.
>> >
>> > We can also keep the current vmstate descriptions, but access fields
>> > from the automatically-generated visitors instead of struct fields.
>> > This keeps the IDL simple.
>>
>> It may be worthwhile as an incremental step though, one nice thing about
>> automatically generated bindings is that with the QIDL Anthony
>> prototyped a while back we assume we serialize by default, so changes in
>> annotated structs automatically trigger changes in the generated
>> bindings unless you explicitly mark fields as immutable/derivable/etc,
>> which we can tie into the build or make check to automatically detect
>> and bring attention to changes in vmstate. This may be worth the effort
>> if we adopt the proposed 4 year migration support cycle for pc-1.0,
>> since that'll continue to rely on vmstate even after we move on to an
>> IDL and newer protocol.
>
> Beyond ASL/IDL I like to be sure that we're not just translating one format to
> other representation but instead we introduce some new functionality like:
> - Ability to negotiate the protocol version

Ack.

> - Bi-direction data exchange, the sender will send data as a function
> of the target release

The reason bi-direction data exchange doesn't exist is because it would add 
latency to the critical path.  I think we should avoid bi-directional data 
exchange unless there's an extremely compelling reason to do so.

> - Include the machine type too
> - Synchronize the entire qemu cmdline and don't relay on management to
> set it up.
> - Along the way, deal w/ hotplug events.

This will be address via QOM.  As we convert backends and machine types, we 
should be able to dump out the full configuration and send it over the wire.

Regards,

Anthony Liguori

>
>>
>> >
>> > Paolo
>> >
>>
>
>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
@ 2012-04-04 11:52             ` Anthony Liguori
  0 siblings, 0 replies; 24+ messages in thread
From: Anthony Liguori @ 2012-04-04 11:52 UTC (permalink / raw)
  To: dlaor
  Cc: Paolo Bonzini, Markus Armbruster, Michael Roth,
	KVM devel mailing list, qemu-devel

On 04/04/2012 05:53 AM, Dor Laor wrote:
> On 04/04/2012 01:37 PM, Michael Roth wrote:
>>
>> On Apr 4, 2012 2:42 AM, "Paolo Bonzini" <pbonzini@redhat.com
>> <mailto:pbonzini@redhat.com>> wrote:
>> >
>> > Il 04/04/2012 03:18, Michael Roth ha scritto:
>> > > Attacking the IDL/schema side first is the more rationale approach.
>> From
>> > > there we can potentially generate ASN.1 BER/DER visitors for the
>> protocol
>> > > side, or potentially even just vmstate bindings as a start. I've
>> recently
>> > > started looking into the latter... it's completely feasible, the only
>> > > downside is it complicates the IDL due requiring support for a lot of
>> > > what are very much vmstate-specific items, but it should be possible to
>> > > do this in a manner where those annotations are self-contained and
>> > > ignorable if we opted to replace vmstate-style declarations.
>> >
>> > We can also keep the current vmstate descriptions, but access fields
>> > from the automatically-generated visitors instead of struct fields.
>> > This keeps the IDL simple.
>>
>> It may be worthwhile as an incremental step though, one nice thing about
>> automatically generated bindings is that with the QIDL Anthony
>> prototyped a while back we assume we serialize by default, so changes in
>> annotated structs automatically trigger changes in the generated
>> bindings unless you explicitly mark fields as immutable/derivable/etc,
>> which we can tie into the build or make check to automatically detect
>> and bring attention to changes in vmstate. This may be worth the effort
>> if we adopt the proposed 4 year migration support cycle for pc-1.0,
>> since that'll continue to rely on vmstate even after we move on to an
>> IDL and newer protocol.
>
> Beyond ASL/IDL I like to be sure that we're not just translating one format to
> other representation but instead we introduce some new functionality like:
> - Ability to negotiate the protocol version

Ack.

> - Bi-direction data exchange, the sender will send data as a function
> of the target release

The reason bi-direction data exchange doesn't exist is because it would add 
latency to the critical path.  I think we should avoid bi-directional data 
exchange unless there's an extremely compelling reason to do so.

> - Include the machine type too
> - Synchronize the entire qemu cmdline and don't relay on management to
> set it up.
> - Along the way, deal w/ hotplug events.

This will be address via QOM.  As we convert backends and machine types, we 
should be able to dump out the full configuration and send it over the wire.

Regards,

Anthony Liguori

>
>>
>> >
>> > Paolo
>> >
>>
>
>

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
  2012-04-04 11:52             ` Anthony Liguori
@ 2012-04-04 12:01               ` Dor Laor
  -1 siblings, 0 replies; 24+ messages in thread
From: Dor Laor @ 2012-04-04 12:01 UTC (permalink / raw)
  To: Anthony Liguori
  Cc: Paolo Bonzini, Markus Armbruster, Michael Roth,
	KVM devel mailing list, qemu-devel

On 04/04/2012 02:52 PM, Anthony Liguori wrote:
> On 04/04/2012 05:53 AM, Dor Laor wrote:
>> On 04/04/2012 01:37 PM, Michael Roth wrote:
>>>
>>> On Apr 4, 2012 2:42 AM, "Paolo Bonzini" <pbonzini@redhat.com
>>> <mailto:pbonzini@redhat.com>> wrote:
>>> >
>>> > Il 04/04/2012 03:18, Michael Roth ha scritto:
>>> > > Attacking the IDL/schema side first is the more rationale approach.
>>> From
>>> > > there we can potentially generate ASN.1 BER/DER visitors for the
>>> protocol
>>> > > side, or potentially even just vmstate bindings as a start. I've
>>> recently
>>> > > started looking into the latter... it's completely feasible, the
>>> only
>>> > > downside is it complicates the IDL due requiring support for a
>>> lot of
>>> > > what are very much vmstate-specific items, but it should be
>>> possible to
>>> > > do this in a manner where those annotations are self-contained and
>>> > > ignorable if we opted to replace vmstate-style declarations.
>>> >
>>> > We can also keep the current vmstate descriptions, but access fields
>>> > from the automatically-generated visitors instead of struct fields.
>>> > This keeps the IDL simple.
>>>
>>> It may be worthwhile as an incremental step though, one nice thing about
>>> automatically generated bindings is that with the QIDL Anthony
>>> prototyped a while back we assume we serialize by default, so changes in
>>> annotated structs automatically trigger changes in the generated
>>> bindings unless you explicitly mark fields as immutable/derivable/etc,
>>> which we can tie into the build or make check to automatically detect
>>> and bring attention to changes in vmstate. This may be worth the effort
>>> if we adopt the proposed 4 year migration support cycle for pc-1.0,
>>> since that'll continue to rely on vmstate even after we move on to an
>>> IDL and newer protocol.
>>
>> Beyond ASL/IDL I like to be sure that we're not just translating one
>> format to
>> other representation but instead we introduce some new functionality
>> like:
>> - Ability to negotiate the protocol version
>
> Ack.
>
>> - Bi-direction data exchange, the sender will send data as a function
>> of the target release
>
> The reason bi-direction data exchange doesn't exist is because it would
> add latency to the critical path. I think we should avoid bi-directional

Not necessarily, there is not need to do the exchange on the down time, 
you can do it ahead of time during the initial connection and few 
additional msec or even a second won't change much.

> data exchange unless there's an extremely compelling reason to do so.

The key advantage is that you'll be able to migrate to an old qemu that 
may not be compatible w/ the standard protocol and the source will be 
able to discover this and adjust.
At the moment I don't have anything more concrete than that but I think 
that's happen in the past and will continue to happen and we can add the 
required hook into the protocol.

>
>> - Include the machine type too
>> - Synchronize the entire qemu cmdline and don't relay on management to
>> set it up.
>> - Along the way, deal w/ hotplug events.
>
> This will be address via QOM. As we convert backends and machine types,
> we should be able to dump out the full configuration and send it over
> the wire.
>
> Regards,
>
> Anthony Liguori
>
>>
>>>
>>> >
>>> > Paolo
>>> >
>>>
>>
>>
>
>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
@ 2012-04-04 12:01               ` Dor Laor
  0 siblings, 0 replies; 24+ messages in thread
From: Dor Laor @ 2012-04-04 12:01 UTC (permalink / raw)
  To: Anthony Liguori
  Cc: qemu-devel, Paolo Bonzini, Markus Armbruster,
	KVM devel mailing list, Michael Roth

On 04/04/2012 02:52 PM, Anthony Liguori wrote:
> On 04/04/2012 05:53 AM, Dor Laor wrote:
>> On 04/04/2012 01:37 PM, Michael Roth wrote:
>>>
>>> On Apr 4, 2012 2:42 AM, "Paolo Bonzini" <pbonzini@redhat.com
>>> <mailto:pbonzini@redhat.com>> wrote:
>>> >
>>> > Il 04/04/2012 03:18, Michael Roth ha scritto:
>>> > > Attacking the IDL/schema side first is the more rationale approach.
>>> From
>>> > > there we can potentially generate ASN.1 BER/DER visitors for the
>>> protocol
>>> > > side, or potentially even just vmstate bindings as a start. I've
>>> recently
>>> > > started looking into the latter... it's completely feasible, the
>>> only
>>> > > downside is it complicates the IDL due requiring support for a
>>> lot of
>>> > > what are very much vmstate-specific items, but it should be
>>> possible to
>>> > > do this in a manner where those annotations are self-contained and
>>> > > ignorable if we opted to replace vmstate-style declarations.
>>> >
>>> > We can also keep the current vmstate descriptions, but access fields
>>> > from the automatically-generated visitors instead of struct fields.
>>> > This keeps the IDL simple.
>>>
>>> It may be worthwhile as an incremental step though, one nice thing about
>>> automatically generated bindings is that with the QIDL Anthony
>>> prototyped a while back we assume we serialize by default, so changes in
>>> annotated structs automatically trigger changes in the generated
>>> bindings unless you explicitly mark fields as immutable/derivable/etc,
>>> which we can tie into the build or make check to automatically detect
>>> and bring attention to changes in vmstate. This may be worth the effort
>>> if we adopt the proposed 4 year migration support cycle for pc-1.0,
>>> since that'll continue to rely on vmstate even after we move on to an
>>> IDL and newer protocol.
>>
>> Beyond ASL/IDL I like to be sure that we're not just translating one
>> format to
>> other representation but instead we introduce some new functionality
>> like:
>> - Ability to negotiate the protocol version
>
> Ack.
>
>> - Bi-direction data exchange, the sender will send data as a function
>> of the target release
>
> The reason bi-direction data exchange doesn't exist is because it would
> add latency to the critical path. I think we should avoid bi-directional

Not necessarily, there is not need to do the exchange on the down time, 
you can do it ahead of time during the initial connection and few 
additional msec or even a second won't change much.

> data exchange unless there's an extremely compelling reason to do so.

The key advantage is that you'll be able to migrate to an old qemu that 
may not be compatible w/ the standard protocol and the source will be 
able to discover this and adjust.
At the moment I don't have anything more concrete than that but I think 
that's happen in the past and will continue to happen and we can add the 
required hook into the protocol.

>
>> - Include the machine type too
>> - Synchronize the entire qemu cmdline and don't relay on management to
>> set it up.
>> - Along the way, deal w/ hotplug events.
>
> This will be address via QOM. As we convert backends and machine types,
> we should be able to dump out the full configuration and send it over
> the wire.
>
> Regards,
>
> Anthony Liguori
>
>>
>>>
>>> >
>>> > Paolo
>>> >
>>>
>>
>>
>
>

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
  2012-04-04 10:53           ` Dor Laor
@ 2012-04-04 12:14             ` Michael Roth
  -1 siblings, 0 replies; 24+ messages in thread
From: Michael Roth @ 2012-04-04 12:14 UTC (permalink / raw)
  To: Dor Laor
  Cc: Paolo Bonzini, qemu-devel, Markus Armbruster, KVM devel mailing list

On Wed, Apr 04, 2012 at 01:53:34PM +0300, Dor Laor wrote:
> On 04/04/2012 01:37 PM, Michael Roth wrote:
> >
> >On Apr 4, 2012 2:42 AM, "Paolo Bonzini" <pbonzini@redhat.com
> ><mailto:pbonzini@redhat.com>> wrote:
> > >
> > > Il 04/04/2012 03:18, Michael Roth ha scritto:
> > > > Attacking the IDL/schema side first is the more rationale approach.
> >From
> > > > there we can potentially generate ASN.1 BER/DER visitors for the
> >protocol
> > > > side, or potentially even just vmstate bindings as a start. I've
> >recently
> > > > started looking into the latter... it's completely feasible, the only
> > > > downside is it complicates the IDL due requiring support for a lot of
> > > > what are very much vmstate-specific items, but it should be possible to
> > > > do this in a manner where those annotations are self-contained and
> > > > ignorable if we opted to replace vmstate-style declarations.
> > >
> > > We can also keep the current vmstate descriptions, but access fields
> > > from the automatically-generated visitors instead of struct fields.
> > > This keeps the IDL simple.
> >
> >It may be worthwhile as an incremental step though, one nice thing about
> >automatically generated bindings is that with the QIDL Anthony
> >prototyped a while back we assume we serialize by default, so changes in
> >annotated structs automatically trigger changes in the generated
> >bindings unless you explicitly mark fields as immutable/derivable/etc,
> >which we can tie into the build or make check to automatically detect
> >and bring attention to changes in vmstate. This may be worth the effort
> >if we adopt the proposed 4 year migration support cycle for pc-1.0,
> >since that'll continue to rely on vmstate even after we move on to an
> >IDL and newer protocol.
> 
> Beyond ASL/IDL I like to be sure that we're not just translating one
> format to other representation but instead we introduce some new
> functionality like:
>  - Ability to negotiate the protocol version
>  - Bi-direction data exchange, the sender will send data as a function
>    of the target release
>    - Include the machine type too

I've been toying with the notion of having the target start up a QMP
limited server that the source talks to to orchestrate negotiation. We
could potentially even send the device state by taking our QIDL-generated
visitors and serializing state via QmpOutputVisitor. QMP can be made
aware of the format of the device state input by taking the intermediate
step of generating QAPI schemas via QIDL, and using the QAPI code
generators to generate the visitors rather than QIDL directly. This
would also address the protocol side: just use QMP rather than ASN.1..

It's not as compact, but device state is such a small amount of data
compared to memory/disk that I don't think it's worth optimizing that
aspect, though we could use compression at the protocol layer if we
were inclined. Anything more suited to an out-of-band protocol, like
memory/disk, could be orchestrated via this interface... source can ask
target for a port to handle such things, negotiate stuff like XBZRLE,
etc.

>  - Synchronize the entire qemu cmdline and don't relay on management to
>    set it up.
>    - Along the way, deal w/ hotplug events.

My initial plan for the QIDL-generated visitors is to associate a QOM
property, "state", with each device, and to serialize device state by
walking the QOM composition tree, the main rationale being that if we
extend that serialization to include other QOM properties, I believe we
have everything we need to recreate all the devices on the target:
parent->child relationships, types, properties set via cmdline,
device state...

A simpler alternative would be to leverage just send the cmdline
options over to the target and assume that it results in the same underlying
machine, then just send off the device state. Much simpler actually...but
the above approach should work regardless of changes to the command-line
options on the source... having an internally stable cmdline scheme
might work as well...

I'm not sure what the right approach is here but whatever we decide on I think
being able to automatically generate visitors from annotations is a good
first-step and should tie into any forseeable approaches.

> 
> >
> > >
> > > Paolo
> > >
> >
> 

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
@ 2012-04-04 12:14             ` Michael Roth
  0 siblings, 0 replies; 24+ messages in thread
From: Michael Roth @ 2012-04-04 12:14 UTC (permalink / raw)
  To: Dor Laor
  Cc: Paolo Bonzini, qemu-devel, KVM devel mailing list, Markus Armbruster

On Wed, Apr 04, 2012 at 01:53:34PM +0300, Dor Laor wrote:
> On 04/04/2012 01:37 PM, Michael Roth wrote:
> >
> >On Apr 4, 2012 2:42 AM, "Paolo Bonzini" <pbonzini@redhat.com
> ><mailto:pbonzini@redhat.com>> wrote:
> > >
> > > Il 04/04/2012 03:18, Michael Roth ha scritto:
> > > > Attacking the IDL/schema side first is the more rationale approach.
> >From
> > > > there we can potentially generate ASN.1 BER/DER visitors for the
> >protocol
> > > > side, or potentially even just vmstate bindings as a start. I've
> >recently
> > > > started looking into the latter... it's completely feasible, the only
> > > > downside is it complicates the IDL due requiring support for a lot of
> > > > what are very much vmstate-specific items, but it should be possible to
> > > > do this in a manner where those annotations are self-contained and
> > > > ignorable if we opted to replace vmstate-style declarations.
> > >
> > > We can also keep the current vmstate descriptions, but access fields
> > > from the automatically-generated visitors instead of struct fields.
> > > This keeps the IDL simple.
> >
> >It may be worthwhile as an incremental step though, one nice thing about
> >automatically generated bindings is that with the QIDL Anthony
> >prototyped a while back we assume we serialize by default, so changes in
> >annotated structs automatically trigger changes in the generated
> >bindings unless you explicitly mark fields as immutable/derivable/etc,
> >which we can tie into the build or make check to automatically detect
> >and bring attention to changes in vmstate. This may be worth the effort
> >if we adopt the proposed 4 year migration support cycle for pc-1.0,
> >since that'll continue to rely on vmstate even after we move on to an
> >IDL and newer protocol.
> 
> Beyond ASL/IDL I like to be sure that we're not just translating one
> format to other representation but instead we introduce some new
> functionality like:
>  - Ability to negotiate the protocol version
>  - Bi-direction data exchange, the sender will send data as a function
>    of the target release
>    - Include the machine type too

I've been toying with the notion of having the target start up a QMP
limited server that the source talks to to orchestrate negotiation. We
could potentially even send the device state by taking our QIDL-generated
visitors and serializing state via QmpOutputVisitor. QMP can be made
aware of the format of the device state input by taking the intermediate
step of generating QAPI schemas via QIDL, and using the QAPI code
generators to generate the visitors rather than QIDL directly. This
would also address the protocol side: just use QMP rather than ASN.1..

It's not as compact, but device state is such a small amount of data
compared to memory/disk that I don't think it's worth optimizing that
aspect, though we could use compression at the protocol layer if we
were inclined. Anything more suited to an out-of-band protocol, like
memory/disk, could be orchestrated via this interface... source can ask
target for a port to handle such things, negotiate stuff like XBZRLE,
etc.

>  - Synchronize the entire qemu cmdline and don't relay on management to
>    set it up.
>    - Along the way, deal w/ hotplug events.

My initial plan for the QIDL-generated visitors is to associate a QOM
property, "state", with each device, and to serialize device state by
walking the QOM composition tree, the main rationale being that if we
extend that serialization to include other QOM properties, I believe we
have everything we need to recreate all the devices on the target:
parent->child relationships, types, properties set via cmdline,
device state...

A simpler alternative would be to leverage just send the cmdline
options over to the target and assume that it results in the same underlying
machine, then just send off the device state. Much simpler actually...but
the above approach should work regardless of changes to the command-line
options on the source... having an internally stable cmdline scheme
might work as well...

I'm not sure what the right approach is here but whatever we decide on I think
being able to automatically generate visitors from annotations is a good
first-step and should tie into any forseeable approaches.

> 
> >
> > >
> > > Paolo
> > >
> >
> 

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
  2012-04-04 12:14             ` Michael Roth
@ 2012-04-04 13:21               ` Igor Mammedov
  -1 siblings, 0 replies; 24+ messages in thread
From: Igor Mammedov @ 2012-04-04 13:21 UTC (permalink / raw)
  To: Michael Roth
  Cc: Dor Laor, Paolo Bonzini, qemu-devel, KVM devel mailing list,
	Markus Armbruster

On 04/04/2012 02:14 PM, Michael Roth wrote:
> On Wed, Apr 04, 2012 at 01:53:34PM +0300, Dor Laor wrote:
>> On 04/04/2012 01:37 PM, Michael Roth wrote:
>>>
>>> On Apr 4, 2012 2:42 AM, "Paolo Bonzini"<pbonzini@redhat.com
>>> <mailto:pbonzini@redhat.com>>  wrote:
>>>>
>>>> Il 04/04/2012 03:18, Michael Roth ha scritto:
>>>>> Attacking the IDL/schema side first is the more rationale approach.
>>> From
>>>>> there we can potentially generate ASN.1 BER/DER visitors for the
>>> protocol
>>>>> side, or potentially even just vmstate bindings as a start. I've
>>> recently
>>>>> started looking into the latter... it's completely feasible, the only
>>>>> downside is it complicates the IDL due requiring support for a lot of
>>>>> what are very much vmstate-specific items, but it should be possible to
>>>>> do this in a manner where those annotations are self-contained and
>>>>> ignorable if we opted to replace vmstate-style declarations.
>>>>
>>>> We can also keep the current vmstate descriptions, but access fields
>>>> from the automatically-generated visitors instead of struct fields.
>>>> This keeps the IDL simple.
>>>
>>> It may be worthwhile as an incremental step though, one nice thing about
>>> automatically generated bindings is that with the QIDL Anthony
>>> prototyped a while back we assume we serialize by default, so changes in
>>> annotated structs automatically trigger changes in the generated
>>> bindings unless you explicitly mark fields as immutable/derivable/etc,
>>> which we can tie into the build or make check to automatically detect
>>> and bring attention to changes in vmstate. This may be worth the effort
>>> if we adopt the proposed 4 year migration support cycle for pc-1.0,
>>> since that'll continue to rely on vmstate even after we move on to an
>>> IDL and newer protocol.
>>
>> Beyond ASL/IDL I like to be sure that we're not just translating one
>> format to other representation but instead we introduce some new
>> functionality like:
>>   - Ability to negotiate the protocol version
>>   - Bi-direction data exchange, the sender will send data as a function
>>     of the target release
>>     - Include the machine type too
>
> I've been toying with the notion of having the target start up a QMP
> limited server that the source talks to to orchestrate negotiation. We
> could potentially even send the device state by taking our QIDL-generated
> visitors and serializing state via QmpOutputVisitor. QMP can be made
> aware of the format of the device state input by taking the intermediate
> step of generating QAPI schemas via QIDL, and using the QAPI code
> generators to generate the visitors rather than QIDL directly. This
> would also address the protocol side: just use QMP rather than ASN.1..
>
> It's not as compact, but device state is such a small amount of data
> compared to memory/disk that I don't think it's worth optimizing that
> aspect, though we could use compression at the protocol layer if we
> were inclined. Anything more suited to an out-of-band protocol, like
> memory/disk, could be orchestrated via this interface... source can ask
> target for a port to handle such things, negotiate stuff like XBZRLE,
> etc.
>
>>   - Synchronize the entire qemu cmdline and don't relay on management to
>>     set it up.
>>     - Along the way, deal w/ hotplug events.
>
> My initial plan for the QIDL-generated visitors is to associate a QOM
> property, "state", with each device, and to serialize device state by
> walking the QOM composition tree, the main rationale being that if we
> extend that serialization to include other QOM properties, I believe we
> have everything we need to recreate all the devices on the target:
> parent->child relationships, types, properties set via cmdline,
> device state...
>
> A simpler alternative would be to leverage just send the cmdline
> options over to the target and assume that it results in the same underlying
> machine, then just send off the device state. Much simpler actually...but
> the above approach should work regardless of changes to the command-line
> options on the source... having an internally stable cmdline scheme
> might work as well...
Will command line take in account hot-plugged devices?

>
> I'm not sure what the right approach is here but whatever we decide on I think
> being able to automatically generate visitors from annotations is a good
> first-step and should tie into any forseeable approaches.
>
>>
>>>
>>>>
>>>> Paolo
>>>>
>>>
>>
>

-- 
-----
  Igor

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
@ 2012-04-04 13:21               ` Igor Mammedov
  0 siblings, 0 replies; 24+ messages in thread
From: Igor Mammedov @ 2012-04-04 13:21 UTC (permalink / raw)
  To: Michael Roth
  Cc: Paolo Bonzini, Dor Laor, qemu-devel, KVM devel mailing list,
	Markus Armbruster

On 04/04/2012 02:14 PM, Michael Roth wrote:
> On Wed, Apr 04, 2012 at 01:53:34PM +0300, Dor Laor wrote:
>> On 04/04/2012 01:37 PM, Michael Roth wrote:
>>>
>>> On Apr 4, 2012 2:42 AM, "Paolo Bonzini"<pbonzini@redhat.com
>>> <mailto:pbonzini@redhat.com>>  wrote:
>>>>
>>>> Il 04/04/2012 03:18, Michael Roth ha scritto:
>>>>> Attacking the IDL/schema side first is the more rationale approach.
>>> From
>>>>> there we can potentially generate ASN.1 BER/DER visitors for the
>>> protocol
>>>>> side, or potentially even just vmstate bindings as a start. I've
>>> recently
>>>>> started looking into the latter... it's completely feasible, the only
>>>>> downside is it complicates the IDL due requiring support for a lot of
>>>>> what are very much vmstate-specific items, but it should be possible to
>>>>> do this in a manner where those annotations are self-contained and
>>>>> ignorable if we opted to replace vmstate-style declarations.
>>>>
>>>> We can also keep the current vmstate descriptions, but access fields
>>>> from the automatically-generated visitors instead of struct fields.
>>>> This keeps the IDL simple.
>>>
>>> It may be worthwhile as an incremental step though, one nice thing about
>>> automatically generated bindings is that with the QIDL Anthony
>>> prototyped a while back we assume we serialize by default, so changes in
>>> annotated structs automatically trigger changes in the generated
>>> bindings unless you explicitly mark fields as immutable/derivable/etc,
>>> which we can tie into the build or make check to automatically detect
>>> and bring attention to changes in vmstate. This may be worth the effort
>>> if we adopt the proposed 4 year migration support cycle for pc-1.0,
>>> since that'll continue to rely on vmstate even after we move on to an
>>> IDL and newer protocol.
>>
>> Beyond ASL/IDL I like to be sure that we're not just translating one
>> format to other representation but instead we introduce some new
>> functionality like:
>>   - Ability to negotiate the protocol version
>>   - Bi-direction data exchange, the sender will send data as a function
>>     of the target release
>>     - Include the machine type too
>
> I've been toying with the notion of having the target start up a QMP
> limited server that the source talks to to orchestrate negotiation. We
> could potentially even send the device state by taking our QIDL-generated
> visitors and serializing state via QmpOutputVisitor. QMP can be made
> aware of the format of the device state input by taking the intermediate
> step of generating QAPI schemas via QIDL, and using the QAPI code
> generators to generate the visitors rather than QIDL directly. This
> would also address the protocol side: just use QMP rather than ASN.1..
>
> It's not as compact, but device state is such a small amount of data
> compared to memory/disk that I don't think it's worth optimizing that
> aspect, though we could use compression at the protocol layer if we
> were inclined. Anything more suited to an out-of-band protocol, like
> memory/disk, could be orchestrated via this interface... source can ask
> target for a port to handle such things, negotiate stuff like XBZRLE,
> etc.
>
>>   - Synchronize the entire qemu cmdline and don't relay on management to
>>     set it up.
>>     - Along the way, deal w/ hotplug events.
>
> My initial plan for the QIDL-generated visitors is to associate a QOM
> property, "state", with each device, and to serialize device state by
> walking the QOM composition tree, the main rationale being that if we
> extend that serialization to include other QOM properties, I believe we
> have everything we need to recreate all the devices on the target:
> parent->child relationships, types, properties set via cmdline,
> device state...
>
> A simpler alternative would be to leverage just send the cmdline
> options over to the target and assume that it results in the same underlying
> machine, then just send off the device state. Much simpler actually...but
> the above approach should work regardless of changes to the command-line
> options on the source... having an internally stable cmdline scheme
> might work as well...
Will command line take in account hot-plugged devices?

>
> I'm not sure what the right approach is here but whatever we decide on I think
> being able to automatically generate visitors from annotations is a good
> first-step and should tie into any forseeable approaches.
>
>>
>>>
>>>>
>>>> Paolo
>>>>
>>>
>>
>

-- 
-----
  Igor

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
  2012-04-04 13:21               ` Igor Mammedov
  (?)
@ 2012-04-04 14:39               ` Michael Roth
  2012-04-05 16:16                   ` Avi Kivity
  -1 siblings, 1 reply; 24+ messages in thread
From: Michael Roth @ 2012-04-04 14:39 UTC (permalink / raw)
  To: Igor Mammedov
  Cc: Paolo Bonzini, Dor Laor, qemu-devel, KVM devel mailing list,
	Markus Armbruster

On Wed, Apr 04, 2012 at 03:21:26PM +0200, Igor Mammedov wrote:
> On 04/04/2012 02:14 PM, Michael Roth wrote:
> >On Wed, Apr 04, 2012 at 01:53:34PM +0300, Dor Laor wrote:
> >>On 04/04/2012 01:37 PM, Michael Roth wrote:
> >>>
> >>>On Apr 4, 2012 2:42 AM, "Paolo Bonzini"<pbonzini@redhat.com
> >>><mailto:pbonzini@redhat.com>>  wrote:
> >>>>
> >>>>Il 04/04/2012 03:18, Michael Roth ha scritto:
> >>>>>Attacking the IDL/schema side first is the more rationale approach.
> >>>From
> >>>>>there we can potentially generate ASN.1 BER/DER visitors for the
> >>>protocol
> >>>>>side, or potentially even just vmstate bindings as a start. I've
> >>>recently
> >>>>>started looking into the latter... it's completely feasible, the only
> >>>>>downside is it complicates the IDL due requiring support for a lot of
> >>>>>what are very much vmstate-specific items, but it should be possible to
> >>>>>do this in a manner where those annotations are self-contained and
> >>>>>ignorable if we opted to replace vmstate-style declarations.
> >>>>
> >>>>We can also keep the current vmstate descriptions, but access fields
> >>>>from the automatically-generated visitors instead of struct fields.
> >>>>This keeps the IDL simple.
> >>>
> >>>It may be worthwhile as an incremental step though, one nice thing about
> >>>automatically generated bindings is that with the QIDL Anthony
> >>>prototyped a while back we assume we serialize by default, so changes in
> >>>annotated structs automatically trigger changes in the generated
> >>>bindings unless you explicitly mark fields as immutable/derivable/etc,
> >>>which we can tie into the build or make check to automatically detect
> >>>and bring attention to changes in vmstate. This may be worth the effort
> >>>if we adopt the proposed 4 year migration support cycle for pc-1.0,
> >>>since that'll continue to rely on vmstate even after we move on to an
> >>>IDL and newer protocol.
> >>
> >>Beyond ASL/IDL I like to be sure that we're not just translating one
> >>format to other representation but instead we introduce some new
> >>functionality like:
> >>  - Ability to negotiate the protocol version
> >>  - Bi-direction data exchange, the sender will send data as a function
> >>    of the target release
> >>    - Include the machine type too
> >
> >I've been toying with the notion of having the target start up a QMP
> >limited server that the source talks to to orchestrate negotiation. We
> >could potentially even send the device state by taking our QIDL-generated
> >visitors and serializing state via QmpOutputVisitor. QMP can be made
> >aware of the format of the device state input by taking the intermediate
> >step of generating QAPI schemas via QIDL, and using the QAPI code
> >generators to generate the visitors rather than QIDL directly. This
> >would also address the protocol side: just use QMP rather than ASN.1..
> >
> >It's not as compact, but device state is such a small amount of data
> >compared to memory/disk that I don't think it's worth optimizing that
> >aspect, though we could use compression at the protocol layer if we
> >were inclined. Anything more suited to an out-of-band protocol, like
> >memory/disk, could be orchestrated via this interface... source can ask
> >target for a port to handle such things, negotiate stuff like XBZRLE,
> >etc.
> >
> >>  - Synchronize the entire qemu cmdline and don't relay on management to
> >>    set it up.
> >>    - Along the way, deal w/ hotplug events.
> >
> >My initial plan for the QIDL-generated visitors is to associate a QOM
> >property, "state", with each device, and to serialize device state by
> >walking the QOM composition tree, the main rationale being that if we
> >extend that serialization to include other QOM properties, I believe we
> >have everything we need to recreate all the devices on the target:
> >parent->child relationships, types, properties set via cmdline,
> >device state...
> >
> >A simpler alternative would be to leverage just send the cmdline
> >options over to the target and assume that it results in the same underlying
> >machine, then just send off the device state. Much simpler actually...but
> >the above approach should work regardless of changes to the command-line
> >options on the source... having an internally stable cmdline scheme
> >might work as well...
> Will command line take in account hot-plugged devices?

No, that's a good point. We'd probably need to generate the options
required to ensure the devices are created on the target, and we'd only
be able to do that just before sending the device state. That means we
need a way to create machines after we've completed tasks like memory
migration, which probably has similar requirements to just being able to
instantiate a machine from a serialized QOM composition tree.

> 
> >
> >I'm not sure what the right approach is here but whatever we decide on I think
> >being able to automatically generate visitors from annotations is a good
> >first-step and should tie into any forseeable approaches.
> >
> >>
> >>>
> >>>>
> >>>>Paolo
> >>>>
> >>>
> >>
> >
> 
> -- 
> -----
>  Igor
> 

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
  2012-04-04 14:39               ` Michael Roth
@ 2012-04-05 16:16                   ` Avi Kivity
  0 siblings, 0 replies; 24+ messages in thread
From: Avi Kivity @ 2012-04-05 16:16 UTC (permalink / raw)
  To: Michael Roth
  Cc: Igor Mammedov, Paolo Bonzini, Dor Laor, qemu-devel,
	KVM devel mailing list, Markus Armbruster

On 04/04/2012 05:39 PM, Michael Roth wrote:
> > Will command line take in account hot-plugged devices?
>
> No, that's a good point. We'd probably need to generate the options
> required to ensure the devices are created on the target, and we'd only
> be able to do that just before sending the device state. That means we
> need a way to create machines after we've completed tasks like memory
> migration, which probably has similar requirements to just being able to
> instantiate a machine from a serialized QOM composition tree.
>

Or we forward the hotplug events over the migration protocol.  This is
needed for devices that include memory (like ivshmem.c)

<page>
<page>
<page>
<hotplug event>
<page>
<page for the new device>
<page>

-- 
error compiling committee.c: too many arguments to function


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Qemu-devel] KVM call minutes April 3
@ 2012-04-05 16:16                   ` Avi Kivity
  0 siblings, 0 replies; 24+ messages in thread
From: Avi Kivity @ 2012-04-05 16:16 UTC (permalink / raw)
  To: Michael Roth
  Cc: KVM devel mailing list, Dor Laor, Markus Armbruster, qemu-devel,
	Paolo Bonzini, Igor Mammedov

On 04/04/2012 05:39 PM, Michael Roth wrote:
> > Will command line take in account hot-plugged devices?
>
> No, that's a good point. We'd probably need to generate the options
> required to ensure the devices are created on the target, and we'd only
> be able to do that just before sending the device state. That means we
> need a way to create machines after we've completed tasks like memory
> migration, which probably has similar requirements to just being able to
> instantiate a machine from a serialized QOM composition tree.
>

Or we forward the hotplug events over the migration protocol.  This is
needed for devices that include memory (like ivshmem.c)

<page>
<page>
<page>
<hotplug event>
<page>
<page for the new device>
<page>

-- 
error compiling committee.c: too many arguments to function

^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2012-04-05 16:17 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-04-03 14:43 KVM call minutes April 3 Markus Armbruster
2012-04-03 14:43 ` [Qemu-devel] " Markus Armbruster
2012-04-03 20:43 ` Dor Laor
2012-04-04  1:18   ` Michael Roth
2012-04-04  1:18     ` Michael Roth
2012-04-04  7:40     ` Paolo Bonzini
2012-04-04  7:40       ` Paolo Bonzini
2012-04-04 10:37       ` Michael Roth
2012-04-04 10:37         ` [Qemu-devel] " Michael Roth
2012-04-04 10:53         ` Dor Laor
2012-04-04 10:53           ` Dor Laor
2012-04-04 11:52           ` Anthony Liguori
2012-04-04 11:52             ` Anthony Liguori
2012-04-04 12:01             ` Dor Laor
2012-04-04 12:01               ` Dor Laor
2012-04-04 12:14           ` Michael Roth
2012-04-04 12:14             ` Michael Roth
2012-04-04 13:21             ` Igor Mammedov
2012-04-04 13:21               ` Igor Mammedov
2012-04-04 14:39               ` Michael Roth
2012-04-05 16:16                 ` Avi Kivity
2012-04-05 16:16                   ` Avi Kivity
2012-04-04 11:48   ` Anthony Liguori
2012-04-04 11:48     ` Anthony Liguori

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.