All of lore.kernel.org
 help / color / mirror / Atom feed
* proposal: deprecate -readconfig/-writeconfig
@ 2020-05-14  8:09 Paolo Bonzini
  2020-05-14  8:54 ` Gerd Hoffmann
  2020-05-14  8:56 ` Daniel P. Berrangé
  0 siblings, 2 replies; 14+ messages in thread
From: Paolo Bonzini @ 2020-05-14  8:09 UTC (permalink / raw)
  To: qemu-devel, Markus Armbruster, Peter Maydell, John Snow

IMHO configuration files are in general a failed experiment.  In
practice, they do not add much value over just a shell script because
they don't allow configuring all QEMU options, they are very much fixed
(by their nature).  I think it's more or less agreed that they are not
solving any problem for higher-level management stacks as well; those
would prefer to configure the VM via QMP or another API.

So, any objections to deprecating -readconfig and -writeconfig?

Thanks,

Paolo



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: proposal: deprecate -readconfig/-writeconfig
  2020-05-14  8:09 proposal: deprecate -readconfig/-writeconfig Paolo Bonzini
@ 2020-05-14  8:54 ` Gerd Hoffmann
  2020-05-15  5:31   ` Markus Armbruster
  2020-05-14  8:56 ` Daniel P. Berrangé
  1 sibling, 1 reply; 14+ messages in thread
From: Gerd Hoffmann @ 2020-05-14  8:54 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: Peter Maydell, John Snow, qemu-devel, Markus Armbruster

On Thu, May 14, 2020 at 10:09:21AM +0200, Paolo Bonzini wrote:
> IMHO configuration files are in general a failed experiment.  In
> practice, they do not add much value over just a shell script because
> they don't allow configuring all QEMU options, they are very much fixed
> (by their nature).  I think it's more or less agreed that they are not
> solving any problem for higher-level management stacks as well; those
> would prefer to configure the VM via QMP or another API.
> 
> So, any objections to deprecating -readconfig and -writeconfig?

-writeconfig surely can go away, it never reached the point where it
could write out an configuration which is actually complete.

-readconfig is a bit more tricky, it's actually useful.  I'm using it
sometimes.  Also we have docs/config/ with a bunch of files you can
pass to -readconfig.

I can see that it'll stand in the way if we want move away from QemuOpts
to something else (say qom-based yaml/json config files), so I wouldn't
veto deprecation, but I'd prefer it not being actually dropped until the
replacement is ready and the stuff in docs/config/ being converted to
the new scheme.

my two cents,
  Gerd



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: proposal: deprecate -readconfig/-writeconfig
  2020-05-14  8:09 proposal: deprecate -readconfig/-writeconfig Paolo Bonzini
  2020-05-14  8:54 ` Gerd Hoffmann
@ 2020-05-14  8:56 ` Daniel P. Berrangé
  2020-05-14 12:37   ` Paolo Bonzini
  2020-05-14 14:40   ` John Snow
  1 sibling, 2 replies; 14+ messages in thread
From: Daniel P. Berrangé @ 2020-05-14  8:56 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: Peter Maydell, John Snow, qemu-devel, Markus Armbruster

On Thu, May 14, 2020 at 10:09:21AM +0200, Paolo Bonzini wrote:
> IMHO configuration files are in general a failed experiment.  In
> practice, they do not add much value over just a shell script because
> they don't allow configuring all QEMU options, they are very much fixed
> (by their nature).  I think it's more or less agreed that they are not
> solving any problem for higher-level management stacks as well; those
> would prefer to configure the VM via QMP or another API.
>
> So, any objections to deprecating -readconfig and -writeconfig?

Libvirt would like to have a config file for QEMU, but it would have
to be one that actually covers all the config options QEMU supports,
and ideally using a data format in common with that used for runtime
changes. So for libvirt's needs the current readconfig is entirely
useless.

For a less general purpose mgmt app, that targets some specific use
cases I could imagine people might have used readconfig. Note that
we have a bunch of documentation that is illustrating usage of
-readconfig to our users. So it is quite possible we have people
relying on this feature even though it is incomplete in its coverage
of options.

If we deprecate them, the only alternative users have right now is
to go back to passing CLI args. This works, as this is what libvirt
has always done, but it isn't pretty to see 1 MB command lines ;-P

So essentially we'd be deciding to kill the feature with no direct
replacement, even though it is potentially useful in some limited
scenarios.

If we have a general strategy to eliminate QemuOpts and move entirely
to QAPI based config, then I can see -readcofig/-writeconfig may be
creating a burden of back compatibility on maintainers.

This could justify us removing the feature with no immediate replacement,
on the basis that would facilitate more important changes that are for
the greater good of the project long term.

Overall, I don't object, just cautioning that we should be aware that
we're likely to have some users of this feature we're conciously going
to break.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: proposal: deprecate -readconfig/-writeconfig
  2020-05-14  8:56 ` Daniel P. Berrangé
@ 2020-05-14 12:37   ` Paolo Bonzini
  2020-05-15  5:54     ` Markus Armbruster
  2020-05-14 14:40   ` John Snow
  1 sibling, 1 reply; 14+ messages in thread
From: Paolo Bonzini @ 2020-05-14 12:37 UTC (permalink / raw)
  To: Daniel P. Berrangé
  Cc: Peter Maydell, John Snow, qemu-devel, Markus Armbruster

On 14/05/20 10:56, Daniel P. Berrangé wrote:
> On Thu, May 14, 2020 at 10:09:21AM +0200, Paolo Bonzini wrote:
>> IMHO configuration files are in general a failed experiment.  In
>> practice, they do not add much value over just a shell script because
>> they don't allow configuring all QEMU options, they are very much fixed
>> (by their nature).  I think it's more or less agreed that they are not
>> solving any problem for higher-level management stacks as well; those
>> would prefer to configure the VM via QMP or another API.
>>
>> So, any objections to deprecating -readconfig and -writeconfig?
> 
> Libvirt would like to have a config file for QEMU, but it would have
> to be one that actually covers all the config options QEMU supports,
> and ideally using a data format in common with that used for runtime
> changes. So for libvirt's needs the current readconfig is entirely
> useless.

Yes, this is what I was thinking about.

> For a less general purpose mgmt app, that targets some specific use
> cases I could imagine people might have used readconfig. [...]
> If we deprecate them, the only alternative users have right now is
> to go back to passing CLI args. [...] We'd be deciding to kill the
> feature with no direct replacement, even though it is potentially
> useful in some limited scenarios.
> 
> If we have a general strategy to eliminate QemuOpts and move entirely
> to QAPI based config, then I can see -readcofig/-writeconfig may be
> creating a burden of back compatibility on maintainers.

I don't see QemuOpts going away anytime soon, but I do see more QMP/QAPI
and less command line in the future as far as management tools are
concerned.  QemuOpts and HMP will remain for direct usage, for the
foreseeable future.

So I guess we can keep -readconfig but deprecate, or even remove,
-writeconfig.

Paolo



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: proposal: deprecate -readconfig/-writeconfig
  2020-05-14  8:56 ` Daniel P. Berrangé
  2020-05-14 12:37   ` Paolo Bonzini
@ 2020-05-14 14:40   ` John Snow
  2020-05-14 15:34     ` Daniel P. Berrangé
  2020-05-15  9:06     ` Gerd Hoffmann
  1 sibling, 2 replies; 14+ messages in thread
From: John Snow @ 2020-05-14 14:40 UTC (permalink / raw)
  To: Daniel P. Berrangé, Paolo Bonzini
  Cc: Peter Maydell, qemu-devel, Markus Armbruster



On 5/14/20 4:56 AM, Daniel P. Berrangé wrote:
> On Thu, May 14, 2020 at 10:09:21AM +0200, Paolo Bonzini wrote:
>> IMHO configuration files are in general a failed experiment.  In
>> practice, they do not add much value over just a shell script because
>> they don't allow configuring all QEMU options, they are very much fixed
>> (by their nature).  I think it's more or less agreed that they are not
>> solving any problem for higher-level management stacks as well; those
>> would prefer to configure the VM via QMP or another API.
>>
>> So, any objections to deprecating -readconfig and -writeconfig?
> 
> Libvirt would like to have a config file for QEMU, but it would have
> to be one that actually covers all the config options QEMU supports,
> and ideally using a data format in common with that used for runtime
> changes. So for libvirt's needs the current readconfig is entirely
> useless.
> 

Yeah. In this sense, would a json/yaml config file help, under the
premise that you could just cat it into the pipe to configure a machine?

(Assuming we had proper runtime configuration commands, of course.)

> For a less general purpose mgmt app, that targets some specific use
> cases I could imagine people might have used readconfig. Note that
> we have a bunch of documentation that is illustrating usage of
> -readconfig to our users. So it is quite possible we have people
> relying on this feature even though it is incomplete in its coverage
> of options.
> 
> If we deprecate them, the only alternative users have right now is
> to go back to passing CLI args. This works, as this is what libvirt
> has always done, but it isn't pretty to see 1 MB command lines ;-P
> 

Can always write a shim that reads the options from a text file. *shrug*

It still clutters the process list, but there's not much we can do right
now.

> So essentially we'd be deciding to kill the feature with no direct
> replacement, even though it is potentially useful in some limited
> scenarios.
> 
> If we have a general strategy to eliminate QemuOpts and move entirely
> to QAPI based config, then I can see -readcofig/-writeconfig may be
> creating a burden of back compatibility on maintainers.
> 
> This could justify us removing the feature with no immediate replacement,
> on the basis that would facilitate more important changes that are for
> the greater good of the project long term.
> 
> Overall, I don't object, just cautioning that we should be aware that
> we're likely to have some users of this feature we're conciously going
> to break.
> 
> Regards,
> Daniel
> 

Sometimes I feel like a broken and impartial solution is really worse
than having none. If we don't truly support the read/write config
options, we shouldn't pretend that we do.

Funneling users back to using the CLI is likely the better thing, even
with no replacement.

I realize this is a pretty hostile thing to do in general, though, but
it might truly be the kinder option to start simplifying and unifying
configuration, documentation, and support efforts.

We don't have to actually remove it right away, either. We can just
start sounding the alarms that we're preparing to remove it, and falling
back to using the CLI would be a safe thing to do for now.

--js



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: proposal: deprecate -readconfig/-writeconfig
  2020-05-14 14:40   ` John Snow
@ 2020-05-14 15:34     ` Daniel P. Berrangé
  2020-05-14 15:51       ` Paolo Bonzini
  2020-05-15  5:51       ` Markus Armbruster
  2020-05-15  9:06     ` Gerd Hoffmann
  1 sibling, 2 replies; 14+ messages in thread
From: Daniel P. Berrangé @ 2020-05-14 15:34 UTC (permalink / raw)
  To: John Snow; +Cc: Peter Maydell, Paolo Bonzini, qemu-devel, Markus Armbruster

On Thu, May 14, 2020 at 10:40:40AM -0400, John Snow wrote:
> 
> 
> On 5/14/20 4:56 AM, Daniel P. Berrangé wrote:
> > On Thu, May 14, 2020 at 10:09:21AM +0200, Paolo Bonzini wrote:
> >> IMHO configuration files are in general a failed experiment.  In
> >> practice, they do not add much value over just a shell script because
> >> they don't allow configuring all QEMU options, they are very much fixed
> >> (by their nature).  I think it's more or less agreed that they are not
> >> solving any problem for higher-level management stacks as well; those
> >> would prefer to configure the VM via QMP or another API.
> >>
> >> So, any objections to deprecating -readconfig and -writeconfig?
> > 
> > Libvirt would like to have a config file for QEMU, but it would have
> > to be one that actually covers all the config options QEMU supports,
> > and ideally using a data format in common with that used for runtime
> > changes. So for libvirt's needs the current readconfig is entirely
> > useless.
> > 
> 
> Yeah. In this sense, would a json/yaml config file help, under the
> premise that you could just cat it into the pipe to configure a machine?
> 
> (Assuming we had proper runtime configuration commands, of course.)

Yeah, the key thing is that you really want to be able to provide the
whole initial config in one go as an atomic action. I don't want to
issue 100 individual QMP commands to load each initial device. With
that in mind it doesn't really matter whether you do

  $ qemu -qmp stdio
  (qmp) load-config /some/file.yaml

vs

  $ qemu -qmp stdio -load-config /some/file.yaml

though the latter is probably slightly nicer as it lets you grep
for the VM based on the filename visible in the CLI. 

> > Overall, I don't object, just cautioning that we should be aware that
> > we're likely to have some users of this feature we're conciously going
> > to break.
> 
> Sometimes I feel like a broken and impartial solution is really worse
> than having none. If we don't truly support the read/write config
> options, we shouldn't pretend that we do.
> 
> Funneling users back to using the CLI is likely the better thing, even
> with no replacement.
> 
> I realize this is a pretty hostile thing to do in general, though, but
> it might truly be the kinder option to start simplifying and unifying
> configuration, documentation, and support efforts.
>
> We don't have to actually remove it right away, either. We can just
> start sounding the alarms that we're preparing to remove it, and falling
> back to using the CLI would be a safe thing to do for now.

My gut feeling is that we need to do something a bit more adventurous
if we're to get QEMU to move forward in solving the 10+ year old problem
of a formally described, consistent configuration syntax.

Just deprecating & removing -readconfig is just one small mole hill out
of 100 so doesn't move things along much on its own.


I tend to think we'd be better served by focusing on introducing a new set
of binaries,  qemu-vm-$TARGET, which exclusively use a new config syntax,
free of any legacy baggage present in qemu-system-$TARGET.

Work on qemu-vm-$TARGET will involve refactoring, and that will certainly
risk causing bugs in qemu-system-$TARGET. The premise though is that this
risk is lower, than if we tried to retrofit a new config syuntax directly
into qemu-system-$TARGET.

In particular I think it is basically impossible to do any meaningful
changes in the main() method of softmmu/vl.c, due to the fragile ordering
for creation of various different devices/backends, and its interaction
with the argv parsing. By using a new qemu-vm-$TARGET we can have a new
softmmu/vm.c, instead of vl.c. We'll still likely need to modify parts of
vl.c to use new/changed API calls, but that's less distruptive, as we
can leave the crazy logic for ordering of device creation untouched.


In the past we've been fairly cautious of any change which might risk
back compat problems in qemu-system-$TARGET. I think we can say we're not
going to gratuitously break qemu-system-$TARGET, but at the same time we
are not going to make heroic efforts at (unachievable) perfection.

IOW, accept that we'll probably cause regressions in some edge cases in
qemu-system-$TARGET. We'll make reasonable effort to fix them if they
are reported, but we're willing to accept the risk of disruption in the
short term, in order to get to a better long term solution.

Depending on how much of a burden it is to keep qemu-system-$TARGET around
long term, it could exist forever without any new feature work on its CLI;
Or it could be turned into a shim around qemu-vm-$TARGET; Or it could
be deprecated over a prolonged period (as opposed to the usual 2 release
minimum).

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: proposal: deprecate -readconfig/-writeconfig
  2020-05-14 15:34     ` Daniel P. Berrangé
@ 2020-05-14 15:51       ` Paolo Bonzini
  2020-05-14 15:55         ` Daniel P. Berrangé
  2020-05-15  5:51       ` Markus Armbruster
  1 sibling, 1 reply; 14+ messages in thread
From: Paolo Bonzini @ 2020-05-14 15:51 UTC (permalink / raw)
  To: Daniel P. Berrangé, John Snow
  Cc: Peter Maydell, qemu-devel, Markus Armbruster

On 14/05/20 17:34, Daniel P. Berrangé wrote:
> Yeah, the key thing is that you really want to be able to provide the
> whole initial config in one go as an atomic action. I don't want to
> issue 100 individual QMP commands to load each initial device.

Why?  I think if we do something like the qemu-vm-$TARGET that you
propose below, there's absolutely no difference between the two.

Then you'd have a

qemu-run /some/file.yaml

(notice the lack of $TARGET) that takes care of starting the VM.

> I tend to think we'd be better served by focusing on introducing a new set
> of binaries,  qemu-vm-$TARGET, which exclusively use a new config syntax,
> free of any legacy baggage present in qemu-system-$TARGET.
> 
> Work on qemu-vm-$TARGET will involve refactoring, and that will certainly
> risk causing bugs in qemu-system-$TARGET. The premise though is that this
> risk is lower, than if we tried to retrofit a new config syuntax directly
> into qemu-system-$TARGET.
> 
> In particular I think it is basically impossible to do any meaningful
> changes in the main() method of softmmu/vl.c, due to the fragile ordering
> for creation of various different devices/backends, and its interaction
> with the argv parsing. By using a new qemu-vm-$TARGET we can have a new
> softmmu/vm.c, instead of vl.c. We'll still likely need to modify parts of
> vl.c to use new/changed API calls, but that's less distruptive, as we
> can leave the crazy logic for ordering of device creation untouched.

Agreed.

Paolo



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: proposal: deprecate -readconfig/-writeconfig
  2020-05-14 15:51       ` Paolo Bonzini
@ 2020-05-14 15:55         ` Daniel P. Berrangé
  2020-05-15  5:44           ` Markus Armbruster
  0 siblings, 1 reply; 14+ messages in thread
From: Daniel P. Berrangé @ 2020-05-14 15:55 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: Peter Maydell, John Snow, qemu-devel, Markus Armbruster

On Thu, May 14, 2020 at 05:51:02PM +0200, Paolo Bonzini wrote:
> On 14/05/20 17:34, Daniel P. Berrangé wrote:
> > Yeah, the key thing is that you really want to be able to provide the
> > whole initial config in one go as an atomic action. I don't want to
> > issue 100 individual QMP commands to load each initial device.
> 
> Why?  I think if we do something like the qemu-vm-$TARGET that you
> propose below, there's absolutely no difference between the two.

Ok, I should clarify. I don't want to do 100 individual serialized
round-trip request+reply, as that'd create latency on startup.
100 pipelined/parallelized request+reply would be ok, as you'll
not have the synchronization delay for each command.

Today the biggest cause of slow startup in libvirt, is issuing
something like 100+ serialized QMP calls to check status of
individual CPUID features. Possibly this is already just a libvirt
bug we can could just stuff all 100 qom-get queries down the pipe
in one go and have 1 wait for replies to arrive.

> Then you'd have a
> 
> qemu-run /some/file.yaml
> 
> (notice the lack of $TARGET) that takes care of starting the VM.
> 
> > I tend to think we'd be better served by focusing on introducing a new set
> > of binaries,  qemu-vm-$TARGET, which exclusively use a new config syntax,
> > free of any legacy baggage present in qemu-system-$TARGET.
> > 
> > Work on qemu-vm-$TARGET will involve refactoring, and that will certainly
> > risk causing bugs in qemu-system-$TARGET. The premise though is that this
> > risk is lower, than if we tried to retrofit a new config syuntax directly
> > into qemu-system-$TARGET.
> > 
> > In particular I think it is basically impossible to do any meaningful
> > changes in the main() method of softmmu/vl.c, due to the fragile ordering
> > for creation of various different devices/backends, and its interaction
> > with the argv parsing. By using a new qemu-vm-$TARGET we can have a new
> > softmmu/vm.c, instead of vl.c. We'll still likely need to modify parts of
> > vl.c to use new/changed API calls, but that's less distruptive, as we
> > can leave the crazy logic for ordering of device creation untouched.
> 
> Agreed.
> 
> Paolo
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: proposal: deprecate -readconfig/-writeconfig
  2020-05-14  8:54 ` Gerd Hoffmann
@ 2020-05-15  5:31   ` Markus Armbruster
  0 siblings, 0 replies; 14+ messages in thread
From: Markus Armbruster @ 2020-05-15  5:31 UTC (permalink / raw)
  To: Gerd Hoffmann; +Cc: Paolo Bonzini, John Snow, qemu-devel, Peter Maydell

Gerd Hoffmann <kraxel@redhat.com> writes:

> On Thu, May 14, 2020 at 10:09:21AM +0200, Paolo Bonzini wrote:
>> IMHO configuration files are in general a failed experiment.  In
>> practice, they do not add much value over just a shell script because
>> they don't allow configuring all QEMU options, they are very much fixed
>> (by their nature).  I think it's more or less agreed that they are not
>> solving any problem for higher-level management stacks as well; those
>> would prefer to configure the VM via QMP or another API.
>> 
>> So, any objections to deprecating -readconfig and -writeconfig?
>
> -writeconfig surely can go away, it never reached the point where it
> could write out an configuration which is actually complete.
>
> -readconfig is a bit more tricky, it's actually useful.  I'm using it
> sometimes.

I use it all the time.

>             Also we have docs/config/ with a bunch of files you can
> pass to -readconfig.

True.

> I can see that it'll stand in the way if we want move away from QemuOpts
> to something else (say qom-based yaml/json config files), so I wouldn't
> veto deprecation, but I'd prefer it not being actually dropped until the
> replacement is ready and the stuff in docs/config/ being converted to
> the new scheme.

I want QemuOpts replaced, and I want its replacement to support
configuration files.

The replacement effort needs some license for incompatible change.  The
less license, the harder the job becomes.

The configuration file format is among the things that'll change.

QemuOpts was a reasonable step forward back when you invented it, and it
served us well for some time, but we've outgrown it.  It's basically
two-level: configuration group ("drive", "chardev", "device", ...) and
option parameter (for "chardev": "backend", "path", "host", ...).  We
really need trees now.  I can't see how to grow the current INI-like
configuration file syntax compatibly to trees without making it overly
verbose.

Telling users now not to rely on the configuration file format to remain
compatible makes sense to me.



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: proposal: deprecate -readconfig/-writeconfig
  2020-05-14 15:55         ` Daniel P. Berrangé
@ 2020-05-15  5:44           ` Markus Armbruster
  0 siblings, 0 replies; 14+ messages in thread
From: Markus Armbruster @ 2020-05-15  5:44 UTC (permalink / raw)
  To: Daniel P. Berrangé
  Cc: Paolo Bonzini, John Snow, qemu-devel, Markus Armbruster, Peter Maydell

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Thu, May 14, 2020 at 05:51:02PM +0200, Paolo Bonzini wrote:
>> On 14/05/20 17:34, Daniel P. Berrangé wrote:
>> > Yeah, the key thing is that you really want to be able to provide the
>> > whole initial config in one go as an atomic action. I don't want to
>> > issue 100 individual QMP commands to load each initial device.
>> 
>> Why?  I think if we do something like the qemu-vm-$TARGET that you
>> propose below, there's absolutely no difference between the two.
>
> Ok, I should clarify. I don't want to do 100 individual serialized
> round-trip request+reply, as that'd create latency on startup.
> 100 pipelined/parallelized request+reply would be ok, as you'll
> not have the synchronization delay for each command.
>
> Today the biggest cause of slow startup in libvirt, is issuing
> something like 100+ serialized QMP calls to check status of
> individual CPUID features. Possibly this is already just a libvirt
> bug we can could just stuff all 100 qom-get queries down the pipe
> in one go and have 1 wait for replies to arrive.

Hundreds of serialized QMP roundtrips are certainly bad.

Sending hundreds of QMP commands before reading any QMP output would
also be bad: it relies on QEMU to buffer the output.  It does, but
unlimited buffering is bad, and you shouldn't rely on it to stay.

Instead, send your input as fast as QEMU accepts it (it *will* stop
reading if you send input faster than it can process it), and also
receive output as fast as you can, until you got it all.

[...]



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: proposal: deprecate -readconfig/-writeconfig
  2020-05-14 15:34     ` Daniel P. Berrangé
  2020-05-14 15:51       ` Paolo Bonzini
@ 2020-05-15  5:51       ` Markus Armbruster
  1 sibling, 0 replies; 14+ messages in thread
From: Markus Armbruster @ 2020-05-15  5:51 UTC (permalink / raw)
  To: Daniel P. Berrangé
  Cc: Peter Maydell, John Snow, qemu-devel, Markus Armbruster, Paolo Bonzini

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Thu, May 14, 2020 at 10:40:40AM -0400, John Snow wrote:
>> 
>> 
>> On 5/14/20 4:56 AM, Daniel P. Berrangé wrote:
>> > On Thu, May 14, 2020 at 10:09:21AM +0200, Paolo Bonzini wrote:
>> >> IMHO configuration files are in general a failed experiment.  In
>> >> practice, they do not add much value over just a shell script because
>> >> they don't allow configuring all QEMU options, they are very much fixed
>> >> (by their nature).  I think it's more or less agreed that they are not
>> >> solving any problem for higher-level management stacks as well; those
>> >> would prefer to configure the VM via QMP or another API.
>> >>
>> >> So, any objections to deprecating -readconfig and -writeconfig?
>> > 
>> > Libvirt would like to have a config file for QEMU, but it would have
>> > to be one that actually covers all the config options QEMU supports,
>> > and ideally using a data format in common with that used for runtime
>> > changes. So for libvirt's needs the current readconfig is entirely
>> > useless.
>> > 
>> 
>> Yeah. In this sense, would a json/yaml config file help, under the
>> premise that you could just cat it into the pipe to configure a machine?
>> 
>> (Assuming we had proper runtime configuration commands, of course.)
>
> Yeah, the key thing is that you really want to be able to provide the
> whole initial config in one go as an atomic action. I don't want to
> issue 100 individual QMP commands to load each initial device. With
> that in mind it doesn't really matter whether you do
>
>   $ qemu -qmp stdio
>   (qmp) load-config /some/file.yaml
>
> vs
>
>   $ qemu -qmp stdio -load-config /some/file.yaml
>
> though the latter is probably slightly nicer as it lets you grep
> for the VM based on the filename visible in the CLI. 

Once QEMU can do the former, providing the latter should be trivial.
Not providing it would be unadvisable.

Yes, a management application that talks QMP anyway can be expected to
be quite happy with having to do initial configuration via QMP, with or
without a load-config command.

But human users won't be happy.  Having to use a wrapper program to feed
their configuration file to QMP adds undue complexity.

[...]



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: proposal: deprecate -readconfig/-writeconfig
  2020-05-14 12:37   ` Paolo Bonzini
@ 2020-05-15  5:54     ` Markus Armbruster
  2020-05-15 10:19       ` Paolo Bonzini
  0 siblings, 1 reply; 14+ messages in thread
From: Markus Armbruster @ 2020-05-15  5:54 UTC (permalink / raw)
  To: Paolo Bonzini
  Cc: John Snow, Peter Maydell, Daniel P. Berrangé, qemu-devel

Paolo Bonzini <pbonzini@redhat.com> writes:

> On 14/05/20 10:56, Daniel P. Berrangé wrote:
>> On Thu, May 14, 2020 at 10:09:21AM +0200, Paolo Bonzini wrote:
>>> IMHO configuration files are in general a failed experiment.  In
>>> practice, they do not add much value over just a shell script because
>>> they don't allow configuring all QEMU options, they are very much fixed
>>> (by their nature).  I think it's more or less agreed that they are not
>>> solving any problem for higher-level management stacks as well; those
>>> would prefer to configure the VM via QMP or another API.
>>>
>>> So, any objections to deprecating -readconfig and -writeconfig?
>> 
>> Libvirt would like to have a config file for QEMU, but it would have
>> to be one that actually covers all the config options QEMU supports,
>> and ideally using a data format in common with that used for runtime
>> changes. So for libvirt's needs the current readconfig is entirely
>> useless.
>
> Yes, this is what I was thinking about.
>
>> For a less general purpose mgmt app, that targets some specific use
>> cases I could imagine people might have used readconfig. [...]
>> If we deprecate them, the only alternative users have right now is
>> to go back to passing CLI args. [...] We'd be deciding to kill the
>> feature with no direct replacement, even though it is potentially
>> useful in some limited scenarios.
>> 
>> If we have a general strategy to eliminate QemuOpts and move entirely
>> to QAPI based config, then I can see -readcofig/-writeconfig may be
>> creating a burden of back compatibility on maintainers.
>
> I don't see QemuOpts going away anytime soon, but I do see more QMP/QAPI
> and less command line in the future as far as management tools are
> concerned.  QemuOpts and HMP will remain for direct usage, for the
> foreseeable future.

I'd prefer not to have two separate configuration infrastructures.

> So I guess we can keep -readconfig but deprecate, or even remove,
> -writeconfig.

Deprecate both as stable interfaces, remove neither.



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: proposal: deprecate -readconfig/-writeconfig
  2020-05-14 14:40   ` John Snow
  2020-05-14 15:34     ` Daniel P. Berrangé
@ 2020-05-15  9:06     ` Gerd Hoffmann
  1 sibling, 0 replies; 14+ messages in thread
From: Gerd Hoffmann @ 2020-05-15  9:06 UTC (permalink / raw)
  To: John Snow
  Cc: Paolo Bonzini, Daniel P. Berrangé,
	qemu-devel, Markus Armbruster, Peter Maydell

  Hi,

> Sometimes I feel like a broken and impartial solution is really worse
> than having none. If we don't truly support the read/write config
> options, we shouldn't pretend that we do.
> 
> Funneling users back to using the CLI is likely the better thing, even
> with no replacement.
> 
> I realize this is a pretty hostile thing to do in general, though, but
> it might truly be the kinder option to start simplifying and unifying
> configuration, documentation, and support efforts.
> 
> We don't have to actually remove it right away, either.

Removing doesn't buy us much as long as both cli and -readconfig are
using QemuOpts as backend.  -readconfig itself isn't that much code
after all, it is just an alternative way to feed QemuOpts.

I see -readconfig getting into the way when moving from QemuOpts to
something else (with proper tree support) though.

> We can just start sounding the alarms that we're preparing to remove
> it, and falling back to using the CLI would be a safe thing to do for
> now.

Yes, we should start sounding the alarms (aka deprecate it).  Not sure
"fallback to cli" is a good recommendation though, I suspect that might
not stay compatible either.

take care,
  Gerd



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: proposal: deprecate -readconfig/-writeconfig
  2020-05-15  5:54     ` Markus Armbruster
@ 2020-05-15 10:19       ` Paolo Bonzini
  0 siblings, 0 replies; 14+ messages in thread
From: Paolo Bonzini @ 2020-05-15 10:19 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: John Snow, Peter Maydell, Daniel P. Berrangé, qemu-devel

On 15/05/20 07:54, Markus Armbruster wrote:
>> I don't see QemuOpts going away anytime soon, but I do see more QMP/QAPI
>> and less command line in the future as far as management tools are
>> concerned.  QemuOpts and HMP will remain for direct usage, for the
>> foreseeable future.
> 
> I'd prefer not to have two separate configuration infrastructures.

We already have two, since the introduction of preconfig state.  The
question is more how to unify the code that implements them, than how to
reduce them to one.

For example, I can imagine (and have even discussed informally with John
Snow) a patch series that starts by adding a new preconfig command:

{ 'execute': 'set-argv', 'arguments': { 'argv': [ '-m', '2048'] } }

and progressively adds more preconfig commands until Libvirt can
reasonably configure a VM without set-argv.

An example implementation would be to special case chardev-add,
object-add etc. to stash their arguments in a QAPI struct in preconfig
mode, and the same for -chardev and -object.  Then the big ugly setup
code in main() would walk these structs instead of QemuOpts.  Bingo: you
have a QAPI-representable machine config.  It may not be the nicest one
(see the problems you raised in the past about chardev-add not using the
latest and greatest QAPI features) but I would love to see a proof of
concept for this.

Thanks,

Paolo



^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2020-05-15 10:21 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-14  8:09 proposal: deprecate -readconfig/-writeconfig Paolo Bonzini
2020-05-14  8:54 ` Gerd Hoffmann
2020-05-15  5:31   ` Markus Armbruster
2020-05-14  8:56 ` Daniel P. Berrangé
2020-05-14 12:37   ` Paolo Bonzini
2020-05-15  5:54     ` Markus Armbruster
2020-05-15 10:19       ` Paolo Bonzini
2020-05-14 14:40   ` John Snow
2020-05-14 15:34     ` Daniel P. Berrangé
2020-05-14 15:51       ` Paolo Bonzini
2020-05-14 15:55         ` Daniel P. Berrangé
2020-05-15  5:44           ` Markus Armbruster
2020-05-15  5:51       ` Markus Armbruster
2020-05-15  9:06     ` Gerd Hoffmann

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.