From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
To: Markus Armbruster <armbru@redhat.com>
Cc: "Laurent Vivier" <lvivier@redhat.com>,
"Thomas Huth" <thuth@redhat.com>,
"Daniel P. Berrangé" <berrange@redhat.com>,
"Eduardo Habkost" <ehabkost@redhat.com>,
"Juan Quintela" <quintela@redhat.com>,
qemu-devel@nongnu.org, "Paolo Bonzini" <pbonzini@redhat.com>
Subject: Re: [PATCH v5 7/8] multifd: Add multifd-zstd-level parameter
Date: Fri, 14 Feb 2020 18:50:53 +0000 [thread overview]
Message-ID: <20200214185053.GN3283@work-vm> (raw)
In-Reply-To: <87imk91rkx.fsf@dusky.pond.sub.org>
* Markus Armbruster (armbru@redhat.com) wrote:
> Juan Quintela <quintela@redhat.com> writes:
>
> > Markus Armbruster <armbru@redhat.com> wrote:
> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> >>
> >>> * Juan Quintela (quintela@redhat.com) wrote:
> >>>> Signed-off-by: Juan Quintela <quintela@redhat.com>
> >>>> ---
> >>>> migration/migration.c | 15 +++++++++++++++
> >>>> monitor/hmp-cmds.c | 4 ++++
> >>>> qapi/migration.json | 29 ++++++++++++++++++++++++++---
> >>>> 3 files changed, 45 insertions(+), 3 deletions(-)
> >>>>
> >>>> diff --git a/migration/migration.c b/migration/migration.c
> >>>> index 3b081e8147..b690500545 100644
> >>>> --- a/migration/migration.c
> >>>> +++ b/migration/migration.c
> >>>> @@ -91,6 +91,8 @@
> >>>> #define DEFAULT_MIGRATE_MULTIFD_METHOD MULTIFD_METHOD_NONE
> >>>> /*0: means nocompress, 1: best speed, ... 9: best compress ratio */
> >>>> #define DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL 1
> >>>> +/* 0: means nocompress, 1: best speed, ... 20: best compress ratio */
> >>>> +#define DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL 1
> >>>>
> >>>> /* Background transfer rate for postcopy, 0 means unlimited, note
> >>>> * that page requests can still exceed this limit.
> >>>> @@ -805,6 +807,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp)
> >>>> params->multifd_method = s->parameters.multifd_method;
> >>>> params->has_multifd_zlib_level = true;
> >>>> params->multifd_zlib_level = s->parameters.multifd_zlib_level;
> >>>> + params->has_multifd_zstd_level = true;
> >>>> + params->multifd_zstd_level = s->parameters.multifd_zstd_level;
> >>>
> >>> Do we really want different 'multifd_...._level's or just one
> >>> 'multifd_compress_level' - or even just reuse the existing
> >>> 'compress-level' parameter.
> >>
> >> compress-level,
> >
> > possible values: 1 to 9 deprecated
> >
> >> multifd-zlib-level
> >
> > possible values: 1 to 9, default 1
> > (zlib default is -1, let the lib decide)
> >
> > , and multifd-zstd-level apply
> >
> > possible values: 1 to 19, default 1
> > (zstd default is 3)
> >
> >> "normal" live migration compression, multifd zlib live migration
> >> compression, and multifd zstd live migration compression, respectively.
> >>
> >> Any live migration can only use one of the three compressions.
> >>
> >> Correct?
> >
> > Yeap.
> >
> >>> The only tricky thing about combining them is how to handle
> >>> the difference in allowed ranges; When would the right time be
> >>> to check it?
> >>>
> >>> Markus/Eric: Any idea?
> >>
> >> To have an informed opinion, I'd have to dig through the migration
> >> code.
> >
> > Problem is _how_ to setup them. if we setup zstd compression method,
> > put the value at 19, and then setup zlib compression method, what should
> > we do?
> >
> > Truncate to 9?
> > Give one error?
> > Don't allow the zlib setup?
> >
> > Too complicated.
>
> The interface pretends the parameters are all independent: you get to
> set them one by one.
>
> They are in fact not independent, and this now leads to difficulties.
>
> To avoid them, the interface should let you specify a desired
> configuration all at once, and its implementation should then do what it
> takes to get from here to there.
>
> Example: current state is multifd compression method "zstd", compression
> level is 19. User wants to switch to method "zlib" level 9 the obvious
> way
>
> With old interface:
> step 1: set method to "zlib"
> step 2: set level to 9
> Problem: after step 1, we have method "zlib" with invalid level 19.
> Workaround: swap the steps.
>
> Note that the workaround only works because the set of levels both
> methods support is non-empty. We might still come up with more
> complicated parameter combinations where that is not the case.
>
> With new interface:
> set compression to "zlib" level 9
>
> The new interface require us to specify a QAPI type capable of holding
> the complete migration configuration.
>
> The stupid way is to throw all migration parameters into a struct, and
> make the ones optional that apply only when others have certain values.
> Thus, the "applies only when" bits are semantical, documented in
> comments, and enforced by C code.
I realised we already have that stupid struct! It's just
MigrationParameters - it has all the parameters as optional values,
and QMP's MigrateSetParameters already takes that - so you can already
provide both the type and the level at the same time; although there's
no semantic correlation between them.
Dave
> With a bit more care, we can make "applies only when" syntactical
> instead. Examples:
>
> @max-bandwidth and @downtime-limit always apply. They go straight
> into the struct.
>
> @tls-creds, @tls-hostname, @tls-authz apply only when TLS is enabled
> by setting @tls-creds. Have an optional member @tls, which is a
> struct with mandatory member @creds, optional members @hostname,
> @authz.
>
> @multifd-zlib-level applies when @multifd-method is "zlib", and
> @multifd-zstd-level applies when it's "zstd". Have a union
> @multifd-compression, cases "none", "zlib" and "zstd", where each
> case's members are the parameters applying in that case.
>
> Please note the purpose of these examples is to show how things can be
> done in the schema. I'm not trying to tell you how these specific
> things *should* be done.
>
> The resulting type is perfectly suited as return value of a query
> command. It's awkward as argument of a "specify desired configuration"
> command, because it requires the user to specify *complete*
> configuration. If we want to support omitting the parts of it we don't
> want to change, we have to make more members optional. Imprecise for
> the query, where we now have to specify "always present" in comments.
> Usually less bad than duplicating a complex type.
>
> >> Documentation of admissible range will become a bit awkward, too.
> >>
> >> Too many migration parameters...
> >
> > Sure, but the other option is taking a value and live with it.
> > I am all for leaving the library default and call it a day.
> >
> > Later, Juan.
>
> Hope this helps some.
>
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
next prev parent reply other threads:[~2020-02-14 18:52 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-29 11:56 [PATCH v5 0/8] Multifd Migration Compression Juan Quintela
2020-01-29 11:56 ` [PATCH v5 1/8] multifd: Add multifd-method parameter Juan Quintela
2020-01-30 7:54 ` Markus Armbruster
2020-01-30 9:11 ` Juan Quintela
2020-01-30 12:17 ` Markus Armbruster
2020-02-11 18:50 ` Daniel P. Berrangé
2020-02-13 19:29 ` Juan Quintela
2020-01-29 11:56 ` [PATCH v5 2/8] migration: Add support for modules Juan Quintela
2020-02-11 10:54 ` Dr. David Alan Gilbert
2020-02-13 19:38 ` Juan Quintela
2020-01-29 11:56 ` [PATCH v5 3/8] multifd: Make no compression operations into its own structure Juan Quintela
2020-02-07 18:45 ` Dr. David Alan Gilbert
2020-02-11 11:23 ` Juan Quintela
2020-01-29 11:56 ` [PATCH v5 4/8] multifd: Add multifd-zlib-level parameter Juan Quintela
2020-01-30 8:03 ` Markus Armbruster
2020-01-30 8:56 ` Juan Quintela
2020-02-11 18:57 ` Daniel P. Berrangé
2020-02-13 13:27 ` Markus Armbruster
2020-02-13 16:33 ` Juan Quintela
2020-01-29 11:56 ` [PATCH v5 5/8] multifd: Add zlib compression multifd support Juan Quintela
2020-01-30 8:04 ` Markus Armbruster
2020-02-11 18:43 ` Dr. David Alan Gilbert
2020-02-13 20:24 ` Juan Quintela
2020-01-29 11:56 ` [PATCH v5 6/8] configure: Enable test and libs for zstd Juan Quintela
2020-02-11 20:11 ` Daniel P. Berrangé
2020-02-13 21:08 ` Juan Quintela
2020-02-14 10:26 ` Daniel P. Berrangé
2020-01-29 11:56 ` [PATCH v5 7/8] multifd: Add multifd-zstd-level parameter Juan Quintela
2020-01-30 8:08 ` Markus Armbruster
2020-02-11 18:47 ` Dr. David Alan Gilbert
2020-02-13 14:04 ` Markus Armbruster
2020-02-13 14:28 ` Dr. David Alan Gilbert
2020-02-13 15:33 ` Juan Quintela
2020-02-14 8:49 ` Markus Armbruster
2020-02-14 18:50 ` Dr. David Alan Gilbert [this message]
2020-01-29 11:56 ` [PATCH v5 8/8] multifd: Add zstd compression multifd support Juan Quintela
2020-01-30 8:08 ` Markus Armbruster
2020-02-11 20:01 ` Dr. David Alan Gilbert
2020-02-13 20:39 ` Juan Quintela
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200214185053.GN3283@work-vm \
--to=dgilbert@redhat.com \
--cc=armbru@redhat.com \
--cc=berrange@redhat.com \
--cc=ehabkost@redhat.com \
--cc=lvivier@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=thuth@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).