Re: [Qemu-devel] [PATCH v5 23/45] migrate_start_postcopy: Command to trigger transition to postcopy

["Dr. David Alan Gilbert" <dgilbert@redhat.com>]

* David Gibson (david@gibson.dropbear.id.au) wrote:
> On Wed, Mar 18, 2015 at 05:59:51PM +0000, Dr. David Alan Gilbert wrote:
> > * David Gibson (david@gibson.dropbear.id.au) wrote:
> > > On Fri, Mar 13, 2015 at 11:19:06AM +0000, Dr. David Alan Gilbert wrote:
> > > > * David Gibson (david@gibson.dropbear.id.au) wrote:
> > > > > On Wed, Feb 25, 2015 at 04:51:46PM +0000, Dr. David Alan Gilbert (git) wrote:
> > > > > > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > > > > > 
> > > > > > Once postcopy is enabled (with migrate_set_capability), the migration
> > > > > > will still start on precopy mode.  To cause a transition into postcopy
> > > > > > the:
> > > > > > 
> > > > > >   migrate_start_postcopy
> > > > > > 
> > > > > > command must be issued.  Postcopy will start sometime after this
> > > > > > (when it's next checked in the migration loop).
> > > > > > 
> > > > > > Issuing the command before migration has started will error,
> > > > > > and issuing after it has finished is ignored.
> > > > > > 
> > > > > > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> > > > > > Reviewed-by: Eric Blake <eblake@redhat.com>
> > > > > > ---
> > > > > >  hmp-commands.hx               | 15 +++++++++++++++
> > > > > >  hmp.c                         |  7 +++++++
> > > > > >  hmp.h                         |  1 +
> > > > > >  include/migration/migration.h |  3 +++
> > > > > >  migration/migration.c         | 22 ++++++++++++++++++++++
> > > > > >  qapi-schema.json              |  8 ++++++++
> > > > > >  qmp-commands.hx               | 19 +++++++++++++++++++
> > > > > >  7 files changed, 75 insertions(+)
> > > > > > 
> > > > > > diff --git a/hmp-commands.hx b/hmp-commands.hx
> > > > > > index e37bc8b..03b8b78 100644
> > > > > > --- a/hmp-commands.hx
> > > > > > +++ b/hmp-commands.hx
> > > > > > @@ -985,6 +985,21 @@ Enable/Disable the usage of a capability @var{capability} for migration.
> > > > > >  ETEXI
> > > > > >  
> > > > > >      {
> > > > > > +        .name       = "migrate_start_postcopy",
> > > > > > +        .args_type  = "",
> > > > > > +        .params     = "",
> > > > > > +        .help       = "Switch migration to postcopy mode",
> > > > > > +        .mhandler.cmd = hmp_migrate_start_postcopy,
> > > > > > +    },
> > > > > > +
> > > > > > +STEXI
> > > > > > +@item migrate_start_postcopy
> > > > > > +@findex migrate_start_postcopy
> > > > > > +Switch in-progress migration to postcopy mode. Ignored after the end of
> > > > > > +migration (or once already in postcopy).
> > > > > > +ETEXI
> > > > > > +
> > > > > > +    {
> > > > > >          .name       = "client_migrate_info",
> > > > > >          .args_type  = "protocol:s,hostname:s,port:i?,tls-port:i?,cert-subject:s?",
> > > > > >          .params     = "protocol hostname port tls-port cert-subject",
> > > > > > diff --git a/hmp.c b/hmp.c
> > > > > > index b47f331..df9736c 100644
> > > > > > --- a/hmp.c
> > > > > > +++ b/hmp.c
> > > > > > @@ -1140,6 +1140,13 @@ void hmp_migrate_set_capability(Monitor *mon, const QDict *qdict)
> > > > > >      }
> > > > > >  }
> > > > > >  
> > > > > > +void hmp_migrate_start_postcopy(Monitor *mon, const QDict *qdict)
> > > > > > +{
> > > > > > +    Error *err = NULL;
> > > > > > +    qmp_migrate_start_postcopy(&err);
> > > > > > +    hmp_handle_error(mon, &err);
> > > > > > +}
> > > > > > +
> > > > > >  void hmp_set_password(Monitor *mon, const QDict *qdict)
> > > > > >  {
> > > > > >      const char *protocol  = qdict_get_str(qdict, "protocol");
> > > > > > diff --git a/hmp.h b/hmp.h
> > > > > > index 4bb5dca..da1334f 100644
> > > > > > --- a/hmp.h
> > > > > > +++ b/hmp.h
> > > > > > @@ -64,6 +64,7 @@ void hmp_migrate_set_downtime(Monitor *mon, const QDict *qdict);
> > > > > >  void hmp_migrate_set_speed(Monitor *mon, const QDict *qdict);
> > > > > >  void hmp_migrate_set_capability(Monitor *mon, const QDict *qdict);
> > > > > >  void hmp_migrate_set_cache_size(Monitor *mon, const QDict *qdict);
> > > > > > +void hmp_migrate_start_postcopy(Monitor *mon, const QDict *qdict);
> > > > > >  void hmp_set_password(Monitor *mon, const QDict *qdict);
> > > > > >  void hmp_expire_password(Monitor *mon, const QDict *qdict);
> > > > > >  void hmp_eject(Monitor *mon, const QDict *qdict);
> > > > > > diff --git a/include/migration/migration.h b/include/migration/migration.h
> > > > > > index e6a814a..293c83e 100644
> > > > > > --- a/include/migration/migration.h
> > > > > > +++ b/include/migration/migration.h
> > > > > > @@ -104,6 +104,9 @@ struct MigrationState
> > > > > >      int64_t xbzrle_cache_size;
> > > > > >      int64_t setup_time;
> > > > > >      int64_t dirty_sync_count;
> > > > > > +
> > > > > > +    /* Flag set once the migration has been asked to enter postcopy */
> > > > > > +    bool start_postcopy;
> > > > > >  };
> > > > > >  
> > > > > >  void process_incoming_migration(QEMUFile *f);
> > > > > > diff --git a/migration/migration.c b/migration/migration.c
> > > > > > index a4fc7d7..43ca656 100644
> > > > > > --- a/migration/migration.c
> > > > > > +++ b/migration/migration.c
> > > > > > @@ -372,6 +372,28 @@ void qmp_migrate_set_capabilities(MigrationCapabilityStatusList *params,
> > > > > >      }
> > > > > >  }
> > > > > >  
> > > > > > +void qmp_migrate_start_postcopy(Error **errp)
> > > > > > +{
> > > > > > +    MigrationState *s = migrate_get_current();
> > > > > > +
> > > > > > +    if (!migrate_postcopy_ram()) {
> > > > > > +        error_setg(errp, "Enable postcopy with migration_set_capability before"
> > > > > > +                         " the start of migration");
> > > > > > +        return;
> > > > > > +    }
> > > > > > +
> > > > > > +    if (s->state == MIG_STATE_NONE) {
> > > > > > +        error_setg(errp, "Postcopy must be started after migration has been"
> > > > > > +                         " started");
> > > > > > +        return;
> > > > > > +    }
> > > > > > +    /*
> > > > > > +     * we don't error if migration has finished since that would be racy
> > > > > > +     * with issuing this command.
> > > > > > +     */
> > > > > > +    atomic_set(&s->start_postcopy, true);
> > > > > 
> > > > > Why atomic_set?
> > > > 
> > > > It's being read by the migration thread, this is happening in the main thread.
> > > > 
> > > > There's no strict ordering requirement or anything.
> > > 
> > > I don't think you need an atomic then.  AFAIK an atomic_set() in
> > > isolation without some sort of atomic on the other side is pretty much
> > > meaningless.
> > 
> > The other side has an atomic_read:
> > 
> >                 if (migrate_postcopy_ram() &&
> >                     s->state != MIGRATION_STATUS_POSTCOPY_ACTIVE &&
> >                     pend_nonpost <= max_size &&
> >                     atomic_read(&s->start_postcopy)) {
> > 
> >                     if (!postcopy_start(s, &old_vm_running)) {
> >                         current_active_type = MIGRATION_STATUS_POSTCOPY_ACTIVE;
> >                         entered_postcopy = true;
> >                     }
> > 
> > so it is at least symmetric.
> 
> But still pointless.  Atomicity isn't magic pixie dust; it only makes
> sense if you're making atomic specific operations that need to be.
> Simple integer loads and stores are already atomic.  Unless at least
> some of the atomic operations are something more complex, there's
> really no point to atomically marked operations.

OK, I'll kill it off.

It'll work in practice, but I still believe that what you're saying isn't
safe C:
   1) There's no barrier after the write, so there's no guarantee the other
      thread will eventually see it (in practice we've got other pthread ops
      we take so we will get a barrier somewhere, and most CPUs eventually
      do propagate the store).

   2) The read side could legally be optimised out of the loop by the compiler.
      (but in practice wont be because compilers won't optimise that far).

Dave

> 
> -- 
> David Gibson			| I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
> 				| _way_ _around_!
> http://www.ozlabs.org/~dgibson


--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK