On Wed, Mar 13, 2019 at 12:48:57PM +0100, Cédric Le Goater wrote:
> On 2/25/19 3:53 AM, David Gibson wrote:
> > On Fri, Feb 22, 2019 at 12:28:33PM +0100, Cédric Le Goater wrote:
> >> When migration of a VM is initiated, a first copy of the RAM is
> >> transferred to the destination before the VM is stopped, but there is
> >> no guarantee that the EQ pages in which the event notification are
> >> queued have not been modified.
> >>
> >> To make sure migration will capture a consistent memory state, the
> >> XIVE device should perform a XIVE quiesce sequence to stop the flow of
> >> event notifications and stabilize the EQs. This is the purpose of the
> >> KVM_DEV_XIVE_EQ_SYNC control which will also marks the EQ pages dirty
> >> to force their transfer.
> >>
> >> Signed-off-by: Cédric Le Goater <clg@kaod.org>
> >> ---
> >>  arch/powerpc/include/uapi/asm/kvm.h        |  1 +
> >>  arch/powerpc/kvm/book3s_xive_native.c      | 67 ++++++++++++++++++++++
> >>  Documentation/virtual/kvm/devices/xive.txt | 29 ++++++++++
> >>  3 files changed, 97 insertions(+)
> >>
> >> diff --git a/arch/powerpc/include/uapi/asm/kvm.h b/arch/powerpc/include/uapi/asm/kvm.h
> >> index 289c504b7c1d..cd78ad1020fe 100644
> >> --- a/arch/powerpc/include/uapi/asm/kvm.h
> >> +++ b/arch/powerpc/include/uapi/asm/kvm.h
> >> @@ -678,6 +678,7 @@ struct kvm_ppc_cpu_char {
> >>  /* POWER9 XIVE Native Interrupt Controller */
> >>  #define KVM_DEV_XIVE_GRP_CTRL		1
> >>  #define   KVM_DEV_XIVE_RESET		1
> >> +#define   KVM_DEV_XIVE_EQ_SYNC		2
> >>  #define KVM_DEV_XIVE_GRP_SOURCE		2	/* 64-bit source attributes */
> >>  #define KVM_DEV_XIVE_GRP_SOURCE_CONFIG	3	/* 64-bit source attributes */
> >>  #define KVM_DEV_XIVE_GRP_EQ_CONFIG	4	/* 64-bit eq attributes */
> >> diff --git a/arch/powerpc/kvm/book3s_xive_native.c b/arch/powerpc/kvm/book3s_xive_native.c
> >> index dd2a9d411fe7..3debc876d5a0 100644
> >> --- a/arch/powerpc/kvm/book3s_xive_native.c
> >> +++ b/arch/powerpc/kvm/book3s_xive_native.c
> >> @@ -640,6 +640,70 @@ static int kvmppc_xive_reset(struct kvmppc_xive *xive)
> >>  	return 0;
> >>  }
> >>  
> >> +static void kvmppc_xive_native_sync_sources(struct kvmppc_xive_src_block *sb)
> >> +{
> >> +	int j;
> >> +
> >> +	for (j = 0; j < KVMPPC_XICS_IRQ_PER_ICS; j++) {
> >> +		struct kvmppc_xive_irq_state *state = &sb->irq_state[j];
> >> +		struct xive_irq_data *xd;
> >> +		u32 hw_num;
> >> +
> >> +		if (!state->valid)
> >> +			continue;
> >> +		if (state->act_priority == MASKED)
> > 
> > Is this correct?  If you masked an irq, then immediately did a sync,
> > couldn't there still be some of the irqs in flight?  I thought the
> > reason we needed a sync was that masking and other such operations
> > _didn't_ implicitly synchronize.
> 
> The struct kvmppc_xive_irq_state reflects the state of the EAS 
> configuration and not the state of the source. The source is masked 
> setting the PQ bits to '-Q', which is what is being done before calling 
> the KVM_DEV_XIVE_EQ_SYNC control. 
> 
> If a source EAS is configured, OPAL syncs the XIVE IC of the source and
> the XIVE IC of the previous target if any.  
> 
> So I think we are fine.

Ok.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson