Re: [PATCH V3 07/22] cpr

From: Stefan Hajnoczi <stefanha@redhat.com>
To: Steven Sistare <steven.sistare@oracle.com>
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	"Jason Zeng" <jason.zeng@linux.intel.com>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	"Juan Quintela" <quintela@redhat.com>,
	qemu-devel@nongnu.org,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	"Alex Williamson" <alex.williamson@redhat.com>,
	"Marc-André Lureau" <marcandre.lureau@redhat.com>,
	"Paolo Bonzini" <pbonzini@redhat.com>,
	"Philippe Mathieu-Daudé" <philmd@redhat.com>,
	"Markus Armbruster" <armbru@redhat.com>
Subject: Re: [PATCH V3 07/22] cpr
Date: Tue, 18 May 2021 14:42:50 +0100	[thread overview]
Message-ID: <YKPEWicpOeh3yo5/@stefanha-x1.localdomain> (raw)
In-Reply-To: <881e5c07-8582-478c-9c13-47b1967cd3da@oracle.com>

[-- Attachment #1: Type: text/plain, Size: 2214 bytes --]

On Fri, May 14, 2021 at 11:14:44AM -0400, Steven Sistare wrote:
> On 5/14/2021 7:28 AM, Stefan Hajnoczi wrote:
> > On Thu, May 13, 2021 at 04:21:02PM -0400, Steven Sistare wrote:
> >> On 5/12/2021 12:19 PM, Stefan Hajnoczi wrote:
> >>> On Fri, May 07, 2021 at 05:25:05AM -0700, Steve Sistare wrote:
> >>>> To use the restart mode, qemu must be started with the memfd-alloc machine
> >>>> option.  The memfd's are saved to the environment and kept open across exec,
> >>>> after which they are found from the environment and re-mmap'd.  Hence guest
> >>>> ram is preserved in place, albeit with new virtual addresses in the qemu
> >>>> process.  The caller resumes the guest by calling cprload, which loads
> >>>> state from the file.  If the VM was running at cprsave time, then VM
> >>>> execution resumes.  cprsave supports any type of guest image and block
> >>>> device, but the caller must not modify guest block devices between cprsave
> >>>> and cprload.
> >>>
> >>> Does QEMU's existing -object memory-backend-file on tmpfs or hugetlbfs
> >>> achieve the same thing?
> >>
> >> Not quite.  Various secondary anonymous memory objects are allocated via ram_block_add
> >> and must be preserved, such as these on x86_64.  
> >>   vga.vram
> >>   pc.ram
> >>   pc.bios
> >>   pc.rom
> >>   vga.rom
> >>   rom@etc/acpi/tables
> >>   rom@etc/table-loader
> >>   rom@etc/acpi/rsdp
> >>
> >> Even the read-only areas must be preserved rather than recreated from files in the updated
> >> qemu, as their contents may have changed.
> > 
> > Migration knows how to save/load these RAM blocks. Only pc.ram is
> > significant in size so I'm not sure it's worth special-casing the
> > others?
> 
> Some of these are mapped for vfio dma as a consequence of the normal memory region callback to
> consumers code.  We get conflict errors vs those existing vfio mappings if they are recreated 
> and remapped in the new process.  The memfd option is a simple and robust solution to that issue.

Okay, if the VFIO device DMAs to them then they need to stay alive. Live
migration cannot copy their contents since they could be DMAed to at any
time and we'd copy stale data.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]