Re: [RFC] virtio_pmem: enable live migration support

From: Pankaj Gupta <pankaj.gupta@ionos.com>
To: David Hildenbrand <david@redhat.com>
Cc: Pankaj Gupta <pankaj.gupta.linux@gmail.com>,
	"Michael S . Tsirkin" <mst@redhat.com>,
	Rik van Riel <riel@surriel.com>,
	Qemu Developers <qemu-devel@nongnu.org>,
	"Dr . David Alan Gilbert" <dgilbert@redhat.com>,
	Stefan Hajnoczi <stefanha@redhat.com>,
	Dan Williams <dan.j.williams@intel.com>
Subject: Re: [RFC] virtio_pmem: enable live migration support
Date: Wed, 12 Jan 2022 17:08:37 +0100	[thread overview]
Message-ID: <CALzYo32zxdL6ET_5Btw=Hoat8i4KtA2iUEpd9+_sXSbw84_SAA@mail.gmail.com> (raw)
In-Reply-To: <d0fc0d8e-fe38-1ab2-2d56-23345de783e7@redhat.com>

> >> I mean, that would be fundamentally broken, because the fsync() would
> >> corrupt the file. So I assume in a sane environment, the dst could only
> >> have stale clean pagecache pages. And we'd have to get rid of these to
> >> re-read everything from file.
> >
> > In case of write back cache mode, we could still have stale dirty
> > pages at the destination
> > host and destination fsync is not the right thing to do. We need to
> > invalidate these pages
> > (Can we invalidate dirty pages resident in page cache with
> > POSIX_FADV_DONTNEED as
> > well?) man pages say, we cannot (unless i misunderstood it).
> >
>
> I think you'd have to fsync + POSIX_FADV_DONTNEED. But I am still
> confused how we could end up with dirty pagecache pages on the
> destination. In my opinion, there should only be clean pagecache pages
> -- can someone enlighten me? :)

because of activity on the page cache pages corresponding to mmap region
in the past which is not synced yet or not reclaimed yet. Maybe this
is hypothetical
or not possible, happy to learn?

>
> >>
> >> IIRC, an existing mmap of the file on the dst should not really be
> >> problematic *as long as* we didn't actually access file content that way
> >> and faulted in the pages. So *maybe*, if we do the POSIX_FADV_DONTNEED
> >> on the dst before accessing file content via the mmap, there shouldn't
> >> be an issue. Unless the mmap itself is already problematic.
> >
> > mmap with shared=ON, might result in stale dirty page cache pages?
>
> But only if actually accessing memory, especially writing to it, no?

yes.

>
> >
> >>
> >> I think we can assume that once QEMU starts on the dst and wants to mmap
> >> the file that it's not mapped into any other process yet. vhost-user
> >> will only mmap *after* being told from QEMU about the mmap region and
> >> the location in GPA.
> >
> > maybe we have an old stale dirty page cache page even if there no mmap process
> > alive before mmaping virtio-pmem backend file in destination?
>
> But how could that happen in a sane environment? As I said, any
> writeback on that file from the destination would actually corrupt the
> file that has been used+modified on the source in the meantime, no?
>
>
> --
> Thanks,
>
> David / dhildenb
>