* [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-09 13:50 Pankaj Gupta
0 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-09 13:50 UTC (permalink / raw)
To: linux-kernel, kvm, qemu-devel, linux-nvdimm, linux-fsdevel,
virtualization, linux-acpi, linux-ext4, linux-xfs
Cc: pagupta, jack, lcapitulino, adilger.kernel, zwisler, eblake,
dave.jiang, darrick.wong, vishal.l.verma, mst, willy, hch,
jmoyer, nilal, riel, stefanha, imammedo, dan.j.williams, tytso,
xiaoguangrong.eric, rjw, pbonzini
This patch series has implementation for "virtio pmem".
"virtio pmem" is fake persistent memory(nvdimm) in guest
which allows to bypass the guest page cache. This also
implements a VIRTIO based asynchronous flush mechanism.
Sharing guest kernel driver in this patchset with the
changes suggested in v2. Tested with Qemu side device
emulation for virtio-pmem [6].
Details of project idea for 'virtio pmem' flushing interface
is shared [3] & [4].
Implementation is divided into two parts:
New virtio pmem guest driver and qemu code changes for new
virtio pmem paravirtualized device.
1. Guest virtio-pmem kernel driver
---------------------------------
- Reads persistent memory range from paravirt device and
registers with 'nvdimm_bus'.
- 'nvdimm/pmem' driver uses this information to allocate
persistent memory region and setup filesystem operations
to the allocated memory.
- virtio pmem driver implements asynchronous flushing
interface to flush from guest to host.
2. Qemu virtio-pmem device
---------------------------------
- Creates virtio pmem device and exposes a memory range to
KVM guest.
- At host side this is file backed memory which acts as
persistent memory.
- Qemu side flush uses aio thread pool API's and virtio
for asynchronous guest multi request handling.
David Hildenbrand CCed also posted a modified version[6] of
qemu virtio-pmem code based on updated Qemu memory device API.
Virtio-pmem errors handling:
----------------------------------------
Checked behaviour of virtio-pmem for below types of errors
Need suggestions on expected behaviour for handling these errors?
- Hardware Errors: Uncorrectable recoverable Errors:
a] virtio-pmem:
- As per current logic if error page belongs to Qemu process,
host MCE handler isolates(hwpoison) that page and send SIGBUS.
Qemu SIGBUS handler injects exception to KVM guest.
- KVM guest then isolates the page and send SIGBUS to guest
userspace process which has mapped the page.
b] Existing implementation for ACPI pmem driver:
- Handles such errors with MCE notifier and creates a list
of bad blocks. Read/direct access DAX operation return EIO
if accessed memory page fall in bad block list.
- It also starts backgound scrubbing.
- Similar functionality can be reused in virtio-pmem with MCE
notifier but without scrubbing(no ACPI/ARS)? Need inputs to
confirm if this behaviour is ok or needs any change?
Changes from PATCH v2: [1]
- Disable MAP_SYNC for ext4 & XFS filesystems - [Dan]
- Use name 'virtio pmem' in place of 'fake dax'
Changes from PATCH v1: [2]
- 0-day build test for build dependency on libnvdimm
Changes suggested by - [Dan Williams]
- Split the driver into two parts virtio & pmem
- Move queuing of async block request to block layer
- Add "sync" parameter in nvdimm_flush function
- Use indirect call for nvdimm_flush
- Don’t move declarations to common global header e.g nd.h
- nvdimm_flush() return 0 or -EIO if it fails
- Teach nsio_rw_bytes() that the flush can fail
- Rename nvdimm_flush() to generic_nvdimm_flush()
- Use 'nd_region->provider_data' for long dereferencing
- Remove virtio_pmem_freeze/restore functions
- Remove BSD license text with SPDX license text
- Add might_sleep() in virtio_pmem_flush - [Luiz]
- Make spin_lock_irqsave() narrow
Changes from RFC v3
- Rebase to latest upstream - Luiz
- Call ndregion->flush in place of nvdimm_flush- Luiz
- kmalloc return check - Luiz
- virtqueue full handling - Stefan
- Don't map entire virtio_pmem_req to device - Stefan
- request leak, correct sizeof req- Stefan
- Move declaration to virtio_pmem.c
Changes from RFC v2:
- Add flush function in the nd_region in place of switching
on a flag - Dan & Stefan
- Add flush completion function with proper locking and wait
for host side flush completion - Stefan & Dan
- Keep userspace API in uapi header file - Stefan, MST
- Use LE fields & New device id - MST
- Indentation & spacing suggestions - MST & Eric
- Remove extra header files & add licensing - Stefan
Changes from RFC v1:
- Reuse existing 'pmem' code for registering persistent
memory and other operations instead of creating an entirely
new block driver.
- Use VIRTIO driver to register memory information with
nvdimm_bus and create region_type accordingly.
- Call VIRTIO flush from existing pmem driver.
Pankaj Gupta (5):
libnvdimm: nd_region flush callback support
virtio-pmem: Add virtio-pmem guest driver
libnvdimm: add nd_region buffered dax_dev flag
ext4: disable map_sync for virtio pmem
xfs: disable map_sync for virtio pmem
[2] https://lkml.org/lkml/2018/8/31/407
[3] https://www.spinics.net/lists/kvm/msg149761.html
[4] https://www.spinics.net/lists/kvm/msg153095.html
[5] https://lkml.org/lkml/2018/8/31/413
[6] https://marc.info/?l=qemu-devel&m=153555721901824&w=2
drivers/acpi/nfit/core.c | 4 -
drivers/dax/super.c | 17 +++++
drivers/nvdimm/claim.c | 6 +
drivers/nvdimm/nd.h | 1
drivers/nvdimm/pmem.c | 15 +++-
drivers/nvdimm/region_devs.c | 45 +++++++++++++-
drivers/nvdimm/virtio_pmem.c | 84 ++++++++++++++++++++++++++
drivers/virtio/Kconfig | 10 +++
drivers/virtio/Makefile | 1
drivers/virtio/pmem.c | 125 +++++++++++++++++++++++++++++++++++++++
fs/ext4/file.c | 11 +++
fs/xfs/xfs_file.c | 8 ++
include/linux/dax.h | 9 ++
include/linux/libnvdimm.h | 11 +++
include/linux/virtio_pmem.h | 60 ++++++++++++++++++
include/uapi/linux/virtio_ids.h | 1
include/uapi/linux/virtio_pmem.h | 10 +++
17 files changed, 406 insertions(+), 12 deletions(-)
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-15 5:35 ` Pankaj Gupta
(?)
@ 2019-01-15 20:42 ` Dave Chinner
-1 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-15 20:42 UTC (permalink / raw)
To: Pankaj Gupta
Cc: Jan Kara, KVM list, David Hildenbrand, linux-nvdimm, Jason Wang,
Qemu Developers,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, Ross Zwisler, Eric Blake, darrick wong,
Michael S. Tsirkin, Matthew Wilcox, Christoph Hellwig,
Linux ACPI, linux-ext4, Rik van Riel, Stefan Hajnoczi,
Igor Mammedov, lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, Kevin Wolf,
Nitesh Narayan Lal, Theodore Ts'o, xiaoguangrong eric
On Tue, Jan 15, 2019 at 12:35:06AM -0500, Pankaj Gupta wrote:
>
> > > > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > > > >
> > > > > > > Until you have images (and hence host page cache) shared between
> > > > > > > multiple guests. People will want to do this, because it means they
> > > > > > > only need a single set of pages in host memory for executable
> > > > > > > binaries rather than a set of pages per guest. Then you have
> > > > > > > multiple guests being able to detect residency of the same set of
> > > > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > > > pages from the host cache, then we have a guest-to-guest
> > > > > > > information
> > > > > > > leak channel.
> > > > > >
> > > > > > I don't think we should ever be considering something that would
> > > > > > allow a
> > > > > > guest to evict page's from the host's pagecache [1]. The guest
> > > > > > should
> > > > > > be able to kick its own references to the host's pagecache out of its
> > > > > > own pagecache, but not be able to influence whether the host or
> > > > > > another
> > > > > > guest has a read-only mapping cached.
> > > > > >
> > > > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > > > truncation, holepunching, etc are going to evict pages from the
> > > > > > host's
> > > > > > page cache.
> > > > >
> > > > > This is so correct. Guest does not not evict host page cache pages
> > > > > directly.
> > > >
> > > > They don't right now.
> > > >
> > > > But someone is going to end up asking for discard to work so that
> > > > the guest can free unused space in the underlying spares image (i.e.
> > > > make use of fstrim or mount -o discard) because they have workloads
> > > > that have bursts of space usage and they need to trim the image
> > > > files afterwards to keep their overall space usage under control.
> > > >
> > > > And then....
> > >
> > > ...we reject / push back on that patch citing the above concern.
> >
> > So at what point do we draw the line?
> >
> > We're allowing writable DAX mappings, but as I've pointed out that
> > means we are going to be allowing a potential information leak via
> > files with shared extents to be directly mapped and written to.
> >
> > But we won't allow useful admin operations that allow better
> > management of host side storage space similar to how normal image
> > files are used by guests because it's an information leak vector?
>
> First of all Thank you for all the useful discussions.
> I am summarizing here:
>
> - We have to live with the limitation to not support fstrim and
> mount -o discard options with virtio-pmem as they will evict
> host page cache pages. We cannot allow this for virtio-pmem
> for security reasons. These filesystem commands will just zero out
> unused pages currently.
Not sure I follow you here - what pages are going to be zeroed and
when will they be zeroed? If discard is not allowed, filesystems
just don't issue such commands and the underlying device will never
seen them.
> - If alot of space is unused and not freed guest can request host
> Administrator for truncating the host backing image.
You can't use truncate to free space in a disk image file. The only
way to do it safely in a generic, filesystem agnositic way is to
mount the disk image (e.g. on loopback) and run fstrim on it. The
loopback device will punches holes in the file where all the free
space is reported by the filesystem via discard requests.
Which is kinda my point - this could only be done if the guest is
shut down, which makes it very difficult for admins to manage.
> We are also planning to support qcow2 sparse image format at
> host side with virtio-pmem.
So you're going to be remapping a huge number of disjoint regions
into a linear pmem mapping? ISTR discussions about similar things
for virtio+fuse+dax that came up against "large numbers of mapped
regions don't scale" and so it wasn't a practical solution compared
to a just using raw sparse files....
> - There is no existing solution for Qemu persistent memory
> emulation with write support currently. This solution provides
> us the paravartualized way of emulating persistent memory.
Sure, but the question is why do you need to create an emulation
that doesn't actually perform like pmem? The whole point of pmem is
performance, and emulating pmem by mmap() of a file on spinning
disks is going to be horrible for performance. Even on SSDs it's
going to be orders of magnitudes slower than real pmem.
So exactly what problem are you trying to solve with this driver?
Cheers,
Dave.
--
Dave Chinner
david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-15 20:42 ` Dave Chinner
0 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-15 20:42 UTC (permalink / raw)
To: Pankaj Gupta
Cc: Dan Williams, Matthew Wilcox, Linux Kernel Mailing List,
KVM list, Qemu Developers, linux-nvdimm, linux-fsdevel,
virtualization, Linux ACPI, linux-ext4, linux-xfs, Jan Kara,
Stefan Hajnoczi, Rik van Riel, Nitesh Narayan Lal, Kevin Wolf,
Paolo Bonzini, Ross Zwisler, vishal l verma, dave jiang,
David Hildenbrand, jmoyer, xiaoguangrong eric, Christoph Hellwig,
Michael S. Tsirkin, Jason Wang, lcapitulino, Igor Mammedov,
Eric Blake, Theodore Ts'o, adilger kernel, darrick wong,
Rafael J. Wysocki
On Tue, Jan 15, 2019 at 12:35:06AM -0500, Pankaj Gupta wrote:
>
> > > > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > > > >
> > > > > > > Until you have images (and hence host page cache) shared between
> > > > > > > multiple guests. People will want to do this, because it means they
> > > > > > > only need a single set of pages in host memory for executable
> > > > > > > binaries rather than a set of pages per guest. Then you have
> > > > > > > multiple guests being able to detect residency of the same set of
> > > > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > > > pages from the host cache, then we have a guest-to-guest
> > > > > > > information
> > > > > > > leak channel.
> > > > > >
> > > > > > I don't think we should ever be considering something that would
> > > > > > allow a
> > > > > > guest to evict page's from the host's pagecache [1]. The guest
> > > > > > should
> > > > > > be able to kick its own references to the host's pagecache out of its
> > > > > > own pagecache, but not be able to influence whether the host or
> > > > > > another
> > > > > > guest has a read-only mapping cached.
> > > > > >
> > > > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > > > truncation, holepunching, etc are going to evict pages from the
> > > > > > host's
> > > > > > page cache.
> > > > >
> > > > > This is so correct. Guest does not not evict host page cache pages
> > > > > directly.
> > > >
> > > > They don't right now.
> > > >
> > > > But someone is going to end up asking for discard to work so that
> > > > the guest can free unused space in the underlying spares image (i.e.
> > > > make use of fstrim or mount -o discard) because they have workloads
> > > > that have bursts of space usage and they need to trim the image
> > > > files afterwards to keep their overall space usage under control.
> > > >
> > > > And then....
> > >
> > > ...we reject / push back on that patch citing the above concern.
> >
> > So at what point do we draw the line?
> >
> > We're allowing writable DAX mappings, but as I've pointed out that
> > means we are going to be allowing a potential information leak via
> > files with shared extents to be directly mapped and written to.
> >
> > But we won't allow useful admin operations that allow better
> > management of host side storage space similar to how normal image
> > files are used by guests because it's an information leak vector?
>
> First of all Thank you for all the useful discussions.
> I am summarizing here:
>
> - We have to live with the limitation to not support fstrim and
> mount -o discard options with virtio-pmem as they will evict
> host page cache pages. We cannot allow this for virtio-pmem
> for security reasons. These filesystem commands will just zero out
> unused pages currently.
Not sure I follow you here - what pages are going to be zeroed and
when will they be zeroed? If discard is not allowed, filesystems
just don't issue such commands and the underlying device will never
seen them.
> - If alot of space is unused and not freed guest can request host
> Administrator for truncating the host backing image.
You can't use truncate to free space in a disk image file. The only
way to do it safely in a generic, filesystem agnositic way is to
mount the disk image (e.g. on loopback) and run fstrim on it. The
loopback device will punches holes in the file where all the free
space is reported by the filesystem via discard requests.
Which is kinda my point - this could only be done if the guest is
shut down, which makes it very difficult for admins to manage.
> We are also planning to support qcow2 sparse image format at
> host side with virtio-pmem.
So you're going to be remapping a huge number of disjoint regions
into a linear pmem mapping? ISTR discussions about similar things
for virtio+fuse+dax that came up against "large numbers of mapped
regions don't scale" and so it wasn't a practical solution compared
to a just using raw sparse files....
> - There is no existing solution for Qemu persistent memory
> emulation with write support currently. This solution provides
> us the paravartualized way of emulating persistent memory.
Sure, but the question is why do you need to create an emulation
that doesn't actually perform like pmem? The whole point of pmem is
performance, and emulating pmem by mmap() of a file on spinning
disks is going to be horrible for performance. Even on SSDs it's
going to be orders of magnitudes slower than real pmem.
So exactly what problem are you trying to solve with this driver?
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-15 20:42 ` Dave Chinner
0 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-15 20:42 UTC (permalink / raw)
To: Pankaj Gupta
Cc: Jan Kara, KVM list, David Hildenbrand, linux-nvdimm, Jason Wang,
Qemu Developers,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, Ross Zwisler, Eric Blake, darrick wong,
Michael S. Tsirkin, Matthew Wilcox, Christoph Hellwig,
Linux ACPI, linux-ext4, Rik van Riel, Stefan Hajnoczi,
Igor Mammedov, lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, Kevin Wolf,
Nitesh Narayan Lal, Theodore Ts'o, xiaoguangrong eric
On Tue, Jan 15, 2019 at 12:35:06AM -0500, Pankaj Gupta wrote:
>
> > > > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > > > >
> > > > > > > Until you have images (and hence host page cache) shared between
> > > > > > > multiple guests. People will want to do this, because it means they
> > > > > > > only need a single set of pages in host memory for executable
> > > > > > > binaries rather than a set of pages per guest. Then you have
> > > > > > > multiple guests being able to detect residency of the same set of
> > > > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > > > pages from the host cache, then we have a guest-to-guest
> > > > > > > information
> > > > > > > leak channel.
> > > > > >
> > > > > > I don't think we should ever be considering something that would
> > > > > > allow a
> > > > > > guest to evict page's from the host's pagecache [1]. The guest
> > > > > > should
> > > > > > be able to kick its own references to the host's pagecache out of its
> > > > > > own pagecache, but not be able to influence whether the host or
> > > > > > another
> > > > > > guest has a read-only mapping cached.
> > > > > >
> > > > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > > > truncation, holepunching, etc are going to evict pages from the
> > > > > > host's
> > > > > > page cache.
> > > > >
> > > > > This is so correct. Guest does not not evict host page cache pages
> > > > > directly.
> > > >
> > > > They don't right now.
> > > >
> > > > But someone is going to end up asking for discard to work so that
> > > > the guest can free unused space in the underlying spares image (i.e.
> > > > make use of fstrim or mount -o discard) because they have workloads
> > > > that have bursts of space usage and they need to trim the image
> > > > files afterwards to keep their overall space usage under control.
> > > >
> > > > And then....
> > >
> > > ...we reject / push back on that patch citing the above concern.
> >
> > So at what point do we draw the line?
> >
> > We're allowing writable DAX mappings, but as I've pointed out that
> > means we are going to be allowing a potential information leak via
> > files with shared extents to be directly mapped and written to.
> >
> > But we won't allow useful admin operations that allow better
> > management of host side storage space similar to how normal image
> > files are used by guests because it's an information leak vector?
>
> First of all Thank you for all the useful discussions.
> I am summarizing here:
>
> - We have to live with the limitation to not support fstrim and
> mount -o discard options with virtio-pmem as they will evict
> host page cache pages. We cannot allow this for virtio-pmem
> for security reasons. These filesystem commands will just zero out
> unused pages currently.
Not sure I follow you here - what pages are going to be zeroed and
when will they be zeroed? If discard is not allowed, filesystems
just don't issue such commands and the underlying device will never
seen them.
> - If alot of space is unused and not freed guest can request host
> Administrator for truncating the host backing image.
You can't use truncate to free space in a disk image file. The only
way to do it safely in a generic, filesystem agnositic way is to
mount the disk image (e.g. on loopback) and run fstrim on it. The
loopback device will punches holes in the file where all the free
space is reported by the filesystem via discard requests.
Which is kinda my point - this could only be done if the guest is
shut down, which makes it very difficult for admins to manage.
> We are also planning to support qcow2 sparse image format at
> host side with virtio-pmem.
So you're going to be remapping a huge number of disjoint regions
into a linear pmem mapping? ISTR discussions about similar things
for virtio+fuse+dax that came up against "large numbers of mapped
regions don't scale" and so it wasn't a practical solution compared
to a just using raw sparse files....
> - There is no existing solution for Qemu persistent memory
> emulation with write support currently. This solution provides
> us the paravartualized way of emulating persistent memory.
Sure, but the question is why do you need to create an emulation
that doesn't actually perform like pmem? The whole point of pmem is
performance, and emulating pmem by mmap() of a file on spinning
disks is going to be horrible for performance. Even on SSDs it's
going to be orders of magnitudes slower than real pmem.
So exactly what problem are you trying to solve with this driver?
Cheers,
Dave.
--
Dave Chinner
david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-15 5:35 ` Pankaj Gupta
` (2 preceding siblings ...)
(?)
@ 2019-01-15 20:42 ` Dave Chinner
-1 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-15 20:42 UTC (permalink / raw)
To: Pankaj Gupta
Cc: Jan Kara, KVM list, linux-nvdimm, Qemu Developers,
virtualization, adilger kernel, Ross Zwisler, Eric Blake,
dave jiang, darrick wong, vishal l verma, Michael S. Tsirkin,
Matthew Wilcox, Christoph Hellwig, Linux ACPI, jmoyer,
linux-ext4, Rik van Riel, Stefan Hajnoczi, Igor Mammedov,
Dan Williams, lcapitulino, Nitesh Narayan Lal, Theodore Ts'o
On Tue, Jan 15, 2019 at 12:35:06AM -0500, Pankaj Gupta wrote:
>
> > > > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > > > >
> > > > > > > Until you have images (and hence host page cache) shared between
> > > > > > > multiple guests. People will want to do this, because it means they
> > > > > > > only need a single set of pages in host memory for executable
> > > > > > > binaries rather than a set of pages per guest. Then you have
> > > > > > > multiple guests being able to detect residency of the same set of
> > > > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > > > pages from the host cache, then we have a guest-to-guest
> > > > > > > information
> > > > > > > leak channel.
> > > > > >
> > > > > > I don't think we should ever be considering something that would
> > > > > > allow a
> > > > > > guest to evict page's from the host's pagecache [1]. The guest
> > > > > > should
> > > > > > be able to kick its own references to the host's pagecache out of its
> > > > > > own pagecache, but not be able to influence whether the host or
> > > > > > another
> > > > > > guest has a read-only mapping cached.
> > > > > >
> > > > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > > > truncation, holepunching, etc are going to evict pages from the
> > > > > > host's
> > > > > > page cache.
> > > > >
> > > > > This is so correct. Guest does not not evict host page cache pages
> > > > > directly.
> > > >
> > > > They don't right now.
> > > >
> > > > But someone is going to end up asking for discard to work so that
> > > > the guest can free unused space in the underlying spares image (i.e.
> > > > make use of fstrim or mount -o discard) because they have workloads
> > > > that have bursts of space usage and they need to trim the image
> > > > files afterwards to keep their overall space usage under control.
> > > >
> > > > And then....
> > >
> > > ...we reject / push back on that patch citing the above concern.
> >
> > So at what point do we draw the line?
> >
> > We're allowing writable DAX mappings, but as I've pointed out that
> > means we are going to be allowing a potential information leak via
> > files with shared extents to be directly mapped and written to.
> >
> > But we won't allow useful admin operations that allow better
> > management of host side storage space similar to how normal image
> > files are used by guests because it's an information leak vector?
>
> First of all Thank you for all the useful discussions.
> I am summarizing here:
>
> - We have to live with the limitation to not support fstrim and
> mount -o discard options with virtio-pmem as they will evict
> host page cache pages. We cannot allow this for virtio-pmem
> for security reasons. These filesystem commands will just zero out
> unused pages currently.
Not sure I follow you here - what pages are going to be zeroed and
when will they be zeroed? If discard is not allowed, filesystems
just don't issue such commands and the underlying device will never
seen them.
> - If alot of space is unused and not freed guest can request host
> Administrator for truncating the host backing image.
You can't use truncate to free space in a disk image file. The only
way to do it safely in a generic, filesystem agnositic way is to
mount the disk image (e.g. on loopback) and run fstrim on it. The
loopback device will punches holes in the file where all the free
space is reported by the filesystem via discard requests.
Which is kinda my point - this could only be done if the guest is
shut down, which makes it very difficult for admins to manage.
> We are also planning to support qcow2 sparse image format at
> host side with virtio-pmem.
So you're going to be remapping a huge number of disjoint regions
into a linear pmem mapping? ISTR discussions about similar things
for virtio+fuse+dax that came up against "large numbers of mapped
regions don't scale" and so it wasn't a practical solution compared
to a just using raw sparse files....
> - There is no existing solution for Qemu persistent memory
> emulation with write support currently. This solution provides
> us the paravartualized way of emulating persistent memory.
Sure, but the question is why do you need to create an emulation
that doesn't actually perform like pmem? The whole point of pmem is
performance, and emulating pmem by mmap() of a file on spinning
disks is going to be horrible for performance. Even on SSDs it's
going to be orders of magnitudes slower than real pmem.
So exactly what problem are you trying to solve with this driver?
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-14 22:21 ` Dave Chinner
(?)
@ 2019-01-15 5:35 ` Pankaj Gupta
-1 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-15 5:35 UTC (permalink / raw)
To: Dave Chinner
Cc: Jan Kara, KVM list, David Hildenbrand, linux-nvdimm, Jason Wang,
Qemu Developers,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, Ross Zwisler, Eric Blake, darrick wong,
Michael S. Tsirkin, Matthew Wilcox, Christoph Hellwig,
Linux ACPI, linux-ext4, Rik van Riel, Stefan Hajnoczi,
Igor Mammedov, lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, Kevin Wolf,
Nitesh Narayan Lal, Theodore Ts'o, xiaoguangrong eric
> > > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > > >
> > > > > > Until you have images (and hence host page cache) shared between
> > > > > > multiple guests. People will want to do this, because it means they
> > > > > > only need a single set of pages in host memory for executable
> > > > > > binaries rather than a set of pages per guest. Then you have
> > > > > > multiple guests being able to detect residency of the same set of
> > > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > > pages from the host cache, then we have a guest-to-guest
> > > > > > information
> > > > > > leak channel.
> > > > >
> > > > > I don't think we should ever be considering something that would
> > > > > allow a
> > > > > guest to evict page's from the host's pagecache [1]. The guest
> > > > > should
> > > > > be able to kick its own references to the host's pagecache out of its
> > > > > own pagecache, but not be able to influence whether the host or
> > > > > another
> > > > > guest has a read-only mapping cached.
> > > > >
> > > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > > truncation, holepunching, etc are going to evict pages from the
> > > > > host's
> > > > > page cache.
> > > >
> > > > This is so correct. Guest does not not evict host page cache pages
> > > > directly.
> > >
> > > They don't right now.
> > >
> > > But someone is going to end up asking for discard to work so that
> > > the guest can free unused space in the underlying spares image (i.e.
> > > make use of fstrim or mount -o discard) because they have workloads
> > > that have bursts of space usage and they need to trim the image
> > > files afterwards to keep their overall space usage under control.
> > >
> > > And then....
> >
> > ...we reject / push back on that patch citing the above concern.
>
> So at what point do we draw the line?
>
> We're allowing writable DAX mappings, but as I've pointed out that
> means we are going to be allowing a potential information leak via
> files with shared extents to be directly mapped and written to.
>
> But we won't allow useful admin operations that allow better
> management of host side storage space similar to how normal image
> files are used by guests because it's an information leak vector?
First of all Thank you for all the useful discussions.
I am summarizing here:
- We have to live with the limitation to not support fstrim and
mount -o discard options with virtio-pmem as they will evict
host page cache pages. We cannot allow this for virtio-pmem
for security reasons. These filesystem commands will just zero out
unused pages currently.
- If alot of space is unused and not freed guest can request host
Administrator for truncating the host backing image.
We are also planning to support qcow2 sparse image format at
host side with virtio-pmem.
- There is no existing solution for Qemu persistent memory
emulation with write support currently. This solution provides
us the paravartualized way of emulating persistent memory. It
does not emulate of ACPI structures instead it just uses VIRTIO
for communication between guest & host. It is fast because of its
asynchronous nature and it works well. This makes use of at guest
side libnvdimm API's
- If disk size freeing problem with guest files trim truncate is
very important for users, they can still use real hardware which
will provide them both (advance disk features & page cache by pass).
Considering all the above reasons I think this feature is useful
from virtualization point of view. As Dave rightly said we should
be careful and I think now we are careful with the security implications
of this device.
Thanks again for all the inputs.
Best regards,
Pankaj
>
> That's splitting some really fine hairs there...
>
> > > > In case of virtio-pmem & DAX, guest clears guest page cache exceptional
> > > > entries.
> > > > Its solely decision of host to take action on the host page cache
> > > > pages.
> > > >
> > > > In case of virtio-pmem, guest does not modify host file directly i.e
> > > > don't
> > > > perform hole punch & truncation operation directly on host file.
> > >
> > > ... this will no longer be true, and the nuclear landmine in this
> > > driver interface will have been armed....
> >
> > I agree with the need to be careful when / if explicit cache control
> > is added, but that's not the case today.
>
> "if"?
>
> I expect it to be "when", not if. Expect the worst, plan for it now.
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org
>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-15 5:35 ` Pankaj Gupta
0 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-15 5:35 UTC (permalink / raw)
To: Dave Chinner
Cc: Dan Williams, Matthew Wilcox, Linux Kernel Mailing List,
KVM list, Qemu Developers, linux-nvdimm, linux-fsdevel,
virtualization, Linux ACPI, linux-ext4, linux-xfs, Jan Kara,
Stefan Hajnoczi, Rik van Riel, Nitesh Narayan Lal, Kevin Wolf,
Paolo Bonzini, Ross Zwisler, vishal l verma, dave jiang,
David Hildenbrand, jmoyer, xiaoguangrong eric, Christoph Hellwig,
Michael S. Tsirkin, Jason Wang, lcapitulino, Igor Mammedov,
Eric Blake, Theodore Ts'o, adilger kernel, darrick wong,
Rafael J. Wysocki
> > > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > > >
> > > > > > Until you have images (and hence host page cache) shared between
> > > > > > multiple guests. People will want to do this, because it means they
> > > > > > only need a single set of pages in host memory for executable
> > > > > > binaries rather than a set of pages per guest. Then you have
> > > > > > multiple guests being able to detect residency of the same set of
> > > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > > pages from the host cache, then we have a guest-to-guest
> > > > > > information
> > > > > > leak channel.
> > > > >
> > > > > I don't think we should ever be considering something that would
> > > > > allow a
> > > > > guest to evict page's from the host's pagecache [1]. The guest
> > > > > should
> > > > > be able to kick its own references to the host's pagecache out of its
> > > > > own pagecache, but not be able to influence whether the host or
> > > > > another
> > > > > guest has a read-only mapping cached.
> > > > >
> > > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > > truncation, holepunching, etc are going to evict pages from the
> > > > > host's
> > > > > page cache.
> > > >
> > > > This is so correct. Guest does not not evict host page cache pages
> > > > directly.
> > >
> > > They don't right now.
> > >
> > > But someone is going to end up asking for discard to work so that
> > > the guest can free unused space in the underlying spares image (i.e.
> > > make use of fstrim or mount -o discard) because they have workloads
> > > that have bursts of space usage and they need to trim the image
> > > files afterwards to keep their overall space usage under control.
> > >
> > > And then....
> >
> > ...we reject / push back on that patch citing the above concern.
>
> So at what point do we draw the line?
>
> We're allowing writable DAX mappings, but as I've pointed out that
> means we are going to be allowing a potential information leak via
> files with shared extents to be directly mapped and written to.
>
> But we won't allow useful admin operations that allow better
> management of host side storage space similar to how normal image
> files are used by guests because it's an information leak vector?
First of all Thank you for all the useful discussions.
I am summarizing here:
- We have to live with the limitation to not support fstrim and
mount -o discard options with virtio-pmem as they will evict
host page cache pages. We cannot allow this for virtio-pmem
for security reasons. These filesystem commands will just zero out
unused pages currently.
- If alot of space is unused and not freed guest can request host
Administrator for truncating the host backing image.
We are also planning to support qcow2 sparse image format at
host side with virtio-pmem.
- There is no existing solution for Qemu persistent memory
emulation with write support currently. This solution provides
us the paravartualized way of emulating persistent memory. It
does not emulate of ACPI structures instead it just uses VIRTIO
for communication between guest & host. It is fast because of its
asynchronous nature and it works well. This makes use of at guest
side libnvdimm API's
- If disk size freeing problem with guest files trim truncate is
very important for users, they can still use real hardware which
will provide them both (advance disk features & page cache by pass).
Considering all the above reasons I think this feature is useful
from virtualization point of view. As Dave rightly said we should
be careful and I think now we are careful with the security implications
of this device.
Thanks again for all the inputs.
Best regards,
Pankaj
>
> That's splitting some really fine hairs there...
>
> > > > In case of virtio-pmem & DAX, guest clears guest page cache exceptional
> > > > entries.
> > > > Its solely decision of host to take action on the host page cache
> > > > pages.
> > > >
> > > > In case of virtio-pmem, guest does not modify host file directly i.e
> > > > don't
> > > > perform hole punch & truncation operation directly on host file.
> > >
> > > ... this will no longer be true, and the nuclear landmine in this
> > > driver interface will have been armed....
> >
> > I agree with the need to be careful when / if explicit cache control
> > is added, but that's not the case today.
>
> "if"?
>
> I expect it to be "when", not if. Expect the worst, plan for it now.
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-15 5:35 ` Pankaj Gupta
0 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-15 5:35 UTC (permalink / raw)
To: Dave Chinner
Cc: Jan Kara, KVM list, David Hildenbrand, linux-nvdimm, Jason Wang,
Qemu Developers,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, Ross Zwisler, Eric Blake, darrick wong,
Michael S. Tsirkin, Matthew Wilcox, Christoph Hellwig,
Linux ACPI, linux-ext4, Rik van Riel, Stefan Hajnoczi,
Igor Mammedov, lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, Kevin Wolf,
Nitesh Narayan Lal, Theodore Ts'o, xiaoguangrong eric
> > > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > > >
> > > > > > Until you have images (and hence host page cache) shared between
> > > > > > multiple guests. People will want to do this, because it means they
> > > > > > only need a single set of pages in host memory for executable
> > > > > > binaries rather than a set of pages per guest. Then you have
> > > > > > multiple guests being able to detect residency of the same set of
> > > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > > pages from the host cache, then we have a guest-to-guest
> > > > > > information
> > > > > > leak channel.
> > > > >
> > > > > I don't think we should ever be considering something that would
> > > > > allow a
> > > > > guest to evict page's from the host's pagecache [1]. The guest
> > > > > should
> > > > > be able to kick its own references to the host's pagecache out of its
> > > > > own pagecache, but not be able to influence whether the host or
> > > > > another
> > > > > guest has a read-only mapping cached.
> > > > >
> > > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > > truncation, holepunching, etc are going to evict pages from the
> > > > > host's
> > > > > page cache.
> > > >
> > > > This is so correct. Guest does not not evict host page cache pages
> > > > directly.
> > >
> > > They don't right now.
> > >
> > > But someone is going to end up asking for discard to work so that
> > > the guest can free unused space in the underlying spares image (i.e.
> > > make use of fstrim or mount -o discard) because they have workloads
> > > that have bursts of space usage and they need to trim the image
> > > files afterwards to keep their overall space usage under control.
> > >
> > > And then....
> >
> > ...we reject / push back on that patch citing the above concern.
>
> So at what point do we draw the line?
>
> We're allowing writable DAX mappings, but as I've pointed out that
> means we are going to be allowing a potential information leak via
> files with shared extents to be directly mapped and written to.
>
> But we won't allow useful admin operations that allow better
> management of host side storage space similar to how normal image
> files are used by guests because it's an information leak vector?
First of all Thank you for all the useful discussions.
I am summarizing here:
- We have to live with the limitation to not support fstrim and
mount -o discard options with virtio-pmem as they will evict
host page cache pages. We cannot allow this for virtio-pmem
for security reasons. These filesystem commands will just zero out
unused pages currently.
- If alot of space is unused and not freed guest can request host
Administrator for truncating the host backing image.
We are also planning to support qcow2 sparse image format at
host side with virtio-pmem.
- There is no existing solution for Qemu persistent memory
emulation with write support currently. This solution provides
us the paravartualized way of emulating persistent memory. It
does not emulate of ACPI structures instead it just uses VIRTIO
for communication between guest & host. It is fast because of its
asynchronous nature and it works well. This makes use of at guest
side libnvdimm API's
- If disk size freeing problem with guest files trim truncate is
very important for users, they can still use real hardware which
will provide them both (advance disk features & page cache by pass).
Considering all the above reasons I think this feature is useful
from virtualization point of view. As Dave rightly said we should
be careful and I think now we are careful with the security implications
of this device.
Thanks again for all the inputs.
Best regards,
Pankaj
>
> That's splitting some really fine hairs there...
>
> > > > In case of virtio-pmem & DAX, guest clears guest page cache exceptional
> > > > entries.
> > > > Its solely decision of host to take action on the host page cache
> > > > pages.
> > > >
> > > > In case of virtio-pmem, guest does not modify host file directly i.e
> > > > don't
> > > > perform hole punch & truncation operation directly on host file.
> > >
> > > ... this will no longer be true, and the nuclear landmine in this
> > > driver interface will have been armed....
> >
> > I agree with the need to be careful when / if explicit cache control
> > is added, but that's not the case today.
>
> "if"?
>
> I expect it to be "when", not if. Expect the worst, plan for it now.
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org
>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-14 22:21 ` Dave Chinner
` (4 preceding siblings ...)
(?)
@ 2019-01-15 5:35 ` Pankaj Gupta
-1 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-15 5:35 UTC (permalink / raw)
To: Dave Chinner
Cc: Jan Kara, KVM list, linux-nvdimm, Qemu Developers,
virtualization, adilger kernel, Ross Zwisler, Eric Blake,
dave jiang, darrick wong, vishal l verma, Michael S. Tsirkin,
Matthew Wilcox, Christoph Hellwig, Linux ACPI, jmoyer,
linux-ext4, Rik van Riel, Stefan Hajnoczi, Igor Mammedov,
Dan Williams, lcapitulino, Nitesh Narayan Lal, Theodore Ts'o
> > > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > > >
> > > > > > Until you have images (and hence host page cache) shared between
> > > > > > multiple guests. People will want to do this, because it means they
> > > > > > only need a single set of pages in host memory for executable
> > > > > > binaries rather than a set of pages per guest. Then you have
> > > > > > multiple guests being able to detect residency of the same set of
> > > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > > pages from the host cache, then we have a guest-to-guest
> > > > > > information
> > > > > > leak channel.
> > > > >
> > > > > I don't think we should ever be considering something that would
> > > > > allow a
> > > > > guest to evict page's from the host's pagecache [1]. The guest
> > > > > should
> > > > > be able to kick its own references to the host's pagecache out of its
> > > > > own pagecache, but not be able to influence whether the host or
> > > > > another
> > > > > guest has a read-only mapping cached.
> > > > >
> > > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > > truncation, holepunching, etc are going to evict pages from the
> > > > > host's
> > > > > page cache.
> > > >
> > > > This is so correct. Guest does not not evict host page cache pages
> > > > directly.
> > >
> > > They don't right now.
> > >
> > > But someone is going to end up asking for discard to work so that
> > > the guest can free unused space in the underlying spares image (i.e.
> > > make use of fstrim or mount -o discard) because they have workloads
> > > that have bursts of space usage and they need to trim the image
> > > files afterwards to keep their overall space usage under control.
> > >
> > > And then....
> >
> > ...we reject / push back on that patch citing the above concern.
>
> So at what point do we draw the line?
>
> We're allowing writable DAX mappings, but as I've pointed out that
> means we are going to be allowing a potential information leak via
> files with shared extents to be directly mapped and written to.
>
> But we won't allow useful admin operations that allow better
> management of host side storage space similar to how normal image
> files are used by guests because it's an information leak vector?
First of all Thank you for all the useful discussions.
I am summarizing here:
- We have to live with the limitation to not support fstrim and
mount -o discard options with virtio-pmem as they will evict
host page cache pages. We cannot allow this for virtio-pmem
for security reasons. These filesystem commands will just zero out
unused pages currently.
- If alot of space is unused and not freed guest can request host
Administrator for truncating the host backing image.
We are also planning to support qcow2 sparse image format at
host side with virtio-pmem.
- There is no existing solution for Qemu persistent memory
emulation with write support currently. This solution provides
us the paravartualized way of emulating persistent memory. It
does not emulate of ACPI structures instead it just uses VIRTIO
for communication between guest & host. It is fast because of its
asynchronous nature and it works well. This makes use of at guest
side libnvdimm API's
- If disk size freeing problem with guest files trim truncate is
very important for users, they can still use real hardware which
will provide them both (advance disk features & page cache by pass).
Considering all the above reasons I think this feature is useful
from virtualization point of view. As Dave rightly said we should
be careful and I think now we are careful with the security implications
of this device.
Thanks again for all the inputs.
Best regards,
Pankaj
>
> That's splitting some really fine hairs there...
>
> > > > In case of virtio-pmem & DAX, guest clears guest page cache exceptional
> > > > entries.
> > > > Its solely decision of host to take action on the host page cache
> > > > pages.
> > > >
> > > > In case of virtio-pmem, guest does not modify host file directly i.e
> > > > don't
> > > > perform hole punch & truncation operation directly on host file.
> > >
> > > ... this will no longer be true, and the nuclear landmine in this
> > > driver interface will have been armed....
> >
> > I agree with the need to be careful when / if explicit cache control
> > is added, but that's not the case today.
>
> "if"?
>
> I expect it to be "when", not if. Expect the worst, plan for it now.
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-14 22:21 ` Dave Chinner
(?)
@ 2019-01-15 2:19 ` Michael S. Tsirkin
-1 siblings, 0 replies; 57+ messages in thread
From: Michael S. Tsirkin @ 2019-01-15 2:19 UTC (permalink / raw)
To: Dave Chinner
Cc: Dan Williams, Pankaj Gupta, Matthew Wilcox,
Linux Kernel Mailing List, KVM list, Qemu Developers,
linux-nvdimm, linux-fsdevel, virtualization, Linux ACPI,
linux-ext4, linux-xfs, Jan Kara, Stefan Hajnoczi, Rik van Riel,
Nitesh Narayan Lal, Kevin Wolf, Paolo Bonzini, Ross Zwisler
On Tue, Jan 15, 2019 at 09:21:32AM +1100, Dave Chinner wrote:
> On Mon, Jan 14, 2019 at 01:35:57PM -0800, Dan Williams wrote:
> > On Mon, Jan 14, 2019 at 1:25 PM Dave Chinner <david@fromorbit.com> wrote:
> > >
> > > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > > >
> > > > > > Until you have images (and hence host page cache) shared between
> > > > > > multiple guests. People will want to do this, because it means they
> > > > > > only need a single set of pages in host memory for executable
> > > > > > binaries rather than a set of pages per guest. Then you have
> > > > > > multiple guests being able to detect residency of the same set of
> > > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > > pages from the host cache, then we have a guest-to-guest information
> > > > > > leak channel.
> > > > >
> > > > > I don't think we should ever be considering something that would allow a
> > > > > guest to evict page's from the host's pagecache [1]. The guest should
> > > > > be able to kick its own references to the host's pagecache out of its
> > > > > own pagecache, but not be able to influence whether the host or another
> > > > > guest has a read-only mapping cached.
> > > > >
> > > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > > truncation, holepunching, etc are going to evict pages from the host's
> > > > > page cache.
> > > >
> > > > This is so correct. Guest does not not evict host page cache pages directly.
> > >
> > > They don't right now.
> > >
> > > But someone is going to end up asking for discard to work so that
> > > the guest can free unused space in the underlying spares image (i.e.
> > > make use of fstrim or mount -o discard) because they have workloads
> > > that have bursts of space usage and they need to trim the image
> > > files afterwards to keep their overall space usage under control.
> > >
> > > And then....
> >
> > ...we reject / push back on that patch citing the above concern.
>
> So at what point do we draw the line?
>
> We're allowing writable DAX mappings, but as I've pointed out that
> means we are going to be allowing a potential information leak via
> files with shared extents to be directly mapped and written to.
>
> But we won't allow useful admin operations that allow better
> management of host side storage space similar to how normal image
> files are used by guests because it's an information leak vector?
>
> That's splitting some really fine hairs there...
May I summarize that th security implications need to
be documented?
In fact that would make a fine security implications section
in the device specification.
> > > > In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> > > > Its solely decision of host to take action on the host page cache pages.
> > > >
> > > > In case of virtio-pmem, guest does not modify host file directly i.e don't
> > > > perform hole punch & truncation operation directly on host file.
> > >
> > > ... this will no longer be true, and the nuclear landmine in this
> > > driver interface will have been armed....
> >
> > I agree with the need to be careful when / if explicit cache control
> > is added, but that's not the case today.
>
> "if"?
>
> I expect it to be "when", not if. Expect the worst, plan for it now.
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-15 2:19 ` Michael S. Tsirkin
0 siblings, 0 replies; 57+ messages in thread
From: Michael S. Tsirkin @ 2019-01-15 2:19 UTC (permalink / raw)
To: Dave Chinner
Cc: Dan Williams, Pankaj Gupta, Matthew Wilcox,
Linux Kernel Mailing List, KVM list, Qemu Developers,
linux-nvdimm, linux-fsdevel, virtualization, Linux ACPI,
linux-ext4, linux-xfs, Jan Kara, Stefan Hajnoczi, Rik van Riel,
Nitesh Narayan Lal, Kevin Wolf, Paolo Bonzini, Ross Zwisler,
vishal l verma, dave jiang, David Hildenbrand, jmoyer,
xiaoguangrong eric, Christoph Hellwig, Jason Wang, lcapitulino,
Igor Mammedov, Eric Blake, Theodore Ts'o, adilger kernel,
darrick wong, Rafael J. Wysocki
On Tue, Jan 15, 2019 at 09:21:32AM +1100, Dave Chinner wrote:
> On Mon, Jan 14, 2019 at 01:35:57PM -0800, Dan Williams wrote:
> > On Mon, Jan 14, 2019 at 1:25 PM Dave Chinner <david@fromorbit.com> wrote:
> > >
> > > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > > >
> > > > > > Until you have images (and hence host page cache) shared between
> > > > > > multiple guests. People will want to do this, because it means they
> > > > > > only need a single set of pages in host memory for executable
> > > > > > binaries rather than a set of pages per guest. Then you have
> > > > > > multiple guests being able to detect residency of the same set of
> > > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > > pages from the host cache, then we have a guest-to-guest information
> > > > > > leak channel.
> > > > >
> > > > > I don't think we should ever be considering something that would allow a
> > > > > guest to evict page's from the host's pagecache [1]. The guest should
> > > > > be able to kick its own references to the host's pagecache out of its
> > > > > own pagecache, but not be able to influence whether the host or another
> > > > > guest has a read-only mapping cached.
> > > > >
> > > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > > truncation, holepunching, etc are going to evict pages from the host's
> > > > > page cache.
> > > >
> > > > This is so correct. Guest does not not evict host page cache pages directly.
> > >
> > > They don't right now.
> > >
> > > But someone is going to end up asking for discard to work so that
> > > the guest can free unused space in the underlying spares image (i.e.
> > > make use of fstrim or mount -o discard) because they have workloads
> > > that have bursts of space usage and they need to trim the image
> > > files afterwards to keep their overall space usage under control.
> > >
> > > And then....
> >
> > ...we reject / push back on that patch citing the above concern.
>
> So at what point do we draw the line?
>
> We're allowing writable DAX mappings, but as I've pointed out that
> means we are going to be allowing a potential information leak via
> files with shared extents to be directly mapped and written to.
>
> But we won't allow useful admin operations that allow better
> management of host side storage space similar to how normal image
> files are used by guests because it's an information leak vector?
>
> That's splitting some really fine hairs there...
May I summarize that th security implications need to
be documented?
In fact that would make a fine security implications section
in the device specification.
> > > > In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> > > > Its solely decision of host to take action on the host page cache pages.
> > > >
> > > > In case of virtio-pmem, guest does not modify host file directly i.e don't
> > > > perform hole punch & truncation operation directly on host file.
> > >
> > > ... this will no longer be true, and the nuclear landmine in this
> > > driver interface will have been armed....
> >
> > I agree with the need to be careful when / if explicit cache control
> > is added, but that's not the case today.
>
> "if"?
>
> I expect it to be "when", not if. Expect the worst, plan for it now.
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-15 2:19 ` Michael S. Tsirkin
0 siblings, 0 replies; 57+ messages in thread
From: Michael S. Tsirkin @ 2019-01-15 2:19 UTC (permalink / raw)
To: Dave Chinner
Cc: Dan Williams, Pankaj Gupta, Matthew Wilcox,
Linux Kernel Mailing List, KVM list, Qemu Developers,
linux-nvdimm, linux-fsdevel, virtualization, Linux ACPI,
linux-ext4, linux-xfs, Jan Kara, Stefan Hajnoczi, Rik van Riel,
Nitesh Narayan Lal, Kevin Wolf, Paolo Bonzini, Ross Zwisler,
vishal l verma
On Tue, Jan 15, 2019 at 09:21:32AM +1100, Dave Chinner wrote:
> On Mon, Jan 14, 2019 at 01:35:57PM -0800, Dan Williams wrote:
> > On Mon, Jan 14, 2019 at 1:25 PM Dave Chinner <david@fromorbit.com> wrote:
> > >
> > > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > > >
> > > > > > Until you have images (and hence host page cache) shared between
> > > > > > multiple guests. People will want to do this, because it means they
> > > > > > only need a single set of pages in host memory for executable
> > > > > > binaries rather than a set of pages per guest. Then you have
> > > > > > multiple guests being able to detect residency of the same set of
> > > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > > pages from the host cache, then we have a guest-to-guest information
> > > > > > leak channel.
> > > > >
> > > > > I don't think we should ever be considering something that would allow a
> > > > > guest to evict page's from the host's pagecache [1]. The guest should
> > > > > be able to kick its own references to the host's pagecache out of its
> > > > > own pagecache, but not be able to influence whether the host or another
> > > > > guest has a read-only mapping cached.
> > > > >
> > > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > > truncation, holepunching, etc are going to evict pages from the host's
> > > > > page cache.
> > > >
> > > > This is so correct. Guest does not not evict host page cache pages directly.
> > >
> > > They don't right now.
> > >
> > > But someone is going to end up asking for discard to work so that
> > > the guest can free unused space in the underlying spares image (i.e.
> > > make use of fstrim or mount -o discard) because they have workloads
> > > that have bursts of space usage and they need to trim the image
> > > files afterwards to keep their overall space usage under control.
> > >
> > > And then....
> >
> > ...we reject / push back on that patch citing the above concern.
>
> So at what point do we draw the line?
>
> We're allowing writable DAX mappings, but as I've pointed out that
> means we are going to be allowing a potential information leak via
> files with shared extents to be directly mapped and written to.
>
> But we won't allow useful admin operations that allow better
> management of host side storage space similar to how normal image
> files are used by guests because it's an information leak vector?
>
> That's splitting some really fine hairs there...
May I summarize that th security implications need to
be documented?
In fact that would make a fine security implications section
in the device specification.
> > > > In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> > > > Its solely decision of host to take action on the host page cache pages.
> > > >
> > > > In case of virtio-pmem, guest does not modify host file directly i.e don't
> > > > perform hole punch & truncation operation directly on host file.
> > >
> > > ... this will no longer be true, and the nuclear landmine in this
> > > driver interface will have been armed....
> >
> > I agree with the need to be careful when / if explicit cache control
> > is added, but that's not the case today.
>
> "if"?
>
> I expect it to be "when", not if. Expect the worst, plan for it now.
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-14 22:21 ` Dave Chinner
(?)
(?)
@ 2019-01-15 2:19 ` Michael S. Tsirkin
-1 siblings, 0 replies; 57+ messages in thread
From: Michael S. Tsirkin @ 2019-01-15 2:19 UTC (permalink / raw)
To: Dave Chinner
Cc: Pankaj Gupta, Jan Kara, KVM list, linux-nvdimm, Qemu Developers,
virtualization, adilger kernel, Ross Zwisler, Eric Blake,
dave jiang, darrick wong, vishal l verma, Matthew Wilcox,
Christoph Hellwig, Linux ACPI, jmoyer, linux-ext4, Rik van Riel,
Stefan Hajnoczi, Igor Mammedov, Dan Williams, lcapitulino,
Nitesh Narayan Lal, Theodore Ts'o, xiaog
On Tue, Jan 15, 2019 at 09:21:32AM +1100, Dave Chinner wrote:
> On Mon, Jan 14, 2019 at 01:35:57PM -0800, Dan Williams wrote:
> > On Mon, Jan 14, 2019 at 1:25 PM Dave Chinner <david@fromorbit.com> wrote:
> > >
> > > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > > >
> > > > > > Until you have images (and hence host page cache) shared between
> > > > > > multiple guests. People will want to do this, because it means they
> > > > > > only need a single set of pages in host memory for executable
> > > > > > binaries rather than a set of pages per guest. Then you have
> > > > > > multiple guests being able to detect residency of the same set of
> > > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > > pages from the host cache, then we have a guest-to-guest information
> > > > > > leak channel.
> > > > >
> > > > > I don't think we should ever be considering something that would allow a
> > > > > guest to evict page's from the host's pagecache [1]. The guest should
> > > > > be able to kick its own references to the host's pagecache out of its
> > > > > own pagecache, but not be able to influence whether the host or another
> > > > > guest has a read-only mapping cached.
> > > > >
> > > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > > truncation, holepunching, etc are going to evict pages from the host's
> > > > > page cache.
> > > >
> > > > This is so correct. Guest does not not evict host page cache pages directly.
> > >
> > > They don't right now.
> > >
> > > But someone is going to end up asking for discard to work so that
> > > the guest can free unused space in the underlying spares image (i.e.
> > > make use of fstrim or mount -o discard) because they have workloads
> > > that have bursts of space usage and they need to trim the image
> > > files afterwards to keep their overall space usage under control.
> > >
> > > And then....
> >
> > ...we reject / push back on that patch citing the above concern.
>
> So at what point do we draw the line?
>
> We're allowing writable DAX mappings, but as I've pointed out that
> means we are going to be allowing a potential information leak via
> files with shared extents to be directly mapped and written to.
>
> But we won't allow useful admin operations that allow better
> management of host side storage space similar to how normal image
> files are used by guests because it's an information leak vector?
>
> That's splitting some really fine hairs there...
May I summarize that th security implications need to
be documented?
In fact that would make a fine security implications section
in the device specification.
> > > > In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> > > > Its solely decision of host to take action on the host page cache pages.
> > > >
> > > > In case of virtio-pmem, guest does not modify host file directly i.e don't
> > > > perform hole punch & truncation operation directly on host file.
> > >
> > > ... this will no longer be true, and the nuclear landmine in this
> > > driver interface will have been armed....
> >
> > I agree with the need to be careful when / if explicit cache control
> > is added, but that's not the case today.
>
> "if"?
>
> I expect it to be "when", not if. Expect the worst, plan for it now.
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-14 21:35 ` Dan Williams
(?)
@ 2019-01-14 22:21 ` Dave Chinner
-1 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-14 22:21 UTC (permalink / raw)
To: Dan Williams
Cc: Jan Kara, KVM list, David Hildenbrand, linux-nvdimm, Jason Wang,
Qemu Developers,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, Ross Zwisler, Eric Blake, darrick wong,
Michael S. Tsirkin, Matthew Wilcox, Christoph Hellwig,
Linux ACPI, linux-ext4, Rik van Riel, Stefan Hajnoczi,
Igor Mammedov, lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, Kevin Wolf,
Nitesh Narayan Lal, Theodore Ts'o, xiaoguangrong eric
On Mon, Jan 14, 2019 at 01:35:57PM -0800, Dan Williams wrote:
> On Mon, Jan 14, 2019 at 1:25 PM Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org> wrote:
> >
> > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > >
> > > > > Until you have images (and hence host page cache) shared between
> > > > > multiple guests. People will want to do this, because it means they
> > > > > only need a single set of pages in host memory for executable
> > > > > binaries rather than a set of pages per guest. Then you have
> > > > > multiple guests being able to detect residency of the same set of
> > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > pages from the host cache, then we have a guest-to-guest information
> > > > > leak channel.
> > > >
> > > > I don't think we should ever be considering something that would allow a
> > > > guest to evict page's from the host's pagecache [1]. The guest should
> > > > be able to kick its own references to the host's pagecache out of its
> > > > own pagecache, but not be able to influence whether the host or another
> > > > guest has a read-only mapping cached.
> > > >
> > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > truncation, holepunching, etc are going to evict pages from the host's
> > > > page cache.
> > >
> > > This is so correct. Guest does not not evict host page cache pages directly.
> >
> > They don't right now.
> >
> > But someone is going to end up asking for discard to work so that
> > the guest can free unused space in the underlying spares image (i.e.
> > make use of fstrim or mount -o discard) because they have workloads
> > that have bursts of space usage and they need to trim the image
> > files afterwards to keep their overall space usage under control.
> >
> > And then....
>
> ...we reject / push back on that patch citing the above concern.
So at what point do we draw the line?
We're allowing writable DAX mappings, but as I've pointed out that
means we are going to be allowing a potential information leak via
files with shared extents to be directly mapped and written to.
But we won't allow useful admin operations that allow better
management of host side storage space similar to how normal image
files are used by guests because it's an information leak vector?
That's splitting some really fine hairs there...
> > > In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> > > Its solely decision of host to take action on the host page cache pages.
> > >
> > > In case of virtio-pmem, guest does not modify host file directly i.e don't
> > > perform hole punch & truncation operation directly on host file.
> >
> > ... this will no longer be true, and the nuclear landmine in this
> > driver interface will have been armed....
>
> I agree with the need to be careful when / if explicit cache control
> is added, but that's not the case today.
"if"?
I expect it to be "when", not if. Expect the worst, plan for it now.
Cheers,
Dave.
--
Dave Chinner
david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-14 22:21 ` Dave Chinner
0 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-14 22:21 UTC (permalink / raw)
To: Dan Williams
Cc: Pankaj Gupta, Matthew Wilcox, Linux Kernel Mailing List,
KVM list, Qemu Developers, linux-nvdimm, linux-fsdevel,
virtualization, Linux ACPI, linux-ext4, linux-xfs, Jan Kara,
Stefan Hajnoczi, Rik van Riel, Nitesh Narayan Lal, Kevin Wolf,
Paolo Bonzini, Ross Zwisler, vishal l verma, dave jiang,
David Hildenbrand, jmoyer, xiaoguangrong eric, Christoph Hellwig,
Michael S. Tsirkin, Jason Wang, lcapitulino, Igor Mammedov,
Eric Blake, Theodore Ts'o, adilger kernel, darrick wong,
Rafael J. Wysocki
On Mon, Jan 14, 2019 at 01:35:57PM -0800, Dan Williams wrote:
> On Mon, Jan 14, 2019 at 1:25 PM Dave Chinner <david@fromorbit.com> wrote:
> >
> > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > >
> > > > > Until you have images (and hence host page cache) shared between
> > > > > multiple guests. People will want to do this, because it means they
> > > > > only need a single set of pages in host memory for executable
> > > > > binaries rather than a set of pages per guest. Then you have
> > > > > multiple guests being able to detect residency of the same set of
> > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > pages from the host cache, then we have a guest-to-guest information
> > > > > leak channel.
> > > >
> > > > I don't think we should ever be considering something that would allow a
> > > > guest to evict page's from the host's pagecache [1]. The guest should
> > > > be able to kick its own references to the host's pagecache out of its
> > > > own pagecache, but not be able to influence whether the host or another
> > > > guest has a read-only mapping cached.
> > > >
> > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > truncation, holepunching, etc are going to evict pages from the host's
> > > > page cache.
> > >
> > > This is so correct. Guest does not not evict host page cache pages directly.
> >
> > They don't right now.
> >
> > But someone is going to end up asking for discard to work so that
> > the guest can free unused space in the underlying spares image (i.e.
> > make use of fstrim or mount -o discard) because they have workloads
> > that have bursts of space usage and they need to trim the image
> > files afterwards to keep their overall space usage under control.
> >
> > And then....
>
> ...we reject / push back on that patch citing the above concern.
So at what point do we draw the line?
We're allowing writable DAX mappings, but as I've pointed out that
means we are going to be allowing a potential information leak via
files with shared extents to be directly mapped and written to.
But we won't allow useful admin operations that allow better
management of host side storage space similar to how normal image
files are used by guests because it's an information leak vector?
That's splitting some really fine hairs there...
> > > In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> > > Its solely decision of host to take action on the host page cache pages.
> > >
> > > In case of virtio-pmem, guest does not modify host file directly i.e don't
> > > perform hole punch & truncation operation directly on host file.
> >
> > ... this will no longer be true, and the nuclear landmine in this
> > driver interface will have been armed....
>
> I agree with the need to be careful when / if explicit cache control
> is added, but that's not the case today.
"if"?
I expect it to be "when", not if. Expect the worst, plan for it now.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-14 22:21 ` Dave Chinner
0 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-14 22:21 UTC (permalink / raw)
To: Dan Williams
Cc: Jan Kara, KVM list, David Hildenbrand, linux-nvdimm, Jason Wang,
Qemu Developers,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, Ross Zwisler, Eric Blake, darrick wong,
Michael S. Tsirkin, Matthew Wilcox, Christoph Hellwig,
Linux ACPI, linux-ext4, Rik van Riel, Stefan Hajnoczi,
Igor Mammedov, lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, Kevin Wolf,
Nitesh Narayan Lal, Theodore Ts'o, xiaoguangrong eric
On Mon, Jan 14, 2019 at 01:35:57PM -0800, Dan Williams wrote:
> On Mon, Jan 14, 2019 at 1:25 PM Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org> wrote:
> >
> > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > >
> > > > > Until you have images (and hence host page cache) shared between
> > > > > multiple guests. People will want to do this, because it means they
> > > > > only need a single set of pages in host memory for executable
> > > > > binaries rather than a set of pages per guest. Then you have
> > > > > multiple guests being able to detect residency of the same set of
> > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > pages from the host cache, then we have a guest-to-guest information
> > > > > leak channel.
> > > >
> > > > I don't think we should ever be considering something that would allow a
> > > > guest to evict page's from the host's pagecache [1]. The guest should
> > > > be able to kick its own references to the host's pagecache out of its
> > > > own pagecache, but not be able to influence whether the host or another
> > > > guest has a read-only mapping cached.
> > > >
> > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > truncation, holepunching, etc are going to evict pages from the host's
> > > > page cache.
> > >
> > > This is so correct. Guest does not not evict host page cache pages directly.
> >
> > They don't right now.
> >
> > But someone is going to end up asking for discard to work so that
> > the guest can free unused space in the underlying spares image (i.e.
> > make use of fstrim or mount -o discard) because they have workloads
> > that have bursts of space usage and they need to trim the image
> > files afterwards to keep their overall space usage under control.
> >
> > And then....
>
> ...we reject / push back on that patch citing the above concern.
So at what point do we draw the line?
We're allowing writable DAX mappings, but as I've pointed out that
means we are going to be allowing a potential information leak via
files with shared extents to be directly mapped and written to.
But we won't allow useful admin operations that allow better
management of host side storage space similar to how normal image
files are used by guests because it's an information leak vector?
That's splitting some really fine hairs there...
> > > In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> > > Its solely decision of host to take action on the host page cache pages.
> > >
> > > In case of virtio-pmem, guest does not modify host file directly i.e don't
> > > perform hole punch & truncation operation directly on host file.
> >
> > ... this will no longer be true, and the nuclear landmine in this
> > driver interface will have been armed....
>
> I agree with the need to be careful when / if explicit cache control
> is added, but that's not the case today.
"if"?
I expect it to be "when", not if. Expect the worst, plan for it now.
Cheers,
Dave.
--
Dave Chinner
david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-14 21:35 ` Dan Williams
(?)
(?)
@ 2019-01-14 22:21 ` Dave Chinner
-1 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-14 22:21 UTC (permalink / raw)
To: Dan Williams
Cc: Pankaj Gupta, Jan Kara, KVM list, linux-nvdimm, Qemu Developers,
virtualization, adilger kernel, Ross Zwisler, Eric Blake,
dave jiang, darrick wong, vishal l verma, Michael S. Tsirkin,
Matthew Wilcox, Christoph Hellwig, Linux ACPI, jmoyer,
linux-ext4, Rik van Riel, Stefan Hajnoczi, Igor Mammedov,
lcapitulino, Nitesh Narayan Lal, Theodore Ts'o, xiaogu
On Mon, Jan 14, 2019 at 01:35:57PM -0800, Dan Williams wrote:
> On Mon, Jan 14, 2019 at 1:25 PM Dave Chinner <david@fromorbit.com> wrote:
> >
> > On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> > >
> > > > > Until you have images (and hence host page cache) shared between
> > > > > multiple guests. People will want to do this, because it means they
> > > > > only need a single set of pages in host memory for executable
> > > > > binaries rather than a set of pages per guest. Then you have
> > > > > multiple guests being able to detect residency of the same set of
> > > > > pages. If the guests can then, in any way, control eviction of the
> > > > > pages from the host cache, then we have a guest-to-guest information
> > > > > leak channel.
> > > >
> > > > I don't think we should ever be considering something that would allow a
> > > > guest to evict page's from the host's pagecache [1]. The guest should
> > > > be able to kick its own references to the host's pagecache out of its
> > > > own pagecache, but not be able to influence whether the host or another
> > > > guest has a read-only mapping cached.
> > > >
> > > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > > truncation, holepunching, etc are going to evict pages from the host's
> > > > page cache.
> > >
> > > This is so correct. Guest does not not evict host page cache pages directly.
> >
> > They don't right now.
> >
> > But someone is going to end up asking for discard to work so that
> > the guest can free unused space in the underlying spares image (i.e.
> > make use of fstrim or mount -o discard) because they have workloads
> > that have bursts of space usage and they need to trim the image
> > files afterwards to keep their overall space usage under control.
> >
> > And then....
>
> ...we reject / push back on that patch citing the above concern.
So at what point do we draw the line?
We're allowing writable DAX mappings, but as I've pointed out that
means we are going to be allowing a potential information leak via
files with shared extents to be directly mapped and written to.
But we won't allow useful admin operations that allow better
management of host side storage space similar to how normal image
files are used by guests because it's an information leak vector?
That's splitting some really fine hairs there...
> > > In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> > > Its solely decision of host to take action on the host page cache pages.
> > >
> > > In case of virtio-pmem, guest does not modify host file directly i.e don't
> > > perform hole punch & truncation operation directly on host file.
> >
> > ... this will no longer be true, and the nuclear landmine in this
> > driver interface will have been armed....
>
> I agree with the need to be careful when / if explicit cache control
> is added, but that's not the case today.
"if"?
I expect it to be "when", not if. Expect the worst, plan for it now.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-14 21:25 ` Dave Chinner
(?)
@ 2019-01-14 21:35 ` Dan Williams
-1 siblings, 0 replies; 57+ messages in thread
From: Dan Williams @ 2019-01-14 21:35 UTC (permalink / raw)
To: Dave Chinner
Cc: Jan Kara, KVM list, David Hildenbrand, linux-nvdimm, Jason Wang,
Qemu Developers,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, Ross Zwisler, Eric Blake, darrick wong,
Michael S. Tsirkin, Matthew Wilcox, Christoph Hellwig,
Linux ACPI, linux-ext4, Rik van Riel, Stefan Hajnoczi,
Igor Mammedov, lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, Kevin Wolf,
Nitesh Narayan Lal, Theodore Ts'o, xiaoguangrong eric
On Mon, Jan 14, 2019 at 1:25 PM Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org> wrote:
>
> On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> >
> > > > Until you have images (and hence host page cache) shared between
> > > > multiple guests. People will want to do this, because it means they
> > > > only need a single set of pages in host memory for executable
> > > > binaries rather than a set of pages per guest. Then you have
> > > > multiple guests being able to detect residency of the same set of
> > > > pages. If the guests can then, in any way, control eviction of the
> > > > pages from the host cache, then we have a guest-to-guest information
> > > > leak channel.
> > >
> > > I don't think we should ever be considering something that would allow a
> > > guest to evict page's from the host's pagecache [1]. The guest should
> > > be able to kick its own references to the host's pagecache out of its
> > > own pagecache, but not be able to influence whether the host or another
> > > guest has a read-only mapping cached.
> > >
> > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > truncation, holepunching, etc are going to evict pages from the host's
> > > page cache.
> >
> > This is so correct. Guest does not not evict host page cache pages directly.
>
> They don't right now.
>
> But someone is going to end up asking for discard to work so that
> the guest can free unused space in the underlying spares image (i.e.
> make use of fstrim or mount -o discard) because they have workloads
> that have bursts of space usage and they need to trim the image
> files afterwards to keep their overall space usage under control.
>
> And then....
...we reject / push back on that patch citing the above concern.
> > In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> > Its solely decision of host to take action on the host page cache pages.
> >
> > In case of virtio-pmem, guest does not modify host file directly i.e don't
> > perform hole punch & truncation operation directly on host file.
>
> ... this will no longer be true, and the nuclear landmine in this
> driver interface will have been armed....
I agree with the need to be careful when / if explicit cache control
is added, but that's not the case today.
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-14 21:35 ` Dan Williams
0 siblings, 0 replies; 57+ messages in thread
From: Dan Williams @ 2019-01-14 21:35 UTC (permalink / raw)
To: Dave Chinner
Cc: Pankaj Gupta, Matthew Wilcox, Linux Kernel Mailing List,
KVM list, Qemu Developers, linux-nvdimm, linux-fsdevel,
virtualization, Linux ACPI, linux-ext4, linux-xfs, Jan Kara,
Stefan Hajnoczi, Rik van Riel, Nitesh Narayan Lal, Kevin Wolf,
Paolo Bonzini, Ross Zwisler, vishal l verma, dave jiang,
David Hildenbrand, jmoyer, xiaoguangrong eric, Christoph Hellwig,
Michael S. Tsirkin, Jason Wang, lcapitulino, Igor Mammedov,
Eric Blake, Theodore Ts'o, adilger kernel, darrick wong,
Rafael J. Wysocki
On Mon, Jan 14, 2019 at 1:25 PM Dave Chinner <david@fromorbit.com> wrote:
>
> On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> >
> > > > Until you have images (and hence host page cache) shared between
> > > > multiple guests. People will want to do this, because it means they
> > > > only need a single set of pages in host memory for executable
> > > > binaries rather than a set of pages per guest. Then you have
> > > > multiple guests being able to detect residency of the same set of
> > > > pages. If the guests can then, in any way, control eviction of the
> > > > pages from the host cache, then we have a guest-to-guest information
> > > > leak channel.
> > >
> > > I don't think we should ever be considering something that would allow a
> > > guest to evict page's from the host's pagecache [1]. The guest should
> > > be able to kick its own references to the host's pagecache out of its
> > > own pagecache, but not be able to influence whether the host or another
> > > guest has a read-only mapping cached.
> > >
> > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > truncation, holepunching, etc are going to evict pages from the host's
> > > page cache.
> >
> > This is so correct. Guest does not not evict host page cache pages directly.
>
> They don't right now.
>
> But someone is going to end up asking for discard to work so that
> the guest can free unused space in the underlying spares image (i.e.
> make use of fstrim or mount -o discard) because they have workloads
> that have bursts of space usage and they need to trim the image
> files afterwards to keep their overall space usage under control.
>
> And then....
...we reject / push back on that patch citing the above concern.
> > In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> > Its solely decision of host to take action on the host page cache pages.
> >
> > In case of virtio-pmem, guest does not modify host file directly i.e don't
> > perform hole punch & truncation operation directly on host file.
>
> ... this will no longer be true, and the nuclear landmine in this
> driver interface will have been armed....
I agree with the need to be careful when / if explicit cache control
is added, but that's not the case today.
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-14 21:35 ` Dan Williams
0 siblings, 0 replies; 57+ messages in thread
From: Dan Williams @ 2019-01-14 21:35 UTC (permalink / raw)
To: Dave Chinner
Cc: Jan Kara, KVM list, David Hildenbrand, linux-nvdimm, Jason Wang,
Qemu Developers,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, Ross Zwisler, Eric Blake, darrick wong,
Michael S. Tsirkin, Matthew Wilcox, Christoph Hellwig,
Linux ACPI, linux-ext4, Rik van Riel, Stefan Hajnoczi,
Igor Mammedov, lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, Kevin Wolf,
Nitesh Narayan Lal, Theodore Ts'o, xiaoguangrong eric
On Mon, Jan 14, 2019 at 1:25 PM Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org> wrote:
>
> On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> >
> > > > Until you have images (and hence host page cache) shared between
> > > > multiple guests. People will want to do this, because it means they
> > > > only need a single set of pages in host memory for executable
> > > > binaries rather than a set of pages per guest. Then you have
> > > > multiple guests being able to detect residency of the same set of
> > > > pages. If the guests can then, in any way, control eviction of the
> > > > pages from the host cache, then we have a guest-to-guest information
> > > > leak channel.
> > >
> > > I don't think we should ever be considering something that would allow a
> > > guest to evict page's from the host's pagecache [1]. The guest should
> > > be able to kick its own references to the host's pagecache out of its
> > > own pagecache, but not be able to influence whether the host or another
> > > guest has a read-only mapping cached.
> > >
> > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > truncation, holepunching, etc are going to evict pages from the host's
> > > page cache.
> >
> > This is so correct. Guest does not not evict host page cache pages directly.
>
> They don't right now.
>
> But someone is going to end up asking for discard to work so that
> the guest can free unused space in the underlying spares image (i.e.
> make use of fstrim or mount -o discard) because they have workloads
> that have bursts of space usage and they need to trim the image
> files afterwards to keep their overall space usage under control.
>
> And then....
...we reject / push back on that patch citing the above concern.
> > In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> > Its solely decision of host to take action on the host page cache pages.
> >
> > In case of virtio-pmem, guest does not modify host file directly i.e don't
> > perform hole punch & truncation operation directly on host file.
>
> ... this will no longer be true, and the nuclear landmine in this
> driver interface will have been armed....
I agree with the need to be careful when / if explicit cache control
is added, but that's not the case today.
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-14 21:25 ` Dave Chinner
(?)
@ 2019-01-14 21:35 ` Dan Williams
-1 siblings, 0 replies; 57+ messages in thread
From: Dan Williams @ 2019-01-14 21:35 UTC (permalink / raw)
To: Dave Chinner
Cc: Pankaj Gupta, Jan Kara, KVM list, linux-nvdimm, Qemu Developers,
virtualization, adilger kernel, Ross Zwisler, Eric Blake,
dave jiang, darrick wong, vishal l verma, Michael S. Tsirkin,
Matthew Wilcox, Christoph Hellwig, Linux ACPI, jmoyer,
linux-ext4, Rik van Riel, Stefan Hajnoczi, Igor Mammedov,
lcapitulino, Nitesh Narayan Lal, Theodore Ts'o, xiaogu
On Mon, Jan 14, 2019 at 1:25 PM Dave Chinner <david@fromorbit.com> wrote:
>
> On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
> >
> > > > Until you have images (and hence host page cache) shared between
> > > > multiple guests. People will want to do this, because it means they
> > > > only need a single set of pages in host memory for executable
> > > > binaries rather than a set of pages per guest. Then you have
> > > > multiple guests being able to detect residency of the same set of
> > > > pages. If the guests can then, in any way, control eviction of the
> > > > pages from the host cache, then we have a guest-to-guest information
> > > > leak channel.
> > >
> > > I don't think we should ever be considering something that would allow a
> > > guest to evict page's from the host's pagecache [1]. The guest should
> > > be able to kick its own references to the host's pagecache out of its
> > > own pagecache, but not be able to influence whether the host or another
> > > guest has a read-only mapping cached.
> > >
> > > [1] Unless the guest is allowed to modify the host's file; obviously
> > > truncation, holepunching, etc are going to evict pages from the host's
> > > page cache.
> >
> > This is so correct. Guest does not not evict host page cache pages directly.
>
> They don't right now.
>
> But someone is going to end up asking for discard to work so that
> the guest can free unused space in the underlying spares image (i.e.
> make use of fstrim or mount -o discard) because they have workloads
> that have bursts of space usage and they need to trim the image
> files afterwards to keep their overall space usage under control.
>
> And then....
...we reject / push back on that patch citing the above concern.
> > In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> > Its solely decision of host to take action on the host page cache pages.
> >
> > In case of virtio-pmem, guest does not modify host file directly i.e don't
> > perform hole punch & truncation operation directly on host file.
>
> ... this will no longer be true, and the nuclear landmine in this
> driver interface will have been armed....
I agree with the need to be careful when / if explicit cache control
is added, but that's not the case today.
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-14 7:15 ` Pankaj Gupta
@ 2019-01-14 21:25 ` Dave Chinner
-1 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-14 21:25 UTC (permalink / raw)
To: Pankaj Gupta
Cc: jack-AlSwsSmVLrQ, kvm-u79uwXL29TY76Z2rM5mHXA,
david-H+wXaHxf7aLQT0dZR+AlfA,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w,
jasowang-H+wXaHxf7aLQT0dZR+AlfA,
qemu-devel-qX2TKyscuCcdnm+yROfE0A,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, zwisler-DgEjT+Ai2ygdnm+yROfE0A,
eblake-H+wXaHxf7aLQT0dZR+AlfA, darrick wong,
mst-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox,
hch-wEGCiKHe2LqWVfeAwA7xHQ, linux-acpi-u79uwXL29TY76Z2rM5mHXA,
linux-ext4-u79uwXL29TY76Z2rM5mHXA, riel-ebMLmSuQjDVBDgjK7y7TUQ,
stefanha-H+wXaHxf7aLQT0dZR+AlfA, imammedo-H+wXaHxf7aLQT0dZR+AlfA,
lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, kwolf-H+wXaHxf7aLQT0dZR+AlfA,
nilal-H+wXaHxf7aLQT0dZR+AlfA, tytso-3s7WtUTddSA,
xiaoguangrong eric, rjw-LthD3rsA81gm4RdzfppkhA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
pbonzini-H+wXaHxf7aLQT0dZR+AlfA
On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
>
> > > Until you have images (and hence host page cache) shared between
> > > multiple guests. People will want to do this, because it means they
> > > only need a single set of pages in host memory for executable
> > > binaries rather than a set of pages per guest. Then you have
> > > multiple guests being able to detect residency of the same set of
> > > pages. If the guests can then, in any way, control eviction of the
> > > pages from the host cache, then we have a guest-to-guest information
> > > leak channel.
> >
> > I don't think we should ever be considering something that would allow a
> > guest to evict page's from the host's pagecache [1]. The guest should
> > be able to kick its own references to the host's pagecache out of its
> > own pagecache, but not be able to influence whether the host or another
> > guest has a read-only mapping cached.
> >
> > [1] Unless the guest is allowed to modify the host's file; obviously
> > truncation, holepunching, etc are going to evict pages from the host's
> > page cache.
>
> This is so correct. Guest does not not evict host page cache pages directly.
They don't right now.
But someone is going to end up asking for discard to work so that
the guest can free unused space in the underlying spares image (i.e.
make use of fstrim or mount -o discard) because they have workloads
that have bursts of space usage and they need to trim the image
files afterwards to keep their overall space usage under control.
And then....
> In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> Its solely decision of host to take action on the host page cache pages.
>
> In case of virtio-pmem, guest does not modify host file directly i.e don't
> perform hole punch & truncation operation directly on host file.
... this will no longer be true, and the nuclear landmine in this
driver interface will have been armed....
Cheers,
Dave.
--
Dave Chinner
david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-14 21:25 ` Dave Chinner
0 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-14 21:25 UTC (permalink / raw)
To: Pankaj Gupta
Cc: Matthew Wilcox, linux-kernel, kvm, qemu-devel, linux-nvdimm,
linux-fsdevel, virtualization, linux-acpi, linux-ext4, linux-xfs,
jack, stefanha, dan j williams, riel, nilal, kwolf, pbonzini,
zwisler, vishal l verma, dave jiang, david, jmoyer,
xiaoguangrong eric, hch, mst, jasowang, lcapitulino, imammedo,
eblake, tytso, adilger kernel, darrick wong, rjw
On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
>
> > > Until you have images (and hence host page cache) shared between
> > > multiple guests. People will want to do this, because it means they
> > > only need a single set of pages in host memory for executable
> > > binaries rather than a set of pages per guest. Then you have
> > > multiple guests being able to detect residency of the same set of
> > > pages. If the guests can then, in any way, control eviction of the
> > > pages from the host cache, then we have a guest-to-guest information
> > > leak channel.
> >
> > I don't think we should ever be considering something that would allow a
> > guest to evict page's from the host's pagecache [1]. The guest should
> > be able to kick its own references to the host's pagecache out of its
> > own pagecache, but not be able to influence whether the host or another
> > guest has a read-only mapping cached.
> >
> > [1] Unless the guest is allowed to modify the host's file; obviously
> > truncation, holepunching, etc are going to evict pages from the host's
> > page cache.
>
> This is so correct. Guest does not not evict host page cache pages directly.
They don't right now.
But someone is going to end up asking for discard to work so that
the guest can free unused space in the underlying spares image (i.e.
make use of fstrim or mount -o discard) because they have workloads
that have bursts of space usage and they need to trim the image
files afterwards to keep their overall space usage under control.
And then....
> In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> Its solely decision of host to take action on the host page cache pages.
>
> In case of virtio-pmem, guest does not modify host file directly i.e don't
> perform hole punch & truncation operation directly on host file.
... this will no longer be true, and the nuclear landmine in this
driver interface will have been armed....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-14 7:15 ` Pankaj Gupta
(?)
@ 2019-01-14 21:25 ` Dave Chinner
-1 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-14 21:25 UTC (permalink / raw)
To: Pankaj Gupta
Cc: jack, kvm, linux-nvdimm, qemu-devel, virtualization,
adilger kernel, zwisler, eblake, dave jiang, darrick wong,
vishal l verma, mst, Matthew Wilcox, hch, linux-acpi, jmoyer,
linux-ext4, riel, stefanha, imammedo, dan j williams,
lcapitulino, nilal, tytso, xiaoguangrong eric, rjw, linux-kernel,
linux-xfs, linux-fsdevel, pbonzini
On Mon, Jan 14, 2019 at 02:15:40AM -0500, Pankaj Gupta wrote:
>
> > > Until you have images (and hence host page cache) shared between
> > > multiple guests. People will want to do this, because it means they
> > > only need a single set of pages in host memory for executable
> > > binaries rather than a set of pages per guest. Then you have
> > > multiple guests being able to detect residency of the same set of
> > > pages. If the guests can then, in any way, control eviction of the
> > > pages from the host cache, then we have a guest-to-guest information
> > > leak channel.
> >
> > I don't think we should ever be considering something that would allow a
> > guest to evict page's from the host's pagecache [1]. The guest should
> > be able to kick its own references to the host's pagecache out of its
> > own pagecache, but not be able to influence whether the host or another
> > guest has a read-only mapping cached.
> >
> > [1] Unless the guest is allowed to modify the host's file; obviously
> > truncation, holepunching, etc are going to evict pages from the host's
> > page cache.
>
> This is so correct. Guest does not not evict host page cache pages directly.
They don't right now.
But someone is going to end up asking for discard to work so that
the guest can free unused space in the underlying spares image (i.e.
make use of fstrim or mount -o discard) because they have workloads
that have bursts of space usage and they need to trim the image
files afterwards to keep their overall space usage under control.
And then....
> In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
> Its solely decision of host to take action on the host page cache pages.
>
> In case of virtio-pmem, guest does not modify host file directly i.e don't
> perform hole punch & truncation operation directly on host file.
... this will no longer be true, and the nuclear landmine in this
driver interface will have been armed....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-13 23:38 ` Matthew Wilcox
@ 2019-01-14 7:15 ` Pankaj Gupta
-1 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-14 7:15 UTC (permalink / raw)
To: Matthew Wilcox
Cc: jack-AlSwsSmVLrQ, kvm-u79uwXL29TY76Z2rM5mHXA,
david-H+wXaHxf7aLQT0dZR+AlfA,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w,
jasowang-H+wXaHxf7aLQT0dZR+AlfA, Dave Chinner,
qemu-devel-qX2TKyscuCcdnm+yROfE0A,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, zwisler-DgEjT+Ai2ygdnm+yROfE0A,
eblake-H+wXaHxf7aLQT0dZR+AlfA, darrick wong,
mst-H+wXaHxf7aLQT0dZR+AlfA, hch-wEGCiKHe2LqWVfeAwA7xHQ,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
linux-ext4-u79uwXL29TY76Z2rM5mHXA, riel-ebMLmSuQjDVBDgjK7y7TUQ,
stefanha-H+wXaHxf7aLQT0dZR+AlfA, imammedo-H+wXaHxf7aLQT0dZR+AlfA,
lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, kwolf-H+wXaHxf7aLQT0dZR+AlfA,
nilal-H+wXaHxf7aLQT0dZR+AlfA, tytso-3s7WtUTddSA,
xiaoguangrong eric, rjw-LthD3rsA81gm4RdzfppkhA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
pbonzini-H+wXaHxf7aLQT0dZR+AlfA
> > Until you have images (and hence host page cache) shared between
> > multiple guests. People will want to do this, because it means they
> > only need a single set of pages in host memory for executable
> > binaries rather than a set of pages per guest. Then you have
> > multiple guests being able to detect residency of the same set of
> > pages. If the guests can then, in any way, control eviction of the
> > pages from the host cache, then we have a guest-to-guest information
> > leak channel.
>
> I don't think we should ever be considering something that would allow a
> guest to evict page's from the host's pagecache [1]. The guest should
> be able to kick its own references to the host's pagecache out of its
> own pagecache, but not be able to influence whether the host or another
> guest has a read-only mapping cached.
>
> [1] Unless the guest is allowed to modify the host's file; obviously
> truncation, holepunching, etc are going to evict pages from the host's
> page cache.
This is so correct. Guest does not not evict host page cache pages directly.
In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
Its solely decision of host to take action on the host page cache pages.
In case of virtio-pmem, guest does not modify host file directly i.e don't
perform hole punch & truncation operation directly on host file.
Thanks,
Pankaj
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-14 7:15 ` Pankaj Gupta
0 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-14 7:15 UTC (permalink / raw)
To: Matthew Wilcox
Cc: Dave Chinner, linux-kernel, kvm, qemu-devel, linux-nvdimm,
linux-fsdevel, virtualization, linux-acpi, linux-ext4, linux-xfs,
jack, stefanha, dan j williams, riel, nilal, kwolf, pbonzini,
zwisler, vishal l verma, dave jiang, david, jmoyer,
xiaoguangrong eric, hch, mst, jasowang, lcapitulino, imammedo,
eblake, tytso, adilger kernel, darrick wong, rjw
> > Until you have images (and hence host page cache) shared between
> > multiple guests. People will want to do this, because it means they
> > only need a single set of pages in host memory for executable
> > binaries rather than a set of pages per guest. Then you have
> > multiple guests being able to detect residency of the same set of
> > pages. If the guests can then, in any way, control eviction of the
> > pages from the host cache, then we have a guest-to-guest information
> > leak channel.
>
> I don't think we should ever be considering something that would allow a
> guest to evict page's from the host's pagecache [1]. The guest should
> be able to kick its own references to the host's pagecache out of its
> own pagecache, but not be able to influence whether the host or another
> guest has a read-only mapping cached.
>
> [1] Unless the guest is allowed to modify the host's file; obviously
> truncation, holepunching, etc are going to evict pages from the host's
> page cache.
This is so correct. Guest does not not evict host page cache pages directly.
In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
Its solely decision of host to take action on the host page cache pages.
In case of virtio-pmem, guest does not modify host file directly i.e don't
perform hole punch & truncation operation directly on host file.
Thanks,
Pankaj
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-13 23:38 ` Matthew Wilcox
` (2 preceding siblings ...)
(?)
@ 2019-01-14 7:15 ` Pankaj Gupta
-1 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-14 7:15 UTC (permalink / raw)
To: Matthew Wilcox
Cc: jack, kvm, linux-nvdimm, Dave Chinner, qemu-devel,
virtualization, adilger kernel, zwisler, eblake, dave jiang,
darrick wong, vishal l verma, mst, hch, linux-acpi, jmoyer,
linux-ext4, riel, stefanha, imammedo, dan j williams,
lcapitulino, nilal, tytso, xiaoguangrong eric, rjw, linux-kernel,
linux-xfs, linux-fsdevel, pbonzini
> > Until you have images (and hence host page cache) shared between
> > multiple guests. People will want to do this, because it means they
> > only need a single set of pages in host memory for executable
> > binaries rather than a set of pages per guest. Then you have
> > multiple guests being able to detect residency of the same set of
> > pages. If the guests can then, in any way, control eviction of the
> > pages from the host cache, then we have a guest-to-guest information
> > leak channel.
>
> I don't think we should ever be considering something that would allow a
> guest to evict page's from the host's pagecache [1]. The guest should
> be able to kick its own references to the host's pagecache out of its
> own pagecache, but not be able to influence whether the host or another
> guest has a read-only mapping cached.
>
> [1] Unless the guest is allowed to modify the host's file; obviously
> truncation, holepunching, etc are going to evict pages from the host's
> page cache.
This is so correct. Guest does not not evict host page cache pages directly.
In case of virtio-pmem & DAX, guest clears guest page cache exceptional entries.
Its solely decision of host to take action on the host page cache pages.
In case of virtio-pmem, guest does not modify host file directly i.e don't
perform hole punch & truncation operation directly on host file.
Thanks,
Pankaj
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-13 23:38 ` Matthew Wilcox
@ 2019-01-14 2:50 ` Dave Chinner
-1 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-14 2:50 UTC (permalink / raw)
To: Matthew Wilcox
Cc: jack-AlSwsSmVLrQ, kvm-u79uwXL29TY76Z2rM5mHXA,
david-H+wXaHxf7aLQT0dZR+AlfA,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w,
jasowang-H+wXaHxf7aLQT0dZR+AlfA,
qemu-devel-qX2TKyscuCcdnm+yROfE0A,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, zwisler-DgEjT+Ai2ygdnm+yROfE0A,
eblake-H+wXaHxf7aLQT0dZR+AlfA, darrick wong,
mst-H+wXaHxf7aLQT0dZR+AlfA, hch-wEGCiKHe2LqWVfeAwA7xHQ,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
linux-ext4-u79uwXL29TY76Z2rM5mHXA, riel-ebMLmSuQjDVBDgjK7y7TUQ,
stefanha-H+wXaHxf7aLQT0dZR+AlfA, imammedo-H+wXaHxf7aLQT0dZR+AlfA,
lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, kwolf-H+wXaHxf7aLQT0dZR+AlfA,
nilal-H+wXaHxf7aLQT0dZR+AlfA, tytso-3s7WtUTddSA,
xiaoguangrong eric, rjw-LthD3rsA81gm4RdzfppkhA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
pbonzini-H+wXaHxf7aLQT0dZR+AlfA
On Sun, Jan 13, 2019 at 03:38:21PM -0800, Matthew Wilcox wrote:
> On Mon, Jan 14, 2019 at 10:29:02AM +1100, Dave Chinner wrote:
> > Until you have images (and hence host page cache) shared between
> > multiple guests. People will want to do this, because it means they
> > only need a single set of pages in host memory for executable
> > binaries rather than a set of pages per guest. Then you have
> > multiple guests being able to detect residency of the same set of
> > pages. If the guests can then, in any way, control eviction of the
> > pages from the host cache, then we have a guest-to-guest information
> > leak channel.
>
> I don't think we should ever be considering something that would allow a
> guest to evict page's from the host's pagecache [1]. The guest should
> be able to kick its own references to the host's pagecache out of its
> own pagecache, but not be able to influence whether the host or another
> guest has a read-only mapping cached.
>
> [1] Unless the guest is allowed to modify the host's file; obviously
> truncation, holepunching, etc are going to evict pages from the host's
> page cache.
Right, and that's exactly what I mean by "we need to be real careful
with functionality like this".
To be honest, I really don't think I've even touched the surface
here.
e.g. Filesystems and storage can share logical and physical extents.
Which means that image files that share storage (e.g. because they
are all cloned from the same master image and/or there's in-line
deduplication running on the storage) and can be directly accessed
by guests may very well be susceptible to detection of host side
deduplication and subsequent copy-on-write operations.
This really doesn't seem much different to me from the guest being
able to infer host side KSM page deduplication and COW operation in
the guest side page cache. The only difference is that DAX is being
used to probe the host side page cache and storage rather than the
guest side.
IOWs, I suspect there's a world of pain waiting for us if we punch
huge holes through the virtual machine abstractions like this.
Improving performance is a laudible goal, but at what price?
Cheers,
Dave.
--
Dave Chinner
david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-14 2:50 ` Dave Chinner
0 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-14 2:50 UTC (permalink / raw)
To: Matthew Wilcox
Cc: Pankaj Gupta, linux-kernel, kvm, qemu-devel, linux-nvdimm,
linux-fsdevel, virtualization, linux-acpi, linux-ext4, linux-xfs,
jack, stefanha, dan j williams, riel, nilal, kwolf, pbonzini,
zwisler, vishal l verma, dave jiang, david, jmoyer,
xiaoguangrong eric, hch, mst, jasowang, lcapitulino, imammedo,
eblake, tytso, adilger kernel, darrick wong, rjw
On Sun, Jan 13, 2019 at 03:38:21PM -0800, Matthew Wilcox wrote:
> On Mon, Jan 14, 2019 at 10:29:02AM +1100, Dave Chinner wrote:
> > Until you have images (and hence host page cache) shared between
> > multiple guests. People will want to do this, because it means they
> > only need a single set of pages in host memory for executable
> > binaries rather than a set of pages per guest. Then you have
> > multiple guests being able to detect residency of the same set of
> > pages. If the guests can then, in any way, control eviction of the
> > pages from the host cache, then we have a guest-to-guest information
> > leak channel.
>
> I don't think we should ever be considering something that would allow a
> guest to evict page's from the host's pagecache [1]. The guest should
> be able to kick its own references to the host's pagecache out of its
> own pagecache, but not be able to influence whether the host or another
> guest has a read-only mapping cached.
>
> [1] Unless the guest is allowed to modify the host's file; obviously
> truncation, holepunching, etc are going to evict pages from the host's
> page cache.
Right, and that's exactly what I mean by "we need to be real careful
with functionality like this".
To be honest, I really don't think I've even touched the surface
here.
e.g. Filesystems and storage can share logical and physical extents.
Which means that image files that share storage (e.g. because they
are all cloned from the same master image and/or there's in-line
deduplication running on the storage) and can be directly accessed
by guests may very well be susceptible to detection of host side
deduplication and subsequent copy-on-write operations.
This really doesn't seem much different to me from the guest being
able to infer host side KSM page deduplication and COW operation in
the guest side page cache. The only difference is that DAX is being
used to probe the host side page cache and storage rather than the
guest side.
IOWs, I suspect there's a world of pain waiting for us if we punch
huge holes through the virtual machine abstractions like this.
Improving performance is a laudible goal, but at what price?
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-13 23:38 ` Matthew Wilcox
(?)
@ 2019-01-14 2:50 ` Dave Chinner
-1 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-14 2:50 UTC (permalink / raw)
To: Matthew Wilcox
Cc: Pankaj Gupta, jack, kvm, linux-nvdimm, qemu-devel,
virtualization, adilger kernel, zwisler, eblake, dave jiang,
darrick wong, vishal l verma, mst, hch, linux-acpi, jmoyer,
linux-ext4, riel, stefanha, imammedo, dan j williams,
lcapitulino, nilal, tytso, xiaoguangrong eric, rjw, linux-kernel,
linux-xfs, linux-fsdevel, pbonzini
On Sun, Jan 13, 2019 at 03:38:21PM -0800, Matthew Wilcox wrote:
> On Mon, Jan 14, 2019 at 10:29:02AM +1100, Dave Chinner wrote:
> > Until you have images (and hence host page cache) shared between
> > multiple guests. People will want to do this, because it means they
> > only need a single set of pages in host memory for executable
> > binaries rather than a set of pages per guest. Then you have
> > multiple guests being able to detect residency of the same set of
> > pages. If the guests can then, in any way, control eviction of the
> > pages from the host cache, then we have a guest-to-guest information
> > leak channel.
>
> I don't think we should ever be considering something that would allow a
> guest to evict page's from the host's pagecache [1]. The guest should
> be able to kick its own references to the host's pagecache out of its
> own pagecache, but not be able to influence whether the host or another
> guest has a read-only mapping cached.
>
> [1] Unless the guest is allowed to modify the host's file; obviously
> truncation, holepunching, etc are going to evict pages from the host's
> page cache.
Right, and that's exactly what I mean by "we need to be real careful
with functionality like this".
To be honest, I really don't think I've even touched the surface
here.
e.g. Filesystems and storage can share logical and physical extents.
Which means that image files that share storage (e.g. because they
are all cloned from the same master image and/or there's in-line
deduplication running on the storage) and can be directly accessed
by guests may very well be susceptible to detection of host side
deduplication and subsequent copy-on-write operations.
This really doesn't seem much different to me from the guest being
able to infer host side KSM page deduplication and COW operation in
the guest side page cache. The only difference is that DAX is being
used to probe the host side page cache and storage rather than the
guest side.
IOWs, I suspect there's a world of pain waiting for us if we punch
huge holes through the virtual machine abstractions like this.
Improving performance is a laudible goal, but at what price?
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-13 23:29 ` Dave Chinner
@ 2019-01-13 23:38 ` Matthew Wilcox
-1 siblings, 0 replies; 57+ messages in thread
From: Matthew Wilcox @ 2019-01-13 23:38 UTC (permalink / raw)
To: Dave Chinner
Cc: jack-AlSwsSmVLrQ, kvm-u79uwXL29TY76Z2rM5mHXA,
david-H+wXaHxf7aLQT0dZR+AlfA,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w,
jasowang-H+wXaHxf7aLQT0dZR+AlfA,
qemu-devel-qX2TKyscuCcdnm+yROfE0A,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, zwisler-DgEjT+Ai2ygdnm+yROfE0A,
eblake-H+wXaHxf7aLQT0dZR+AlfA, darrick wong,
mst-H+wXaHxf7aLQT0dZR+AlfA, hch-wEGCiKHe2LqWVfeAwA7xHQ,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
linux-ext4-u79uwXL29TY76Z2rM5mHXA, riel-ebMLmSuQjDVBDgjK7y7TUQ,
stefanha-H+wXaHxf7aLQT0dZR+AlfA, imammedo-H+wXaHxf7aLQT0dZR+AlfA,
lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, kwolf-H+wXaHxf7aLQT0dZR+AlfA,
nilal-H+wXaHxf7aLQT0dZR+AlfA, tytso-3s7WtUTddSA,
xiaoguangrong eric, rjw-LthD3rsA81gm4RdzfppkhA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
pbonzini-H+wXaHxf7aLQT0dZR+AlfA
On Mon, Jan 14, 2019 at 10:29:02AM +1100, Dave Chinner wrote:
> Until you have images (and hence host page cache) shared between
> multiple guests. People will want to do this, because it means they
> only need a single set of pages in host memory for executable
> binaries rather than a set of pages per guest. Then you have
> multiple guests being able to detect residency of the same set of
> pages. If the guests can then, in any way, control eviction of the
> pages from the host cache, then we have a guest-to-guest information
> leak channel.
I don't think we should ever be considering something that would allow a
guest to evict page's from the host's pagecache [1]. The guest should
be able to kick its own references to the host's pagecache out of its
own pagecache, but not be able to influence whether the host or another
guest has a read-only mapping cached.
[1] Unless the guest is allowed to modify the host's file; obviously
truncation, holepunching, etc are going to evict pages from the host's
page cache.
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-13 23:38 ` Matthew Wilcox
0 siblings, 0 replies; 57+ messages in thread
From: Matthew Wilcox @ 2019-01-13 23:38 UTC (permalink / raw)
To: Dave Chinner
Cc: Pankaj Gupta, linux-kernel, kvm, qemu-devel, linux-nvdimm,
linux-fsdevel, virtualization, linux-acpi, linux-ext4, linux-xfs,
jack, stefanha, dan j williams, riel, nilal, kwolf, pbonzini,
zwisler, vishal l verma, dave jiang, david, jmoyer,
xiaoguangrong eric, hch, mst, jasowang, lcapitulino, imammedo,
eblake, tytso, adilger kernel, darrick wong, rjw
On Mon, Jan 14, 2019 at 10:29:02AM +1100, Dave Chinner wrote:
> Until you have images (and hence host page cache) shared between
> multiple guests. People will want to do this, because it means they
> only need a single set of pages in host memory for executable
> binaries rather than a set of pages per guest. Then you have
> multiple guests being able to detect residency of the same set of
> pages. If the guests can then, in any way, control eviction of the
> pages from the host cache, then we have a guest-to-guest information
> leak channel.
I don't think we should ever be considering something that would allow a
guest to evict page's from the host's pagecache [1]. The guest should
be able to kick its own references to the host's pagecache out of its
own pagecache, but not be able to influence whether the host or another
guest has a read-only mapping cached.
[1] Unless the guest is allowed to modify the host's file; obviously
truncation, holepunching, etc are going to evict pages from the host's
page cache.
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-13 23:29 ` Dave Chinner
(?)
@ 2019-01-13 23:38 ` Matthew Wilcox
-1 siblings, 0 replies; 57+ messages in thread
From: Matthew Wilcox @ 2019-01-13 23:38 UTC (permalink / raw)
To: Dave Chinner
Cc: Pankaj Gupta, jack, kvm, linux-nvdimm, qemu-devel,
virtualization, adilger kernel, zwisler, eblake, dave jiang,
darrick wong, vishal l verma, mst, hch, linux-acpi, jmoyer,
linux-ext4, riel, stefanha, imammedo, dan j williams,
lcapitulino, nilal, tytso, xiaoguangrong eric, rjw, linux-kernel,
linux-xfs, linux-fsdevel, pbonzini
On Mon, Jan 14, 2019 at 10:29:02AM +1100, Dave Chinner wrote:
> Until you have images (and hence host page cache) shared between
> multiple guests. People will want to do this, because it means they
> only need a single set of pages in host memory for executable
> binaries rather than a set of pages per guest. Then you have
> multiple guests being able to detect residency of the same set of
> pages. If the guests can then, in any way, control eviction of the
> pages from the host cache, then we have a guest-to-guest information
> leak channel.
I don't think we should ever be considering something that would allow a
guest to evict page's from the host's pagecache [1]. The guest should
be able to kick its own references to the host's pagecache out of its
own pagecache, but not be able to influence whether the host or another
guest has a read-only mapping cached.
[1] Unless the guest is allowed to modify the host's file; obviously
truncation, holepunching, etc are going to evict pages from the host's
page cache.
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-11 7:45 ` Pankaj Gupta
@ 2019-01-13 23:29 ` Dave Chinner
-1 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-13 23:29 UTC (permalink / raw)
To: Pankaj Gupta
Cc: jack-AlSwsSmVLrQ, kvm-u79uwXL29TY76Z2rM5mHXA,
david-H+wXaHxf7aLQT0dZR+AlfA,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w,
jasowang-H+wXaHxf7aLQT0dZR+AlfA,
qemu-devel-qX2TKyscuCcdnm+yROfE0A,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, zwisler-DgEjT+Ai2ygdnm+yROfE0A,
eblake-H+wXaHxf7aLQT0dZR+AlfA, darrick wong,
mst-H+wXaHxf7aLQT0dZR+AlfA, willy-wEGCiKHe2LqWVfeAwA7xHQ,
hch-wEGCiKHe2LqWVfeAwA7xHQ, linux-acpi-u79uwXL29TY76Z2rM5mHXA,
nilal-H+wXaHxf7aLQT0dZR+AlfA, riel-ebMLmSuQjDVBDgjK7y7TUQ,
stefanha-H+wXaHxf7aLQT0dZR+AlfA, imammedo-H+wXaHxf7aLQT0dZR+AlfA,
lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, kwolf-H+wXaHxf7aLQT0dZR+AlfA,
linux-ext4-u79uwXL29TY76Z2rM5mHXA, tytso-3s7WtUTddSA,
xiaoguangrong eric, rjw-LthD3rsA81gm4RdzfppkhA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
pbonzini-H+wXaHxf7aLQT0dZR+AlfA
On Fri, Jan 11, 2019 at 02:45:04AM -0500, Pankaj Gupta wrote:
>
> >
> > On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > > This patch series has implementation for "virtio pmem".
> > > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > > which allows to bypass the guest page cache. This also
> > > implements a VIRTIO based asynchronous flush mechanism.
> >
> > Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> > like a good idea, but.....
> >
> > This means the guest VM can now run timing attacks to observe host
> > side page cache residency, and depending on the implementation I'm
> > guessing that the guest will be able to control host side page
> > cache eviction, too (e.g. via discard or hole punch operations).
>
> Not sure how? this is similar to mmapping virtual memory by any userspace
> process. Any host userspace process can do such attack on host page cache
> using mincore & mmap shared file.
Mincore is for monitoring, not cached eviction. And it's not
required to observe cache residency, either. That's a wide open
field containing an uncountable number of moles...
> But i don't think guest can do this alone. For virtio-pmem usecase
> guest won't be using page cache so timing attack from only guest
> side is not possible unless host userspace can run checks on page
> cache eviction state using mincore etc. As rightly described by
> Rik, guest will only access its own page cache pages and if guest
> page cache is managed directly by host, this saves alot of effort
> for guest in transferring guest state of page cache.
Until you have images (and hence host page cache) shared between
multiple guests. People will want to do this, because it means they
only need a single set of pages in host memory for executable
binaries rather than a set of pages per guest. Then you have
multiple guests being able to detect residency of the same set of
pages. If the guests can then, in any way, control eviction of the
pages from the host cache, then we have a guest-to-guest information
leak channel.
i.e. it's something we need to be aware of and really careful about
enabling infrastructure that /will/ be abused if guests can find a
way to influence the host side cache residency.
Cheers,
Dave.
--
Dave Chinner
david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-13 23:29 ` Dave Chinner
0 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-13 23:29 UTC (permalink / raw)
To: Pankaj Gupta
Cc: linux-kernel, kvm, qemu-devel, linux-nvdimm, linux-fsdevel,
virtualization, linux-acpi, linux-ext4, linux-xfs, jack,
stefanha, dan j williams, riel, nilal, kwolf, pbonzini, zwisler,
vishal l verma, dave jiang, david, jmoyer, xiaoguangrong eric,
hch, mst, jasowang, lcapitulino, imammedo, eblake, willy, tytso,
adilger kernel, darrick wong, rjw
On Fri, Jan 11, 2019 at 02:45:04AM -0500, Pankaj Gupta wrote:
>
> >
> > On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > > This patch series has implementation for "virtio pmem".
> > > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > > which allows to bypass the guest page cache. This also
> > > implements a VIRTIO based asynchronous flush mechanism.
> >
> > Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> > like a good idea, but.....
> >
> > This means the guest VM can now run timing attacks to observe host
> > side page cache residency, and depending on the implementation I'm
> > guessing that the guest will be able to control host side page
> > cache eviction, too (e.g. via discard or hole punch operations).
>
> Not sure how? this is similar to mmapping virtual memory by any userspace
> process. Any host userspace process can do such attack on host page cache
> using mincore & mmap shared file.
Mincore is for monitoring, not cached eviction. And it's not
required to observe cache residency, either. That's a wide open
field containing an uncountable number of moles...
> But i don't think guest can do this alone. For virtio-pmem usecase
> guest won't be using page cache so timing attack from only guest
> side is not possible unless host userspace can run checks on page
> cache eviction state using mincore etc. As rightly described by
> Rik, guest will only access its own page cache pages and if guest
> page cache is managed directly by host, this saves alot of effort
> for guest in transferring guest state of page cache.
Until you have images (and hence host page cache) shared between
multiple guests. People will want to do this, because it means they
only need a single set of pages in host memory for executable
binaries rather than a set of pages per guest. Then you have
multiple guests being able to detect residency of the same set of
pages. If the guests can then, in any way, control eviction of the
pages from the host cache, then we have a guest-to-guest information
leak channel.
i.e. it's something we need to be aware of and really careful about
enabling infrastructure that /will/ be abused if guests can find a
way to influence the host side cache residency.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-11 7:45 ` Pankaj Gupta
(?)
(?)
@ 2019-01-13 23:29 ` Dave Chinner
-1 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-13 23:29 UTC (permalink / raw)
To: Pankaj Gupta
Cc: jack, kvm, linux-nvdimm, qemu-devel, virtualization,
adilger kernel, zwisler, eblake, dave jiang, darrick wong,
vishal l verma, mst, willy, hch, linux-acpi, jmoyer, nilal, riel,
stefanha, imammedo, dan j williams, lcapitulino, linux-ext4,
tytso, xiaoguangrong eric, rjw, linux-kernel, linux-xfs,
linux-fsdevel, pbonzini
On Fri, Jan 11, 2019 at 02:45:04AM -0500, Pankaj Gupta wrote:
>
> >
> > On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > > This patch series has implementation for "virtio pmem".
> > > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > > which allows to bypass the guest page cache. This also
> > > implements a VIRTIO based asynchronous flush mechanism.
> >
> > Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> > like a good idea, but.....
> >
> > This means the guest VM can now run timing attacks to observe host
> > side page cache residency, and depending on the implementation I'm
> > guessing that the guest will be able to control host side page
> > cache eviction, too (e.g. via discard or hole punch operations).
>
> Not sure how? this is similar to mmapping virtual memory by any userspace
> process. Any host userspace process can do such attack on host page cache
> using mincore & mmap shared file.
Mincore is for monitoring, not cached eviction. And it's not
required to observe cache residency, either. That's a wide open
field containing an uncountable number of moles...
> But i don't think guest can do this alone. For virtio-pmem usecase
> guest won't be using page cache so timing attack from only guest
> side is not possible unless host userspace can run checks on page
> cache eviction state using mincore etc. As rightly described by
> Rik, guest will only access its own page cache pages and if guest
> page cache is managed directly by host, this saves alot of effort
> for guest in transferring guest state of page cache.
Until you have images (and hence host page cache) shared between
multiple guests. People will want to do this, because it means they
only need a single set of pages in host memory for executable
binaries rather than a set of pages per guest. Then you have
multiple guests being able to detect residency of the same set of
pages. If the guests can then, in any way, control eviction of the
pages from the host cache, then we have a guest-to-guest information
leak channel.
i.e. it's something we need to be aware of and really careful about
enabling infrastructure that /will/ be abused if guests can find a
way to influence the host side cache residency.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-13 1:38 ` Pankaj Gupta
@ 2019-01-13 1:43 ` Dan Williams
-1 siblings, 0 replies; 57+ messages in thread
From: Dan Williams @ 2019-01-13 1:43 UTC (permalink / raw)
To: Pankaj Gupta
Cc: Jan Kara, KVM list, David Hildenbrand, linux-nvdimm, Jason Wang,
Dave Chinner, Qemu Developers,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, Ross Zwisler, Eric Blake, darrick wong,
Michael S. Tsirkin, Matthew Wilcox, Christoph Hellwig,
Linux ACPI, linux-ext4, Rik van Riel, Stefan Hajnoczi,
Igor Mammedov, lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, Kevin Wolf,
Nitesh Narayan Lal, Theodore Ts'o, xiaoguangro
On Sat, Jan 12, 2019 at 5:38 PM Pankaj Gupta <pagupta-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>
>
>
> >
> > On Thu 10-01-19 12:26:17, Dave Chinner wrote:
> > > On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > > > This patch series has implementation for "virtio pmem".
> > > > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > > > which allows to bypass the guest page cache. This also
> > > > implements a VIRTIO based asynchronous flush mechanism.
> > >
> > > Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> > > like a good idea, but.....
> > >
> > > This means the guest VM can now run timing attacks to observe host
> > > side page cache residency, and depending on the implementation I'm
> > > guessing that the guest will be able to control host side page
> > > cache eviction, too (e.g. via discard or hole punch operations).
> > >
> > > Which means this functionality looks to me like a new vector for
> > > information leakage into and out of the guest VM via guest
> > > controlled host page cache manipulation.
> > >
> > > https://arxiv.org/pdf/1901.01161
> > >
> > > I might be wrong, but if I'm not we're going to have to be very
> > > careful about how guest VMs can access and manipulate host side
> > > resources like the page cache.....
> >
> > Right. Thinking about this I would be more concerned about the fact that
> > guest can effectively pin amount of host's page cache upto size of the
> > device/file passed to guest as PMEM, can't it Pankaj? Or is there some QEMU
> > magic that avoids this?
>
> Yes, guest will pin these host page cache pages using 'get_user_pages' by
> elevating the page reference count. But these pages can be reclaimed by host
> at any time when there is memory pressure.
Wait, how can the guest pin the host pages? I would expect this to
happen only when using vfio and device assignment. Otherwise, no the
host can't reclaim a pinned page, that's the whole point of a pin to
prevent the mm from reclaiming ownership.
> KVM does not permanently pin pages. vfio does that but we are not using
> it here.
Right, so I'm confused by your pin assertion above.
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-13 1:43 ` Dan Williams
0 siblings, 0 replies; 57+ messages in thread
From: Dan Williams @ 2019-01-13 1:43 UTC (permalink / raw)
To: Pankaj Gupta
Cc: Jan Kara, Dave Chinner, Linux Kernel Mailing List, KVM list,
Qemu Developers, linux-nvdimm, linux-fsdevel, virtualization,
Linux ACPI, linux-ext4, linux-xfs, Stefan Hajnoczi, Rik van Riel,
Nitesh Narayan Lal, Kevin Wolf, Paolo Bonzini, Ross Zwisler,
vishal l verma, dave jiang, David Hildenbrand, jmoyer,
xiaoguangrong eric, Christoph Hellwig, Michael S. Tsirkin,
Jason Wang, lcapitulino, Igor Mammedov, Eric Blake,
Matthew Wilcox, Theodore Ts'o, adilger kernel, darrick wong,
Rafael J. Wysocki
On Sat, Jan 12, 2019 at 5:38 PM Pankaj Gupta <pagupta@redhat.com> wrote:
>
>
>
> >
> > On Thu 10-01-19 12:26:17, Dave Chinner wrote:
> > > On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > > > This patch series has implementation for "virtio pmem".
> > > > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > > > which allows to bypass the guest page cache. This also
> > > > implements a VIRTIO based asynchronous flush mechanism.
> > >
> > > Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> > > like a good idea, but.....
> > >
> > > This means the guest VM can now run timing attacks to observe host
> > > side page cache residency, and depending on the implementation I'm
> > > guessing that the guest will be able to control host side page
> > > cache eviction, too (e.g. via discard or hole punch operations).
> > >
> > > Which means this functionality looks to me like a new vector for
> > > information leakage into and out of the guest VM via guest
> > > controlled host page cache manipulation.
> > >
> > > https://arxiv.org/pdf/1901.01161
> > >
> > > I might be wrong, but if I'm not we're going to have to be very
> > > careful about how guest VMs can access and manipulate host side
> > > resources like the page cache.....
> >
> > Right. Thinking about this I would be more concerned about the fact that
> > guest can effectively pin amount of host's page cache upto size of the
> > device/file passed to guest as PMEM, can't it Pankaj? Or is there some QEMU
> > magic that avoids this?
>
> Yes, guest will pin these host page cache pages using 'get_user_pages' by
> elevating the page reference count. But these pages can be reclaimed by host
> at any time when there is memory pressure.
Wait, how can the guest pin the host pages? I would expect this to
happen only when using vfio and device assignment. Otherwise, no the
host can't reclaim a pinned page, that's the whole point of a pin to
prevent the mm from reclaiming ownership.
> KVM does not permanently pin pages. vfio does that but we are not using
> it here.
Right, so I'm confused by your pin assertion above.
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-13 1:38 ` Pankaj Gupta
(?)
(?)
@ 2019-01-13 1:43 ` Dan Williams
-1 siblings, 0 replies; 57+ messages in thread
From: Dan Williams @ 2019-01-13 1:43 UTC (permalink / raw)
To: Pankaj Gupta
Cc: Jan Kara, KVM list, linux-nvdimm, Dave Chinner, Qemu Developers,
virtualization, adilger kernel, Ross Zwisler, Eric Blake,
dave jiang, darrick wong, vishal l verma, Michael S. Tsirkin,
Matthew Wilcox, Christoph Hellwig, Linux ACPI, jmoyer,
linux-ext4, Rik van Riel, Stefan Hajnoczi, Igor Mammedov,
lcapitulino, Nitesh Narayan Lal, Theodore Ts'o, xiao
On Sat, Jan 12, 2019 at 5:38 PM Pankaj Gupta <pagupta@redhat.com> wrote:
>
>
>
> >
> > On Thu 10-01-19 12:26:17, Dave Chinner wrote:
> > > On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > > > This patch series has implementation for "virtio pmem".
> > > > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > > > which allows to bypass the guest page cache. This also
> > > > implements a VIRTIO based asynchronous flush mechanism.
> > >
> > > Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> > > like a good idea, but.....
> > >
> > > This means the guest VM can now run timing attacks to observe host
> > > side page cache residency, and depending on the implementation I'm
> > > guessing that the guest will be able to control host side page
> > > cache eviction, too (e.g. via discard or hole punch operations).
> > >
> > > Which means this functionality looks to me like a new vector for
> > > information leakage into and out of the guest VM via guest
> > > controlled host page cache manipulation.
> > >
> > > https://arxiv.org/pdf/1901.01161
> > >
> > > I might be wrong, but if I'm not we're going to have to be very
> > > careful about how guest VMs can access and manipulate host side
> > > resources like the page cache.....
> >
> > Right. Thinking about this I would be more concerned about the fact that
> > guest can effectively pin amount of host's page cache upto size of the
> > device/file passed to guest as PMEM, can't it Pankaj? Or is there some QEMU
> > magic that avoids this?
>
> Yes, guest will pin these host page cache pages using 'get_user_pages' by
> elevating the page reference count. But these pages can be reclaimed by host
> at any time when there is memory pressure.
Wait, how can the guest pin the host pages? I would expect this to
happen only when using vfio and device assignment. Otherwise, no the
host can't reclaim a pinned page, that's the whole point of a pin to
prevent the mm from reclaiming ownership.
> KVM does not permanently pin pages. vfio does that but we are not using
> it here.
Right, so I'm confused by your pin assertion above.
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-10 10:17 ` Jan Kara
@ 2019-01-13 1:38 ` Pankaj Gupta
-1 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-13 1:38 UTC (permalink / raw)
To: Jan Kara
Cc: kvm-u79uwXL29TY76Z2rM5mHXA, david-H+wXaHxf7aLQT0dZR+AlfA,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w,
jasowang-H+wXaHxf7aLQT0dZR+AlfA, Dave Chinner,
qemu-devel-qX2TKyscuCcdnm+yROfE0A,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger kernel, zwisler-DgEjT+Ai2ygdnm+yROfE0A,
eblake-H+wXaHxf7aLQT0dZR+AlfA, darrick wong,
mst-H+wXaHxf7aLQT0dZR+AlfA, willy-wEGCiKHe2LqWVfeAwA7xHQ,
hch-wEGCiKHe2LqWVfeAwA7xHQ, linux-acpi-u79uwXL29TY76Z2rM5mHXA,
linux-ext4-u79uwXL29TY76Z2rM5mHXA, riel-ebMLmSuQjDVBDgjK7y7TUQ,
stefanha-H+wXaHxf7aLQT0dZR+AlfA, imammedo-H+wXaHxf7aLQT0dZR+AlfA,
lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, kwolf-H+wXaHxf7aLQT0dZR+AlfA,
nilal-H+wXaHxf7aLQT0dZR+AlfA, tytso-3s7WtUTddSA,
xiaoguangrong eric, rjw-LthD3rsA81gm4RdzfppkhA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
pbonzini-H+wXaHxf7aLQT0dZR+AlfA
>
> On Thu 10-01-19 12:26:17, Dave Chinner wrote:
> > On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > > This patch series has implementation for "virtio pmem".
> > > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > > which allows to bypass the guest page cache. This also
> > > implements a VIRTIO based asynchronous flush mechanism.
> >
> > Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> > like a good idea, but.....
> >
> > This means the guest VM can now run timing attacks to observe host
> > side page cache residency, and depending on the implementation I'm
> > guessing that the guest will be able to control host side page
> > cache eviction, too (e.g. via discard or hole punch operations).
> >
> > Which means this functionality looks to me like a new vector for
> > information leakage into and out of the guest VM via guest
> > controlled host page cache manipulation.
> >
> > https://arxiv.org/pdf/1901.01161
> >
> > I might be wrong, but if I'm not we're going to have to be very
> > careful about how guest VMs can access and manipulate host side
> > resources like the page cache.....
>
> Right. Thinking about this I would be more concerned about the fact that
> guest can effectively pin amount of host's page cache upto size of the
> device/file passed to guest as PMEM, can't it Pankaj? Or is there some QEMU
> magic that avoids this?
Yes, guest will pin these host page cache pages using 'get_user_pages' by
elevating the page reference count. But these pages can be reclaimed by host
at any time when there is memory pressure.
KVM does not permanently pin pages. vfio does that but we are not using
it here.
Could you please elaborate what you are thinking?
Thanks,
Pankaj
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-13 1:38 ` Pankaj Gupta
0 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-13 1:38 UTC (permalink / raw)
To: Jan Kara
Cc: Dave Chinner, linux-kernel, kvm, qemu-devel, linux-nvdimm,
linux-fsdevel, virtualization, linux-acpi, linux-ext4, linux-xfs,
stefanha, dan j williams, riel, nilal, kwolf, pbonzini, zwisler,
vishal l verma, dave jiang, david, jmoyer, xiaoguangrong eric,
hch, mst, jasowang, lcapitulino, imammedo, eblake, willy, tytso,
adilger kernel, darrick wong, rjw
>
> On Thu 10-01-19 12:26:17, Dave Chinner wrote:
> > On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > > This patch series has implementation for "virtio pmem".
> > > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > > which allows to bypass the guest page cache. This also
> > > implements a VIRTIO based asynchronous flush mechanism.
> >
> > Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> > like a good idea, but.....
> >
> > This means the guest VM can now run timing attacks to observe host
> > side page cache residency, and depending on the implementation I'm
> > guessing that the guest will be able to control host side page
> > cache eviction, too (e.g. via discard or hole punch operations).
> >
> > Which means this functionality looks to me like a new vector for
> > information leakage into and out of the guest VM via guest
> > controlled host page cache manipulation.
> >
> > https://arxiv.org/pdf/1901.01161
> >
> > I might be wrong, but if I'm not we're going to have to be very
> > careful about how guest VMs can access and manipulate host side
> > resources like the page cache.....
>
> Right. Thinking about this I would be more concerned about the fact that
> guest can effectively pin amount of host's page cache upto size of the
> device/file passed to guest as PMEM, can't it Pankaj? Or is there some QEMU
> magic that avoids this?
Yes, guest will pin these host page cache pages using 'get_user_pages' by
elevating the page reference count. But these pages can be reclaimed by host
at any time when there is memory pressure.
KVM does not permanently pin pages. vfio does that but we are not using
it here.
Could you please elaborate what you are thinking?
Thanks,
Pankaj
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-10 10:17 ` Jan Kara
(?)
(?)
@ 2019-01-13 1:38 ` Pankaj Gupta
-1 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-13 1:38 UTC (permalink / raw)
To: Jan Kara
Cc: kvm, linux-nvdimm, Dave Chinner, qemu-devel, virtualization,
adilger kernel, zwisler, eblake, dave jiang, darrick wong,
vishal l verma, mst, willy, hch, linux-acpi, jmoyer, linux-ext4,
riel, stefanha, imammedo, dan j williams, lcapitulino, nilal,
tytso, xiaoguangrong eric, rjw, linux-kernel, linux-xfs,
linux-fsdevel, pbonzini
>
> On Thu 10-01-19 12:26:17, Dave Chinner wrote:
> > On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > > This patch series has implementation for "virtio pmem".
> > > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > > which allows to bypass the guest page cache. This also
> > > implements a VIRTIO based asynchronous flush mechanism.
> >
> > Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> > like a good idea, but.....
> >
> > This means the guest VM can now run timing attacks to observe host
> > side page cache residency, and depending on the implementation I'm
> > guessing that the guest will be able to control host side page
> > cache eviction, too (e.g. via discard or hole punch operations).
> >
> > Which means this functionality looks to me like a new vector for
> > information leakage into and out of the guest VM via guest
> > controlled host page cache manipulation.
> >
> > https://arxiv.org/pdf/1901.01161
> >
> > I might be wrong, but if I'm not we're going to have to be very
> > careful about how guest VMs can access and manipulate host side
> > resources like the page cache.....
>
> Right. Thinking about this I would be more concerned about the fact that
> guest can effectively pin amount of host's page cache upto size of the
> device/file passed to guest as PMEM, can't it Pankaj? Or is there some QEMU
> magic that avoids this?
Yes, guest will pin these host page cache pages using 'get_user_pages' by
elevating the page reference count. But these pages can be reclaimed by host
at any time when there is memory pressure.
KVM does not permanently pin pages. vfio does that but we are not using
it here.
Could you please elaborate what you are thinking?
Thanks,
Pankaj
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-10 1:26 ` Dave Chinner
(?)
@ 2019-01-11 7:45 ` Pankaj Gupta
-1 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-11 7:45 UTC (permalink / raw)
To: Dave Chinner
Cc: linux-kernel, kvm, qemu-devel, linux-nvdimm, linux-fsdevel,
virtualization, linux-acpi, linux-ext4, linux-xfs, jack,
stefanha, dan j williams, riel, nilal, kwolf, pbonzini, zwisler,
vishal l verma, dave jiang, david, jmoyer, xiaoguangrong eric,
hch, mst, jasowang, lcapitulino, imammedo, eblake, willy, tytso,
adilger kernel
>
> On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > This patch series has implementation for "virtio pmem".
> > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > which allows to bypass the guest page cache. This also
> > implements a VIRTIO based asynchronous flush mechanism.
>
> Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> like a good idea, but.....
>
> This means the guest VM can now run timing attacks to observe host
> side page cache residency, and depending on the implementation I'm
> guessing that the guest will be able to control host side page
> cache eviction, too (e.g. via discard or hole punch operations).
Not sure how? this is similar to mmapping virtual memory by any userspace
process. Any host userspace process can do such attack on host page cache
using mincore & mmap shared file.
But i don't think guest can do this alone. For virtio-pmem usecase guest
won't be using page cache so timing attack from only guest side is not
possible unless host userspace can run checks on page cache eviction state
using mincore etc.
As rightly described by Rik, guest will only access its own page cache pages
and if guest page cache is managed directly by host, this saves alot of
effort for guest in transferring guest state of page cache.
>
> Which means this functionality looks to me like a new vector for
> information leakage into and out of the guest VM via guest
> controlled host page cache manipulation.
>
> https://arxiv.org/pdf/1901.01161
>
> I might be wrong, but if I'm not we're going to have to be very
> careful about how guest VMs can access and manipulate host side
> resources like the page cache.....
If I am following correctly the discussions in MM thread.
Important steps to mitigate this:
* Avoid running mincore in privilege mode: to safeguard page evict state of any
page cache page.
* tweaking RWF_NOWAIT
I think if we secure ways to find current state(cached/evicted) of a page in host,
we should be able to mitigate the impact for any page cache page access attack
including virtio-pmem.
Thanks,
Pankaj
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-11 7:45 ` Pankaj Gupta
0 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-11 7:45 UTC (permalink / raw)
To: Dave Chinner
Cc: linux-kernel, kvm, qemu-devel, linux-nvdimm, linux-fsdevel,
virtualization, linux-acpi, linux-ext4, linux-xfs, jack,
stefanha, dan j williams, riel, nilal, kwolf, pbonzini, zwisler,
vishal l verma, dave jiang, david, jmoyer, xiaoguangrong eric,
hch, mst, jasowang, lcapitulino, imammedo, eblake, willy, tytso,
adilger kernel, darrick wong, rjw
>
> On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > This patch series has implementation for "virtio pmem".
> > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > which allows to bypass the guest page cache. This also
> > implements a VIRTIO based asynchronous flush mechanism.
>
> Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> like a good idea, but.....
>
> This means the guest VM can now run timing attacks to observe host
> side page cache residency, and depending on the implementation I'm
> guessing that the guest will be able to control host side page
> cache eviction, too (e.g. via discard or hole punch operations).
Not sure how? this is similar to mmapping virtual memory by any userspace
process. Any host userspace process can do such attack on host page cache
using mincore & mmap shared file.
But i don't think guest can do this alone. For virtio-pmem usecase guest
won't be using page cache so timing attack from only guest side is not
possible unless host userspace can run checks on page cache eviction state
using mincore etc.
As rightly described by Rik, guest will only access its own page cache pages
and if guest page cache is managed directly by host, this saves alot of
effort for guest in transferring guest state of page cache.
>
> Which means this functionality looks to me like a new vector for
> information leakage into and out of the guest VM via guest
> controlled host page cache manipulation.
>
> https://arxiv.org/pdf/1901.01161
>
> I might be wrong, but if I'm not we're going to have to be very
> careful about how guest VMs can access and manipulate host side
> resources like the page cache.....
If I am following correctly the discussions in MM thread.
Important steps to mitigate this:
* Avoid running mincore in privilege mode: to safeguard page evict state of any
page cache page.
* tweaking RWF_NOWAIT
I think if we secure ways to find current state(cached/evicted) of a page in host,
we should be able to mitigate the impact for any page cache page access attack
including virtio-pmem.
Thanks,
Pankaj
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-11 7:45 ` Pankaj Gupta
0 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-11 7:45 UTC (permalink / raw)
To: Dave Chinner
Cc: linux-kernel, kvm, qemu-devel, linux-nvdimm, linux-fsdevel,
virtualization, linux-acpi, linux-ext4, linux-xfs, jack,
stefanha, dan j williams, riel, nilal, kwolf, pbonzini, zwisler,
vishal l verma, dave jiang, david, jmoyer, xiaoguangrong eric,
hch, mst, jasowang, lcapitulino, imammedo, eblake, willy, tytso,
adilger kernel
>
> On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > This patch series has implementation for "virtio pmem".
> > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > which allows to bypass the guest page cache. This also
> > implements a VIRTIO based asynchronous flush mechanism.
>
> Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> like a good idea, but.....
>
> This means the guest VM can now run timing attacks to observe host
> side page cache residency, and depending on the implementation I'm
> guessing that the guest will be able to control host side page
> cache eviction, too (e.g. via discard or hole punch operations).
Not sure how? this is similar to mmapping virtual memory by any userspace
process. Any host userspace process can do such attack on host page cache
using mincore & mmap shared file.
But i don't think guest can do this alone. For virtio-pmem usecase guest
won't be using page cache so timing attack from only guest side is not
possible unless host userspace can run checks on page cache eviction state
using mincore etc.
As rightly described by Rik, guest will only access its own page cache pages
and if guest page cache is managed directly by host, this saves alot of
effort for guest in transferring guest state of page cache.
>
> Which means this functionality looks to me like a new vector for
> information leakage into and out of the guest VM via guest
> controlled host page cache manipulation.
>
> https://arxiv.org/pdf/1901.01161
>
> I might be wrong, but if I'm not we're going to have to be very
> careful about how guest VMs can access and manipulate host side
> resources like the page cache.....
If I am following correctly the discussions in MM thread.
Important steps to mitigate this:
* Avoid running mincore in privilege mode: to safeguard page evict state of any
page cache page.
* tweaking RWF_NOWAIT
I think if we secure ways to find current state(cached/evicted) of a page in host,
we should be able to mitigate the impact for any page cache page access attack
including virtio-pmem.
Thanks,
Pankaj
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-10 1:26 ` Dave Chinner
` (3 preceding siblings ...)
(?)
@ 2019-01-11 7:45 ` Pankaj Gupta
-1 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-11 7:45 UTC (permalink / raw)
To: Dave Chinner
Cc: jack, kvm, linux-nvdimm, qemu-devel, virtualization,
adilger kernel, zwisler, eblake, dave jiang, darrick wong,
vishal l verma, mst, willy, hch, linux-acpi, jmoyer, nilal, riel,
stefanha, imammedo, dan j williams, lcapitulino, linux-ext4,
tytso, xiaoguangrong eric, rjw, linux-kernel, linux-xfs,
linux-fsdevel, pbonzini
>
> On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > This patch series has implementation for "virtio pmem".
> > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > which allows to bypass the guest page cache. This also
> > implements a VIRTIO based asynchronous flush mechanism.
>
> Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> like a good idea, but.....
>
> This means the guest VM can now run timing attacks to observe host
> side page cache residency, and depending on the implementation I'm
> guessing that the guest will be able to control host side page
> cache eviction, too (e.g. via discard or hole punch operations).
Not sure how? this is similar to mmapping virtual memory by any userspace
process. Any host userspace process can do such attack on host page cache
using mincore & mmap shared file.
But i don't think guest can do this alone. For virtio-pmem usecase guest
won't be using page cache so timing attack from only guest side is not
possible unless host userspace can run checks on page cache eviction state
using mincore etc.
As rightly described by Rik, guest will only access its own page cache pages
and if guest page cache is managed directly by host, this saves alot of
effort for guest in transferring guest state of page cache.
>
> Which means this functionality looks to me like a new vector for
> information leakage into and out of the guest VM via guest
> controlled host page cache manipulation.
>
> https://arxiv.org/pdf/1901.01161
>
> I might be wrong, but if I'm not we're going to have to be very
> careful about how guest VMs can access and manipulate host side
> resources like the page cache.....
If I am following correctly the discussions in MM thread.
Important steps to mitigate this:
* Avoid running mincore in privilege mode: to safeguard page evict state of any
page cache page.
* tweaking RWF_NOWAIT
I think if we secure ways to find current state(cached/evicted) of a page in host,
we should be able to mitigate the impact for any page cache page access attack
including virtio-pmem.
Thanks,
Pankaj
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-10 1:26 ` Dave Chinner
@ 2019-01-10 10:17 ` Jan Kara
-1 siblings, 0 replies; 57+ messages in thread
From: Jan Kara @ 2019-01-10 10:17 UTC (permalink / raw)
To: Dave Chinner
Cc: jack-AlSwsSmVLrQ, kvm-u79uwXL29TY76Z2rM5mHXA,
david-H+wXaHxf7aLQT0dZR+AlfA,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w,
jasowang-H+wXaHxf7aLQT0dZR+AlfA,
qemu-devel-qX2TKyscuCcdnm+yROfE0A,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger.kernel-m1MBpc4rdrD3fQ9qLvQP4Q,
zwisler-DgEjT+Ai2ygdnm+yROfE0A, eblake-H+wXaHxf7aLQT0dZR+AlfA,
darrick.wong-QHcLZuEGTsvQT0dZR+AlfA, mst-H+wXaHxf7aLQT0dZR+AlfA,
willy-wEGCiKHe2LqWVfeAwA7xHQ, hch-wEGCiKHe2LqWVfeAwA7xHQ,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
linux-ext4-u79uwXL29TY76Z2rM5mHXA, riel-ebMLmSuQjDVBDgjK7y7TUQ,
stefanha-H+wXaHxf7aLQT0dZR+AlfA, imammedo-H+wXaHxf7aLQT0dZR+AlfA,
lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, kwolf-H+wXaHxf7aLQT0dZR+AlfA,
nilal-H+wXaHxf7aLQT0dZR+AlfA, tytso-3s7WtUTddSA,
xiaoguangrong.eric-Re5JQEeQqe8AvxtiuMwx3w,
rjw-LthD3rsA81gm4RdzfppkhA, linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
pbonzini-H+wXaHxf7aLQT0dZR+AlfA
On Thu 10-01-19 12:26:17, Dave Chinner wrote:
> On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > This patch series has implementation for "virtio pmem".
> > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > which allows to bypass the guest page cache. This also
> > implements a VIRTIO based asynchronous flush mechanism.
>
> Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> like a good idea, but.....
>
> This means the guest VM can now run timing attacks to observe host
> side page cache residency, and depending on the implementation I'm
> guessing that the guest will be able to control host side page
> cache eviction, too (e.g. via discard or hole punch operations).
>
> Which means this functionality looks to me like a new vector for
> information leakage into and out of the guest VM via guest
> controlled host page cache manipulation.
>
> https://arxiv.org/pdf/1901.01161
>
> I might be wrong, but if I'm not we're going to have to be very
> careful about how guest VMs can access and manipulate host side
> resources like the page cache.....
Right. Thinking about this I would be more concerned about the fact that
guest can effectively pin amount of host's page cache upto size of the
device/file passed to guest as PMEM, can't it Pankaj? Or is there some QEMU
magic that avoids this?
Honza
--
Jan Kara <jack-IBi9RG/b67k@public.gmane.org>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-10 10:17 ` Jan Kara
0 siblings, 0 replies; 57+ messages in thread
From: Jan Kara @ 2019-01-10 10:17 UTC (permalink / raw)
To: Dave Chinner
Cc: Pankaj Gupta, linux-kernel, kvm, qemu-devel, linux-nvdimm,
linux-fsdevel, virtualization, linux-acpi, linux-ext4, linux-xfs,
jack, stefanha, dan.j.williams, riel, nilal, kwolf, pbonzini,
zwisler, vishal.l.verma, dave.jiang, david, jmoyer,
xiaoguangrong.eric, hch, mst, jasowang, lcapitulino, imammedo,
eblake, willy, tytso, adilger.kernel, darrick.wong, rjw
On Thu 10-01-19 12:26:17, Dave Chinner wrote:
> On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > This patch series has implementation for "virtio pmem".
> > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > which allows to bypass the guest page cache. This also
> > implements a VIRTIO based asynchronous flush mechanism.
>
> Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> like a good idea, but.....
>
> This means the guest VM can now run timing attacks to observe host
> side page cache residency, and depending on the implementation I'm
> guessing that the guest will be able to control host side page
> cache eviction, too (e.g. via discard or hole punch operations).
>
> Which means this functionality looks to me like a new vector for
> information leakage into and out of the guest VM via guest
> controlled host page cache manipulation.
>
> https://arxiv.org/pdf/1901.01161
>
> I might be wrong, but if I'm not we're going to have to be very
> careful about how guest VMs can access and manipulate host side
> resources like the page cache.....
Right. Thinking about this I would be more concerned about the fact that
guest can effectively pin amount of host's page cache upto size of the
device/file passed to guest as PMEM, can't it Pankaj? Or is there some QEMU
magic that avoids this?
Honza
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-10 1:26 ` Dave Chinner
(?)
(?)
@ 2019-01-10 10:17 ` Jan Kara
-1 siblings, 0 replies; 57+ messages in thread
From: Jan Kara @ 2019-01-10 10:17 UTC (permalink / raw)
To: Dave Chinner
Cc: Pankaj Gupta, jack, kvm, linux-nvdimm, qemu-devel,
virtualization, adilger.kernel, zwisler, eblake, dave.jiang,
darrick.wong, vishal.l.verma, mst, willy, hch, linux-acpi,
jmoyer, linux-ext4, riel, stefanha, imammedo, dan.j.williams,
lcapitulino, nilal, tytso, xiaoguangrong.eric, rjw, linux-kernel,
linux-xfs, linux-fsdevel, pbonzini
On Thu 10-01-19 12:26:17, Dave Chinner wrote:
> On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > This patch series has implementation for "virtio pmem".
> > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > which allows to bypass the guest page cache. This also
> > implements a VIRTIO based asynchronous flush mechanism.
>
> Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> like a good idea, but.....
>
> This means the guest VM can now run timing attacks to observe host
> side page cache residency, and depending on the implementation I'm
> guessing that the guest will be able to control host side page
> cache eviction, too (e.g. via discard or hole punch operations).
>
> Which means this functionality looks to me like a new vector for
> information leakage into and out of the guest VM via guest
> controlled host page cache manipulation.
>
> https://arxiv.org/pdf/1901.01161
>
> I might be wrong, but if I'm not we're going to have to be very
> careful about how guest VMs can access and manipulate host side
> resources like the page cache.....
Right. Thinking about this I would be more concerned about the fact that
guest can effectively pin amount of host's page cache upto size of the
device/file passed to guest as PMEM, can't it Pankaj? Or is there some QEMU
magic that avoids this?
Honza
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-10 1:26 ` Dave Chinner
(?)
@ 2019-01-10 2:40 ` Rik van Riel
-1 siblings, 0 replies; 57+ messages in thread
From: Rik van Riel @ 2019-01-10 2:40 UTC (permalink / raw)
To: Dave Chinner, Pankaj Gupta
Cc: linux-kernel, kvm, qemu-devel, linux-nvdimm, linux-fsdevel,
virtualization, linux-acpi, linux-ext4, linux-xfs, jack,
stefanha, dan.j.williams, nilal, kwolf, pbonzini, zwisler,
vishal.l.verma, dave.jiang, david, jmoyer, xiaoguangrong.eric,
hch, mst, jasowang, lcapitulino, imammedo, eblake, willy, tytso,
adilger.kernel, darrick.wong, rjw
[-- Attachment #1: Type: text/plain, Size: 1243 bytes --]
On Thu, 2019-01-10 at 12:26 +1100, Dave Chinner wrote:
> On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> > This patch series has implementation for "virtio pmem".
> > "virtio pmem" is fake persistent memory(nvdimm) in guest
> > which allows to bypass the guest page cache. This also
> > implements a VIRTIO based asynchronous flush mechanism.
>
> Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
> like a good idea, but.....
>
> This means the guest VM can now run timing attacks to observe host
> side page cache residency, and depending on the implementation I'm
> guessing that the guest will be able to control host side page
> cache eviction, too (e.g. via discard or hole punch operations).
Even if the guest can only access its own disk image,
using its own page cache pages, Pankaj's patches could
still be a win because they allow the host kernel to
easily evict page cache pages without doing IO.
Evicting page cache pages that live inside a guest
involves something like swapping, ballooning, or some
sort of callback into the guest to figure out whether
a page is clean.
Evicting page cache pages that live in the host, OTOH...
--
All Rights Reversed.
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-09 14:47 Pankaj Gupta
@ 2019-01-10 1:26 ` Dave Chinner
[not found] ` <20190109144736.17452-1-pagupta-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
1 sibling, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-10 1:26 UTC (permalink / raw)
To: Pankaj Gupta
Cc: jack-AlSwsSmVLrQ, kvm-u79uwXL29TY76Z2rM5mHXA,
david-H+wXaHxf7aLQT0dZR+AlfA,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w,
jasowang-H+wXaHxf7aLQT0dZR+AlfA,
qemu-devel-qX2TKyscuCcdnm+yROfE0A,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
adilger.kernel-m1MBpc4rdrD3fQ9qLvQP4Q,
zwisler-DgEjT+Ai2ygdnm+yROfE0A, eblake-H+wXaHxf7aLQT0dZR+AlfA,
darrick.wong-QHcLZuEGTsvQT0dZR+AlfA, mst-H+wXaHxf7aLQT0dZR+AlfA,
willy-wEGCiKHe2LqWVfeAwA7xHQ, hch-wEGCiKHe2LqWVfeAwA7xHQ,
linux-acpi-u79uwXL29TY76Z2rM5mHXA, nilal-H+wXaHxf7aLQT0dZR+AlfA,
riel-ebMLmSuQjDVBDgjK7y7TUQ, stefanha-H+wXaHxf7aLQT0dZR+AlfA,
imammedo-H+wXaHxf7aLQT0dZR+AlfA,
lcapitulino-H+wXaHxf7aLQT0dZR+AlfA, kwolf-H+wXaHxf7aLQT0dZR+AlfA,
linux-ext4-u79uwXL29TY76Z2rM5mHXA, tytso-3s7WtUTddSA,
xiaoguangrong.eric-Re5JQEeQqe8AvxtiuMwx3w,
rjw-LthD3rsA81gm4RdzfppkhA, linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
pbonzini-H+wXaHxf7aLQT0dZR+AlfA
On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> This patch series has implementation for "virtio pmem".
> "virtio pmem" is fake persistent memory(nvdimm) in guest
> which allows to bypass the guest page cache. This also
> implements a VIRTIO based asynchronous flush mechanism.
Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
like a good idea, but.....
This means the guest VM can now run timing attacks to observe host
side page cache residency, and depending on the implementation I'm
guessing that the guest will be able to control host side page
cache eviction, too (e.g. via discard or hole punch operations).
Which means this functionality looks to me like a new vector for
information leakage into and out of the guest VM via guest
controlled host page cache manipulation.
https://arxiv.org/pdf/1901.01161
I might be wrong, but if I'm not we're going to have to be very
careful about how guest VMs can access and manipulate host side
resources like the page cache.....
Cheers,
Dave.
--
Dave Chinner
david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-10 1:26 ` Dave Chinner
0 siblings, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-10 1:26 UTC (permalink / raw)
To: Pankaj Gupta
Cc: linux-kernel, kvm, qemu-devel, linux-nvdimm, linux-fsdevel,
virtualization, linux-acpi, linux-ext4, linux-xfs, jack,
stefanha, dan.j.williams, riel, nilal, kwolf, pbonzini, zwisler,
vishal.l.verma, dave.jiang, david, jmoyer, xiaoguangrong.eric,
hch, mst, jasowang, lcapitulino, imammedo, eblake, willy, tytso,
adilger.kernel, darrick.wong, rjw
On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> This patch series has implementation for "virtio pmem".
> "virtio pmem" is fake persistent memory(nvdimm) in guest
> which allows to bypass the guest page cache. This also
> implements a VIRTIO based asynchronous flush mechanism.
Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
like a good idea, but.....
This means the guest VM can now run timing attacks to observe host
side page cache residency, and depending on the implementation I'm
guessing that the guest will be able to control host side page
cache eviction, too (e.g. via discard or hole punch operations).
Which means this functionality looks to me like a new vector for
information leakage into and out of the guest VM via guest
controlled host page cache manipulation.
https://arxiv.org/pdf/1901.01161
I might be wrong, but if I'm not we're going to have to be very
careful about how guest VMs can access and manipulate host side
resources like the page cache.....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v3 0/5] kvm "virtio pmem" device
2019-01-09 14:47 Pankaj Gupta
@ 2019-01-10 1:26 ` Dave Chinner
[not found] ` <20190109144736.17452-1-pagupta-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
1 sibling, 0 replies; 57+ messages in thread
From: Dave Chinner @ 2019-01-10 1:26 UTC (permalink / raw)
To: Pankaj Gupta
Cc: jack, kvm, linux-nvdimm, qemu-devel, virtualization,
adilger.kernel, zwisler, eblake, dave.jiang, darrick.wong,
vishal.l.verma, mst, willy, hch, linux-acpi, jmoyer, nilal, riel,
stefanha, imammedo, dan.j.williams, lcapitulino, linux-ext4,
tytso, xiaoguangrong.eric, rjw, linux-kernel, linux-xfs,
linux-fsdevel, pbonzini
On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote:
> This patch series has implementation for "virtio pmem".
> "virtio pmem" is fake persistent memory(nvdimm) in guest
> which allows to bypass the guest page cache. This also
> implements a VIRTIO based asynchronous flush mechanism.
Hmmmm. Sharing the host page cache direct into the guest VM. Sounds
like a good idea, but.....
This means the guest VM can now run timing attacks to observe host
side page cache residency, and depending on the implementation I'm
guessing that the guest will be able to control host side page
cache eviction, too (e.g. via discard or hole punch operations).
Which means this functionality looks to me like a new vector for
information leakage into and out of the guest VM via guest
controlled host page cache manipulation.
https://arxiv.org/pdf/1901.01161
I might be wrong, but if I'm not we're going to have to be very
careful about how guest VMs can access and manipulate host side
resources like the page cache.....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 57+ messages in thread
* [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-09 14:47 Pankaj Gupta
2019-01-10 1:26 ` Dave Chinner
[not found] ` <20190109144736.17452-1-pagupta-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
0 siblings, 2 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-09 14:47 UTC (permalink / raw)
To: linux-kernel, kvm, qemu-devel, linux-nvdimm, linux-fsdevel,
virtualization, linux-acpi, linux-ext4, linux-xfs
Cc: jack, stefanha, dan.j.williams, riel, nilal, kwolf, pbonzini,
zwisler, vishal.l.verma, dave.jiang, david, jmoyer,
xiaoguangrong.eric, hch, mst, jasowang, lcapitulino, imammedo,
eblake, willy, tytso, adilger.kernel, darrick.wong, rjw, pagupta
This patch series has implementation for "virtio pmem".
"virtio pmem" is fake persistent memory(nvdimm) in guest
which allows to bypass the guest page cache. This also
implements a VIRTIO based asynchronous flush mechanism.
Sharing guest kernel driver in this patchset with the
changes suggested in v2. Tested with Qemu side device
emulation for virtio-pmem [6].
Details of project idea for 'virtio pmem' flushing interface
is shared [3] & [4].
Implementation is divided into two parts:
New virtio pmem guest driver and qemu code changes for new
virtio pmem paravirtualized device.
1. Guest virtio-pmem kernel driver
---------------------------------
- Reads persistent memory range from paravirt device and
registers with 'nvdimm_bus'.
- 'nvdimm/pmem' driver uses this information to allocate
persistent memory region and setup filesystem operations
to the allocated memory.
- virtio pmem driver implements asynchronous flushing
interface to flush from guest to host.
2. Qemu virtio-pmem device
---------------------------------
- Creates virtio pmem device and exposes a memory range to
KVM guest.
- At host side this is file backed memory which acts as
persistent memory.
- Qemu side flush uses aio thread pool API's and virtio
for asynchronous guest multi request handling.
David Hildenbrand CCed also posted a modified version[6] of
qemu virtio-pmem code based on updated Qemu memory device API.
Virtio-pmem errors handling:
----------------------------------------
Checked behaviour of virtio-pmem for below types of errors
Need suggestions on expected behaviour for handling these errors?
- Hardware Errors: Uncorrectable recoverable Errors:
a] virtio-pmem:
- As per current logic if error page belongs to Qemu process,
host MCE handler isolates(hwpoison) that page and send SIGBUS.
Qemu SIGBUS handler injects exception to KVM guest.
- KVM guest then isolates the page and send SIGBUS to guest
userspace process which has mapped the page.
b] Existing implementation for ACPI pmem driver:
- Handles such errors with MCE notifier and creates a list
of bad blocks. Read/direct access DAX operation return EIO
if accessed memory page fall in bad block list.
- It also starts backgound scrubbing.
- Similar functionality can be reused in virtio-pmem with MCE
notifier but without scrubbing(no ACPI/ARS)? Need inputs to
confirm if this behaviour is ok or needs any change?
Changes from PATCH v2: [1]
- Disable MAP_SYNC for ext4 & XFS filesystems - [Dan]
- Use name 'virtio pmem' in place of 'fake dax'
Changes from PATCH v1: [2]
- 0-day build test for build dependency on libnvdimm
Changes suggested by - [Dan Williams]
- Split the driver into two parts virtio & pmem
- Move queuing of async block request to block layer
- Add "sync" parameter in nvdimm_flush function
- Use indirect call for nvdimm_flush
- Don’t move declarations to common global header e.g nd.h
- nvdimm_flush() return 0 or -EIO if it fails
- Teach nsio_rw_bytes() that the flush can fail
- Rename nvdimm_flush() to generic_nvdimm_flush()
- Use 'nd_region->provider_data' for long dereferencing
- Remove virtio_pmem_freeze/restore functions
- Remove BSD license text with SPDX license text
- Add might_sleep() in virtio_pmem_flush - [Luiz]
- Make spin_lock_irqsave() narrow
Changes from RFC v3
- Rebase to latest upstream - Luiz
- Call ndregion->flush in place of nvdimm_flush- Luiz
- kmalloc return check - Luiz
- virtqueue full handling - Stefan
- Don't map entire virtio_pmem_req to device - Stefan
- request leak, correct sizeof req- Stefan
- Move declaration to virtio_pmem.c
Changes from RFC v2:
- Add flush function in the nd_region in place of switching
on a flag - Dan & Stefan
- Add flush completion function with proper locking and wait
for host side flush completion - Stefan & Dan
- Keep userspace API in uapi header file - Stefan, MST
- Use LE fields & New device id - MST
- Indentation & spacing suggestions - MST & Eric
- Remove extra header files & add licensing - Stefan
Changes from RFC v1:
- Reuse existing 'pmem' code for registering persistent
memory and other operations instead of creating an entirely
new block driver.
- Use VIRTIO driver to register memory information with
nvdimm_bus and create region_type accordingly.
- Call VIRTIO flush from existing pmem driver.
Pankaj Gupta (5):
libnvdimm: nd_region flush callback support
virtio-pmem: Add virtio-pmem guest driver
libnvdimm: add nd_region buffered dax_dev flag
ext4: disable map_sync for virtio pmem
xfs: disable map_sync for virtio pmem
[2] https://lkml.org/lkml/2018/8/31/407
[3] https://www.spinics.net/lists/kvm/msg149761.html
[4] https://www.spinics.net/lists/kvm/msg153095.html
[5] https://lkml.org/lkml/2018/8/31/413
[6] https://marc.info/?l=qemu-devel&m=153555721901824&w=2
drivers/acpi/nfit/core.c | 4 -
drivers/dax/super.c | 17 +++++
drivers/nvdimm/claim.c | 6 +
drivers/nvdimm/nd.h | 1
drivers/nvdimm/pmem.c | 15 +++-
drivers/nvdimm/region_devs.c | 45 +++++++++++++-
drivers/nvdimm/virtio_pmem.c | 84 ++++++++++++++++++++++++++
drivers/virtio/Kconfig | 10 +++
drivers/virtio/Makefile | 1
drivers/virtio/pmem.c | 125 +++++++++++++++++++++++++++++++++++++++
fs/ext4/file.c | 11 +++
fs/xfs/xfs_file.c | 8 ++
include/linux/dax.h | 9 ++
include/linux/libnvdimm.h | 11 +++
include/linux/virtio_pmem.h | 60 ++++++++++++++++++
include/uapi/linux/virtio_ids.h | 1
include/uapi/linux/virtio_pmem.h | 10 +++
17 files changed, 406 insertions(+), 12 deletions(-)
^ permalink raw reply [flat|nested] 57+ messages in thread
* [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-09 14:47 Pankaj Gupta
0 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-09 14:47 UTC (permalink / raw)
To: linux-kernel, kvm, qemu-devel, linux-nvdimm, linux-fsdevel,
virtualization, linux-acpi, linux-ext4, linux-xfs
Cc: pagupta, jack, lcapitulino, adilger.kernel, zwisler, eblake,
dave.jiang, darrick.wong, vishal.l.verma, mst, willy, hch,
jmoyer, nilal, riel, stefanha, imammedo, dan.j.williams, tytso,
xiaoguangrong.eric, rjw, pbonzini
This patch series has implementation for "virtio pmem".
"virtio pmem" is fake persistent memory(nvdimm) in guest
which allows to bypass the guest page cache. This also
implements a VIRTIO based asynchronous flush mechanism.
Sharing guest kernel driver in this patchset with the
changes suggested in v2. Tested with Qemu side device
emulation for virtio-pmem [6].
Details of project idea for 'virtio pmem' flushing interface
is shared [3] & [4].
Implementation is divided into two parts:
New virtio pmem guest driver and qemu code changes for new
virtio pmem paravirtualized device.
1. Guest virtio-pmem kernel driver
---------------------------------
- Reads persistent memory range from paravirt device and
registers with 'nvdimm_bus'.
- 'nvdimm/pmem' driver uses this information to allocate
persistent memory region and setup filesystem operations
to the allocated memory.
- virtio pmem driver implements asynchronous flushing
interface to flush from guest to host.
2. Qemu virtio-pmem device
---------------------------------
- Creates virtio pmem device and exposes a memory range to
KVM guest.
- At host side this is file backed memory which acts as
persistent memory.
- Qemu side flush uses aio thread pool API's and virtio
for asynchronous guest multi request handling.
David Hildenbrand CCed also posted a modified version[6] of
qemu virtio-pmem code based on updated Qemu memory device API.
Virtio-pmem errors handling:
----------------------------------------
Checked behaviour of virtio-pmem for below types of errors
Need suggestions on expected behaviour for handling these errors?
- Hardware Errors: Uncorrectable recoverable Errors:
a] virtio-pmem:
- As per current logic if error page belongs to Qemu process,
host MCE handler isolates(hwpoison) that page and send SIGBUS.
Qemu SIGBUS handler injects exception to KVM guest.
- KVM guest then isolates the page and send SIGBUS to guest
userspace process which has mapped the page.
b] Existing implementation for ACPI pmem driver:
- Handles such errors with MCE notifier and creates a list
of bad blocks. Read/direct access DAX operation return EIO
if accessed memory page fall in bad block list.
- It also starts backgound scrubbing.
- Similar functionality can be reused in virtio-pmem with MCE
notifier but without scrubbing(no ACPI/ARS)? Need inputs to
confirm if this behaviour is ok or needs any change?
Changes from PATCH v2: [1]
- Disable MAP_SYNC for ext4 & XFS filesystems - [Dan]
- Use name 'virtio pmem' in place of 'fake dax'
Changes from PATCH v1: [2]
- 0-day build test for build dependency on libnvdimm
Changes suggested by - [Dan Williams]
- Split the driver into two parts virtio & pmem
- Move queuing of async block request to block layer
- Add "sync" parameter in nvdimm_flush function
- Use indirect call for nvdimm_flush
- Don’t move declarations to common global header e.g nd.h
- nvdimm_flush() return 0 or -EIO if it fails
- Teach nsio_rw_bytes() that the flush can fail
- Rename nvdimm_flush() to generic_nvdimm_flush()
- Use 'nd_region->provider_data' for long dereferencing
- Remove virtio_pmem_freeze/restore functions
- Remove BSD license text with SPDX license text
- Add might_sleep() in virtio_pmem_flush - [Luiz]
- Make spin_lock_irqsave() narrow
Changes from RFC v3
- Rebase to latest upstream - Luiz
- Call ndregion->flush in place of nvdimm_flush- Luiz
- kmalloc return check - Luiz
- virtqueue full handling - Stefan
- Don't map entire virtio_pmem_req to device - Stefan
- request leak, correct sizeof req- Stefan
- Move declaration to virtio_pmem.c
Changes from RFC v2:
- Add flush function in the nd_region in place of switching
on a flag - Dan & Stefan
- Add flush completion function with proper locking and wait
for host side flush completion - Stefan & Dan
- Keep userspace API in uapi header file - Stefan, MST
- Use LE fields & New device id - MST
- Indentation & spacing suggestions - MST & Eric
- Remove extra header files & add licensing - Stefan
Changes from RFC v1:
- Reuse existing 'pmem' code for registering persistent
memory and other operations instead of creating an entirely
new block driver.
- Use VIRTIO driver to register memory information with
nvdimm_bus and create region_type accordingly.
- Call VIRTIO flush from existing pmem driver.
Pankaj Gupta (5):
libnvdimm: nd_region flush callback support
virtio-pmem: Add virtio-pmem guest driver
libnvdimm: add nd_region buffered dax_dev flag
ext4: disable map_sync for virtio pmem
xfs: disable map_sync for virtio pmem
[2] https://lkml.org/lkml/2018/8/31/407
[3] https://www.spinics.net/lists/kvm/msg149761.html
[4] https://www.spinics.net/lists/kvm/msg153095.html
[5] https://lkml.org/lkml/2018/8/31/413
[6] https://marc.info/?l=qemu-devel&m=153555721901824&w=2
drivers/acpi/nfit/core.c | 4 -
drivers/dax/super.c | 17 +++++
drivers/nvdimm/claim.c | 6 +
drivers/nvdimm/nd.h | 1
drivers/nvdimm/pmem.c | 15 +++-
drivers/nvdimm/region_devs.c | 45 +++++++++++++-
drivers/nvdimm/virtio_pmem.c | 84 ++++++++++++++++++++++++++
drivers/virtio/Kconfig | 10 +++
drivers/virtio/Makefile | 1
drivers/virtio/pmem.c | 125 +++++++++++++++++++++++++++++++++++++++
fs/ext4/file.c | 11 +++
fs/xfs/xfs_file.c | 8 ++
include/linux/dax.h | 9 ++
include/linux/libnvdimm.h | 11 +++
include/linux/virtio_pmem.h | 60 ++++++++++++++++++
include/uapi/linux/virtio_ids.h | 1
include/uapi/linux/virtio_pmem.h | 10 +++
17 files changed, 406 insertions(+), 12 deletions(-)
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 57+ messages in thread
* [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-09 13:50 ` Pankaj Gupta
0 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-09 13:50 UTC (permalink / raw)
To: linux-kernel-u79uwXL29TY76Z2rM5mHXA, kvm-u79uwXL29TY76Z2rM5mHXA,
qemu-devel-qX2TKyscuCcdnm+yROfE0A,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
linux-ext4-u79uwXL29TY76Z2rM5mHXA,
linux-xfs-u79uwXL29TY76Z2rM5mHXA
Cc: jack-AlSwsSmVLrQ, david-H+wXaHxf7aLQT0dZR+AlfA,
jasowang-H+wXaHxf7aLQT0dZR+AlfA,
lcapitulino-H+wXaHxf7aLQT0dZR+AlfA,
adilger.kernel-m1MBpc4rdrD3fQ9qLvQP4Q,
zwisler-DgEjT+Ai2ygdnm+yROfE0A, eblake-H+wXaHxf7aLQT0dZR+AlfA,
darrick.wong-QHcLZuEGTsvQT0dZR+AlfA, mst-H+wXaHxf7aLQT0dZR+AlfA,
willy-wEGCiKHe2LqWVfeAwA7xHQ, hch-wEGCiKHe2LqWVfeAwA7xHQ,
nilal-H+wXaHxf7aLQT0dZR+AlfA, riel-ebMLmSuQjDVBDgjK7y7TUQ,
stefanha-H+wXaHxf7aLQT0dZR+AlfA, imammedo-H+wXaHxf7aLQT0dZR+AlfA,
kwolf-H+wXaHxf7aLQT0dZR+AlfA, tytso-3s7WtUTddSA,
xiaoguangrong.eric-Re5JQEeQqe8AvxtiuMwx3w,
rjw-LthD3rsA81gm4RdzfppkhA, pbonzini-H+wXaHxf7aLQT0dZR+AlfA
This patch series has implementation for "virtio pmem".
"virtio pmem" is fake persistent memory(nvdimm) in guest
which allows to bypass the guest page cache. This also
implements a VIRTIO based asynchronous flush mechanism.
Sharing guest kernel driver in this patchset with the
changes suggested in v2. Tested with Qemu side device
emulation for virtio-pmem [6].
Details of project idea for 'virtio pmem' flushing interface
is shared [3] & [4].
Implementation is divided into two parts:
New virtio pmem guest driver and qemu code changes for new
virtio pmem paravirtualized device.
1. Guest virtio-pmem kernel driver
---------------------------------
- Reads persistent memory range from paravirt device and
registers with 'nvdimm_bus'.
- 'nvdimm/pmem' driver uses this information to allocate
persistent memory region and setup filesystem operations
to the allocated memory.
- virtio pmem driver implements asynchronous flushing
interface to flush from guest to host.
2. Qemu virtio-pmem device
---------------------------------
- Creates virtio pmem device and exposes a memory range to
KVM guest.
- At host side this is file backed memory which acts as
persistent memory.
- Qemu side flush uses aio thread pool API's and virtio
for asynchronous guest multi request handling.
David Hildenbrand CCed also posted a modified version[6] of
qemu virtio-pmem code based on updated Qemu memory device API.
Virtio-pmem errors handling:
----------------------------------------
Checked behaviour of virtio-pmem for below types of errors
Need suggestions on expected behaviour for handling these errors?
- Hardware Errors: Uncorrectable recoverable Errors:
a] virtio-pmem:
- As per current logic if error page belongs to Qemu process,
host MCE handler isolates(hwpoison) that page and send SIGBUS.
Qemu SIGBUS handler injects exception to KVM guest.
- KVM guest then isolates the page and send SIGBUS to guest
userspace process which has mapped the page.
b] Existing implementation for ACPI pmem driver:
- Handles such errors with MCE notifier and creates a list
of bad blocks. Read/direct access DAX operation return EIO
if accessed memory page fall in bad block list.
- It also starts backgound scrubbing.
- Similar functionality can be reused in virtio-pmem with MCE
notifier but without scrubbing(no ACPI/ARS)? Need inputs to
confirm if this behaviour is ok or needs any change?
Changes from PATCH v2: [1]
- Disable MAP_SYNC for ext4 & XFS filesystems - [Dan]
- Use name 'virtio pmem' in place of 'fake dax'
Changes from PATCH v1: [2]
- 0-day build test for build dependency on libnvdimm
Changes suggested by - [Dan Williams]
- Split the driver into two parts virtio & pmem
- Move queuing of async block request to block layer
- Add "sync" parameter in nvdimm_flush function
- Use indirect call for nvdimm_flush
- Don’t move declarations to common global header e.g nd.h
- nvdimm_flush() return 0 or -EIO if it fails
- Teach nsio_rw_bytes() that the flush can fail
- Rename nvdimm_flush() to generic_nvdimm_flush()
- Use 'nd_region->provider_data' for long dereferencing
- Remove virtio_pmem_freeze/restore functions
- Remove BSD license text with SPDX license text
- Add might_sleep() in virtio_pmem_flush - [Luiz]
- Make spin_lock_irqsave() narrow
Changes from RFC v3
- Rebase to latest upstream - Luiz
- Call ndregion->flush in place of nvdimm_flush- Luiz
- kmalloc return check - Luiz
- virtqueue full handling - Stefan
- Don't map entire virtio_pmem_req to device - Stefan
- request leak, correct sizeof req- Stefan
- Move declaration to virtio_pmem.c
Changes from RFC v2:
- Add flush function in the nd_region in place of switching
on a flag - Dan & Stefan
- Add flush completion function with proper locking and wait
for host side flush completion - Stefan & Dan
- Keep userspace API in uapi header file - Stefan, MST
- Use LE fields & New device id - MST
- Indentation & spacing suggestions - MST & Eric
- Remove extra header files & add licensing - Stefan
Changes from RFC v1:
- Reuse existing 'pmem' code for registering persistent
memory and other operations instead of creating an entirely
new block driver.
- Use VIRTIO driver to register memory information with
nvdimm_bus and create region_type accordingly.
- Call VIRTIO flush from existing pmem driver.
Pankaj Gupta (5):
libnvdimm: nd_region flush callback support
virtio-pmem: Add virtio-pmem guest driver
libnvdimm: add nd_region buffered dax_dev flag
ext4: disable map_sync for virtio pmem
xfs: disable map_sync for virtio pmem
[2] https://lkml.org/lkml/2018/8/31/407
[3] https://www.spinics.net/lists/kvm/msg149761.html
[4] https://www.spinics.net/lists/kvm/msg153095.html
[5] https://lkml.org/lkml/2018/8/31/413
[6] https://marc.info/?l=qemu-devel&m=153555721901824&w=2
drivers/acpi/nfit/core.c | 4 -
drivers/dax/super.c | 17 +++++
drivers/nvdimm/claim.c | 6 +
drivers/nvdimm/nd.h | 1
drivers/nvdimm/pmem.c | 15 +++-
drivers/nvdimm/region_devs.c | 45 +++++++++++++-
drivers/nvdimm/virtio_pmem.c | 84 ++++++++++++++++++++++++++
drivers/virtio/Kconfig | 10 +++
drivers/virtio/Makefile | 1
drivers/virtio/pmem.c | 125 +++++++++++++++++++++++++++++++++++++++
fs/ext4/file.c | 11 +++
fs/xfs/xfs_file.c | 8 ++
include/linux/dax.h | 9 ++
include/linux/libnvdimm.h | 11 +++
include/linux/virtio_pmem.h | 60 ++++++++++++++++++
include/uapi/linux/virtio_ids.h | 1
include/uapi/linux/virtio_pmem.h | 10 +++
17 files changed, 406 insertions(+), 12 deletions(-)
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply [flat|nested] 57+ messages in thread
* [PATCH v3 0/5] kvm "virtio pmem" device
@ 2019-01-09 13:50 ` Pankaj Gupta
0 siblings, 0 replies; 57+ messages in thread
From: Pankaj Gupta @ 2019-01-09 13:50 UTC (permalink / raw)
To: linux-kernel, kvm, qemu-devel, linux-nvdimm, linux-fsdevel,
virtualization, linux-acpi, linux-ext4, linux-xfs
Cc: jack, stefanha, dan.j.williams, riel, nilal, kwolf, pbonzini,
zwisler, vishal.l.verma, dave.jiang, david, jmoyer,
xiaoguangrong.eric, hch, mst, jasowang, lcapitulino, imammedo,
eblake, willy, tytso, adilger.kernel, darrick.wong, rjw, pagupta
This patch series has implementation for "virtio pmem".
"virtio pmem" is fake persistent memory(nvdimm) in guest
which allows to bypass the guest page cache. This also
implements a VIRTIO based asynchronous flush mechanism.
Sharing guest kernel driver in this patchset with the
changes suggested in v2. Tested with Qemu side device
emulation for virtio-pmem [6].
Details of project idea for 'virtio pmem' flushing interface
is shared [3] & [4].
Implementation is divided into two parts:
New virtio pmem guest driver and qemu code changes for new
virtio pmem paravirtualized device.
1. Guest virtio-pmem kernel driver
---------------------------------
- Reads persistent memory range from paravirt device and
registers with 'nvdimm_bus'.
- 'nvdimm/pmem' driver uses this information to allocate
persistent memory region and setup filesystem operations
to the allocated memory.
- virtio pmem driver implements asynchronous flushing
interface to flush from guest to host.
2. Qemu virtio-pmem device
---------------------------------
- Creates virtio pmem device and exposes a memory range to
KVM guest.
- At host side this is file backed memory which acts as
persistent memory.
- Qemu side flush uses aio thread pool API's and virtio
for asynchronous guest multi request handling.
David Hildenbrand CCed also posted a modified version[6] of
qemu virtio-pmem code based on updated Qemu memory device API.
Virtio-pmem errors handling:
----------------------------------------
Checked behaviour of virtio-pmem for below types of errors
Need suggestions on expected behaviour for handling these errors?
- Hardware Errors: Uncorrectable recoverable Errors:
a] virtio-pmem:
- As per current logic if error page belongs to Qemu process,
host MCE handler isolates(hwpoison) that page and send SIGBUS.
Qemu SIGBUS handler injects exception to KVM guest.
- KVM guest then isolates the page and send SIGBUS to guest
userspace process which has mapped the page.
b] Existing implementation for ACPI pmem driver:
- Handles such errors with MCE notifier and creates a list
of bad blocks. Read/direct access DAX operation return EIO
if accessed memory page fall in bad block list.
- It also starts backgound scrubbing.
- Similar functionality can be reused in virtio-pmem with MCE
notifier but without scrubbing(no ACPI/ARS)? Need inputs to
confirm if this behaviour is ok or needs any change?
Changes from PATCH v2: [1]
- Disable MAP_SYNC for ext4 & XFS filesystems - [Dan]
- Use name 'virtio pmem' in place of 'fake dax'
Changes from PATCH v1: [2]
- 0-day build test for build dependency on libnvdimm
Changes suggested by - [Dan Williams]
- Split the driver into two parts virtio & pmem
- Move queuing of async block request to block layer
- Add "sync" parameter in nvdimm_flush function
- Use indirect call for nvdimm_flush
- Don’t move declarations to common global header e.g nd.h
- nvdimm_flush() return 0 or -EIO if it fails
- Teach nsio_rw_bytes() that the flush can fail
- Rename nvdimm_flush() to generic_nvdimm_flush()
- Use 'nd_region->provider_data' for long dereferencing
- Remove virtio_pmem_freeze/restore functions
- Remove BSD license text with SPDX license text
- Add might_sleep() in virtio_pmem_flush - [Luiz]
- Make spin_lock_irqsave() narrow
Changes from RFC v3
- Rebase to latest upstream - Luiz
- Call ndregion->flush in place of nvdimm_flush- Luiz
- kmalloc return check - Luiz
- virtqueue full handling - Stefan
- Don't map entire virtio_pmem_req to device - Stefan
- request leak, correct sizeof req- Stefan
- Move declaration to virtio_pmem.c
Changes from RFC v2:
- Add flush function in the nd_region in place of switching
on a flag - Dan & Stefan
- Add flush completion function with proper locking and wait
for host side flush completion - Stefan & Dan
- Keep userspace API in uapi header file - Stefan, MST
- Use LE fields & New device id - MST
- Indentation & spacing suggestions - MST & Eric
- Remove extra header files & add licensing - Stefan
Changes from RFC v1:
- Reuse existing 'pmem' code for registering persistent
memory and other operations instead of creating an entirely
new block driver.
- Use VIRTIO driver to register memory information with
nvdimm_bus and create region_type accordingly.
- Call VIRTIO flush from existing pmem driver.
Pankaj Gupta (5):
libnvdimm: nd_region flush callback support
virtio-pmem: Add virtio-pmem guest driver
libnvdimm: add nd_region buffered dax_dev flag
ext4: disable map_sync for virtio pmem
xfs: disable map_sync for virtio pmem
[2] https://lkml.org/lkml/2018/8/31/407
[3] https://www.spinics.net/lists/kvm/msg149761.html
[4] https://www.spinics.net/lists/kvm/msg153095.html
[5] https://lkml.org/lkml/2018/8/31/413
[6] https://marc.info/?l=qemu-devel&m=153555721901824&w=2
drivers/acpi/nfit/core.c | 4 -
drivers/dax/super.c | 17 +++++
drivers/nvdimm/claim.c | 6 +
drivers/nvdimm/nd.h | 1
drivers/nvdimm/pmem.c | 15 +++-
drivers/nvdimm/region_devs.c | 45 +++++++++++++-
drivers/nvdimm/virtio_pmem.c | 84 ++++++++++++++++++++++++++
drivers/virtio/Kconfig | 10 +++
drivers/virtio/Makefile | 1
drivers/virtio/pmem.c | 125 +++++++++++++++++++++++++++++++++++++++
fs/ext4/file.c | 11 +++
fs/xfs/xfs_file.c | 8 ++
include/linux/dax.h | 9 ++
include/linux/libnvdimm.h | 11 +++
include/linux/virtio_pmem.h | 60 ++++++++++++++++++
include/uapi/linux/virtio_ids.h | 1
include/uapi/linux/virtio_pmem.h | 10 +++
17 files changed, 406 insertions(+), 12 deletions(-)
^ permalink raw reply [flat|nested] 57+ messages in thread
end of thread, other threads:[~2019-01-15 20:42 UTC | newest]
Thread overview: 57+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-01-09 13:50 [PATCH v3 0/5] kvm "virtio pmem" device Pankaj Gupta
-- strict thread matches above, loose matches on Subject: below --
2019-01-09 14:47 Pankaj Gupta
2019-01-09 14:47 Pankaj Gupta
2019-01-10 1:26 ` Dave Chinner
[not found] ` <20190109144736.17452-1-pagupta-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2019-01-10 1:26 ` Dave Chinner
2019-01-10 1:26 ` Dave Chinner
2019-01-10 2:40 ` Rik van Riel
2019-01-10 10:17 ` Jan Kara
2019-01-10 10:17 ` Jan Kara
2019-01-10 10:17 ` Jan Kara
[not found] ` <20190110101757.GC15790-4I4JzKEfoa/jFM9bn6wA6Q@public.gmane.org>
2019-01-13 1:38 ` Pankaj Gupta
2019-01-13 1:38 ` Pankaj Gupta
[not found] ` <1354249849.63357171.1547343519970.JavaMail.zimbra-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2019-01-13 1:43 ` Dan Williams
2019-01-13 1:43 ` Dan Williams
2019-01-13 1:43 ` Dan Williams
2019-01-13 1:38 ` Pankaj Gupta
2019-01-11 7:45 ` Pankaj Gupta
2019-01-11 7:45 ` Pankaj Gupta
2019-01-11 7:45 ` Pankaj Gupta
2019-01-11 7:45 ` Pankaj Gupta
2019-01-13 23:29 ` Dave Chinner
[not found] ` <1326478078.61913951.1547192704870.JavaMail.zimbra-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2019-01-13 23:29 ` Dave Chinner
2019-01-13 23:29 ` Dave Chinner
2019-01-13 23:38 ` Matthew Wilcox
2019-01-13 23:38 ` Matthew Wilcox
2019-01-13 23:38 ` Matthew Wilcox
2019-01-14 2:50 ` Dave Chinner
[not found] ` <20190113233820.GX6310-PfSpb0PWhxZc2C7mugBRk2EX/6BAtgUQ@public.gmane.org>
2019-01-14 2:50 ` Dave Chinner
2019-01-14 2:50 ` Dave Chinner
2019-01-14 7:15 ` Pankaj Gupta
2019-01-14 7:15 ` Pankaj Gupta
2019-01-14 21:25 ` Dave Chinner
[not found] ` <942065073.64011540.1547450140670.JavaMail.zimbra-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2019-01-14 21:25 ` Dave Chinner
2019-01-14 21:25 ` Dave Chinner
2019-01-14 21:35 ` Dan Williams
2019-01-14 21:35 ` Dan Williams
2019-01-14 21:35 ` Dan Williams
2019-01-14 21:35 ` Dan Williams
2019-01-14 22:21 ` Dave Chinner
[not found] ` <CAPcyv4jtPcLV-s0sKNHwwk0ug7GLBV6699dpm1h3r2xSo879dg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2019-01-14 22:21 ` Dave Chinner
2019-01-14 22:21 ` Dave Chinner
2019-01-14 22:21 ` Dave Chinner
2019-01-15 2:19 ` Michael S. Tsirkin
2019-01-15 2:19 ` Michael S. Tsirkin
2019-01-15 2:19 ` Michael S. Tsirkin
2019-01-15 2:19 ` Michael S. Tsirkin
2019-01-15 5:35 ` Pankaj Gupta
2019-01-15 5:35 ` Pankaj Gupta
2019-01-15 5:35 ` Pankaj Gupta
[not found] ` <1684638419.64320214.1547530506805.JavaMail.zimbra-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2019-01-15 20:42 ` Dave Chinner
2019-01-15 20:42 ` Dave Chinner
2019-01-15 20:42 ` Dave Chinner
2019-01-15 20:42 ` Dave Chinner
2019-01-15 5:35 ` Pankaj Gupta
2019-01-14 7:15 ` Pankaj Gupta
2019-01-09 13:50 Pankaj Gupta
2019-01-09 13:50 ` Pankaj Gupta
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.