[RFC] KVM "fake DAX" device flushing

[RFC] KVM "fake DAX" device flushing

[Pankaj Gupta]

We are sharing the prototype version of 'fake DAX' flushing
interface for the initial feedback. This is still work in progress
and not yet ready for merging.

Protoype right now just implements basic functionality without advanced
features with two major parts:

- Qemu virtio-pmem device
  It exposes a persistent memory range to KVM guest which at host side is file
  backed memory and works as persistent memory device. In addition to this it
  provides a virtio flushing interface for KVM guest to do a Qemu side sync for
  guest DAX persistent memory range.  

- Guest virtio-pmem driver
  Reads persistent memory range from paravirt device and reserves system memory map.
  It also allocates a block device corresponding to the pmem range which is accessed
  by DAX capable file systems. (file system support is still pending).  
  
We shared the project idea for 'fake DAX' flushing interface here [1].
Based on suggestions here [2], we implemented guest 'virtio-pmem'
driver and Qemu paravirt device.

[1] https://www.spinics.net/lists/kvm/msg149761.html
[2] https://www.spinics.net/lists/kvm/msg153095.html

Work yet to be done:

- Separate out the common code used by ACPI pmem interface and
  reuse it.

- In pmem device memmap allocation and working. There is some parallel work
  going on upstream related to 'memory_hotplug restructuring' [3] and also hitting
  a memory section alignment issue [4].
  
  [3] https://lwn.net/Articles/712099/
  [4] https://www.mail-archive.com/linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org/msg02978.html
  
- Provide DAX capable file-system(ext4 & XFS) support.
- Qemu device flush functionality.
- Qemu live migration work when host page cache is used.
- Multiple virtio-pmem disks support.

Prototype implementation for feedback:

Kernel: https://github.com/pagupta/linux/commit/d15cf90074eae91aeed7a228da3faf319566dd40
Qemu  : https://github.com/pagupta/qemu/commit/9c428db1e1076970e097e2b0ef8afe52509af823

Please provide feedback. Also, I would be attending KVM Forum in Prague from (25-27 Oct). 
If you are attending KVM forum/Linux conference, I would love to have a discussion on ideas 
and future work.

Thank you,
Pankaj Gupta

Re: [RFC] KVM "fake DAX" device flushing

[Dan Williams]

On Wed, Oct 11, 2017 at 11:51 AM, Pankaj Gupta <pagupta@redhat.com> wrote:
> We are sharing the prototype version of 'fake DAX' flushing
> interface for the initial feedback. This is still work in progress
> and not yet ready for merging.
>
> Protoype right now just implements basic functionality without advanced
> features with two major parts:
>
> - Qemu virtio-pmem device
>   It exposes a persistent memory range to KVM guest which at host side is file
>   backed memory and works as persistent memory device. In addition to this it
>   provides a virtio flushing interface for KVM guest to do a Qemu side sync for
>   guest DAX persistent memory range.
>
> - Guest virtio-pmem driver
>   Reads persistent memory range from paravirt device and reserves system memory map.
>   It also allocates a block device corresponding to the pmem range which is accessed
>   by DAX capable file systems. (file system support is still pending).
>
> We shared the project idea for 'fake DAX' flushing interface here [1].
> Based on suggestions here [2], we implemented guest 'virtio-pmem'
> driver and Qemu paravirt device.
>
> [1] https://www.spinics.net/lists/kvm/msg149761.html
> [2] https://www.spinics.net/lists/kvm/msg153095.html
>
> Work yet to be done:
>
> - Separate out the common code used by ACPI pmem interface and
>   reuse it.
>
> - In pmem device memmap allocation and working. There is some parallel work
>   going on upstream related to 'memory_hotplug restructuring' [3] and also hitting
>   a memory section alignment issue [4].
>
>   [3] https://lwn.net/Articles/712099/
>   [4] https://www.mail-archive.com/linux-nvdimm@lists.01.org/msg02978.html
>
> - Provide DAX capable file-system(ext4 & XFS) support.
> - Qemu device flush functionality.
> - Qemu live migration work when host page cache is used.
> - Multiple virtio-pmem disks support.
>
> Prototype implementation for feedback:
>
> Kernel: https://github.com/pagupta/linux/commit/d15cf90074eae91aeed7a228da3faf319566dd40

Please send this as a patch so it can be reviewed over email.

Re: [RFC] KVM "fake DAX" device flushing

[Stefan Hajnoczi]

On Thu, Oct 12, 2017 at 12:21:46AM +0530, Pankaj Gupta wrote:
> We are sharing the prototype version of 'fake DAX' flushing
> interface for the initial feedback. This is still work in progress
> and not yet ready for merging.
> 
> Protoype right now just implements basic functionality without advanced
> features with two major parts:
> 
> - Qemu virtio-pmem device
>   It exposes a persistent memory range to KVM guest which at host side is file
>   backed memory and works as persistent memory device. In addition to this it
>   provides a virtio flushing interface for KVM guest to do a Qemu side sync for
>   guest DAX persistent memory range.

Please post a draft VIRTIO device specification.

The VIRTIO Technical Committee resources and mailing lists are here:

https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=virtio#feedback

Stefan

Re: [RFC] KVM "fake DAX" device flushing

[Pankaj Gupta]

> On Thu, Oct 12, 2017 at 12:21:46AM +0530, Pankaj Gupta wrote:
> > We are sharing the prototype version of 'fake DAX' flushing
> > interface for the initial feedback. This is still work in progress
> > and not yet ready for merging.
> > 
> > Protoype right now just implements basic functionality without advanced
> > features with two major parts:
> > 
> > - Qemu virtio-pmem device
> >   It exposes a persistent memory range to KVM guest which at host side is
> >   file
> >   backed memory and works as persistent memory device. In addition to this
> >   it
> >   provides a virtio flushing interface for KVM guest to do a Qemu side sync
> >   for
> >   guest DAX persistent memory range.
> 
> Please post a draft VIRTIO device specification.

Sure! will prepare and share.

Thanks,
Pankaj
 
> 
> The VIRTIO Technical Committee resources and mailing lists are here:
> 
> https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=virtio#feedback
> 
> Stefan
>