From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Message-ID: <1500818683.4073.31.camel@redhat.com> Subject: Re: KVM "fake DAX" flushing interface - discussion From: Rik van Riel Date: Sun, 23 Jul 2017 10:04:43 -0400 In-Reply-To: References: <1455443283.33337333.1500618150787.JavaMail.zimbra@redhat.com> <945864462.33340808.1500620194836.JavaMail.zimbra@redhat.com> <20170721121241.GA18014@stefanha-x1.localdomain> <46101617.33460557.1500643755247.JavaMail.zimbra@redhat.com> <20170721155848.GO18014@stefanha-x1.localdomain> Mime-Version: 1.0 List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: Dan Williams , Stefan Hajnoczi Cc: Kevin Wolf , Pankaj Gupta , xiaoguangrong eric , kvm-devel , "linux-nvdimm@lists.01.org" , Qemu Developers , Stefan Hajnoczi , Paolo Bonzini , Nitesh Narayan Lal List-ID: On Sat, 2017-07-22 at 12:34 -0700, Dan Williams wrote: > On Fri, Jul 21, 2017 at 8:58 AM, Stefan Hajnoczi > wrote: > > > > Maybe the NVDIMM folks can comment on this idea. > > I think it's unworkable to use the flush hints as a guest-to-host > fsync mechanism. That mechanism was designed to flush small memory > controller buffers, not large swaths of dirty memory. What about > running the guests in a writethrough cache mode to avoid needing > dirty > cache management altogether? Either way I think you need to use > device-dax on the host, or one of the two work-in-progress filesystem > mechanisms (synchronous-faults or S_IOMAP_FROZEN) to avoid need any > metadata coordination between guests and the host. The thing Pankaj is looking at is to use the DAX mechanisms inside the guest (disk image as memory mapped nvdimm area), with that disk image backed by a regular storage device on the host. The goal is to increase density of guests, by moving page cache into the host (where it can be easily reclaimed). If we assume the guests will be backed by relatively fast SSDs, a "whole device flush" from filesystem journaling code (issued where the filesystem issues a barrier or disk cache flush today) may be just what we need to make that work. _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm From mboxrd@z Thu Jan 1 00:00:00 1970 From: Rik van Riel Subject: Re: KVM "fake DAX" flushing interface - discussion Date: Sun, 23 Jul 2017 10:04:43 -0400 Message-ID: <1500818683.4073.31.camel@redhat.com> References: <1455443283.33337333.1500618150787.JavaMail.zimbra@redhat.com> <945864462.33340808.1500620194836.JavaMail.zimbra@redhat.com> <20170721121241.GA18014@stefanha-x1.localdomain> <46101617.33460557.1500643755247.JavaMail.zimbra@redhat.com> <20170721155848.GO18014@stefanha-x1.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: Kevin Wolf , Pankaj Gupta , xiaoguangrong eric , kvm-devel , "linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org" , Qemu Developers , Stefan Hajnoczi , Paolo Bonzini , Nitesh Narayan Lal To: Dan Williams , Stefan Hajnoczi Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linux-nvdimm-bounces-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org Sender: "Linux-nvdimm" List-Id: kvm.vger.kernel.org On Sat, 2017-07-22 at 12:34 -0700, Dan Williams wrote: > On Fri, Jul 21, 2017 at 8:58 AM, Stefan Hajnoczi > wrote: > > > > Maybe the NVDIMM folks can comment on this idea. > > I think it's unworkable to use the flush hints as a guest-to-host > fsync mechanism. That mechanism was designed to flush small memory > controller buffers, not large swaths of dirty memory. What about > running the guests in a writethrough cache mode to avoid needing > dirty > cache management altogether? Either way I think you need to use > device-dax on the host, or one of the two work-in-progress filesystem > mechanisms (synchronous-faults or S_IOMAP_FROZEN) to avoid need any > metadata coordination between guests and the host. The thing Pankaj is looking at is to use the DAX mechanisms inside the guest (disk image as memory mapped nvdimm area), with that disk image backed by a regular storage device on the host. The goal is to increase density of guests, by moving page cache into the host (where it can be easily reclaimed). If we assume the guests will be backed by relatively fast SSDs, a "whole device flush" from filesystem journaling code (issued where the filesystem issues a barrier or disk cache flush today) may be just what we need to make that work. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35211) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dZHV8-0000q0-16 for qemu-devel@nongnu.org; Sun, 23 Jul 2017 10:04:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dZHV4-00035H-T9 for qemu-devel@nongnu.org; Sun, 23 Jul 2017 10:04:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:44374) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dZHV4-00032e-Mo for qemu-devel@nongnu.org; Sun, 23 Jul 2017 10:04:50 -0400 Message-ID: <1500818683.4073.31.camel@redhat.com> From: Rik van Riel Date: Sun, 23 Jul 2017 10:04:43 -0400 In-Reply-To: References: <1455443283.33337333.1500618150787.JavaMail.zimbra@redhat.com> <945864462.33340808.1500620194836.JavaMail.zimbra@redhat.com> <20170721121241.GA18014@stefanha-x1.localdomain> <46101617.33460557.1500643755247.JavaMail.zimbra@redhat.com> <20170721155848.GO18014@stefanha-x1.localdomain> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] KVM "fake DAX" flushing interface - discussion List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Dan Williams , Stefan Hajnoczi Cc: Pankaj Gupta , Stefan Hajnoczi , kvm-devel , Qemu Developers , "linux-nvdimm@lists.01.org" , ross zwisler , Paolo Bonzini , Kevin Wolf , Nitesh Narayan Lal , xiaoguangrong eric , Haozhong Zhang On Sat, 2017-07-22 at 12:34 -0700, Dan Williams wrote: > On Fri, Jul 21, 2017 at 8:58 AM, Stefan Hajnoczi > wrote: > > > > Maybe the NVDIMM folks can comment on this idea. > > I think it's unworkable to use the flush hints as a guest-to-host > fsync mechanism. That mechanism was designed to flush small memory > controller buffers, not large swaths of dirty memory. What about > running the guests in a writethrough cache mode to avoid needing > dirty > cache management altogether? Either way I think you need to use > device-dax on the host, or one of the two work-in-progress filesystem > mechanisms (synchronous-faults or S_IOMAP_FROZEN) to avoid need any > metadata coordination between guests and the host. The thing Pankaj is looking at is to use the DAX mechanisms inside the guest (disk image as memory mapped nvdimm area), with that disk image backed by a regular storage device on the host. The goal is to increase density of guests, by moving page cache into the host (where it can be easily reclaimed). If we assume the guests will be backed by relatively fast SSDs, a "whole device flush" from filesystem journaling code (issued where the filesystem issues a barrier or disk cache flush today) may be just what we need to make that work.