Linux-PM Archive on lore.kernel.org
 help / color / Atom feed
From: "Roger Pau Monné" <roger.pau@citrix.com>
To: "Durrant, Paul" <pdurrant@amazon.co.uk>
Cc: "Agarwal, Anchal" <anchalag@amazon.com>,
	"Valentin, Eduardo" <eduval@amazon.com>,
	"len.brown@intel.com" <len.brown@intel.com>,
	"peterz@infradead.org" <peterz@infradead.org>,
	"benh@kernel.crashing.org" <benh@kernel.crashing.org>,
	"x86@kernel.org" <x86@kernel.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	"pavel@ucw.cz" <pavel@ucw.cz>, "hpa@zytor.com" <hpa@zytor.com>,
	"tglx@linutronix.de" <tglx@linutronix.de>,
	"sstabellini@kernel.org" <sstabellini@kernel.org>,
	"fllinden@amaozn.com" <fllinden@amaozn.com>,
	"Kamata, Munehisa" <kamatam@amazon.com>,
	"mingo@redhat.com" <mingo@redhat.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"Singh, Balbir" <sblbir@amazon.com>,
	"axboe@kernel.dk" <axboe@kernel.dk>,
	"konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"bp@alien8.de" <bp@alien8.de>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
	"jgross@suse.com" <jgross@suse.com>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"linux-pm@vger.kernel.org" <linux-pm@vger.kernel.org>,
	"rjw@rjwysocki.net" <rjw@rjwysocki.net>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"vkuznets@redhat.com" <vkuznets@redhat.com>,
	"davem@davemloft.net" <davem@davemloft.net>,
	"Woodhouse, David" <dwmw@amazon.co.uk>
Subject: Re: [Xen-devel] [RFC PATCH v3 06/12] xen-blkfront: add callbacks for PM suspend and hibernation
Date: Fri, 21 Feb 2020 12:51:03 +0100
Message-ID: <20200221115103.GY4679@Air-de-Roger> (raw)
In-Reply-To: <66a211bae1de4be9861ef8393607d1b3@EX13D32EUC003.ant.amazon.com>

On Fri, Feb 21, 2020 at 10:33:42AM +0000, Durrant, Paul wrote:
> > -----Original Message-----
> > From: Roger Pau Monné <roger.pau@citrix.com>
> > Sent: 21 February 2020 10:22
> > To: Durrant, Paul <pdurrant@amazon.co.uk>
> > Cc: Agarwal, Anchal <anchalag@amazon.com>; Valentin, Eduardo
> > <eduval@amazon.com>; len.brown@intel.com; peterz@infradead.org;
> > benh@kernel.crashing.org; x86@kernel.org; linux-mm@kvack.org;
> > pavel@ucw.cz; hpa@zytor.com; tglx@linutronix.de; sstabellini@kernel.org;
> > fllinden@amaozn.com; Kamata, Munehisa <kamatam@amazon.com>;
> > mingo@redhat.com; xen-devel@lists.xenproject.org; Singh, Balbir
> > <sblbir@amazon.com>; axboe@kernel.dk; konrad.wilk@oracle.com;
> > bp@alien8.de; boris.ostrovsky@oracle.com; jgross@suse.com;
> > netdev@vger.kernel.org; linux-pm@vger.kernel.org; rjw@rjwysocki.net;
> > linux-kernel@vger.kernel.org; vkuznets@redhat.com; davem@davemloft.net;
> > Woodhouse, David <dwmw@amazon.co.uk>
> > Subject: Re: [Xen-devel] [RFC PATCH v3 06/12] xen-blkfront: add callbacks
> > for PM suspend and hibernation
> > 
> > On Fri, Feb 21, 2020 at 09:56:54AM +0000, Durrant, Paul wrote:
> > > > -----Original Message-----
> > > > From: Roger Pau Monné <roger.pau@citrix.com>
> > > > Sent: 21 February 2020 09:22
> > > > To: Durrant, Paul <pdurrant@amazon.co.uk>
> > > > Cc: Agarwal, Anchal <anchalag@amazon.com>; Valentin, Eduardo
> > > > <eduval@amazon.com>; len.brown@intel.com; peterz@infradead.org;
> > > > benh@kernel.crashing.org; x86@kernel.org; linux-mm@kvack.org;
> > > > pavel@ucw.cz; hpa@zytor.com; tglx@linutronix.de;
> > sstabellini@kernel.org;
> > > > fllinden@amaozn.com; Kamata, Munehisa <kamatam@amazon.com>;
> > > > mingo@redhat.com; xen-devel@lists.xenproject.org; Singh, Balbir
> > > > <sblbir@amazon.com>; axboe@kernel.dk; konrad.wilk@oracle.com;
> > > > bp@alien8.de; boris.ostrovsky@oracle.com; jgross@suse.com;
> > > > netdev@vger.kernel.org; linux-pm@vger.kernel.org; rjw@rjwysocki.net;
> > > > linux-kernel@vger.kernel.org; vkuznets@redhat.com;
> > davem@davemloft.net;
> > > > Woodhouse, David <dwmw@amazon.co.uk>
> > > > Subject: Re: [Xen-devel] [RFC PATCH v3 06/12] xen-blkfront: add
> > callbacks
> > > > for PM suspend and hibernation
> > > >
> > > > On Thu, Feb 20, 2020 at 05:01:52PM +0000, Durrant, Paul wrote:
> > > > > > > Hopefully what I said above illustrates why it may not be 100%
> > > > common.
> > > > > >
> > > > > > Yes, that's fine. I don't expect it to be 100% common (as I guess
> > > > > > that the hooks will have different prototypes), but I expect
> > > > > > that routines can be shared, and that the approach taken can be
> > the
> > > > > > same.
> > > > > >
> > > > > > For example one necessary difference will be that xenbus initiated
> > > > > > suspend won't close the PV connection, in case suspension fails.
> > On PM
> > > > > > suspend you seem to always close the connection beforehand, so you
> > > > > > will always have to re-negotiate on resume even if suspension
> > failed.
> > > > > >
> > > > > > What I'm mostly worried about is the different approach to ring
> > > > > > draining. Ie: either xenbus is changed to freeze the queues and
> > drain
> > > > > > the shared rings, or PM uses the already existing logic of not
> > > > > > flushing the rings an re-issuing in-flight requests on resume.
> > > > > >
> > > > >
> > > > > Yes, that's needs consideration. I don’t think the same semantic can
> > be
> > > > suitable for both. E.g. in a xen-suspend we need to freeze with as
> > little
> > > > processing as possible to avoid dirtying RAM late in the migration
> > cycle,
> > > > and we know that in-flight data can wait. But in a transition to S4 we
> > > > need to make sure that at least all the in-flight blkif requests get
> > > > completed, since they probably contain bits of the guest's memory
> > image
> > > > and that's not going to get saved any other way.
> > > >
> > > > Thanks, that makes sense and something along this lines should be
> > > > added to the commit message IMO.
> > > >
> > > > Wondering about S4, shouldn't we expect the queues to already be
> > > > empty? As any subsystem that wanted to store something to disk should
> > > > make sure requests have been successfully completed before
> > > > suspending.
> > >
> > > What about writing the suspend image itself? Normal filesystem I/O
> > > will have been flushed of course, but whatever vestigial kernel
> > > actually writes out the hibernation file may well expect a final
> > > D0->D3 on the storage device to cause a flush.
> > 
> > Hm, I have no idea really. I think whatever writes to the disk before
> > suspend should actually make sure requests have completed, but what
> > you suggest might also be a possibility.
> > 
> > Can you figure out whether there are requests on the ring or in the
> > queue before suspending?
> 
> Well there's clearly pending stuff in the ring if rsp_prod != req_prod :-)

Right, I assume there's no document that states what's the expected
state for queues &c when switching PM states, so we have to assume
that there might be in-flight requests on the ring and in the driver
queues.

> As for internal queues, I don't know how blkfront manages that (or whether it has any pending work queue at all).

There are no internal queues, just the generic ones from blk_mq which
every block device has IIRC.

Thanks, Roger.

  reply index

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-14 23:21 [RFC RESEND PATCH v3 00/12] Enable PM hibernation on guest VMs Anchal Agarwal
2020-02-14 23:22 ` [RFC PATCH v3 01/12] xen/manage: keep track of the on-going suspend mode Anchal Agarwal
2020-02-14 23:23 ` [RFC PATCH v3 02/12] xenbus: add freeze/thaw/restore callbacks support Anchal Agarwal
2020-02-14 23:23 ` [RFC PATCH v3 03/12] x86/xen: Introduce new function to map HYPERVISOR_shared_info on Resume Anchal Agarwal
2020-02-14 23:24 ` [RFC PATCH v3 04/12] x86/xen: add system core suspend and resume callbacks Anchal Agarwal
2020-02-14 23:24 ` [RFC PATCH v3 05/12] xen-netfront: add callbacks for PM suspend and hibernation support Anchal Agarwal
2020-02-14 23:25 ` [RFC PATCH v3 06/12] xen-blkfront: add callbacks for PM suspend and hibernation Anchal Agarwal
2020-02-17 10:05   ` Roger Pau Monné
2020-02-17 23:05     ` Anchal Agarwal
2020-02-18  9:16       ` Roger Pau Monné
2020-02-19 18:04         ` Anchal Agarwal
2020-02-20  8:39           ` Roger Pau Monné
2020-02-20  8:54             ` [Xen-devel] " Durrant, Paul
2020-02-20 15:45               ` Roger Pau Monné
2020-02-20 16:23                 ` Durrant, Paul
2020-02-20 16:48                   ` Roger Pau Monné
2020-02-20 17:01                     ` Durrant, Paul
2020-02-21  0:49                       ` Anchal Agarwal
2020-02-21  9:47                         ` Roger Pau Monné
2020-02-21  9:22                       ` Roger Pau Monné
2020-02-21  9:56                         ` Durrant, Paul
2020-02-21 10:21                           ` Roger Pau Monné
2020-02-21 10:33                             ` Durrant, Paul
2020-02-21 11:51                               ` Roger Pau Monné [this message]
2020-02-21 14:24   ` Roger Pau Monné
2020-03-06 18:40     ` Anchal Agarwal
2020-03-09  9:54       ` Roger Pau Monné
     [not found]         ` <FA688A68-5372-4757-B075-A69A45671CB9@amazon.com>
     [not found]           ` <20200312090435.GK24449@Air-de-Roger.citrite.net>
2020-03-13 17:21             ` Anchal Agarwal
2020-02-14 23:25 ` [RFC PATCH v3 07/12] genirq: Shutdown irq chips in suspend/resume during hibernation Anchal Agarwal
2020-03-06 23:03   ` Thomas Gleixner
2020-03-09 22:37     ` [EXTERNAL][RFC " Anchal Agarwal
2020-02-14 23:26 ` [RFC PATCH v3 08/12] xen/time: introduce xen_{save,restore}_steal_clock Anchal Agarwal
2020-02-14 23:27 ` [RFC PATCH v3 09/12] x86/xen: save and restore steal clock Anchal Agarwal
2020-02-14 23:27 ` [RFC PATCH v3 10/12] xen: Introduce wrapper for save/restore sched clock offset Anchal Agarwal
2020-02-14 23:27 ` [RFC PATCH v3 11/12] xen: Update sched clock offset to avoid system instability in hibernation Anchal Agarwal
2020-02-14 23:28 ` [RFC PATCH v3 12/12] PM / hibernate: update the resume offset on SNAPSHOT_SET_SWAP_AREA Anchal Agarwal

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200221115103.GY4679@Air-de-Roger \
    --to=roger.pau@citrix.com \
    --cc=anchalag@amazon.com \
    --cc=axboe@kernel.dk \
    --cc=benh@kernel.crashing.org \
    --cc=boris.ostrovsky@oracle.com \
    --cc=bp@alien8.de \
    --cc=davem@davemloft.net \
    --cc=dwmw@amazon.co.uk \
    --cc=eduval@amazon.com \
    --cc=fllinden@amaozn.com \
    --cc=hpa@zytor.com \
    --cc=jgross@suse.com \
    --cc=kamatam@amazon.com \
    --cc=konrad.wilk@oracle.com \
    --cc=len.brown@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-pm@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=pavel@ucw.cz \
    --cc=pdurrant@amazon.co.uk \
    --cc=peterz@infradead.org \
    --cc=rjw@rjwysocki.net \
    --cc=sblbir@amazon.com \
    --cc=sstabellini@kernel.org \
    --cc=tglx@linutronix.de \
    --cc=vkuznets@redhat.com \
    --cc=x86@kernel.org \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-PM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-pm/0 linux-pm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-pm linux-pm/ https://lore.kernel.org/linux-pm \
		linux-pm@vger.kernel.org
	public-inbox-index linux-pm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-pm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git