From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C7CFC3565C for ; Fri, 21 Feb 2020 11:51:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B0A3924650 for ; Fri, 21 Feb 2020 11:51:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="PQAnIeMg" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B0A3924650 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 03B446B0003; Fri, 21 Feb 2020 06:51:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F08CE6B0006; Fri, 21 Feb 2020 06:51:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA7B06B0007; Fri, 21 Feb 2020 06:51:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0102.hostedemail.com [216.40.44.102]) by kanga.kvack.org (Postfix) with ESMTP id BF2AD6B0003 for ; Fri, 21 Feb 2020 06:51:14 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 61E17180AD807 for ; Fri, 21 Feb 2020 11:51:14 +0000 (UTC) X-FDA: 76513968468.19.hate21_7f1cc99951820 X-HE-Tag: hate21_7f1cc99951820 X-Filterd-Recvd-Size: 10642 Received: from esa4.hc3370-68.iphmx.com (esa4.hc3370-68.iphmx.com [216.71.155.144]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Fri, 21 Feb 2020 11:51:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1582285873; h=date:from:to:cc:subject:message-id:references: mime-version:content-transfer-encoding:in-reply-to; bh=rkTHwapBz7ayghLmZpskUOnKPxyODDcUxzL7ge8Xcgw=; b=PQAnIeMgICrQHpmZKDOIReVztVnq6KEAwa1ppX6B3Qm3WK/WlTjN6TaX AVwv9+RLhQznSyx/B01gM4Ol7XilKwxWEw7CoWGNzVKb4DKX9j+UfXO9W X2BuHZGhsxe+UHQ69DiD01BRVExQec9z7PRyAjV14j3777Q9XGkp3+M89 M=; Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa4.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: Iy6vkmYvTTmAxcv0sDnk287OqaApR945+t3bbH8+KbvqO+kHUjaYPybqDsL2Uz3ckkNdz7m7Ij Y/kSARbsrffvGwZ0DdF9joLHQxzx+xK9WlQ/XDSamvWxHYxvaUCzUm0cfcLxehNY2mp7FjoWjT D4xjlWsuciis2JlVC9Q0xgZJzdXhb7ZyOnQTX4w+09xcbPdlJMXCastv+6vAPcXH/ReEzapMJk MexwEdNGylwcSepie7bUtm+7uwAyQBY4nMkkPM9+s4Fdfth/Af4mJg7YaOlCyON2+hTYr7itc9 ax0= X-SBRS: 2.7 X-MesageID: 13435837 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.70,468,1574139600"; d="scan'208";a="13435837" Date: Fri, 21 Feb 2020 12:51:03 +0100 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: "Durrant, Paul" CC: "Agarwal, Anchal" , "Valentin, Eduardo" , "len.brown@intel.com" , "peterz@infradead.org" , "benh@kernel.crashing.org" , "x86@kernel.org" , "linux-mm@kvack.org" , "pavel@ucw.cz" , "hpa@zytor.com" , "tglx@linutronix.de" , "sstabellini@kernel.org" , "fllinden@amaozn.com" , "Kamata, Munehisa" , "mingo@redhat.com" , "xen-devel@lists.xenproject.org" , "Singh, Balbir" , "axboe@kernel.dk" , "konrad.wilk@oracle.com" , "bp@alien8.de" , "boris.ostrovsky@oracle.com" , "jgross@suse.com" , "netdev@vger.kernel.org" , "linux-pm@vger.kernel.org" , "rjw@rjwysocki.net" , "linux-kernel@vger.kernel.org" , "vkuznets@redhat.com" , "davem@davemloft.net" , "Woodhouse, David" Subject: Re: [Xen-devel] [RFC PATCH v3 06/12] xen-blkfront: add callbacks for PM suspend and hibernation Message-ID: <20200221115103.GY4679@Air-de-Roger> References: <20200220083904.GI4679@Air-de-Roger> <20200220154507.GO4679@Air-de-Roger> <20200220164839.GR4679@Air-de-Roger> <20200221092219.GU4679@Air-de-Roger> <5ddf980a3fba4fb39571184e688cefc5@EX13D32EUC003.ant.amazon.com> <20200221102130.GW4679@Air-de-Roger> <66a211bae1de4be9861ef8393607d1b3@EX13D32EUC003.ant.amazon.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline In-Reply-To: <66a211bae1de4be9861ef8393607d1b3@EX13D32EUC003.ant.amazon.com> X-ClientProxiedBy: AMSPEX02CAS02.citrite.net (10.69.22.113) To AMSPEX02CL01.citrite.net (10.69.22.125) Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Feb 21, 2020 at 10:33:42AM +0000, Durrant, Paul wrote: > > -----Original Message----- > > From: Roger Pau Monn=C3=A9 > > Sent: 21 February 2020 10:22 > > To: Durrant, Paul > > Cc: Agarwal, Anchal ; Valentin, Eduardo > > ; len.brown@intel.com; peterz@infradead.org; > > benh@kernel.crashing.org; x86@kernel.org; linux-mm@kvack.org; > > pavel@ucw.cz; hpa@zytor.com; tglx@linutronix.de; sstabellini@kernel.o= rg; > > fllinden@amaozn.com; Kamata, Munehisa ; > > mingo@redhat.com; xen-devel@lists.xenproject.org; Singh, Balbir > > ; axboe@kernel.dk; konrad.wilk@oracle.com; > > bp@alien8.de; boris.ostrovsky@oracle.com; jgross@suse.com; > > netdev@vger.kernel.org; linux-pm@vger.kernel.org; rjw@rjwysocki.net; > > linux-kernel@vger.kernel.org; vkuznets@redhat.com; davem@davemloft.ne= t; > > Woodhouse, David > > Subject: Re: [Xen-devel] [RFC PATCH v3 06/12] xen-blkfront: add callb= acks > > for PM suspend and hibernation > >=20 > > On Fri, Feb 21, 2020 at 09:56:54AM +0000, Durrant, Paul wrote: > > > > -----Original Message----- > > > > From: Roger Pau Monn=C3=A9 > > > > Sent: 21 February 2020 09:22 > > > > To: Durrant, Paul > > > > Cc: Agarwal, Anchal ; Valentin, Eduardo > > > > ; len.brown@intel.com; peterz@infradead.org; > > > > benh@kernel.crashing.org; x86@kernel.org; linux-mm@kvack.org; > > > > pavel@ucw.cz; hpa@zytor.com; tglx@linutronix.de; > > sstabellini@kernel.org; > > > > fllinden@amaozn.com; Kamata, Munehisa ; > > > > mingo@redhat.com; xen-devel@lists.xenproject.org; Singh, Balbir > > > > ; axboe@kernel.dk; konrad.wilk@oracle.com; > > > > bp@alien8.de; boris.ostrovsky@oracle.com; jgross@suse.com; > > > > netdev@vger.kernel.org; linux-pm@vger.kernel.org; rjw@rjwysocki.n= et; > > > > linux-kernel@vger.kernel.org; vkuznets@redhat.com; > > davem@davemloft.net; > > > > Woodhouse, David > > > > Subject: Re: [Xen-devel] [RFC PATCH v3 06/12] xen-blkfront: add > > callbacks > > > > for PM suspend and hibernation > > > > > > > > On Thu, Feb 20, 2020 at 05:01:52PM +0000, Durrant, Paul wrote: > > > > > > > Hopefully what I said above illustrates why it may not be 1= 00% > > > > common. > > > > > > > > > > > > Yes, that's fine. I don't expect it to be 100% common (as I g= uess > > > > > > that the hooks will have different prototypes), but I expect > > > > > > that routines can be shared, and that the approach taken can = be > > the > > > > > > same. > > > > > > > > > > > > For example one necessary difference will be that xenbus init= iated > > > > > > suspend won't close the PV connection, in case suspension fai= ls. > > On PM > > > > > > suspend you seem to always close the connection beforehand, s= o you > > > > > > will always have to re-negotiate on resume even if suspension > > failed. > > > > > > > > > > > > What I'm mostly worried about is the different approach to ri= ng > > > > > > draining. Ie: either xenbus is changed to freeze the queues a= nd > > drain > > > > > > the shared rings, or PM uses the already existing logic of no= t > > > > > > flushing the rings an re-issuing in-flight requests on resume= . > > > > > > > > > > > > > > > > Yes, that's needs consideration. I don=E2=80=99t think the same= semantic can > > be > > > > suitable for both. E.g. in a xen-suspend we need to freeze with a= s > > little > > > > processing as possible to avoid dirtying RAM late in the migratio= n > > cycle, > > > > and we know that in-flight data can wait. But in a transition to = S4 we > > > > need to make sure that at least all the in-flight blkif requests = get > > > > completed, since they probably contain bits of the guest's memory > > image > > > > and that's not going to get saved any other way. > > > > > > > > Thanks, that makes sense and something along this lines should be > > > > added to the commit message IMO. > > > > > > > > Wondering about S4, shouldn't we expect the queues to already be > > > > empty? As any subsystem that wanted to store something to disk sh= ould > > > > make sure requests have been successfully completed before > > > > suspending. > > > > > > What about writing the suspend image itself? Normal filesystem I/O > > > will have been flushed of course, but whatever vestigial kernel > > > actually writes out the hibernation file may well expect a final > > > D0->D3 on the storage device to cause a flush. > >=20 > > Hm, I have no idea really. I think whatever writes to the disk before > > suspend should actually make sure requests have completed, but what > > you suggest might also be a possibility. > >=20 > > Can you figure out whether there are requests on the ring or in the > > queue before suspending? >=20 > Well there's clearly pending stuff in the ring if rsp_prod !=3D req_pro= d :-) Right, I assume there's no document that states what's the expected state for queues &c when switching PM states, so we have to assume that there might be in-flight requests on the ring and in the driver queues. > As for internal queues, I don't know how blkfront manages that (or whet= her it has any pending work queue at all). There are no internal queues, just the generic ones from blk_mq which every block device has IIRC. Thanks, Roger.