From: Anchal Agarwal <anchalag@amazon.com>
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: eduval@amazon.com, len.brown@intel.com, peterz@infradead.org,
benh@kernel.crashing.org, x86@kernel.org, linux-mm@kvack.org,
pavel@ucw.cz, hpa@zytor.com, sstabellini@kernel.org,
kamatam@amazon.com, mingo@redhat.com,
xen-devel@lists.xenproject.org, sblbir@amazon.com,
axboe@kernel.dk, konrad.wilk@oracle.com, anchalag@amazon.com,
bp@alien8.de, tglx@linutronix.de, jgross@suse.com,
netdev@vger.kernel.org, linux-pm@vger.kernel.org,
rjw@rjwysocki.net, linux-kernel@vger.kernel.org,
vkuznets@redhat.com, davem@davemloft.net, dwmw@amazon.co.uk,
roger.pau@citrix.com
Subject: Re: [PATCH v2 01/11] xen/manage: keep track of the on-going suspend mode
Date: Wed, 15 Jul 2020 20:49:43 +0000 [thread overview]
Message-ID: <20200715204943.GB17938@dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com> (raw)
In-Reply-To: <50298859-0d0e-6eb0-029b-30df2a4ecd63@oracle.com>
On Mon, Jul 13, 2020 at 11:52:01AM -0400, Boris Ostrovsky wrote:
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> On 7/2/20 2:21 PM, Anchal Agarwal wrote:
> > From: Munehisa Kamata <kamatam@amazon.com>
> >
> > Guest hibernation is different from xen suspend/resume/live migration.
> > Xen save/restore does not use pm_ops as is needed by guest hibernation.
> > Hibernation in guest follows ACPI path and is guest inititated , the
> > hibernation image is saved within guest as compared to later modes
> > which are xen toolstack assisted and image creation/storage is in
> > control of hypervisor/host machine.
> > To differentiate between Xen suspend and PM hibernation, keep track
> > of the on-going suspend mode by mainly using a new PM notifier.
> > Introduce simple functions which help to know the on-going suspend mode
> > so that other Xen-related code can behave differently according to the
> > current suspend mode.
> > Since Xen suspend doesn't have corresponding PM event, its main logic
> > is modfied to acquire pm_mutex and set the current mode.
> >
> > Though, acquirng pm_mutex is still right thing to do, we may
> > see deadlock if PM hibernation is interrupted by Xen suspend.
> > PM hibernation depends on xenwatch thread to process xenbus state
> > transactions, but the thread will sleep to wait pm_mutex which is
> > already held by PM hibernation context in the scenario. Xen shutdown
> > code may need some changes to avoid the issue.
> >
> > [Anchal Agarwal: Changelog]:
> > RFC v1->v2: Code refactoring
> > v1->v2: Remove unused functions for PM SUSPEND/PM hibernation
> >
> > Signed-off-by: Anchal Agarwal <anchalag@amazon.com>
> > Signed-off-by: Munehisa Kamata <kamatam@amazon.com>
> > ---
> > drivers/xen/manage.c | 60 +++++++++++++++++++++++++++++++++++++++++++
> > include/xen/xen-ops.h | 1 +
> > 2 files changed, 61 insertions(+)
> >
> > diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c
> > index cd046684e0d1..69833fd6cfd1 100644
> > --- a/drivers/xen/manage.c
> > +++ b/drivers/xen/manage.c
> > @@ -14,6 +14,7 @@
> > #include <linux/freezer.h>
> > #include <linux/syscore_ops.h>
> > #include <linux/export.h>
> > +#include <linux/suspend.h>
> >
> > #include <xen/xen.h>
> > #include <xen/xenbus.h>
> > @@ -40,6 +41,20 @@ enum shutdown_state {
> > /* Ignore multiple shutdown requests. */
> > static enum shutdown_state shutting_down = SHUTDOWN_INVALID;
> >
> > +enum suspend_modes {
> > + NO_SUSPEND = 0,
> > + XEN_SUSPEND,
> > + PM_HIBERNATION,
> > +};
> > +
> > +/* Protected by pm_mutex */
> > +static enum suspend_modes suspend_mode = NO_SUSPEND;
> > +
> > +bool xen_is_xen_suspend(void)
>
>
> Weren't you going to call this pv suspend? (And also --- is this suspend
> or hibernation? Your commit messages and cover letter talk about fixing
> hibernation).
>
>
This is for hibernation is for pvhvm/hvm/pv-on-hvm guests as you may call it.
The method is just there to check if "xen suspend" is in progress.
I do not see "xen_suspend" differentiating between pv or hvm
domain until later in the code hence, I abstracted it to xen_is_xen_suspend.
> > +{
> > + return suspend_mode == XEN_SUSPEND;
> > +}
> > +
>
>
>
> > +
> > +static int xen_pm_notifier(struct notifier_block *notifier,
> > + unsigned long pm_event, void *unused)
> > +{
> > + switch (pm_event) {
> > + case PM_SUSPEND_PREPARE:
> > + case PM_HIBERNATION_PREPARE:
> > + case PM_RESTORE_PREPARE:
> > + suspend_mode = PM_HIBERNATION;
>
>
> Do you ever use this mode? It seems to me all you care about is whether
> or not we are doing XEN_SUSPEND. And so perhaps suspend_mode could
> become a boolean. And then maybe even drop it altogether because it you
> should be able to key off (shutting_down == SHUTDOWN_SUSPEND).
>
>
The mode was left there in case its needed for restore prepare cases. But you
are right the only thing I currently care about whether shutting_down ==
SHUTDOWN_SUSPEND. Infact, the notifier may not be needed in first place.
xen_is_xen_suspend could work right off the bat using 'shutting_down' variable
itself. *I think so* I will test it on my end and send an updated patch.
> > + break;
> > + case PM_POST_SUSPEND:
> > + case PM_POST_RESTORE:
> > + case PM_POST_HIBERNATION:
> > + /* Set back to the default */
> > + suspend_mode = NO_SUSPEND;
> > + break;
> > + default:
> > + pr_warn("Receive unknown PM event 0x%lx\n", pm_event);
> > + return -EINVAL;
> > + }
> > +
> > + return 0;
> > +};
>
>
>
> > +static int xen_setup_pm_notifier(void)
> > +{
> > + if (!xen_hvm_domain())
> > + return -ENODEV;
>
>
> I forgot --- what did we decide about non-x86 (i.e. ARM)?
It would be great to support that however, its out of
scope for this patch set.
I’ll be happy to discuss it separately.
>
>
> And PVH dom0.
That's another good use case to make it work with however, I still
think that should be tested/worked upon separately as the feature itself
(PVH Dom0) is very new.
>
>
Thanks,
Anchal
> -boris
>
>
>
next prev parent reply other threads:[~2020-07-15 20:50 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-02 18:21 [PATCH v2 00/11] Fix PM hibernation in Xen guests Anchal Agarwal
2020-07-02 18:21 ` [PATCH v2 01/11] xen/manage: keep track of the on-going suspend mode Anchal Agarwal
2020-07-13 15:52 ` Boris Ostrovsky
2020-07-15 20:49 ` Anchal Agarwal [this message]
2020-07-15 21:18 ` Boris Ostrovsky
2020-07-17 19:10 ` Anchal Agarwal
2020-07-19 1:47 ` Boris Ostrovsky
2020-07-20 9:37 ` Roger Pau Monné
2020-07-21 0:17 ` Anchal Agarwal
2020-07-21 8:30 ` Roger Pau Monné
2020-07-21 19:55 ` Anchal Agarwal
2020-07-22 8:27 ` Roger Pau Monné
2020-07-21 0:03 ` Anchal Agarwal
2020-07-21 21:48 ` Boris Ostrovsky
2020-07-22 0:18 ` Stefano Stabellini
2020-07-22 18:02 ` Anchal Agarwal
2020-07-22 22:45 ` Boris Ostrovsky
2020-07-22 23:49 ` Stefano Stabellini
2020-07-23 22:57 ` Anchal Agarwal
2020-07-24 23:01 ` Stefano Stabellini
2020-07-27 22:08 ` Boris Ostrovsky
2020-07-30 23:06 ` Anchal Agarwal
2020-07-31 14:13 ` Boris Ostrovsky
2020-07-31 14:25 ` Rafael J. Wysocki
2020-08-04 23:42 ` Anchal Agarwal
2020-08-05 13:31 ` Boris Ostrovsky
2020-08-05 17:42 ` Anchal Agarwal
2020-07-02 18:21 ` [PATCH v2 03/11] x86/xen: Introduce new function to map HYPERVISOR_shared_info on Resume Anchal Agarwal
2020-07-02 18:22 ` [PATCH v2 04/11] x86/xen: add system core suspend and resume callbacks Anchal Agarwal
2020-07-22 9:08 ` Julien Grall
2020-07-02 18:22 ` [PATCH v2 05/11] genirq: Shutdown irq chips in suspend/resume during hibernation Anchal Agarwal
2020-07-02 18:22 ` [PATCH v2 06/11] xen-blkfront: add callbacks for PM suspend and hibernation Anchal Agarwal
2020-07-02 18:22 ` [PATCH v2 07/11] xen-netfront: " Anchal Agarwal
2020-07-02 18:22 ` [PATCH v2 08/11] x86/xen: save and restore steal clock during PM hibernation Anchal Agarwal
2020-07-02 18:23 ` [PATCH v2 09/11] xen: Introduce wrapper for save/restore sched clock offset Anchal Agarwal
2020-07-02 18:23 ` [PATCH v2 10/11] xen: Update sched clock offset to avoid system instability in hibernation Anchal Agarwal
2020-07-02 18:23 ` [PATCH v2 11/11] PM / hibernate: update the resume offset on SNAPSHOT_SET_SWAP_AREA Anchal Agarwal
2020-07-02 18:25 ` [PATCH v2 02/11] xenbus: add freeze/thaw/restore callbacks support Anchal Agarwal
2020-07-10 18:17 ` [PATCH v2 00/11] Fix PM hibernation in Xen guests Agarwal, Anchal
2020-07-13 19:43 ` Boris Ostrovsky
2020-07-15 19:49 ` Anchal Agarwal
2020-07-15 20:49 ` Boris Ostrovsky
2020-07-16 23:28 ` Anchal Agarwal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200715204943.GB17938@dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com \
--to=anchalag@amazon.com \
--cc=axboe@kernel.dk \
--cc=benh@kernel.crashing.org \
--cc=boris.ostrovsky@oracle.com \
--cc=bp@alien8.de \
--cc=davem@davemloft.net \
--cc=dwmw@amazon.co.uk \
--cc=eduval@amazon.com \
--cc=hpa@zytor.com \
--cc=jgross@suse.com \
--cc=kamatam@amazon.com \
--cc=konrad.wilk@oracle.com \
--cc=len.brown@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-pm@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=pavel@ucw.cz \
--cc=peterz@infradead.org \
--cc=rjw@rjwysocki.net \
--cc=roger.pau@citrix.com \
--cc=sblbir@amazon.com \
--cc=sstabellini@kernel.org \
--cc=tglx@linutronix.de \
--cc=vkuznets@redhat.com \
--cc=x86@kernel.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).