From: Dan Williams <dan.j.williams@intel.com> To: linux-nvdimm@lists.01.org Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>, "Rafael J. Wysocki" <rafael@kernel.org>, Pavel Machek <pavel@ucw.cz>, Len Brown <len.brown@intel.com>, linux-acpi@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 11/12] PM, libnvdimm: Add syscore_quiesced() callback for firmware activation Date: Thu, 25 Jun 2020 16:51:19 -0700 [thread overview] Message-ID: <159312907937.1850128.15890323251117466770.stgit@dwillia2-desk3.amr.corp.intel.com> (raw) In-Reply-To: <159312902033.1850128.1712559453279208264.stgit@dwillia2-desk3.amr.corp.intel.com> The runtime firmware activation capability of Intel NVDIMM devices requires memory transactions to be disabled for 100s of microseconds. This timeout is large enough to cause in-flight DMA to fail and other application detectable timeouts. Arrange for firmware activation to be executed while the system is "quiesced" all suspend operations have completed successfully. Note that the placement of syscore_quiesced() before suspend_disable_secondary_cpus() and the "TEST_PLATFORM" early exit in suspend_enter(): if (suspend_test(TEST_PLATFORM)) goto Platform_wake; ...is a deliberate tradeoff. suspend_disable_secondary_cpus() causes violence to drivers with many interrupts allocated (server-class network adapters for example). So, allow for triggering firmware-activation without requiring all irq vectors to be routed (oversubscribed) to a single CPU. Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: "Rafael J. Wysocki" <rafael@kernel.org> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Vishal Verma <vishal.l.verma@intel.com> Cc: Dave Jiang <dave.jiang@intel.com> Cc: Ira Weiny <ira.weiny@intel.com> Cc: Pavel Machek <pavel@ucw.cz> Cc: Len Brown <len.brown@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com> --- drivers/base/syscore.c | 18 ++++++++++++++++++ drivers/nvdimm/bus.c | 30 ++++++++++++++++++++++++++++++ drivers/nvdimm/core.c | 3 +++ include/linux/syscore_ops.h | 2 ++ kernel/power/suspend.c | 2 ++ 5 files changed, 55 insertions(+) diff --git a/drivers/base/syscore.c b/drivers/base/syscore.c index 0d346a307140..9fbe47d5b50a 100644 --- a/drivers/base/syscore.c +++ b/drivers/base/syscore.c @@ -108,6 +108,24 @@ void syscore_resume(void) trace_suspend_resume(TPS("syscore_resume"), 0, false); } EXPORT_SYMBOL_GPL(syscore_resume); + +/** + * syscore_quiesced - Execute callbacks that need a quiesced system + * + * This function is executed after all syscore_suspend() callbacks have + * completed successfully. + */ +void syscore_quiesced(void) +{ + struct syscore_ops *ops; + + list_for_each_entry(ops, &syscore_ops_list, node) { + if (!ops->quiesced) + continue; + ops->quiesced(); + } +} + #endif /* CONFIG_PM_SLEEP */ /** diff --git a/drivers/nvdimm/bus.c b/drivers/nvdimm/bus.c index 955265656b96..c1b6ffe3d2fc 100644 --- a/drivers/nvdimm/bus.c +++ b/drivers/nvdimm/bus.c @@ -3,6 +3,7 @@ * Copyright(c) 2013-2015 Intel Corporation. All rights reserved. */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt +#include <linux/syscore_ops.h> #include <linux/libnvdimm.h> #include <linux/sched/mm.h> #include <linux/vmalloc.h> @@ -1289,6 +1290,33 @@ static const struct file_operations nvdimm_fops = { .llseek = noop_llseek, }; +static void trigger_fw_activate(struct nvdimm_bus_descriptor *nd_desc) +{ + if (!nd_desc->fw_ops) + return; + nd_desc->fw_ops->activate(nd_desc); +} + +static void nvdimm_syscore_quiesced(void) +{ + struct nvdimm_bus *nvdimm_bus; + + mutex_lock(&nvdimm_bus_list_mutex); + list_for_each_entry(nvdimm_bus, &nvdimm_bus_list, list) { + struct nvdimm_bus_descriptor *nd_desc = nvdimm_bus->nd_desc; + struct device *dev = &nvdimm_bus->dev; + + nvdimm_bus_lock(dev); + trigger_fw_activate(nd_desc); + nvdimm_bus_unlock(dev); + } + mutex_unlock(&nvdimm_bus_list_mutex); +} + +static struct syscore_ops nvdimm_syscore_ops = { + .quiesced = nvdimm_syscore_quiesced, +}; + int __init nvdimm_bus_init(void) { int rc; @@ -1317,6 +1345,8 @@ int __init nvdimm_bus_init(void) if (rc) goto err_nd_bus; + register_syscore_ops(&nvdimm_syscore_ops); + return 0; err_nd_bus: diff --git a/drivers/nvdimm/core.c b/drivers/nvdimm/core.c index b1cc7b35bd51..0cbb5620cd45 100644 --- a/drivers/nvdimm/core.c +++ b/drivers/nvdimm/core.c @@ -417,6 +417,9 @@ static ssize_t firmware_activate_store(struct device *dev, if (!nd_desc->fw_ops) return -EOPNOTSUPP; + if (!sysfs_streq(buf, "activate")) + return -EINVAL; + nvdimm_bus_lock(dev); state = nd_desc->fw_ops->activate_state(nd_desc); diff --git a/include/linux/syscore_ops.h b/include/linux/syscore_ops.h index ae4d48e4c970..2bffe7cca613 100644 --- a/include/linux/syscore_ops.h +++ b/include/linux/syscore_ops.h @@ -15,6 +15,7 @@ struct syscore_ops { int (*suspend)(void); void (*resume)(void); void (*shutdown)(void); + void (*quiesced)(void); }; extern void register_syscore_ops(struct syscore_ops *ops); @@ -22,6 +23,7 @@ extern void unregister_syscore_ops(struct syscore_ops *ops); #ifdef CONFIG_PM_SLEEP extern int syscore_suspend(void); extern void syscore_resume(void); +extern void syscore_quiesced(void); #endif extern void syscore_shutdown(void); diff --git a/kernel/power/suspend.c b/kernel/power/suspend.c index 8b1bb5ee7e5d..5929f49dc44c 100644 --- a/kernel/power/suspend.c +++ b/kernel/power/suspend.c @@ -414,6 +414,8 @@ static int suspend_enter(suspend_state_t state, bool *wakeup) if (error) goto Platform_wake; + syscore_quiesced(); + if (suspend_test(TEST_PLATFORM)) goto Platform_wake; _______________________________________________ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
WARNING: multiple messages have this Message-ID (diff)
From: Dan Williams <dan.j.williams@intel.com> To: linux-nvdimm@lists.01.org Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>, "Rafael J. Wysocki" <rafael@kernel.org>, Vishal Verma <vishal.l.verma@intel.com>, Dave Jiang <dave.jiang@intel.com>, Ira Weiny <ira.weiny@intel.com>, Pavel Machek <pavel@ucw.cz>, Len Brown <len.brown@intel.com>, linux-acpi@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 11/12] PM, libnvdimm: Add syscore_quiesced() callback for firmware activation Date: Thu, 25 Jun 2020 16:51:19 -0700 [thread overview] Message-ID: <159312907937.1850128.15890323251117466770.stgit@dwillia2-desk3.amr.corp.intel.com> (raw) In-Reply-To: <159312902033.1850128.1712559453279208264.stgit@dwillia2-desk3.amr.corp.intel.com> The runtime firmware activation capability of Intel NVDIMM devices requires memory transactions to be disabled for 100s of microseconds. This timeout is large enough to cause in-flight DMA to fail and other application detectable timeouts. Arrange for firmware activation to be executed while the system is "quiesced" all suspend operations have completed successfully. Note that the placement of syscore_quiesced() before suspend_disable_secondary_cpus() and the "TEST_PLATFORM" early exit in suspend_enter(): if (suspend_test(TEST_PLATFORM)) goto Platform_wake; ...is a deliberate tradeoff. suspend_disable_secondary_cpus() causes violence to drivers with many interrupts allocated (server-class network adapters for example). So, allow for triggering firmware-activation without requiring all irq vectors to be routed (oversubscribed) to a single CPU. Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: "Rafael J. Wysocki" <rafael@kernel.org> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Vishal Verma <vishal.l.verma@intel.com> Cc: Dave Jiang <dave.jiang@intel.com> Cc: Ira Weiny <ira.weiny@intel.com> Cc: Pavel Machek <pavel@ucw.cz> Cc: Len Brown <len.brown@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com> --- drivers/base/syscore.c | 18 ++++++++++++++++++ drivers/nvdimm/bus.c | 30 ++++++++++++++++++++++++++++++ drivers/nvdimm/core.c | 3 +++ include/linux/syscore_ops.h | 2 ++ kernel/power/suspend.c | 2 ++ 5 files changed, 55 insertions(+) diff --git a/drivers/base/syscore.c b/drivers/base/syscore.c index 0d346a307140..9fbe47d5b50a 100644 --- a/drivers/base/syscore.c +++ b/drivers/base/syscore.c @@ -108,6 +108,24 @@ void syscore_resume(void) trace_suspend_resume(TPS("syscore_resume"), 0, false); } EXPORT_SYMBOL_GPL(syscore_resume); + +/** + * syscore_quiesced - Execute callbacks that need a quiesced system + * + * This function is executed after all syscore_suspend() callbacks have + * completed successfully. + */ +void syscore_quiesced(void) +{ + struct syscore_ops *ops; + + list_for_each_entry(ops, &syscore_ops_list, node) { + if (!ops->quiesced) + continue; + ops->quiesced(); + } +} + #endif /* CONFIG_PM_SLEEP */ /** diff --git a/drivers/nvdimm/bus.c b/drivers/nvdimm/bus.c index 955265656b96..c1b6ffe3d2fc 100644 --- a/drivers/nvdimm/bus.c +++ b/drivers/nvdimm/bus.c @@ -3,6 +3,7 @@ * Copyright(c) 2013-2015 Intel Corporation. All rights reserved. */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt +#include <linux/syscore_ops.h> #include <linux/libnvdimm.h> #include <linux/sched/mm.h> #include <linux/vmalloc.h> @@ -1289,6 +1290,33 @@ static const struct file_operations nvdimm_fops = { .llseek = noop_llseek, }; +static void trigger_fw_activate(struct nvdimm_bus_descriptor *nd_desc) +{ + if (!nd_desc->fw_ops) + return; + nd_desc->fw_ops->activate(nd_desc); +} + +static void nvdimm_syscore_quiesced(void) +{ + struct nvdimm_bus *nvdimm_bus; + + mutex_lock(&nvdimm_bus_list_mutex); + list_for_each_entry(nvdimm_bus, &nvdimm_bus_list, list) { + struct nvdimm_bus_descriptor *nd_desc = nvdimm_bus->nd_desc; + struct device *dev = &nvdimm_bus->dev; + + nvdimm_bus_lock(dev); + trigger_fw_activate(nd_desc); + nvdimm_bus_unlock(dev); + } + mutex_unlock(&nvdimm_bus_list_mutex); +} + +static struct syscore_ops nvdimm_syscore_ops = { + .quiesced = nvdimm_syscore_quiesced, +}; + int __init nvdimm_bus_init(void) { int rc; @@ -1317,6 +1345,8 @@ int __init nvdimm_bus_init(void) if (rc) goto err_nd_bus; + register_syscore_ops(&nvdimm_syscore_ops); + return 0; err_nd_bus: diff --git a/drivers/nvdimm/core.c b/drivers/nvdimm/core.c index b1cc7b35bd51..0cbb5620cd45 100644 --- a/drivers/nvdimm/core.c +++ b/drivers/nvdimm/core.c @@ -417,6 +417,9 @@ static ssize_t firmware_activate_store(struct device *dev, if (!nd_desc->fw_ops) return -EOPNOTSUPP; + if (!sysfs_streq(buf, "activate")) + return -EINVAL; + nvdimm_bus_lock(dev); state = nd_desc->fw_ops->activate_state(nd_desc); diff --git a/include/linux/syscore_ops.h b/include/linux/syscore_ops.h index ae4d48e4c970..2bffe7cca613 100644 --- a/include/linux/syscore_ops.h +++ b/include/linux/syscore_ops.h @@ -15,6 +15,7 @@ struct syscore_ops { int (*suspend)(void); void (*resume)(void); void (*shutdown)(void); + void (*quiesced)(void); }; extern void register_syscore_ops(struct syscore_ops *ops); @@ -22,6 +23,7 @@ extern void unregister_syscore_ops(struct syscore_ops *ops); #ifdef CONFIG_PM_SLEEP extern int syscore_suspend(void); extern void syscore_resume(void); +extern void syscore_quiesced(void); #endif extern void syscore_shutdown(void); diff --git a/kernel/power/suspend.c b/kernel/power/suspend.c index 8b1bb5ee7e5d..5929f49dc44c 100644 --- a/kernel/power/suspend.c +++ b/kernel/power/suspend.c @@ -414,6 +414,8 @@ static int suspend_enter(suspend_state_t state, bool *wakeup) if (error) goto Platform_wake; + syscore_quiesced(); + if (suspend_test(TEST_PLATFORM)) goto Platform_wake;
next prev parent reply other threads:[~2020-06-26 0:07 UTC|newest] Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-06-25 23:50 [PATCH 00/12] ACPI/NVDIMM: Runtime Firmware Activation Dan Williams 2020-06-25 23:50 ` Dan Williams 2020-06-25 23:50 ` [PATCH 01/12] libnvdimm: Validate command family indices Dan Williams 2020-06-25 23:50 ` Dan Williams 2020-07-01 19:33 ` Sasha Levin 2020-07-01 19:33 ` Sasha Levin 2020-07-10 14:02 ` Sasha Levin 2020-07-10 14:02 ` Sasha Levin 2020-06-25 23:50 ` [PATCH 02/12] ACPI: NFIT: Move bus_dsm_mask out of generic nvdimm_bus_descriptor Dan Williams 2020-06-25 23:50 ` Dan Williams 2020-06-25 23:50 ` [PATCH 03/12] ACPI: NFIT: Define runtime firmware activation commands Dan Williams 2020-06-25 23:50 ` Dan Williams 2020-06-25 23:50 ` [PATCH 04/12] tools/testing/nvdimm: Cleanup dimm index passing Dan Williams 2020-06-25 23:50 ` Dan Williams 2020-06-25 23:50 ` [PATCH 05/12] tools/testing/nvdimm: Add command debug messages Dan Williams 2020-06-25 23:50 ` Dan Williams 2020-06-25 23:50 ` [PATCH 06/12] tools/testing/nvdimm: Prepare nfit_ctl_test() for ND_CMD_CALL emulation Dan Williams 2020-06-25 23:50 ` Dan Williams 2020-06-25 23:50 ` [PATCH 07/12] tools/testing/nvdimm: Emulate firmware activation commands Dan Williams 2020-06-25 23:50 ` Dan Williams 2020-06-25 23:51 ` [PATCH 08/12] driver-core: Introduce DEVICE_ATTR_ADMIN_{RO,RW} Dan Williams 2020-06-25 23:51 ` Dan Williams 2020-06-26 5:06 ` Greg Kroah-Hartman 2020-06-26 5:06 ` Greg Kroah-Hartman 2020-06-26 5:09 ` Dan Williams 2020-06-26 5:09 ` Dan Williams 2020-06-25 23:51 ` [PATCH 09/12] libnvdimm: Convert to DEVICE_ATTR_ADMIN_RO() Dan Williams 2020-06-25 23:51 ` Dan Williams 2020-06-25 23:51 ` [PATCH 10/12] libnvdimm: Add runtime firmware activation sysfs interface Dan Williams 2020-06-25 23:51 ` Dan Williams 2020-06-25 23:51 ` Dan Williams [this message] 2020-06-25 23:51 ` [PATCH 11/12] PM, libnvdimm: Add syscore_quiesced() callback for firmware activation Dan Williams 2020-06-26 14:23 ` Rafael J. Wysocki 2020-06-26 14:23 ` Rafael J. Wysocki 2020-06-25 23:51 ` [PATCH 12/12] ACPI: NFIT: Add runtime firmware activate support Dan Williams 2020-06-25 23:51 ` Dan Williams 2020-06-26 14:22 ` [PATCH 00/12] ACPI/NVDIMM: Runtime Firmware Activation Rafael J. Wysocki 2020-06-26 14:22 ` Rafael J. Wysocki 2020-06-26 18:43 ` Dan Williams 2020-06-26 18:43 ` Dan Williams 2020-06-28 17:22 ` Rafael J. Wysocki 2020-06-28 17:22 ` Rafael J. Wysocki 2020-06-29 23:37 ` Dan Williams 2020-06-29 23:37 ` Dan Williams 2020-06-30 10:55 ` Rafael J. Wysocki 2020-06-30 10:55 ` Rafael J. Wysocki
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=159312907937.1850128.15890323251117466770.stgit@dwillia2-desk3.amr.corp.intel.com \ --to=dan.j.williams@intel.com \ --cc=gregkh@linuxfoundation.org \ --cc=len.brown@intel.com \ --cc=linux-acpi@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-nvdimm@lists.01.org \ --cc=pavel@ucw.cz \ --cc=rafael@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.