From: "Rafael J. Wysocki" <rjw@rjwysocki.net>
To: Dan Williams <dan.j.williams@intel.com>
Cc: rafael.j.wysocki@intel.com, stable@vger.kernel.org,
Len Brown <lenb@kernel.org>, Borislav Petkov <bp@alien8.de>,
Ira Weiny <ira.weiny@intel.com>,
James Morse <james.morse@arm.com>,
Erik Kaneda <erik.kaneda@intel.com>,
Myron Stowe <myron.stowe@redhat.com>,
Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org,
linux-nvdimm@lists.01.org
Subject: [RFT][PATCH] ACPI: OSL: Use rwlock instead of RCU for memory management
Date: Fri, 05 Jun 2020 16:06:28 +0200 [thread overview]
Message-ID: <2643462.teTRrieJB7@kreacher> (raw)
In-Reply-To: <158889473309.2292982.18007035454673387731.stgit@dwillia2-desk3.amr.corp.intel.com>
From: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Subject: [PATCH] ACPI: OSL: Use rwlock instead of RCU for memory management
The ACPI OS layer uses RCU to protect the list of ACPI memory
mappings from being walked while it is updated. Among other
situations, that list can be walked in non-NMI interrupt context,
so using a sleeping lock to protect it is not an option.
However, performance issues related to the RCU usage in there
appear, as described by Dan Williams:
"Recently a performance problem was reported for a process invoking
a non-trival ASL program. The method call in this case ends up
repetitively triggering a call path like:
acpi_ex_store
acpi_ex_store_object_to_node
acpi_ex_write_data_to_field
acpi_ex_insert_into_field
acpi_ex_write_with_update_rule
acpi_ex_field_datum_io
acpi_ex_access_region
acpi_ev_address_space_dispatch
acpi_ex_system_memory_space_handler
acpi_os_map_cleanup.part.14
_synchronize_rcu_expedited.constprop.89
schedule
The end result of frequent synchronize_rcu_expedited() invocation is
tiny sub-millisecond spurts of execution where the scheduler freely
migrates this apparently sleepy task. The overhead of frequent
scheduler invocation multiplies the execution time by a factor
of 2-3X."
In order to avoid these issues, replace the RCU in the ACPI OS
layer by an rwlock.
That rwlock should not be frequently contended, so the performance
impact of it is not expected to be significant.
Reported-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
---
Hi Dan,
This is a possible fix for the ACPI OSL RCU-related performance issues, but
can you please arrange for the testing of it on the affected systems?
Cheers!
---
drivers/acpi/osl.c | 50 ++++++++++++++++++++++++++++++++++----------------
1 file changed, 34 insertions(+), 16 deletions(-)
Index: linux-pm/drivers/acpi/osl.c
===================================================================
--- linux-pm.orig/drivers/acpi/osl.c
+++ linux-pm/drivers/acpi/osl.c
@@ -81,8 +81,8 @@ struct acpi_ioremap {
};
static LIST_HEAD(acpi_ioremaps);
+static DEFINE_RWLOCK(acpi_ioremaps_list_lock);
static DEFINE_MUTEX(acpi_ioremap_lock);
-#define acpi_ioremap_lock_held() lock_is_held(&acpi_ioremap_lock.dep_map)
static void __init acpi_request_region (struct acpi_generic_address *gas,
unsigned int length, char *desc)
@@ -214,13 +214,13 @@ acpi_physical_address __init acpi_os_get
return pa;
}
-/* Must be called with 'acpi_ioremap_lock' or RCU read lock held. */
+/* Must be called with 'acpi_ioremap_lock' or 'acpi_ioremaps_list_lock' held. */
static struct acpi_ioremap *
acpi_map_lookup(acpi_physical_address phys, acpi_size size)
{
struct acpi_ioremap *map;
- list_for_each_entry_rcu(map, &acpi_ioremaps, list, acpi_ioremap_lock_held())
+ list_for_each_entry(map, &acpi_ioremaps, list)
if (map->phys <= phys &&
phys + size <= map->phys + map->size)
return map;
@@ -228,7 +228,7 @@ acpi_map_lookup(acpi_physical_address ph
return NULL;
}
-/* Must be called with 'acpi_ioremap_lock' or RCU read lock held. */
+/* Must be called with 'acpi_ioremap_lock' or 'acpi_ioremaps_list_lock' held. */
static void __iomem *
acpi_map_vaddr_lookup(acpi_physical_address phys, unsigned int size)
{
@@ -257,13 +257,13 @@ void __iomem *acpi_os_get_iomem(acpi_phy
}
EXPORT_SYMBOL_GPL(acpi_os_get_iomem);
-/* Must be called with 'acpi_ioremap_lock' or RCU read lock held. */
+/* Must be called with 'acpi_ioremap_lock' or 'acpi_ioremaps_list_lock' held. */
static struct acpi_ioremap *
acpi_map_lookup_virt(void __iomem *virt, acpi_size size)
{
struct acpi_ioremap *map;
- list_for_each_entry_rcu(map, &acpi_ioremaps, list, acpi_ioremap_lock_held())
+ list_for_each_entry(map, &acpi_ioremaps, list)
if (map->virt <= virt &&
virt + size <= map->virt + map->size)
return map;
@@ -360,7 +360,11 @@ void __iomem __ref
map->size = pg_sz;
map->refcount = 1;
- list_add_tail_rcu(&map->list, &acpi_ioremaps);
+ write_lock_irq(&acpi_ioremaps_list_lock);
+
+ list_add_tail(&map->list, &acpi_ioremaps);
+
+ write_unlock_irq(&acpi_ioremaps_list_lock);
out:
mutex_unlock(&acpi_ioremap_lock);
@@ -379,14 +383,18 @@ static unsigned long acpi_os_drop_map_re
{
unsigned long refcount = --map->refcount;
- if (!refcount)
- list_del_rcu(&map->list);
+ if (!refcount) {
+ write_lock_irq(&acpi_ioremaps_list_lock);
+
+ list_del(&map->list);
+
+ write_unlock_irq(&acpi_ioremaps_list_lock);
+ }
return refcount;
}
static void acpi_os_map_cleanup(struct acpi_ioremap *map)
{
- synchronize_rcu_expedited();
acpi_unmap(map->phys, map->virt);
kfree(map);
}
@@ -704,18 +712,23 @@ acpi_status
acpi_os_read_memory(acpi_physical_address phys_addr, u64 *value, u32 width)
{
void __iomem *virt_addr;
+ unsigned long flags;
unsigned int size = width / 8;
bool unmap = false;
u64 dummy;
int error;
- rcu_read_lock();
+ read_lock_irqsave(&acpi_ioremaps_list_lock, flags);
+
virt_addr = acpi_map_vaddr_lookup(phys_addr, size);
if (!virt_addr) {
- rcu_read_unlock();
+
+ read_unlock_irqrestore(&acpi_ioremaps_list_lock, flags);
+
virt_addr = acpi_os_ioremap(phys_addr, size);
if (!virt_addr)
return AE_BAD_ADDRESS;
+
unmap = true;
}
@@ -728,7 +741,7 @@ acpi_os_read_memory(acpi_physical_addres
if (unmap)
iounmap(virt_addr);
else
- rcu_read_unlock();
+ read_unlock_irqrestore(&acpi_ioremaps_list_lock, flags);
return AE_OK;
}
@@ -737,16 +750,21 @@ acpi_status
acpi_os_write_memory(acpi_physical_address phys_addr, u64 value, u32 width)
{
void __iomem *virt_addr;
+ unsigned long flags;
unsigned int size = width / 8;
bool unmap = false;
- rcu_read_lock();
+ read_lock_irqsave(&acpi_ioremaps_list_lock, flags);
+
virt_addr = acpi_map_vaddr_lookup(phys_addr, size);
if (!virt_addr) {
- rcu_read_unlock();
+
+ read_unlock_irqrestore(&acpi_ioremaps_list_lock, flags);
+
virt_addr = acpi_os_ioremap(phys_addr, size);
if (!virt_addr)
return AE_BAD_ADDRESS;
+
unmap = true;
}
@@ -770,7 +788,7 @@ acpi_os_write_memory(acpi_physical_addre
if (unmap)
iounmap(virt_addr);
else
- rcu_read_unlock();
+ read_unlock_irqrestore(&acpi_ioremaps_list_lock, flags);
return AE_OK;
}
next prev parent reply other threads:[~2020-06-05 14:06 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-07 23:39 [PATCH v2] ACPI: Drop rcu usage for MMIO mappings Dan Williams
2020-05-13 8:52 ` [ACPI] 5a91d41f89: BUG:sleeping_function_called_from_invalid_context_at_kernel/locking/mutex.c kernel test robot
2020-06-05 13:32 ` [PATCH v2] ACPI: Drop rcu usage for MMIO mappings Rafael J. Wysocki
2020-06-05 16:18 ` Dan Williams
2020-06-05 16:21 ` Rafael J. Wysocki
2020-06-05 16:39 ` Dan Williams
2020-06-05 17:02 ` Rafael J. Wysocki
2020-06-05 14:06 ` Rafael J. Wysocki [this message]
2020-06-05 17:08 ` [RFT][PATCH] ACPI: OSL: Use rwlock instead of RCU for memory management Dan Williams
2020-06-06 6:56 ` Rafael J. Wysocki
2020-06-08 15:33 ` Rafael J. Wysocki
2020-06-08 16:29 ` Rafael J. Wysocki
2020-06-05 19:40 ` Andy Shevchenko
2020-06-06 6:48 ` Rafael J. Wysocki
2020-06-10 12:17 ` [RFT][PATCH 0/3] ACPI: ACPICA / OSL: Avoid unmapping ACPI memory inside of the AML interpreter Rafael J. Wysocki
2020-06-10 12:20 ` [RFT][PATCH 1/3] ACPICA: Defer unmapping of memory used in memory opregions Rafael J. Wysocki
2020-06-10 12:21 ` [RFT][PATCH 2/3] ACPICA: Remove unused memory mappings on interpreter exit Rafael J. Wysocki
2020-06-12 0:12 ` Kaneda, Erik
2020-06-12 12:05 ` Rafael J. Wysocki
2020-06-13 19:28 ` Rafael J. Wysocki
2020-06-15 19:06 ` Dan Williams
2020-06-10 12:22 ` [RFT][PATCH 3/3] ACPI: OSL: Define ACPI_OS_MAP_MEMORY_FAST_PATH() Rafael J. Wysocki
2020-06-13 19:19 ` [RFT][PATCH 0/3] ACPI: ACPICA / OSL: Avoid unmapping ACPI memory inside of the AML interpreter Rafael J. Wysocki
2020-06-22 13:50 ` [RFT][PATCH v2 0/4] " Rafael J. Wysocki
2020-06-22 13:52 ` [RFT][PATCH v2 1/4] ACPICA: Defer unmapping of opregion memory if supported by OS Rafael J. Wysocki
2020-06-22 13:53 ` [RFT][PATCH v2 2/4] ACPI: OSL: Add support for deferred unmapping of ACPI memory Rafael J. Wysocki
2020-06-22 14:56 ` Andy Shevchenko
2020-06-22 15:27 ` Rafael J. Wysocki
2020-06-22 15:46 ` Andy Shevchenko
2020-06-22 14:01 ` [RFT][PATCH v2 3/4] ACPICA: Preserve memory opregion mappings if supported by OS Rafael J. Wysocki
2020-06-26 22:53 ` Kaneda, Erik
2020-06-29 13:02 ` Rafael J. Wysocki
2020-06-22 14:02 ` [RFT][PATCH v2 4/4] ACPI: OSL: Implement acpi_os_map_memory_fast_path() Rafael J. Wysocki
2020-06-26 17:28 ` [RFT][PATCH v3 0/4] ACPI: ACPICA / OSL: Avoid unmapping ACPI memory inside of the AML interpreter Rafael J. Wysocki
2020-06-26 17:31 ` [RFT][PATCH v3 1/4] ACPICA: Take deferred unmapping of memory into account Rafael J. Wysocki
2020-06-26 17:31 ` [RFT][PATCH v3 2/4] ACPI: OSL: Implement deferred unmapping of ACPI memory Rafael J. Wysocki
2020-06-26 17:32 ` [RFT][PATCH v3 3/4] ACPICA: Preserve memory opregion mappings if supported by OS Rafael J. Wysocki
2020-06-26 17:33 ` [RFT][PATCH v3 4/4] ACPI: OSL: Implement acpi_os_map_memory_fast_path() Rafael J. Wysocki
2020-06-26 18:41 ` [RFT][PATCH v3 0/4] ACPI: ACPICA / OSL: Avoid unmapping ACPI memory inside of the AML interpreter Dan Williams
2020-06-28 17:09 ` Rafael J. Wysocki
2020-06-29 20:46 ` Dan Williams
2020-06-30 11:04 ` Rafael J. Wysocki
2020-06-29 16:31 ` [PATCH v4 0/2] " Rafael J. Wysocki
2020-06-29 16:33 ` [PATCH v4 1/2] ACPI: OSL: Implement deferred unmapping of ACPI memory Rafael J. Wysocki
2020-06-29 16:33 ` [PATCH v4 2/2] ACPICA: Preserve memory opregion mappings Rafael J. Wysocki
2020-06-29 20:57 ` Al Stone
2020-06-30 11:44 ` Rafael J. Wysocki
2020-06-30 15:31 ` Al Stone
2020-06-30 15:52 ` Rafael J. Wysocki
2020-06-30 19:57 ` Al Stone
2020-07-16 19:22 ` Verma, Vishal L
2020-07-19 19:14 ` Rafael J. Wysocki
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2643462.teTRrieJB7@kreacher \
--to=rjw@rjwysocki.net \
--cc=andriy.shevchenko@linux.intel.com \
--cc=bp@alien8.de \
--cc=dan.j.williams@intel.com \
--cc=erik.kaneda@intel.com \
--cc=ira.weiny@intel.com \
--cc=james.morse@arm.com \
--cc=lenb@kernel.org \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvdimm@lists.01.org \
--cc=myron.stowe@redhat.com \
--cc=rafael.j.wysocki@intel.com \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).