From: Andi Kleen <andi@firstfloor.org>
To: bhelgaas@google.com
Cc: x86@kernel.org, linux-pci@vger.kernel.org, eranian@google.com,
peterz@infradead.org, linux-kernel@vger.kernel.org,
Andi Kleen <ak@linux.intel.com>
Subject: [PATCH 1/4] pci: Allow lockless access path to PCI mmconfig
Date: Thu, 2 Mar 2017 15:21:01 -0800 [thread overview]
Message-ID: <20170302232104.10136-1-andi@firstfloor.org> (raw)
From: Andi Kleen <ak@linux.intel.com>
The Intel uncore driver can do a lot of PCI config accesses to read
performance counters. I had a situation on a 4S system where it
was spending 40+% of CPU time grabbing the pci_cfg_lock due to that.
For 64bit x86 with MMCONFIG there isn't really any reason to take
a lock. The access is directly mapped to an underlying MMIO area,
which can fully operate lockless.
Add a new flag that allows the PCI mid layer to skip the lock
and set it for the 64bit mmconfig code.
There's a small risk that someone relies on this lock for synchronization,
but I think that's unlikely because there isn't really any useful
synchronization at this individual operation level. Any useful
synchronization would likely need to protect at least a
read-modify-write or similar. So I made it unconditional without opt-in.
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
arch/x86/pci/mmconfig_64.c | 1 +
drivers/pci/access.c | 14 ++++++++++----
include/linux/pci.h | 2 ++
3 files changed, 13 insertions(+), 4 deletions(-)
diff --git a/arch/x86/pci/mmconfig_64.c b/arch/x86/pci/mmconfig_64.c
index bea52496aea6..8bf10f41e626 100644
--- a/arch/x86/pci/mmconfig_64.c
+++ b/arch/x86/pci/mmconfig_64.c
@@ -121,6 +121,7 @@ int __init pci_mmcfg_arch_init(void)
}
raw_pci_ext_ops = &pci_mmcfg;
+ pci_root_ops.ll_allowed = true;
return 1;
}
diff --git a/drivers/pci/access.c b/drivers/pci/access.c
index db239547fefd..22552c6606c1 100644
--- a/drivers/pci/access.c
+++ b/drivers/pci/access.c
@@ -32,11 +32,14 @@ int pci_bus_read_config_##size \
int res; \
unsigned long flags; \
u32 data = 0; \
+ bool ll_allowed = bus->ops->ll_allowed; \
if (PCI_##size##_BAD) return PCIBIOS_BAD_REGISTER_NUMBER; \
- raw_spin_lock_irqsave(&pci_lock, flags); \
+ if (!ll_allowed) \
+ raw_spin_lock_irqsave(&pci_lock, flags); \
res = bus->ops->read(bus, devfn, pos, len, &data); \
*value = (type)data; \
- raw_spin_unlock_irqrestore(&pci_lock, flags); \
+ if (!ll_allowed) \
+ raw_spin_unlock_irqrestore(&pci_lock, flags); \
return res; \
}
@@ -46,10 +49,13 @@ int pci_bus_write_config_##size \
{ \
int res; \
unsigned long flags; \
+ bool ll_allowed = bus->ops->ll_allowed; \
if (PCI_##size##_BAD) return PCIBIOS_BAD_REGISTER_NUMBER; \
- raw_spin_lock_irqsave(&pci_lock, flags); \
+ if (!ll_allowed) \
+ raw_spin_lock_irqsave(&pci_lock, flags); \
res = bus->ops->write(bus, devfn, pos, len, value); \
- raw_spin_unlock_irqrestore(&pci_lock, flags); \
+ if (!ll_allowed) \
+ raw_spin_unlock_irqrestore(&pci_lock, flags); \
return res; \
}
diff --git a/include/linux/pci.h b/include/linux/pci.h
index e2d1a124216a..9b234cbc7ae1 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -612,6 +612,8 @@ struct pci_ops {
void __iomem *(*map_bus)(struct pci_bus *bus, unsigned int devfn, int where);
int (*read)(struct pci_bus *bus, unsigned int devfn, int where, int size, u32 *val);
int (*write)(struct pci_bus *bus, unsigned int devfn, int where, int size, u32 val);
+ /* Set to true when pci_lock is not needed for read/write */
+ bool ll_allowed;
};
/*
--
2.9.3
next reply other threads:[~2017-03-03 2:09 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-02 23:21 Andi Kleen [this message]
2017-03-02 23:21 ` [PATCH 2/4] pci: Add generic pci_bus_force_mmconfig interface Andi Kleen
2017-03-14 17:34 ` H. Peter Anvin
2017-03-02 23:21 ` [PATCH 3/4] x86, pci: Add interface to force mmconfig Andi Kleen
2017-03-14 13:55 ` Thomas Gleixner
2017-03-14 15:41 ` Andi Kleen
2017-03-14 16:40 ` Thomas Gleixner
2017-03-14 17:02 ` Andi Kleen
2017-03-14 17:56 ` Thomas Gleixner
2017-03-14 19:47 ` Bjorn Helgaas
2017-03-15 2:24 ` Andi Kleen
2017-03-15 2:55 ` Bjorn Helgaas
2017-03-15 10:00 ` Thomas Gleixner
2017-03-15 14:09 ` Bjorn Helgaas
2017-03-16 0:02 ` Andi Kleen
2017-03-16 22:45 ` Thomas Gleixner
2017-03-02 23:21 ` [PATCH 4/4] perf/x86/intel/uncore: Enable forced mmconfig for Intel uncore Andi Kleen
2017-03-14 13:06 ` [PATCH 1/4] pci: Allow lockless access path to PCI mmconfig Thomas Gleixner
2017-03-14 17:28 ` H. Peter Anvin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170302232104.10136-1-andi@firstfloor.org \
--to=andi@firstfloor.org \
--cc=ak@linux.intel.com \
--cc=bhelgaas@google.com \
--cc=eranian@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).