From: Sergey Miroshnichenko <s.miroshnichenko@yadro.com>
To: <linux-pci@vger.kernel.org>, <linuxppc-dev@lists.ozlabs.org>
Cc: Sergey Miroshnichenko <s.miroshnichenko@yadro.com>,
Bjorn Helgaas <helgaas@kernel.org>,
linux@yadro.com
Subject: [PATCH v5 11/23] PCI: hotplug: movable BARs: Calculate immovable parts of bridge windows
Date: Fri, 16 Aug 2019 19:50:49 +0300 [thread overview]
Message-ID: <20190816165101.911-12-s.miroshnichenko@yadro.com> (raw)
In-Reply-To: <20190816165101.911-1-s.miroshnichenko@yadro.com>
When movable BARs are enabled, and if a bridge contains a device with fixed
(IORESOURCE_PCI_FIXED) or immovable BARs, the corresponing windows can't be
moved too far away from their original positions - they must still contain
all the fixed/immovable BARs, like that:
1) Window position before a bus rescan:
| <-- root bridge window --> |
| |
| | <-- bridge window --> | |
| | movable BARs | **fixed BAR** | |
2) Possible valid outcome after rescan and move:
| <-- root bridge window --> |
| |
| | <-- bridge window --> | |
| | **fixed BAR** | Movable BARs | |
An immovable area of a bridge (separare for IO, MEM and MEM64 window types)
is a range that covers all the fixed and immovable BARs of direct children,
and all the fixed area of children bridges:
| <-- root bridge window --> |
| |
| | <-- bridge window level 1 --> | |
| | ******** immovable area of this bridge window ******** | |
| | | |
| | **fixed BAR** | <-- bridge window level 2 --> | BARs | |
| | | ***** fixed area of this bridge ***** | | |
| | | | | |
| | | ***fixed BAR*** | | ***fixed BAR*** | | |
To store these areas, the .immovable_range field has been added to struct
pci_bus. It is filled recursively from leaves to the root before a rescan.
Also make pbus_size_io() and pbus_size_mem() return their usual result OR
the size of an immovable range of according type, depending on which one is
larger.
Signed-off-by: Sergey Miroshnichenko <s.miroshnichenko@yadro.com>
---
drivers/pci/pci.h | 14 +++++++
drivers/pci/probe.c | 88 +++++++++++++++++++++++++++++++++++++++++
drivers/pci/setup-bus.c | 17 ++++++++
include/linux/pci.h | 6 +++
4 files changed, 125 insertions(+)
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 53249cbc21b6..12add575faf1 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -371,6 +371,20 @@ static inline bool pci_dev_is_disconnected(const struct pci_dev *dev)
return dev->error_state == pci_channel_io_perm_failure;
}
+static inline int pci_get_bridge_resource_idx(struct resource *r)
+{
+ int idx = 1;
+
+ if (r->flags & IORESOURCE_IO)
+ idx = 0;
+ else if (!(r->flags & IORESOURCE_PREFETCH))
+ idx = 1;
+ else if (r->flags & IORESOURCE_MEM_64)
+ idx = 2;
+
+ return idx;
+}
+
/* pci_dev priv_flags */
#define PCI_DEV_ADDED 0
#define PCI_DEV_DISABLED_BARS 1
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index bf0a7d1c5d09..5f52a19738aa 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -550,6 +550,7 @@ void pci_read_bridge_bases(struct pci_bus *child)
static struct pci_bus *pci_alloc_bus(struct pci_bus *parent)
{
struct pci_bus *b;
+ int idx;
b = kzalloc(sizeof(*b), GFP_KERNEL);
if (!b)
@@ -566,6 +567,11 @@ static struct pci_bus *pci_alloc_bus(struct pci_bus *parent)
if (parent)
b->domain_nr = parent->domain_nr;
#endif
+ for (idx = 0; idx < PCI_BRIDGE_RESOURCE_NUM; ++idx) {
+ b->immovable_range[idx].start = 0;
+ b->immovable_range[idx].end = 0;
+ }
+
return b;
}
@@ -3512,6 +3518,87 @@ static void pci_setup_bridges(struct pci_bus *bus)
pci_setup_bridge(bus);
}
+static void pci_bus_update_immovable_range(struct pci_bus *bus)
+{
+ struct pci_dev *dev;
+ int idx;
+ resource_size_t start, end;
+
+ for (idx = 0; idx < PCI_BRIDGE_RESOURCE_NUM; ++idx) {
+ bus->immovable_range[idx].start = 0;
+ bus->immovable_range[idx].end = 0;
+ }
+
+ list_for_each_entry(dev, &bus->devices, bus_list)
+ if (dev->subordinate)
+ pci_bus_update_immovable_range(dev->subordinate);
+
+ list_for_each_entry(dev, &bus->devices, bus_list) {
+ int i;
+ bool dev_is_movable = pci_dev_movable_bars_supported(dev);
+ struct pci_bus *child = dev->subordinate;
+
+ for (i = 0; i < PCI_BRIDGE_RESOURCES; ++i) {
+ struct resource *r = &dev->resource[i];
+
+ if (!r->flags || (r->flags & IORESOURCE_UNSET) || !r->parent)
+ continue;
+
+ if (!dev_is_movable || (r->flags & IORESOURCE_PCI_FIXED)) {
+ idx = pci_get_bridge_resource_idx(r);
+ start = bus->immovable_range[idx].start;
+ end = bus->immovable_range[idx].end;
+
+ if (!start || start > r->start)
+ start = r->start;
+ if (end < r->end)
+ end = r->end;
+
+ if (bus->immovable_range[idx].start != start ||
+ bus->immovable_range[idx].end != end) {
+ dev_dbg(&bus->dev, "Found fixed 0x%llx-0x%llx in %s, expand the fixed bridge window %d to 0x%llx-0x%llx\n",
+ (unsigned long long)r->start,
+ (unsigned long long)r->end,
+ dev_name(&dev->dev), idx,
+ (unsigned long long)start,
+ (unsigned long long)end);
+ bus->immovable_range[idx].start = start;
+ bus->immovable_range[idx].end = end;
+ }
+ }
+ }
+
+ if (child) {
+ for (idx = 0; idx < PCI_BRIDGE_RESOURCE_NUM; ++idx) {
+ struct resource *child_immovable_range =
+ &child->immovable_range[idx];
+
+ if (child_immovable_range->start >=
+ child_immovable_range->end)
+ continue;
+
+ start = bus->immovable_range[idx].start;
+ end = bus->immovable_range[idx].end;
+
+ if (!start || start > child_immovable_range->start)
+ start = child_immovable_range->start;
+ if (end < child_immovable_range->end)
+ end = child_immovable_range->end;
+
+ if (start < bus->immovable_range[idx].start ||
+ end > bus->immovable_range[idx].end) {
+ dev_dbg(&bus->dev, "Expand the fixed bridge window %d from %s to 0x%llx-0x%llx\n",
+ idx, dev_name(&child->dev),
+ (unsigned long long)start,
+ (unsigned long long)end);
+ bus->immovable_range[idx].start = start;
+ bus->immovable_range[idx].end = end;
+ }
+ }
+ }
+ }
+}
+
static struct pci_dev *pci_find_next_new_device(struct pci_bus *bus)
{
struct pci_dev *dev;
@@ -3610,6 +3697,7 @@ unsigned int pci_rescan_bus(struct pci_bus *bus)
pci_bus_rescan_prepare(root);
max = pci_scan_child_bus(root);
+ pci_bus_update_immovable_range(root);
pci_reassign_root_bus_resources(root);
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 33f709095675..420510a1a257 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -882,9 +882,17 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
resource_size_t children_add_size = 0;
resource_size_t min_align, align;
+ resource_size_t fixed_start = bus->immovable_range[0].start;
+ resource_size_t fixed_end = bus->immovable_range[0].end;
+ resource_size_t fixed_size = (fixed_start < fixed_end) ?
+ (fixed_end - fixed_start + 1) : 0;
+
if (!b_res)
return;
+ if (min_size < fixed_size)
+ min_size = fixed_size;
+
min_align = window_alignment(bus, IORESOURCE_IO);
list_for_each_entry(dev, &bus->devices, bus_list) {
int i;
@@ -993,6 +1001,15 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
resource_size_t children_add_size = 0;
resource_size_t children_add_align = 0;
resource_size_t add_align = 0;
+ bool is_mem64 = (mask & IORESOURCE_MEM_64);
+
+ resource_size_t fixed_start = bus->immovable_range[is_mem64 ? 2 : 1].start;
+ resource_size_t fixed_end = bus->immovable_range[is_mem64 ? 2 : 1].end;
+ resource_size_t fixed_size = (fixed_start < fixed_end) ?
+ (fixed_end - fixed_start + 1) : 0;
+
+ if (min_size < fixed_size)
+ min_size = fixed_size;
if (!b_res)
return -ENOSPC;
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 95a8113c2157..efafbf816fe6 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -581,6 +581,12 @@ struct pci_bus {
struct list_head resources; /* Address space routed to this bus */
struct resource busn_res; /* Bus numbers routed to this bus */
+ /*
+ * If there are fixed or immovable resources in the bridge window, this range
+ * contains the lowest start address and highest end address of them.
+ */
+ struct resource immovable_range[PCI_BRIDGE_RESOURCE_NUM];
+
struct pci_ops *ops; /* Configuration access functions */
struct msi_controller *msi; /* MSI controller */
void *sysdata; /* Hook for sys-specific extension */
--
2.21.0
next prev parent reply other threads:[~2019-08-16 17:18 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-16 16:50 [PATCH v5 00/23] PCI: Allow BAR movement during hotplug Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 01/23] PCI: Fix race condition in pci_enable/disable_device() Sergey Miroshnichenko
2019-08-22 12:37 ` Marta Rybczynska
2019-09-27 21:59 ` Bjorn Helgaas
2019-09-30 8:53 ` Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 02/23] PCI: Enable bridge's I/O and MEM access for hotplugged devices Sergey Miroshnichenko
2019-09-27 22:01 ` Bjorn Helgaas
2019-08-16 16:50 ` [PATCH v5 03/23] PCI: hotplug: Add a flag for the movable BARs feature Sergey Miroshnichenko
2019-09-27 22:02 ` Bjorn Helgaas
2019-09-30 8:44 ` David Laight
2019-09-30 16:17 ` Sergey Miroshnichenko
2019-09-30 12:59 ` Sergey Miroshnichenko
2019-10-15 22:14 ` Bjorn Helgaas
2019-10-16 15:50 ` Sergey Miroshnichenko
2019-10-16 17:29 ` Bjorn Helgaas
2019-08-16 16:50 ` [PATCH v5 04/23] PCI: Define PCI-specific version of the release_child_resources() Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 05/23] PCI: hotplug: movable BARs: Fix reassigning the released bridge windows Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 06/23] PCI: hotplug: movable BARs: Recalculate all bridge windows during rescan Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 07/23] PCI: hotplug: movable BARs: Don't allow added devices to steal resources Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 08/23] PCI: Include fixed and immovable BARs into the bus size calculating Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 09/23] PCI: Prohibit assigning BARs and bridge windows to non-direct parents Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 10/23] PCI: hotplug: movable BARs: Try to assign unassigned resources only once Sergey Miroshnichenko
2019-08-16 16:50 ` Sergey Miroshnichenko [this message]
2019-08-16 16:50 ` [PATCH v5 12/23] PCI: hotplug: movable BARs: Compute limits for relocated bridge windows Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 13/23] PCI: Make sure bridge windows include their fixed BARs Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 14/23] PCI: Fix assigning the fixed prefetchable resources Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 15/23] PCI: hotplug: movable BARs: Assign fixed and immovable BARs before others Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 16/23] PCI: hotplug: movable BARs: Don't reserve IO/mem bus space Sergey Miroshnichenko
2019-09-04 5:42 ` Oliver O'Halloran
2019-09-04 11:22 ` Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 17/23] powerpc/pci: Fix crash with enabled movable BARs Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 18/23] powerpc/pci: Handle BAR movement Sergey Miroshnichenko
2019-09-04 5:37 ` Oliver O'Halloran
2019-09-06 16:24 ` Sergey Miroshnichenko
2019-09-09 14:02 ` Oliver O'Halloran
2019-08-16 16:50 ` [PATCH v5 19/23] PCI: hotplug: Configure MPS for hot-added bridges during bus rescan Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 20/23] PCI: hotplug: movable BARs: Enable the feature by default Sergey Miroshnichenko
2019-08-16 16:50 ` [PATCH v5 21/23] nvme-pci: Handle movable BARs Sergey Miroshnichenko
2019-08-16 16:51 ` [PATCH v5 22/23] PCI/portdrv: Declare support of " Sergey Miroshnichenko
2019-08-16 16:51 ` [PATCH v5 23/23] PCI: pciehp: movable BARs: Trigger a domain rescan on hp events Sergey Miroshnichenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190816165101.911-12-s.miroshnichenko@yadro.com \
--to=s.miroshnichenko@yadro.com \
--cc=helgaas@kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux@yadro.com \
--cc=linuxppc-dev@lists.ozlabs.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).