* Fixes for 64bit window handling on AMD CPUs
@ 2017-11-29 14:12 Christian König
2017-11-29 14:12 ` [PATCH 1/3] x86/PCI: fix infinity loop in search for 64bit BAR placement Christian König
` (3 more replies)
0 siblings, 4 replies; 9+ messages in thread
From: Christian König @ 2017-11-29 14:12 UTC (permalink / raw)
To: helgaas, linux-pci, linux-kernel
Hi Bjorn,
please include the following three patches in your next fix pull request for -rc2.
They just restrict the fixup of the 64bit window on AMD CPU to avoid problems with Xen and multi socket systems.
If you prefer a pull request just say so and I will setup a branch on a public server.
Thanks,
Christian.
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH 1/3] x86/PCI: fix infinity loop in search for 64bit BAR placement 2017-11-29 14:12 Fixes for 64bit window handling on AMD CPUs Christian König @ 2017-11-29 14:12 ` Christian König 2017-11-29 14:12 ` [PATCH 2/3] x86/PCI: only enable a 64bit BAR on single socket AMD Family 15h systems Christian König ` (2 subsequent siblings) 3 siblings, 0 replies; 9+ messages in thread From: Christian König @ 2017-11-29 14:12 UTC (permalink / raw) To: helgaas, linux-pci, linux-kernel Break the loop if we can't find some address space for a 64bit BAR. Signed-off-by: Christian König <christian.koenig@amd.com> --- arch/x86/pci/fixup.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c index e59378bf37d9..e857b3ac5755 100644 --- a/arch/x86/pci/fixup.c +++ b/arch/x86/pci/fixup.c @@ -695,8 +695,13 @@ static void pci_amd_enable_64bit_bar(struct pci_dev *dev) res->end = 0xfd00000000ull - 1; /* Just grab the free area behind system memory for this */ - while ((conflict = request_resource_conflict(&iomem_resource, res))) + while ((conflict = request_resource_conflict(&iomem_resource, res))) { + if (conflict->end >= res->end) { + kfree(res); + return; + } res->start = conflict->end + 1; + } dev_info(&dev->dev, "adding root bus resource %pR\n", res); -- 2.11.0 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 2/3] x86/PCI: only enable a 64bit BAR on single socket AMD Family 15h systems 2017-11-29 14:12 Fixes for 64bit window handling on AMD CPUs Christian König 2017-11-29 14:12 ` [PATCH 1/3] x86/PCI: fix infinity loop in search for 64bit BAR placement Christian König @ 2017-11-29 14:12 ` Christian König 2017-11-29 14:12 ` [PATCH 3/3] x86/PCI: limit the size of the 64bit BAR to 256GB Christian König 2017-12-06 20:51 ` Fixes for 64bit window handling on AMD CPUs Bjorn Helgaas 3 siblings, 0 replies; 9+ messages in thread From: Christian König @ 2017-11-29 14:12 UTC (permalink / raw) To: helgaas, linux-pci, linux-kernel When we have a multi socket system each CPU core needs the same setup. Since this is tricky to do in the fixup code disable enabling a 64bit BAR on multi socket systems for now. Signed-off-by: Christian König <christian.koenig@amd.com> --- arch/x86/pci/fixup.c | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c index e857b3ac5755..c817ab85dc82 100644 --- a/arch/x86/pci/fixup.c +++ b/arch/x86/pci/fixup.c @@ -664,6 +664,16 @@ static void pci_amd_enable_64bit_bar(struct pci_dev *dev) unsigned i; u32 base, limit, high; struct resource *res, *conflict; + struct pci_dev *other; + + /* Check that we are the only device of that type */ + other = pci_get_device(dev->vendor, dev->device, NULL); + if (other != dev || + (other = pci_get_device(dev->vendor, dev->device, other))) { + /* This is a multi socket system, don't touch it for now */ + pci_dev_put(other); + return; + } for (i = 0; i < 8; i++) { pci_read_config_dword(dev, AMD_141b_MMIO_BASE(i), &base); @@ -718,10 +728,10 @@ static void pci_amd_enable_64bit_bar(struct pci_dev *dev) pci_bus_add_resource(dev->bus, res, 0); } -DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x1401, pci_amd_enable_64bit_bar); -DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x141b, pci_amd_enable_64bit_bar); -DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x1571, pci_amd_enable_64bit_bar); -DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x15b1, pci_amd_enable_64bit_bar); -DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x1601, pci_amd_enable_64bit_bar); +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD, 0x1401, pci_amd_enable_64bit_bar); +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD, 0x141b, pci_amd_enable_64bit_bar); +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD, 0x1571, pci_amd_enable_64bit_bar); +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD, 0x15b1, pci_amd_enable_64bit_bar); +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD, 0x1601, pci_amd_enable_64bit_bar); #endif -- 2.11.0 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 3/3] x86/PCI: limit the size of the 64bit BAR to 256GB 2017-11-29 14:12 Fixes for 64bit window handling on AMD CPUs Christian König 2017-11-29 14:12 ` [PATCH 1/3] x86/PCI: fix infinity loop in search for 64bit BAR placement Christian König 2017-11-29 14:12 ` [PATCH 2/3] x86/PCI: only enable a 64bit BAR on single socket AMD Family 15h systems Christian König @ 2017-11-29 14:12 ` Christian König 2017-12-06 19:51 ` Bjorn Helgaas 2017-12-06 20:51 ` Fixes for 64bit window handling on AMD CPUs Bjorn Helgaas 3 siblings, 1 reply; 9+ messages in thread From: Christian König @ 2017-11-29 14:12 UTC (permalink / raw) To: helgaas, linux-pci, linux-kernel This avoids problems with Xen which hides some memory resources from the OS and potentially also allows memory hotplug while this fixup is enabled. Signed-off-by: Christian König <christian.koenig@amd.com> --- arch/x86/pci/fixup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c index c817ab85dc82..149adbc7f2a3 100644 --- a/arch/x86/pci/fixup.c +++ b/arch/x86/pci/fixup.c @@ -701,7 +701,7 @@ static void pci_amd_enable_64bit_bar(struct pci_dev *dev) res->name = "PCI Bus 0000:00"; res->flags = IORESOURCE_PREFETCH | IORESOURCE_MEM | IORESOURCE_MEM_64 | IORESOURCE_WINDOW; - res->start = 0x100000000ull; + res->start = 0xbd00000000ull; res->end = 0xfd00000000ull - 1; /* Just grab the free area behind system memory for this */ -- 2.11.0 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 3/3] x86/PCI: limit the size of the 64bit BAR to 256GB 2017-11-29 14:12 ` [PATCH 3/3] x86/PCI: limit the size of the 64bit BAR to 256GB Christian König @ 2017-12-06 19:51 ` Bjorn Helgaas 2017-12-08 17:56 ` Bjorn Helgaas 0 siblings, 1 reply; 9+ messages in thread From: Bjorn Helgaas @ 2017-12-06 19:51 UTC (permalink / raw) To: Christian König; +Cc: linux-pci, linux-kernel On Wed, Nov 29, 2017 at 03:12:29PM +0100, Christian König wrote: > This avoids problems with Xen which hides some memory resources from the > OS and potentially also allows memory hotplug while this fixup is > enabled. The patch itself is OK, but the changelog doesn't say enough about what the problem is. I have no clue about what the Xen issue is or why limiting the BAR to 256GB avoids the problem or what this has to do with memory hotplug. For example, we should be able to tell why 256GB is the right number. Maybe there's something specific in Xen you can reference? Maybe an example of what goes wrong with some details? > Signed-off-by: Christian König <christian.koenig@amd.com> > --- > arch/x86/pci/fixup.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c > index c817ab85dc82..149adbc7f2a3 100644 > --- a/arch/x86/pci/fixup.c > +++ b/arch/x86/pci/fixup.c > @@ -701,7 +701,7 @@ static void pci_amd_enable_64bit_bar(struct pci_dev *dev) > res->name = "PCI Bus 0000:00"; > res->flags = IORESOURCE_PREFETCH | IORESOURCE_MEM | > IORESOURCE_MEM_64 | IORESOURCE_WINDOW; > - res->start = 0x100000000ull; > + res->start = 0xbd00000000ull; > res->end = 0xfd00000000ull - 1; > > /* Just grab the free area behind system memory for this */ > -- > 2.11.0 > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 3/3] x86/PCI: limit the size of the 64bit BAR to 256GB 2017-12-06 19:51 ` Bjorn Helgaas @ 2017-12-08 17:56 ` Bjorn Helgaas 2017-12-08 21:30 ` Boris Ostrovsky 2017-12-10 10:03 ` Christian König 0 siblings, 2 replies; 9+ messages in thread From: Bjorn Helgaas @ 2017-12-08 17:56 UTC (permalink / raw) To: Christian König; +Cc: linux-pci, linux-kernel On Wed, Dec 06, 2017 at 01:51:18PM -0600, Bjorn Helgaas wrote: > On Wed, Nov 29, 2017 at 03:12:29PM +0100, Christian König wrote: > > This avoids problems with Xen which hides some memory resources from the > > OS and potentially also allows memory hotplug while this fixup is > > enabled. > > The patch itself is OK, but the changelog doesn't say enough about > what the problem is. I have no clue about what the Xen issue is or > why limiting the BAR to 256GB avoids the problem or what this has to > do with memory hotplug. > > For example, we should be able to tell why 256GB is the right number. > Maybe there's something specific in Xen you can reference? Maybe an > example of what goes wrong with some details? Ping? Is this change required to fix issues people are seeing? If so, we either need to rework the changelog and get it merged, or revert the quirk as a whole. I tentatively applied the first two patches to for-linus, but I haven't asked Linus to pull them because I assumed we really needed all three. Bjorn > > Signed-off-by: Christian König <christian.koenig@amd.com> > > --- > > arch/x86/pci/fixup.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c > > index c817ab85dc82..149adbc7f2a3 100644 > > --- a/arch/x86/pci/fixup.c > > +++ b/arch/x86/pci/fixup.c > > @@ -701,7 +701,7 @@ static void pci_amd_enable_64bit_bar(struct pci_dev *dev) > > res->name = "PCI Bus 0000:00"; > > res->flags = IORESOURCE_PREFETCH | IORESOURCE_MEM | > > IORESOURCE_MEM_64 | IORESOURCE_WINDOW; > > - res->start = 0x100000000ull; > > + res->start = 0xbd00000000ull; > > res->end = 0xfd00000000ull - 1; > > > > /* Just grab the free area behind system memory for this */ > > -- > > 2.11.0 > > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 3/3] x86/PCI: limit the size of the 64bit BAR to 256GB 2017-12-08 17:56 ` Bjorn Helgaas @ 2017-12-08 21:30 ` Boris Ostrovsky 2017-12-10 10:03 ` Christian König 1 sibling, 0 replies; 9+ messages in thread From: Boris Ostrovsky @ 2017-12-08 21:30 UTC (permalink / raw) To: Bjorn Helgaas, Christian König Cc: linux-pci, linux-kernel, xen-devel, Juergen Gross On 12/08/2017 12:56 PM, Bjorn Helgaas wrote: > On Wed, Dec 06, 2017 at 01:51:18PM -0600, Bjorn Helgaas wrote: >> On Wed, Nov 29, 2017 at 03:12:29PM +0100, Christian König wrote: >>> This avoids problems with Xen which hides some memory resources from the >>> OS and potentially also allows memory hotplug while this fixup is >>> enabled. >> The patch itself is OK, but the changelog doesn't say enough about >> what the problem is. I have no clue about what the Xen issue is or >> why limiting the BAR to 256GB avoids the problem or what this has to >> do with memory hotplug. >> >> For example, we should be able to tell why 256GB is the right number. >> Maybe there's something specific in Xen you can reference? Maybe an >> example of what goes wrong with some details? > Ping? Is this change required to fix issues people are seeing? If > so, we either need to rework the changelog and get it merged, or > revert the quirk as a whole. > > I tentatively applied the first two patches to for-linus, but I > haven't asked Linus to pull them because I assumed we really needed > all three. This is not a fix but rather is a workaround. The problem is that Xen dom0 may be running with less than all of the system memory and the chunk of host memory that dom0 doesn't have is not exposed in e820 as reserved. And so pci_amd_enable_64bit_bar() assumes that it can be used for MMIO, with predictable results. Only trying to use very high addresses limits chances that there is memory there. The alternative is to revert f5775e0b6116b7e2425ccf535243b21768566d87. I have been working on a proper fix but haven't been able to finish it yet. -boris > > Bjorn > >>> Signed-off-by: Christian König <christian.koenig@amd.com> >>> --- >>> arch/x86/pci/fixup.c | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c >>> index c817ab85dc82..149adbc7f2a3 100644 >>> --- a/arch/x86/pci/fixup.c >>> +++ b/arch/x86/pci/fixup.c >>> @@ -701,7 +701,7 @@ static void pci_amd_enable_64bit_bar(struct pci_dev *dev) >>> res->name = "PCI Bus 0000:00"; >>> res->flags = IORESOURCE_PREFETCH | IORESOURCE_MEM | >>> IORESOURCE_MEM_64 | IORESOURCE_WINDOW; >>> - res->start = 0x100000000ull; >>> + res->start = 0xbd00000000ull; >>> res->end = 0xfd00000000ull - 1; >>> >>> /* Just grab the free area behind system memory for this */ >>> -- >>> 2.11.0 >>> ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 3/3] x86/PCI: limit the size of the 64bit BAR to 256GB 2017-12-08 17:56 ` Bjorn Helgaas 2017-12-08 21:30 ` Boris Ostrovsky @ 2017-12-10 10:03 ` Christian König 1 sibling, 0 replies; 9+ messages in thread From: Christian König @ 2017-12-10 10:03 UTC (permalink / raw) To: Bjorn Helgaas; +Cc: linux-pci, linux-kernel Am 08.12.2017 um 18:56 schrieb Bjorn Helgaas: > On Wed, Dec 06, 2017 at 01:51:18PM -0600, Bjorn Helgaas wrote: >> On Wed, Nov 29, 2017 at 03:12:29PM +0100, Christian König wrote: >>> This avoids problems with Xen which hides some memory resources from the >>> OS and potentially also allows memory hotplug while this fixup is >>> enabled. >> The patch itself is OK, but the changelog doesn't say enough about >> what the problem is. I have no clue about what the Xen issue is or >> why limiting the BAR to 256GB avoids the problem or what this has to >> do with memory hotplug. >> >> For example, we should be able to tell why 256GB is the right number. >> Maybe there's something specific in Xen you can reference? Maybe an >> example of what goes wrong with some details? > Ping? Sorry for the delay, first been to busy and then got a bad cold and today is the first day I've got out of bed again. > Is this change required to fix issues people are seeing? If > so, we either need to rework the changelog and get it merged, or > revert the quirk as a whole. It's just a precaution to eventually avoid problems, I will try to provided a patch with updated commit log tomorrow. > I tentatively applied the first two patches to for-linus, but I > haven't asked Linus to pull them because I assumed we really needed > all three. The first two patches are perfectly enough for now. Thanks, Christian. > > Bjorn > >>> Signed-off-by: Christian König <christian.koenig@amd.com> >>> --- >>> arch/x86/pci/fixup.c | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c >>> index c817ab85dc82..149adbc7f2a3 100644 >>> --- a/arch/x86/pci/fixup.c >>> +++ b/arch/x86/pci/fixup.c >>> @@ -701,7 +701,7 @@ static void pci_amd_enable_64bit_bar(struct pci_dev *dev) >>> res->name = "PCI Bus 0000:00"; >>> res->flags = IORESOURCE_PREFETCH | IORESOURCE_MEM | >>> IORESOURCE_MEM_64 | IORESOURCE_WINDOW; >>> - res->start = 0x100000000ull; >>> + res->start = 0xbd00000000ull; >>> res->end = 0xfd00000000ull - 1; >>> >>> /* Just grab the free area behind system memory for this */ >>> -- >>> 2.11.0 >>> ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Fixes for 64bit window handling on AMD CPUs 2017-11-29 14:12 Fixes for 64bit window handling on AMD CPUs Christian König ` (2 preceding siblings ...) 2017-11-29 14:12 ` [PATCH 3/3] x86/PCI: limit the size of the 64bit BAR to 256GB Christian König @ 2017-12-06 20:51 ` Bjorn Helgaas 3 siblings, 0 replies; 9+ messages in thread From: Bjorn Helgaas @ 2017-12-06 20:51 UTC (permalink / raw) To: Christian König; +Cc: linux-pci, linux-kernel On Wed, Nov 29, 2017 at 03:12:26PM +0100, Christian König wrote: > Hi Bjorn, > > please include the following three patches in your next fix pull request for -rc2. > > They just restrict the fixup of the 64bit window on AMD CPU to avoid problems with Xen and multi socket systems. Applied the first two to for-linus for v4.15, thanks! I'll apply the third as soon as we get the changelog clarified. ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2017-12-10 10:03 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-11-29 14:12 Fixes for 64bit window handling on AMD CPUs Christian König 2017-11-29 14:12 ` [PATCH 1/3] x86/PCI: fix infinity loop in search for 64bit BAR placement Christian König 2017-11-29 14:12 ` [PATCH 2/3] x86/PCI: only enable a 64bit BAR on single socket AMD Family 15h systems Christian König 2017-11-29 14:12 ` [PATCH 3/3] x86/PCI: limit the size of the 64bit BAR to 256GB Christian König 2017-12-06 19:51 ` Bjorn Helgaas 2017-12-08 17:56 ` Bjorn Helgaas 2017-12-08 21:30 ` Boris Ostrovsky 2017-12-10 10:03 ` Christian König 2017-12-06 20:51 ` Fixes for 64bit window handling on AMD CPUs Bjorn Helgaas
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).