[[RFC PATCH v1] 0/1] Add pci=nobbn to ignore ACPI _BBN method to override host bridge bus window

[[RFC PATCH v1] 0/1] Add pci=nobbn to ignore ACPI _BBN method to override host bridge bus window

[Nicholas Johnson]

[Re-posting as the first time I sent cover letter twice instead of cover 
letter and patch, oops]

Hi all,

I want to be able to override the bus resource from ACPI, but nocrs does 
not do it. I am putting this out here to get a feel for the sentiment 
for doing something like this.

What is my motivation for doing this?

I have a Gigabyte Z170X Designare motherboard which only gives resource 
[bus 00-7e]. I want the full [bus 00-ff] because I am trying to add as 
many Thunderbolt 3 ports with add-in cards as possible. Thunderbolt 
consumes bus numbers quickly. An Intel Ice Lake implementation (ideal) 
consumes 42 busses per port, but prior solutions consume 50 busses per 
port and have additional busses required for the NHI and USB 
controllers, as well as the bridges from the root port.

Why not change nocrs to do this? Why the new kernel parameter?

I imagine that on systems with multiple PCI root complexes, things will 
get hairy if we do this, if they are not placed on separate segments / 
domains by the firmware. I do not own such a beast, but from what I 
understand, the firmware normally places them on the same segment / 
domain with non-overlapping bus numbers. But we may still want to use 
nocrs for other reasons. I need to use nocrs to allow Linux to allocate 
vast amounts of MMIO and MMIO_PREF under the Thunderbolt root ports 
without the BIOS support for Thunderbolt. Hence, they should be kept 
separate.

Why do this in general?

The bus resource is still a resource which is specified from ACPI, just 
like those overridden by nocrs. Even if we do not use pci=nocrs to 
override it, it should be possible to override it, just as it is 
possible to override _CRS.

Topics for discussion include, but are not limited to:

	Is my code great, good, bad, ugly, or does it need work, etc?

	Which way to skin the cat / achieve the end goal, in terms of 
	code?

	Should this be done?

	Is pci=nobbn a good name for the new parameter?

	Should the documentation notice say to report a bug if you use 
	this, like the nocrs one does?

	What are the potential risks and fallout if it is done, if any? 
	My stance on this is that it will be limited to people who use 
	the parameter, so it should be safe.

	What would it take to convince you to support this?

	In relation to arch/x86/pci/acpi.c in the function 
	pci_acpi_root_prepare_resources(): When using CRS, we remove the 
	resource that satisfies resource_is_pcicfg_ioport() without the 
	dev_printk(). But when doing nocrs, we remove it along with all 
	of the others and do the dev_printk() notice. Should it be 
	changed in another patch to skip printing the notice for this 
	resource when using nocrs?

Lastly (semi-unrelated), you may notice that the email linkage is 
broken. It is because I made the mistake of using an @outlook email 
address. It interferes with the encoding. If I send patches with git 
send-email, the encoding is broken. If I use mutt -H, the encoding 
works, but the linkage is broken. I have heard that Gmail also has 
problems, but I tested it the other day with an old Gmail address I 
have, and it appears to work. But I am open for suggestions on what my 
new email domain for kernel development should be. I hope I can use a 
consumer email provider, and not mess with hosting my own domain (it 
sounds like a lot of work). I will switch over soon and send an email 
from this account to confirm that the new account is actually me.

Thanks for any comments or insights.

Kind regards,

Nicholas Johnson

Nicholas Johnson (1):
  PCI: Add pci=nobbn to ignore ACPI _BBN method to override host bridge
    bus window

 Documentation/admin-guide/kernel-parameters.txt |  2 ++
 arch/x86/include/asm/pci_x86.h                  |  1 +
 arch/x86/pci/acpi.c                             | 11 +++++++++++
 arch/x86/pci/common.c                           |  3 +++
 4 files changed, 17 insertions(+)

-- 
2.24.0

Re: [[RFC PATCH v1] 0/1] Add pci=nobbn to ignore ACPI _BBN method to override host bridge bus window

[Nicholas Johnson]

On Mon, Dec 09, 2019 at 10:10:55AM -0600, Bjorn Helgaas wrote:
> On Thu, Nov 21, 2019 at 9:02 PM Nicholas Johnson
> <nicholas.johnson-opensource@outlook.com.au> wrote:
> >
> > Hi all,
> >
> > I want to be able to override the bus resource from ACPI, but nocrs does
> > not do it. I am putting this out here to get a feel for the sentiment
> > for doing something like this.
> 
> This should be cc'd to linus-pci and linux-acpi (added).  I only
> noticed this message by accident.  And I don't see the patch at all.
I thought I had you cc'd into this - you were in the get_maintainer.pl 
output.

Looks like I posted the cover letter twice instead of the patch. I will 
re-post shortly.

> 
> > What is my motivation for doing this?
> >
> > I have a Gigabyte Z170X Designare motherboard which only gives resource
> > [bus 00-7e]. I want the full [bus 00-ff] because I am trying to add as
> > many Thunderbolt 3 ports with add-in cards as possible. Thunderbolt
> > consumes bus numbers quickly. An Intel Ice Lake implementation (ideal)
> > consumes 42 busses per port, but prior solutions consume 50 busses per
> > port and have additional busses required for the NHI and USB
> > controllers, as well as the bridges from the root port.
> >
> > Why not change nocrs to do this? Why the new kernel parameter?
> >
> > I imagine that on systems with multiple PCI root complexes, things will
> > get hairy if we do this, if they are not placed on separate segments /
> > domains by the firmware. I do not own such a beast, but from what I
> > understand, the firmware normally places them on the same segment /
> > domain with non-overlapping bus numbers. But we may still want to use
> > nocrs for other reasons. I need to use nocrs to allow Linux to allocate
> > vast amounts of MMIO and MMIO_PREF under the Thunderbolt root ports
> > without the BIOS support for Thunderbolt. Hence, they should be kept
> > separate.
> >
> > Why do this in general?
> >
> > The bus resource is still a resource which is specified from ACPI, just
> > like those overridden by nocrs. Even if we do not use pci=nocrs to
> > override it, it should be possible to override it, just as it is
> > possible to override _CRS.
> 
> pci=nocrs is for working around defects in firmware or Linux.  The
> firmware knows more about the platform than Linux, and in general we
> have to trust it.  We probably should taint the kernel when we use it.
> 
> Any parameter like this should work the same on all ACPI systems,
> including ia64 and arm64, and should probably also taint the kernel.
> 
> I can't see the patch itself, but I'm a little confused because we
> normally get the bus number range from _CRS in acpi_pci_root_add() and
> your patch doesn't appear to touch that.
Because pci=nocrs is x86-only, I was making pci=nobbn x86-only. Which 
means I only touched the areas in arch/x86 where pci=nocrs is handled.

If you want, I can change nocrs to work on any arch, although I will 
need somebody who owns ACPI systems of different architectures to test.

The only such system I know of (that is not server hardware) is the 
Microsoft Surface Pro X, which is an ARM64 system with ACPI and at least 
three PCIe root complexes on PCI domains/segments 0, 2, 3. Mainline 
Linux does not run properly even on X86 Surface devices, so I imagine it 
will be an absolute nightmare getting Linux to run on this. I also have 
no intention of buying anything without Thunderbolt / USB4.

Kind regards,
Nicholas

> 
> > Nicholas Johnson (1):
> >   PCI: Add pci=nobbn to ignore ACPI _BBN method to override host bridge
> >     bus window
> >
> >  Documentation/admin-guide/kernel-parameters.txt |  2 ++
> >  arch/x86/include/asm/pci_x86.h                  |  1 +
> >  arch/x86/pci/acpi.c                             | 11 +++++++++++
> >  arch/x86/pci/common.c                           |  3 +++
> >  4 files changed, 17 insertions(+)
> >
> > --
> > 2.24.0
> >

Re: [[RFC PATCH v1] 0/1] Add pci=nobbn to ignore ACPI _BBN method to override host bridge bus window

[Bjorn Helgaas]

On Thu, Nov 21, 2019 at 9:02 PM Nicholas Johnson
<nicholas.johnson-opensource@outlook.com.au> wrote:
>
> Hi all,
>
> I want to be able to override the bus resource from ACPI, but nocrs does
> not do it. I am putting this out here to get a feel for the sentiment
> for doing something like this.

This should be cc'd to linus-pci and linux-acpi (added).  I only
noticed this message by accident.  And I don't see the patch at all.

> What is my motivation for doing this?
>
> I have a Gigabyte Z170X Designare motherboard which only gives resource
> [bus 00-7e]. I want the full [bus 00-ff] because I am trying to add as
> many Thunderbolt 3 ports with add-in cards as possible. Thunderbolt
> consumes bus numbers quickly. An Intel Ice Lake implementation (ideal)
> consumes 42 busses per port, but prior solutions consume 50 busses per
> port and have additional busses required for the NHI and USB
> controllers, as well as the bridges from the root port.
>
> Why not change nocrs to do this? Why the new kernel parameter?
>
> I imagine that on systems with multiple PCI root complexes, things will
> get hairy if we do this, if they are not placed on separate segments /
> domains by the firmware. I do not own such a beast, but from what I
> understand, the firmware normally places them on the same segment /
> domain with non-overlapping bus numbers. But we may still want to use
> nocrs for other reasons. I need to use nocrs to allow Linux to allocate
> vast amounts of MMIO and MMIO_PREF under the Thunderbolt root ports
> without the BIOS support for Thunderbolt. Hence, they should be kept
> separate.
>
> Why do this in general?
>
> The bus resource is still a resource which is specified from ACPI, just
> like those overridden by nocrs. Even if we do not use pci=nocrs to
> override it, it should be possible to override it, just as it is
> possible to override _CRS.

pci=nocrs is for working around defects in firmware or Linux.  The
firmware knows more about the platform than Linux, and in general we
have to trust it.  We probably should taint the kernel when we use it.

Any parameter like this should work the same on all ACPI systems,
including ia64 and arm64, and should probably also taint the kernel.

I can't see the patch itself, but I'm a little confused because we
normally get the bus number range from _CRS in acpi_pci_root_add() and
your patch doesn't appear to touch that.

> Nicholas Johnson (1):
>   PCI: Add pci=nobbn to ignore ACPI _BBN method to override host bridge
>     bus window
>
>  Documentation/admin-guide/kernel-parameters.txt |  2 ++
>  arch/x86/include/asm/pci_x86.h                  |  1 +
>  arch/x86/pci/acpi.c                             | 11 +++++++++++
>  arch/x86/pci/common.c                           |  3 +++
>  4 files changed, 17 insertions(+)
>
> --
> 2.24.0
>

[[RFC PATCH v1] 0/1] Add pci=nobbn to ignore ACPI _BBN method to override host bridge bus window

[Nicholas Johnson]

Hi all,

I want to be able to override the bus resource from ACPI, but nocrs does 
not do it. I am putting this out here to get a feel for the sentiment 
for doing something like this.

What is my motivation for doing this?

I have a Gigabyte Z170X Designare motherboard which only gives resource 
[bus 00-7e]. I want the full [bus 00-ff] because I am trying to add as 
many Thunderbolt 3 ports with add-in cards as possible. Thunderbolt 
consumes bus numbers quickly. An Intel Ice Lake implementation (ideal) 
consumes 42 busses per port, but prior solutions consume 50 busses per 
port and have additional busses required for the NHI and USB 
controllers, as well as the bridges from the root port.

Why not change nocrs to do this? Why the new kernel parameter?

I imagine that on systems with multiple PCI root complexes, things will 
get hairy if we do this, if they are not placed on separate segments / 
domains by the firmware. I do not own such a beast, but from what I 
understand, the firmware normally places them on the same segment / 
domain with non-overlapping bus numbers. But we may still want to use 
nocrs for other reasons. I need to use nocrs to allow Linux to allocate 
vast amounts of MMIO and MMIO_PREF under the Thunderbolt root ports 
without the BIOS support for Thunderbolt. Hence, they should be kept 
separate.

Why do this in general?

The bus resource is still a resource which is specified from ACPI, just 
like those overridden by nocrs. Even if we do not use pci=nocrs to 
override it, it should be possible to override it, just as it is 
possible to override _CRS.

Topics for discussion include, but are not limited to:

	Is my code great, good, bad, ugly, or does it need work, etc?

	Which way to skin the cat / achieve the end goal, in terms of 
	code?

	Should this be done?

	Is pci=nobbn a good name for the new parameter?

	Should the documentation notice say to report a bug if you use 
	this, like the nocrs one does?

	What are the potential risks and fallout if it is done, if any? 
	My stance on this is that it will be limited to people who use 
	the parameter, so it should be safe.

	What would it take to convince you to support this?

	In relation to arch/x86/pci/acpi.c in the function 
	pci_acpi_root_prepare_resources(): When using CRS, we remove the 
	resource that satisfies resource_is_pcicfg_ioport() without the 
	dev_printk(). But when doing nocrs, we remove it along with all 
	of the others and do the dev_printk() notice. Should it be 
	changed in another patch to skip printing the notice for this 
	resource when using nocrs?

Lastly (semi-unrelated), you may notice that the email linkage is 
broken. It is because I made the mistake of using an @outlook email 
address. It interferes with the encoding. If I send patches with git 
send-email, the encoding is broken. If I use mutt -H, the encoding 
works, but the linkage is broken. I have heard that Gmail also has 
problems, but I tested it the other day with an old Gmail address I 
have, and it appears to work. But I am open for suggestions on what my 
new email domain for kernel development should be. I hope I can use a 
consumer email provider, and not mess with hosting my own domain (it 
sounds like a lot of work). I will switch over soon and send an email 
from this account to confirm that the new account is actually me.

Thanks for any comments or insights.

Kind regards,

Nicholas Johnson

Nicholas Johnson (1):
  PCI: Add pci=nobbn to ignore ACPI _BBN method to override host bridge
    bus window

 Documentation/admin-guide/kernel-parameters.txt |  2 ++
 arch/x86/include/asm/pci_x86.h                  |  1 +
 arch/x86/pci/acpi.c                             | 11 +++++++++++
 arch/x86/pci/common.c                           |  3 +++
 4 files changed, 17 insertions(+)

-- 
2.24.0

[[RFC PATCH v1] 0/1] Add pci=nobbn to ignore ACPI _BBN method to override host bridge bus window

[Nicholas Johnson]

Hi all,

I want to be able to override the bus resource from ACPI, but nocrs does 
not do it. I am putting this out here to get a feel for the sentiment 
for doing something like this.

What is my motivation for doing this?

I have a Gigabyte Z170X Designare motherboard which only gives resource 
[bus 00-7e]. I want the full [bus 00-ff] because I am trying to add as 
many Thunderbolt 3 ports with add-in cards as possible. Thunderbolt 
consumes bus numbers quickly. An Intel Ice Lake implementation (ideal) 
consumes 42 busses per port, but prior solutions consume 50 busses per 
port and have additional busses required for the NHI and USB 
controllers, as well as the bridges from the root port.

Why not change nocrs to do this? Why the new kernel parameter?

I imagine that on systems with multiple PCI root complexes, things will 
get hairy if we do this, if they are not placed on separate segments / 
domains by the firmware. I do not own such a beast, but from what I 
understand, the firmware normally places them on the same segment / 
domain with non-overlapping bus numbers. But we may still want to use 
nocrs for other reasons. I need to use nocrs to allow Linux to allocate 
vast amounts of MMIO and MMIO_PREF under the Thunderbolt root ports 
without the BIOS support for Thunderbolt. Hence, they should be kept 
separate.

Why do this in general?

The bus resource is still a resource which is specified from ACPI, just 
like those overridden by nocrs. Even if we do not use pci=nocrs to 
override it, it should be possible to override it, just as it is 
possible to override _CRS.

Topics for discussion include, but are not limited to:

	Is my code great, good, bad, ugly, or does it need work, etc?

	Which way to skin the cat / achieve the end goal, in terms of 
	code?

	Should this be done?

	Is pci=nobbn a good name for the new parameter?

	Should the documentation notice say to report a bug if you use 
	this, like the nocrs one does?

	What are the potential risks and fallout if it is done, if any? 
	My stance on this is that it will be limited to people who use 
	the parameter, so it should be safe.

	What would it take to convince you to support this?

	In relation to arch/x86/pci/acpi.c in the function 
	pci_acpi_root_prepare_resources(): When using CRS, we remove the 
	resource that satisfies resource_is_pcicfg_ioport() without the 
	dev_printk(). But when doing nocrs, we remove it along with all 
	of the others and do the dev_printk() notice. Should it be 
	changed in another patch to skip printing the notice for this 
	resource when using nocrs?

Lastly (semi-unrelated), you may notice that the email linkage is 
broken. It is because I made the mistake of using an @outlook email 
address. It interferes with the encoding. If I send patches with git 
send-email, the encoding is broken. If I use mutt -H, the encoding 
works, but the linkage is broken. I have heard that Gmail also has 
problems, but I tested it the other day with an old Gmail address I 
have, and it appears to work. But I am open for suggestions on what my 
new email domain for kernel development should be. I hope I can use a 
consumer email provider, and not mess with hosting my own domain (it 
sounds like a lot of work). I will switch over soon and send an email 
from this account to confirm that the new account is actually me.

Thanks for any comments or insights.

Kind regards,

Nicholas Johnson

Nicholas Johnson (1):
  PCI: Add pci=nobbn to ignore ACPI _BBN method to override host bridge
    bus window

 Documentation/admin-guide/kernel-parameters.txt |  2 ++
 arch/x86/include/asm/pci_x86.h                  |  1 +
 arch/x86/pci/acpi.c                             | 11 +++++++++++
 arch/x86/pci/common.c                           |  3 +++
 4 files changed, 17 insertions(+)

-- 
2.24.0