linux-pci.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Sergei Miroshnichenko <s.miroshnichenko@yadro.com>
To: "mika.westerberg@linux.intel.com" <mika.westerberg@linux.intel.com>
Cc: "David.Laight@aculab.com" <David.Laight@aculab.com>,
	"christian@kellner.me" <christian@kellner.me>,
	"christian.koenig@amd.com" <christian.koenig@amd.com>,
	"rajatja@google.com" <rajatja@google.com>,
	"YehezkelShB@gmail.com" <YehezkelShB@gmail.com>,
	"mario.limonciello@dell.com" <mario.limonciello@dell.com>,
	"helgaas@kernel.org" <helgaas@kernel.org>,
	"linux@yadro.com" <linux@yadro.com>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"sr@denx.de" <sr@denx.de>, "lukas@wunner.de" <lukas@wunner.de>,
	"andy.lavr@gmail.com" <andy.lavr@gmail.com>
Subject: Re: [PATCH v9 00/26] PCI: Allow BAR movement during boot and hotplug
Date: Thu, 11 Feb 2021 17:45:20 +0000	[thread overview]
Message-ID: <52dd963fc697059d3db39c25eda222f4b7197761.camel@yadro.com> (raw)
In-Reply-To: <20210211113941.GF2542@lahna.fi.intel.com>

On Thu, 2021-02-11 at 13:39 +0200, mika.westerberg@linux.intel.com
wrote:
> On Wed, Feb 10, 2021 at 07:40:06PM +0000, Sergei Miroshnichenko
> wrote:
> > On Thu, 2021-02-04 at 12:49 +0200, Mika Westerberg
> > wrote:
> > > ...
> > The branch is finally ready, so if you still have time for that,
> > please
> > take a look:
> > 
> > https://github.com/YADRO-KNS/linux/tree/yadro/pcie_hotplug/movable_bars_v9.1
> 
> Thanks for sharing!
> 
> I tried this series on top of v5.11-rc7 on a Dell XPS 9380 so that I
> have two TBT3 devices connected. Each device includes PCIe switch and
> a
> xHCI endpoint.
> 
> What I see that the hotplug downstream port does not have enough bus
> numbers (and other resources allocated) so when attaching the second
> device it does not fit there anymore. The resulting 'lspci -t' output
> looks like below:
> 
> -[0000:00]-+-00.0
>            +-02.0
>            +-04.0
>            +-08.0
>            +-12.0
>            +-14.0
>            +-14.2
>            +-15.0
>            +-15.1
>            +-16.0
>            +-1c.0-[01]----00.0
>            +-1c.6-[02]----00.0
>            +-1d.0-[03-3b]----00.0-[04-3b]--+-00.0-[05]----00.0
>            |                               +-01.0-[06-1f]----00.0-
> [07-09]--+-02.0-[08]----00.0
>            |                               |                         
>       \-04.0-[09]----00.0-[0a]--
>            |                               +-02.0-[20]----00.0
>            |                               \-04.0-[21-3b]--
>            +-1d.4-[3c]----00.0
>            +-1f.0
>            +-1f.3
>            +-1f.4
>            \-1f.5
> 
> So the last PCIE switch is not visible anymore, and the xHCI on the
> second TBT device is not functional either.
> 
> On the mainline kernel I get this:
> 
> -[0000:00]-+-00.0
>            +-02.0
>            +-04.0
>            +-08.0
>            +-12.0
>            +-14.0
>            +-14.2
>            +-15.0
>            +-15.1
>            +-16.0
>            +-1c.0-[01]----00.0
>            +-1c.6-[02]----00.0
>            +-1d.0-[03-3b]----00.0-[04-3b]--+-00.0-[05]----00.0
>            |                               +-01.0-[06-1f]----00.0-
> [07-1f]--+-02.0-[08]----00.0
>            |                               |                         
>       \-04.0-[09-1f]----00.0-[0a-1f]--+-02.0-[0b]----00.0
>            |                               |                         
>                                       \-04.0-[0c-1f]--
>            |                               +-02.0-[20]----00.0
>            |                               \-04.0-[21-3b]--
>            +-1d.4-[3c]----00.0
>            +-1f.0
>            +-1f.3
>            +-1f.4
>            \-1f.5
> 
> In this topology I can add yet another TBT device and there are still
> resources available and all the endpoints are functional too.
> 
> I can send you the full dmesg and lspci -vv output if needed.


What a pity. Yes, please, I would of course like to take a look why
that happened, and compare, what went wrong (before and after the
hotplug: lspci -tv, dmesg -t and /proc/iomem with /proc/ioports, if it
wouldn't be too much trouble).

The first patchset wasn't intended to change the bus numbers behavior,
so I also have to check that locally. And despite of lack of bus
numbers, new endpoints still should be visible, this is for me to check
as well.

Thanks a lot for testing!

Serge

  reply	other threads:[~2021-02-11 17:52 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-18 17:39 [PATCH v9 00/26] PCI: Allow BAR movement during boot and hotplug Sergei Miroshnichenko
2020-12-18 17:39 ` [PATCH v9 01/26] PCI: Fix race condition in pci_enable/disable_device() Sergei Miroshnichenko
2020-12-28 15:37   ` Sergei Miroshnichenko
2020-12-18 17:39 ` [PATCH v9 02/26] PCI: Ensure a bridge has I/O and MEM access for hot-added devices Sergei Miroshnichenko
2020-12-18 17:39 ` [PATCH v9 03/26] PCI: hotplug: Initial support of the movable BARs feature Sergei Miroshnichenko
2020-12-18 17:39 ` [PATCH v9 04/26] PCI: Add version of release_child_resources() aware of fixed BARs Sergei Miroshnichenko
2020-12-18 17:39 ` [PATCH v9 05/26] PCI: hotplug: Fix reassigning the released BARs Sergei Miroshnichenko
2020-12-18 17:39 ` [PATCH v9 06/26] PCI: hotplug: Recalculate every bridge window during rescan Sergei Miroshnichenko
2020-12-18 17:39 ` [PATCH v9 07/26] PCI: hotplug: Don't allow hot-added devices to steal resources Sergei Miroshnichenko
2020-12-18 17:39 ` [PATCH v9 08/26] PCI: Reassign BARs if BIOS/bootloader had assigned not all of them Sergei Miroshnichenko
2020-12-18 17:39 ` [PATCH v9 09/26] PCI: Movable BARs: Make just a single attempt to assign bridge resources Sergei Miroshnichenko
2020-12-18 17:39 ` [PATCH v9 10/26] PCI: hotplug: Calculate fixed parts of bridge windows Sergei Miroshnichenko
2020-12-18 17:39 ` [PATCH v9 11/26] PCI: Include fixed BARs into the bus size calculating Sergei Miroshnichenko
2020-12-18 17:39 ` [PATCH v9 12/26] PCI: Make sure bridge windows include their fixed BARs Sergei Miroshnichenko
2020-12-18 17:39 ` [PATCH v9 13/26] PCI: hotplug: Add support of fixed BARs to pci_assign_resource() Sergei Miroshnichenko
2020-12-18 17:39 ` [PATCH v9 14/26] PCI: hotplug: Sort fixed BARs before assignment Sergei Miroshnichenko
2020-12-18 17:40 ` [PATCH v9 15/26] x86/PCI/ACPI: Fix up PCIBIOS_MIN_MEM if value computed from e820 is invalid Sergei Miroshnichenko
2020-12-18 17:40 ` [PATCH v9 16/26] PCI: hotplug: Configure MPS after manual bus rescan Sergei Miroshnichenko
2020-12-18 17:40 ` [PATCH v9 17/26] PCI: hotplug: Don't disable the released bridge windows immediately Sergei Miroshnichenko
2020-12-18 17:40 ` [PATCH v9 18/26] PCI: pciehp: Trigger a domain rescan on hp events when enabled movable BARs Sergei Miroshnichenko
2020-12-18 17:40 ` [PATCH v9 19/26] PCI: Don't claim fixed BARs Sergei Miroshnichenko
2020-12-18 17:40 ` [PATCH v9 20/26] PCI: hotplug: Retry bus assignment without reserved space Sergei Miroshnichenko
2020-12-18 17:40 ` [PATCH v9 21/26] PCI: Rescan the bus to resize a BAR Sergei Miroshnichenko
2020-12-18 17:40 ` [PATCH v9 22/26] PCI: hotplug: Enable the movable BARs feature by default Sergei Miroshnichenko
2020-12-18 17:40 ` [PATCH v9 23/26] PCI/portdrv: Declare support of movable BARs Sergei Miroshnichenko
2020-12-18 17:40 ` [PATCH v9 24/26] nvme-pci: Handle " Sergei Miroshnichenko
2020-12-18 17:40 ` [PATCH v9 25/26] PCI: Add a message for updating BARs Sergei Miroshnichenko
2020-12-18 17:40 ` [PATCH v9 26/26] resource: increase max nesting level for /proc/iomem Sergei Miroshnichenko
2021-01-28 14:53 ` [PATCH v9 00/26] PCI: Allow BAR movement during boot and hotplug Bjorn Helgaas
2021-01-28 20:39   ` Lukas Wunner
2021-02-01 12:55     ` Mika Westerberg
2021-02-03 20:17       ` Sergei Miroshnichenko
2021-02-04 10:49         ` mika.westerberg
2021-02-10 19:40           ` Sergei Miroshnichenko
2021-02-10 21:46             ` Lukas Wunner
2021-02-11 11:39             ` mika.westerberg
2021-02-11 17:45               ` Sergei Miroshnichenko [this message]
2021-02-12 12:52                 ` mika.westerberg
2021-02-12 20:54                   ` Sergei Miroshnichenko
2021-02-15 14:56                     ` mika.westerberg
2021-02-03 20:01     ` Sergei Miroshnichenko
2021-02-04  9:34       ` David Laight
2021-02-03 19:59   ` Sergei Miroshnichenko
2021-02-04  8:26     ` Hinko Kocevar

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=52dd963fc697059d3db39c25eda222f4b7197761.camel@yadro.com \
    --to=s.miroshnichenko@yadro.com \
    --cc=David.Laight@aculab.com \
    --cc=YehezkelShB@gmail.com \
    --cc=andy.lavr@gmail.com \
    --cc=christian.koenig@amd.com \
    --cc=christian@kellner.me \
    --cc=helgaas@kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=linux@yadro.com \
    --cc=lukas@wunner.de \
    --cc=mario.limonciello@dell.com \
    --cc=mika.westerberg@linux.intel.com \
    --cc=rajatja@google.com \
    --cc=sr@denx.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).