RE: [External] Re: [PATCH v1] PCI: Set no io resource for bridges that behind VMD controller

[Xiaochun XC17 Li <lixc17@lenovo.com>]

On Sat, Nov 11, 2022 at 07:54:25, "Bjorn Helgaas" <helgaas@kernel.org> wrote:
> On Tue, Sep 13, 2022 at 09:24:45PM +0800, Xiaochun Lee wrote:
> > From: Xiaochun Lee <lixc17@lenovo.com>
> >
> > When enable VMDs on Intel CPUs, VMD controllers(8086:28c0) be
> > recognized by VMD driver and there are many failed messages of BAR 13
> > when scan the bridges and assign IO resource behind it as listed
> > below, the bridge wants to get 0x6000 as its IO resource, but there is
> > no IO resources on the host bridge.
> >
> > VMD host bridge resources:
> > vmd 0000:64:00.5: PCI host bridge to bus 10000:80 pci_bus 10000:80:
> > root bus resource [bus 80-9f] pci_bus 10000:80: root bus resource [mem
> > 0xe0000000-0xe1ffffff] pci_bus 10000:80: root bus resource [mem
> > 0x24ffff02010-0x24fffffffff 64bit]
> >
> > Failed messages of BAR#13:
> > pci 10000:80:02.0: BAR 13: no space for [io  size 0x1000] pci
> > 10000:80:02.0: BAR 13: failed to assign [io  size 0x1000] pci
> > 10000:80:03.0: BAR 13: no space for [io  size 0x1000] pci
> > 10000:80:03.0: BAR 13: failed to assign [io  size 0x1000]
> >
> > VMD-enabled root ports use
> > Enhanced Configuration Access Mechanism (ECAM) access PCI Express
> > configuration space, and offer VMD_CFGBAR as base of PCI Express
> > configuration space for the bridges behind it. The configuration space
> > includes IO resources, but these IO resources are not actually used on
> > X86, especially the NVMes as device connected on this hot plug
> > bridges, and it can result in BAR#13 assign IO resource failed. So we
> > clear IO resources by setting an IO base value greater than limit to
> > these bridges. Hence, we can leverage kernel parameter
> > "pci=hpiosize=0KB" to avoid this failed messages show out.
> >
> > Signed-off-by: Xiaochun Lee <lixc17@lenovo.com>
> 
> Some of the discussion here got lost because of email issues.  Lore has
> some:
> https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.
> kernel.org%2Fall%2F1663075485-20591-1-git-send-email-
> lixiaochun.2888%40163.com%2FT%2F%23u&amp;data=05%7C01%7Clixc17
> %40lenovo.com%7C9cd095ffdb584e492dec08dac440139b%7C5c7d0b28bdf
> 8410caa934df372b16203%7C0%7C0%7C638038076734438158%7CUnknown
> %7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> WwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=JzWYkIsaEfINofiqo
> XyjEh43VjXO3HZw2JLSsmhpUiQ%3D&amp;reserved=0,
> and patchwork has a v2 with a little more discussion:
> https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatc
> hwork.kernel.org%2Fproject%2Flinux-pci%2Fpatch%2F1664288166-7432-1-
> git-send-email-
> lixiaochun.2888%40163.com%2F&amp;data=05%7C01%7Clixc17%40lenovo.
> com%7C9cd095ffdb584e492dec08dac440139b%7C5c7d0b28bdf8410caa934
> df372b16203%7C0%7C0%7C638038076734438158%7CUnknown%7CTWFpb
> GZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI
> 6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=y%2BtIsepTpyLeoHW6CrgkZz2
> tiMjY0TONfK7zNCKXQ90%3D&amp;reserved=0
> 
> But the v2 patch doesn't seem to have made it to the mailing lists or to
> lore
> (https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flor
> e.kernel.org%2Fall%2F%3Fq%3Df%253Alixc17&amp;data=05%7C01%7Clixc1
> 7%40lenovo.com%7C9cd095ffdb584e492dec08dac440139b%7C5c7d0b28bd
> f8410caa934df372b16203%7C0%7C0%7C638038076734438158%7CUnknow
> n%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h
> aWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=zxGz1hZOD2tvQP
> EsbxQTzjHwQvXvqeO%2FUd6I9S%2Fj314%3D&amp;reserved=0) and I don't
> apply things until they appear on the mailing list.
> 
> I *would* like to get rid of those "no space" and "failed to assign"
> messages.  This is an issue for platforms other than VMD, too.  Just an FYI
> that you need to follow up on this if we want make progress.
> 
> Bjorn

Thanks for your comments, so do you mean we'd better come up with a
solution to avoid messages like "no space" and "failed to assign" using
a common way, for both VMD and other platforms?

Xiaochun