From: "Singh, Brijesh" <brijesh.singh@amd.com>
To: Alex Williamson <alex.williamson@redhat.com>,
Peter Xu <zhexu@redhat.com>
Cc: "Singh, Brijesh" <brijesh.singh@amd.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
"eric.auger@redhat.com" <eric.auger@redhat.com>,
"Suthikulpanit, Suravee" <Suravee.Suthikulpanit@amd.com>
Subject: Re: [Qemu-devel] [RFC PATCH] pci: Use PCI aliases when determining device IOMMU address space
Date: Thu, 25 Jul 2019 14:34:24 +0000 [thread overview]
Message-ID: <5ec9b21e-cf89-5d28-7671-aa84e9a4f360@amd.com> (raw)
In-Reply-To: <20190724134247.61c2104a@x1.home>
On 7/24/19 2:42 PM, Alex Williamson wrote:
> On Wed, 24 Jul 2019 08:43:55 -0600
> Alex Williamson <alex.williamson@redhat.com> wrote:
>
>> On Wed, 24 Jul 2019 18:03:31 +0800
>> Peter Xu <zhexu@redhat.com> wrote:
>>
>>> On Wed, Jul 24, 2019 at 05:39:22AM -0400, Michael S. Tsirkin wrote:
>>>> On Wed, Jul 24, 2019 at 03:14:39PM +0800, Peter Xu wrote:
>>>>> On Tue, Jul 23, 2019 at 11:26:18AM -0600, Alex Williamson wrote:
>>>>>>> On 3/29/19 11:49 AM, Alex Williamson wrote:
>>>>>>>> [Cc +Brijesh]
>>>>>>>>
>>>>>>>> Hi Brijesh, will the change below require the IVRS to be updated to
>>>>>>>> include aliases for all BDF ranges behind a conventional bridge? I
>>>>>>>> think the Linux code handles this regardless of the firmware provided
>>>>>>>> aliases, but is it required per spec for the ACPI tables to include
>>>>>>>> bridge aliases? Thanks,
> [snip]
>
> For a data point, I fired up an old 990FX system which includes a
> PCIe-to-PCI bridge and I added a plugin PCIe-to-PCI bridge just to keep
> things interesting... guess how many alias ranges are in the IVRS...
> Yep, just the one built into the motherboard:
>
> AMD-Vi: Using IVHD type 0x10
> AMD-Vi: device: 00:00.2 cap: 0040 seg: 0 flags: 3e info 1300
> AMD-Vi: mmio-addr: 00000000fec30000
> AMD-Vi: DEV_SELECT_RANGE_START devid: 00:00.0 flags: 00
> AMD-Vi: DEV_RANGE_END devid: 00:00.2
> AMD-Vi: DEV_SELECT devid: 00:09.0 flags: 00
> AMD-Vi: DEV_SELECT devid: 01:00.0 flags: 00
> AMD-Vi: DEV_SELECT devid: 00:0a.0 flags: 00
> AMD-Vi: DEV_SELECT devid: 02:00.0 flags: 00
> AMD-Vi: DEV_SELECT devid: 00:11.0 flags: 00
> AMD-Vi: DEV_SELECT_RANGE_START devid: 00:12.0 flags: 00
> AMD-Vi: DEV_RANGE_END devid: 00:12.2
> AMD-Vi: DEV_SELECT_RANGE_START devid: 00:13.0 flags: 00
> AMD-Vi: DEV_RANGE_END devid: 00:13.2
> AMD-Vi: DEV_SELECT devid: 00:14.0 flags: d7
> AMD-Vi: DEV_SELECT devid: 00:14.2 flags: 00
> AMD-Vi: DEV_SELECT devid: 00:14.3 flags: 00
> AMD-Vi: DEV_SELECT devid: 00:14.4 flags: 00
> AMD-Vi: DEV_ALIAS_RANGE devid: 03:00.0 flags: 00 devid_to: 00:14.4
> AMD-Vi: DEV_RANGE_END devid: 03:1f.7
>
> [Everything on bus 03:xx.x is aliased to device 00:14.4, the builtin PCIe-to-PCI bridge]
>
> AMD-Vi: DEV_SELECT devid: 00:14.5 flags: 00
> AMD-Vi: DEV_SELECT devid: 00:15.0 flags: 00
> AMD-Vi: DEV_SELECT_RANGE_START devid: 04:00.0 flags: 00
> AMD-Vi: DEV_RANGE_END devid: 04:1f.7
> AMD-Vi: DEV_SELECT devid: 00:15.1 flags: 00
> AMD-Vi: DEV_SELECT_RANGE_START devid: 05:00.0 flags: 00
> AMD-Vi: DEV_RANGE_END devid: 05:1f.7
> AMD-Vi: DEV_SELECT devid: 00:15.2 flags: 00
> AMD-Vi: DEV_SELECT_RANGE_START devid: 06:00.0 flags: 00
> AMD-Vi: DEV_RANGE_END devid: 06:1f.7
> AMD-Vi: DEV_SELECT devid: 00:15.3 flags: 00
> AMD-Vi: DEV_SELECT_RANGE_START devid: 08:00.0 flags: 00
> AMD-Vi: DEV_RANGE_END devid: 08:1f.7
> AMD-Vi: DEV_SELECT_RANGE_START devid: 00:16.0 flags: 00
> AMD-Vi: DEV_RANGE_END devid: 00:16.2
> AMD-Vi: DEV_SPECIAL(IOAPIC[8]) devid: 00:14.0
> AMD-Vi: DEV_SPECIAL(HPET[0]) devid: 00:14.0
> AMD-Vi: DEV_SPECIAL(IOAPIC[8]) devid: 00:00.1
>
> -[0000:00]-+-00.0 Advanced Micro Devices, Inc. [AMD/ATI] RD9x0/RX980 Host Bridge
> +-00.2 Advanced Micro Devices, Inc. [AMD/ATI] RD890S/RD990 I/O Memory Management Unit (IOMMU)
> +-09.0-[01]----00.0 Etron Technology, Inc. EJ168 USB 3.0 Host Controller
> +-0a.0-[02]----00.0 Marvell Technology Group Ltd. 88SE9172 SATA 6Gb/s Controller
> +-11.0 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode]
> +-12.0 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
> +-12.2 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
> +-13.0 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
> +-13.2 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
> +-14.0 Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller
> +-14.2 Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA)
> +-14.3 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller
> +-14.4-[03]--+-06.0 NVidia / SGS Thomson (Joint Venture) Riva128
> | \-0e.0 VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller
> +-14.5 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
> +-15.0-[04]----00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
> +-15.1-[05]----00.0 Etron Technology, Inc. EJ168 USB 3.0 Host Controller
> +-15.2-[06-07]----00.0-[07]----00.0 Realtek Semiconductor Co., Ltd. Device 8149
> +-15.3-[08]--
> +-16.0 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
> +-16.2 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
> +-18.0 Advanced Micro Devices, Inc. [AMD] Family 10h Processor HyperTransport Configuration
> +-18.1 Advanced Micro Devices, Inc. [AMD] Family 10h Processor Address Map
> +-18.2 Advanced Micro Devices, Inc. [AMD] Family 10h Processor DRAM Controller
> +-18.3 Advanced Micro Devices, Inc. [AMD] Family 10h Processor Miscellaneous Control
> \-18.4 Advanced Micro Devices, Inc. [AMD] Family 10h Processor Link Control
>
> 00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge (rev 40) (prog-if 01 [Subtractive decode])
> Bus: primary=00, secondary=03, subordinate=03, sec-latency=64
>
> 06:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 03) (prog-if 00 [Normal decode])
> Bus: primary=06, secondary=07, subordinate=07, sec-latency=32
> Capabilities: [80] Express (v1) PCI-Express to PCI/PCI-X Bridge, MSI 00
>
> I realized as I was writing, that the alias caused by 06:00.0 would be
> devfn 00.0 on the secondary bus 07, ie. 07:00.0 would alias to
> 07:00.0, so technically there's really not an alias for this small
> case. So I replaced the NIC with this:
>
> +-15.2-[06-07]----00.0-[07]--+-00.0 NEC Corporation OHCI USB Controller
> +-00.1 NEC Corporation OHCI USB Controller
> \-00.2 NEC Corporation uPD72010x USB 2.0 Controller
>
> Now functions 07:00.[12] also alias to 07:00.0. The IVRS table is
> unaffected.
>
> I'm tempted to say that QEMU would be no worse than bare metal if we
> simply ignore IVHD device alias entries. I know that Linux will figure
> out the aliasing regardless of the IVRS. Will Windows guests? I'd
> love to hear from Bijesh or Suravee regarding the behavior above versus
> what AMD expects from system vendors. Thanks,
>
Alex,
I am not well versed on the expected IOMMU address space behavior yet,
Suravee will know more about. I will ask him to take a look at this
thread.
thanks
> Alex
>
next prev parent reply other threads:[~2019-07-25 14:34 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <155364082689.15803.7062874513041742278.stgit@gimli.home>
[not found] ` <20190329104904.450fefef@x1.home>
2019-04-01 13:41 ` [Qemu-devel] [RFC PATCH] pci: Use PCI aliases when determining device IOMMU address space Singh, Brijesh
2019-07-23 17:26 ` Alex Williamson
2019-07-23 18:45 ` Michael S. Tsirkin
2019-07-24 7:14 ` Peter Xu
2019-07-24 9:39 ` Michael S. Tsirkin
2019-07-24 10:03 ` Peter Xu
2019-07-24 14:43 ` Alex Williamson
2019-07-24 19:42 ` Alex Williamson
2019-07-25 14:34 ` Singh, Brijesh [this message]
2019-07-25 6:37 ` Peter Xu
2019-07-25 10:43 ` Michael S. Tsirkin
2019-07-25 14:00 ` Alex Williamson
2019-07-25 15:22 ` Michael S. Tsirkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5ec9b21e-cf89-5d28-7671-aa84e9a4f360@amd.com \
--to=brijesh.singh@amd.com \
--cc=Suravee.Suthikulpanit@amd.com \
--cc=alex.williamson@redhat.com \
--cc=eric.auger@redhat.com \
--cc=mst@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=zhexu@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.