From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54534) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bhihC-000691-FB for qemu-devel@nongnu.org; Wed, 07 Sep 2016 15:39:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bhih8-0007VI-7r for qemu-devel@nongnu.org; Wed, 07 Sep 2016 15:39:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51276) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bhih7-0007V5-Vf for qemu-devel@nongnu.org; Wed, 07 Sep 2016 15:39:38 -0400 References: <1472736127-18137-1-git-send-email-marcel@redhat.com> <6c9103e7-70ad-55c6-5533-d6d9bbaa39b4@redhat.com> <739af9d6-2382-c4bd-679d-87b153124491@redhat.com> <1473229310.28663.14.camel@redhat.com> <3ed4337a-f53a-a148-88d6-a3d35ffeca79@redhat.com> From: Marcel Apfelbaum Message-ID: Date: Wed, 7 Sep 2016 22:39:28 +0300 MIME-Version: 1.0 In-Reply-To: <3ed4337a-f53a-a148-88d6-a3d35ffeca79@redhat.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH RFC] docs: add PCIe devices placement guidelines List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Laine Stump , qemu-devel@nongnu.org Cc: Gerd Hoffmann , Laszlo Ersek , mst@redhat.com, Peter Maydell , Drew Jones , Andrea Bolognani , Alex Williamson On 09/07/2016 08:55 PM, Laine Stump wrote: > On 09/07/2016 04:06 AM, Marcel Apfelbaum wrote: >> On 09/07/2016 09:21 AM, Gerd Hoffmann wrote: >>> Hi, >>> >>>>>> ports, if that's allowed). For example: >>>>>> >>>>>> - 1-32 ports needed: use root ports only >>>>>> >>>>>> - 33-64 ports needed: use 31 root ports, and one switch with 2-32 >>>>>> downstream ports >>> >>> I expect you rarely need any switches. You can go multifunction with >>> the pcie root ports. Which is how physical q35 works too btw, typically >>> the root ports are on slot 1c for intel chipsets: >>> >>> nilsson root ~# lspci -s1c >>> 00:1c.0 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset >>> Family PCI Express Root Port 1 (rev c4) >>> 00:1c.1 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset >>> Family PCI Express Root Port 2 (rev c4) >>> 00:1c.2 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset >>> Family PCI Express Root Port 3 (rev c4) >>> >>> Root bus has 32 slots, a few are taken (host bridge @ 00.0, lpc+sata @ >>> 1f.*, pci bridge @ 1e.0, maybe vga @ 01.0), leaving 28 free slots. With >>> 8 functions each you can have up to 224 root ports without any switches, >>> and you have not many pci bus numbers left until you hit the 256 busses >>> limit ... >>> >> >> Good point, maybe libvirt can avoid adding switches unless the user >> explicitly >> asked for them. I checked and it a actually works fine in QEMU. > > I'm just now writing the code that auto-adds *-ports as they are needed, and doing it this way simplifies it *immensely*. > > When I had to think about the possibility of needing upstream/downstream switches, as an endpoint device was added, I would need to check if a (root|downstream)-port was available and if not I might > be able to just add a root-port, or I might have to add a downstream-port; if the only option was a downstream port, then *that* might require adding a new *upstream* port. > > If I can limit libvirt to only auto-adding root-ports (and if there is no downside to putting multiple root ports on a single root bus port), then I just need to find an empty function of an empty > slot on the root bus, add a root-port, and I'm done (and since 224 is *a lot*, I think at least for now it's okay to punt once they get past that point). > > So, *is* there any downside to doing this? > No downside I can think of. Just be sure to emphasize the auto-add mechanism stops at 'x' devices. If the user needs more, he should manually add switches and manually assign the devices to the Downstream Ports. Thanks, Marcel >