From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A114C433DF for ; Fri, 3 Jul 2020 00:52:34 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5B1942073E for ; Fri, 3 Jul 2020 00:52:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5B1942073E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=bugs.launchpad.net Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:55662 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jr9wP-0004yq-Ig for qemu-devel@archiver.kernel.org; Thu, 02 Jul 2020 20:52:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46220) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jr9uk-0002Cf-Lg for qemu-devel@nongnu.org; Thu, 02 Jul 2020 20:50:51 -0400 Received: from indium.canonical.com ([91.189.90.7]:38222) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jr9ug-0007QB-IE for qemu-devel@nongnu.org; Thu, 02 Jul 2020 20:50:50 -0400 Received: from loganberry.canonical.com ([91.189.90.37]) by indium.canonical.com with esmtp (Exim 4.86_2 #2 (Debian)) id 1jr9uc-0001gS-6u for ; Fri, 03 Jul 2020 00:50:42 +0000 Received: from loganberry.canonical.com (localhost [127.0.0.1]) by loganberry.canonical.com (Postfix) with ESMTP id E0EF42E811D for ; Fri, 3 Jul 2020 00:50:40 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Date: Fri, 03 Jul 2020 00:36:12 -0000 From: costinel <1869006@bugs.launchpad.net> To: qemu-devel@nongnu.org X-Launchpad-Notification-Type: bug X-Launchpad-Bug: product=qemu; status=New; importance=Undecided; assignee=None; X-Launchpad-Bug-Tags: arm passthrough tcg X-Launchpad-Bug-Information-Type: Public X-Launchpad-Bug-Private: no X-Launchpad-Bug-Security-Vulnerability: no X-Launchpad-Bug-Commenters: alex-l-williamson costinel dgilbert-h hrw pmaydell X-Launchpad-Bug-Reporter: Marcin Juszkiewicz (hrw) X-Launchpad-Bug-Modifier: costinel (costinel) References: <158514404728.11288.8869885318197124821.malonedeb@soybean.canonical.com> Message-Id: <159373657302.3426.5144140385853263611.malone@wampee.canonical.com> Subject: [Bug 1869006] Re: PCIe cards passthrough to TCG guest works on 2GB of guest memory but fails on 4GB (vfio_dma_map invalid arg) X-Launchpad-Message-Rationale: Subscriber (QEMU) @qemu-devel-ml X-Launchpad-Message-For: qemu-devel-ml Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="1cbd0aa39df153c901321817f9b57cf3f232b507"; Instance="production-secrets-lazr.conf" X-Launchpad-Hash: edc29670eae8dd88aa5f9e90bfdc8ca43d2921ee Received-SPF: none client-ip=91.189.90.7; envelope-from=bounces@canonical.com; helo=indium.canonical.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/02 20:50:44 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -58 X-Spam_score: -5.9 X-Spam_bar: ----- X-Spam_report: (-5.9 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Bug 1869006 <1869006@bugs.launchpad.net> Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Alex, thanks for the quick answer, but sadly I still do not fully understand the implications, even if I read the pdf paper on RH website you mention, as well as the vendor advisory at https://support.hpe.com/hpesc/public/docDisplay?docId=3Demr_na-c04781229 When you say "qemu has no support", do you actually mean "qemu people are unable to help you if you break things by bypassing the in-place restrictions", or "qemu is designed to not work when restrictions are bypassed"? Do I understand correctly that the BIOS can modify portions of the system usable RAM, so the vendor specific software tools can read those addresses, and if yes, does this mean is there a risk for data corruption if the RMRR restrictions are bypassed? I have eventually managed to passthrough an nvidia card in the microserver gen8 to a windows vm using patched kernel 5.3, along with the vendor instructions to exclude the pcie slot aka the conrep solution but for it to work it still needed the "rmrr patch" aka removing the "return -EPERM" line below the "Device is ineligible [...]" in drivers/iommu/intel-iommu.c However applying the same modification to kernel 5.4 leads to the "VFIO_MAP= _DMA: -22" error. Is there other place in the kernel 5.4 source that must be modified to bring back the v5.3 kernel behaviour? (ie. I have a stable home windows vm with the gpu passthrough despite all) -- = You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1869006 Title: PCIe cards passthrough to TCG guest works on 2GB of guest memory but fails on 4GB (vfio_dma_map invalid arg) Status in QEMU: New Bug description: During one meeting coworker asked "did someone tried to passthrough PCIe card to other arch guest?" and I decided to check it. Plugged SATA and USB3 controllers into spare slots on mainboard and started playing. On 1GB VM instance it worked (both cold- and hot- plugged). On 4GB one it did not: B=C5=82=C4=85d podczas uruchamiania domeny: internal error: process exite= d while connecting to monitor: 2020-03-25T13:43:39.107524Z qemu-system-aarc= h64: -device vfio-pci,host=3D0000:29:00.0,id=3Dhostdev0,bus=3Dpci.3,addr=3D= 0x0: VFIO_MAP_DMA: -22 2020-03-25T13:43:39.107560Z qemu-system-aarch64: -device vfio-pci,host=3D= 0000:29:00.0,id=3Dhostdev0,bus=3Dpci.3,addr=3D0x0: vfio 0000:29:00.0: faile= d to setup container for group 28: memory listener initialization failed: R= egion mach-virt.ram: vfio_dma_map(0x563169753c80, 0x40000000, 0x100000000, = 0x7fb2a3e00000) =3D -22 (Invalid argument) Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/asyncjob.py", line 75, in cb_= wrapper callback(asyncjob, *args, **kwargs) File "/usr/share/virt-manager/virtManager/asyncjob.py", line 111, in tm= pcb callback(*args, **kwargs) File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", lin= e 66, in newfn ret =3D fn(self, *args, **kwargs) File "/usr/share/virt-manager/virtManager/object/domain.py", line 1279,= in startup self._backend.create() File "/usr/lib64/python3.8/site-packages/libvirt.py", line 1234, in cre= ate if ret =3D=3D -1: raise libvirtError ('virDomainCreate() failed', dom= =3Dself) libvirt.libvirtError: internal error: process exited while connecting to = monitor: 2020-03-25T13:43:39.107524Z qemu-system-aarch64: -device vfio-pci,= host=3D0000:29:00.0,id=3Dhostdev0,bus=3Dpci.3,addr=3D0x0: VFIO_MAP_DMA: -22 2020-03-25T13:43:39.107560Z qemu-system-aarch64: -device vfio-pci,host=3D= 0000:29:00.0,id=3Dhostdev0,bus=3Dpci.3,addr=3D0x0: vfio 0000:29:00.0: faile= d to setup container for group 28: memory listener initialization failed: R= egion mach-virt.ram: vfio_dma_map(0x563169753c80, 0x40000000, 0x100000000, = 0x7fb2a3e00000) =3D -22 (Invalid argument) = I played with memory and 3054 MB is maximum value possible to boot VM wit= h coldplugged host PCIe cards. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1869006/+subscriptions