From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751284AbbKJINB (ORCPT ); Tue, 10 Nov 2015 03:13:01 -0500 Received: from david.siemens.de ([192.35.17.14]:58826 "EHLO david.siemens.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750905AbbKJIM7 (ORCPT ); Tue, 10 Nov 2015 03:12:59 -0500 X-Greylist: delayed 2625 seconds by postgrey-1.27 at vger.kernel.org; Tue, 10 Nov 2015 03:12:58 EST Subject: Re: [PATCH v4 0/6] virtio core DMA API conversion To: Andy Lutomirski , Benjamin Herrenschmidt References: <20151109133624-mutt-send-email-mst@redhat.com> <1447109937.31884.42.camel@kernel.crashing.org> <1447121076.31884.61.camel@kernel.crashing.org> Cc: Andy Lutomirski , David Woodhouse , "linux-kernel@vger.kernel.org" , "David S. Miller" , sparclinux@vger.kernel.org, Joerg Roedel , Christian Borntraeger , Cornelia Huck , Sebastian Ott , Paolo Bonzini , Christoph Hellwig , KVM , Martin Schwidefsky , linux-s390 , Linux Virtualization , "Michael S. Tsirkin" From: Jan Kiszka Message-ID: <56419C92.9050704@siemens.com> Date: Tue, 10 Nov 2015 08:28:18 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2015-11-10 03:18, Andy Lutomirski wrote: > On Mon, Nov 9, 2015 at 6:04 PM, Benjamin Herrenschmidt >> I thus go back to my original statement, it's a LOT easier to handle if >> the device itself is self describing, indicating whether it is set to >> bypass a host iommu or not. For L1->L2, well, that wouldn't be the >> first time qemu/VFIO plays tricks with the passed through device >> configuration space... > > Which leaves the special case of Xen, where even preexisting devices > don't bypass the IOMMU. Can we keep this specific to powerpc and > sparc? On x86, this problem is basically nonexistent, since the IOMMU > is properly self-describing. > > IOW, I think that on x86 we should assume that all virtio devices > honor the IOMMU. >>From the guest driver POV, that is OK because either there is no IOMMU to program (the current situation with qemu), there can be one that doesn't need it (the current situation with qemu and iommu=on) or there is (Xen) or will be (future qemu) one that requires it. > >> >> Note that the above can be solved via some kind of compromise: The >> device self describes the ability to honor the iommu, along with the >> property (or ACPI table entry) that indicates whether or not it does. >> >> IE. We could use the revision or ProgIf field of the config space for >> example. Or something in virtio config. If it's an "old" device, we >> know it always bypass. If it's a new device, we know it only bypasses >> if the corresponding property is in. I still would have to sort out the >> openbios case for mac among others but it's at least a workable >> direction. >> >> BTW. Don't you have a similar problem on x86 that today qemu claims >> that everything honors the iommu in ACPI ? > > Only on a single experimental configuration, and that can apparently > just be fixed going forward without any real problems being caused. BTW, I once tried to describe the current situation on QEMU x86 with IOMMU enabled via ACPI. While you can easily add IOMMU device exceptions to the static tables, the fun starts when considering device hotplug for virtio. Unless I missed some trick, ACPI doesn't seem like being designed for that level of flexibility. You would have to reserve a complete PCI bus, declare that one as not being IOMMU-governed, and then only add new virtio devices to that bus. Possible, but a lot of restrictions that existing management software would have to be aware of as well. Jan -- Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jan Kiszka Subject: Re: [PATCH v4 0/6] virtio core DMA API conversion Date: Tue, 10 Nov 2015 08:28:18 +0100 Message-ID: <56419C92.9050704@siemens.com> References: <20151109133624-mutt-send-email-mst@redhat.com> <1447109937.31884.42.camel@kernel.crashing.org> <1447121076.31884.61.camel@kernel.crashing.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org List-Archive: List-Post: To: Andy Lutomirski , Benjamin Herrenschmidt Cc: Joerg Roedel , KVM , linux-s390 , "Michael S. Tsirkin" , Sebastian Ott , "linux-kernel@vger.kernel.org" , Christoph Hellwig , Christian Borntraeger , Andy Lutomirski , sparclinux@vger.kernel.org, Paolo Bonzini , Linux Virtualization , David Woodhouse , "David S. Miller" , Martin Schwidefsky List-ID: On 2015-11-10 03:18, Andy Lutomirski wrote: > On Mon, Nov 9, 2015 at 6:04 PM, Benjamin Herrenschmidt >> I thus go back to my original statement, it's a LOT easier to handle if >> the device itself is self describing, indicating whether it is set to >> bypass a host iommu or not. For L1->L2, well, that wouldn't be the >> first time qemu/VFIO plays tricks with the passed through device >> configuration space... > > Which leaves the special case of Xen, where even preexisting devices > don't bypass the IOMMU. Can we keep this specific to powerpc and > sparc? On x86, this problem is basically nonexistent, since the IOMMU > is properly self-describing. > > IOW, I think that on x86 we should assume that all virtio devices > honor the IOMMU. >From the guest driver POV, that is OK because either there is no IOMMU to program (the current situation with qemu), there can be one that doesn't need it (the current situation with qemu and iommu=on) or there is (Xen) or will be (future qemu) one that requires it. > >> >> Note that the above can be solved via some kind of compromise: The >> device self describes the ability to honor the iommu, along with the >> property (or ACPI table entry) that indicates whether or not it does. >> >> IE. We could use the revision or ProgIf field of the config space for >> example. Or something in virtio config. If it's an "old" device, we >> know it always bypass. If it's a new device, we know it only bypasses >> if the corresponding property is in. I still would have to sort out the >> openbios case for mac among others but it's at least a workable >> direction. >> >> BTW. Don't you have a similar problem on x86 that today qemu claims >> that everything honors the iommu in ACPI ? > > Only on a single experimental configuration, and that can apparently > just be fixed going forward without any real problems being caused. BTW, I once tried to describe the current situation on QEMU x86 with IOMMU enabled via ACPI. While you can easily add IOMMU device exceptions to the static tables, the fun starts when considering device hotplug for virtio. Unless I missed some trick, ACPI doesn't seem like being designed for that level of flexibility. You would have to reserve a complete PCI bus, declare that one as not being IOMMU-governed, and then only add new virtio devices to that bus. Possible, but a lot of restrictions that existing management software would have to be aware of as well. Jan -- Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jan Kiszka Date: Tue, 10 Nov 2015 07:28:18 +0000 Subject: Re: [PATCH v4 0/6] virtio core DMA API conversion Message-Id: <56419C92.9050704@siemens.com> List-Id: References: <20151109133624-mutt-send-email-mst@redhat.com> <1447109937.31884.42.camel@kernel.crashing.org> <1447121076.31884.61.camel@kernel.crashing.org> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Andy Lutomirski , Benjamin Herrenschmidt Cc: Joerg Roedel , KVM , linux-s390 , "Michael S. Tsirkin" , Sebastian Ott , "linux-kernel@vger.kernel.org" , Christoph Hellwig , Christian Borntraeger , Andy Lutomirski , sparclinux@vger.kernel.org, Paolo Bonzini , Linux Virtualization , David Woodhouse , "David S. Miller" , Martin Schwidefsky On 2015-11-10 03:18, Andy Lutomirski wrote: > On Mon, Nov 9, 2015 at 6:04 PM, Benjamin Herrenschmidt >> I thus go back to my original statement, it's a LOT easier to handle if >> the device itself is self describing, indicating whether it is set to >> bypass a host iommu or not. For L1->L2, well, that wouldn't be the >> first time qemu/VFIO plays tricks with the passed through device >> configuration space... > > Which leaves the special case of Xen, where even preexisting devices > don't bypass the IOMMU. Can we keep this specific to powerpc and > sparc? On x86, this problem is basically nonexistent, since the IOMMU > is properly self-describing. > > IOW, I think that on x86 we should assume that all virtio devices > honor the IOMMU. >From the guest driver POV, that is OK because either there is no IOMMU to program (the current situation with qemu), there can be one that doesn't need it (the current situation with qemu and iommu=on) or there is (Xen) or will be (future qemu) one that requires it. > >> >> Note that the above can be solved via some kind of compromise: The >> device self describes the ability to honor the iommu, along with the >> property (or ACPI table entry) that indicates whether or not it does. >> >> IE. We could use the revision or ProgIf field of the config space for >> example. Or something in virtio config. If it's an "old" device, we >> know it always bypass. If it's a new device, we know it only bypasses >> if the corresponding property is in. I still would have to sort out the >> openbios case for mac among others but it's at least a workable >> direction. >> >> BTW. Don't you have a similar problem on x86 that today qemu claims >> that everything honors the iommu in ACPI ? > > Only on a single experimental configuration, and that can apparently > just be fixed going forward without any real problems being caused. BTW, I once tried to describe the current situation on QEMU x86 with IOMMU enabled via ACPI. While you can easily add IOMMU device exceptions to the static tables, the fun starts when considering device hotplug for virtio. Unless I missed some trick, ACPI doesn't seem like being designed for that level of flexibility. You would have to reserve a complete PCI bus, declare that one as not being IOMMU-governed, and then only add new virtio devices to that bus. Possible, but a lot of restrictions that existing management software would have to be aware of as well. Jan -- Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jan Kiszka Subject: Re: [PATCH v4 0/6] virtio core DMA API conversion Date: Tue, 10 Nov 2015 08:28:18 +0100 Message-ID: <56419C92.9050704@siemens.com> References: <20151109133624-mutt-send-email-mst@redhat.com> <1447109937.31884.42.camel@kernel.crashing.org> <1447121076.31884.61.camel@kernel.crashing.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: Joerg Roedel , KVM , linux-s390 , "Michael S. Tsirkin" , Sebastian Ott , "linux-kernel@vger.kernel.org" , Christoph Hellwig , Christian Borntraeger , Andy Lutomirski , sparclinux@vger.kernel.org, Paolo Bonzini , Linux Virtualization , David Woodhouse , "David S. Miller" , Martin Schwidefsky To: Andy Lutomirski , Benjamin Herrenschmidt Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org List-Id: kvm.vger.kernel.org On 2015-11-10 03:18, Andy Lutomirski wrote: > On Mon, Nov 9, 2015 at 6:04 PM, Benjamin Herrenschmidt >> I thus go back to my original statement, it's a LOT easier to handle if >> the device itself is self describing, indicating whether it is set to >> bypass a host iommu or not. For L1->L2, well, that wouldn't be the >> first time qemu/VFIO plays tricks with the passed through device >> configuration space... > > Which leaves the special case of Xen, where even preexisting devices > don't bypass the IOMMU. Can we keep this specific to powerpc and > sparc? On x86, this problem is basically nonexistent, since the IOMMU > is properly self-describing. > > IOW, I think that on x86 we should assume that all virtio devices > honor the IOMMU. >>From the guest driver POV, that is OK because either there is no IOMMU to program (the current situation with qemu), there can be one that doesn't need it (the current situation with qemu and iommu=on) or there is (Xen) or will be (future qemu) one that requires it. > >> >> Note that the above can be solved via some kind of compromise: The >> device self describes the ability to honor the iommu, along with the >> property (or ACPI table entry) that indicates whether or not it does. >> >> IE. We could use the revision or ProgIf field of the config space for >> example. Or something in virtio config. If it's an "old" device, we >> know it always bypass. If it's a new device, we know it only bypasses >> if the corresponding property is in. I still would have to sort out the >> openbios case for mac among others but it's at least a workable >> direction. >> >> BTW. Don't you have a similar problem on x86 that today qemu claims >> that everything honors the iommu in ACPI ? > > Only on a single experimental configuration, and that can apparently > just be fixed going forward without any real problems being caused. BTW, I once tried to describe the current situation on QEMU x86 with IOMMU enabled via ACPI. While you can easily add IOMMU device exceptions to the static tables, the fun starts when considering device hotplug for virtio. Unless I missed some trick, ACPI doesn't seem like being designed for that level of flexibility. You would have to reserve a complete PCI bus, declare that one as not being IOMMU-governed, and then only add new virtio devices to that bus. Possible, but a lot of restrictions that existing management software would have to be aware of as well. Jan -- Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux