From: Oleksandr Tyshchenko <olekstysh@gmail.com> To: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, virtualization@lists.linux-foundation.org, x86@kernel.org Cc: "Oleksandr Tyshchenko" <oleksandr_tyshchenko@epam.com>, "Michael S. Tsirkin" <mst@redhat.com>, "Christoph Hellwig" <hch@infradead.org>, "Stefano Stabellini" <sstabellini@kernel.org>, "Boris Ostrovsky" <boris.ostrovsky@oracle.com>, "Juergen Gross" <jgross@suse.com>, "Julien Grall" <julien@xen.org>, "Bertrand Marquis" <bertrand.marquis@arm.com>, "Wei Chen" <Wei.Chen@arm.com>, "Henry Wang" <Henry.Wang@arm.com>, "Kaly Xin" <Kaly.Xin@arm.com>, "Jiamei Xie" <Jiamei.Xie@arm.com>, "Alex Bennée" <alex.bennee@linaro.org> Subject: [PATCH V3 0/8] virtio: Solution to restrict memory access under Xen using xen-grant DMA-mapping layer Date: Tue, 31 May 2022 00:00:09 +0300 [thread overview] Message-ID: <1653944417-17168-1-git-send-email-olekstysh@gmail.com> (raw) From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com> Hello all. The purpose of this patch series is to add support for restricting memory access under Xen using specific grant table [1] based DMA-mapping layer. Patch series is based on Juergen Gross’ initial work [2] which implies using grant references instead of raw guest physical addresses (GPA) for the virtio communications (some kind of the software IOMMU). You can find RFC-V2 patch series (and previous discussions) at [3]. The high level idea is to create new Xen’s grant table based DMA-mapping layer for the guest Linux whose main purpose is to provide a special 64-bit DMA address which is formed by using the grant reference (for a page to be shared with the backend) with offset and setting the highest address bit (this is for the backend to be able to distinguish grant ref based DMA address from normal GPA). For this to work we need the ability to allocate contiguous (consecutive) grant references for multi-page allocations. And the backend then needs to offer VIRTIO_F_ACCESS_PLATFORM and VIRTIO_F_VERSION_1 feature bits (it must support virtio-mmio modern transport for 64-bit addresses in the virtqueue). Xen's grant mapping mechanism is the secure and safe solution to share pages between domains which proven to work and works for years (in the context of traditional Xen PV drivers for example). So far, the foreign mapping is used for the virtio backend to map and access guest memory. With the foreign mapping, the backend is able to map arbitrary pages from the guest memory (or even from Dom0 memory). And as the result, the malicious backend which runs in a non-trusted domain can take advantage of this. Instead, with the grant mapping the backend is only allowed to map pages which were explicitly granted by the guest before and nothing else. According to the discussions in various mainline threads this solution would likely be welcome because it perfectly fits in the security model Xen provides. What is more, the grant table based solution requires zero changes to the Xen hypervisor itself at least with virtio-mmio and DT (in comparison, for example, with "foreign mapping + virtio-iommu" solution which would require the whole new complex emulator in hypervisor in addition to new functionality/hypercall to pass IOVA from the virtio backend running elsewhere to the hypervisor and translate it to the GPA before mapping into P2M or denying the foreign mapping request if no corresponding IOVA-GPA mapping present in the IOMMU page table for that particular device). We only need to update toolstack to insert "xen,grant-dma" IOMMU node (to be referred by the virtio-mmio device using "iommus" property) when creating a guest device-tree (this is an indicator for the guest to use Xen grant mappings scheme for that device with the endpoint ID being used as ID of Xen domain where the corresponding backend is running, the backend domid is used as an argument to the grant mapping APIs). It worth mentioning that toolstack patch is based on non upstreamed yet “Virtio support for toolstack on Arm” series which is on review now [4]. Please note the following: - Patch series only covers Arm and virtio-mmio (device-tree) for now. To enable the restricted memory access feature on Arm the following option should be set: CONFIG_XEN_VIRTIO=y - Patch series is based on "kernel: add new infrastructure for platform_has() support" patch series which is on review now [5] - Xen should be built with the following options: CONFIG_IOREQ_SERVER=y CONFIG_EXPERT=y Patch series is rebased on Linux 5.18-rc7 tag with "kernel: add new infrastructure for platform_has() support" and "xen: simplify frontend side ring setup" series applied and tested on Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with standalone userspace (non-Qemu) virtio-mmio based virtio-disk backend running in Driver domain and Linux guest running on existing virtio-blk driver (frontend). No issues were observed. Guest domain 'reboot/destroy' use-cases work properly. I have also tested other use-cases such as assigning several virtio block devices or a mix of virtio and Xen PV block devices to the guest. Patch series was build-tested on Arm32 and x86. 1. Xen changes located at (last patch): https://github.com/otyshchenko1/xen/commits/libxl_virtio_next2_1 2. Linux changes located at (last 8 patches): https://github.com/otyshchenko1/linux/commits/virtio_grant8_1 3. virtio-disk changes located at: https://github.com/otyshchenko1/virtio-disk/commits/virtio_grant Any feedback/help would be highly appreciated. [1] https://xenbits.xenproject.org/docs/4.16-testing/misc/grant-tables.txt [2] https://www.youtube.com/watch?v=IrlEdaIUDPk [3] https://lore.kernel.org/xen-devel/1649963973-22879-1-git-send-email-olekstysh@gmail.com/ https://lore.kernel.org/xen-devel/1650646263-22047-1-git-send-email-olekstysh@gmail.com/ https://lore.kernel.org/xen-devel/1651947548-4055-1-git-send-email-olekstysh@gmail.com/ [4] https://lore.kernel.org/xen-devel/1651598763-12162-1-git-send-email-olekstysh@gmail.com/ [5] https://lore.kernel.org/xen-devel/20220504155703.13336-1-jgross@suse.com/ Juergen Gross (3): xen/grants: support allocating consecutive grants xen/grant-dma-ops: Add option to restrict memory access under Xen xen/virtio: Enable restricted memory access using Xen grant mappings Oleksandr Tyshchenko (5): arm/xen: Introduce xen_setup_dma_ops() dt-bindings: Add xen,grant-dma IOMMU description for xen-grant DMA ops xen/grant-dma-iommu: Introduce stub IOMMU driver xen/grant-dma-ops: Retrieve the ID of backend's domain for DT devices arm/xen: Assign xen-grant DMA ops for xen-grant DMA devices .../devicetree/bindings/iommu/xen,grant-dma.yaml | 49 +++ arch/arm/include/asm/xen/xen-ops.h | 2 + arch/arm/mm/dma-mapping.c | 7 +- arch/arm/xen/enlighten.c | 2 + arch/arm64/include/asm/xen/xen-ops.h | 2 + arch/arm64/mm/dma-mapping.c | 7 +- arch/x86/xen/enlighten_hvm.c | 2 + arch/x86/xen/enlighten_pv.c | 2 + drivers/xen/Kconfig | 20 ++ drivers/xen/Makefile | 2 + drivers/xen/grant-dma-iommu.c | 78 +++++ drivers/xen/grant-dma-ops.c | 345 +++++++++++++++++++++ drivers/xen/grant-table.c | 251 ++++++++++++--- include/xen/arm/xen-ops.h | 18 ++ include/xen/grant_table.h | 4 + include/xen/xen-ops.h | 13 + include/xen/xen.h | 8 + 17 files changed, 766 insertions(+), 46 deletions(-) create mode 100644 Documentation/devicetree/bindings/iommu/xen,grant-dma.yaml create mode 100644 arch/arm/include/asm/xen/xen-ops.h create mode 100644 arch/arm64/include/asm/xen/xen-ops.h create mode 100644 drivers/xen/grant-dma-iommu.c create mode 100644 drivers/xen/grant-dma-ops.c create mode 100644 include/xen/arm/xen-ops.h -- 2.7.4
WARNING: multiple messages have this Message-ID (diff)
From: Oleksandr Tyshchenko <olekstysh@gmail.com> To: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, virtualization@lists.linux-foundation.org, x86@kernel.org Cc: "Oleksandr Tyshchenko" <oleksandr_tyshchenko@epam.com>, "Michael S. Tsirkin" <mst@redhat.com>, "Christoph Hellwig" <hch@infradead.org>, "Stefano Stabellini" <sstabellini@kernel.org>, "Boris Ostrovsky" <boris.ostrovsky@oracle.com>, "Juergen Gross" <jgross@suse.com>, "Julien Grall" <julien@xen.org>, "Bertrand Marquis" <bertrand.marquis@arm.com>, "Wei Chen" <Wei.Chen@arm.com>, "Henry Wang" <Henry.Wang@arm.com>, "Kaly Xin" <Kaly.Xin@arm.com>, "Jiamei Xie" <Jiamei.Xie@arm.com>, "Alex Bennée" <alex.bennee@linaro.org> Subject: [PATCH V3 0/8] virtio: Solution to restrict memory access under Xen using xen-grant DMA-mapping layer Date: Tue, 31 May 2022 00:00:09 +0300 [thread overview] Message-ID: <1653944417-17168-1-git-send-email-olekstysh@gmail.com> (raw) From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com> Hello all. The purpose of this patch series is to add support for restricting memory access under Xen using specific grant table [1] based DMA-mapping layer. Patch series is based on Juergen Gross’ initial work [2] which implies using grant references instead of raw guest physical addresses (GPA) for the virtio communications (some kind of the software IOMMU). You can find RFC-V2 patch series (and previous discussions) at [3]. The high level idea is to create new Xen’s grant table based DMA-mapping layer for the guest Linux whose main purpose is to provide a special 64-bit DMA address which is formed by using the grant reference (for a page to be shared with the backend) with offset and setting the highest address bit (this is for the backend to be able to distinguish grant ref based DMA address from normal GPA). For this to work we need the ability to allocate contiguous (consecutive) grant references for multi-page allocations. And the backend then needs to offer VIRTIO_F_ACCESS_PLATFORM and VIRTIO_F_VERSION_1 feature bits (it must support virtio-mmio modern transport for 64-bit addresses in the virtqueue). Xen's grant mapping mechanism is the secure and safe solution to share pages between domains which proven to work and works for years (in the context of traditional Xen PV drivers for example). So far, the foreign mapping is used for the virtio backend to map and access guest memory. With the foreign mapping, the backend is able to map arbitrary pages from the guest memory (or even from Dom0 memory). And as the result, the malicious backend which runs in a non-trusted domain can take advantage of this. Instead, with the grant mapping the backend is only allowed to map pages which were explicitly granted by the guest before and nothing else. According to the discussions in various mainline threads this solution would likely be welcome because it perfectly fits in the security model Xen provides. What is more, the grant table based solution requires zero changes to the Xen hypervisor itself at least with virtio-mmio and DT (in comparison, for example, with "foreign mapping + virtio-iommu" solution which would require the whole new complex emulator in hypervisor in addition to new functionality/hypercall to pass IOVA from the virtio backend running elsewhere to the hypervisor and translate it to the GPA before mapping into P2M or denying the foreign mapping request if no corresponding IOVA-GPA mapping present in the IOMMU page table for that particular device). We only need to update toolstack to insert "xen,grant-dma" IOMMU node (to be referred by the virtio-mmio device using "iommus" property) when creating a guest device-tree (this is an indicator for the guest to use Xen grant mappings scheme for that device with the endpoint ID being used as ID of Xen domain where the corresponding backend is running, the backend domid is used as an argument to the grant mapping APIs). It worth mentioning that toolstack patch is based on non upstreamed yet “Virtio support for toolstack on Arm” series which is on review now [4]. Please note the following: - Patch series only covers Arm and virtio-mmio (device-tree) for now. To enable the restricted memory access feature on Arm the following option should be set: CONFIG_XEN_VIRTIO=y - Patch series is based on "kernel: add new infrastructure for platform_has() support" patch series which is on review now [5] - Xen should be built with the following options: CONFIG_IOREQ_SERVER=y CONFIG_EXPERT=y Patch series is rebased on Linux 5.18-rc7 tag with "kernel: add new infrastructure for platform_has() support" and "xen: simplify frontend side ring setup" series applied and tested on Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with standalone userspace (non-Qemu) virtio-mmio based virtio-disk backend running in Driver domain and Linux guest running on existing virtio-blk driver (frontend). No issues were observed. Guest domain 'reboot/destroy' use-cases work properly. I have also tested other use-cases such as assigning several virtio block devices or a mix of virtio and Xen PV block devices to the guest. Patch series was build-tested on Arm32 and x86. 1. Xen changes located at (last patch): https://github.com/otyshchenko1/xen/commits/libxl_virtio_next2_1 2. Linux changes located at (last 8 patches): https://github.com/otyshchenko1/linux/commits/virtio_grant8_1 3. virtio-disk changes located at: https://github.com/otyshchenko1/virtio-disk/commits/virtio_grant Any feedback/help would be highly appreciated. [1] https://xenbits.xenproject.org/docs/4.16-testing/misc/grant-tables.txt [2] https://www.youtube.com/watch?v=IrlEdaIUDPk [3] https://lore.kernel.org/xen-devel/1649963973-22879-1-git-send-email-olekstysh@gmail.com/ https://lore.kernel.org/xen-devel/1650646263-22047-1-git-send-email-olekstysh@gmail.com/ https://lore.kernel.org/xen-devel/1651947548-4055-1-git-send-email-olekstysh@gmail.com/ [4] https://lore.kernel.org/xen-devel/1651598763-12162-1-git-send-email-olekstysh@gmail.com/ [5] https://lore.kernel.org/xen-devel/20220504155703.13336-1-jgross@suse.com/ Juergen Gross (3): xen/grants: support allocating consecutive grants xen/grant-dma-ops: Add option to restrict memory access under Xen xen/virtio: Enable restricted memory access using Xen grant mappings Oleksandr Tyshchenko (5): arm/xen: Introduce xen_setup_dma_ops() dt-bindings: Add xen,grant-dma IOMMU description for xen-grant DMA ops xen/grant-dma-iommu: Introduce stub IOMMU driver xen/grant-dma-ops: Retrieve the ID of backend's domain for DT devices arm/xen: Assign xen-grant DMA ops for xen-grant DMA devices .../devicetree/bindings/iommu/xen,grant-dma.yaml | 49 +++ arch/arm/include/asm/xen/xen-ops.h | 2 + arch/arm/mm/dma-mapping.c | 7 +- arch/arm/xen/enlighten.c | 2 + arch/arm64/include/asm/xen/xen-ops.h | 2 + arch/arm64/mm/dma-mapping.c | 7 +- arch/x86/xen/enlighten_hvm.c | 2 + arch/x86/xen/enlighten_pv.c | 2 + drivers/xen/Kconfig | 20 ++ drivers/xen/Makefile | 2 + drivers/xen/grant-dma-iommu.c | 78 +++++ drivers/xen/grant-dma-ops.c | 345 +++++++++++++++++++++ drivers/xen/grant-table.c | 251 ++++++++++++--- include/xen/arm/xen-ops.h | 18 ++ include/xen/grant_table.h | 4 + include/xen/xen-ops.h | 13 + include/xen/xen.h | 8 + 17 files changed, 766 insertions(+), 46 deletions(-) create mode 100644 Documentation/devicetree/bindings/iommu/xen,grant-dma.yaml create mode 100644 arch/arm/include/asm/xen/xen-ops.h create mode 100644 arch/arm64/include/asm/xen/xen-ops.h create mode 100644 drivers/xen/grant-dma-iommu.c create mode 100644 drivers/xen/grant-dma-ops.c create mode 100644 include/xen/arm/xen-ops.h -- 2.7.4 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next reply other threads:[~2022-05-30 21:00 UTC|newest] Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-05-30 21:00 Oleksandr Tyshchenko [this message] 2022-05-30 21:00 ` [PATCH V3 0/8] virtio: Solution to restrict memory access under Xen using xen-grant DMA-mapping layer Oleksandr Tyshchenko 2022-05-30 21:00 ` [PATCH V3 1/8] arm/xen: Introduce xen_setup_dma_ops() Oleksandr Tyshchenko 2022-05-30 21:00 ` Oleksandr Tyshchenko 2022-05-30 21:00 ` [PATCH V3 2/8] xen/grants: support allocating consecutive grants Oleksandr Tyshchenko 2022-05-30 21:00 ` Oleksandr Tyshchenko 2022-05-30 21:00 ` [PATCH V3 3/8] xen/grant-dma-ops: Add option to restrict memory access under Xen Oleksandr Tyshchenko 2022-05-30 21:00 ` Oleksandr Tyshchenko 2022-06-01 0:19 ` Stefano Stabellini 2022-06-01 0:19 ` Stefano Stabellini 2022-05-30 21:00 ` [PATCH V3 4/8] xen/virtio: Enable restricted memory access using Xen grant mappings Oleksandr Tyshchenko 2022-05-30 21:00 ` Oleksandr Tyshchenko 2022-06-02 12:49 ` Oleksandr 2022-06-02 12:49 ` Oleksandr 2022-06-02 19:09 ` Boris Ostrovsky 2022-06-02 19:09 ` Boris Ostrovsky 2022-05-30 21:00 ` [PATCH V3 5/8] dt-bindings: Add xen, grant-dma IOMMU description for xen-grant DMA ops Oleksandr Tyshchenko 2022-05-30 21:00 ` [PATCH V3 5/8] dt-bindings: Add xen,grant-dma " Oleksandr Tyshchenko 2022-05-30 21:00 ` Oleksandr Tyshchenko 2022-05-31 11:52 ` Krzysztof Kozlowski 2022-05-31 11:52 ` Krzysztof Kozlowski 2022-05-31 11:52 ` Krzysztof Kozlowski 2022-05-31 17:09 ` Oleksandr 2022-05-31 17:09 ` Oleksandr 2022-05-31 17:09 ` Oleksandr 2022-06-01 0:34 ` Stefano Stabellini 2022-06-01 0:34 ` Stefano Stabellini 2022-06-01 0:34 ` Stefano Stabellini 2022-06-01 13:32 ` Oleksandr 2022-06-01 13:32 ` Oleksandr 2022-06-01 13:32 ` Oleksandr 2022-05-30 21:00 ` [PATCH V3 6/8] xen/grant-dma-iommu: Introduce stub IOMMU driver Oleksandr Tyshchenko 2022-05-30 21:00 ` Oleksandr Tyshchenko 2022-06-01 0:38 ` Stefano Stabellini 2022-06-01 0:38 ` Stefano Stabellini 2022-05-30 21:00 ` [PATCH V3 7/8] xen/grant-dma-ops: Retrieve the ID of backend's domain for DT devices Oleksandr Tyshchenko 2022-05-30 21:00 ` Oleksandr Tyshchenko 2022-06-01 0:53 ` Stefano Stabellini 2022-06-01 0:53 ` Stefano Stabellini 2022-05-30 21:00 ` [PATCH V3 8/8] arm/xen: Assign xen-grant DMA ops for xen-grant DMA devices Oleksandr Tyshchenko 2022-05-30 21:00 ` Oleksandr Tyshchenko
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1653944417-17168-1-git-send-email-olekstysh@gmail.com \ --to=olekstysh@gmail.com \ --cc=Henry.Wang@arm.com \ --cc=Jiamei.Xie@arm.com \ --cc=Kaly.Xin@arm.com \ --cc=Wei.Chen@arm.com \ --cc=alex.bennee@linaro.org \ --cc=bertrand.marquis@arm.com \ --cc=boris.ostrovsky@oracle.com \ --cc=hch@infradead.org \ --cc=jgross@suse.com \ --cc=julien@xen.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mst@redhat.com \ --cc=oleksandr_tyshchenko@epam.com \ --cc=sstabellini@kernel.org \ --cc=virtualization@lists.linux-foundation.org \ --cc=x86@kernel.org \ --cc=xen-devel@lists.xenproject.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.