From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03A80C433F5 for ; Mon, 13 Sep 2021 20:49:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D67C1610E7 for ; Mon, 13 Sep 2021 20:49:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243505AbhIMUuY (ORCPT ); Mon, 13 Sep 2021 16:50:24 -0400 Received: from mail.kernel.org ([198.145.29.99]:39372 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242743AbhIMUuV (ORCPT ); Mon, 13 Sep 2021 16:50:21 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 6463A610FE; Mon, 13 Sep 2021 20:49:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1631566145; bh=hUiqmRGtEXepWEODLRwwvcp5j41ZXDQIlBCemxiVKNw=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=DHwsMX7ikvWdd/BI8zDeHuw+7Tapq9FDRv/8U/Wc5GJhsdDEr0PZBkADcNC3Cw4/l XVIYVS3fpf4IB/pEf/TBDYctfkgqE5TknKGwK5w28r1T0PfvZgGxfh5u4emxheYskE tfJGnP6f/saAr4DBa9u2FuM//73eD3+3Ma38STWt9N64jckrDfpLuZsst7TasKtIXx vFlplwL562zn3HK9rwxFYC1y4bQe7btGfdnMb5FDi/TilVePV2YX60eoSVjbN+C9Oj eGh0ylJCU9jw2iP9ocX1Qp0BMnuv15otYWsJrZYReV/tsLjmDecY0Nc2ev8c2d0QaN wqK6HMy1fcSRg== Date: Mon, 13 Sep 2021 13:49:04 -0700 (PDT) From: Stefano Stabellini X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s To: Jan Beulich cc: Stefano Stabellini , Juergen Gross , Boris Ostrovsky , lkml , "xen-devel@lists.xenproject.org" , the arch/x86 maintainers , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Christoph Hellwig Subject: Re: [PATCH 12/12] swiotlb-xen: this is PV-only on x86 In-Reply-To: Message-ID: References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <004feaef-f3bb-e4bb-fb10-f205a9f69f28@suse.com> User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 13 Sep 2021, Jan Beulich wrote: > On 11.09.2021 01:48, Stefano Stabellini wrote: > > On Wed, 8 Sep 2021, Christoph Hellwig wrote: > >> On Tue, Sep 07, 2021 at 02:13:21PM +0200, Jan Beulich wrote: > >>> The code is unreachable for HVM or PVH, and it also makes little sense > >>> in auto-translated environments. On Arm, with > >>> xen_{create,destroy}_contiguous_region() both being stubs, I have a hard > >>> time seeing what good the Xen specific variant does - the generic one > >>> ought to be fine for all purposes there. Still Arm code explicitly > >>> references symbols here, so the code will continue to be included there. > >> > >> Can the Xen/arm folks look into that? Getting ARM out of using > >> swiotlb-xen would be a huge step forward cleaning up some DMA APIs. > > > > On ARM swiotlb-xen is used for a different purpose compared to x86. > > > > Many ARM SoCs still don't have an IOMMU covering all DMA-mastering > > devices (e.g. Raspberry Pi 4). As a consequence we map Dom0 1:1 (guest > > physical == physical address). > > > > Now if it was just for Dom0, thanks to the 1:1 mapping, we wouldn't need > > swiotlb-xen. But when we start using PV drivers to share the network or > > disk between Dom0 and DomU we are going to get DomU pages mapped in > > Dom0, we call them "foreign pages". They are not mapped 1:1. It can > > happen that one of these foreign pages are used for DMA operations > > (e.g. related to the NIC). swiotlb-xen is used to detect these > > situations and translate the guest physical address to physical address > > of foreign pages appropriately. > > Thinking about this some more - if Dom0 is 1:1 mapped, why don't you > map foreign pages 1:1 as well then? That's because the foreign page, from Linux POV, would appear out of thin air. It would just show up in a region not considered memory just few moments before, so there would be no memblock, no struct page, nothing. At least in the past that caused serious issues to the kernel. This is the reason why the kernel is using ballooned-out pages to map foreign pages even on x86: drivers/block/xen-blkback/blkback.c:xen_blkbk_map -> gnttab_page_cache_get drivers/xen/grant-table.c:gnttab_page_cache_get drivers/xen/grant-table.c:gnttab_alloc_pages drivers/xen/unpopulated-alloc.c:xen_alloc_unpopulated_pages drivers/xen/balloon.c:alloc_xenballooned_pages From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92D92C433F5 for ; Mon, 13 Sep 2021 20:49:21 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 57F46610F9 for ; Mon, 13 Sep 2021 20:49:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 57F46610F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.185924.334677 (Exim 4.92) (envelope-from ) id 1mPst2-00005B-5z; Mon, 13 Sep 2021 20:49:08 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 185924.334677; Mon, 13 Sep 2021 20:49:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mPst2-000050-2r; Mon, 13 Sep 2021 20:49:08 +0000 Received: by outflank-mailman (input) for mailman id 185924; Mon, 13 Sep 2021 20:49:07 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mPst1-00004u-Ix for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 20:49:07 +0000 Received: from mail.kernel.org (unknown [198.145.29.99]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 092b0662-14d4-11ec-b3cc-12813bfff9fa; Mon, 13 Sep 2021 20:49:06 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6463A610FE; Mon, 13 Sep 2021 20:49:05 +0000 (UTC) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 092b0662-14d4-11ec-b3cc-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1631566145; bh=hUiqmRGtEXepWEODLRwwvcp5j41ZXDQIlBCemxiVKNw=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=DHwsMX7ikvWdd/BI8zDeHuw+7Tapq9FDRv/8U/Wc5GJhsdDEr0PZBkADcNC3Cw4/l XVIYVS3fpf4IB/pEf/TBDYctfkgqE5TknKGwK5w28r1T0PfvZgGxfh5u4emxheYskE tfJGnP6f/saAr4DBa9u2FuM//73eD3+3Ma38STWt9N64jckrDfpLuZsst7TasKtIXx vFlplwL562zn3HK9rwxFYC1y4bQe7btGfdnMb5FDi/TilVePV2YX60eoSVjbN+C9Oj eGh0ylJCU9jw2iP9ocX1Qp0BMnuv15otYWsJrZYReV/tsLjmDecY0Nc2ev8c2d0QaN wqK6HMy1fcSRg== Date: Mon, 13 Sep 2021 13:49:04 -0700 (PDT) From: Stefano Stabellini X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s To: Jan Beulich cc: Stefano Stabellini , Juergen Gross , Boris Ostrovsky , lkml , "xen-devel@lists.xenproject.org" , the arch/x86 maintainers , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Christoph Hellwig Subject: Re: [PATCH 12/12] swiotlb-xen: this is PV-only on x86 In-Reply-To: Message-ID: References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <004feaef-f3bb-e4bb-fb10-f205a9f69f28@suse.com> User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII On Mon, 13 Sep 2021, Jan Beulich wrote: > On 11.09.2021 01:48, Stefano Stabellini wrote: > > On Wed, 8 Sep 2021, Christoph Hellwig wrote: > >> On Tue, Sep 07, 2021 at 02:13:21PM +0200, Jan Beulich wrote: > >>> The code is unreachable for HVM or PVH, and it also makes little sense > >>> in auto-translated environments. On Arm, with > >>> xen_{create,destroy}_contiguous_region() both being stubs, I have a hard > >>> time seeing what good the Xen specific variant does - the generic one > >>> ought to be fine for all purposes there. Still Arm code explicitly > >>> references symbols here, so the code will continue to be included there. > >> > >> Can the Xen/arm folks look into that? Getting ARM out of using > >> swiotlb-xen would be a huge step forward cleaning up some DMA APIs. > > > > On ARM swiotlb-xen is used for a different purpose compared to x86. > > > > Many ARM SoCs still don't have an IOMMU covering all DMA-mastering > > devices (e.g. Raspberry Pi 4). As a consequence we map Dom0 1:1 (guest > > physical == physical address). > > > > Now if it was just for Dom0, thanks to the 1:1 mapping, we wouldn't need > > swiotlb-xen. But when we start using PV drivers to share the network or > > disk between Dom0 and DomU we are going to get DomU pages mapped in > > Dom0, we call them "foreign pages". They are not mapped 1:1. It can > > happen that one of these foreign pages are used for DMA operations > > (e.g. related to the NIC). swiotlb-xen is used to detect these > > situations and translate the guest physical address to physical address > > of foreign pages appropriately. > > Thinking about this some more - if Dom0 is 1:1 mapped, why don't you > map foreign pages 1:1 as well then? That's because the foreign page, from Linux POV, would appear out of thin air. It would just show up in a region not considered memory just few moments before, so there would be no memblock, no struct page, nothing. At least in the past that caused serious issues to the kernel. This is the reason why the kernel is using ballooned-out pages to map foreign pages even on x86: drivers/block/xen-blkback/blkback.c:xen_blkbk_map -> gnttab_page_cache_get drivers/xen/grant-table.c:gnttab_page_cache_get drivers/xen/grant-table.c:gnttab_alloc_pages drivers/xen/unpopulated-alloc.c:xen_alloc_unpopulated_pages drivers/xen/balloon.c:alloc_xenballooned_pages