From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73031C433ED for ; Wed, 12 May 2021 17:29:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C835F61418 for ; Wed, 12 May 2021 17:29:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C835F61418 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DAAE56B006C; Wed, 12 May 2021 13:29:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D5C1D6B006E; Wed, 12 May 2021 13:29:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD5CD6B0070; Wed, 12 May 2021 13:29:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0216.hostedemail.com [216.40.44.216]) by kanga.kvack.org (Postfix) with ESMTP id 8B4176B006C for ; Wed, 12 May 2021 13:29:42 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 1AD04181AEF30 for ; Wed, 12 May 2021 17:29:42 +0000 (UTC) X-FDA: 78133266204.01.53ACC28 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf27.hostedemail.com (Postfix) with ESMTP id 38C2C80192E4 for ; Wed, 12 May 2021 17:29:40 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3F2BB31B; Wed, 12 May 2021 10:29:40 -0700 (PDT) Received: from [192.168.1.123] (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 37C463F719; Wed, 12 May 2021 10:29:35 -0700 (PDT) Subject: Re: [Resend RFC PATCH V2 10/12] HV/IOMMU: Add Hyper-V dma ops support To: Tianyu Lan , Christoph Hellwig , konrad.wilk@oracle.com Cc: kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, wei.liu@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, akpm@linux-foundation.org, gregkh@linuxfoundation.org, m.szyprowski@samsung.com, joro@8bytes.org, will@kernel.org, davem@davemloft.net, kuba@kernel.org, jejb@linux.ibm.com, martin.petersen@oracle.com, Tianyu Lan , iommu@lists.linux-foundation.org, linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-scsi@vger.kernel.org, netdev@vger.kernel.org, vkuznets@redhat.com, thomas.lendacky@amd.com, brijesh.singh@amd.com, sunilmut@microsoft.com References: <20210414144945.3460554-1-ltykernel@gmail.com> <20210414144945.3460554-11-ltykernel@gmail.com> <20210414154729.GD32045@lst.de> From: Robin Murphy Message-ID: <4793ae97-5e29-4a45-d2b6-61c151322065@arm.com> Date: Wed, 12 May 2021 18:29:29 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB X-Rspamd-Queue-Id: 38C2C80192E4 Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf27.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com X-Rspamd-Server: rspam04 X-Stat-Signature: x94xhgq8zkybp8ackp5k1yrz4gd8g9ar Received-SPF: none (arm.com>: No applicable sender policy available) receiver=imf27; identity=mailfrom; envelope-from=""; helo=foss.arm.com; client-ip=217.140.110.172 X-HE-DKIM-Result: none/none X-HE-Tag: 1620840580-575357 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2021-05-12 17:01, Tianyu Lan wrote: > Hi Christoph and Konrad: > =C2=A0=C2=A0=C2=A0=C2=A0 Current Swiotlb bounce buffer uses a pool for= all devices. There > is a high overhead to get or free bounce buffer during performance test= . > Swiotlb code now use a global spin lock to protect bounce buffer data. > Several device queues try to acquire the spin lock and this introduce > additional overhead. >=20 > For performance and security perspective, each devices should have a > separate swiotlb bounce buffer pool and so this part needs to rework. > I want to check this is right way to resolve performance issues with=20 > swiotlb bounce buffer. If you have some other suggestions,welcome. We're already well on the way to factoring out SWIOTLB to allow for just=20 this sort of more flexible usage like per-device bounce pools - see here: https://lore.kernel.org/linux-iommu/20210510095026.3477496-1-tientzu@chro= mium.org/T/#t FWIW this looks to have an awful lot in common with what's going to be=20 needed for Android's protected KVM and Arm's Confidential Compute=20 Architecture, so we'll all be better off by doing it right. I'm getting=20 the feeling that set_memory_decrypted() wants to grow into a more=20 general abstraction that can return an alias at a different GPA if=20 necessary. Robin. >=20 > Thanks. >=20 > On 4/14/2021 11:47 PM, Christoph Hellwig wrote: >>> +static dma_addr_t hyperv_map_page(struct device *dev, struct page=20 >>> *page, >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 unsigned long offset, size_t size, >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 enum dma_data_direction dir, >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 unsigned long attrs) >>> +{ >>> +=C2=A0=C2=A0=C2=A0 phys_addr_t map, phys =3D (page_to_pfn(page) << P= AGE_SHIFT) + offset; >>> + >>> +=C2=A0=C2=A0=C2=A0 if (!hv_is_isolation_supported()) >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return phys; >>> + >>> +=C2=A0=C2=A0=C2=A0 map =3D swiotlb_tbl_map_single(dev, phys, size, H= V_HYP_PAGE_SIZE,=20 >>> dir, >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 attrs); >>> +=C2=A0=C2=A0=C2=A0 if (map =3D=3D (phys_addr_t)DMA_MAPPING_ERROR) >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return DMA_MAPPING_ERROR; >>> + >>> +=C2=A0=C2=A0=C2=A0 return map; >>> +} >> >> This largerly duplicates what dma-direct + swiotlb does.=C2=A0 Please = use >> force_dma_unencrypted to force bounce buffering and just use the gener= ic >> code. >> >>> +=C2=A0=C2=A0=C2=A0 if (hv_isolation_type_snp()) { >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ret =3D hv_set_mem_host_v= isibility( >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 phys_to_virt(hyperv_io_tlb_start), >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 hyperv_io_tlb_size, >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 VMBUS_PAGE_VISIBLE_READ_WRITE); >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (ret) >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 p= anic("%s: Fail to mark Hyper-v swiotlb buffer visible=20 >>> to host. err=3D%d\n", >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __func__, ret); >>> + >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hyperv_io_tlb_remap =3D i= oremap_cache(hyperv_io_tlb_start >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 + m= s_hyperv.shared_gpa_boundary, >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 hyperv_io_tlb_size); >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (!hyperv_io_tlb_remap) >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 p= anic("%s: Fail to remap io tlb.\n", __func__); >>> + >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 memset(hyperv_io_tlb_rema= p, 0x00, hyperv_io_tlb_size); >>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 swiotlb_set_bounce_remap(= hyperv_io_tlb_remap); >> >> And this really needs to go into a common hook where we currently just >> call set_memory_decrypted so that all the different schemes for these >> trusted VMs (we have about half a dozen now) can share code rather tha= n >> reinventing it. >>