From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6D52C432BE for ; Wed, 4 Aug 2021 05:02:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BA4C360F14 for ; Wed, 4 Aug 2021 05:02:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234982AbhHDFC1 (ORCPT ); Wed, 4 Aug 2021 01:02:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38700 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234937AbhHDFC0 (ORCPT ); Wed, 4 Aug 2021 01:02:26 -0400 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5BE72C061798 for ; Tue, 3 Aug 2021 22:02:14 -0700 (PDT) Received: by mail-ed1-x529.google.com with SMTP id z11so1875535edb.11 for ; Tue, 03 Aug 2021 22:02:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=N+beALY95MrjF7EQlWQ+7qrLEnnewleV96sjHNxuEQs=; b=1rLCLS4VnBkloqKEKCZnZytWQey60VsL3htB4tjjDdGtlLAiqoh4sNKJRTSD6PwHMR 9vnNHw1iSS/Fx6i0RCE4RQQ4vyqq5rtIc36Im9QY2RqHEKf/w7Ug3oZhwlGcXQQVWu+G DaqQtL3jX0bh/ha8PFkv/P+nkTaCYS0lRvDdZ7eTBoSHDgR6igcI7RQpNA/OnHviNAOp B3a2lMoBgB2yx2PIRIb70LzX8dilNPja4adYWApEFw1rdfLmQVuZMJXeYvIXWxCdUk2b bPhkn+mywOFmHQT2BPGuByMD4SU6R+J4yLdJWCNVm7lh1t2vV6tsAG7vZpgywjnOueBE IJ+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=N+beALY95MrjF7EQlWQ+7qrLEnnewleV96sjHNxuEQs=; b=J2oin2p1r6WukfaBoqVtjVTeQmi3CueMyF5gMB/wBURZVX/LwzUIPFcOIbVVef5Vet 3cvejbvFbh/ju6MOik2Ic4f3ErinF9/FsP+s8TjS1olFMpCiu70Q8q+Mlr+p0zqHNQCl ryRypEd7T1sSKR16Qade4yQqZjC8vyGV+7tmia1Cc4lbVxI8L9tp800VdmKzATapo3lk pwmCbV3BOCwF0s2g8S1EIeQs6W4xYVKDcaVnkjEle3if3o/vlX7yD+LhIPsoqQU3DQJt 9jIiSb42xWdMW8qL8F7UQxVbCSCS5Tqev7hlefs4Hu3eLAFMXF80hGwZiy8nXskX6mqR 0THg== X-Gm-Message-State: AOAM530/uRmglW7t1AhUjOJWlPsYVPbOlXpwQWuje6npHfK1BP5VS21X DY51xYRGF8W19Jt118Im9LUXi5Ob056wCjBu/UHV X-Google-Smtp-Source: ABdhPJyt1k/RSWy83+1+OmhkaAmWeFnxG6MZmwR66lsbk5H/EN4omlE0NKvZ4kUYOjDCLtkcwCUQ7smkV99GEEOO3As= X-Received: by 2002:aa7:c50a:: with SMTP id o10mr29218603edq.118.1628053332808; Tue, 03 Aug 2021 22:02:12 -0700 (PDT) MIME-Version: 1.0 References: <20210729073503.187-1-xieyongji@bytedance.com> <20210729073503.187-2-xieyongji@bytedance.com> <43d88942-1cd3-c840-6fec-4155fd544d80@redhat.com> <6e05e25e-e569-402e-d81b-8ac2cff1c0e8@arm.com> In-Reply-To: <6e05e25e-e569-402e-d81b-8ac2cff1c0e8@arm.com> From: Yongji Xie Date: Wed, 4 Aug 2021 13:02:01 +0800 Message-ID: Subject: Re: [PATCH v10 01/17] iova: Export alloc_iova_fast() and free_iova_fast() To: Robin Murphy Cc: Jason Wang , kvm , "Michael S. Tsirkin" , virtualization , Christian Brauner , Jonathan Corbet , Matthew Wilcox , Christoph Hellwig , Dan Carpenter , Stefano Garzarella , Liu Xiaodong , linux-fsdevel@vger.kernel.org, Al Viro , Stefan Hajnoczi , songmuchun@bytedance.com, Jens Axboe , He Zhe , Greg KH , Randy Dunlap , linux-kernel , iommu@lists.linux-foundation.org, bcrl@kvack.org, netdev@vger.kernel.org, Joe Perches , =?UTF-8?Q?Mika_Penttil=C3=A4?= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Tue, Aug 3, 2021 at 6:54 PM Robin Murphy wrote: > > On 2021-08-03 09:54, Yongji Xie wrote: > > On Tue, Aug 3, 2021 at 3:41 PM Jason Wang wrote: > >> > >> > >> =E5=9C=A8 2021/7/29 =E4=B8=8B=E5=8D=883:34, Xie Yongji =E5=86=99=E9=81= =93: > >>> Export alloc_iova_fast() and free_iova_fast() so that > >>> some modules can use it to improve iova allocation efficiency. > >> > >> > >> It's better to explain why alloc_iova() is not sufficient here. > >> > > > > Fine. > > What I fail to understand from the later patches is what the IOVA domain > actually represents. If the "device" is a userspace process then > logically the "IOVA" would be the userspace address, so presumably > somewhere you're having to translate between this arbitrary address > space and actual usable addresses - if you're worried about efficiency > surely it would be even better to not do that? > Yes, userspace daemon needs to translate the "IOVA" in a DMA descriptor to the VA (from mmap(2)). But this actually doesn't affect performance since it's an identical mapping in most cases. > Presumably userspace doesn't have any concern about alignment and the > things we have to worry about for the DMA API in general, so it's pretty > much just allocating slots in a buffer, and there are far more effective > ways to do that than a full-blown address space manager. Considering iova allocation efficiency, I think the iova allocator is better here. In most cases, we don't even need to hold a spin lock during iova allocation. > If you're going > to reuse any infrastructure I'd have expected it to be SWIOTLB rather > than the IOVA allocator. Because, y'know, you're *literally implementing > a software I/O TLB* ;) > But actually what we can reuse in SWIOTLB is the IOVA allocator. And the IOVA management in SWIOTLB is not what we want. For example, SWIOTLB allocates and uses contiguous memory for bouncing, which is not necessary in VDUSE case. And VDUSE needs coherent mapping which is not supported by the SWIOTLB. Besides, the SWIOTLB works in singleton mode (designed for platform IOMMU) , but VDUSE is based on on-chip IOMMU (supports multiple instances). So I still prefer to reuse the IOVA allocator to implement a MMU-based software IOTLB. Thanks, Yongji