From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED459C4320A for ; Wed, 4 Aug 2021 05:02:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D0C5060F10 for ; Wed, 4 Aug 2021 05:02:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235067AbhHDFC1 (ORCPT ); Wed, 4 Aug 2021 01:02:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234955AbhHDFC0 (ORCPT ); Wed, 4 Aug 2021 01:02:26 -0400 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B3EFC0613D5 for ; Tue, 3 Aug 2021 22:02:14 -0700 (PDT) Received: by mail-ed1-x52e.google.com with SMTP id b7so1928853edu.3 for ; Tue, 03 Aug 2021 22:02:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=N+beALY95MrjF7EQlWQ+7qrLEnnewleV96sjHNxuEQs=; b=1rLCLS4VnBkloqKEKCZnZytWQey60VsL3htB4tjjDdGtlLAiqoh4sNKJRTSD6PwHMR 9vnNHw1iSS/Fx6i0RCE4RQQ4vyqq5rtIc36Im9QY2RqHEKf/w7Ug3oZhwlGcXQQVWu+G DaqQtL3jX0bh/ha8PFkv/P+nkTaCYS0lRvDdZ7eTBoSHDgR6igcI7RQpNA/OnHviNAOp B3a2lMoBgB2yx2PIRIb70LzX8dilNPja4adYWApEFw1rdfLmQVuZMJXeYvIXWxCdUk2b bPhkn+mywOFmHQT2BPGuByMD4SU6R+J4yLdJWCNVm7lh1t2vV6tsAG7vZpgywjnOueBE IJ+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=N+beALY95MrjF7EQlWQ+7qrLEnnewleV96sjHNxuEQs=; b=brcNfyzTPJbQsqq7roUjU44Uv8eTSJOCFBIkQ7V5wMBu19un9WDsgslLF0uq8MSD37 gSZ+5yjYRkR3BID8QXpfyTEOjroyIX1RPTCHRh5d7cTUMGQcxn0i0YrmdQImxzOyvGvl 3NaD8bM+kaV5ohf6O5JEyLredtoxtglA7nFH107Nbx09wTCwxE/CngPIDuXqot5mV4Vo XEpQJ1w3cSkTOYxmAgSVriPzuXz1Bm6apxD/TtIRa3mCvB/u4lztPkSUyAsvCWIe8izb +slkiq+Y5ohJKYQ1j6xUPjsM1fVnls1+Dh9VdJ8YibSlnoZ2SK8dX1hsK7kNvbFOsAdl zcbQ== X-Gm-Message-State: AOAM533v+TrujyzEJyZSItawi73QScUun5ekh3gN2A5FXwkWPR7GJV1G Bx2ATHg2fL4WQkSD1h7+hlts1V7dchZq/JGX2emB X-Google-Smtp-Source: ABdhPJyt1k/RSWy83+1+OmhkaAmWeFnxG6MZmwR66lsbk5H/EN4omlE0NKvZ4kUYOjDCLtkcwCUQ7smkV99GEEOO3As= X-Received: by 2002:aa7:c50a:: with SMTP id o10mr29218603edq.118.1628053332808; Tue, 03 Aug 2021 22:02:12 -0700 (PDT) MIME-Version: 1.0 References: <20210729073503.187-1-xieyongji@bytedance.com> <20210729073503.187-2-xieyongji@bytedance.com> <43d88942-1cd3-c840-6fec-4155fd544d80@redhat.com> <6e05e25e-e569-402e-d81b-8ac2cff1c0e8@arm.com> In-Reply-To: <6e05e25e-e569-402e-d81b-8ac2cff1c0e8@arm.com> From: Yongji Xie Date: Wed, 4 Aug 2021 13:02:01 +0800 Message-ID: Subject: Re: [PATCH v10 01/17] iova: Export alloc_iova_fast() and free_iova_fast() To: Robin Murphy Cc: Jason Wang , kvm , "Michael S. Tsirkin" , virtualization , Christian Brauner , Jonathan Corbet , Matthew Wilcox , Christoph Hellwig , Dan Carpenter , Stefano Garzarella , Liu Xiaodong , linux-fsdevel@vger.kernel.org, Al Viro , Stefan Hajnoczi , songmuchun@bytedance.com, Jens Axboe , He Zhe , Greg KH , Randy Dunlap , linux-kernel , iommu@lists.linux-foundation.org, bcrl@kvack.org, netdev@vger.kernel.org, Joe Perches , =?UTF-8?Q?Mika_Penttil=C3=A4?= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Aug 3, 2021 at 6:54 PM Robin Murphy wrote: > > On 2021-08-03 09:54, Yongji Xie wrote: > > On Tue, Aug 3, 2021 at 3:41 PM Jason Wang wrote: > >> > >> > >> =E5=9C=A8 2021/7/29 =E4=B8=8B=E5=8D=883:34, Xie Yongji =E5=86=99=E9=81= =93: > >>> Export alloc_iova_fast() and free_iova_fast() so that > >>> some modules can use it to improve iova allocation efficiency. > >> > >> > >> It's better to explain why alloc_iova() is not sufficient here. > >> > > > > Fine. > > What I fail to understand from the later patches is what the IOVA domain > actually represents. If the "device" is a userspace process then > logically the "IOVA" would be the userspace address, so presumably > somewhere you're having to translate between this arbitrary address > space and actual usable addresses - if you're worried about efficiency > surely it would be even better to not do that? > Yes, userspace daemon needs to translate the "IOVA" in a DMA descriptor to the VA (from mmap(2)). But this actually doesn't affect performance since it's an identical mapping in most cases. > Presumably userspace doesn't have any concern about alignment and the > things we have to worry about for the DMA API in general, so it's pretty > much just allocating slots in a buffer, and there are far more effective > ways to do that than a full-blown address space manager. Considering iova allocation efficiency, I think the iova allocator is better here. In most cases, we don't even need to hold a spin lock during iova allocation. > If you're going > to reuse any infrastructure I'd have expected it to be SWIOTLB rather > than the IOVA allocator. Because, y'know, you're *literally implementing > a software I/O TLB* ;) > But actually what we can reuse in SWIOTLB is the IOVA allocator. And the IOVA management in SWIOTLB is not what we want. For example, SWIOTLB allocates and uses contiguous memory for bouncing, which is not necessary in VDUSE case. And VDUSE needs coherent mapping which is not supported by the SWIOTLB. Besides, the SWIOTLB works in singleton mode (designed for platform IOMMU) , but VDUSE is based on on-chip IOMMU (supports multiple instances). So I still prefer to reuse the IOVA allocator to implement a MMU-based software IOTLB. Thanks, Yongji