From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF5D0C11F67 for ; Tue, 29 Jun 2021 06:41:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C2FC361D6C for ; Tue, 29 Jun 2021 06:41:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232183AbhF2Gnm (ORCPT ); Tue, 29 Jun 2021 02:43:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54134 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231958AbhF2Gnj (ORCPT ); Tue, 29 Jun 2021 02:43:39 -0400 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 65390C061767 for ; Mon, 28 Jun 2021 23:41:11 -0700 (PDT) Received: by mail-ed1-x533.google.com with SMTP id i5so29786642eds.1 for ; Mon, 28 Jun 2021 23:41:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=llkKHmBkfnr2e7nGu5wo9NWF1Rvx5VVNTXNZeIaOEY8=; b=b6IxjHxgrsOeV79Ht0Ax4/VA7XYqaQCz8mEI89nlODOpkbbNQKkT86lVUMKt1PlsZi aO9kArwDvjiDefLm8Gqo0xUt1Mk7mf97PWF41AYIIn47zWADydgm2jEpt5HbuvJNgu5D ntPRGt8AK87ZDLLlBie2K5g/5+KlMcWQeGjcg4WsoHO2ksy66nVpAOJnvc90t5QRYxsZ lmYM79ne39YAIfMIugI5AFk+AfwA9H9Bbs5SlIVioXOv0ITHb/z3Mum0jibvHXkilmpY vBesh85ghF0T5N8Mk3dWMoelFiG83Ubtv9E3WAB1/u1Mox5/MvGtN3SeVQ5/0Ut6/dR8 OyRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=llkKHmBkfnr2e7nGu5wo9NWF1Rvx5VVNTXNZeIaOEY8=; b=VZwLlI/uFheDLlpaYkGBq2qC/6XHYf5GwQf1YEpoLwASPuLaPHIbKma5vuMB5j1fzK Qogn0MAjWkRPhAkbmwe219JY/6dWoSkiHDGo/HOjqu8CNINkfx956zFqPsqVUjleoDOJ XUTXQ5kWZnwndHkoD7t5UoM2p0FNGQk6/aN1NJphNXLtbfjptoaseTu+58VebaOHb3Zl RavetJceqPspK2CjbMhu1ZIKab/Iso8R66NzU3MEk51iF23M4rZKiDy8TUJ1Xi4w0AqI mQpqsdjjx+FGXyG8/ilFEbBm82bQmqG8deztGyBmbY6qVDvTv3c8w7JhikgTpcABNcBl o5wg== X-Gm-Message-State: AOAM530wJ/sWkX+BnPykvz9crkQCZ2+WQCgK0ufrSSYUPcDxdyH9/j4l Da2PcMb2wuu0mb/srIAVg7cmt3yVtYz9vvHWzPL+ X-Google-Smtp-Source: ABdhPJztCqFdUxDVPRMjQB4Q3BvYvtgtIsSUG1gu1hpU8fUwWsabUxu+tfnt4bzd/O6vpsvz+4mWTWvqMD3uAaEQxFk= X-Received: by 2002:a50:ff01:: with SMTP id a1mr37794534edu.253.1624948869940; Mon, 28 Jun 2021 23:41:09 -0700 (PDT) MIME-Version: 1.0 References: <20210615141331.407-1-xieyongji@bytedance.com> <20210628103309.GA205554@storage2.sh.intel.com> In-Reply-To: From: Yongji Xie Date: Tue, 29 Jun 2021 14:40:59 +0800 Message-ID: Subject: Re: Re: [PATCH v8 00/10] Introduce VDUSE - vDPA Device in Userspace To: Jason Wang Cc: Yongji Xie , Liu Xiaodong , "Michael S. Tsirkin" , Stefan Hajnoczi , Stefano Garzarella , Parav Pandit , Christoph Hellwig , Christian Brauner , Randy Dunlap , Matthew Wilcox , Al Viro , Jens Axboe , bcrl@kvack.org, Jonathan Corbet , =?UTF-8?Q?Mika_Penttil=C3=A4?= , Dan Carpenter , joro@8bytes.org, Greg KH , kvm , netdev@vger.kernel.org, linux-kernel , virtualization , iommu@lists.linux-foundation.org, songmuchun@bytedance.com, linux-fsdevel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jun 29, 2021 at 12:13 PM Jason Wang wrote: > > > =E5=9C=A8 2021/6/28 =E4=B8=8B=E5=8D=886:32, Yongji Xie =E5=86=99=E9=81=93= : > >> The large barrier is bounce-buffer mapping: SPDK requires hugepages > >> for NVMe over PCIe and RDMA, so take some preallcoated hugepages to > >> map as bounce buffer is necessary. Or it's hard to avoid an extra > >> memcpy from bounce-buffer to hugepage. > >> If you can add an option to map hugepages as bounce-buffer, > >> then SPDK could also be a potential user of vduse. > >> > > I think we can support registering user space memory for bounce-buffer > > use like XDP does. But this needs to pin the pages, so I didn't > > consider it in this initial version. > > > > Note that userspace should be unaware of the existence of the bounce buff= er. > If so, it might be hard to use umem. Because we can't use umem for coherent mapping which needs physical address contiguous space. Thanks, Yongji From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48421C11F68 for ; Tue, 29 Jun 2021 06:41:18 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C197361DC5 for ; Tue, 29 Jun 2021 06:41:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C197361DC5 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 895AB83443; Tue, 29 Jun 2021 06:41:17 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id z4Iqyp91ZAhk; Tue, 29 Jun 2021 06:41:16 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id A937C82B51; Tue, 29 Jun 2021 06:41:15 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 647E3C001A; Tue, 29 Jun 2021 06:41:15 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 66975C000E for ; Tue, 29 Jun 2021 06:41:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 507C3607B4 for ; Tue, 29 Jun 2021 06:41:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=bytedance-com.20150623.gappssmtp.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3defO0DsuO8B for ; Tue, 29 Jun 2021 06:41:12 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by smtp3.osuosl.org (Postfix) with ESMTPS id DA9436078F for ; Tue, 29 Jun 2021 06:41:11 +0000 (UTC) Received: by mail-ed1-x52e.google.com with SMTP id n25so3210986edw.9 for ; Mon, 28 Jun 2021 23:41:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=llkKHmBkfnr2e7nGu5wo9NWF1Rvx5VVNTXNZeIaOEY8=; b=b6IxjHxgrsOeV79Ht0Ax4/VA7XYqaQCz8mEI89nlODOpkbbNQKkT86lVUMKt1PlsZi aO9kArwDvjiDefLm8Gqo0xUt1Mk7mf97PWF41AYIIn47zWADydgm2jEpt5HbuvJNgu5D ntPRGt8AK87ZDLLlBie2K5g/5+KlMcWQeGjcg4WsoHO2ksy66nVpAOJnvc90t5QRYxsZ lmYM79ne39YAIfMIugI5AFk+AfwA9H9Bbs5SlIVioXOv0ITHb/z3Mum0jibvHXkilmpY vBesh85ghF0T5N8Mk3dWMoelFiG83Ubtv9E3WAB1/u1Mox5/MvGtN3SeVQ5/0Ut6/dR8 OyRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=llkKHmBkfnr2e7nGu5wo9NWF1Rvx5VVNTXNZeIaOEY8=; b=rcxPNBJVl55VrCfOxYhVaRblJx9YfhOMzzPmLbC/c5/Ac1IM8kawsOoUow9iYXT3gx qlSPBE8JeKBRGiheVoMlCNZf3lai/GrDxl8CHPGNdNW5CjybPoowWUyfzt3dvbGSrs1O cJv7eqHDQEGptl4iWpADNdbf9BXX3vnZvSAndI7PzG9pr6a3e3RnlzMu2IhWZyUGSA6p KDS3DEq9X7wfZPswoCP8g7ckWmE+Nc3fWr3lSsXlkytj0E+v/pFcx00fToCUzq7jTSOz d0QRGuc2Zlj3YCx5twVUyu0M50TImvhDeto0W5nPpS+Ki2H7nbv1jSDatETLojGCrXpx hVnQ== X-Gm-Message-State: AOAM531nuuAdPKmyUR2PysjORzvZZFzZ3X9TBf8b6Ekg9uB3oFcMOhvd KUjaQ62qzNrURsV8o9WEZJ0SN2Rqcnx8wu+BdrkN X-Google-Smtp-Source: ABdhPJztCqFdUxDVPRMjQB4Q3BvYvtgtIsSUG1gu1hpU8fUwWsabUxu+tfnt4bzd/O6vpsvz+4mWTWvqMD3uAaEQxFk= X-Received: by 2002:a50:ff01:: with SMTP id a1mr37794534edu.253.1624948869940; Mon, 28 Jun 2021 23:41:09 -0700 (PDT) MIME-Version: 1.0 References: <20210615141331.407-1-xieyongji@bytedance.com> <20210628103309.GA205554@storage2.sh.intel.com> In-Reply-To: From: Yongji Xie Date: Tue, 29 Jun 2021 14:40:59 +0800 Message-ID: Subject: Re: Re: [PATCH v8 00/10] Introduce VDUSE - vDPA Device in Userspace To: Jason Wang Cc: kvm , "Michael S. Tsirkin" , virtualization , Christian Brauner , Jonathan Corbet , Matthew Wilcox , Christoph Hellwig , Dan Carpenter , Stefano Garzarella , Liu Xiaodong , Yongji Xie , Al Viro , Stefan Hajnoczi , songmuchun@bytedance.com, Jens Axboe , Greg KH , Randy Dunlap , linux-kernel , iommu@lists.linux-foundation.org, bcrl@kvack.org, netdev@vger.kernel.org, linux-fsdevel@vger.kernel.org, =?UTF-8?Q?Mika_Penttil=C3=A4?= X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" T24gVHVlLCBKdW4gMjksIDIwMjEgYXQgMTI6MTMgUE0gSmFzb24gV2FuZyA8amFzb3dhbmdAcmVk aGF0LmNvbT4gd3JvdGU6Cj4KPgo+IOWcqCAyMDIxLzYvMjgg5LiL5Y2INjozMiwgWW9uZ2ppIFhp ZSDlhpnpgZM6Cj4gPj4gVGhlIGxhcmdlIGJhcnJpZXIgaXMgYm91bmNlLWJ1ZmZlciBtYXBwaW5n OiBTUERLIHJlcXVpcmVzIGh1Z2VwYWdlcwo+ID4+IGZvciBOVk1lIG92ZXIgUENJZSBhbmQgUkRN QSwgc28gdGFrZSBzb21lIHByZWFsbGNvYXRlZCBodWdlcGFnZXMgdG8KPiA+PiBtYXAgYXMgYm91 bmNlIGJ1ZmZlciBpcyBuZWNlc3NhcnkuIE9yIGl0J3MgaGFyZCB0byBhdm9pZCBhbiBleHRyYQo+ ID4+IG1lbWNweSBmcm9tIGJvdW5jZS1idWZmZXIgdG8gaHVnZXBhZ2UuCj4gPj4gSWYgeW91IGNh biBhZGQgYW4gb3B0aW9uIHRvIG1hcCBodWdlcGFnZXMgYXMgYm91bmNlLWJ1ZmZlciwKPiA+PiB0 aGVuIFNQREsgY291bGQgYWxzbyBiZSBhIHBvdGVudGlhbCB1c2VyIG9mIHZkdXNlLgo+ID4+Cj4g PiBJIHRoaW5rIHdlIGNhbiBzdXBwb3J0IHJlZ2lzdGVyaW5nIHVzZXIgc3BhY2UgbWVtb3J5IGZv ciBib3VuY2UtYnVmZmVyCj4gPiB1c2UgbGlrZSBYRFAgZG9lcy4gQnV0IHRoaXMgbmVlZHMgdG8g cGluIHRoZSBwYWdlcywgc28gSSBkaWRuJ3QKPiA+IGNvbnNpZGVyIGl0IGluIHRoaXMgaW5pdGlh bCB2ZXJzaW9uLgo+ID4KPgo+IE5vdGUgdGhhdCB1c2Vyc3BhY2Ugc2hvdWxkIGJlIHVuYXdhcmUg b2YgdGhlIGV4aXN0ZW5jZSBvZiB0aGUgYm91bmNlIGJ1ZmZlci4KPgoKSWYgc28sIGl0IG1pZ2h0 IGJlIGhhcmQgdG8gdXNlIHVtZW0uIEJlY2F1c2Ugd2UgY2FuJ3QgdXNlIHVtZW0gZm9yCmNvaGVy ZW50IG1hcHBpbmcgd2hpY2ggbmVlZHMgcGh5c2ljYWwgYWRkcmVzcyBjb250aWd1b3VzIHNwYWNl LgoKVGhhbmtzLApZb25namkKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX18KaW9tbXUgbWFpbGluZyBsaXN0CmlvbW11QGxpc3RzLmxpbnV4LWZvdW5kYXRpb24u b3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9uLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2lv bW11