From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 521E6C11F64 for ; Thu, 1 Jul 2021 08:01:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3115061480 for ; Thu, 1 Jul 2021 08:01:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234895AbhGAIDx (ORCPT ); Thu, 1 Jul 2021 04:03:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:29085 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234635AbhGAIDw (ORCPT ); Thu, 1 Jul 2021 04:03:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625126482; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3kkO+UX+kTqcYTfctkZezZWjIyhRJe4+TF27ND409mQ=; b=J2qV2RZYCbAb7MK5fW02J+Syk+yohBnlzYGZT+HqISyo8gZ0xneDto/vO6UdDIzyka5AWF aTgNRFmMvHJk7gMnI8u8mV3+U1yFNF0kMkq2+wgu5mqstFkV2UFQMflnZJtLARWd2Kzvd3 Qwjj+6FQn242MDHYgi6bgG/Aa9UZU3o= Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-500-RGpIk38SONulxYakwZ68_A-1; Thu, 01 Jul 2021 04:01:21 -0400 X-MC-Unique: RGpIk38SONulxYakwZ68_A-1 Received: by mail-pl1-f200.google.com with SMTP id q2-20020a170902e302b0290128ec775274so2372934plc.20 for ; Thu, 01 Jul 2021 01:01:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=3kkO+UX+kTqcYTfctkZezZWjIyhRJe4+TF27ND409mQ=; b=tSq7lNDo+xR+QzDfEKAMzbdLQIYFpZxf6HoN6lUQwbd8YJlWxOuaKOgkC++/MMcXZo hmrj9r5IXR+7VhQB51h4UDvRiu4iGTVEoB+JFyd25dOIoKNNG9z3K+QMziXpe9w2Ibu0 oS9Foy/zL+SSnMqewZJ/iruHoiYrqbjp0/sKF8FcyVowf4TtgpF9tt3rcupq1JWOKb7J YTdGVRFsgP3SA8vc0I24g5jpY1xY2C81Ti0aENG2mGCxK87fA0jrpTR8dwNfiJx2jDam dfHV+krC5kNpytXAEwNt8/wPfUytlaW6ESUbR+jUtrQELdvpSgqZvMJWI6S8Btkukdya W7fw== X-Gm-Message-State: AOAM531R/c5Wf1ueB6/SkgTyRhEn5K7sDRj84e/uCzqdWmj1ID7Bcdmh MX6DBi84NGvu/SWvBaFglKpF99R643GYj9dTJDEXC2Hv2ShfeK7bCr/fMjV3wtHObjXKDgp+yuj nbpW8g991q19yvTJ9nS98WDR9XYtz9T+s6HhN/Iqg/5tcQiVHkSB3EU4wPHY7SLoqNy+TUzLrhn Fx X-Received: by 2002:a63:794:: with SMTP id 142mr6983334pgh.198.1625126151794; Thu, 01 Jul 2021 00:55:51 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx+FJQkpNj2SGJqmkRE0B+JqnTbGGKE300PtWwx2ZJ5+wT2j2YuZ2VlEIodQZCrR0j2AFzwoQ== X-Received: by 2002:a63:794:: with SMTP id 142mr6983291pgh.198.1625126151412; Thu, 01 Jul 2021 00:55:51 -0700 (PDT) Received: from wangxiaodeMacBook-Air.local ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id a6sm23966070pfo.212.2021.07.01.00.55.42 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 01 Jul 2021 00:55:50 -0700 (PDT) Subject: Re: [PATCH v8 09/10] vduse: Introduce VDUSE - vDPA Device in Userspace To: Yongji Xie , Stefan Hajnoczi Cc: "Michael S. Tsirkin" , Stefano Garzarella , Parav Pandit , Christoph Hellwig , Christian Brauner , Randy Dunlap , Matthew Wilcox , Al Viro , Jens Axboe , bcrl@kvack.org, Jonathan Corbet , =?UTF-8?Q?Mika_Penttil=c3=a4?= , Dan Carpenter , joro@8bytes.org, Greg KH , songmuchun@bytedance.com, virtualization , netdev@vger.kernel.org, kvm , linux-fsdevel@vger.kernel.org, iommu@lists.linux-foundation.org, linux-kernel References: <20210615141331.407-1-xieyongji@bytedance.com> <20210615141331.407-10-xieyongji@bytedance.com> From: Jason Wang Message-ID: <7264cb0b-7072-098e-3d22-2b7e89216545@redhat.com> Date: Thu, 1 Jul 2021 15:55:36 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 在 2021/7/1 下午2:50, Yongji Xie 写道: > On Wed, Jun 30, 2021 at 5:51 PM Stefan Hajnoczi wrote: >> On Tue, Jun 29, 2021 at 10:59:51AM +0800, Yongji Xie wrote: >>> On Mon, Jun 28, 2021 at 9:02 PM Stefan Hajnoczi wrote: >>>> On Tue, Jun 15, 2021 at 10:13:30PM +0800, Xie Yongji wrote: >>>>> +/* ioctls */ >>>>> + >>>>> +struct vduse_dev_config { >>>>> + char name[VDUSE_NAME_MAX]; /* vduse device name */ >>>>> + __u32 vendor_id; /* virtio vendor id */ >>>>> + __u32 device_id; /* virtio device id */ >>>>> + __u64 features; /* device features */ >>>>> + __u64 bounce_size; /* bounce buffer size for iommu */ >>>>> + __u16 vq_size_max; /* the max size of virtqueue */ >>>> The VIRTIO specification allows per-virtqueue sizes. A device can have >>>> two virtqueues, where the first one allows up to 1024 descriptors and >>>> the second one allows only 128 descriptors, for example. >>>> >>> Good point! But it looks like virtio-vdpa/virtio-pci doesn't support >>> that now. All virtqueues have the same maximum size. >> I see struct vpda_config_ops only supports a per-device max vq size: >> u16 (*get_vq_num_max)(struct vdpa_device *vdev); >> >> virtio-pci supports per-virtqueue sizes because the struct >> virtio_pci_common_cfg->queue_size register is per-queue (controlled by >> queue_select). >> > Oh, yes. I miss queue_select. > >> I guess this is a question for Jason: will vdpa will keep this limitation? >> If yes, then VDUSE can stick to it too without running into problems in >> the future. I think it's better to extend the get_vq_num_max() per virtqueue. Currently, vDPA assumes the parent to have a global max size. This seems to work on most of the parents but not vp-vDPA (which could be backed by QEMU, in that case cvq's size is smaller). Fortunately, we haven't enabled had cvq support in the userspace now. I can post the fixes. >> >>>>> + __u16 padding; /* padding */ >>>>> + __u32 vq_num; /* the number of virtqueues */ >>>>> + __u32 vq_align; /* the allocation alignment of virtqueue's metadata */ >>>> I'm not sure what this is? >>>> >>> This will be used by vring_create_virtqueue() too. >> If there is no official definition for the meaning of this value then >> "/* same as vring_create_virtqueue()'s vring_align parameter */" would >> be clearer. That way the reader knows what to research in order to >> understand how this field works. >> > OK. > >> I don't remember but maybe it was used to support vrings when the >> host/guest have non-4KB page sizes. I wonder if anyone has an official >> definition for this value? > Not sure. Maybe we might need some alignment which is less than > PAGE_SIZE sometimes. So I see CCW always use 4096, but I'm not sure whether or not it's smaller than PAGE_SIZE. Thanks > > Thanks, > Yongji > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.1 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,URIBL_RED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFF3AC11F67 for ; Thu, 1 Jul 2021 07:55:59 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6675561480 for ; Thu, 1 Jul 2021 07:55:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6675561480 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 3D55B40148; Thu, 1 Jul 2021 07:55:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ecSjHwKzJ5KH; Thu, 1 Jul 2021 07:55:58 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 353494015D; Thu, 1 Jul 2021 07:55:58 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 03A19C001A; Thu, 1 Jul 2021 07:55:58 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 13676C000E for ; Thu, 1 Jul 2021 07:55:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 0D6EB40637 for ; Thu, 1 Jul 2021 07:55:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gKoi9HOk1Gh3 for ; Thu, 1 Jul 2021 07:55:55 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id E70DE40636 for ; Thu, 1 Jul 2021 07:55:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625126153; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3kkO+UX+kTqcYTfctkZezZWjIyhRJe4+TF27ND409mQ=; b=jHLWW9mKaiIBq4ooDnWmNq2ZVMEIEwm193hJjk11OAmBtTv4vXADaVQEV8gNwru1vonUks vHjmivUcl79+Ze+lVQEMLS4KjDu7wqaI/qwO3zm7WKcIjH5saMeDYZqNY9yYGXW3c2LatM g3FpI80j/kdOcpf2ib0XbYLfptxd1lc= Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-520-dEuxhVOlNDmTEfg_LZJ8xw-1; Thu, 01 Jul 2021 03:55:52 -0400 X-MC-Unique: dEuxhVOlNDmTEfg_LZJ8xw-1 Received: by mail-pg1-f197.google.com with SMTP id m13-20020a633f0d0000b0290222ece48979so3619806pga.1 for ; Thu, 01 Jul 2021 00:55:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=3kkO+UX+kTqcYTfctkZezZWjIyhRJe4+TF27ND409mQ=; b=okM/BJFQmadbBXPLyXWLI7BZ+Q83OEDEu/jU9eTSURHG4sPsBIrMHq+mzVG4xAbFhG SuDiAXfFEd094X7gysq/S/61BkLjywGNzPJdKCCymRodKwavXGRvyzQmtQUbSt0mb3D6 2wQKXEoMRgdIk9RKJl6OVsObl6mA8BPb7pe4jFfhOdYhsHMjJ9R6FiT/qfzVKdjHQYlf ecwEpmX2UZcf/alpAJr5ckL+IrgpQOFJVB1h/0B/F2dIngMkxNf4a4g4qtiKjPdbIIWR cpU3/Rot6svGvYzKA/3iaq5E8UltXgfMfiaf+GPgAdhmzS0LVC+/0f12u1dIepqp0F18 SdVw== X-Gm-Message-State: AOAM533HbXhj2+FSpCTNSSPoqnLYbzETPEnvsNMENIRYW1FSHjWjbcm0 1xmaZ2rgxVNDVDRtAWvbmX89hjSOyEwF0Fpx5O+N9ferOxoerqQcIcP6nypsQbCK7OH0Tvi4QvX GZoxnabPh8cWlX1eSfEjiMqEjSt7iGA== X-Received: by 2002:a63:794:: with SMTP id 142mr6983327pgh.198.1625126151682; Thu, 01 Jul 2021 00:55:51 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx+FJQkpNj2SGJqmkRE0B+JqnTbGGKE300PtWwx2ZJ5+wT2j2YuZ2VlEIodQZCrR0j2AFzwoQ== X-Received: by 2002:a63:794:: with SMTP id 142mr6983291pgh.198.1625126151412; Thu, 01 Jul 2021 00:55:51 -0700 (PDT) Received: from wangxiaodeMacBook-Air.local ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id a6sm23966070pfo.212.2021.07.01.00.55.42 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 01 Jul 2021 00:55:50 -0700 (PDT) Subject: Re: [PATCH v8 09/10] vduse: Introduce VDUSE - vDPA Device in Userspace To: Yongji Xie , Stefan Hajnoczi References: <20210615141331.407-1-xieyongji@bytedance.com> <20210615141331.407-10-xieyongji@bytedance.com> From: Jason Wang Message-ID: <7264cb0b-7072-098e-3d22-2b7e89216545@redhat.com> Date: Thu, 1 Jul 2021 15:55:36 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Cc: kvm , "Michael S. Tsirkin" , virtualization , Christian Brauner , Jonathan Corbet , Matthew Wilcox , Christoph Hellwig , Dan Carpenter , Stefano Garzarella , Al Viro , songmuchun@bytedance.com, Jens Axboe , Greg KH , Randy Dunlap , linux-kernel , iommu@lists.linux-foundation.org, bcrl@kvack.org, netdev@vger.kernel.org, linux-fsdevel@vger.kernel.org, =?UTF-8?Q?Mika_Penttil=c3=a4?= X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" CuWcqCAyMDIxLzcvMSDkuIvljYgyOjUwLCBZb25namkgWGllIOWGmemBkzoKPiBPbiBXZWQsIEp1 biAzMCwgMjAyMSBhdCA1OjUxIFBNIFN0ZWZhbiBIYWpub2N6aSA8c3RlZmFuaGFAcmVkaGF0LmNv bT4gd3JvdGU6Cj4+IE9uIFR1ZSwgSnVuIDI5LCAyMDIxIGF0IDEwOjU5OjUxQU0gKzA4MDAsIFlv bmdqaSBYaWUgd3JvdGU6Cj4+PiBPbiBNb24sIEp1biAyOCwgMjAyMSBhdCA5OjAyIFBNIFN0ZWZh biBIYWpub2N6aSA8c3RlZmFuaGFAcmVkaGF0LmNvbT4gd3JvdGU6Cj4+Pj4gT24gVHVlLCBKdW4g MTUsIDIwMjEgYXQgMTA6MTM6MzBQTSArMDgwMCwgWGllIFlvbmdqaSB3cm90ZToKPj4+Pj4gKy8q IGlvY3RscyAqLwo+Pj4+PiArCj4+Pj4+ICtzdHJ1Y3QgdmR1c2VfZGV2X2NvbmZpZyB7Cj4+Pj4+ ICsgICAgIGNoYXIgbmFtZVtWRFVTRV9OQU1FX01BWF07IC8qIHZkdXNlIGRldmljZSBuYW1lICov Cj4+Pj4+ICsgICAgIF9fdTMyIHZlbmRvcl9pZDsgLyogdmlydGlvIHZlbmRvciBpZCAqLwo+Pj4+ PiArICAgICBfX3UzMiBkZXZpY2VfaWQ7IC8qIHZpcnRpbyBkZXZpY2UgaWQgKi8KPj4+Pj4gKyAg ICAgX191NjQgZmVhdHVyZXM7IC8qIGRldmljZSBmZWF0dXJlcyAqLwo+Pj4+PiArICAgICBfX3U2 NCBib3VuY2Vfc2l6ZTsgLyogYm91bmNlIGJ1ZmZlciBzaXplIGZvciBpb21tdSAqLwo+Pj4+PiAr ICAgICBfX3UxNiB2cV9zaXplX21heDsgLyogdGhlIG1heCBzaXplIG9mIHZpcnRxdWV1ZSAqLwo+ Pj4+IFRoZSBWSVJUSU8gc3BlY2lmaWNhdGlvbiBhbGxvd3MgcGVyLXZpcnRxdWV1ZSBzaXplcy4g QSBkZXZpY2UgY2FuIGhhdmUKPj4+PiB0d28gdmlydHF1ZXVlcywgd2hlcmUgdGhlIGZpcnN0IG9u ZSBhbGxvd3MgdXAgdG8gMTAyNCBkZXNjcmlwdG9ycyBhbmQKPj4+PiB0aGUgc2Vjb25kIG9uZSBh bGxvd3Mgb25seSAxMjggZGVzY3JpcHRvcnMsIGZvciBleGFtcGxlLgo+Pj4+Cj4+PiBHb29kIHBv aW50ISBCdXQgaXQgbG9va3MgbGlrZSB2aXJ0aW8tdmRwYS92aXJ0aW8tcGNpIGRvZXNuJ3Qgc3Vw cG9ydAo+Pj4gdGhhdCBub3cuIEFsbCB2aXJ0cXVldWVzIGhhdmUgdGhlIHNhbWUgbWF4aW11bSBz aXplLgo+PiBJIHNlZSBzdHJ1Y3QgdnBkYV9jb25maWdfb3BzIG9ubHkgc3VwcG9ydHMgYSBwZXIt ZGV2aWNlIG1heCB2cSBzaXplOgo+PiB1MTYgKCpnZXRfdnFfbnVtX21heCkoc3RydWN0IHZkcGFf ZGV2aWNlICp2ZGV2KTsKPj4KPj4gdmlydGlvLXBjaSBzdXBwb3J0cyBwZXItdmlydHF1ZXVlIHNp emVzIGJlY2F1c2UgdGhlIHN0cnVjdAo+PiB2aXJ0aW9fcGNpX2NvbW1vbl9jZmctPnF1ZXVlX3Np emUgcmVnaXN0ZXIgaXMgcGVyLXF1ZXVlIChjb250cm9sbGVkIGJ5Cj4+IHF1ZXVlX3NlbGVjdCku Cj4+Cj4gT2gsIHllcy4gSSBtaXNzIHF1ZXVlX3NlbGVjdC4KPgo+PiBJIGd1ZXNzIHRoaXMgaXMg YSBxdWVzdGlvbiBmb3IgSmFzb246IHdpbGwgdmRwYSB3aWxsIGtlZXAgdGhpcyBsaW1pdGF0aW9u Pwo+PiBJZiB5ZXMsIHRoZW4gVkRVU0UgY2FuIHN0aWNrIHRvIGl0IHRvbyB3aXRob3V0IHJ1bm5p bmcgaW50byBwcm9ibGVtcyBpbgo+PiB0aGUgZnV0dXJlLgoKCkkgdGhpbmsgaXQncyBiZXR0ZXIg dG8gZXh0ZW5kIHRoZSBnZXRfdnFfbnVtX21heCgpIHBlciB2aXJ0cXVldWUuCgpDdXJyZW50bHks IHZEUEEgYXNzdW1lcyB0aGUgcGFyZW50IHRvIGhhdmUgYSBnbG9iYWwgbWF4IHNpemUuIFRoaXMg c2VlbXMgCnRvIHdvcmsgb24gbW9zdCBvZiB0aGUgcGFyZW50cyBidXQgbm90IHZwLXZEUEEgKHdo aWNoIGNvdWxkIGJlIGJhY2tlZCBieSAKUUVNVSwgaW4gdGhhdCBjYXNlIGN2cSdzIHNpemUgaXMg c21hbGxlcikuCgpGb3J0dW5hdGVseSwgd2UgaGF2ZW4ndCBlbmFibGVkIGhhZCBjdnEgc3VwcG9y dCBpbiB0aGUgdXNlcnNwYWNlIG5vdy4KCkkgY2FuIHBvc3QgdGhlIGZpeGVzLgoKCj4+Cj4+Pj4+ ICsgICAgIF9fdTE2IHBhZGRpbmc7IC8qIHBhZGRpbmcgKi8KPj4+Pj4gKyAgICAgX191MzIgdnFf bnVtOyAvKiB0aGUgbnVtYmVyIG9mIHZpcnRxdWV1ZXMgKi8KPj4+Pj4gKyAgICAgX191MzIgdnFf YWxpZ247IC8qIHRoZSBhbGxvY2F0aW9uIGFsaWdubWVudCBvZiB2aXJ0cXVldWUncyBtZXRhZGF0 YSAqLwo+Pj4+IEknbSBub3Qgc3VyZSB3aGF0IHRoaXMgaXM/Cj4+Pj4KPj4+ICAgVGhpcyB3aWxs IGJlIHVzZWQgYnkgdnJpbmdfY3JlYXRlX3ZpcnRxdWV1ZSgpIHRvby4KPj4gSWYgdGhlcmUgaXMg bm8gb2ZmaWNpYWwgZGVmaW5pdGlvbiBmb3IgdGhlIG1lYW5pbmcgb2YgdGhpcyB2YWx1ZSB0aGVu Cj4+ICIvKiBzYW1lIGFzIHZyaW5nX2NyZWF0ZV92aXJ0cXVldWUoKSdzIHZyaW5nX2FsaWduIHBh cmFtZXRlciAqLyIgd291bGQKPj4gYmUgY2xlYXJlci4gVGhhdCB3YXkgdGhlIHJlYWRlciBrbm93 cyB3aGF0IHRvIHJlc2VhcmNoIGluIG9yZGVyIHRvCj4+IHVuZGVyc3RhbmQgaG93IHRoaXMgZmll bGQgd29ya3MuCj4+Cj4gT0suCj4KPj4gSSBkb24ndCByZW1lbWJlciBidXQgbWF5YmUgaXQgd2Fz IHVzZWQgdG8gc3VwcG9ydCB2cmluZ3Mgd2hlbiB0aGUKPj4gaG9zdC9ndWVzdCBoYXZlIG5vbi00 S0IgcGFnZSBzaXplcy4gSSB3b25kZXIgaWYgYW55b25lIGhhcyBhbiBvZmZpY2lhbAo+PiBkZWZp bml0aW9uIGZvciB0aGlzIHZhbHVlPwo+IE5vdCBzdXJlLiBNYXliZSB3ZSBtaWdodCBuZWVkIHNv bWUgYWxpZ25tZW50IHdoaWNoIGlzIGxlc3MgdGhhbgo+IFBBR0VfU0laRSBzb21ldGltZXMuCgoK U28gSSBzZWUgQ0NXIGFsd2F5cyB1c2UgNDA5NiwgYnV0IEknbSBub3Qgc3VyZSB3aGV0aGVyIG9y IG5vdCBpdCdzIApzbWFsbGVyIHRoYW4gUEFHRV9TSVpFLgoKVGhhbmtzCgoKPgo+IFRoYW5rcywK PiBZb25namkKPgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X18KaW9tbXUgbWFpbGluZyBsaXN0CmlvbW11QGxpc3RzLmxpbnV4LWZvdW5kYXRpb24ub3JnCmh0 dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9uLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2lvbW11 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.1 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,URIBL_RED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49D32C11F69 for ; Thu, 1 Jul 2021 07:55:59 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DB7BA61493 for ; Thu, 1 Jul 2021 07:55:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DB7BA61493 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=virtualization-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 9C36740640; Thu, 1 Jul 2021 07:55:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id B2xz6Soxffj3; Thu, 1 Jul 2021 07:55:57 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id EFBD340636; Thu, 1 Jul 2021 07:55:56 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C1945C001A; Thu, 1 Jul 2021 07:55:56 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id ADEF1C000E for ; Thu, 1 Jul 2021 07:55:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 9C5AB82B1B for ; Thu, 1 Jul 2021 07:55:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6UrjEVhGtXIC for ; Thu, 1 Jul 2021 07:55:55 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id E78F082925 for ; Thu, 1 Jul 2021 07:55:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625126153; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3kkO+UX+kTqcYTfctkZezZWjIyhRJe4+TF27ND409mQ=; b=jHLWW9mKaiIBq4ooDnWmNq2ZVMEIEwm193hJjk11OAmBtTv4vXADaVQEV8gNwru1vonUks vHjmivUcl79+Ze+lVQEMLS4KjDu7wqaI/qwO3zm7WKcIjH5saMeDYZqNY9yYGXW3c2LatM g3FpI80j/kdOcpf2ib0XbYLfptxd1lc= Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-392-wTJOKR_RNMiLObUvVVYKjg-1; Thu, 01 Jul 2021 03:55:52 -0400 X-MC-Unique: wTJOKR_RNMiLObUvVVYKjg-1 Received: by mail-pg1-f199.google.com with SMTP id h1-20020a63df410000b0290222939c0dd7so3627633pgj.4 for ; Thu, 01 Jul 2021 00:55:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=3kkO+UX+kTqcYTfctkZezZWjIyhRJe4+TF27ND409mQ=; b=BI2B2SMh9877aYInW2CNdCih0yurbnPyTMZgsOzT+s3EW18oil2tgfsU8k3Hn3nUQZ cnbyKeGUcN/hDeThnmgRUIvL1+d/XPaogYKIx6JYuiZ85JhAerpK9hn0wqo4OolXeEwt kDJF59rkyzw3GL7nkPe8tkaoG6G7dLDY1Y7DyvdsAMdFK4+KYfP/EUUaT/lBOFcgsMPf KYJvoY2ro7AgfT9nM3vlLWv2BwhUqRLEkWtdTtSSHFwX//udCFjVWSskLFHurGKnWPAd wheKaiIlXDV396BZY4+gMTs5sgdw/7knSDRhQef5CRmO0O9h3f6YK7eBPS17ndMlxyo7 AmSg== X-Gm-Message-State: AOAM531lHMaHBkU9oMyzpvFykl0B1OAznS3AofmGyQfUnirJjcKZpfWE IS+ax5Q4dAabXXkhDN3hyRNWD/OnTcHTnSb/oOTlH/pn/tnfzra7AvfAACm4ENWXE/57Z/1q4C4 kHG+/+J6yr5nSfRitpjjVUhnIgPxKUHBYdEQqzqLitg== X-Received: by 2002:a63:794:: with SMTP id 142mr6983318pgh.198.1625126151674; Thu, 01 Jul 2021 00:55:51 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx+FJQkpNj2SGJqmkRE0B+JqnTbGGKE300PtWwx2ZJ5+wT2j2YuZ2VlEIodQZCrR0j2AFzwoQ== X-Received: by 2002:a63:794:: with SMTP id 142mr6983291pgh.198.1625126151412; Thu, 01 Jul 2021 00:55:51 -0700 (PDT) Received: from wangxiaodeMacBook-Air.local ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id a6sm23966070pfo.212.2021.07.01.00.55.42 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 01 Jul 2021 00:55:50 -0700 (PDT) Subject: Re: [PATCH v8 09/10] vduse: Introduce VDUSE - vDPA Device in Userspace To: Yongji Xie , Stefan Hajnoczi References: <20210615141331.407-1-xieyongji@bytedance.com> <20210615141331.407-10-xieyongji@bytedance.com> From: Jason Wang Message-ID: <7264cb0b-7072-098e-3d22-2b7e89216545@redhat.com> Date: Thu, 1 Jul 2021 15:55:36 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Cc: kvm , "Michael S. Tsirkin" , virtualization , Christian Brauner , Jonathan Corbet , joro@8bytes.org, Matthew Wilcox , Christoph Hellwig , Dan Carpenter , Al Viro , songmuchun@bytedance.com, Jens Axboe , Greg KH , Randy Dunlap , linux-kernel , iommu@lists.linux-foundation.org, bcrl@kvack.org, netdev@vger.kernel.org, linux-fsdevel@vger.kernel.org, =?UTF-8?Q?Mika_Penttil=c3=a4?= X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" CuWcqCAyMDIxLzcvMSDkuIvljYgyOjUwLCBZb25namkgWGllIOWGmemBkzoKPiBPbiBXZWQsIEp1 biAzMCwgMjAyMSBhdCA1OjUxIFBNIFN0ZWZhbiBIYWpub2N6aSA8c3RlZmFuaGFAcmVkaGF0LmNv bT4gd3JvdGU6Cj4+IE9uIFR1ZSwgSnVuIDI5LCAyMDIxIGF0IDEwOjU5OjUxQU0gKzA4MDAsIFlv bmdqaSBYaWUgd3JvdGU6Cj4+PiBPbiBNb24sIEp1biAyOCwgMjAyMSBhdCA5OjAyIFBNIFN0ZWZh biBIYWpub2N6aSA8c3RlZmFuaGFAcmVkaGF0LmNvbT4gd3JvdGU6Cj4+Pj4gT24gVHVlLCBKdW4g MTUsIDIwMjEgYXQgMTA6MTM6MzBQTSArMDgwMCwgWGllIFlvbmdqaSB3cm90ZToKPj4+Pj4gKy8q IGlvY3RscyAqLwo+Pj4+PiArCj4+Pj4+ICtzdHJ1Y3QgdmR1c2VfZGV2X2NvbmZpZyB7Cj4+Pj4+ ICsgICAgIGNoYXIgbmFtZVtWRFVTRV9OQU1FX01BWF07IC8qIHZkdXNlIGRldmljZSBuYW1lICov Cj4+Pj4+ICsgICAgIF9fdTMyIHZlbmRvcl9pZDsgLyogdmlydGlvIHZlbmRvciBpZCAqLwo+Pj4+ PiArICAgICBfX3UzMiBkZXZpY2VfaWQ7IC8qIHZpcnRpbyBkZXZpY2UgaWQgKi8KPj4+Pj4gKyAg ICAgX191NjQgZmVhdHVyZXM7IC8qIGRldmljZSBmZWF0dXJlcyAqLwo+Pj4+PiArICAgICBfX3U2 NCBib3VuY2Vfc2l6ZTsgLyogYm91bmNlIGJ1ZmZlciBzaXplIGZvciBpb21tdSAqLwo+Pj4+PiAr ICAgICBfX3UxNiB2cV9zaXplX21heDsgLyogdGhlIG1heCBzaXplIG9mIHZpcnRxdWV1ZSAqLwo+ Pj4+IFRoZSBWSVJUSU8gc3BlY2lmaWNhdGlvbiBhbGxvd3MgcGVyLXZpcnRxdWV1ZSBzaXplcy4g QSBkZXZpY2UgY2FuIGhhdmUKPj4+PiB0d28gdmlydHF1ZXVlcywgd2hlcmUgdGhlIGZpcnN0IG9u ZSBhbGxvd3MgdXAgdG8gMTAyNCBkZXNjcmlwdG9ycyBhbmQKPj4+PiB0aGUgc2Vjb25kIG9uZSBh bGxvd3Mgb25seSAxMjggZGVzY3JpcHRvcnMsIGZvciBleGFtcGxlLgo+Pj4+Cj4+PiBHb29kIHBv aW50ISBCdXQgaXQgbG9va3MgbGlrZSB2aXJ0aW8tdmRwYS92aXJ0aW8tcGNpIGRvZXNuJ3Qgc3Vw cG9ydAo+Pj4gdGhhdCBub3cuIEFsbCB2aXJ0cXVldWVzIGhhdmUgdGhlIHNhbWUgbWF4aW11bSBz aXplLgo+PiBJIHNlZSBzdHJ1Y3QgdnBkYV9jb25maWdfb3BzIG9ubHkgc3VwcG9ydHMgYSBwZXIt ZGV2aWNlIG1heCB2cSBzaXplOgo+PiB1MTYgKCpnZXRfdnFfbnVtX21heCkoc3RydWN0IHZkcGFf ZGV2aWNlICp2ZGV2KTsKPj4KPj4gdmlydGlvLXBjaSBzdXBwb3J0cyBwZXItdmlydHF1ZXVlIHNp emVzIGJlY2F1c2UgdGhlIHN0cnVjdAo+PiB2aXJ0aW9fcGNpX2NvbW1vbl9jZmctPnF1ZXVlX3Np emUgcmVnaXN0ZXIgaXMgcGVyLXF1ZXVlIChjb250cm9sbGVkIGJ5Cj4+IHF1ZXVlX3NlbGVjdCku Cj4+Cj4gT2gsIHllcy4gSSBtaXNzIHF1ZXVlX3NlbGVjdC4KPgo+PiBJIGd1ZXNzIHRoaXMgaXMg YSBxdWVzdGlvbiBmb3IgSmFzb246IHdpbGwgdmRwYSB3aWxsIGtlZXAgdGhpcyBsaW1pdGF0aW9u Pwo+PiBJZiB5ZXMsIHRoZW4gVkRVU0UgY2FuIHN0aWNrIHRvIGl0IHRvbyB3aXRob3V0IHJ1bm5p bmcgaW50byBwcm9ibGVtcyBpbgo+PiB0aGUgZnV0dXJlLgoKCkkgdGhpbmsgaXQncyBiZXR0ZXIg dG8gZXh0ZW5kIHRoZSBnZXRfdnFfbnVtX21heCgpIHBlciB2aXJ0cXVldWUuCgpDdXJyZW50bHks IHZEUEEgYXNzdW1lcyB0aGUgcGFyZW50IHRvIGhhdmUgYSBnbG9iYWwgbWF4IHNpemUuIFRoaXMg c2VlbXMgCnRvIHdvcmsgb24gbW9zdCBvZiB0aGUgcGFyZW50cyBidXQgbm90IHZwLXZEUEEgKHdo aWNoIGNvdWxkIGJlIGJhY2tlZCBieSAKUUVNVSwgaW4gdGhhdCBjYXNlIGN2cSdzIHNpemUgaXMg c21hbGxlcikuCgpGb3J0dW5hdGVseSwgd2UgaGF2ZW4ndCBlbmFibGVkIGhhZCBjdnEgc3VwcG9y dCBpbiB0aGUgdXNlcnNwYWNlIG5vdy4KCkkgY2FuIHBvc3QgdGhlIGZpeGVzLgoKCj4+Cj4+Pj4+ ICsgICAgIF9fdTE2IHBhZGRpbmc7IC8qIHBhZGRpbmcgKi8KPj4+Pj4gKyAgICAgX191MzIgdnFf bnVtOyAvKiB0aGUgbnVtYmVyIG9mIHZpcnRxdWV1ZXMgKi8KPj4+Pj4gKyAgICAgX191MzIgdnFf YWxpZ247IC8qIHRoZSBhbGxvY2F0aW9uIGFsaWdubWVudCBvZiB2aXJ0cXVldWUncyBtZXRhZGF0 YSAqLwo+Pj4+IEknbSBub3Qgc3VyZSB3aGF0IHRoaXMgaXM/Cj4+Pj4KPj4+ICAgVGhpcyB3aWxs IGJlIHVzZWQgYnkgdnJpbmdfY3JlYXRlX3ZpcnRxdWV1ZSgpIHRvby4KPj4gSWYgdGhlcmUgaXMg bm8gb2ZmaWNpYWwgZGVmaW5pdGlvbiBmb3IgdGhlIG1lYW5pbmcgb2YgdGhpcyB2YWx1ZSB0aGVu Cj4+ICIvKiBzYW1lIGFzIHZyaW5nX2NyZWF0ZV92aXJ0cXVldWUoKSdzIHZyaW5nX2FsaWduIHBh cmFtZXRlciAqLyIgd291bGQKPj4gYmUgY2xlYXJlci4gVGhhdCB3YXkgdGhlIHJlYWRlciBrbm93 cyB3aGF0IHRvIHJlc2VhcmNoIGluIG9yZGVyIHRvCj4+IHVuZGVyc3RhbmQgaG93IHRoaXMgZmll bGQgd29ya3MuCj4+Cj4gT0suCj4KPj4gSSBkb24ndCByZW1lbWJlciBidXQgbWF5YmUgaXQgd2Fz IHVzZWQgdG8gc3VwcG9ydCB2cmluZ3Mgd2hlbiB0aGUKPj4gaG9zdC9ndWVzdCBoYXZlIG5vbi00 S0IgcGFnZSBzaXplcy4gSSB3b25kZXIgaWYgYW55b25lIGhhcyBhbiBvZmZpY2lhbAo+PiBkZWZp bml0aW9uIGZvciB0aGlzIHZhbHVlPwo+IE5vdCBzdXJlLiBNYXliZSB3ZSBtaWdodCBuZWVkIHNv bWUgYWxpZ25tZW50IHdoaWNoIGlzIGxlc3MgdGhhbgo+IFBBR0VfU0laRSBzb21ldGltZXMuCgoK U28gSSBzZWUgQ0NXIGFsd2F5cyB1c2UgNDA5NiwgYnV0IEknbSBub3Qgc3VyZSB3aGV0aGVyIG9y IG5vdCBpdCdzIApzbWFsbGVyIHRoYW4gUEFHRV9TSVpFLgoKVGhhbmtzCgoKPgo+IFRoYW5rcywK PiBZb25namkKPgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X18KVmlydHVhbGl6YXRpb24gbWFpbGluZyBsaXN0ClZpcnR1YWxpemF0aW9uQGxpc3RzLmxpbnV4 LWZvdW5kYXRpb24ub3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9uLm9yZy9tYWlsbWFu L2xpc3RpbmZvL3ZpcnR1YWxpemF0aW9u