From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 952BAC433F5 for ; Mon, 21 Feb 2022 07:15:27 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 4362E40242; Mon, 21 Feb 2022 07:15:27 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zcilCwcHKss3; Mon, 21 Feb 2022 07:15:25 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 1AECB404F0; Mon, 21 Feb 2022 07:15:25 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id DF7EEC0033; Mon, 21 Feb 2022 07:15:24 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 157BAC001A for ; Mon, 21 Feb 2022 07:15:23 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id F0161404D7 for ; Mon, 21 Feb 2022 07:15:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uxCJRPryInuf for ; Mon, 21 Feb 2022 07:15:21 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 3260140242 for ; Mon, 21 Feb 2022 07:15:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1645427719; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jrN8XsW/BOgueqLR4Y6t/TA6fNns23B14ddLmdsBcG0=; b=RsHoMm9Z06XHCfOaugtYdKqR8Uc4FEoTzdScJh0YyrQMLAPOXX7g38bcjq/3XnxKM9+NUq D84YLvEO43yDbt6Ec2NQrRzyDWnpC+SuaE+BgYfMK24kS+8nJ8o2maY3PH0ItjcyO2ysS5 0tvwCzQBgsmpX6UadLWqjBuQrL9oSWo= Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-166-9l4wXcU0MO2zOa9PNxHS4g-1; Mon, 21 Feb 2022 02:15:17 -0500 X-MC-Unique: 9l4wXcU0MO2zOa9PNxHS4g-1 Received: by mail-pg1-f200.google.com with SMTP id b9-20020a63e709000000b00362f44b02aeso9017189pgi.17 for ; Sun, 20 Feb 2022 23:15:17 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=jrN8XsW/BOgueqLR4Y6t/TA6fNns23B14ddLmdsBcG0=; b=n0wbAA48i4ba0/6yZ1VQ1v/N2VIM/MUj4dU539gAXAWKP897C3MR8rNCrwFUSThBqY UU/dimyQoYlSdw9ZErNFln4gQibDZqew4hWEn4rArJ/oAA+NZGyJ+ly0NuBDoQGBSKTA hxVGQqF4Va15Pen7PgmlWu3dDogwXu+mWMJ8fqHI4dS+A8PJXSqYP4oHpxbjzxvMe/ce sX4s7n16l4pkCeY/Vv7RVHuEPBi49jk8jENnTNd6ibrWl6Q1FlN3t8+GaAjgqFIRAinp COokKfiUiNd4t7W7HbeUuE2LUV2aywMUNqHfrETIsWmQA6JqFOQvoockjhYdS996EfaN EsGA== X-Gm-Message-State: AOAM531xG+kIex2Od9H+HTqEe1TcrAiPkB8hgvnN0UCni7fWjTNvSXdk 8mZzYO0FRQLgOHxGAPWpSXuirU+MgHavjKDkl1p7Ii13OuD21zQI18Ok1nvoJSDOu7gFwSVPzPT mi1nEv0I6ce4d8qMErFzFJdoPV+lyJpP2yH2OaxI83w== X-Received: by 2002:a17:902:d642:b0:14c:e9a7:d542 with SMTP id y2-20020a170902d64200b0014ce9a7d542mr17838066plh.38.1645427716012; Sun, 20 Feb 2022 23:15:16 -0800 (PST) X-Google-Smtp-Source: ABdhPJwXEdYXXygo2Bn/aG8AOxUtXK0uqh288rJiPaDSm5yyd3B2kkBgqHyxfMkmGU8ykvmhchguIg== X-Received: by 2002:a17:902:d642:b0:14c:e9a7:d542 with SMTP id y2-20020a170902d64200b0014ce9a7d542mr17838032plh.38.1645427715684; Sun, 20 Feb 2022 23:15:15 -0800 (PST) Received: from [10.72.12.96] ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id i22sm959549pjz.3.2022.02.20.23.15.06 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 20 Feb 2022 23:15:15 -0800 (PST) Message-ID: <02f29b62-6656-ba2f-1475-251b16e0e978@redhat.com> Date: Mon, 21 Feb 2022 15:15:03 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.6.0 Subject: Re: [PATCH 17/31] vdpa: adapt vhost_ops callbacks to svq To: Eugenio Perez Martin References: <20220121202733.404989-1-eperezma@redhat.com> <20220121202733.404989-18-eperezma@redhat.com> <82b8c3bf-1b11-86c7-4fad-294f5ccf1278@redhat.com> <3d0dfaaa-a67c-6f48-fd03-45d2661ba92a@redhat.com> From: Jason Wang In-Reply-To: Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Cc: Laurent Vivier , Parav Pandit , Cindy Lu , "Michael S. Tsirkin" , Richard Henderson , qemu-level , Gautam Dawar , Markus Armbruster , Eduardo Habkost , Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Paolo Bonzini , Zhu Lingshan , virtualization , Eric Blake X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" CuWcqCAyMDIyLzIvMTgg5LiK5Y2IMToxMywgRXVnZW5pbyBQZXJleiBNYXJ0aW4g5YaZ6YGTOgo+ IE9uIFR1ZSwgRmViIDgsIDIwMjIgYXQgNDo1OCBBTSBKYXNvbiBXYW5nIDxqYXNvd2FuZ0ByZWRo YXQuY29tPiB3cm90ZToKPj4KPj4g5ZyoIDIwMjIvMi8xIOS4iuWNiDI6NTgsIEV1Z2VuaW8gUGVy ZXogTWFydGluIOWGmemBkzoKPj4+IE9uIFN1biwgSmFuIDMwLCAyMDIyIGF0IDU6MDMgQU0gSmFz b24gV2FuZyA8amFzb3dhbmdAcmVkaGF0LmNvbT4gd3JvdGU6Cj4+Pj4g5ZyoIDIwMjIvMS8yMiDk uIrljYg0OjI3LCBFdWdlbmlvIFDDqXJleiDlhpnpgZM6Cj4+Pj4+IEZpcnN0IGhhbGYgb2YgdGhl IGJ1ZmZlcnMgZm9yd2FyZGluZyBwYXJ0LCBwcmVwYXJpbmcgdmhvc3QtdmRwYQo+Pj4+PiBjYWxs YmFja3MgdG8gU1ZRIHRvIG9mZmVyIGl0LiBRRU1VIGNhbm5vdCBlbmFibGUgaXQgYXQgdGhpcyBt b21lbnQsIHNvCj4+Pj4+IHRoaXMgaXMgZWZmZWN0aXZlbHkgZGVhZCBjb2RlIGF0IHRoZSBtb21l bnQsIGJ1dCBpdCBoZWxwcyB0byByZWR1Y2UKPj4+Pj4gcGF0Y2ggc2l6ZS4KPj4+Pj4KPj4+Pj4g U2lnbmVkLW9mZi1ieTogRXVnZW5pbyBQw6lyZXogPGVwZXJlem1hQHJlZGhhdC5jb20+Cj4+Pj4+ IC0tLQo+Pj4+PiAgICAgaHcvdmlydGlvL3Zob3N0LXNoYWRvdy12aXJ0cXVldWUuaCB8ICAgMiAr LQo+Pj4+PiAgICAgaHcvdmlydGlvL3Zob3N0LXNoYWRvdy12aXJ0cXVldWUuYyB8ICAyMSArKysr LQo+Pj4+PiAgICAgaHcvdmlydGlvL3Zob3N0LXZkcGEuYyAgICAgICAgICAgICB8IDEzMyArKysr KysrKysrKysrKysrKysrKysrKysrKy0tLQo+Pj4+PiAgICAgMyBmaWxlcyBjaGFuZ2VkLCAxNDMg aW5zZXJ0aW9ucygrKSwgMTMgZGVsZXRpb25zKC0pCj4+Pj4+Cj4+Pj4+IGRpZmYgLS1naXQgYS9o dy92aXJ0aW8vdmhvc3Qtc2hhZG93LXZpcnRxdWV1ZS5oIGIvaHcvdmlydGlvL3Zob3N0LXNoYWRv dy12aXJ0cXVldWUuaAo+Pj4+PiBpbmRleCAwMzUyMDdhNDY5Li4zOWFlZjVmZmRmIDEwMDY0NAo+ Pj4+PiAtLS0gYS9ody92aXJ0aW8vdmhvc3Qtc2hhZG93LXZpcnRxdWV1ZS5oCj4+Pj4+ICsrKyBi L2h3L3ZpcnRpby92aG9zdC1zaGFkb3ctdmlydHF1ZXVlLmgKPj4+Pj4gQEAgLTM1LDcgKzM1LDcg QEAgc2l6ZV90IHZob3N0X3N2cV9kZXZpY2VfYXJlYV9zaXplKGNvbnN0IFZob3N0U2hhZG93Vmly dHF1ZXVlICpzdnEpOwo+Pj4+Pgo+Pj4+PiAgICAgdm9pZCB2aG9zdF9zdnFfc3RvcChWaG9zdFNo YWRvd1ZpcnRxdWV1ZSAqc3ZxKTsKPj4+Pj4KPj4+Pj4gLVZob3N0U2hhZG93VmlydHF1ZXVlICp2 aG9zdF9zdnFfbmV3KHZvaWQpOwo+Pj4+PiArVmhvc3RTaGFkb3dWaXJ0cXVldWUgKnZob3N0X3N2 cV9uZXcodWludDE2X3QgcXNpemUpOwo+Pj4+Pgo+Pj4+PiAgICAgdm9pZCB2aG9zdF9zdnFfZnJl ZShWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAqdnEpOwo+Pj4+Pgo+Pj4+PiBkaWZmIC0tZ2l0IGEvaHcv dmlydGlvL3Zob3N0LXNoYWRvdy12aXJ0cXVldWUuYyBiL2h3L3ZpcnRpby92aG9zdC1zaGFkb3ct dmlydHF1ZXVlLmMKPj4+Pj4gaW5kZXggZjEyOWVjODM5NS4uN2MxNjgwNzVkNyAxMDA2NDQKPj4+ Pj4gLS0tIGEvaHcvdmlydGlvL3Zob3N0LXNoYWRvdy12aXJ0cXVldWUuYwo+Pj4+PiArKysgYi9o dy92aXJ0aW8vdmhvc3Qtc2hhZG93LXZpcnRxdWV1ZS5jCj4+Pj4+IEBAIC0yNzcsOSArMjc3LDE3 IEBAIHZvaWQgdmhvc3Rfc3ZxX3N0b3AoVmhvc3RTaGFkb3dWaXJ0cXVldWUgKnN2cSkKPj4+Pj4g ICAgIC8qKgo+Pj4+PiAgICAgICogQ3JlYXRlcyB2aG9zdCBzaGFkb3cgdmlydHF1ZXVlLCBhbmQg aW5zdHJ1Y3Qgdmhvc3QgZGV2aWNlIHRvIHVzZSB0aGUgc2hhZG93Cj4+Pj4+ICAgICAgKiBtZXRo b2RzIGFuZCBmaWxlIGRlc2NyaXB0b3JzLgo+Pj4+PiArICoKPj4+Pj4gKyAqIEBxc2l6ZSBTaGFk b3cgVmlydFF1ZXVlIHNpemUKPj4+Pj4gKyAqCj4+Pj4+ICsgKiBSZXR1cm5zIHRoZSBuZXcgdmly dHF1ZXVlIG9yIE5VTEwuCj4+Pj4+ICsgKgo+Pj4+PiArICogSW4gY2FzZSBvZiBlcnJvciwgcmVh c29uIGlzIHJlcG9ydGVkIHRocm91Z2ggZXJyb3JfcmVwb3J0Lgo+Pj4+PiAgICAgICovCj4+Pj4+ IC1WaG9zdFNoYWRvd1ZpcnRxdWV1ZSAqdmhvc3Rfc3ZxX25ldyh2b2lkKQo+Pj4+PiArVmhvc3RT aGFkb3dWaXJ0cXVldWUgKnZob3N0X3N2cV9uZXcodWludDE2X3QgcXNpemUpCj4+Pj4+ICAgICB7 Cj4+Pj4+ICsgICAgc2l6ZV90IGRlc2Nfc2l6ZSA9IHNpemVvZih2cmluZ19kZXNjX3QpICogcXNp emU7Cj4+Pj4+ICsgICAgc2l6ZV90IGRldmljZV9zaXplLCBkcml2ZXJfc2l6ZTsKPj4+Pj4gICAg ICAgICBnX2F1dG9mcmVlIFZob3N0U2hhZG93VmlydHF1ZXVlICpzdnEgPSBnX25ldzAoVmhvc3RT aGFkb3dWaXJ0cXVldWUsIDEpOwo+Pj4+PiAgICAgICAgIGludCByOwo+Pj4+Pgo+Pj4+PiBAQCAt MzAwLDYgKzMwOCwxNSBAQCBWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAqdmhvc3Rfc3ZxX25ldyh2b2lk KQo+Pj4+PiAgICAgICAgIC8qIFBsYWNlaG9sZGVyIGRlc2NyaXB0b3IsIGl0IHNob3VsZCBiZSBk ZWxldGVkIGF0IHNldF9raWNrX2ZkICovCj4+Pj4+ICAgICAgICAgZXZlbnRfbm90aWZpZXJfaW5p dF9mZCgmc3ZxLT5zdnFfa2ljaywgSU5WQUxJRF9TVlFfS0lDS19GRCk7Cj4+Pj4+Cj4+Pj4+ICsg ICAgc3ZxLT52cmluZy5udW0gPSBxc2l6ZTsKPj4+PiBJIHdvbmRlciBpZiB0aGlzIGlzIHRoZSBi ZXN0LiBFLmcgc29tZSBoYXJkd2FyZSBjYW4gc3VwcG9ydCB1cCB0byAzMksKPj4+PiBxdWV1ZSBz aXplLiBTbyB0aGlzIHdpbGwgcHJvYmFibHkgZW5kIHVwIHdpdGg6Cj4+Pj4KPj4+PiAxKSBTVlEg dXNlIDMySyBxdWV1ZSBzaXplCj4+Pj4gMikgaGFyZHdhcmUgcXVldWUgdXNlcyAyNTYKPj4+Pgo+ Pj4gSW4gdGhhdCBjYXNlIFNWUSB2cmluZyBxdWV1ZSBzaXplIHdpbGwgYmUgMzJLIGFuZCBndWVz dCdzIHZyaW5nIGNhbgo+Pj4gbmVnb3RpYXRlIGFueSBudW1iZXIgd2l0aCBTVlEgZXF1YWwgb3Ig bGVzcyB0aGFuIDMySywKPj4KPj4gU29ycnkgZm9yIGJlaW5nIHVuY2xlYXIgd2hhdCBJIG1lYW50 IGlzIGFjdHVhbGx5Cj4+Cj4+IDEpIFNWUSB1c2VzIDMySyBxdWV1ZSBzaXplCj4+Cj4+IDIpIGd1 ZXN0IHZxIHVzZXMgMjU2Cj4+Cj4+IFRoaXMgbG9va3MgbGlrZSBhIGJ1cmRlbiB0aGF0IG5lZWRz IGV4dHJhIGxvZ2ljIGFuZCBtYXkgZGFtYWdlIHRoZQo+PiBwZXJmb3JtYW5jZS4KPj4KPiBTdGls bCBub3QgZ2V0dGluZyB0aGlzIHBvaW50Lgo+Cj4gQW4gYXZhaWxhYmxlIGd1ZXN0IGJ1ZmZlciwg YWx0aG91Z2ggY29udGlndW91cyBpbiBHUEEvR1ZBLCBjYW4gZXhwYW5kCj4gaW4gbXVsdGlwbGUg YnVmZmVycyBpZiBpdCdzIG5vdCBjb250aWd1b3VzIGluIHFlbXUncyBWQSAoYnkgdGhlIHdoaWxl Cj4gbG9vcCBpbiB2aXJ0cXVldWVfbWFwX2Rlc2MgWzFdKS4gSW4gdGhhdCBzY2VuYXJpbyBpdCBp cyBiZXR0ZXIgdG8gaGF2ZQo+ICJwbGVudHkiIG9mIFNWUSBidWZmZXJzLgoKClllcywgYnV0IHRo aXMgY2FzZSBzaG91bGQgYmUgcmFyZS4gU28gaW4gdGhpcyBjYXNlIHdlIHNob3VsZCBkZWFsIHdp dGggCm92ZXJydW4gb24gU1ZRLCB0aGF0IGlzCgoxKSBTVlEgaXMgZnVsbAoyKSBndWVzdCBWUSBp c24ndAoKV2UgbmVlZCB0bwoKMSkgY2hlY2sgdGhlIGF2YWlsYWJsZSBidWZmZXIgc2xvdHMKMikg ZGlzYWJsZSBndWVzdCBraWNrIGFuZCB3YWl0IGZvciB0aGUgdXNlZCBidWZmZXJzCgpCdXQgaXQg bG9va3MgdG8gbWUgdGhlIGN1cnJlbnQgY29kZSBpcyBub3QgcmVhZHkgZm9yIGRlYWxpbmcgd2l0 aCB0aGlzIGNhc2U/CgoKPgo+IEknbSBvayBpZiB3ZSBkZWNpZGUgdG8gcHV0IGFuIHVwcGVyIGxp bWl0IHRob3VnaCwgb3IgaWYgd2UgZGVjaWRlIG5vdAo+IHRvIGhhbmRsZSB0aGlzIHNpdHVhdGlv bi4gQnV0IHdlIHdvdWxkIGxlYXZlIG91dCB2YWxpZCB2aXJ0aW8gZHJpdmVycy4KPiBNYXliZSB0 byBzZXQgYSBmaXhlZCB1cHBlciBsaW1pdCAoMTAyND8pPyBUbyBhZGQgYW5vdGhlciBwYXJhbWV0 ZXIKPiAoeC1zdnEtc2l6ZS1uPU4pPwo+Cj4gSWYgeW91IG1lYW4gd2UgbG9zZSBwZXJmb3JtYW5j ZSBiZWNhdXNlIG1lbW9yeSBnZXRzIG1vcmUgc3BhcnNlIEkKPiB0aGluayB0aGUgb25seSBwb3Nz aWJpbGl0eSBpcyB0byBsaW1pdCB0aGF0IHdheS4KCgpJZiBndWVzdCBpcyBub3QgdXNpbmcgMzJL LCBoYXZpbmcgYSAzMksgZm9yIHN2cSBtYXkgZ2l2ZXMgZXh0cmEgc3RyZXNzIApvbiB0aGUgY2Fj aGUgc2luY2Ugd2Ugd2lsbCBlbmQgdXAgd2l0aCBhIHByZXR0eSBsYXJnZSB3b3JraW5nIHNldC4K Cgo+Cj4+IEFuZCB0aGlzIGNhbiBsZWFkIG90aGVyIGludGVyZXN0aW5nIHNpdHVhdGlvbjoKPj4K Pj4gMSkgU1ZRIHVzZXMgMjU2Cj4+Cj4+IDIpIGd1ZXN0IHZxIHVzZXMgMTAyNAo+Pgo+PiBXaGVy ZSBhIGxvdCBvZiBtb3JlIFNWUSBsb2dpYyBpcyBuZWVkZWQuCj4+Cj4gSWYgd2UgYWdyZWUgdGhh dCBhIGd1ZXN0IGRlc2NyaXB0b3IgY2FuIGV4cGFuZCBpbiBtdWx0aXBsZSBTVlEKPiBkZXNjcmlw dG9ycywgdGhpcyBzaG91bGQgYmUgYWxyZWFkeSBoYW5kbGVkIGJ5IHRoZSBwcmV2aW91cyBsb2dp YyB0b28uCj4KPiBCdXQgdGhpcyBzaG91bGQgb25seSBoYXBwZW4gaW4gY2FzZSB0aGF0IHFlbXUg aXMgbGF1bmNoZWQgd2l0aCBhICJiYWQiCj4gY21kbGluZSwgaXNuJ3QgaXQ/CgoKVGhpcyBzZWVt cyBjYW4gaGFwcGVuIHdoZW4gd2UgdXNlIC1kZXZpY2UgCnZpcnRpby1uZXQtcGNpLHR4X3F1ZXVl X3NpemU9MTAyNCB3aXRoIGEgMjU2IHNpemUgdnBfdmRwYSBkZXZpY2UgYXQgbGVhc3Q/CgoKPgo+ IElmIEkgcnVuIHRoYXQgZXhhbXBsZSB3aXRoIHZwX3ZkcGEsIEwwIHFlbXUgd2lsbCBoYXBwaWx5 IGFjY2VwdCAxMDI0Cj4gYXMgYSBxdWV1ZSBzaXplIFsyXS4gQnV0IGlmIHRoZSB2ZHBhIGRldmlj ZSBtYXhpbXVtIHF1ZXVlIHNpemUgaXMKPiBlZmZlY3RpdmVseSAyNTYsIHRoaXMgd2lsbCByZXN1 bHQgaW4gYW4gZXJyb3I6IFdlJ3JlIG5vdCBleHBvc2luZyBpdAo+IHRvIHRoZSBndWVzdCBhdCBh bnkgbW9tZW50IGJ1dCB3aXRoIHFlbXUncyBjbWRsaW5lLgo+Cj4+PiBpbmNsdWRpbmcgMjU2Lgo+ Pj4gSXMgdGhhdCB3aGF0IHlvdSBtZWFuPwo+Pgo+PiBJIG1lYW4sIGl0IGxvb2tzIHRvIG1lIHRo ZSBsb2dpYyB3aWxsIGJlIG11Y2ggbW9yZSBzaW1wbGlmaWVkIGlmIHdlIGp1c3QKPj4gYWxsb2Nh dGUgdGhlIHNoYWRvdyB2aXJ0cXVldWUgd2l0aCB0aGUgc2l6ZSB3aGF0IGd1ZXN0IGNhbiBzZWUg KGd1ZXN0Cj4+IHZyaW5nKS4KPj4KPj4gVGhlbiB3ZSBkb24ndCBuZWVkIHRvIHRoaW5rIGlmIHRo ZSBkaWZmZXJlbmNlIG9mIHRoZSBxdWV1ZSBzaXplIGNhbiBoYXZlCj4+IGFueSBzaWRlIGVmZmVj dHMuCj4+Cj4gSSB0aGluayB0aGF0IHdlIGNhbm5vdCBhdm9pZCB0aGF0IGV4dHJhIGxvZ2ljIHVu bGVzcyB3ZSBmb3JjZSBHUEEgdG8KPiBiZSBjb250aWd1b3VzIGluIElPVkEuIElmIHdlIGFyZSBz dXJlIHRoZSBndWVzdCdzIGJ1ZmZlcnMgY2Fubm90IGJlIGF0Cj4gbW9yZSB0aGFuIG9uZSBkZXNj cmlwdG9yIGluIFNWUSwgdGhlbiB5ZXMsIHdlIGNhbiBzaW1wbGlmeSB0aGluZ3MuIElmCj4gbm90 LCBJIHRoaW5rIHdlIGFyZSBmb3JjZWQgdG8gY2FycnkgYWxsIG9mIGl0LgoKClllcywgSSBhZ3Jl ZSwgdGhlIGNvZGUgc2hvdWxkIGJlIHJvYnVzdCB0byBoYW5kbGUgYW55IGNhc2UuCgpUaGFua3MK Cgo+Cj4gQnV0IGlmIHdlIHByb3ZlIGl0IEknbSBub3Qgb3Bwb3NlZCB0byBzaW1wbGlmeWluZyB0 aGluZ3MgYW5kIG1ha2luZwo+IGhlYWQgYXQgU1ZRID09IGhlYWQgYXQgZ3Vlc3QuCj4KPiBUaGFu a3MhCj4KPiBbMV0gaHR0cHM6Ly9naXRsYWIuY29tL3FlbXUtcHJvamVjdC9xZW11Ly0vYmxvYi8x N2UzMTM0MC9ody92aXJ0aW8vdmlydGlvLmMjTDEyOTcKPiBbMl0gQnV0IHRoYXQncyBub3QgdGhl IHdob2xlIHN0b3J5OiBJJ3ZlIGJlZW4gcnVubmluZyBsaW1pdGVkIGluIHR4Cj4gZGVzY3JpcHRv cnMgYmVjYXVzZSBvZiB2aXJ0aW9fbmV0X21heF90eF9xdWV1ZV9zaXplLCB3aGljaCBwcmVkYXRl cwo+IHZkcGEuIEknbGwgc2VuZCBhIHBhdGNoIHRvIGFsc28gdW4tbGltaXQgaXQuCj4KPj4+IElm IHdpdGggaGFyZHdhcmUgcXVldWVzIHlvdSBtZWFuIGd1ZXN0J3MgdnJpbmcsIG5vdCBzdXJlIHdo eSBpdCBpcwo+Pj4gInByb2JhYmx5IDI1NiIuIEknZCBzYXkgdGhhdCBpbiB0aGF0IGNhc2Ugd2l0 aCB0aGUgdmlydGlvLW5ldCBrZXJuZWwKPj4+IGRyaXZlciB0aGUgcmluZyBzaXplIHdpbGwgYmUg dGhlIHNhbWUgYXMgdGhlIGRldmljZSBleHBvcnQsIGZvcgo+Pj4gZXhhbXBsZSwgaXNuJ3QgaXQ/ Cj4+Pgo+Pj4gVGhlIGltcGxlbWVudGF0aW9uIHNob3VsZCBzdXBwb3J0IGFueSBjb21iaW5hdGlv biBvZiBzaXplcywgYnV0IHRoZQo+Pj4gcmluZyBzaXplIGV4cG9zZWQgdG8gdGhlIGd1ZXN0IGlz IG5ldmVyIGJpZ2dlciB0aGFuIGhhcmR3YXJlIG9uZS4KPj4+Cj4+Pj4gPyBPciB3ZSBTVlEgY2Fu IHN0aWNrIHRvIDI1NiBidXQgdGhpcyB3aWxsIHRoaXMgY2F1c2UgdHJvdWJsZSBpZiB3ZSB3YW50 Cj4+Pj4gdG8gYWRkIGV2ZW50IGluZGV4IHN1cHBvcnQ/Cj4+Pj4KPj4+IEkgdGhpbmsgd2Ugc2hv dWxkIG5vdCBoYXZlIGFueSBwcm9ibGVtIHdpdGggZXZlbnQgaWR4LiBJZiB5b3UgbWVhbgo+Pj4g dGhhdCB0aGUgZ3Vlc3QgY291bGQgbWFyayBtb3JlIGJ1ZmZlcnMgYXZhaWxhYmxlIHRoYW4gU1ZR IHZyaW5nJ3MKPj4+IHNpemUsIHRoYXQgc2hvdWxkIG5vdCBoYXBwZW4gYmVjYXVzZSB0aGVyZSBt dXN0IGJlIGxlc3MgZW50cmllcyBpbiB0aGUKPj4+IGd1ZXN0IHRoYW4gU1ZRLgo+Pj4KPj4+IEJ1 dCBpZiBJIHVuZGVyc3Rvb2QgeW91IGNvcnJlY3RseSwgYSBzaW1pbGFyIHNpdHVhdGlvbiBjb3Vs ZCBoYXBwZW4gaWYKPj4+IGEgZ3Vlc3QncyBjb250aWd1b3VzIGJ1ZmZlciBpcyBzY2F0dGVyZWQg YWNyb3NzIG1hbnkgcWVtdSdzIFZBIGNodW5rcy4KPj4+IEV2ZW4gaWYgdGhhdCB3b3VsZCBoYXBw ZW4sIHRoZSBzaXR1YXRpb24gc2hvdWxkIGJlIG9rIHRvbzogU1ZRIGtub3dzCj4+PiB0aGUgZ3Vl c3QncyBhdmFpbCBpZHggYW5kLCBpZiBTVlEgaXMgZnVsbCwgaXQgd2lsbCBjb250aW51ZSBmb3J3 YXJkaW5nCj4+PiBhdmFpbCBidWZmZXJzIHdoZW4gdGhlIGRldmljZSB1c2VzIG1vcmUgYnVmZmVy cy4KPj4+Cj4+PiBEb2VzIHRoYXQgbWFrZSBzZW5zZSB0byB5b3U/Cj4+Cj4+IFllcy4KPj4KPj4g VGhhbmtzCj4+CgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f XwpWaXJ0dWFsaXphdGlvbiBtYWlsaW5nIGxpc3QKVmlydHVhbGl6YXRpb25AbGlzdHMubGludXgt Zm91bmRhdGlvbi5vcmcKaHR0cHM6Ly9saXN0cy5saW51eGZvdW5kYXRpb24ub3JnL21haWxtYW4v bGlzdGluZm8vdmlydHVhbGl6YXRpb24= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BB8C6C433F5 for ; Mon, 21 Feb 2022 07:21:39 +0000 (UTC) Received: from localhost ([::1]:34304 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nM30s-0001fJ-So for qemu-devel@archiver.kernel.org; Mon, 21 Feb 2022 02:21:38 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48636) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nM2uo-000695-R8 for qemu-devel@nongnu.org; Mon, 21 Feb 2022 02:15:22 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:56188) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nM2um-0007tG-KP for qemu-devel@nongnu.org; Mon, 21 Feb 2022 02:15:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1645427719; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jrN8XsW/BOgueqLR4Y6t/TA6fNns23B14ddLmdsBcG0=; b=RsHoMm9Z06XHCfOaugtYdKqR8Uc4FEoTzdScJh0YyrQMLAPOXX7g38bcjq/3XnxKM9+NUq D84YLvEO43yDbt6Ec2NQrRzyDWnpC+SuaE+BgYfMK24kS+8nJ8o2maY3PH0ItjcyO2ysS5 0tvwCzQBgsmpX6UadLWqjBuQrL9oSWo= Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-288-WrJjN9pPMIaLhsHap-wBOw-1; Mon, 21 Feb 2022 02:15:17 -0500 X-MC-Unique: WrJjN9pPMIaLhsHap-wBOw-1 Received: by mail-pf1-f197.google.com with SMTP id y28-20020aa793dc000000b004e160274e3eso4509037pff.18 for ; Sun, 20 Feb 2022 23:15:17 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=jrN8XsW/BOgueqLR4Y6t/TA6fNns23B14ddLmdsBcG0=; b=TPb55TzZIJRyF95EY34HQv+azKROyl+zd68nXqZyoA+pbGt43lBq9bPOMBWXO1OXyi xukXEGBh7bF0lOHDj5qmejxPxJYBJ6Wg3gh5pQNvLJEsEYkSvghEJjhnSLWYs10UpPgV CGgbwkNYLsOpl943bpC8U4nCYkIzo8+mmk781qQUdA3y1OCN4Ngsfx3vjSqhiTAG/7HZ WJ0wIL7meIth5vqwlW3ZYzhuTpGoP3fUearkCz1Nqx7qEUKs1yAL2QGAFLSP8tL/XOHt JoXE0JLpcZeebZZZN4LZRtQuBa+EwqvfqLxhhrrR7eskbnFC1ujX2j3S0tX0IwzU9+Ng 6jZg== X-Gm-Message-State: AOAM531JN20D1bCC3JrSGVEVH+42KE41UWt4w7C7zcHIzK/4U2qdgNQc Jm4WkQe7GyC0DXRwKAJ17zAGLuOWLweto72b2pgxJbJ475594TqymkX6YafDEwbkQdLt1fZ5Ef2 4IyWxwI5RTK3CrV4= X-Received: by 2002:a17:902:d642:b0:14c:e9a7:d542 with SMTP id y2-20020a170902d64200b0014ce9a7d542mr17838073plh.38.1645427716057; Sun, 20 Feb 2022 23:15:16 -0800 (PST) X-Google-Smtp-Source: ABdhPJwXEdYXXygo2Bn/aG8AOxUtXK0uqh288rJiPaDSm5yyd3B2kkBgqHyxfMkmGU8ykvmhchguIg== X-Received: by 2002:a17:902:d642:b0:14c:e9a7:d542 with SMTP id y2-20020a170902d64200b0014ce9a7d542mr17838032plh.38.1645427715684; Sun, 20 Feb 2022 23:15:15 -0800 (PST) Received: from [10.72.12.96] ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id i22sm959549pjz.3.2022.02.20.23.15.06 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 20 Feb 2022 23:15:15 -0800 (PST) Message-ID: <02f29b62-6656-ba2f-1475-251b16e0e978@redhat.com> Date: Mon, 21 Feb 2022 15:15:03 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.6.0 Subject: Re: [PATCH 17/31] vdpa: adapt vhost_ops callbacks to svq To: Eugenio Perez Martin References: <20220121202733.404989-1-eperezma@redhat.com> <20220121202733.404989-18-eperezma@redhat.com> <82b8c3bf-1b11-86c7-4fad-294f5ccf1278@redhat.com> <3d0dfaaa-a67c-6f48-fd03-45d2661ba92a@redhat.com> From: Jason Wang In-Reply-To: Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=170.10.133.124; envelope-from=jasowang@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , Cindy Lu , "Michael S. Tsirkin" , Juan Quintela , Richard Henderson , qemu-level , Gautam Dawar , Markus Armbruster , Eduardo Habkost , Harpreet Singh Anand , Xiao W Wang , Peter Xu , Stefan Hajnoczi , Eli Cohen , Paolo Bonzini , Zhu Lingshan , virtualization , Eric Blake , Stefano Garzarella Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" 在 2022/2/18 上午1:13, Eugenio Perez Martin 写道: > On Tue, Feb 8, 2022 at 4:58 AM Jason Wang wrote: >> >> 在 2022/2/1 上午2:58, Eugenio Perez Martin 写道: >>> On Sun, Jan 30, 2022 at 5:03 AM Jason Wang wrote: >>>> 在 2022/1/22 上午4:27, Eugenio Pérez 写道: >>>>> First half of the buffers forwarding part, preparing vhost-vdpa >>>>> callbacks to SVQ to offer it. QEMU cannot enable it at this moment, so >>>>> this is effectively dead code at the moment, but it helps to reduce >>>>> patch size. >>>>> >>>>> Signed-off-by: Eugenio Pérez >>>>> --- >>>>> hw/virtio/vhost-shadow-virtqueue.h | 2 +- >>>>> hw/virtio/vhost-shadow-virtqueue.c | 21 ++++- >>>>> hw/virtio/vhost-vdpa.c | 133 ++++++++++++++++++++++++++--- >>>>> 3 files changed, 143 insertions(+), 13 deletions(-) >>>>> >>>>> diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h >>>>> index 035207a469..39aef5ffdf 100644 >>>>> --- a/hw/virtio/vhost-shadow-virtqueue.h >>>>> +++ b/hw/virtio/vhost-shadow-virtqueue.h >>>>> @@ -35,7 +35,7 @@ size_t vhost_svq_device_area_size(const VhostShadowVirtqueue *svq); >>>>> >>>>> void vhost_svq_stop(VhostShadowVirtqueue *svq); >>>>> >>>>> -VhostShadowVirtqueue *vhost_svq_new(void); >>>>> +VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize); >>>>> >>>>> void vhost_svq_free(VhostShadowVirtqueue *vq); >>>>> >>>>> diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c >>>>> index f129ec8395..7c168075d7 100644 >>>>> --- a/hw/virtio/vhost-shadow-virtqueue.c >>>>> +++ b/hw/virtio/vhost-shadow-virtqueue.c >>>>> @@ -277,9 +277,17 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) >>>>> /** >>>>> * Creates vhost shadow virtqueue, and instruct vhost device to use the shadow >>>>> * methods and file descriptors. >>>>> + * >>>>> + * @qsize Shadow VirtQueue size >>>>> + * >>>>> + * Returns the new virtqueue or NULL. >>>>> + * >>>>> + * In case of error, reason is reported through error_report. >>>>> */ >>>>> -VhostShadowVirtqueue *vhost_svq_new(void) >>>>> +VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize) >>>>> { >>>>> + size_t desc_size = sizeof(vring_desc_t) * qsize; >>>>> + size_t device_size, driver_size; >>>>> g_autofree VhostShadowVirtqueue *svq = g_new0(VhostShadowVirtqueue, 1); >>>>> int r; >>>>> >>>>> @@ -300,6 +308,15 @@ VhostShadowVirtqueue *vhost_svq_new(void) >>>>> /* Placeholder descriptor, it should be deleted at set_kick_fd */ >>>>> event_notifier_init_fd(&svq->svq_kick, INVALID_SVQ_KICK_FD); >>>>> >>>>> + svq->vring.num = qsize; >>>> I wonder if this is the best. E.g some hardware can support up to 32K >>>> queue size. So this will probably end up with: >>>> >>>> 1) SVQ use 32K queue size >>>> 2) hardware queue uses 256 >>>> >>> In that case SVQ vring queue size will be 32K and guest's vring can >>> negotiate any number with SVQ equal or less than 32K, >> >> Sorry for being unclear what I meant is actually >> >> 1) SVQ uses 32K queue size >> >> 2) guest vq uses 256 >> >> This looks like a burden that needs extra logic and may damage the >> performance. >> > Still not getting this point. > > An available guest buffer, although contiguous in GPA/GVA, can expand > in multiple buffers if it's not contiguous in qemu's VA (by the while > loop in virtqueue_map_desc [1]). In that scenario it is better to have > "plenty" of SVQ buffers. Yes, but this case should be rare. So in this case we should deal with overrun on SVQ, that is 1) SVQ is full 2) guest VQ isn't We need to 1) check the available buffer slots 2) disable guest kick and wait for the used buffers But it looks to me the current code is not ready for dealing with this case? > > I'm ok if we decide to put an upper limit though, or if we decide not > to handle this situation. But we would leave out valid virtio drivers. > Maybe to set a fixed upper limit (1024?)? To add another parameter > (x-svq-size-n=N)? > > If you mean we lose performance because memory gets more sparse I > think the only possibility is to limit that way. If guest is not using 32K, having a 32K for svq may gives extra stress on the cache since we will end up with a pretty large working set. > >> And this can lead other interesting situation: >> >> 1) SVQ uses 256 >> >> 2) guest vq uses 1024 >> >> Where a lot of more SVQ logic is needed. >> > If we agree that a guest descriptor can expand in multiple SVQ > descriptors, this should be already handled by the previous logic too. > > But this should only happen in case that qemu is launched with a "bad" > cmdline, isn't it? This seems can happen when we use -device virtio-net-pci,tx_queue_size=1024 with a 256 size vp_vdpa device at least? > > If I run that example with vp_vdpa, L0 qemu will happily accept 1024 > as a queue size [2]. But if the vdpa device maximum queue size is > effectively 256, this will result in an error: We're not exposing it > to the guest at any moment but with qemu's cmdline. > >>> including 256. >>> Is that what you mean? >> >> I mean, it looks to me the logic will be much more simplified if we just >> allocate the shadow virtqueue with the size what guest can see (guest >> vring). >> >> Then we don't need to think if the difference of the queue size can have >> any side effects. >> > I think that we cannot avoid that extra logic unless we force GPA to > be contiguous in IOVA. If we are sure the guest's buffers cannot be at > more than one descriptor in SVQ, then yes, we can simplify things. If > not, I think we are forced to carry all of it. Yes, I agree, the code should be robust to handle any case. Thanks > > But if we prove it I'm not opposed to simplifying things and making > head at SVQ == head at guest. > > Thanks! > > [1] https://gitlab.com/qemu-project/qemu/-/blob/17e31340/hw/virtio/virtio.c#L1297 > [2] But that's not the whole story: I've been running limited in tx > descriptors because of virtio_net_max_tx_queue_size, which predates > vdpa. I'll send a patch to also un-limit it. > >>> If with hardware queues you mean guest's vring, not sure why it is >>> "probably 256". I'd say that in that case with the virtio-net kernel >>> driver the ring size will be the same as the device export, for >>> example, isn't it? >>> >>> The implementation should support any combination of sizes, but the >>> ring size exposed to the guest is never bigger than hardware one. >>> >>>> ? Or we SVQ can stick to 256 but this will this cause trouble if we want >>>> to add event index support? >>>> >>> I think we should not have any problem with event idx. If you mean >>> that the guest could mark more buffers available than SVQ vring's >>> size, that should not happen because there must be less entries in the >>> guest than SVQ. >>> >>> But if I understood you correctly, a similar situation could happen if >>> a guest's contiguous buffer is scattered across many qemu's VA chunks. >>> Even if that would happen, the situation should be ok too: SVQ knows >>> the guest's avail idx and, if SVQ is full, it will continue forwarding >>> avail buffers when the device uses more buffers. >>> >>> Does that make sense to you? >> >> Yes. >> >> Thanks >>