From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 90FA2C433EF for ; Tue, 22 Feb 2022 03:16:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id F39DC408E7; Tue, 22 Feb 2022 03:16:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qI652JhjMn3V; Tue, 22 Feb 2022 03:16:44 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 0D069408DF; Tue, 22 Feb 2022 03:16:43 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D9A76C001A; Tue, 22 Feb 2022 03:16:43 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 231A5C0011 for ; Tue, 22 Feb 2022 03:16:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 10FEC60AD7 for ; Tue, 22 Feb 2022 03:16:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id plzmafRkDeBZ for ; Tue, 22 Feb 2022 03:16:40 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id A0D7F60AC2 for ; Tue, 22 Feb 2022 03:16:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1645499798; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gHY8uuapAXsf5KULdO4mEh+Xne/2fBaTYxBgL4rXrS8=; b=NzCWF168EQDkJYuu0TSDcatQPI6yJmp+gqq0kmw8NyX0r8QhnHdtmqx2XRb8s+DPwiHxe9 hN7b38+1Eyt7zK0MlkHfPe/JrOL5y3aqlDzKoWqY3qMsYW+e2oXxxeWoPxyPwFS3PHCjPK /JPMIZ3ehSInX1TXJWZP8aBnWPD0HKQ= Received: from mail-lj1-f200.google.com (mail-lj1-f200.google.com [209.85.208.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-354-nFh8q42cPhyF87T2w8UGtQ-1; Mon, 21 Feb 2022 22:16:36 -0500 X-MC-Unique: nFh8q42cPhyF87T2w8UGtQ-1 Received: by mail-lj1-f200.google.com with SMTP id q17-20020a2e7511000000b0023c95987502so4988510ljc.16 for ; Mon, 21 Feb 2022 19:16:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=gHY8uuapAXsf5KULdO4mEh+Xne/2fBaTYxBgL4rXrS8=; b=A7Q4YGdm/rmC4jrekktN/POD4+XfuOiZOh02IB/rL/XbVnCy91y8RsPg9Rj8069prW YbCieW6hrodvpGRw3Z93j160Mr0RL1WAZ0OM8hqFbtVZUyF09AhIII+5xN/8rqpGlkV2 ZCtQ7NYvD/jxXmCq86fchxh/t8CXN+WU3JAjW8RznA9koRFeI8dnMpDioGGTOq4R0D/K 4dXd24eNTnxamAYauO3pl1Hol9YgSwSi0ghNIDDOF+pK9Inizp3RotrMiHotdQ/400P8 l+2u/P/HO/lSMdI0Fr9nD9KpqrDGCiu4rcPVzO5+XlQNN2K0x20on6aC8XeIVyV1LWbk YOhQ== X-Gm-Message-State: AOAM533BOFZEDreSoAHKqXIffUUw6vE76Zu0g+hW1s3fe9Xa76tYtJLc Y2BdguAVMiK7Vmr11zk8PNiDujlstBe1a4EKdhWAud6gawdDv38vc/rH2XXxzKEHSPWtdTlDO9o RE0i6pNbnB5huDBX9BwFIPklrokEbp1xkRhoeCO8LTcBi2mdw7GN44tDG+A== X-Received: by 2002:ac2:4194:0:b0:442:ed9e:4a25 with SMTP id z20-20020ac24194000000b00442ed9e4a25mr15714980lfh.629.1645499795320; Mon, 21 Feb 2022 19:16:35 -0800 (PST) X-Google-Smtp-Source: ABdhPJzFQXM4KXylMKw7WbdxhpVZyCLXX/jI0XWCwc7r24OEqocGHtKuqW/T+EEktOtw/EgdVyRb88GzMPXSR/TEEqw= X-Received: by 2002:ac2:4194:0:b0:442:ed9e:4a25 with SMTP id z20-20020ac24194000000b00442ed9e4a25mr15714952lfh.629.1645499795020; Mon, 21 Feb 2022 19:16:35 -0800 (PST) MIME-Version: 1.0 References: <20220121202733.404989-1-eperezma@redhat.com> <20220121202733.404989-18-eperezma@redhat.com> <82b8c3bf-1b11-86c7-4fad-294f5ccf1278@redhat.com> <3d0dfaaa-a67c-6f48-fd03-45d2661ba92a@redhat.com> <02f29b62-6656-ba2f-1475-251b16e0e978@redhat.com> In-Reply-To: From: Jason Wang Date: Tue, 22 Feb 2022 11:16:23 +0800 Message-ID: Subject: Re: [PATCH 17/31] vdpa: adapt vhost_ops callbacks to svq To: Eugenio Perez Martin Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: Laurent Vivier , Parav Pandit , Cindy Lu , "Michael S. Tsirkin" , Richard Henderson , qemu-level , Gautam Dawar , Markus Armbruster , Eduardo Habkost , Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Paolo Bonzini , Zhu Lingshan , virtualization , Eric Blake X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" T24gVHVlLCBGZWIgMjIsIDIwMjIgYXQgMToyMyBBTSBFdWdlbmlvIFBlcmV6IE1hcnRpbgo8ZXBl cmV6bWFAcmVkaGF0LmNvbT4gd3JvdGU6Cj4KPiBPbiBNb24sIEZlYiAyMSwgMjAyMiBhdCA4OjE1 IEFNIEphc29uIFdhbmcgPGphc293YW5nQHJlZGhhdC5jb20+IHdyb3RlOgo+ID4KPiA+Cj4gPiDl nKggMjAyMi8yLzE4IOS4iuWNiDE6MTMsIEV1Z2VuaW8gUGVyZXogTWFydGluIOWGmemBkzoKPiA+ ID4gT24gVHVlLCBGZWIgOCwgMjAyMiBhdCA0OjU4IEFNIEphc29uIFdhbmcgPGphc293YW5nQHJl ZGhhdC5jb20+IHdyb3RlOgo+ID4gPj4KPiA+ID4+IOWcqCAyMDIyLzIvMSDkuIrljYgyOjU4LCBF dWdlbmlvIFBlcmV6IE1hcnRpbiDlhpnpgZM6Cj4gPiA+Pj4gT24gU3VuLCBKYW4gMzAsIDIwMjIg YXQgNTowMyBBTSBKYXNvbiBXYW5nIDxqYXNvd2FuZ0ByZWRoYXQuY29tPiB3cm90ZToKPiA+ID4+ Pj4g5ZyoIDIwMjIvMS8yMiDkuIrljYg0OjI3LCBFdWdlbmlvIFDDqXJleiDlhpnpgZM6Cj4gPiA+ Pj4+PiBGaXJzdCBoYWxmIG9mIHRoZSBidWZmZXJzIGZvcndhcmRpbmcgcGFydCwgcHJlcGFyaW5n IHZob3N0LXZkcGEKPiA+ID4+Pj4+IGNhbGxiYWNrcyB0byBTVlEgdG8gb2ZmZXIgaXQuIFFFTVUg Y2Fubm90IGVuYWJsZSBpdCBhdCB0aGlzIG1vbWVudCwgc28KPiA+ID4+Pj4+IHRoaXMgaXMgZWZm ZWN0aXZlbHkgZGVhZCBjb2RlIGF0IHRoZSBtb21lbnQsIGJ1dCBpdCBoZWxwcyB0byByZWR1Y2UK PiA+ID4+Pj4+IHBhdGNoIHNpemUuCj4gPiA+Pj4+Pgo+ID4gPj4+Pj4gU2lnbmVkLW9mZi1ieTog RXVnZW5pbyBQw6lyZXogPGVwZXJlem1hQHJlZGhhdC5jb20+Cj4gPiA+Pj4+PiAtLS0KPiA+ID4+ Pj4+ICAgICBody92aXJ0aW8vdmhvc3Qtc2hhZG93LXZpcnRxdWV1ZS5oIHwgICAyICstCj4gPiA+ Pj4+PiAgICAgaHcvdmlydGlvL3Zob3N0LXNoYWRvdy12aXJ0cXVldWUuYyB8ICAyMSArKysrLQo+ ID4gPj4+Pj4gICAgIGh3L3ZpcnRpby92aG9zdC12ZHBhLmMgICAgICAgICAgICAgfCAxMzMgKysr KysrKysrKysrKysrKysrKysrKysrKystLS0KPiA+ID4+Pj4+ICAgICAzIGZpbGVzIGNoYW5nZWQs IDE0MyBpbnNlcnRpb25zKCspLCAxMyBkZWxldGlvbnMoLSkKPiA+ID4+Pj4+Cj4gPiA+Pj4+PiBk aWZmIC0tZ2l0IGEvaHcvdmlydGlvL3Zob3N0LXNoYWRvdy12aXJ0cXVldWUuaCBiL2h3L3ZpcnRp by92aG9zdC1zaGFkb3ctdmlydHF1ZXVlLmgKPiA+ID4+Pj4+IGluZGV4IDAzNTIwN2E0NjkuLjM5 YWVmNWZmZGYgMTAwNjQ0Cj4gPiA+Pj4+PiAtLS0gYS9ody92aXJ0aW8vdmhvc3Qtc2hhZG93LXZp cnRxdWV1ZS5oCj4gPiA+Pj4+PiArKysgYi9ody92aXJ0aW8vdmhvc3Qtc2hhZG93LXZpcnRxdWV1 ZS5oCj4gPiA+Pj4+PiBAQCAtMzUsNyArMzUsNyBAQCBzaXplX3Qgdmhvc3Rfc3ZxX2RldmljZV9h cmVhX3NpemUoY29uc3QgVmhvc3RTaGFkb3dWaXJ0cXVldWUgKnN2cSk7Cj4gPiA+Pj4+Pgo+ID4g Pj4+Pj4gICAgIHZvaWQgdmhvc3Rfc3ZxX3N0b3AoVmhvc3RTaGFkb3dWaXJ0cXVldWUgKnN2cSk7 Cj4gPiA+Pj4+Pgo+ID4gPj4+Pj4gLVZob3N0U2hhZG93VmlydHF1ZXVlICp2aG9zdF9zdnFfbmV3 KHZvaWQpOwo+ID4gPj4+Pj4gK1Zob3N0U2hhZG93VmlydHF1ZXVlICp2aG9zdF9zdnFfbmV3KHVp bnQxNl90IHFzaXplKTsKPiA+ID4+Pj4+Cj4gPiA+Pj4+PiAgICAgdm9pZCB2aG9zdF9zdnFfZnJl ZShWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAqdnEpOwo+ID4gPj4+Pj4KPiA+ID4+Pj4+IGRpZmYgLS1n aXQgYS9ody92aXJ0aW8vdmhvc3Qtc2hhZG93LXZpcnRxdWV1ZS5jIGIvaHcvdmlydGlvL3Zob3N0 LXNoYWRvdy12aXJ0cXVldWUuYwo+ID4gPj4+Pj4gaW5kZXggZjEyOWVjODM5NS4uN2MxNjgwNzVk NyAxMDA2NDQKPiA+ID4+Pj4+IC0tLSBhL2h3L3ZpcnRpby92aG9zdC1zaGFkb3ctdmlydHF1ZXVl LmMKPiA+ID4+Pj4+ICsrKyBiL2h3L3ZpcnRpby92aG9zdC1zaGFkb3ctdmlydHF1ZXVlLmMKPiA+ ID4+Pj4+IEBAIC0yNzcsOSArMjc3LDE3IEBAIHZvaWQgdmhvc3Rfc3ZxX3N0b3AoVmhvc3RTaGFk b3dWaXJ0cXVldWUgKnN2cSkKPiA+ID4+Pj4+ICAgICAvKioKPiA+ID4+Pj4+ICAgICAgKiBDcmVh dGVzIHZob3N0IHNoYWRvdyB2aXJ0cXVldWUsIGFuZCBpbnN0cnVjdCB2aG9zdCBkZXZpY2UgdG8g dXNlIHRoZSBzaGFkb3cKPiA+ID4+Pj4+ICAgICAgKiBtZXRob2RzIGFuZCBmaWxlIGRlc2NyaXB0 b3JzLgo+ID4gPj4+Pj4gKyAqCj4gPiA+Pj4+PiArICogQHFzaXplIFNoYWRvdyBWaXJ0UXVldWUg c2l6ZQo+ID4gPj4+Pj4gKyAqCj4gPiA+Pj4+PiArICogUmV0dXJucyB0aGUgbmV3IHZpcnRxdWV1 ZSBvciBOVUxMLgo+ID4gPj4+Pj4gKyAqCj4gPiA+Pj4+PiArICogSW4gY2FzZSBvZiBlcnJvciwg cmVhc29uIGlzIHJlcG9ydGVkIHRocm91Z2ggZXJyb3JfcmVwb3J0Lgo+ID4gPj4+Pj4gICAgICAq Lwo+ID4gPj4+Pj4gLVZob3N0U2hhZG93VmlydHF1ZXVlICp2aG9zdF9zdnFfbmV3KHZvaWQpCj4g PiA+Pj4+PiArVmhvc3RTaGFkb3dWaXJ0cXVldWUgKnZob3N0X3N2cV9uZXcodWludDE2X3QgcXNp emUpCj4gPiA+Pj4+PiAgICAgewo+ID4gPj4+Pj4gKyAgICBzaXplX3QgZGVzY19zaXplID0gc2l6 ZW9mKHZyaW5nX2Rlc2NfdCkgKiBxc2l6ZTsKPiA+ID4+Pj4+ICsgICAgc2l6ZV90IGRldmljZV9z aXplLCBkcml2ZXJfc2l6ZTsKPiA+ID4+Pj4+ICAgICAgICAgZ19hdXRvZnJlZSBWaG9zdFNoYWRv d1ZpcnRxdWV1ZSAqc3ZxID0gZ19uZXcwKFZob3N0U2hhZG93VmlydHF1ZXVlLCAxKTsKPiA+ID4+ Pj4+ICAgICAgICAgaW50IHI7Cj4gPiA+Pj4+Pgo+ID4gPj4+Pj4gQEAgLTMwMCw2ICszMDgsMTUg QEAgVmhvc3RTaGFkb3dWaXJ0cXVldWUgKnZob3N0X3N2cV9uZXcodm9pZCkKPiA+ID4+Pj4+ICAg ICAgICAgLyogUGxhY2Vob2xkZXIgZGVzY3JpcHRvciwgaXQgc2hvdWxkIGJlIGRlbGV0ZWQgYXQg c2V0X2tpY2tfZmQgKi8KPiA+ID4+Pj4+ICAgICAgICAgZXZlbnRfbm90aWZpZXJfaW5pdF9mZCgm c3ZxLT5zdnFfa2ljaywgSU5WQUxJRF9TVlFfS0lDS19GRCk7Cj4gPiA+Pj4+Pgo+ID4gPj4+Pj4g KyAgICBzdnEtPnZyaW5nLm51bSA9IHFzaXplOwo+ID4gPj4+PiBJIHdvbmRlciBpZiB0aGlzIGlz IHRoZSBiZXN0LiBFLmcgc29tZSBoYXJkd2FyZSBjYW4gc3VwcG9ydCB1cCB0byAzMksKPiA+ID4+ Pj4gcXVldWUgc2l6ZS4gU28gdGhpcyB3aWxsIHByb2JhYmx5IGVuZCB1cCB3aXRoOgo+ID4gPj4+ Pgo+ID4gPj4+PiAxKSBTVlEgdXNlIDMySyBxdWV1ZSBzaXplCj4gPiA+Pj4+IDIpIGhhcmR3YXJl IHF1ZXVlIHVzZXMgMjU2Cj4gPiA+Pj4+Cj4gPiA+Pj4gSW4gdGhhdCBjYXNlIFNWUSB2cmluZyBx dWV1ZSBzaXplIHdpbGwgYmUgMzJLIGFuZCBndWVzdCdzIHZyaW5nIGNhbgo+ID4gPj4+IG5lZ290 aWF0ZSBhbnkgbnVtYmVyIHdpdGggU1ZRIGVxdWFsIG9yIGxlc3MgdGhhbiAzMkssCj4gPiA+Pgo+ ID4gPj4gU29ycnkgZm9yIGJlaW5nIHVuY2xlYXIgd2hhdCBJIG1lYW50IGlzIGFjdHVhbGx5Cj4g PiA+Pgo+ID4gPj4gMSkgU1ZRIHVzZXMgMzJLIHF1ZXVlIHNpemUKPiA+ID4+Cj4gPiA+PiAyKSBn dWVzdCB2cSB1c2VzIDI1Ngo+ID4gPj4KPiA+ID4+IFRoaXMgbG9va3MgbGlrZSBhIGJ1cmRlbiB0 aGF0IG5lZWRzIGV4dHJhIGxvZ2ljIGFuZCBtYXkgZGFtYWdlIHRoZQo+ID4gPj4gcGVyZm9ybWFu Y2UuCj4gPiA+Pgo+ID4gPiBTdGlsbCBub3QgZ2V0dGluZyB0aGlzIHBvaW50Lgo+ID4gPgo+ID4g PiBBbiBhdmFpbGFibGUgZ3Vlc3QgYnVmZmVyLCBhbHRob3VnaCBjb250aWd1b3VzIGluIEdQQS9H VkEsIGNhbiBleHBhbmQKPiA+ID4gaW4gbXVsdGlwbGUgYnVmZmVycyBpZiBpdCdzIG5vdCBjb250 aWd1b3VzIGluIHFlbXUncyBWQSAoYnkgdGhlIHdoaWxlCj4gPiA+IGxvb3AgaW4gdmlydHF1ZXVl X21hcF9kZXNjIFsxXSkuIEluIHRoYXQgc2NlbmFyaW8gaXQgaXMgYmV0dGVyIHRvIGhhdmUKPiA+ ID4gInBsZW50eSIgb2YgU1ZRIGJ1ZmZlcnMuCj4gPgo+ID4KPiA+IFllcywgYnV0IHRoaXMgY2Fz ZSBzaG91bGQgYmUgcmFyZS4gU28gaW4gdGhpcyBjYXNlIHdlIHNob3VsZCBkZWFsIHdpdGgKPiA+ IG92ZXJydW4gb24gU1ZRLCB0aGF0IGlzCj4gPgo+ID4gMSkgU1ZRIGlzIGZ1bGwKPiA+IDIpIGd1 ZXN0IFZRIGlzbid0Cj4gPgo+ID4gV2UgbmVlZCB0bwo+ID4KPiA+IDEpIGNoZWNrIHRoZSBhdmFp bGFibGUgYnVmZmVyIHNsb3RzCj4gPiAyKSBkaXNhYmxlIGd1ZXN0IGtpY2sgYW5kIHdhaXQgZm9y IHRoZSB1c2VkIGJ1ZmZlcnMKPiA+Cj4gPiBCdXQgaXQgbG9va3MgdG8gbWUgdGhlIGN1cnJlbnQg Y29kZSBpcyBub3QgcmVhZHkgZm9yIGRlYWxpbmcgd2l0aCB0aGlzIGNhc2U/Cj4gPgo+Cj4gWWVz IGl0IGRlYWxzLCB0aGF0J3MgdGhlIG1lYW5pbmcgb2Ygc3ZxLT5uZXh0X2d1ZXN0X2F2YWlsX2Vs ZW0uCgpPaCByaWdodCwgSSBtaXNzZWQgdGhhdC4KCj4KPiA+Cj4gPiA+Cj4gPiA+IEknbSBvayBp ZiB3ZSBkZWNpZGUgdG8gcHV0IGFuIHVwcGVyIGxpbWl0IHRob3VnaCwgb3IgaWYgd2UgZGVjaWRl IG5vdAo+ID4gPiB0byBoYW5kbGUgdGhpcyBzaXR1YXRpb24uIEJ1dCB3ZSB3b3VsZCBsZWF2ZSBv dXQgdmFsaWQgdmlydGlvIGRyaXZlcnMuCj4gPiA+IE1heWJlIHRvIHNldCBhIGZpeGVkIHVwcGVy IGxpbWl0ICgxMDI0Pyk/IFRvIGFkZCBhbm90aGVyIHBhcmFtZXRlcgo+ID4gPiAoeC1zdnEtc2l6 ZS1uPU4pPwo+ID4gPgo+ID4gPiBJZiB5b3UgbWVhbiB3ZSBsb3NlIHBlcmZvcm1hbmNlIGJlY2F1 c2UgbWVtb3J5IGdldHMgbW9yZSBzcGFyc2UgSQo+ID4gPiB0aGluayB0aGUgb25seSBwb3NzaWJp bGl0eSBpcyB0byBsaW1pdCB0aGF0IHdheS4KPiA+Cj4gPgo+ID4gSWYgZ3Vlc3QgaXMgbm90IHVz aW5nIDMySywgaGF2aW5nIGEgMzJLIGZvciBzdnEgbWF5IGdpdmVzIGV4dHJhIHN0cmVzcwo+ID4g b24gdGhlIGNhY2hlIHNpbmNlIHdlIHdpbGwgZW5kIHVwIHdpdGggYSBwcmV0dHkgbGFyZ2Ugd29y a2luZyBzZXQuCj4gPgo+Cj4gVGhhdCBtaWdodCBiZSB0cnVlLiBNeSBndWVzcyBpcyB0aGF0IGl0 IHNob3VsZCBub3QgbWF0dGVyLCBzaW5jZSBTVlEKPiBhbmQgdGhlIGd1ZXN0J3MgdnJpbmcgd2ls bCBoYXZlIHRoZSBzYW1lIG51bWJlcnMgb2Ygc2NhdHRlcmVkIGJ1ZmZlcnMKPiBhbmQgdGhlIGF2 YWlsIC8gdXNlZCAvIHBhY2tlZCByaW5nIHdpbGwgYmUgY29uc3VtZWQgbW9yZSBvciBsZXNzCj4g c2VxdWVudGlhbGx5LiBCdXQgSSBoYXZlbid0IHRlc3RlZC4KPgo+IEkgdGhpbmsgaXQncyBiZXR0 ZXIgdG8gYWRkIGFuIHVwcGVyIGxpbWl0IChlaXRoZXIgZml4ZWQgb3IgaW4gdGhlCj4gcWVtdSdz IGJhY2tlbmQncyBjbWRsaW5lKSBsYXRlciBpZiB3ZSBzZWUgdGhhdCB0aGlzIGlzIGEgcHJvYmxl bS4KCkknZCBzdWdnZXN0IHVzaW5nIHRoZSBzYW1lIHNpemUgYXMgd2hhdCB0aGUgZ3Vlc3Qgc2F3 LgoKPiBBbm90aGVyIHNvbHV0aW9uIG5vdyB3b3VsZCBiZSB0byBnZXQgdGhlIG51bWJlciBmcm9t IHRoZSBmcm9udGVuZAo+IGRldmljZSBjbWRsaW5lIGluc3RlYWQgb2YgZnJvbSB0aGUgdmRwYSBk ZXZpY2UuIEknbSBvayB3aXRoIHRoYXQsIGJ1dAo+IGl0IGRvZXNuJ3QgZGVsZXRlIHRoZSBzdnEt Pm5leHRfZ3Vlc3RfYXZhaWxfZWxlbSBwcm9jZXNzaW5nLCBhbmQgaXQKPiBjb21lcyB3aXRoIGRp c2FkdmFudGFnZXMgaW4gbXkgb3Bpbmlvbi4gTW9yZSBiZWxvdy4KClJpZ2h0LCB3ZSBzaG91bGQg a2VlcCBuZXh0X2d1ZXN0X2F2YWlsX2VsZW0uIFVzaW5nIHRoZSBzYW1lIHF1ZXVlIHNpemUKaXMg YSBiYWxhbmNlIGJldHdlZW46CgoxKSB1c2luZyBuZXh0X2d1ZXN0X2F2YWlsX2VsZW0gKHJhcmUp CjIpIG5vdCBnaXZlIHRvbyBtdWNoIHN0cmVzcyBvbiB0aGUgY2FjaGUKCj4KPiA+Cj4gPiA+Cj4g PiA+PiBBbmQgdGhpcyBjYW4gbGVhZCBvdGhlciBpbnRlcmVzdGluZyBzaXR1YXRpb246Cj4gPiA+ Pgo+ID4gPj4gMSkgU1ZRIHVzZXMgMjU2Cj4gPiA+Pgo+ID4gPj4gMikgZ3Vlc3QgdnEgdXNlcyAx MDI0Cj4gPiA+Pgo+ID4gPj4gV2hlcmUgYSBsb3Qgb2YgbW9yZSBTVlEgbG9naWMgaXMgbmVlZGVk Lgo+ID4gPj4KPiA+ID4gSWYgd2UgYWdyZWUgdGhhdCBhIGd1ZXN0IGRlc2NyaXB0b3IgY2FuIGV4 cGFuZCBpbiBtdWx0aXBsZSBTVlEKPiA+ID4gZGVzY3JpcHRvcnMsIHRoaXMgc2hvdWxkIGJlIGFs cmVhZHkgaGFuZGxlZCBieSB0aGUgcHJldmlvdXMgbG9naWMgdG9vLgo+ID4gPgo+ID4gPiBCdXQg dGhpcyBzaG91bGQgb25seSBoYXBwZW4gaW4gY2FzZSB0aGF0IHFlbXUgaXMgbGF1bmNoZWQgd2l0 aCBhICJiYWQiCj4gPiA+IGNtZGxpbmUsIGlzbid0IGl0Pwo+ID4KPiA+Cj4gPiBUaGlzIHNlZW1z IGNhbiBoYXBwZW4gd2hlbiB3ZSB1c2UgLWRldmljZQo+ID4gdmlydGlvLW5ldC1wY2ksdHhfcXVl dWVfc2l6ZT0xMDI0IHdpdGggYSAyNTYgc2l6ZSB2cF92ZHBhIGRldmljZSBhdCBsZWFzdD8KPiA+ Cj4KPiBJJ20gZ29pbmcgdG8gdXNlIHRoZSByeCBxdWV1ZSBoZXJlIHNpbmNlIGl0J3MgbW9yZSBh Y2N1cmF0ZSwgdHggaGFzCj4gaXRzIG93biBsaW1pdCBzZXBhcmF0ZWx5Lgo+Cj4gSWYgd2UgdXNl IHJ4X3F1ZXVlX3NpemU9MjU2IGluIEwwIGFuZCByeF9xdWV1ZV9zaXplPTEwMjQgaW4gTDEgd2l0 aCBubwo+IFNWUSwgTDAgcWVtdSB3aWxsIGhhcHBpbHkgYWNjZXB0IDEwMjQgYXMgc2l6ZQoKSW50 ZXJlc3RpbmcsIGxvb2tzIGxpa2UgYSBidWcgKEkgZ3Vlc3MgaXQgd29ya3Mgc2luY2UgeW91IGVu YWJsZSB2aG9zdD8pOgoKUGVyIHZpcnRpby1zcGVjOgoKIiIiClF1ZXVlIFNpemUuIE9uIHJlc2V0 LCBzcGVjaWZpZXMgdGhlIG1heGltdW0gcXVldWUgc2l6ZSBzdXBwb3J0ZWQgYnkKdGhlIGRldmlj ZS4gVGhpcyBjYW4gYmUgbW9kaWZpZWQgYnkgdGhlIGRyaXZlciB0byByZWR1Y2UgbWVtb3J5CnJl cXVpcmVtZW50cy4gQSAwIG1lYW5zIHRoZSBxdWV1ZSBpcyB1bmF2YWlsYWJsZS4KIiIiCgpXZSBj YW4ndCBpbmNyZWFzZSB0aGUgcXVldWVfc2l6ZSBmcm9tIDI1NiB0byAxMDI0IGFjdHVhbGx5LiAo T25seQpkZWNyZWFzZSBpcyBhbGxvd2VkKS4KCj4gd2hlbiBMMSBxZW11IHdyaXRlcyB0aGF0Cj4g dmFsdWUgYXQgdmhvc3RfdmlydHF1ZXVlX3N0YXJ0LiBJJ20gbm90IHN1cmUgd2hhdCB3b3VsZCBo YXBwZW4gd2l0aCBhCj4gcmVhbCBkZXZpY2UsIG15IGd1ZXNzIGlzIHRoYXQgdGhlIGRldmljZSB3 aWxsIGZhaWwgc29tZWhvdy4gVGhhdCdzCj4gd2hhdCBJIG1lYW50IHdpdGggYSAiYmFkIGNtZGxp bmUiLCBJIHNob3VsZCBoYXZlIGJlZW4gbW9yZSBzcGVjaWZpYy4KCkkgc2hvdWxkIHNheSB0aGF0 IGl0J3Mgc29tZXRoaW5nIHRoYXQgaXMgcHJvYmFibHkgdW5yZWxhdGVkIHRvIHRoaXMKc2VyaWVz IGJ1dCBuZWVkcyB0byBiZSBhZGRyZXNzZWQuCgo+Cj4gSWYgd2UgYWRkIFNWUSB0byB0aGUgbWl4 LCB0aGUgZ3Vlc3QgZmlyc3QgbmVnb3RpYXRlcyB0aGUgMTAyNCB3aXRoIHRoZQo+IHFlbXUgZGV2 aWNlIG1vZGVsLiBBZnRlciB0aGF0LCB2aG9zdC5jIHdpbGwgdHJ5IHRvIHdyaXRlIDEwMjQgdG9v IGJ1dAo+IHRoaXMgaXMgdG90YWxseSBpZ25vcmVkIGJ5IHRoaXMgcGF0Y2gncyBjaGFuZ2VzIGF0 Cj4gdmhvc3RfdmRwYV9zZXRfdnJpbmdfbnVtLiBGaW5hbGx5LCBTVlEgd2lsbCBzZXQgMjU2IGFz IGEgcmluZyBzaXplIHRvCj4gdGhlIGRldmljZSwgc2luY2UgaXQncyB0aGUgcmVhZCB2YWx1ZSBm cm9tIHRoZSBkZXZpY2UsIGxlYWRpbmcgdG8geW91cgo+IHNjZW5hcmlvLiBTbyBTVlEgZWZmZWN0 aXZlbHkgaXNvbGF0ZXMgYm90aCBzaWRlcyBhbmQgbWFrZXMgcG9zc2libGUKPiB0aGUgY29tbXVu aWNhdGlvbiwgZXZlbiB3aXRoIGEgZGV2aWNlIHRoYXQgZG9lcyBub3Qgc3VwcG9ydCBzbyBtYW55 Cj4gZGVzY3JpcHRvcnMuCj4KPiBCdXQgU1ZRIGFscmVhZHkgaGFuZGxlcyB0aGlzIGNhc2U6IEl0 J3MgdGhlIHNhbWUgYXMgaWYgdGhlIGJ1ZmZlcnMgYXJlCj4gZnJhZ21lbnRlZCBpbiBIVkEgYW5k IHF1ZXVlIHNpemUgaXMgZXF1YWwgYXQgYm90aCBzaWRlcy4gVGhhdCdzIHdoeSBJCj4gdGhpbmsg U1ZRIHNpemUgc2hvdWxkIGRlcGVuZCBvbiB0aGUgYmFja2VuZCBkZXZpY2UncyBzaXplLCBub3QK PiBmcm9udGVuZCBjbWRsaW5lLgoKUmlnaHQuCgpUaGFua3MKCj4KPiBUaGFua3MhCj4KPiA+Cj4g PiA+Cj4gPiA+IElmIEkgcnVuIHRoYXQgZXhhbXBsZSB3aXRoIHZwX3ZkcGEsIEwwIHFlbXUgd2ls bCBoYXBwaWx5IGFjY2VwdCAxMDI0Cj4gPiA+IGFzIGEgcXVldWUgc2l6ZSBbMl0uIEJ1dCBpZiB0 aGUgdmRwYSBkZXZpY2UgbWF4aW11bSBxdWV1ZSBzaXplIGlzCj4gPiA+IGVmZmVjdGl2ZWx5IDI1 NiwgdGhpcyB3aWxsIHJlc3VsdCBpbiBhbiBlcnJvcjogV2UncmUgbm90IGV4cG9zaW5nIGl0Cj4g PiA+IHRvIHRoZSBndWVzdCBhdCBhbnkgbW9tZW50IGJ1dCB3aXRoIHFlbXUncyBjbWRsaW5lLgo+ ID4gPgo+ID4gPj4+IGluY2x1ZGluZyAyNTYuCj4gPiA+Pj4gSXMgdGhhdCB3aGF0IHlvdSBtZWFu Pwo+ID4gPj4KPiA+ID4+IEkgbWVhbiwgaXQgbG9va3MgdG8gbWUgdGhlIGxvZ2ljIHdpbGwgYmUg bXVjaCBtb3JlIHNpbXBsaWZpZWQgaWYgd2UganVzdAo+ID4gPj4gYWxsb2NhdGUgdGhlIHNoYWRv dyB2aXJ0cXVldWUgd2l0aCB0aGUgc2l6ZSB3aGF0IGd1ZXN0IGNhbiBzZWUgKGd1ZXN0Cj4gPiA+ PiB2cmluZykuCj4gPiA+Pgo+ID4gPj4gVGhlbiB3ZSBkb24ndCBuZWVkIHRvIHRoaW5rIGlmIHRo ZSBkaWZmZXJlbmNlIG9mIHRoZSBxdWV1ZSBzaXplIGNhbiBoYXZlCj4gPiA+PiBhbnkgc2lkZSBl ZmZlY3RzLgo+ID4gPj4KPiA+ID4gSSB0aGluayB0aGF0IHdlIGNhbm5vdCBhdm9pZCB0aGF0IGV4 dHJhIGxvZ2ljIHVubGVzcyB3ZSBmb3JjZSBHUEEgdG8KPiA+ID4gYmUgY29udGlndW91cyBpbiBJ T1ZBLiBJZiB3ZSBhcmUgc3VyZSB0aGUgZ3Vlc3QncyBidWZmZXJzIGNhbm5vdCBiZSBhdAo+ID4g PiBtb3JlIHRoYW4gb25lIGRlc2NyaXB0b3IgaW4gU1ZRLCB0aGVuIHllcywgd2UgY2FuIHNpbXBs aWZ5IHRoaW5ncy4gSWYKPiA+ID4gbm90LCBJIHRoaW5rIHdlIGFyZSBmb3JjZWQgdG8gY2Fycnkg YWxsIG9mIGl0Lgo+ID4KPiA+Cj4gPiBZZXMsIEkgYWdyZWUsIHRoZSBjb2RlIHNob3VsZCBiZSBy b2J1c3QgdG8gaGFuZGxlIGFueSBjYXNlLgo+ID4KPiA+IFRoYW5rcwo+ID4KPiA+Cj4gPiA+Cj4g PiA+IEJ1dCBpZiB3ZSBwcm92ZSBpdCBJJ20gbm90IG9wcG9zZWQgdG8gc2ltcGxpZnlpbmcgdGhp bmdzIGFuZCBtYWtpbmcKPiA+ID4gaGVhZCBhdCBTVlEgPT0gaGVhZCBhdCBndWVzdC4KPiA+ID4K PiA+ID4gVGhhbmtzIQo+ID4gPgo+ID4gPiBbMV0gaHR0cHM6Ly9naXRsYWIuY29tL3FlbXUtcHJv amVjdC9xZW11Ly0vYmxvYi8xN2UzMTM0MC9ody92aXJ0aW8vdmlydGlvLmMjTDEyOTcKPiA+ID4g WzJdIEJ1dCB0aGF0J3Mgbm90IHRoZSB3aG9sZSBzdG9yeTogSSd2ZSBiZWVuIHJ1bm5pbmcgbGlt aXRlZCBpbiB0eAo+ID4gPiBkZXNjcmlwdG9ycyBiZWNhdXNlIG9mIHZpcnRpb19uZXRfbWF4X3R4 X3F1ZXVlX3NpemUsIHdoaWNoIHByZWRhdGVzCj4gPiA+IHZkcGEuIEknbGwgc2VuZCBhIHBhdGNo IHRvIGFsc28gdW4tbGltaXQgaXQuCj4gPiA+Cj4gPiA+Pj4gSWYgd2l0aCBoYXJkd2FyZSBxdWV1 ZXMgeW91IG1lYW4gZ3Vlc3QncyB2cmluZywgbm90IHN1cmUgd2h5IGl0IGlzCj4gPiA+Pj4gInBy b2JhYmx5IDI1NiIuIEknZCBzYXkgdGhhdCBpbiB0aGF0IGNhc2Ugd2l0aCB0aGUgdmlydGlvLW5l dCBrZXJuZWwKPiA+ID4+PiBkcml2ZXIgdGhlIHJpbmcgc2l6ZSB3aWxsIGJlIHRoZSBzYW1lIGFz IHRoZSBkZXZpY2UgZXhwb3J0LCBmb3IKPiA+ID4+PiBleGFtcGxlLCBpc24ndCBpdD8KPiA+ID4+ Pgo+ID4gPj4+IFRoZSBpbXBsZW1lbnRhdGlvbiBzaG91bGQgc3VwcG9ydCBhbnkgY29tYmluYXRp b24gb2Ygc2l6ZXMsIGJ1dCB0aGUKPiA+ID4+PiByaW5nIHNpemUgZXhwb3NlZCB0byB0aGUgZ3Vl c3QgaXMgbmV2ZXIgYmlnZ2VyIHRoYW4gaGFyZHdhcmUgb25lLgo+ID4gPj4+Cj4gPiA+Pj4+ID8g T3Igd2UgU1ZRIGNhbiBzdGljayB0byAyNTYgYnV0IHRoaXMgd2lsbCB0aGlzIGNhdXNlIHRyb3Vi bGUgaWYgd2Ugd2FudAo+ID4gPj4+PiB0byBhZGQgZXZlbnQgaW5kZXggc3VwcG9ydD8KPiA+ID4+ Pj4KPiA+ID4+PiBJIHRoaW5rIHdlIHNob3VsZCBub3QgaGF2ZSBhbnkgcHJvYmxlbSB3aXRoIGV2 ZW50IGlkeC4gSWYgeW91IG1lYW4KPiA+ID4+PiB0aGF0IHRoZSBndWVzdCBjb3VsZCBtYXJrIG1v cmUgYnVmZmVycyBhdmFpbGFibGUgdGhhbiBTVlEgdnJpbmcncwo+ID4gPj4+IHNpemUsIHRoYXQg c2hvdWxkIG5vdCBoYXBwZW4gYmVjYXVzZSB0aGVyZSBtdXN0IGJlIGxlc3MgZW50cmllcyBpbiB0 aGUKPiA+ID4+PiBndWVzdCB0aGFuIFNWUS4KPiA+ID4+Pgo+ID4gPj4+IEJ1dCBpZiBJIHVuZGVy c3Rvb2QgeW91IGNvcnJlY3RseSwgYSBzaW1pbGFyIHNpdHVhdGlvbiBjb3VsZCBoYXBwZW4gaWYK PiA+ID4+PiBhIGd1ZXN0J3MgY29udGlndW91cyBidWZmZXIgaXMgc2NhdHRlcmVkIGFjcm9zcyBt YW55IHFlbXUncyBWQSBjaHVua3MuCj4gPiA+Pj4gRXZlbiBpZiB0aGF0IHdvdWxkIGhhcHBlbiwg dGhlIHNpdHVhdGlvbiBzaG91bGQgYmUgb2sgdG9vOiBTVlEga25vd3MKPiA+ID4+PiB0aGUgZ3Vl c3QncyBhdmFpbCBpZHggYW5kLCBpZiBTVlEgaXMgZnVsbCwgaXQgd2lsbCBjb250aW51ZSBmb3J3 YXJkaW5nCj4gPiA+Pj4gYXZhaWwgYnVmZmVycyB3aGVuIHRoZSBkZXZpY2UgdXNlcyBtb3JlIGJ1 ZmZlcnMuCj4gPiA+Pj4KPiA+ID4+PiBEb2VzIHRoYXQgbWFrZSBzZW5zZSB0byB5b3U/Cj4gPiA+ Pgo+ID4gPj4gWWVzLgo+ID4gPj4KPiA+ID4+IFRoYW5rcwo+ID4gPj4KPiA+Cj4KCl9fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fClZpcnR1YWxpemF0aW9uIG1h aWxpbmcgbGlzdApWaXJ0dWFsaXphdGlvbkBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRw czovL2xpc3RzLmxpbnV4Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0aW5mby92aXJ0dWFsaXph dGlvbg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B7AA9C43219 for ; Tue, 22 Feb 2022 03:17:55 +0000 (UTC) Received: from localhost ([::1]:53480 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nMLgY-0007eD-Ad for qemu-devel@archiver.kernel.org; Mon, 21 Feb 2022 22:17:54 -0500 Received: from eggs.gnu.org ([209.51.188.92]:34546) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nMLfU-0006wI-PS for qemu-devel@nongnu.org; Mon, 21 Feb 2022 22:16:48 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:59617) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nMLfR-00006e-Bp for qemu-devel@nongnu.org; Mon, 21 Feb 2022 22:16:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1645499798; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gHY8uuapAXsf5KULdO4mEh+Xne/2fBaTYxBgL4rXrS8=; b=NzCWF168EQDkJYuu0TSDcatQPI6yJmp+gqq0kmw8NyX0r8QhnHdtmqx2XRb8s+DPwiHxe9 hN7b38+1Eyt7zK0MlkHfPe/JrOL5y3aqlDzKoWqY3qMsYW+e2oXxxeWoPxyPwFS3PHCjPK /JPMIZ3ehSInX1TXJWZP8aBnWPD0HKQ= Received: from mail-lj1-f200.google.com (mail-lj1-f200.google.com [209.85.208.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-477-OeoXBTxuPPOc2RRmOCVsmQ-1; Mon, 21 Feb 2022 22:16:37 -0500 X-MC-Unique: OeoXBTxuPPOc2RRmOCVsmQ-1 Received: by mail-lj1-f200.google.com with SMTP id h21-20020a05651c125500b002464536cf4eso1624591ljh.23 for ; Mon, 21 Feb 2022 19:16:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=gHY8uuapAXsf5KULdO4mEh+Xne/2fBaTYxBgL4rXrS8=; b=llCB8siKDXhd/fgsAgf2Gj+oOcuyohrLQSVC24Z0GACwp8DmDsuZEt5bS6zGzgqEFf ihtFQ3VGp+vpMbjzVV0IYO/umE6byqNjCbpjiBsncM10S+ePV9zNuR9nSHdeM5h2muQK COZiyXPItwjBImkP247s4OSZF2klAEVhHRAoLAlFTmzCLhhFYNDbdyHaI2oZBCNfuUx/ vQ5teVs49nKOJ/8IWYDLh8MeObEG6674VibGLhmDfc6OBtkpY1lF5GcXyBHIaAhBMgCO Bnj6MEtt8e+EeDswDK9RjRceMydOJqve2cnTAWq5bwv2pdozsU7qmPhp/z91djXsuUzc 4b6Q== X-Gm-Message-State: AOAM531G2wGhMPO2DXDV5ZfSByuo306o81YcYrOWlP6oyP69pfHjkDkp 3dNuYYt+SSjv4ojhePw3P6Mzeupo5Yq4D5AmpnJzLQDbAYetrytsfgG6p7gwilAWk2VLhqBY3ln d0zBlgzVhBoTrvtNBUkOePbUIjcXaiLo= X-Received: by 2002:ac2:4194:0:b0:442:ed9e:4a25 with SMTP id z20-20020ac24194000000b00442ed9e4a25mr15714990lfh.629.1645499795430; Mon, 21 Feb 2022 19:16:35 -0800 (PST) X-Google-Smtp-Source: ABdhPJzFQXM4KXylMKw7WbdxhpVZyCLXX/jI0XWCwc7r24OEqocGHtKuqW/T+EEktOtw/EgdVyRb88GzMPXSR/TEEqw= X-Received: by 2002:ac2:4194:0:b0:442:ed9e:4a25 with SMTP id z20-20020ac24194000000b00442ed9e4a25mr15714952lfh.629.1645499795020; Mon, 21 Feb 2022 19:16:35 -0800 (PST) MIME-Version: 1.0 References: <20220121202733.404989-1-eperezma@redhat.com> <20220121202733.404989-18-eperezma@redhat.com> <82b8c3bf-1b11-86c7-4fad-294f5ccf1278@redhat.com> <3d0dfaaa-a67c-6f48-fd03-45d2661ba92a@redhat.com> <02f29b62-6656-ba2f-1475-251b16e0e978@redhat.com> In-Reply-To: From: Jason Wang Date: Tue, 22 Feb 2022 11:16:23 +0800 Message-ID: Subject: Re: [PATCH 17/31] vdpa: adapt vhost_ops callbacks to svq To: Eugenio Perez Martin Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=170.10.129.124; envelope-from=jasowang@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , Cindy Lu , "Michael S. Tsirkin" , Juan Quintela , Richard Henderson , qemu-level , Gautam Dawar , Markus Armbruster , Eduardo Habkost , Harpreet Singh Anand , Xiao W Wang , Peter Xu , Stefan Hajnoczi , Eli Cohen , Paolo Bonzini , Zhu Lingshan , virtualization , Eric Blake , Stefano Garzarella Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On Tue, Feb 22, 2022 at 1:23 AM Eugenio Perez Martin wrote: > > On Mon, Feb 21, 2022 at 8:15 AM Jason Wang wrote: > > > > > > =E5=9C=A8 2022/2/18 =E4=B8=8A=E5=8D=881:13, Eugenio Perez Martin =E5=86= =99=E9=81=93: > > > On Tue, Feb 8, 2022 at 4:58 AM Jason Wang wrote= : > > >> > > >> =E5=9C=A8 2022/2/1 =E4=B8=8A=E5=8D=882:58, Eugenio Perez Martin =E5= =86=99=E9=81=93: > > >>> On Sun, Jan 30, 2022 at 5:03 AM Jason Wang wr= ote: > > >>>> =E5=9C=A8 2022/1/22 =E4=B8=8A=E5=8D=884:27, Eugenio P=C3=A9rez =E5= =86=99=E9=81=93: > > >>>>> First half of the buffers forwarding part, preparing vhost-vdpa > > >>>>> callbacks to SVQ to offer it. QEMU cannot enable it at this momen= t, so > > >>>>> this is effectively dead code at the moment, but it helps to redu= ce > > >>>>> patch size. > > >>>>> > > >>>>> Signed-off-by: Eugenio P=C3=A9rez > > >>>>> --- > > >>>>> hw/virtio/vhost-shadow-virtqueue.h | 2 +- > > >>>>> hw/virtio/vhost-shadow-virtqueue.c | 21 ++++- > > >>>>> hw/virtio/vhost-vdpa.c | 133 ++++++++++++++++++++= ++++++--- > > >>>>> 3 files changed, 143 insertions(+), 13 deletions(-) > > >>>>> > > >>>>> diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost= -shadow-virtqueue.h > > >>>>> index 035207a469..39aef5ffdf 100644 > > >>>>> --- a/hw/virtio/vhost-shadow-virtqueue.h > > >>>>> +++ b/hw/virtio/vhost-shadow-virtqueue.h > > >>>>> @@ -35,7 +35,7 @@ size_t vhost_svq_device_area_size(const VhostSh= adowVirtqueue *svq); > > >>>>> > > >>>>> void vhost_svq_stop(VhostShadowVirtqueue *svq); > > >>>>> > > >>>>> -VhostShadowVirtqueue *vhost_svq_new(void); > > >>>>> +VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize); > > >>>>> > > >>>>> void vhost_svq_free(VhostShadowVirtqueue *vq); > > >>>>> > > >>>>> diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost= -shadow-virtqueue.c > > >>>>> index f129ec8395..7c168075d7 100644 > > >>>>> --- a/hw/virtio/vhost-shadow-virtqueue.c > > >>>>> +++ b/hw/virtio/vhost-shadow-virtqueue.c > > >>>>> @@ -277,9 +277,17 @@ void vhost_svq_stop(VhostShadowVirtqueue *sv= q) > > >>>>> /** > > >>>>> * Creates vhost shadow virtqueue, and instruct vhost device = to use the shadow > > >>>>> * methods and file descriptors. > > >>>>> + * > > >>>>> + * @qsize Shadow VirtQueue size > > >>>>> + * > > >>>>> + * Returns the new virtqueue or NULL. > > >>>>> + * > > >>>>> + * In case of error, reason is reported through error_report. > > >>>>> */ > > >>>>> -VhostShadowVirtqueue *vhost_svq_new(void) > > >>>>> +VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize) > > >>>>> { > > >>>>> + size_t desc_size =3D sizeof(vring_desc_t) * qsize; > > >>>>> + size_t device_size, driver_size; > > >>>>> g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShad= owVirtqueue, 1); > > >>>>> int r; > > >>>>> > > >>>>> @@ -300,6 +308,15 @@ VhostShadowVirtqueue *vhost_svq_new(void) > > >>>>> /* Placeholder descriptor, it should be deleted at set_ki= ck_fd */ > > >>>>> event_notifier_init_fd(&svq->svq_kick, INVALID_SVQ_KICK_F= D); > > >>>>> > > >>>>> + svq->vring.num =3D qsize; > > >>>> I wonder if this is the best. E.g some hardware can support up to = 32K > > >>>> queue size. So this will probably end up with: > > >>>> > > >>>> 1) SVQ use 32K queue size > > >>>> 2) hardware queue uses 256 > > >>>> > > >>> In that case SVQ vring queue size will be 32K and guest's vring can > > >>> negotiate any number with SVQ equal or less than 32K, > > >> > > >> Sorry for being unclear what I meant is actually > > >> > > >> 1) SVQ uses 32K queue size > > >> > > >> 2) guest vq uses 256 > > >> > > >> This looks like a burden that needs extra logic and may damage the > > >> performance. > > >> > > > Still not getting this point. > > > > > > An available guest buffer, although contiguous in GPA/GVA, can expand > > > in multiple buffers if it's not contiguous in qemu's VA (by the while > > > loop in virtqueue_map_desc [1]). In that scenario it is better to hav= e > > > "plenty" of SVQ buffers. > > > > > > Yes, but this case should be rare. So in this case we should deal with > > overrun on SVQ, that is > > > > 1) SVQ is full > > 2) guest VQ isn't > > > > We need to > > > > 1) check the available buffer slots > > 2) disable guest kick and wait for the used buffers > > > > But it looks to me the current code is not ready for dealing with this = case? > > > > Yes it deals, that's the meaning of svq->next_guest_avail_elem. Oh right, I missed that. > > > > > > > > > I'm ok if we decide to put an upper limit though, or if we decide not > > > to handle this situation. But we would leave out valid virtio drivers= . > > > Maybe to set a fixed upper limit (1024?)? To add another parameter > > > (x-svq-size-n=3DN)? > > > > > > If you mean we lose performance because memory gets more sparse I > > > think the only possibility is to limit that way. > > > > > > If guest is not using 32K, having a 32K for svq may gives extra stress > > on the cache since we will end up with a pretty large working set. > > > > That might be true. My guess is that it should not matter, since SVQ > and the guest's vring will have the same numbers of scattered buffers > and the avail / used / packed ring will be consumed more or less > sequentially. But I haven't tested. > > I think it's better to add an upper limit (either fixed or in the > qemu's backend's cmdline) later if we see that this is a problem. I'd suggest using the same size as what the guest saw. > Another solution now would be to get the number from the frontend > device cmdline instead of from the vdpa device. I'm ok with that, but > it doesn't delete the svq->next_guest_avail_elem processing, and it > comes with disadvantages in my opinion. More below. Right, we should keep next_guest_avail_elem. Using the same queue size is a balance between: 1) using next_guest_avail_elem (rare) 2) not give too much stress on the cache > > > > > > > > >> And this can lead other interesting situation: > > >> > > >> 1) SVQ uses 256 > > >> > > >> 2) guest vq uses 1024 > > >> > > >> Where a lot of more SVQ logic is needed. > > >> > > > If we agree that a guest descriptor can expand in multiple SVQ > > > descriptors, this should be already handled by the previous logic too= . > > > > > > But this should only happen in case that qemu is launched with a "bad= " > > > cmdline, isn't it? > > > > > > This seems can happen when we use -device > > virtio-net-pci,tx_queue_size=3D1024 with a 256 size vp_vdpa device at l= east? > > > > I'm going to use the rx queue here since it's more accurate, tx has > its own limit separately. > > If we use rx_queue_size=3D256 in L0 and rx_queue_size=3D1024 in L1 with n= o > SVQ, L0 qemu will happily accept 1024 as size Interesting, looks like a bug (I guess it works since you enable vhost?): Per virtio-spec: """ Queue Size. On reset, specifies the maximum queue size supported by the device. This can be modified by the driver to reduce memory requirements. A 0 means the queue is unavailable. """ We can't increase the queue_size from 256 to 1024 actually. (Only decrease is allowed). > when L1 qemu writes that > value at vhost_virtqueue_start. I'm not sure what would happen with a > real device, my guess is that the device will fail somehow. That's > what I meant with a "bad cmdline", I should have been more specific. I should say that it's something that is probably unrelated to this series but needs to be addressed. > > If we add SVQ to the mix, the guest first negotiates the 1024 with the > qemu device model. After that, vhost.c will try to write 1024 too but > this is totally ignored by this patch's changes at > vhost_vdpa_set_vring_num. Finally, SVQ will set 256 as a ring size to > the device, since it's the read value from the device, leading to your > scenario. So SVQ effectively isolates both sides and makes possible > the communication, even with a device that does not support so many > descriptors. > > But SVQ already handles this case: It's the same as if the buffers are > fragmented in HVA and queue size is equal at both sides. That's why I > think SVQ size should depend on the backend device's size, not > frontend cmdline. Right. Thanks > > Thanks! > > > > > > > > > If I run that example with vp_vdpa, L0 qemu will happily accept 1024 > > > as a queue size [2]. But if the vdpa device maximum queue size is > > > effectively 256, this will result in an error: We're not exposing it > > > to the guest at any moment but with qemu's cmdline. > > > > > >>> including 256. > > >>> Is that what you mean? > > >> > > >> I mean, it looks to me the logic will be much more simplified if we = just > > >> allocate the shadow virtqueue with the size what guest can see (gues= t > > >> vring). > > >> > > >> Then we don't need to think if the difference of the queue size can = have > > >> any side effects. > > >> > > > I think that we cannot avoid that extra logic unless we force GPA to > > > be contiguous in IOVA. If we are sure the guest's buffers cannot be a= t > > > more than one descriptor in SVQ, then yes, we can simplify things. If > > > not, I think we are forced to carry all of it. > > > > > > Yes, I agree, the code should be robust to handle any case. > > > > Thanks > > > > > > > > > > But if we prove it I'm not opposed to simplifying things and making > > > head at SVQ =3D=3D head at guest. > > > > > > Thanks! > > > > > > [1] https://gitlab.com/qemu-project/qemu/-/blob/17e31340/hw/virtio/vi= rtio.c#L1297 > > > [2] But that's not the whole story: I've been running limited in tx > > > descriptors because of virtio_net_max_tx_queue_size, which predates > > > vdpa. I'll send a patch to also un-limit it. > > > > > >>> If with hardware queues you mean guest's vring, not sure why it is > > >>> "probably 256". I'd say that in that case with the virtio-net kerne= l > > >>> driver the ring size will be the same as the device export, for > > >>> example, isn't it? > > >>> > > >>> The implementation should support any combination of sizes, but the > > >>> ring size exposed to the guest is never bigger than hardware one. > > >>> > > >>>> ? Or we SVQ can stick to 256 but this will this cause trouble if w= e want > > >>>> to add event index support? > > >>>> > > >>> I think we should not have any problem with event idx. If you mean > > >>> that the guest could mark more buffers available than SVQ vring's > > >>> size, that should not happen because there must be less entries in = the > > >>> guest than SVQ. > > >>> > > >>> But if I understood you correctly, a similar situation could happen= if > > >>> a guest's contiguous buffer is scattered across many qemu's VA chun= ks. > > >>> Even if that would happen, the situation should be ok too: SVQ know= s > > >>> the guest's avail idx and, if SVQ is full, it will continue forward= ing > > >>> avail buffers when the device uses more buffers. > > >>> > > >>> Does that make sense to you? > > >> > > >> Yes. > > >> > > >> Thanks > > >> > > >