From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7A89BC433EF for ; Tue, 8 Mar 2022 07:55:40 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 1DEF283E4A; Tue, 8 Mar 2022 07:55:40 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mbHD8R0wDrYZ; Tue, 8 Mar 2022 07:55:38 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 2F113827CE; Tue, 8 Mar 2022 07:55:38 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 10E9CC0012; Tue, 8 Mar 2022 07:55:38 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 772B2C000B for ; Tue, 8 Mar 2022 07:55:36 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 4FBB7408E9 for ; Tue, 8 Mar 2022 07:55:36 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NpzPA6QrWY6n for ; Tue, 8 Mar 2022 07:55:34 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 4A49F404D0 for ; Tue, 8 Mar 2022 07:55:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646726133; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UFezi77WesJBzFK1fhaAbsIyGgE1choY2kdQrE0mkS4=; b=d0NoughBUnNxcq3vKmH1ARJz09Gs9Fw/AZ62U4VudeHCT4Lt2VCzOq9lDpLZcW+KVVQw5P 7BEov1o5OoZUKmkZH7Dqaz6rSmdAOzmb3XT4YGJ1HhI/1NRVxU3haP2opviaumkQy0PSyv gUKT3yeyGu42kdu+62/oWBBaB7hZLiI= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-593-GSFRqrGoO2u2ZAOUKa2l0w-1; Tue, 08 Mar 2022 02:55:31 -0500 X-MC-Unique: GSFRqrGoO2u2ZAOUKa2l0w-1 Received: by mail-ed1-f69.google.com with SMTP id n11-20020aa7c68b000000b0041641550e11so3746945edq.8 for ; Mon, 07 Mar 2022 23:55:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=UFezi77WesJBzFK1fhaAbsIyGgE1choY2kdQrE0mkS4=; b=xtNnugkKzgD6Drt+HQImflA6aT9r2AQX/ggJadiwFNKJA2eGchhh8BHlEOhGAWFiFH fifOh0XGTePR16V7Zdw7NSPGXFg67m+4NUFQ0DB1H57eoaJu/SYmfKY9FEO59ttGTedc hM+yHF5cAYfNzC9/hakfSikHI8TUPIqnMzZIcMCsSWYuDyVs+byf4f5g1kTBt1bF8xzw C4YB/pNqvCl9kyFr039j6P+PYC3dl9FnPVjUycoR6/qdh809x8lxmqeAoH7Xl+qslW3E bsLPmAoTYm0kN/2/sfz68d9fHw71SLw5190+Xua33AsFKhJr57o58VB2ZwhfsuvCVdMG Y7fg== X-Gm-Message-State: AOAM532aEHbeakbt6FomhuhEL2gOVgWHVOCjIatKRC91g2uJenAqtI3J Jss5F3l6DbaSbrwY2yUv0eWFchXNDnSSPv+J5m5bwTFCz/1D3xjgcMyLvb/EwxmcX3fcsldIxqh O+tQ5695FWawk1FOePluBrfLtgj2zVpc6g+vKxYEuVw== X-Received: by 2002:a17:907:160d:b0:6db:49bb:2c7a with SMTP id hb13-20020a170907160d00b006db49bb2c7amr1760428ejc.228.1646726129729; Mon, 07 Mar 2022 23:55:29 -0800 (PST) X-Google-Smtp-Source: ABdhPJzPLIh9zjOVAW9d23M/7A7pRnUKI7iQh9+Zh18H1bTRewg2yW6RP5RnWMHUmqjcD9h29Kldxw== X-Received: by 2002:a17:907:160d:b0:6db:49bb:2c7a with SMTP id hb13-20020a170907160d00b006db49bb2c7amr1760400ejc.228.1646726129410; Mon, 07 Mar 2022 23:55:29 -0800 (PST) Received: from redhat.com ([2.55.138.228]) by smtp.gmail.com with ESMTPSA id c1-20020a170906762100b006d00ae72b0csm5465608ejn.221.2022.03.07.23.55.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Mar 2022 23:55:28 -0800 (PST) Date: Tue, 8 Mar 2022 02:55:23 -0500 From: "Michael S. Tsirkin" To: Jason Wang Subject: Re: [PATCH v5 00/15] vDPA shadow virtqueue Message-ID: <20220308024724-mutt-send-email-mst@kernel.org> References: <20220307153334.3854134-1-eperezma@redhat.com> <14d4fde4-6ea5-4805-b684-c33f6b448565@redhat.com> <20220308020348-mutt-send-email-mst@kernel.org> <20220308022300-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 In-Reply-To: Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mst@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline Cc: qemu-devel , virtualization , Eli Cohen , Eric Blake , Eduardo Habkost , Cindy Lu , "Fangyi \(Eric\)" , Markus Armbruster , yebiaoxiang@huawei.com, Eugenio =?iso-8859-1?Q?P=E9rez?= , Liuxiangdong , Laurent Vivier , Parav Pandit , Richard Henderson , Gautam Dawar , Xiao W Wang , Stefan Hajnoczi , Harpreet Singh Anand , Lingshan X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" T24gVHVlLCBNYXIgMDgsIDIwMjIgYXQgMDM6MzQ6MTdQTSArMDgwMCwgSmFzb24gV2FuZyB3cm90 ZToKPiBPbiBUdWUsIE1hciA4LCAyMDIyIGF0IDM6MjggUE0gTWljaGFlbCBTLiBUc2lya2luIDxt c3RAcmVkaGF0LmNvbT4gd3JvdGU6Cj4gPgo+ID4gT24gVHVlLCBNYXIgMDgsIDIwMjIgYXQgMDM6 MTQ6MzVQTSArMDgwMCwgSmFzb24gV2FuZyB3cm90ZToKPiA+ID4gT24gVHVlLCBNYXIgOCwgMjAy MiBhdCAzOjExIFBNIE1pY2hhZWwgUy4gVHNpcmtpbiA8bXN0QHJlZGhhdC5jb20+IHdyb3RlOgo+ ID4gPiA+Cj4gPiA+ID4gT24gVHVlLCBNYXIgMDgsIDIwMjIgYXQgMDI6MDM6MzJQTSArMDgwMCwg SmFzb24gV2FuZyB3cm90ZToKPiA+ID4gPiA+Cj4gPiA+ID4gPiDlnKggMjAyMi8zLzcg5LiL5Y2I MTE6MzMsIEV1Z2VuaW8gUMOpcmV6IOWGmemBkzoKPiA+ID4gPiA+ID4gVGhpcyBzZXJpZXMgZW5h YmxlIHNoYWRvdyB2aXJ0cXVldWUgKFNWUSkgZm9yIHZob3N0LXZkcGEgZGV2aWNlcy4gVGhpcwo+ ID4gPiA+ID4gPiBpcyBpbnRlbmRlZCBhcyBhIG5ldyBtZXRob2Qgb2YgdHJhY2tpbmcgdGhlIG1l bW9yeSB0aGUgZGV2aWNlcyB0b3VjaAo+ID4gPiA+ID4gPiBkdXJpbmcgYSBtaWdyYXRpb24gcHJv Y2VzczogSW5zdGVhZCBvZiByZWxheSBvbiB2aG9zdCBkZXZpY2UncyBkaXJ0eQo+ID4gPiA+ID4g PiBsb2dnaW5nIGNhcGFiaWxpdHksIFNWUSBpbnRlcmNlcHRzIHRoZSBWUSBkYXRhcGxhbmUgZm9y d2FyZGluZyB0aGUKPiA+ID4gPiA+ID4gZGVzY3JpcHRvcnMgYmV0d2VlbiBWTSBhbmQgZGV2aWNl LiBUaGlzIHdheSBxZW11IGlzIHRoZSBlZmZlY3RpdmUKPiA+ID4gPiA+ID4gd3JpdGVyIG9mIGd1 ZXN0cyBtZW1vcnksIGxpa2UgaW4gcWVtdSdzIHZpcnRpbyBkZXZpY2Ugb3BlcmF0aW9uLgo+ID4g PiA+ID4gPgo+ID4gPiA+ID4gPiBXaGVuIFNWUSBpcyBlbmFibGVkIHFlbXUgb2ZmZXJzIGEgbmV3 IHZpcnR1YWwgYWRkcmVzcyBzcGFjZSB0byB0aGUKPiA+ID4gPiA+ID4gZGV2aWNlIHRvIHJlYWQg YW5kIHdyaXRlIGludG8sIGFuZCBpdCBtYXBzIG5ldyB2cmluZ3MgYW5kIHRoZSBndWVzdAo+ID4g PiA+ID4gPiBtZW1vcnkgaW4gaXQuIFNWUSBhbHNvIGludGVyY2VwdHMga2lja3MgYW5kIGNhbGxz IGJldHdlZW4gdGhlIGRldmljZQo+ID4gPiA+ID4gPiBhbmQgdGhlIGd1ZXN0LiBVc2VkIGJ1ZmZl cnMgcmVsYXkgd291bGQgY2F1c2UgZGlydHkgbWVtb3J5IGJlaW5nCj4gPiA+ID4gPiA+IHRyYWNr ZWQuCj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+IFRoaXMgZWZmZWN0aXZlbHkgbWVhbnMgdGhhdCB2 RFBBIGRldmljZSBwYXNzdGhyb3VnaCBpcyBpbnRlcmNlcHRlZCBieQo+ID4gPiA+ID4gPiBxZW11 LiBXaGlsZSBTVlEgc2hvdWxkIG9ubHkgYmUgZW5hYmxlZCBhdCBtaWdyYXRpb24gdGltZSwgdGhl IHN3aXRjaGluZwo+ID4gPiA+ID4gPiBmcm9tIHJlZ3VsYXIgbW9kZSB0byBTVlEgbW9kZSBpcyBs ZWZ0IGZvciBhIGZ1dHVyZSBzZXJpZXMuCj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+IEl0IGlzIGJh c2VkIG9uIHRoZSBpZGVhcyBvZiBEUERLIFNXIGFzc2lzdGVkIExNLCBpbiB0aGUgc2VyaWVzIG9m Cj4gPiA+ID4gPiA+IERQREsncyBodHRwczovL3BhdGNod29yay5kcGRrLm9yZy9jb3Zlci80ODM3 MC8gLiBIb3dldmVyLCB0aGVzZSBkb2VzCj4gPiA+ID4gPiA+IG5vdCBtYXAgdGhlIHNoYWRvdyB2 cSBpbiBndWVzdCdzIFZBLCBidXQgaW4gcWVtdSdzLgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBG b3IgcWVtdSB0byB1c2Ugc2hhZG93IHZpcnRxdWV1ZXMgdGhlIGd1ZXN0IHZpcnRpbyBkcml2ZXIg bXVzdCBub3QgdXNlCj4gPiA+ID4gPiA+IGZlYXR1cmVzIGxpa2UgZXZlbnRfaWR4Lgo+ID4gPiA+ ID4gPgo+ID4gPiA+ID4gPiBTVlEgbmVlZHMgdG8gYmUgZW5hYmxlZCB3aXRoIGNtZGxpbmU6Cj4g PiA+ID4gPiA+Cj4gPiA+ID4gPiA+IC1uZXRkZXYgdHlwZT12aG9zdC12ZHBhLHZob3N0ZGV2PXZo b3N0LXZkcGEtMCxpZD12aG9zdC12ZHBhMCxzdnE9b24KPiA+ID4gPgo+ID4gPiA+IEEgc3RhYmxl IEFQSSBmb3IgYW4gaW5jb21wbGV0ZSBmZWF0dXJlIGlzIGEgcHJvYmxlbSBpbWhvLgo+ID4gPgo+ ID4gPiBJdCBzaG91bGQgYmUgIngtc3ZxIi4KPiA+Cj4gPgo+ID4gV2VsbCBsb29rIGF0IHBhdGNo IDE1Lgo+IAo+IEl0J3MgYSBidWcgdGhhdCBuZWVkcyB0byBiZSBmaXhlZC4KPiAKPiA+Cj4gPiA+ ID4KPiA+ID4gPgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBUaGUgZmlyc3QgdGhyZWUgcGF0Y2hl cyBlbmFibGVzIG5vdGlmaWNhdGlvbnMgZm9yd2FyZGluZyB3aXRoCj4gPiA+ID4gPiA+IGFzc2lz dGFuY2Ugb2YgcWVtdS4gSXQncyBlYXN5IHRvIGVuYWJsZSBvbmx5IHRoaXMgaWYgdGhlIHJlbGV2 YW50Cj4gPiA+ID4gPiA+IGNtZGxpbmUgcGFydCBvZiB0aGUgbGFzdCBwYXRjaCBpcyBhcHBsaWVk IG9uIHRvcCBvZiB0aGVzZS4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gTmV4dCBmb3VyIHBhdGNo ZXMgaW1wbGVtZW50IHRoZSBhY3R1YWwgYnVmZmVyIGZvcndhcmRpbmcuIEhvd2V2ZXIsCj4gPiA+ ID4gPiA+IGFkZHJlc3MgYXJlIG5vdCB0cmFuc2xhdGVkIGZyb20gSFZBIHNvIHRoZXkgd2lsbCBu ZWVkIGEgaG9zdCBkZXZpY2Ugd2l0aAo+ID4gPiA+ID4gPiBhbiBpb21tdSBhbGxvd2luZyB0aGVt IHRvIGFjY2VzcyBhbGwgb2YgdGhlIEhWQSByYW5nZS4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4g VGhlIGxhc3QgcGFydCBvZiB0aGUgc2VyaWVzIHVzZXMgcHJvcGVybHkgdGhlIGhvc3QgaW9tbXUs IHNvIHFlbXUKPiA+ID4gPiA+ID4gY3JlYXRlcyBhIG5ldyBpb3ZhIGFkZHJlc3Mgc3BhY2UgaW4g dGhlIGRldmljZSdzIHJhbmdlIGFuZCB0cmFuc2xhdGVzCj4gPiA+ID4gPiA+IHRoZSBidWZmZXJz IGluIGl0LiBGaW5hbGx5LCBpdCBhZGRzIHRoZSBjbWRsaW5lIHBhcmFtZXRlci4KPiA+ID4gPiA+ ID4KPiA+ID4gPiA+ID4gU29tZSBzaW1wbGUgcGVyZm9ybWFuY2UgdGVzdHMgd2l0aCBuZXRwZXJm IHdlcmUgZG9uZS4gVGhleSB1c2VkIGEgbmVzdGVkCj4gPiA+ID4gPiA+IGd1ZXN0IHdpdGggdnBf dmRwYSwgdmhvc3Qta2VybmVsIGF0IEwwIGhvc3QuIFN0YXJ0aW5nIHdpdGggbm8gc3ZxIGFuZCBh Cj4gPiA+ID4gPiA+IGJhc2VsaW5lIGF2ZXJhZ2Ugb2YgfjkwMDkuOTZNYnBzOgo+ID4gPiA+ID4g PiBSZWN2ICAgU2VuZCAgICBTZW5kCj4gPiA+ID4gPiA+IFNvY2tldCBTb2NrZXQgIE1lc3NhZ2Ug IEVsYXBzZWQKPiA+ID4gPiA+ID4gU2l6ZSAgIFNpemUgICAgU2l6ZSAgICAgVGltZSAgICAgVGhy b3VnaHB1dAo+ID4gPiA+ID4gPiBieXRlcyAgYnl0ZXMgICBieXRlcyAgICBzZWNzLiAgICAxMF42 Yml0cy9zZWMKPiA+ID4gPiA+ID4gMTMxMDcyICAxNjM4NCAgMTYzODQgICAgMzAuMDEgICAgOTA2 MS4wMwo+ID4gPiA+ID4gPiAxMzEwNzIgIDE2Mzg0ICAxNjM4NCAgICAzMC4wMSAgICA4OTYyLjk0 Cj4gPiA+ID4gPiA+IDEzMTA3MiAgMTYzODQgIDE2Mzg0ICAgIDMwLjAxICAgIDkwMDUuOTIKPiA+ ID4gPiA+ID4KPiA+ID4gPiA+ID4gVG8gZW5hYmxlIFNWUSBidWZmZXJzIGZvcndhcmRpbmcgcmVk dWNlIHRocm91Z2hwdXQgdG8gYWJvdXQKPiA+ID4gPiA+ID4gUmVjdiAgIFNlbmQgICAgU2VuZAo+ ID4gPiA+ID4gPiBTb2NrZXQgU29ja2V0ICBNZXNzYWdlICBFbGFwc2VkCj4gPiA+ID4gPiA+IFNp emUgICBTaXplICAgIFNpemUgICAgIFRpbWUgICAgIFRocm91Z2hwdXQKPiA+ID4gPiA+ID4gYnl0 ZXMgIGJ5dGVzICAgYnl0ZXMgICAgc2Vjcy4gICAgMTBeNmJpdHMvc2VjCj4gPiA+ID4gPiA+IDEz MTA3MiAgMTYzODQgIDE2Mzg0ICAgIDMwLjAxICAgIDc2ODkuNzIKPiA+ID4gPiA+ID4gMTMxMDcy ICAxNjM4NCAgMTYzODQgICAgMzAuMDAgICAgNzc1Mi4wNwo+ID4gPiA+ID4gPiAxMzEwNzIgIDE2 Mzg0ICAxNjM4NCAgICAzMC4wMSAgICA3NzUwLjMwCj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+IEhv d2V2ZXIsIG1hbnkgcGVyZm9ybWFuY2UgaW1wcm92ZW1lbnRzIHdlcmUgbGVmdCBvdXQgb2YgdGhp cyBzZXJpZXMgZm9yCj4gPiA+ID4gPiA+IHNpbXBsaWNpdHksIHNvIGRpZmZlcmVuY2Ugc2hvdWxk IHNocmluayBpbiB0aGUgZnV0dXJlLgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBDb21tZW50cyBh cmUgd2VsY29tZS4KPiA+ID4gPiA+Cj4gPiA+ID4gPgo+ID4gPiA+ID4gSGkgTWljaGFlbDoKPiA+ ID4gPiA+Cj4gPiA+ID4gPiBXaGF0IGRvIHlvdSB0aGluayBvZiB0aGlzIHNlcmllcz8gSXQgbG9v a3MgZ29vZCB0byBtZSBhcyBhIHN0YXJ0LiBUaGUKPiA+ID4gPiA+IGZlYXR1cmUgY291bGQgb25s eSBiZSBlbmFibGVkIGFzIGEgZGVkaWNhdGVkIHBhcmFtZXRlci4gSWYgeW91J3JlIG9rLCBJJ2QK PiA+ID4gPiA+IHRyeSB0byBtYWtlIGl0IGZvciA3LjAuCj4gPiA+ID4gPgo+ID4gPiA+ID4gVGhh bmtzCj4gPiA+ID4KPiA+ID4gPiBXZWxsIHRoYXQncyBjdXR0aW5nIGl0IGF3ZnVsbHkgY2xvc2Us IGFuZCBpdCdzIG5vdCByZWFsbHkgdXNlZnVsCj4gPiA+ID4gYXQgdGhlIGN1cnJlbnQgc3RhZ2Us IGlzIGl0Pwo+ID4gPgo+ID4gPiBUaGlzIGFsbG93cyB2RFBBIHRvIGJlIG1pZ3JhdGVkIHdoZW4g dXNpbmcgIngtc3ZxPW9uIi4KPiA+ID4gQnV0IGFueWhvdyBpdCdzCj4gPiA+IGV4cGVyaW1lbnRh bC4KPiA+Cj4gPiBpdCdzIGxlc3MgZXhwZXJpbWVudGFsIHRoYW4gaW5jb21wbGV0ZS4gSXQgc2Vl bXMgcHJldHR5IGNsZWFybHkgbm90Cj4gPiB0aGUgd2F5IGl0IHdpbGwgd29yayBkb3duIHRoZSBy b2FkLCB3ZSBkb24ndCB3YW50IHN2cSBpbnZvbHZlZAo+ID4gYXQgYWxsIHRpbWVzLgo+IAo+IFJp Z2h0LCBidXQgU1ZRIGNvdWxkIGJlIHVzZWQgZm9yIG90aGVyIHBsYWNlcyBlLmcgcHJvdmlkaW5n IG1pZ3JhdGlvbgo+IGNvbXBhdGliaWxpdHkgd2hlbiB0aGUgZGVzdGluYXRpb24gbGFja3Mgc29t ZSBmZWF0dXJlcy4KCkluIGl0cyBjdXJyZW50IGZvcm0/IEkgZG9uJ3Qgc2VlIGhvdy4gIEdlbmVy YWxseT8gTWF5YmUgYnV0IEkgc3VzcGVjdAp3ZSdsbCBoYXZlIHRvIHJld29yayBpdCBjb21wbGV0 ZWx5IGZvciB0aGF0LgoKPiA+Cj4gPiA+ID4KPiA+ID4gPiBUaGUgSU9WQSB0cmljayBkb2VzIG5v dCBmZWVsIGNvbXBsZXRlIGVpdGhlci4KPiA+ID4KPiA+ID4gSSBkb24ndCBnZXQgaGVyZS4gV2Ug ZG9uJ3QgdXNlIGFueSBJT1ZBIHRyaWNrIGFzIERQREsgKGl0IHJlc2VydmUgSU9WQQo+ID4gPiBm b3Igc2hhZG93IHZxKSBkaWQuIFNvIHdlIHdvbid0IHN1ZmZlciBmcm9tIHRoZSBpc3N1ZXMgb2Yg RFBESy4KPiA+ID4KPiA+ID4gVGhhbmtzCj4gPgo+ID4gTWF5YmUgSSBtaXN1bmRyc3RhbmQgaG93 IHRoaXMgYWxsIHdvcmtzLgo+ID4gSSByZWZlciB0byBhbGwgdGhlIGlvdmFfdHJlZV9hbGxvY19t YXAgdGhpbmdzLgo+IAo+IEl0J3MgYSBzaW1wbGUgSU9WQSBhbGxvY2F0ZXIgYWN0dWFsbHkuIEFu eXRoaW5nIHdyb25nIHdpdGggdGhhdD8KCk5vdCBieSBpdHNlbGYgYnV0IEknbSBub3Qgc3VyZSB3 ZSBjYW4gZ3VhcmFudGVlIGd1ZXN0IHdpbGwgbm90CmF0dGVtcHQgdG8gdXNlIHRoZSBJT1ZBIGFk ZHJlc3NlcyB3ZSBhcmUgcmVzZXJ2aW5nIGRvd24KdGhlIHJvYWQuCgo+IEknbSBmaW5lIHdpdGgg bWFraW5nIGl0IGZvciB0aGUgZnV0dXJlIHJlbGVhc2UuCj4gCj4gVGhhbmtzCj4gCj4gPgo+ID4g PiA+Cj4gPiA+ID4gPgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBUT0RPIG9uIGZ1dHVyZSBzZXJp ZXM6Cj4gPiA+ID4gPiA+ICogRXZlbnQsIGluZGlyZWN0LCBwYWNrZWQsIGFuZCBvdGhlcnMgZmVh dHVyZXMgb2YgdmlydGlvLgo+ID4gPiA+ID4gPiAqIFRvIHN1cHBvcnQgZGlmZmVyZW50IHNldCBv ZiBmZWF0dXJlcyBiZXR3ZWVuIHRoZSBkZXZpY2U8LT5TVlEgYW5kIHRoZQo+ID4gPiA+ID4gPiAg ICBTVlE8LT5ndWVzdCBjb21tdW5pY2F0aW9uLgo+ID4gPiA+ID4gPiAqIFN1cHBvcnQgb2YgZGV2 aWNlIGhvc3Qgbm90aWZpZXIgbWVtb3J5IHJlZ2lvbnMuCj4gPiA+ID4gPiA+ICogVG8gc2VwcGFy YXRlIGJ1ZmZlcnMgZm9yd2FyZGluZyBpbiBpdHMgb3duIEFJTyBjb250ZXh0LCBzbyB3ZSBjYW4K PiA+ID4gPiA+ID4gICAgdGhyb3cgbW9yZSB0aHJlYWRzIHRvIHRoYXQgdGFzayBhbmQgd2UgZG9u J3QgbmVlZCB0byBzdG9wIHRoZSBtYWluCj4gPiA+ID4gPiA+ICAgIGV2ZW50IGxvb3AuCj4gPiA+ ID4gPiA+ICogU3VwcG9ydCBtdWx0aXF1ZXVlIHZpcnRpby1uZXQgdmRwYS4KPiA+ID4gPiA+ID4g KiBQcm9wZXIgZG9jdW1lbnRhdGlvbi4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gQ2hhbmdlcyBm cm9tIHY0Ogo+ID4gPiA+ID4gPiAqIEl0ZXJhdGUgaW92YS0+aHZhIHRyZWUgaW5zdGVhZCBvbiBt YWludGFpbiBvd24gdHJlZSBzbyB3ZSBzdXBwb3J0IEhWQQo+ID4gPiA+ID4gPiAgICBvdmVybGFw cy4KPiA+ID4gPiA+ID4gKiBGaXg6IEVycm5vIGNvbXBsZXRpb24gYXQgZmFpbHVyZS4KPiA+ID4g PiA+ID4gKiBSZW5hbWUgeC1zdnEgdG8gc3ZxLCBzbyBjaGFuZ2VzIHRvIHN0YWJsZSBkb2VzIG5v dCBhZmZlY3QgY21kbGluZSBwYXJhbWV0ZXIuCj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+IENoYW5n ZXMgZnJvbSB2MzoKPiA+ID4gPiA+ID4gKiBBZGQgQHVuc3RhYmxlIGZlYXR1cmUgdG8gTmV0ZGV2 Vmhvc3RWRFBBT3B0aW9ucy54LXN2cS4KPiA+ID4gPiA+ID4gKiBGaXggdW5jb21wbGV0ZSBtYXBw aW5nIChieSAxIGJ5dGUpIG9mIG1lbW9yeSByZWdpb25zIGlmIHN2cSBpcyBlbmFibGVkLgo+ID4g PiA+ID4gPiB2MyBsaW5rOgo+ID4gPiA+ID4gPiBodHRwczovL2xvcmUua2VybmVsLm9yZy9xZW11 LWRldmVsLzIwMjIwMzAyMjAzMDEyLjM0NzY4MzUtMS1lcGVyZXptYUByZWRoYXQuY29tLwo+ID4g PiA+ID4gPgo+ID4gPiA+ID4gPiBDaGFuZ2VzIGZyb20gdjI6Cj4gPiA+ID4gPiA+ICogTGVzcyBh c3NlcnRpb25zIGFuZCBtb3JlIGVycm9yIGhhbmRsaW5nIGluIGlvdmEgdHJlZSBjb2RlLgo+ID4g PiA+ID4gPiAqIEJldHRlciBkb2N1bWVudGF0aW9uLCBib3RoIGZpeGluZyBlcnJvcnMgYW5kIG1h a2luZyBAcGFyYW06IGZvcm1hdAo+ID4gPiA+ID4gPiAqIEhvbW9nZW5laXplIFNWUSBhdmFpbF9p ZHhfc2hhZG93IGFuZCBzaGFkb3dfdXNlZF9pZHggdG8gbWFrZSBzaGFkb3cgYQo+ID4gPiA+ID4g PiAgICBwcmVmaXggYXQgYm90aCB0aW1lcy4KPiA+ID4gPiA+ID4gKiBGaXg6IEZvIG5vdCB1c2Ug VmlydFF1ZXVlRWxlbWVudC0+bGVuIGZpZWxkLCB0cmFjayBzZXBhcmF0ZWRseS4KPiA+ID4gPiA+ ID4gKiBTcGxpdCB2aG9zdF9zdnFfe2VuYWJsZSxkaXNhYmxlfV9ub3RpZmljYXRpb24sIHNvIHRo ZSBjb2RlIGxvb2tzIG1vcmUKPiA+ID4gPiA+ID4gICAgbGlrZSB0aGUga2VybmVsIGRyaXZlciBj b2RlLgo+ID4gPiA+ID4gPiAqIFNtYWxsIGltcHJvdmVtZW50cy4KPiA+ID4gPiA+ID4gdjIgbGlu azoKPiA+ID4gPiA+ID4gaHR0cHM6Ly9sb3JlLmtlcm5lbC5vcmcvYWxsL0NBSmFxeVdmWEhFMEM1 NFJfLU9pd0p6akMwZ1Bwa0UzZVgwTDhCZWVaWEdtMUVSWVB0QUBtYWlsLmdtYWlsLmNvbS8KPiA+ ID4gPiA+ID4KPiA+ID4gPiA+ID4gQ2hhbmdlcyBmcm9tIHYxOgo+ID4gPiA+ID4gPiAqIEZlYXR1 cmUgc2V0IGF0IGRldmljZS0+U1ZRIGlzIG5vdyB0aGUgc2FtZSBhcyBTVlEtPmd1ZXN0Lgo+ID4g PiA+ID4gPiAqIFNpemUgb2YgU1ZRIGlzIG5vdCBtYXggYXZhaWxhYmxlIGRldmljZSBzaXplIGFu eW1vcmUsIGJ1dCBndWVzdCdzCj4gPiA+ID4gPiA+ICAgIG5lZ290aWF0ZWQuCj4gPiA+ID4gPiA+ ICogQWRkIFZIT1NUX0ZJTEVfVU5CSU5EIGtpY2sgYW5kIGNhbGwgZmQgdHJlYXRtZW50Lgo+ID4g PiA+ID4gPiAqIE1ha2UgU1ZRIGEgcHVibGljIHN0cnVjdAo+ID4gPiA+ID4gPiAqIENvbWUgYmFj ayB0byBwcmV2aW91cyBhcHByb2FjaCB0byBpb3ZhLXRyZWUKPiA+ID4gPiA+ID4gKiBTb21lIGFz c2VydGlvbnMgYXJlIG5vdyBmYWlsIHBhdGhzLiBTb21lIGVycm9ycyBhcmUgbm93IGxvZ19ndWVz dC4KPiA+ID4gPiA+ID4gKiBPbmx5IG1hc2sgX0ZfTE9HIGZlYXR1cmUgYXQgdmRwYV9zZXRfZmVh dHVyZXMgc3ZxIGVuYWJsZSBwYXRoLgo+ID4gPiA+ID4gPiAqIFJlZmFjdG9yIHNvbWUgZXJyb3Jz IGFuZCBtZXNzYWdlcy4gQWRkIG1pc3NpbmcgZXJyb3IgdW53aW5kaW5ncy4KPiA+ID4gPiA+ID4g KiBBZGQgbWVtb3J5IGJhcnJpZXIgYXQgX0ZfTk9fTk9USUZZIHNldC4KPiA+ID4gPiA+ID4gKiBT dG9wIGNoZWNraW5nIGZvciBmZWF0dXJlcyBmbGFncyBvdXQgb2YgdHJhbnNwb3J0IHJhbmdlLgo+ ID4gPiA+ID4gPiB2MSBsaW5rOgo+ID4gPiA+ID4gPiBodHRwczovL2xvcmUua2VybmVsLm9yZy92 aXJ0dWFsaXphdGlvbi83ZDg2YzcxNS02ZDcxLThhMjctOTFmNS04ZDQ3YjcxZTMyMDFAcmVkaGF0 LmNvbS8KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gQ2hhbmdlcyBmcm9tIHY0IFJGQzoKPiA+ID4g PiA+ID4gKiBTdXBwb3J0IG9mIGFsbG9jYXRpbmcgLyBmcmVlaW5nIGlvdmEgcmFuZ2VzIGluIElP VkEgdHJlZS4gRXh0ZW5kaW5nCj4gPiA+ID4gPiA+ICAgIGFscmVhZHkgcHJlc2VudCBpb3ZhLXRy ZWUgZm9yIHRoYXQuCj4gPiA+ID4gPiA+ICogUHJvcGVyIHZhbGlkYXRpb24gb2YgZ3Vlc3QgZmVh dHVyZXMuIE5vdyBTVlEgY2FuIG5lZ290aWF0ZSBhCj4gPiA+ID4gPiA+ICAgIGRpZmZlcmVudCBz ZXQgb2YgZmVhdHVyZXMgd2l0aCB0aGUgZGV2aWNlIHdoZW4gZW5hYmxlZC4KPiA+ID4gPiA+ID4g KiBTdXBwb3J0IG9mIGhvc3Qgbm90aWZpZXJzIG1lbW9yeSByZWdpb25zCj4gPiA+ID4gPiA+ICog SGFuZGxpbmcgb2YgU1ZRIGZ1bGwgcXVldWUgaW4gY2FzZSBndWVzdCdzIGRlc2NyaXB0b3JzIHNw YW4gdG8KPiA+ID4gPiA+ID4gICAgZGlmZmVyZW50IG1lbW9yeSByZWdpb25zIChxZW11J3MgVkEg Y2h1bmtzKS4KPiA+ID4gPiA+ID4gKiBGbHVzaCBwZW5kaW5nIHVzZWQgYnVmZmVycyBhdCBlbmQg b2YgU1ZRIG9wZXJhdGlvbi4KPiA+ID4gPiA+ID4gKiBRTVAgY29tbWFuZCBub3cgbG9va3MgYnkg TmV0Q2xpZW50U3RhdGUgbmFtZS4gT3RoZXIgZGV2aWNlcyB3aWxsIG5lZWQKPiA+ID4gPiA+ID4g ICAgdG8gaW1wbGVtZW50IGl0J3Mgd2F5IHRvIGVuYWJsZSB2ZHBhLgo+ID4gPiA+ID4gPiAqIFJl bmFtZSBRTVAgY29tbWFuZCB0byBzZXQsIHNvIGl0IGxvb2tzIG1vcmUgbGlrZSBhIHdheSBvZiB3 b3JraW5nCj4gPiA+ID4gPiA+ICogQmV0dGVyIHVzZSBvZiBxZW11IGVycm9yIHN5c3RlbQo+ID4g PiA+ID4gPiAqIE1ha2UgYSBmZXcgYXNzZXJ0aW9ucyBwcm9wZXIgZXJyb3ItaGFuZGxpbmcgcGF0 aHMuCj4gPiA+ID4gPiA+ICogQWRkIG1vcmUgZG9jdW1lbnRhdGlvbgo+ID4gPiA+ID4gPiAqIExl c3MgY291cGxpbmcgb2YgdmlydGlvIC8gdmhvc3QsIHRoYXQgY291bGQgY2F1c2UgZnJpY3Rpb24g b24gY2hhbmdlcwo+ID4gPiA+ID4gPiAqIEFkZHJlc3NlZCBtYW55IG90aGVyIHNtYWxsIGNvbW1l bnRzIGFuZCBzbWFsbCBmaXhlcy4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gQ2hhbmdlcyBmcm9t IHYzIFJGQzoKPiA+ID4gPiA+ID4gICAgKiBNb3ZlIGV2ZXJ5dGhpbmcgdG8gdmhvc3QtdmRwYSBi YWNrZW5kLiBBIGJpZyBjaGFuZ2UsIHRoaXMgYWxsb3dlZAo+ID4gPiA+ID4gPiAgICAgIHNvbWUg Y2xlYW51cCBidXQgbW9yZSBjb2RlIGhhcyBiZWVuIGFkZGVkIGluIG90aGVyIHBsYWNlcy4KPiA+ ID4gPiA+ID4gICAgKiBNb3JlIHVzZSBvZiBnbGliIHV0aWxpdGllcywgZXNwZWNpYWxseSB0byBt YW5hZ2UgbWVtb3J5Lgo+ID4gPiA+ID4gPiB2MyBsaW5rOgo+ID4gPiA+ID4gPiBodHRwczovL2xp c3RzLm5vbmdudS5vcmcvYXJjaGl2ZS9odG1sL3FlbXUtZGV2ZWwvMjAyMS0wNS9tc2cwNjAzMi5o dG1sCj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+IENoYW5nZXMgZnJvbSB2MiBSRkM6Cj4gPiA+ID4g PiA+ICAgICogQWRkaW5nIHZob3N0LXZkcGEgZGV2aWNlcyBzdXBwb3J0Cj4gPiA+ID4gPiA+ICAg ICogRml4ZWQgc29tZSBtZW1vcnkgbGVha3MgcG9pbnRlZCBieSBkaWZmZXJlbnQgY29tbWVudHMK PiA+ID4gPiA+ID4gdjIgbGluazoKPiA+ID4gPiA+ID4gaHR0cHM6Ly9saXN0cy5ub25nbnUub3Jn L2FyY2hpdmUvaHRtbC9xZW11LWRldmVsLzIwMjEtMDMvbXNnMDU2MDAuaHRtbAo+ID4gPiA+ID4g Pgo+ID4gPiA+ID4gPiBDaGFuZ2VzIGZyb20gdjEgUkZDOgo+ID4gPiA+ID4gPiAgICAqIFVzZSBR TVAgaW5zdGVhZCBvZiBtaWdyYXRpb24gdG8gc3RhcnQgU1ZRIG1vZGUuCj4gPiA+ID4gPiA+ICAg ICogT25seSBhY2NlcHRpbmcgSU9NTVUgZGV2aWNlcywgY2xvc2VyIGJlaGF2aW9yIHdpdGggdGFy Z2V0IGRldmljZXMKPiA+ID4gPiA+ID4gICAgICAodkRQQSkKPiA+ID4gPiA+ID4gICAgKiBGaXgg aW52YWxpZCBtYXNraW5nL3VubWFza2luZyBvZiB2aG9zdCBjYWxsIGZkLgo+ID4gPiA+ID4gPiAg ICAqIFVzZSBvZiBwcm9wZXIgbWV0aG9kcyBmb3Igc3luY2hyb25pemF0aW9uLgo+ID4gPiA+ID4g PiAgICAqIE5vIG5lZWQgdG8gbW9kaWZ5IFZpcnRJTyBkZXZpY2UgY29kZSwgYWxsIG9mIHRoZSBj aGFuZ2VzIGFyZQo+ID4gPiA+ID4gPiAgICAgIGNvbnRhaW5lZCBpbiB2aG9zdCBjb2RlLgo+ID4g PiA+ID4gPiAgICAqIERlbGV0ZSBzdXBlcmZsdW91cyBjb2RlLgo+ID4gPiA+ID4gPiAgICAqIEFu IGludGVybWVkaWF0ZSBSRkMgd2FzIHNlbnQgd2l0aCBvbmx5IHRoZSBub3RpZmljYXRpb25zIGZv cndhcmRpbmcKPiA+ID4gPiA+ID4gICAgICBjaGFuZ2VzLiBJdCBjYW4gYmUgc2VlbiBpbgo+ID4g PiA+ID4gPiAgICAgIGh0dHBzOi8vcGF0Y2hldy5vcmcvUUVNVS8yMDIxMDEyOTIwNTQxNS44NzYy OTAtMS1lcGVyZXptYUByZWRoYXQuY29tLwo+ID4gPiA+ID4gPiB2MSBsaW5rOgo+ID4gPiA+ID4g PiBodHRwczovL2xpc3RzLmdudS5vcmcvYXJjaGl2ZS9odG1sL3FlbXUtZGV2ZWwvMjAyMC0xMS9t c2cwNTM3Mi5odG1sCj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+IEV1Z2VuaW8gUMOpcmV6ICgyMCk6 Cj4gPiA+ID4gPiA+ICAgICAgICB2aXJ0aW86IEFkZCBWSVJUSU9fRl9RVUVVRV9TVEFURQo+ID4g PiA+ID4gPiAgICAgICAgdmlydGlvLW5ldDogSG9ub3IgVklSVElPX0NPTkZJR19TX0RFVklDRV9T VE9QUEVECj4gPiA+ID4gPiA+ICAgICAgICB2aXJ0aW86IEFkZCB2aXJ0aW9fcXVldWVfaXNfaG9z dF9ub3RpZmllcl9lbmFibGVkCj4gPiA+ID4gPiA+ICAgICAgICB2aG9zdDogTWFrZSB2aG9zdF92 aXJ0cXVldWVfe3N0YXJ0LHN0b3B9IHB1YmxpYwo+ID4gPiA+ID4gPiAgICAgICAgdmhvc3Q6IEFk ZCB4LXZob3N0LWVuYWJsZS1zaGFkb3ctdnEgcW1wCj4gPiA+ID4gPiA+ICAgICAgICB2aG9zdDog QWRkIFZob3N0U2hhZG93VmlydHF1ZXVlCj4gPiA+ID4gPiA+ICAgICAgICB2ZHBhOiBSZWdpc3Rl ciB2ZHBhIGRldmljZXMgaW4gYSBsaXN0Cj4gPiA+ID4gPiA+ICAgICAgICB2aG9zdDogUm91dGUg Z3Vlc3QtPmhvc3Qgbm90aWZpY2F0aW9uIHRocm91Z2ggc2hhZG93IHZpcnRxdWV1ZQo+ID4gPiA+ ID4gPiAgICAgICAgQWRkIHZob3N0X3N2cV9nZXRfc3ZxX2NhbGxfbm90aWZpZXIKPiA+ID4gPiA+ ID4gICAgICAgIEFkZCB2aG9zdF9zdnFfc2V0X2d1ZXN0X2NhbGxfbm90aWZpZXIKPiA+ID4gPiA+ ID4gICAgICAgIHZkcGE6IFNhdmUgY2FsbF9mZCBpbiB2aG9zdC12ZHBhCj4gPiA+ID4gPiA+ICAg ICAgICB2aG9zdC12ZHBhOiBUYWtlIGludG8gYWNjb3VudCBTVlEgaW4gdmhvc3RfdmRwYV9zZXRf dnJpbmdfY2FsbAo+ID4gPiA+ID4gPiAgICAgICAgdmhvc3Q6IFJvdXRlIGhvc3QtPmd1ZXN0IG5v dGlmaWNhdGlvbiB0aHJvdWdoIHNoYWRvdyB2aXJ0cXVldWUKPiA+ID4gPiA+ID4gICAgICAgIHZp cnRpbzogQWRkIHZob3N0X3NoYWRvd192cV9nZXRfdnJpbmdfYWRkcgo+ID4gPiA+ID4gPiAgICAg ICAgdmRwYTogU2F2ZSBob3N0IGFuZCBndWVzdCBmZWF0dXJlcwo+ID4gPiA+ID4gPiAgICAgICAg dmhvc3Q6IEFkZCB2aG9zdF9zdnFfdmFsaWRfZGV2aWNlX2ZlYXR1cmVzIHRvIHNoYWRvdyB2cQo+ ID4gPiA+ID4gPiAgICAgICAgdmhvc3Q6IFNoYWRvdyB2aXJ0cXVldWUgYnVmZmVycyBmb3J3YXJk aW5nCj4gPiA+ID4gPiA+ICAgICAgICB2aG9zdDogQWRkIFZob3N0SU9WQVRyZWUKPiA+ID4gPiA+ ID4gICAgICAgIHZob3N0OiBVc2UgYSB0cmVlIHRvIHN0b3JlIG1lbW9yeSBtYXBwaW5ncwo+ID4g PiA+ID4gPiAgICAgICAgdmRwYTogQWRkIGN1c3RvbSBJT1RMQiB0cmFuc2xhdGlvbnMgdG8gU1ZR Cj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+IEV1Z2VuaW8gUMOpcmV6ICgxNSk6Cj4gPiA+ID4gPiA+ ICAgIHZob3N0OiBBZGQgVmhvc3RTaGFkb3dWaXJ0cXVldWUKPiA+ID4gPiA+ID4gICAgdmhvc3Q6 IEFkZCBTaGFkb3cgVmlydFF1ZXVlIGtpY2sgZm9yd2FyZGluZyBjYXBhYmlsaXRpZXMKPiA+ID4g PiA+ID4gICAgdmhvc3Q6IEFkZCBTaGFkb3cgVmlydFF1ZXVlIGNhbGwgZm9yd2FyZGluZyBjYXBh YmlsaXRpZXMKPiA+ID4gPiA+ID4gICAgdmhvc3Q6IEFkZCB2aG9zdF9zdnFfdmFsaWRfZmVhdHVy ZXMgdG8gc2hhZG93IHZxCj4gPiA+ID4gPiA+ICAgIHZpcnRpbzogQWRkIHZob3N0X3N2cV9nZXRf dnJpbmdfYWRkcgo+ID4gPiA+ID4gPiAgICB2ZHBhOiBhZGFwdCB2aG9zdF9vcHMgY2FsbGJhY2tz IHRvIHN2cQo+ID4gPiA+ID4gPiAgICB2aG9zdDogU2hhZG93IHZpcnRxdWV1ZSBidWZmZXJzIGZv cndhcmRpbmcKPiA+ID4gPiA+ID4gICAgdXRpbDogQWRkIGlvdmFfdHJlZV9hbGxvY19tYXAKPiA+ ID4gPiA+ID4gICAgdXRpbDogYWRkIGlvdmFfdHJlZV9maW5kX2lvdmEKPiA+ID4gPiA+ID4gICAg dmhvc3Q6IEFkZCBWaG9zdElPVkFUcmVlCj4gPiA+ID4gPiA+ICAgIHZkcGE6IEFkZCBjdXN0b20g SU9UTEIgdHJhbnNsYXRpb25zIHRvIFNWUQo+ID4gPiA+ID4gPiAgICB2ZHBhOiBBZGFwdCB2aG9z dF92ZHBhX2dldF92cmluZ19iYXNlIHRvIFNWUQo+ID4gPiA+ID4gPiAgICB2ZHBhOiBOZXZlciBz ZXQgbG9nX2Jhc2UgYWRkciBpZiBTVlEgaXMgZW5hYmxlZAo+ID4gPiA+ID4gPiAgICB2ZHBhOiBF eHBvc2UgVkhPU1RfRl9MT0dfQUxMIG9uIFNWUQo+ID4gPiA+ID4gPiAgICB2ZHBhOiBBZGQgeC1z dnEgdG8gTmV0ZGV2Vmhvc3RWRFBBT3B0aW9ucwo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiAgIHFh cGkvbmV0Lmpzb24gICAgICAgICAgICAgICAgICAgICAgfCAgIDggKy0KPiA+ID4gPiA+ID4gICBo dy92aXJ0aW8vdmhvc3QtaW92YS10cmVlLmggICAgICAgIHwgIDI3ICsrCj4gPiA+ID4gPiA+ICAg aHcvdmlydGlvL3Zob3N0LXNoYWRvdy12aXJ0cXVldWUuaCB8ICA4NyArKysrCj4gPiA+ID4gPiA+ ICAgaW5jbHVkZS9ody92aXJ0aW8vdmhvc3QtdmRwYS5oICAgICB8ICAgOCArCj4gPiA+ID4gPiA+ ICAgaW5jbHVkZS9xZW11L2lvdmEtdHJlZS5oICAgICAgICAgICB8ICAzOCArLQo+ID4gPiA+ID4g PiAgIGh3L3ZpcnRpby92aG9zdC1pb3ZhLXRyZWUuYyAgICAgICAgfCAxMTAgKysrKysKPiA+ID4g PiA+ID4gICBody92aXJ0aW8vdmhvc3Qtc2hhZG93LXZpcnRxdWV1ZS5jIHwgNjM3ICsrKysrKysr KysrKysrKysrKysrKysrKysrKysrCj4gPiA+ID4gPiA+ICAgaHcvdmlydGlvL3Zob3N0LXZkcGEu YyAgICAgICAgICAgICB8IDUyNSArKysrKysrKysrKysrKysrKysrKysrKy0KPiA+ID4gPiA+ID4g ICBuZXQvdmhvc3QtdmRwYS5jICAgICAgICAgICAgICAgICAgIHwgIDQ4ICsrLQo+ID4gPiA+ID4g PiAgIHV0aWwvaW92YS10cmVlLmMgICAgICAgICAgICAgICAgICAgfCAxNjkgKysrKysrKysKPiA+ ID4gPiA+ID4gICBody92aXJ0aW8vbWVzb24uYnVpbGQgICAgICAgICAgICAgIHwgICAyICstCj4g PiA+ID4gPiA+ICAgMTEgZmlsZXMgY2hhbmdlZCwgMTYzMyBpbnNlcnRpb25zKCspLCAyNiBkZWxl dGlvbnMoLSkKPiA+ID4gPiA+ID4gICBjcmVhdGUgbW9kZSAxMDA2NDQgaHcvdmlydGlvL3Zob3N0 LWlvdmEtdHJlZS5oCj4gPiA+ID4gPiA+ICAgY3JlYXRlIG1vZGUgMTAwNjQ0IGh3L3ZpcnRpby92 aG9zdC1zaGFkb3ctdmlydHF1ZXVlLmgKPiA+ID4gPiA+ID4gICBjcmVhdGUgbW9kZSAxMDA2NDQg aHcvdmlydGlvL3Zob3N0LWlvdmEtdHJlZS5jCj4gPiA+ID4gPiA+ICAgY3JlYXRlIG1vZGUgMTAw NjQ0IGh3L3ZpcnRpby92aG9zdC1zaGFkb3ctdmlydHF1ZXVlLmMKPiA+ID4gPiA+ID4KPiA+ID4g Pgo+ID4KCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fClZp cnR1YWxpemF0aW9uIG1haWxpbmcgbGlzdApWaXJ0dWFsaXphdGlvbkBsaXN0cy5saW51eC1mb3Vu ZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0 aW5mby92aXJ0dWFsaXphdGlvbg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2E99C43217 for ; Tue, 8 Mar 2022 08:02:13 +0000 (UTC) Received: from localhost ([::1]:59346 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nRUnM-0000dk-Il for qemu-devel@archiver.kernel.org; Tue, 08 Mar 2022 03:02:12 -0500 Received: from eggs.gnu.org ([209.51.188.92]:50954) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRUgy-0000kQ-Ne for qemu-devel@nongnu.org; Tue, 08 Mar 2022 02:55:36 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:31928) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRUgv-0002yy-Sc for qemu-devel@nongnu.org; Tue, 08 Mar 2022 02:55:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646726132; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UFezi77WesJBzFK1fhaAbsIyGgE1choY2kdQrE0mkS4=; b=TxSEOjaWCyzaU8p/9BN5E56ld8k2nKKFISXNFYUTZgqqaqJ7I86ME0Jxj0+jYnzVPFPZCN uoPYl9KXBb1US+dzIPsBUTnyy6nbextAm3c4B3tFoYWLJzXCJ8/IJNJixyJdglS9UDA/Se l/54tXQ5vsnPqsKOzVP7UwpyT5AqfAM= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-593-skTA-VQtMySDyinFBCJNkg-1; Tue, 08 Mar 2022 02:55:31 -0500 X-MC-Unique: skTA-VQtMySDyinFBCJNkg-1 Received: by mail-ed1-f71.google.com with SMTP id cm27-20020a0564020c9b00b004137effc24bso10135386edb.10 for ; Mon, 07 Mar 2022 23:55:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=UFezi77WesJBzFK1fhaAbsIyGgE1choY2kdQrE0mkS4=; b=bvI3KyEyk75F/1+EYvh9rYRIIL+rYowAfGL3HNewOkyS7u6PNfVkIyzFz87kWMpjI9 Ri9VLc+Ozx6UrUV7JTaggpG6QGANRdgoqKI9T6JeQwHZxttw271bFvfuN3WTj/lH9jOQ St9oYVePfE3xA+rSuh6zp6ebsmKeOVPu+80yd0E4pLG86+VueG4uQljhwUD+H3MTviD8 lxMaMPo47+j081lkg9QBq15BRfaYNaL4X0bZqbKvs+48cIM7CHEkvF4CPFjid10CirdQ EPQZRypptYpQ322Xv98NHgiwHlsOGno2ol5eSCnEcwoT6+/SRIDGZwIqIcZ7Ysv1Z3Rd vPRA== X-Gm-Message-State: AOAM530Fi4q8FgI62mXO/h3CkqBeDIS5ZBHaLXi4OUUCl0S955UPEl0L 5X4eyRvulOqr8NzfctW9oC3GCw0TQmiUbz1zdYjxVQbipJSbwPXv/Hteh6vui+VzveyD1G/GVtU 620PO+Huf5BwZhXg= X-Received: by 2002:a17:907:160d:b0:6db:49bb:2c7a with SMTP id hb13-20020a170907160d00b006db49bb2c7amr1760431ejc.228.1646726129745; Mon, 07 Mar 2022 23:55:29 -0800 (PST) X-Google-Smtp-Source: ABdhPJzPLIh9zjOVAW9d23M/7A7pRnUKI7iQh9+Zh18H1bTRewg2yW6RP5RnWMHUmqjcD9h29Kldxw== X-Received: by 2002:a17:907:160d:b0:6db:49bb:2c7a with SMTP id hb13-20020a170907160d00b006db49bb2c7amr1760400ejc.228.1646726129410; Mon, 07 Mar 2022 23:55:29 -0800 (PST) Received: from redhat.com ([2.55.138.228]) by smtp.gmail.com with ESMTPSA id c1-20020a170906762100b006d00ae72b0csm5465608ejn.221.2022.03.07.23.55.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Mar 2022 23:55:28 -0800 (PST) Date: Tue, 8 Mar 2022 02:55:23 -0500 From: "Michael S. Tsirkin" To: Jason Wang Subject: Re: [PATCH v5 00/15] vDPA shadow virtqueue Message-ID: <20220308024724-mutt-send-email-mst@kernel.org> References: <20220307153334.3854134-1-eperezma@redhat.com> <14d4fde4-6ea5-4805-b684-c33f6b448565@redhat.com> <20220308020348-mutt-send-email-mst@kernel.org> <20220308022300-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 In-Reply-To: Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mst@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel , Peter Xu , virtualization , Eli Cohen , Eric Blake , Eduardo Habkost , Cindy Lu , "Fangyi \(Eric\)" , Markus Armbruster , yebiaoxiang@huawei.com, Eugenio =?iso-8859-1?Q?P=E9rez?= , Liuxiangdong , Stefano Garzarella , Laurent Vivier , Parav Pandit , Richard Henderson , Gautam Dawar , Xiao W Wang , Stefan Hajnoczi , Juan Quintela , Harpreet Singh Anand , Lingshan Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On Tue, Mar 08, 2022 at 03:34:17PM +0800, Jason Wang wrote: > On Tue, Mar 8, 2022 at 3:28 PM Michael S. Tsirkin wrote: > > > > On Tue, Mar 08, 2022 at 03:14:35PM +0800, Jason Wang wrote: > > > On Tue, Mar 8, 2022 at 3:11 PM Michael S. Tsirkin wrote: > > > > > > > > On Tue, Mar 08, 2022 at 02:03:32PM +0800, Jason Wang wrote: > > > > > > > > > > 在 2022/3/7 下午11:33, Eugenio Pérez 写道: > > > > > > This series enable shadow virtqueue (SVQ) for vhost-vdpa devices. This > > > > > > is intended as a new method of tracking the memory the devices touch > > > > > > during a migration process: Instead of relay on vhost device's dirty > > > > > > logging capability, SVQ intercepts the VQ dataplane forwarding the > > > > > > descriptors between VM and device. This way qemu is the effective > > > > > > writer of guests memory, like in qemu's virtio device operation. > > > > > > > > > > > > When SVQ is enabled qemu offers a new virtual address space to the > > > > > > device to read and write into, and it maps new vrings and the guest > > > > > > memory in it. SVQ also intercepts kicks and calls between the device > > > > > > and the guest. Used buffers relay would cause dirty memory being > > > > > > tracked. > > > > > > > > > > > > This effectively means that vDPA device passthrough is intercepted by > > > > > > qemu. While SVQ should only be enabled at migration time, the switching > > > > > > from regular mode to SVQ mode is left for a future series. > > > > > > > > > > > > It is based on the ideas of DPDK SW assisted LM, in the series of > > > > > > DPDK's https://patchwork.dpdk.org/cover/48370/ . However, these does > > > > > > not map the shadow vq in guest's VA, but in qemu's. > > > > > > > > > > > > For qemu to use shadow virtqueues the guest virtio driver must not use > > > > > > features like event_idx. > > > > > > > > > > > > SVQ needs to be enabled with cmdline: > > > > > > > > > > > > -netdev type=vhost-vdpa,vhostdev=vhost-vdpa-0,id=vhost-vdpa0,svq=on > > > > > > > > A stable API for an incomplete feature is a problem imho. > > > > > > It should be "x-svq". > > > > > > Well look at patch 15. > > It's a bug that needs to be fixed. > > > > > > > > > > > > > > > > > > > > > > > The first three patches enables notifications forwarding with > > > > > > assistance of qemu. It's easy to enable only this if the relevant > > > > > > cmdline part of the last patch is applied on top of these. > > > > > > > > > > > > Next four patches implement the actual buffer forwarding. However, > > > > > > address are not translated from HVA so they will need a host device with > > > > > > an iommu allowing them to access all of the HVA range. > > > > > > > > > > > > The last part of the series uses properly the host iommu, so qemu > > > > > > creates a new iova address space in the device's range and translates > > > > > > the buffers in it. Finally, it adds the cmdline parameter. > > > > > > > > > > > > Some simple performance tests with netperf were done. They used a nested > > > > > > guest with vp_vdpa, vhost-kernel at L0 host. Starting with no svq and a > > > > > > baseline average of ~9009.96Mbps: > > > > > > Recv Send Send > > > > > > Socket Socket Message Elapsed > > > > > > Size Size Size Time Throughput > > > > > > bytes bytes bytes secs. 10^6bits/sec > > > > > > 131072 16384 16384 30.01 9061.03 > > > > > > 131072 16384 16384 30.01 8962.94 > > > > > > 131072 16384 16384 30.01 9005.92 > > > > > > > > > > > > To enable SVQ buffers forwarding reduce throughput to about > > > > > > Recv Send Send > > > > > > Socket Socket Message Elapsed > > > > > > Size Size Size Time Throughput > > > > > > bytes bytes bytes secs. 10^6bits/sec > > > > > > 131072 16384 16384 30.01 7689.72 > > > > > > 131072 16384 16384 30.00 7752.07 > > > > > > 131072 16384 16384 30.01 7750.30 > > > > > > > > > > > > However, many performance improvements were left out of this series for > > > > > > simplicity, so difference should shrink in the future. > > > > > > > > > > > > Comments are welcome. > > > > > > > > > > > > > > > Hi Michael: > > > > > > > > > > What do you think of this series? It looks good to me as a start. The > > > > > feature could only be enabled as a dedicated parameter. If you're ok, I'd > > > > > try to make it for 7.0. > > > > > > > > > > Thanks > > > > > > > > Well that's cutting it awfully close, and it's not really useful > > > > at the current stage, is it? > > > > > > This allows vDPA to be migrated when using "x-svq=on". > > > But anyhow it's > > > experimental. > > > > it's less experimental than incomplete. It seems pretty clearly not > > the way it will work down the road, we don't want svq involved > > at all times. > > Right, but SVQ could be used for other places e.g providing migration > compatibility when the destination lacks some features. In its current form? I don't see how. Generally? Maybe but I suspect we'll have to rework it completely for that. > > > > > > > > > > The IOVA trick does not feel complete either. > > > > > > I don't get here. We don't use any IOVA trick as DPDK (it reserve IOVA > > > for shadow vq) did. So we won't suffer from the issues of DPDK. > > > > > > Thanks > > > > Maybe I misundrstand how this all works. > > I refer to all the iova_tree_alloc_map things. > > It's a simple IOVA allocater actually. Anything wrong with that? Not by itself but I'm not sure we can guarantee guest will not attempt to use the IOVA addresses we are reserving down the road. > I'm fine with making it for the future release. > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > TODO on future series: > > > > > > * Event, indirect, packed, and others features of virtio. > > > > > > * To support different set of features between the device<->SVQ and the > > > > > > SVQ<->guest communication. > > > > > > * Support of device host notifier memory regions. > > > > > > * To sepparate buffers forwarding in its own AIO context, so we can > > > > > > throw more threads to that task and we don't need to stop the main > > > > > > event loop. > > > > > > * Support multiqueue virtio-net vdpa. > > > > > > * Proper documentation. > > > > > > > > > > > > Changes from v4: > > > > > > * Iterate iova->hva tree instead on maintain own tree so we support HVA > > > > > > overlaps. > > > > > > * Fix: Errno completion at failure. > > > > > > * Rename x-svq to svq, so changes to stable does not affect cmdline parameter. > > > > > > > > > > > > Changes from v3: > > > > > > * Add @unstable feature to NetdevVhostVDPAOptions.x-svq. > > > > > > * Fix uncomplete mapping (by 1 byte) of memory regions if svq is enabled. > > > > > > v3 link: > > > > > > https://lore.kernel.org/qemu-devel/20220302203012.3476835-1-eperezma@redhat.com/ > > > > > > > > > > > > Changes from v2: > > > > > > * Less assertions and more error handling in iova tree code. > > > > > > * Better documentation, both fixing errors and making @param: format > > > > > > * Homogeneize SVQ avail_idx_shadow and shadow_used_idx to make shadow a > > > > > > prefix at both times. > > > > > > * Fix: Fo not use VirtQueueElement->len field, track separatedly. > > > > > > * Split vhost_svq_{enable,disable}_notification, so the code looks more > > > > > > like the kernel driver code. > > > > > > * Small improvements. > > > > > > v2 link: > > > > > > https://lore.kernel.org/all/CAJaqyWfXHE0C54R_-OiwJzjC0gPpkE3eX0L8BeeZXGm1ERYPtA@mail.gmail.com/ > > > > > > > > > > > > Changes from v1: > > > > > > * Feature set at device->SVQ is now the same as SVQ->guest. > > > > > > * Size of SVQ is not max available device size anymore, but guest's > > > > > > negotiated. > > > > > > * Add VHOST_FILE_UNBIND kick and call fd treatment. > > > > > > * Make SVQ a public struct > > > > > > * Come back to previous approach to iova-tree > > > > > > * Some assertions are now fail paths. Some errors are now log_guest. > > > > > > * Only mask _F_LOG feature at vdpa_set_features svq enable path. > > > > > > * Refactor some errors and messages. Add missing error unwindings. > > > > > > * Add memory barrier at _F_NO_NOTIFY set. > > > > > > * Stop checking for features flags out of transport range. > > > > > > v1 link: > > > > > > https://lore.kernel.org/virtualization/7d86c715-6d71-8a27-91f5-8d47b71e3201@redhat.com/ > > > > > > > > > > > > Changes from v4 RFC: > > > > > > * Support of allocating / freeing iova ranges in IOVA tree. Extending > > > > > > already present iova-tree for that. > > > > > > * Proper validation of guest features. Now SVQ can negotiate a > > > > > > different set of features with the device when enabled. > > > > > > * Support of host notifiers memory regions > > > > > > * Handling of SVQ full queue in case guest's descriptors span to > > > > > > different memory regions (qemu's VA chunks). > > > > > > * Flush pending used buffers at end of SVQ operation. > > > > > > * QMP command now looks by NetClientState name. Other devices will need > > > > > > to implement it's way to enable vdpa. > > > > > > * Rename QMP command to set, so it looks more like a way of working > > > > > > * Better use of qemu error system > > > > > > * Make a few assertions proper error-handling paths. > > > > > > * Add more documentation > > > > > > * Less coupling of virtio / vhost, that could cause friction on changes > > > > > > * Addressed many other small comments and small fixes. > > > > > > > > > > > > Changes from v3 RFC: > > > > > > * Move everything to vhost-vdpa backend. A big change, this allowed > > > > > > some cleanup but more code has been added in other places. > > > > > > * More use of glib utilities, especially to manage memory. > > > > > > v3 link: > > > > > > https://lists.nongnu.org/archive/html/qemu-devel/2021-05/msg06032.html > > > > > > > > > > > > Changes from v2 RFC: > > > > > > * Adding vhost-vdpa devices support > > > > > > * Fixed some memory leaks pointed by different comments > > > > > > v2 link: > > > > > > https://lists.nongnu.org/archive/html/qemu-devel/2021-03/msg05600.html > > > > > > > > > > > > Changes from v1 RFC: > > > > > > * Use QMP instead of migration to start SVQ mode. > > > > > > * Only accepting IOMMU devices, closer behavior with target devices > > > > > > (vDPA) > > > > > > * Fix invalid masking/unmasking of vhost call fd. > > > > > > * Use of proper methods for synchronization. > > > > > > * No need to modify VirtIO device code, all of the changes are > > > > > > contained in vhost code. > > > > > > * Delete superfluous code. > > > > > > * An intermediate RFC was sent with only the notifications forwarding > > > > > > changes. It can be seen in > > > > > > https://patchew.org/QEMU/20210129205415.876290-1-eperezma@redhat.com/ > > > > > > v1 link: > > > > > > https://lists.gnu.org/archive/html/qemu-devel/2020-11/msg05372.html > > > > > > > > > > > > Eugenio Pérez (20): > > > > > > virtio: Add VIRTIO_F_QUEUE_STATE > > > > > > virtio-net: Honor VIRTIO_CONFIG_S_DEVICE_STOPPED > > > > > > virtio: Add virtio_queue_is_host_notifier_enabled > > > > > > vhost: Make vhost_virtqueue_{start,stop} public > > > > > > vhost: Add x-vhost-enable-shadow-vq qmp > > > > > > vhost: Add VhostShadowVirtqueue > > > > > > vdpa: Register vdpa devices in a list > > > > > > vhost: Route guest->host notification through shadow virtqueue > > > > > > Add vhost_svq_get_svq_call_notifier > > > > > > Add vhost_svq_set_guest_call_notifier > > > > > > vdpa: Save call_fd in vhost-vdpa > > > > > > vhost-vdpa: Take into account SVQ in vhost_vdpa_set_vring_call > > > > > > vhost: Route host->guest notification through shadow virtqueue > > > > > > virtio: Add vhost_shadow_vq_get_vring_addr > > > > > > vdpa: Save host and guest features > > > > > > vhost: Add vhost_svq_valid_device_features to shadow vq > > > > > > vhost: Shadow virtqueue buffers forwarding > > > > > > vhost: Add VhostIOVATree > > > > > > vhost: Use a tree to store memory mappings > > > > > > vdpa: Add custom IOTLB translations to SVQ > > > > > > > > > > > > Eugenio Pérez (15): > > > > > > vhost: Add VhostShadowVirtqueue > > > > > > vhost: Add Shadow VirtQueue kick forwarding capabilities > > > > > > vhost: Add Shadow VirtQueue call forwarding capabilities > > > > > > vhost: Add vhost_svq_valid_features to shadow vq > > > > > > virtio: Add vhost_svq_get_vring_addr > > > > > > vdpa: adapt vhost_ops callbacks to svq > > > > > > vhost: Shadow virtqueue buffers forwarding > > > > > > util: Add iova_tree_alloc_map > > > > > > util: add iova_tree_find_iova > > > > > > vhost: Add VhostIOVATree > > > > > > vdpa: Add custom IOTLB translations to SVQ > > > > > > vdpa: Adapt vhost_vdpa_get_vring_base to SVQ > > > > > > vdpa: Never set log_base addr if SVQ is enabled > > > > > > vdpa: Expose VHOST_F_LOG_ALL on SVQ > > > > > > vdpa: Add x-svq to NetdevVhostVDPAOptions > > > > > > > > > > > > qapi/net.json | 8 +- > > > > > > hw/virtio/vhost-iova-tree.h | 27 ++ > > > > > > hw/virtio/vhost-shadow-virtqueue.h | 87 ++++ > > > > > > include/hw/virtio/vhost-vdpa.h | 8 + > > > > > > include/qemu/iova-tree.h | 38 +- > > > > > > hw/virtio/vhost-iova-tree.c | 110 +++++ > > > > > > hw/virtio/vhost-shadow-virtqueue.c | 637 +++++++++++++++++++++++++++++ > > > > > > hw/virtio/vhost-vdpa.c | 525 +++++++++++++++++++++++- > > > > > > net/vhost-vdpa.c | 48 ++- > > > > > > util/iova-tree.c | 169 ++++++++ > > > > > > hw/virtio/meson.build | 2 +- > > > > > > 11 files changed, 1633 insertions(+), 26 deletions(-) > > > > > > create mode 100644 hw/virtio/vhost-iova-tree.h > > > > > > create mode 100644 hw/virtio/vhost-shadow-virtqueue.h > > > > > > create mode 100644 hw/virtio/vhost-iova-tree.c > > > > > > create mode 100644 hw/virtio/vhost-shadow-virtqueue.c > > > > > > > > > > > >