From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05B52C433F5 for ; Tue, 2 Nov 2021 07:59:33 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8D35860F56 for ; Tue, 2 Nov 2021 07:59:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8D35860F56 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 5D35160682; Tue, 2 Nov 2021 07:59:32 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 594aLSNvW2Py; Tue, 2 Nov 2021 07:59:30 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 3A31E6067B; Tue, 2 Nov 2021 07:59:30 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0DE45C0019; Tue, 2 Nov 2021 07:59:30 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 05606C000E for ; Tue, 2 Nov 2021 07:59:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id DFD4060682 for ; Tue, 2 Nov 2021 07:59:28 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 83lrrT5gQQIK for ; Tue, 2 Nov 2021 07:59:27 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 4FA1C6067B for ; Tue, 2 Nov 2021 07:59:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635839966; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9CvVJ9im3mG5R3xIrXuhU7ug7R+jbBZPgpFPtRunZGA=; b=F3CilOBoa2Rnn1fOek45Rz8zs1CTOW9AyKN0N6vK9tKo0IgNwlMZVdqwljVgAU2CBxfrtH 8NeKqfBuwh0P8X5kE9sxnORv3dk6NLZ5RP3lU20Auej37T38MYAdVdANISs/AMdxIwk5GJ 8DbNyC4NRL6cCqJ4w3Shhu31/pPVUEU= Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-292-2DI-DuzyPSKQzwBg6jLssg-1; Tue, 02 Nov 2021 03:59:25 -0400 X-MC-Unique: 2DI-DuzyPSKQzwBg6jLssg-1 Received: by mail-pf1-f198.google.com with SMTP id y124-20020a623282000000b0047a09271e49so11071663pfy.16 for ; Tue, 02 Nov 2021 00:59:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=9CvVJ9im3mG5R3xIrXuhU7ug7R+jbBZPgpFPtRunZGA=; b=0Eb6L+anjifwMQzXnPoqlD32rqZNXQr0QSIuCn9EqJnRnW6XJ97aOauLZcJB4RUerX uS/PG1tXesaWujm0Sz4/JLD2wme2o9rLqwHxryoIitndLh7nceNum7RRZM7Zsg4zlHvD NgUrAmnFAV8LFfPGl5aqxejnQ26/5kTkZiDm6qK8U9QPBmwONEoa82bj5IGRgdyoUkey MRmzBRCtRTdZ4OQlmRDtqzLNh9jk3kwH003a3LA2tmx+z77fxucTKtWAVzWr+TvvKCP9 1ZUlfABSTC7znJbSi2lv2B0TVH+Q9dDdZQ7ve3cAaiA39zzw0xAI/lCWc5xHj2uKWMVf c2tA== X-Gm-Message-State: AOAM531IOBwov4Aly1PPKgCyehHThC5Twull82XB4tlPQuHKzdxMtmap qYN5iCKAU9Ib9ZmwVleWenX+ka9Co85BaLU7oM3yC4JtbUlZbI7fPKh7uyC1cyHH3bSfgA34C+d x0wTktrHqfEKm4h4aStZgyGABKpecB6xfi/GmdpjlXmC8gN3NI/cZmVEf1HzqAoDH2d/Tx0LNGw wGfpF5bXZ50iYHcJVR6w== X-Received: by 2002:a17:90a:7e86:: with SMTP id j6mr4779255pjl.167.1635839963697; Tue, 02 Nov 2021 00:59:23 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw9gC2wIEhXsLE7drd3TjM966xz3oOWtKGRoZPZuFmAxiFvKOoj/qO+ciofZ51gY0dQ+LL1MA== X-Received: by 2002:a17:90a:7e86:: with SMTP id j6mr4779195pjl.167.1635839963141; Tue, 02 Nov 2021 00:59:23 -0700 (PDT) Received: from wangxiaodeMacBook-Air.local ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id m186sm17712178pfb.165.2021.11.02.00.59.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 02 Nov 2021 00:59:22 -0700 (PDT) Subject: Re: [RFC PATCH v5 22/26] vhost: Shadow virtqueue buffers forwarding To: =?UTF-8?Q?Eugenio_P=c3=a9rez?= , qemu-devel@nongnu.org References: <20211029183525.1776416-1-eperezma@redhat.com> <20211029183525.1776416-23-eperezma@redhat.com> From: Jason Wang Message-ID: <5cc9cb09-8b10-d406-25bf-b8de5b510a07@redhat.com> Date: Tue, 2 Nov 2021 15:59:10 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 MIME-Version: 1.0 In-Reply-To: <20211029183525.1776416-23-eperezma@redhat.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Richard Henderson , Stefan Hajnoczi , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" CuWcqCAyMDIxLzEwLzMwIOS4iuWNiDI6MzUsIEV1Z2VuaW8gUMOpcmV6IOWGmemBkzoKPiBJbml0 aWFsIHZlcnNpb24gb2Ygc2hhZG93IHZpcnRxdWV1ZSB0aGF0IGFjdHVhbGx5IGZvcndhcmQgYnVm ZmVycy4gVGhlcmUKPiBhcmUgbm8gaW9tbXUgc3VwcG9ydCBhdCB0aGUgbW9tZW50LCBhbmQgdGhh dCB3aWxsIGJlIGFkZHJlc3NlZCBpbiBmdXR1cmUKPiBwYXRjaGVzIG9mIHRoaXMgc2VyaWVzLiBT aW5jZSBhbGwgdmhvc3QtdmRwYSBkZXZpY2VzIHVzZXMgZm9yY2VkIElPTU1VLAo+IHRoaXMgbWVh bnMgdGhhdCBTVlEgaXMgbm90IHVzYWJsZSBhdCB0aGlzIHBvaW50IG9mIHRoZSBzZXJpZXMgb24g YW55Cj4gZGV2aWNlLgo+Cj4gRm9yIHNpbXBsaWNpdHkgaXQgb25seSBzdXBwb3J0cyBtb2Rlcm4g ZGV2aWNlcywgdGhhdCBleHBlY3RzIHZyaW5nCj4gaW4gbGl0dGxlIGVuZGlhbiwgd2l0aCBzcGxp dCByaW5nIGFuZCBubyBldmVudCBpZHggb3IgaW5kaXJlY3QKPiBkZXNjcmlwdG9ycy4gU3VwcG9y dCBmb3IgdGhlbSB3aWxsIG5vdCBiZSBhZGRlZCBpbiB0aGlzIHNlcmllcy4KPgo+IEl0IHJldXNl cyB0aGUgVmlydFF1ZXVlIGNvZGUgZm9yIHRoZSBkZXZpY2UgcGFydC4gVGhlIGRyaXZlciBwYXJ0 IGlzCj4gYmFzZWQgb24gTGludXgncyB2aXJ0aW9fcmluZyBkcml2ZXIsIGJ1dCB3aXRoIHN0cmlw cGVkIGZ1bmN0aW9uYWxpdHkKPiBhbmQgb3B0aW1pemF0aW9ucyBzbyBpdCdzIGVhc2llciB0byBy ZXZpZXcuIExhdGVyIGNvbW1pdHMgYWRkIHNpbXBsZXIKPiBvbmVzLgo+Cj4gSG93ZXZlciB0byBm b3J3YXJkaW5nIGJ1ZmZlcnMgaGF2ZSBzb21lIHBhcnRpY3VsYXIgcGllY2VzOiBPbmUgb2YgdGhl Cj4gbW9zdCB1bmV4cGVjdGVkIG9uZXMgaXMgdGhhdCBhIGd1ZXN0J3MgYnVmZmVyIGNhbiBleHBh bmQgdGhyb3VnaCBtb3JlCj4gdGhhbiBvbmUgZGVzY3JpcHRvciBpbiBTVlEuIFdoaWxlIHRoaXMg aXMgaGFuZGxlZCBncmFjZWZ1bGx5IGJ5IHFlbXUncwo+IGVtdWxhdGVkIHZpcnRpbyBkZXZpY2Vz LCBpdCBtYXkgY2F1c2UgdW5leHBlY3RlZCBTVlEgcXVldWUgZnVsbC4gVGhpcwo+IHBhdGNoIGFs c28gc29sdmVzIGl0IGNoZWNraW5nIGZvciB0aGlzIGNvbmRpdGlvbiBhdCBib3RoIGd1ZXN0J3Mg a2lja3MKPiBhbmQgZGV2aWNlJ3MgY2FsbHMuIFRoZSBjb2RlIG1heSBiZSBtb3JlIGVsZWdhbnQg aW4gdGhlIGZ1dHVyZSBpZiBTVlEKPiBjb2RlIHJ1bnMgaW4gaXRzIG93biBpb2NvbnRleHQuCj4K PiBOb3RlIHRoYXQgdmhvc3RfdmRwYV9nZXRfdnFfc3RhdGUgdHJ1c3QgdGhlIGRldmljZSB0byB3 cml0ZSBpdHMgc3RhdHVzCj4gdG8gdXNlZF9pZHggYXQgcGF1c2UoKSwgZmluaXNoaW5nIGFsbCBp bi1mbGlnaHQgZGVzY3JpcHRvcnMuIFRoaXMgbWF5Cj4gbm90IGJlIGVub3VnaCBmb3IgY29tcGxl eCBkZXZpY2VzLCBidXQgb3RoZXIgZGV2ZWxvcG1lbnQgbGlrZSB1c2FnZSBvZgo+IGluZmxpZ2h0 X2ZkIG9uIHRvcCBvZiB0aGlzIHNvbHV0aW9uIG1heSBleHRlbmQgdGhlIHVzYWdlIGluIHRoZSBm dXR1cmUuCj4KPiBJbiBwYXJ0aWN1bGFyLCBTVlEgdHJ1c3QgaXQgdG8gcmVjb3ZlciBndWVzdCdz IHZpcnRxdWV1ZSBhdCBzdGFydCwgYW5kCj4gdG8gbWFyayBhcyB1c2VkIHRoZSBsYXRlc3QgZGVz Y3JpcHRvcnMgdXNlZCBieSB0aGUgZGV2aWNlIGluIHRoZQo+IG1lYW50aW1lLgo+Cj4gU2lnbmVk LW9mZi1ieTogRXVnZW5pbyBQw6lyZXogPGVwZXJlem1hQHJlZGhhdC5jb20+Cj4gLS0tCj4gICBx YXBpL25ldC5qc29uICAgICAgICAgICAgICAgICAgICAgIHwgICA1ICstCj4gICBody92aXJ0aW8v dmhvc3Qtc2hhZG93LXZpcnRxdWV1ZS5jIHwgNDAwICsrKysrKysrKysrKysrKysrKysrKysrKysr Ky0tCj4gICBody92aXJ0aW8vdmhvc3QtdmRwYS5jICAgICAgICAgICAgIHwgMTQ0ICsrKysrKysr KystCj4gICAzIGZpbGVzIGNoYW5nZWQsIDUyMSBpbnNlcnRpb25zKCspLCAyOCBkZWxldGlvbnMo LSkKPgo+IGRpZmYgLS1naXQgYS9xYXBpL25ldC5qc29uIGIvcWFwaS9uZXQuanNvbgo+IGluZGV4 IGZjYTJmNmViY2EuLjFjNmQzYjIxNzkgMTAwNjQ0Cj4gLS0tIGEvcWFwaS9uZXQuanNvbgo+ICsr KyBiL3FhcGkvbmV0Lmpzb24KPiBAQCAtODQsMTIgKzg0LDkgQEAKPiAgICMKPiAgICMgVXNlIHZo b3N0IHNoYWRvdyB2aXJ0cXVldWUuCj4gICAjCj4gLSMgU1ZRIGNhbiBqdXN0IGZvcndhcmQgbm90 aWZpY2F0aW9ucyBiZXR3ZWVuIHRoZSBkZXZpY2UgYW5kIHRoZSBndWVzdCBhdCB0aGlzCj4gLSMg bW9tZW50LiBUaGlzIHdpbGwgZXhwYW5kIGluIGZ1dHVyZSBjaGFuZ2VzLgo+IC0jCj4gICAjIEBu YW1lOiB0aGUgZGV2aWNlIG5hbWUgb2YgdGhlIFZpcnRJTyBkZXZpY2UKPiAgICMKPiAtIyBAc2V0 OiB0cnVlIHRvIHVzZSB0aGUgYWx0ZXJuYXRlIHNoYWRvdyBWUSBub3RpZmljYXRpb25zCj4gKyMg QHNldDogdHJ1ZSB0byB1c2UgdGhlIGFsdGVybmF0ZSBzaGFkb3cgVlEKPiAgICMKPiAgICMgU2lu Y2U6IDYuMgo+ICAgIwo+IGRpZmYgLS1naXQgYS9ody92aXJ0aW8vdmhvc3Qtc2hhZG93LXZpcnRx dWV1ZS5jIGIvaHcvdmlydGlvL3Zob3N0LXNoYWRvdy12aXJ0cXVldWUuYwo+IGluZGV4IGNiOWZm Y2IwMTUuLmFkMWIyMzQyYmUgMTAwNjQ0Cj4gLS0tIGEvaHcvdmlydGlvL3Zob3N0LXNoYWRvdy12 aXJ0cXVldWUuYwo+ICsrKyBiL2h3L3ZpcnRpby92aG9zdC1zaGFkb3ctdmlydHF1ZXVlLmMKPiBA QCAtOSw2ICs5LDkgQEAKPiAgIAo+ICAgI2luY2x1ZGUgInFlbXUvb3NkZXAuaCIKPiAgICNpbmNs dWRlICJody92aXJ0aW8vdmhvc3Qtc2hhZG93LXZpcnRxdWV1ZS5oIgo+ICsjaW5jbHVkZSAiaHcv dmlydGlvL3Zob3N0LmgiCj4gKyNpbmNsdWRlICJody92aXJ0aW8vdmlydGlvLWFjY2Vzcy5oIgo+ ICsKPiAgICNpbmNsdWRlICJzdGFuZGFyZC1oZWFkZXJzL2xpbnV4L3Zob3N0X3R5cGVzLmgiCj4g ICAKPiAgICNpbmNsdWRlICJxZW11L2Vycm9yLXJlcG9ydC5oIgo+IEBAIC00NSw2ICs0OCwyNyBA QCB0eXBlZGVmIHN0cnVjdCBWaG9zdFNoYWRvd1ZpcnRxdWV1ZSB7Cj4gICAKPiAgICAgICAvKiBW aXJ0aW8gZGV2aWNlICovCj4gICAgICAgVmlydElPRGV2aWNlICp2ZGV2Owo+ICsKPiArICAgIC8q IE1hcCBmb3IgcmV0dXJuaW5nIGd1ZXN0J3MgZGVzY3JpcHRvcnMgKi8KPiArICAgIFZpcnRRdWV1 ZUVsZW1lbnQgKipyaW5nX2lkX21hcHM7Cj4gKwo+ICsgICAgLyogTmV4dCBWaXJ0UXVldWUgZWxl bWVudCB0aGF0IGd1ZXN0IG1hZGUgYXZhaWxhYmxlICovCj4gKyAgICBWaXJ0UXVldWVFbGVtZW50 ICpuZXh0X2d1ZXN0X2F2YWlsX2VsZW07Cj4gKwo+ICsgICAgLyogTmV4dCBoZWFkIHRvIGV4cG9z ZSB0byBkZXZpY2UgKi8KPiArICAgIHVpbnQxNl90IGF2YWlsX2lkeF9zaGFkb3c7Cj4gKwo+ICsg ICAgLyogTmV4dCBmcmVlIGRlc2NyaXB0b3IgKi8KPiArICAgIHVpbnQxNl90IGZyZWVfaGVhZDsK PiArCj4gKyAgICAvKiBMYXN0IHNlZW4gdXNlZCBpZHggKi8KPiArICAgIHVpbnQxNl90IHNoYWRv d191c2VkX2lkeDsKPiArCj4gKyAgICAvKiBOZXh0IGhlYWQgdG8gY29uc3VtZSBmcm9tIGRldmlj ZSAqLwo+ICsgICAgdWludDE2X3QgbGFzdF91c2VkX2lkeDsKPiArCj4gKyAgICAvKiBDYWNoZSBm b3IgdGhlIGV4cG9zZWQgbm90aWZpY2F0aW9uIGZsYWcgKi8KPiArICAgIGJvb2wgbm90aWZpY2F0 aW9uOwo+ICAgfSBWaG9zdFNoYWRvd1ZpcnRxdWV1ZTsKPiAgIAo+ICAgLyoqCj4gQEAgLTU2LDI1 ICs4MCwxNzQgQEAgY29uc3QgRXZlbnROb3RpZmllciAqdmhvc3Rfc3ZxX2dldF9kZXZfa2lja19u b3RpZmllcigKPiAgICAgICByZXR1cm4gJnN2cS0+aGRldl9raWNrOwo+ICAgfQo+ICAgCj4gLS8q IElmIHRoZSBkZXZpY2UgaXMgdXNpbmcgc29tZSBvZiB0aGVzZSwgU1ZRIGNhbm5vdCBjb21tdW5p Y2F0ZSAqLwo+ICsvKioKPiArICogVmlydElPIHRyYW5zcG9ydCBkZXZpY2UgZmVhdHVyZSBhY2tu b3dsZWRnZQo+ICsgKgo+ICsgKiBAZGV2X2ZlYXR1cmVzICBUaGUgZGV2aWNlIGZlYXR1cmVzLiBJ ZiBzdWNjZXNzLCB0aGUgYWNrbm93bGVkZ2VkIGZlYXR1cmVzLgo+ICsgKgo+ICsgKiBSZXR1cm5z IHRydWUgaWYgU1ZRIGNhbiBnbyB3aXRoIGEgc3Vic2V0IG9mIHRoZXNlLCBmYWxzZSBvdGhlcndp c2UuCj4gKyAqLwo+ICAgYm9vbCB2aG9zdF9zdnFfdmFsaWRfZGV2aWNlX2ZlYXR1cmVzKHVpbnQ2 NF90ICpkZXZfZmVhdHVyZXMpCj4gICB7Cj4gLSAgICByZXR1cm4gdHJ1ZTsKPiArICAgIHVpbnQ2 NF90IGI7Cj4gKyAgICBib29sIHIgPSB0cnVlOwo+ICsKPiArICAgIGZvciAoYiA9IFZJUlRJT19U UkFOU1BPUlRfRl9TVEFSVDsgYiA8PSBWSVJUSU9fVFJBTlNQT1JUX0ZfRU5EOyArK2IpIHsKPiAr ICAgICAgICBzd2l0Y2ggKGIpIHsKPiArICAgICAgICBjYXNlIFZJUlRJT19GX05PVElGWV9PTl9F TVBUWToKPiArICAgICAgICBjYXNlIFZJUlRJT19GX0FOWV9MQVlPVVQ6Cj4gKyAgICAgICAgICAg IGNvbnRpbnVlOwo+ICsKPiArICAgICAgICBjYXNlIFZJUlRJT19GX0FDQ0VTU19QTEFURk9STToK PiArICAgICAgICAgICAgLyogU1ZRIGRvZXMgbm90IGtub3cgaG93IHRvIHRyYW5zbGF0ZSBhZGRy ZXNzZXMgKi8KPiArICAgICAgICAgICAgaWYgKCpkZXZfZmVhdHVyZXMgJiBCSVRfVUxMKGIpKSB7 Cj4gKyAgICAgICAgICAgICAgICBjbGVhcl9iaXQoYiwgZGV2X2ZlYXR1cmVzKTsKPiArICAgICAg ICAgICAgICAgIHIgPSBmYWxzZTsKPiArICAgICAgICAgICAgfQo+ICsgICAgICAgICAgICBicmVh azsKPiArCj4gKyAgICAgICAgY2FzZSBWSVJUSU9fRl9WRVJTSU9OXzE6Cj4gKyAgICAgICAgICAg IC8qIFNWUSB0cnVzdCB0aGF0IGd1ZXN0IHZyaW5nIGlzIGxpdHRsZSBlbmRpYW4gKi8KPiArICAg ICAgICAgICAgaWYgKCEoKmRldl9mZWF0dXJlcyAmIEJJVF9VTEwoYikpKSB7Cj4gKyAgICAgICAg ICAgICAgICBzZXRfYml0KGIsIGRldl9mZWF0dXJlcyk7Cj4gKyAgICAgICAgICAgICAgICByID0g ZmFsc2U7Cj4gKyAgICAgICAgICAgIH0KPiArICAgICAgICAgICAgY29udGludWU7Cj4gKwo+ICsg ICAgICAgIGRlZmF1bHQ6Cj4gKyAgICAgICAgICAgIGlmICgqZGV2X2ZlYXR1cmVzICYgQklUX1VM TChiKSkgewo+ICsgICAgICAgICAgICAgICAgY2xlYXJfYml0KGIsIGRldl9mZWF0dXJlcyk7Cj4g KyAgICAgICAgICAgIH0KPiArICAgICAgICB9Cj4gKyAgICB9Cj4gKwo+ICsgICAgcmV0dXJuIHI7 Cj4gICB9Cj4gICAKPiAtLyogSWYgdGhlIGd1ZXN0IGlzIHVzaW5nIHNvbWUgb2YgdGhlc2UsIFNW USBjYW5ub3QgY29tbXVuaWNhdGUgKi8KPiArLyoqCj4gKyAqIENoZWNrIG9mIGd1ZXN0J3MgYWNr bm93bGVkZ2UgZmVhdHVyZXMuCj4gKyAqCj4gKyAqIEBndWVzdF9mZWF0dXJlcyAgVGhlIGd1ZXN0 J3MgYWNrbm93bGVkZ2VkIGZlYXR1cmVzCj4gKyAqCj4gKyAqIFJldHVybnMgdHJ1ZSBpZiBTVlEg Y2FuIGhhbmRsZSB0aGVtLCBmYWxzZSBvdGhlcndpc2UuCj4gKyAqLwo+ICAgYm9vbCB2aG9zdF9z dnFfdmFsaWRfZ3Vlc3RfZmVhdHVyZXModWludDY0X3QgKmd1ZXN0X2ZlYXR1cmVzKQo+ICAgewo+ IC0gICAgcmV0dXJuIHRydWU7Cj4gKyAgICBzdGF0aWMgY29uc3QgdWludDY0X3QgdHJhbnNwb3J0 ID0gTUFLRV82NEJJVF9NQVNLKFZJUlRJT19UUkFOU1BPUlRfRl9TVEFSVCwKPiArICAgICAgICAg ICAgICAgICAgICAgICAgICAgIFZJUlRJT19UUkFOU1BPUlRfRl9FTkQgLSBWSVJUSU9fVFJBTlNQ T1JUX0ZfU1RBUlQpOwo+ICsKPiArICAgIC8qIFRoZXNlIHRyYW5zcG9ydCBmZWF0dXJlcyBhcmUg aGFuZGxlZCBieSBWaXJ0UXVldWUgKi8KPiArICAgIHN0YXRpYyBjb25zdCB1aW50NjRfdCB2YWxp ZCA9IChCSVRfVUxMKFZJUlRJT19SSU5HX0ZfSU5ESVJFQ1RfREVTQykgfAo+ICsgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgIEJJVF9VTEwoVklSVElPX0ZfVkVSU0lPTl8xKSk7Cj4g Kwo+ICsgICAgLyogV2UgYXJlIG9ubHkgaW50ZXJlc3RlZCBpbiB0cmFuc3BvcnQtcmVsYXRlZCBm ZWF0dXJlIGJpdHMgKi8KPiArICAgIHVpbnQ2NF90IGd1ZXN0X3RyYW5zcG9ydF9mZWF0dXJlcyA9 ICgqZ3Vlc3RfZmVhdHVyZXMpICYgdHJhbnNwb3J0Owo+ICsKPiArICAgICpndWVzdF9mZWF0dXJl cyAmPSAodmFsaWQgfCB+dHJhbnNwb3J0KTsKPiArICAgIHJldHVybiAhKGd1ZXN0X3RyYW5zcG9y dF9mZWF0dXJlcyAmICh0cmFuc3BvcnQgXiB2YWxpZCkpOwo+ICAgfQo+ICAgCj4gLS8qIEZvcndh cmQgZ3Vlc3Qgbm90aWZpY2F0aW9ucyAqLwo+IC1zdGF0aWMgdm9pZCB2aG9zdF9oYW5kbGVfZ3Vl c3Rfa2ljayhFdmVudE5vdGlmaWVyICpuKQo+ICsvKioKPiArICogTnVtYmVyIG9mIGRlc2NyaXB0 b3JzIHRoYXQgU1ZRIGNhbiBtYWtlIGF2YWlsYWJsZSBmcm9tIHRoZSBndWVzdC4KPiArICoKPiAr ICogQHN2cSAgIFRoZSBzdnEKPiArICovCj4gK3N0YXRpYyB1aW50MTZfdCB2aG9zdF9zdnFfYXZh aWxhYmxlX3Nsb3RzKGNvbnN0IFZob3N0U2hhZG93VmlydHF1ZXVlICpzdnEpCj4gICB7Cj4gLSAg ICBWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAqc3ZxID0gY29udGFpbmVyX29mKG4sIFZob3N0U2hhZG93 VmlydHF1ZXVlLAo+IC0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBzdnFfa2ljayk7Cj4gKyAgICByZXR1cm4gc3ZxLT52cmluZy5udW0gLSAoc3ZxLT5hdmFpbF9p ZHhfc2hhZG93IC0gc3ZxLT5zaGFkb3dfdXNlZF9pZHgpOwo+ICt9Cj4gICAKPiAtICAgIGlmICh1 bmxpa2VseSghZXZlbnRfbm90aWZpZXJfdGVzdF9hbmRfY2xlYXIobikpKSB7Cj4gK3N0YXRpYyB2 b2lkIHZob3N0X3N2cV9zZXRfbm90aWZpY2F0aW9uKFZob3N0U2hhZG93VmlydHF1ZXVlICpzdnEs IGJvb2wgZW5hYmxlKQo+ICt7Cj4gKyAgICB1aW50MTZfdCBub3RpZmljYXRpb25fZmxhZzsKPiAr Cj4gKyAgICBpZiAoc3ZxLT5ub3RpZmljYXRpb24gPT0gZW5hYmxlKSB7Cj4gKyAgICAgICAgcmV0 dXJuOwo+ICsgICAgfQo+ICsKPiArICAgIG5vdGlmaWNhdGlvbl9mbGFnID0gY3B1X3RvX2xlMTYo VlJJTkdfQVZBSUxfRl9OT19JTlRFUlJVUFQpOwo+ICsKPiArICAgIHN2cS0+bm90aWZpY2F0aW9u ID0gZW5hYmxlOwo+ICsgICAgaWYgKGVuYWJsZSkgewo+ICsgICAgICAgIHN2cS0+dnJpbmcuYXZh aWwtPmZsYWdzICY9IH5ub3RpZmljYXRpb25fZmxhZzsKPiArICAgIH0gZWxzZSB7Cj4gKyAgICAg ICAgc3ZxLT52cmluZy5hdmFpbC0+ZmxhZ3MgfD0gbm90aWZpY2F0aW9uX2ZsYWc7Cj4gKyAgICB9 Cj4gK30KPiArCj4gK3N0YXRpYyB2b2lkIHZob3N0X3ZyaW5nX3dyaXRlX2Rlc2NzKFZob3N0U2hh ZG93VmlydHF1ZXVlICpzdnEsCj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IGNvbnN0IHN0cnVjdCBpb3ZlYyAqaW92ZWMsCj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgIHNpemVfdCBudW0sIGJvb2wgbW9yZV9kZXNjcywgYm9vbCB3cml0ZSkKPiArewo+ ICsgICAgdWludDE2X3QgaSA9IHN2cS0+ZnJlZV9oZWFkLCBsYXN0ID0gc3ZxLT5mcmVlX2hlYWQ7 Cj4gKyAgICB1bnNpZ25lZCBuOwo+ICsgICAgdWludDE2X3QgZmxhZ3MgPSB3cml0ZSA/IGNwdV90 b19sZTE2KFZSSU5HX0RFU0NfRl9XUklURSkgOiAwOwo+ICsgICAgdnJpbmdfZGVzY190ICpkZXNj cyA9IHN2cS0+dnJpbmcuZGVzYzsKPiArCj4gKyAgICBpZiAobnVtID09IDApIHsKPiArICAgICAg ICByZXR1cm47Cj4gKyAgICB9Cj4gKwo+ICsgICAgZm9yIChuID0gMDsgbiA8IG51bTsgbisrKSB7 Cj4gKyAgICAgICAgaWYgKG1vcmVfZGVzY3MgfHwgKG4gKyAxIDwgbnVtKSkgewo+ICsgICAgICAg ICAgICBkZXNjc1tpXS5mbGFncyA9IGZsYWdzIHwgY3B1X3RvX2xlMTYoVlJJTkdfREVTQ19GX05F WFQpOwo+ICsgICAgICAgIH0gZWxzZSB7Cj4gKyAgICAgICAgICAgIGRlc2NzW2ldLmZsYWdzID0g ZmxhZ3M7Cj4gKyAgICAgICAgfQo+ICsgICAgICAgIGRlc2NzW2ldLmFkZHIgPSBjcHVfdG9fbGU2 NCgoaHdhZGRyKWlvdmVjW25dLmlvdl9iYXNlKTsKPiArICAgICAgICBkZXNjc1tpXS5sZW4gPSBj cHVfdG9fbGUzMihpb3ZlY1tuXS5pb3ZfbGVuKTsKPiArCj4gKyAgICAgICAgbGFzdCA9IGk7Cj4g KyAgICAgICAgaSA9IGNwdV90b19sZTE2KGRlc2NzW2ldLm5leHQpOwo+ICsgICAgfQo+ICsKPiAr ICAgIHN2cS0+ZnJlZV9oZWFkID0gbGUxNl90b19jcHUoZGVzY3NbbGFzdF0ubmV4dCk7Cj4gK30K PiArCj4gK3N0YXRpYyB1bnNpZ25lZCB2aG9zdF9zdnFfYWRkX3NwbGl0KFZob3N0U2hhZG93Vmly dHF1ZXVlICpzdnEsCj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFZpcnRR dWV1ZUVsZW1lbnQgKmVsZW0pCj4gK3sKPiArICAgIGludCBoZWFkOwo+ICsgICAgdW5zaWduZWQg YXZhaWxfaWR4Owo+ICsgICAgdnJpbmdfYXZhaWxfdCAqYXZhaWwgPSBzdnEtPnZyaW5nLmF2YWls Owo+ICsKPiArICAgIGhlYWQgPSBzdnEtPmZyZWVfaGVhZDsKPiArCj4gKyAgICAvKiBXZSBuZWVk IHNvbWUgZGVzY3JpcHRvcnMgaGVyZSAqLwo+ICsgICAgYXNzZXJ0KGVsZW0tPm91dF9udW0gfHwg ZWxlbS0+aW5fbnVtKTsKPiArCj4gKyAgICB2aG9zdF92cmluZ193cml0ZV9kZXNjcyhzdnEsIGVs ZW0tPm91dF9zZywgZWxlbS0+b3V0X251bSwKPiArICAgICAgICAgICAgICAgICAgICAgICAgICAg IGVsZW0tPmluX251bSA+IDAsIGZhbHNlKTsKPiArICAgIHZob3N0X3ZyaW5nX3dyaXRlX2Rlc2Nz KHN2cSwgZWxlbS0+aW5fc2csIGVsZW0tPmluX251bSwgZmFsc2UsIHRydWUpOwo+ICsKPiArICAg IC8qCj4gKyAgICAgKiBQdXQgZW50cnkgaW4gYXZhaWxhYmxlIGFycmF5IChidXQgZG9uJ3QgdXBk YXRlIGF2YWlsLT5pZHggdW50aWwgdGhleQo+ICsgICAgICogZG8gc3luYykuCj4gKyAgICAgKi8K PiArICAgIGF2YWlsX2lkeCA9IHN2cS0+YXZhaWxfaWR4X3NoYWRvdyAmIChzdnEtPnZyaW5nLm51 bSAtIDEpOwo+ICsgICAgYXZhaWwtPnJpbmdbYXZhaWxfaWR4XSA9IGNwdV90b19sZTE2KGhlYWQp Owo+ICsgICAgc3ZxLT5hdmFpbF9pZHhfc2hhZG93Kys7Cj4gKwo+ICsgICAgLyogVXBkYXRlIGF2 YWlsIGluZGV4IGFmdGVyIHRoZSBkZXNjcmlwdG9yIGlzIHdyb3RlICovCj4gKyAgICBzbXBfd21i KCk7CgoKQSBxdWVzdGlvbiwgc2luY2Ugd2UgbWF5IHRhbGsgd2l0aCB0aGUgcmVhbCBoYXJkd2Fy ZSwgaXMgc21wX3dtYigpIApzdWZmaWNpZW50IGluIHRoaXMgY2FzZSBvciBkbyB3ZSBuZWVkIHRv IGhvbmVyIFZJUlRJT19GX09SREVSX1BMQVRGT1JNPwoKCj4gKyAgICBhdmFpbC0+aWR4ID0gY3B1 X3RvX2xlMTYoc3ZxLT5hdmFpbF9pZHhfc2hhZG93KTsKPiArCj4gKyAgICByZXR1cm4gaGVhZDsK PiArCj4gK30KPiArCj4gK3N0YXRpYyB2b2lkIHZob3N0X3N2cV9hZGQoVmhvc3RTaGFkb3dWaXJ0 cXVldWUgKnN2cSwgVmlydFF1ZXVlRWxlbWVudCAqZWxlbSkKPiArewo+ICsgICAgdW5zaWduZWQg cWVtdV9oZWFkID0gdmhvc3Rfc3ZxX2FkZF9zcGxpdChzdnEsIGVsZW0pOwo+ICsKPiArICAgIHN2 cS0+cmluZ19pZF9tYXBzW3FlbXVfaGVhZF0gPSBlbGVtOwo+ICt9Cj4gKwo+ICtzdGF0aWMgdm9p ZCB2aG9zdF9zdnFfa2ljayhWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAqc3ZxKQo+ICt7Cj4gKyAgICAv KiBXZSBuZWVkIHRvIGV4cG9zZSBhdmFpbGFibGUgYXJyYXkgZW50cmllcyBiZWZvcmUgY2hlY2tp bmcgdXNlZCBmbGFncyAqLwo+ICsgICAgc21wX21iKCk7Cj4gKyAgICBpZiAoc3ZxLT52cmluZy51 c2VkLT5mbGFncyAmIFZSSU5HX1VTRURfRl9OT19OT1RJRlkpIHsKPiAgICAgICAgICAgcmV0dXJu Owo+ICAgICAgIH0KPiAgIAo+IEBAIC04NiwyNSArMjU5LDE4OCBAQCBzdGF0aWMgdm9pZCB2aG9z dF9oYW5kbGVfZ3Vlc3Rfa2ljayhFdmVudE5vdGlmaWVyICpuKQo+ICAgICAgIH0KPiAgIH0KPiAg IAo+IC0vKgo+IC0gKiBTZXQgdGhlIGRldmljZSdzIG1lbW9yeSByZWdpb24gbm90aWZpZXIuIGFk ZHIgPSBOVUxMIGNsZWFyIGl0Lgo+ICsvKioKPiArICogRm9yd2FyZCBhdmFpbGFibGUgYnVmZmVy cy4KPiArICoKPiArICogQHN2cSBTaGFkb3cgVmlydFF1ZXVlCj4gKyAqCj4gKyAqIE5vdGUgdGhh dCB0aGlzIGZ1bmN0aW9uIGRvZXMgbm90IGd1YXJhbnRlZSB0aGF0IGFsbCBndWVzdCdzIGF2YWls YWJsZQo+ICsgKiBidWZmZXJzIGFyZSBhdmFpbGFibGUgdG8gdGhlIGRldmljZSBpbiBTVlEgYXZh aWwgcmluZy4gVGhlIGd1ZXN0IG1heSBoYXZlCj4gKyAqIGV4cG9zZWQgYSBHUEEgLyBHSU9WQSBj b25naXVvdXMgYnVmZmVyLCBidXQgaXQgbWF5IG5vdCBiZSBjb250aWd1b3VzIGluIHFlbXUKPiAr ICogdmFkZHIuCj4gKyAqCj4gKyAqIElmIHRoYXQgaGFwcGVucywgZ3Vlc3QncyBraWNrIG5vdGlm aWNhdGlvbnMgd2lsbCBiZSBkaXNhYmxlZCB1bnRpbCBkZXZpY2UKPiArICogbWFrZXMgc29tZSBi dWZmZXJzIHVzZWQuCj4gICAgKi8KPiAtdm9pZCB2aG9zdF9zdnFfc2V0X2hvc3RfbXJfbm90aWZp ZXIoVmhvc3RTaGFkb3dWaXJ0cXVldWUgKnN2cSwgdm9pZCAqYWRkcikKPiArc3RhdGljIHZvaWQg dmhvc3RfaGFuZGxlX2d1ZXN0X2tpY2soVmhvc3RTaGFkb3dWaXJ0cXVldWUgKnN2cSkKPiAgIHsK PiAtICAgIHN2cS0+aG9zdF9ub3RpZmllcl9tciA9IGFkZHI7Cj4gKyAgICAvKiBDbGVhciBldmVu dCBub3RpZmllciAqLwo+ICsgICAgZXZlbnRfbm90aWZpZXJfdGVzdF9hbmRfY2xlYXIoJnN2cS0+ c3ZxX2tpY2spOwo+ICsKPiArICAgIC8qIE1ha2UgYXZhaWxhYmxlIGFzIG1hbnkgYnVmZmVycyBh cyBwb3NzaWJsZSAqLwo+ICsgICAgZG8gewo+ICsgICAgICAgIGlmICh2aXJ0aW9fcXVldWVfZ2V0 X25vdGlmaWNhdGlvbihzdnEtPnZxKSkgewo+ICsgICAgICAgICAgICB2aXJ0aW9fcXVldWVfc2V0 X25vdGlmaWNhdGlvbihzdnEtPnZxLCBmYWxzZSk7Cj4gKyAgICAgICAgfQo+ICsKPiArICAgICAg ICB3aGlsZSAodHJ1ZSkgewo+ICsgICAgICAgICAgICBWaXJ0UXVldWVFbGVtZW50ICplbGVtOwo+ ICsKPiArICAgICAgICAgICAgaWYgKHN2cS0+bmV4dF9ndWVzdF9hdmFpbF9lbGVtKSB7Cj4gKyAg ICAgICAgICAgICAgICBlbGVtID0gZ19zdGVhbF9wb2ludGVyKCZzdnEtPm5leHRfZ3Vlc3RfYXZh aWxfZWxlbSk7Cj4gKyAgICAgICAgICAgIH0gZWxzZSB7Cj4gKyAgICAgICAgICAgICAgICBlbGVt ID0gdmlydHF1ZXVlX3BvcChzdnEtPnZxLCBzaXplb2YoKmVsZW0pKTsKPiArICAgICAgICAgICAg fQo+ICsKPiArICAgICAgICAgICAgaWYgKCFlbGVtKSB7Cj4gKyAgICAgICAgICAgICAgICBicmVh azsKPiArICAgICAgICAgICAgfQo+ICsKPiArICAgICAgICAgICAgaWYgKGVsZW0tPm91dF9udW0g KyBlbGVtLT5pbl9udW0gPgo+ICsgICAgICAgICAgICAgICAgdmhvc3Rfc3ZxX2F2YWlsYWJsZV9z bG90cyhzdnEpKSB7Cj4gKyAgICAgICAgICAgICAgICAvKgo+ICsgICAgICAgICAgICAgICAgICog VGhpcyBjb25kaXRpb24gaXMgcG9zc2libGUgc2luY2UgYSBjb250aWd1b3VzIGJ1ZmZlciBpbiBH UEEKPiArICAgICAgICAgICAgICAgICAqIGRvZXMgbm90IGltcGx5IGEgY29udGlndW91cyBidWZm ZXIgaW4gcWVtdSdzIFZBCj4gKyAgICAgICAgICAgICAgICAgKiBzY2F0dGVyLWdhdGhlciBzZWdt ZW50cy4gSWYgdGhhdCBoYXBwZW4sIHRoZSBidWZmZXIgZXhwb3NlZAo+ICsgICAgICAgICAgICAg ICAgICogdG8gdGhlIGRldmljZSBuZWVkcyB0byBiZSBhIGNoYWluIG9mIGRlc2NyaXB0b3JzIGF0 IHRoaXMKPiArICAgICAgICAgICAgICAgICAqIG1vbWVudC4KPiArICAgICAgICAgICAgICAgICAq Cj4gKyAgICAgICAgICAgICAgICAgKiBTVlEgY2Fubm90IGhvbGQgbW9yZSBhdmFpbGFibGUgYnVm ZmVycyBpZiB3ZSBhcmUgaGVyZToKPiArICAgICAgICAgICAgICAgICAqIHF1ZXVlIHRoZSBjdXJy ZW50IGd1ZXN0IGRlc2NyaXB0b3IgYW5kIGlnbm9yZSBmdXJ0aGVyIGtpY2tzCj4gKyAgICAgICAg ICAgICAgICAgKiB1bnRpbCBzb21lIGVsZW1lbnRzIGFyZSB1c2VkLgo+ICsgICAgICAgICAgICAg ICAgICovCgoKSSB3b25kZXIgd2hhdCdzIHRoZSBhZHZhbnRhZ2Ugb2YgdHJhY2tpbmcgdGhlIHBl bmRpbmcgZWxlbSBsaWtlIHRoaXMuIEl0IApsb29rcyB0byBtZSB3ZSBjYW4gc2ltcGx5IHJld2lu ZCBsYXN0X2F2YWlsX2lkeCBpbiB0aGlzIGNhc2U/CgoKPiArICAgICAgICAgICAgICAgIHN2cS0+ bmV4dF9ndWVzdF9hdmFpbF9lbGVtID0gZWxlbTsKPiArICAgICAgICAgICAgICAgIHJldHVybjsK PiArICAgICAgICAgICAgfQo+ICsKPiArICAgICAgICAgICAgdmhvc3Rfc3ZxX2FkZChzdnEsIGVs ZW0pOwo+ICsgICAgICAgICAgICB2aG9zdF9zdnFfa2ljayhzdnEpOwo+ICsgICAgICAgIH0KPiAr Cj4gKyAgICAgICAgdmlydGlvX3F1ZXVlX3NldF9ub3RpZmljYXRpb24oc3ZxLT52cSwgdHJ1ZSk7 Cj4gKyAgICB9IHdoaWxlICghdmlydGlvX3F1ZXVlX2VtcHR5KHN2cS0+dnEpKTsKPiArfQo+ICsK PiArLyoqCj4gKyAqIEhhbmRsZSBndWVzdCdzIGtpY2suCj4gKyAqCj4gKyAqIEBuIGd1ZXN0IGtp Y2sgZXZlbnQgbm90aWZpZXIsIHRoZSBvbmUgdGhhdCBndWVzdCBzZXQgdG8gbm90aWZ5IHN2cS4K PiArICovCj4gK3N0YXRpYyB2b2lkIHZob3N0X2hhbmRsZV9ndWVzdF9raWNrX25vdGlmaWVyKEV2 ZW50Tm90aWZpZXIgKm4pCj4gK3sKPiArICAgIFZob3N0U2hhZG93VmlydHF1ZXVlICpzdnEgPSBj b250YWluZXJfb2YobiwgVmhvc3RTaGFkb3dWaXJ0cXVldWUsCj4gKyAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN2cV9raWNrKTsKPiArICAgIHZob3N0X2hhbmRs ZV9ndWVzdF9raWNrKHN2cSk7Cj4gK30KPiArCj4gK3N0YXRpYyBib29sIHZob3N0X3N2cV9tb3Jl X3VzZWQoVmhvc3RTaGFkb3dWaXJ0cXVldWUgKnN2cSkKPiArewo+ICsgICAgaWYgKHN2cS0+bGFz dF91c2VkX2lkeCAhPSBzdnEtPnNoYWRvd191c2VkX2lkeCkgewo+ICsgICAgICAgIHJldHVybiB0 cnVlOwo+ICsgICAgfQo+ICsKPiArICAgIHN2cS0+c2hhZG93X3VzZWRfaWR4ID0gY3B1X3RvX2xl MTYoc3ZxLT52cmluZy51c2VkLT5pZHgpOwo+ICsKPiArICAgIHJldHVybiBzdnEtPmxhc3RfdXNl ZF9pZHggIT0gc3ZxLT5zaGFkb3dfdXNlZF9pZHg7Cj4gK30KPiArCj4gK3N0YXRpYyBWaXJ0UXVl dWVFbGVtZW50ICp2aG9zdF9zdnFfZ2V0X2J1ZihWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAqc3ZxKQo+ ICt7Cj4gKyAgICB2cmluZ19kZXNjX3QgKmRlc2NzID0gc3ZxLT52cmluZy5kZXNjOwo+ICsgICAg Y29uc3QgdnJpbmdfdXNlZF90ICp1c2VkID0gc3ZxLT52cmluZy51c2VkOwo+ICsgICAgdnJpbmdf dXNlZF9lbGVtX3QgdXNlZF9lbGVtOwo+ICsgICAgdWludDE2X3QgbGFzdF91c2VkOwo+ICsKPiAr ICAgIGlmICghdmhvc3Rfc3ZxX21vcmVfdXNlZChzdnEpKSB7Cj4gKyAgICAgICAgcmV0dXJuIE5V TEw7Cj4gKyAgICB9Cj4gKwo+ICsgICAgLyogT25seSBnZXQgdXNlZCBhcnJheSBlbnRyaWVzIGFm dGVyIHRoZXkgaGF2ZSBiZWVuIGV4cG9zZWQgYnkgZGV2ICovCj4gKyAgICBzbXBfcm1iKCk7Cj4g KyAgICBsYXN0X3VzZWQgPSBzdnEtPmxhc3RfdXNlZF9pZHggJiAoc3ZxLT52cmluZy5udW0gLSAx KTsKPiArICAgIHVzZWRfZWxlbS5pZCA9IGxlMzJfdG9fY3B1KHVzZWQtPnJpbmdbbGFzdF91c2Vk XS5pZCk7Cj4gKyAgICB1c2VkX2VsZW0ubGVuID0gbGUzMl90b19jcHUodXNlZC0+cmluZ1tsYXN0 X3VzZWRdLmxlbik7Cj4gKwo+ICsgICAgc3ZxLT5sYXN0X3VzZWRfaWR4Kys7Cj4gKyAgICBpZiAo dW5saWtlbHkodXNlZF9lbGVtLmlkID49IHN2cS0+dnJpbmcubnVtKSkgewo+ICsgICAgICAgIGVy cm9yX3JlcG9ydCgiRGV2aWNlICVzIHNheXMgaW5kZXggJXUgaXMgdXNlZCIsIHN2cS0+dmRldi0+ bmFtZSwKPiArICAgICAgICAgICAgICAgICAgICAgdXNlZF9lbGVtLmlkKTsKPiArICAgICAgICBy ZXR1cm4gTlVMTDsKPiArICAgIH0KPiArCj4gKyAgICBpZiAodW5saWtlbHkoIXN2cS0+cmluZ19p ZF9tYXBzW3VzZWRfZWxlbS5pZF0pKSB7Cj4gKyAgICAgICAgZXJyb3JfcmVwb3J0KAo+ICsgICAg ICAgICAgICAiRGV2aWNlICVzIHNheXMgaW5kZXggJXUgaXMgdXNlZCwgYnV0IGl0IHdhcyBub3Qg YXZhaWxhYmxlIiwKPiArICAgICAgICAgICAgc3ZxLT52ZGV2LT5uYW1lLCB1c2VkX2VsZW0uaWQp Owo+ICsgICAgICAgIHJldHVybiBOVUxMOwo+ICsgICAgfQo+ICsKPiArICAgIGRlc2NzW3VzZWRf ZWxlbS5pZF0ubmV4dCA9IHN2cS0+ZnJlZV9oZWFkOwo+ICsgICAgc3ZxLT5mcmVlX2hlYWQgPSB1 c2VkX2VsZW0uaWQ7Cj4gKwo+ICsgICAgc3ZxLT5yaW5nX2lkX21hcHNbdXNlZF9lbGVtLmlkXS0+ bGVuID0gdXNlZF9lbGVtLmxlbjsKPiArICAgIHJldHVybiBnX3N0ZWFsX3BvaW50ZXIoJnN2cS0+ cmluZ19pZF9tYXBzW3VzZWRfZWxlbS5pZF0pOwo+ICAgfQo+ICAgCj4gLS8qIEZvcndhcmQgdmhv c3Qgbm90aWZpY2F0aW9ucyAqLwo+ICtzdGF0aWMgdm9pZCB2aG9zdF9zdnFfZmx1c2goVmhvc3RT aGFkb3dWaXJ0cXVldWUgKnN2cSwKPiArICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJvb2wg Y2hlY2tfZm9yX2F2YWlsX3F1ZXVlKQo+ICt7Cj4gKyAgICBWaXJ0UXVldWUgKnZxID0gc3ZxLT52 cTsKPiArCj4gKyAgICAvKiBNYWtlIGFzIG1hbnkgYnVmZmVycyBhcyBwb3NzaWJsZSB1c2VkLiAq Lwo+ICsgICAgZG8gewo+ICsgICAgICAgIHVuc2lnbmVkIGkgPSAwOwo+ICsKPiArICAgICAgICB2 aG9zdF9zdnFfc2V0X25vdGlmaWNhdGlvbihzdnEsIGZhbHNlKTsKPiArICAgICAgICB3aGlsZSAo dHJ1ZSkgewo+ICsgICAgICAgICAgICBnX2F1dG9mcmVlIFZpcnRRdWV1ZUVsZW1lbnQgKmVsZW0g PSB2aG9zdF9zdnFfZ2V0X2J1ZihzdnEpOwo+ICsgICAgICAgICAgICBpZiAoIWVsZW0pIHsKPiAr ICAgICAgICAgICAgICAgIGJyZWFrOwo+ICsgICAgICAgICAgICB9Cj4gKwo+ICsgICAgICAgICAg ICBpZiAodW5saWtlbHkoaSA+PSBzdnEtPnZyaW5nLm51bSkpIHsKPiArICAgICAgICAgICAgICAg IHZpcnRpb19lcnJvcihzdnEtPnZkZXYsCj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAiTW9y ZSB0aGFuICV1IHVzZWQgYnVmZmVycyBvYnRhaW5lZCBpbiBhICV1IHNpemUgU1ZRIiwKPiArICAg ICAgICAgICAgICAgICAgICAgICAgIGksIHN2cS0+dnJpbmcubnVtKTsKPiArICAgICAgICAgICAg ICAgIHZpcnRxdWV1ZV9maWxsKHZxLCBlbGVtLCBlbGVtLT5sZW4sIGkpOwo+ICsgICAgICAgICAg ICAgICAgdmlydHF1ZXVlX2ZsdXNoKHZxLCBpKTsKPiArICAgICAgICAgICAgICAgIGkgPSAwOwo+ ICsgICAgICAgICAgICB9Cj4gKyAgICAgICAgICAgIHZpcnRxdWV1ZV9maWxsKHZxLCBlbGVtLCBl bGVtLT5sZW4sIGkrKyk7Cj4gKyAgICAgICAgfQo+ICsKPiArICAgICAgICB2aXJ0cXVldWVfZmx1 c2godnEsIGkpOwo+ICsgICAgICAgIGV2ZW50X25vdGlmaWVyX3NldCgmc3ZxLT5zdnFfY2FsbCk7 Cj4gKwo+ICsgICAgICAgIGlmIChjaGVja19mb3JfYXZhaWxfcXVldWUgJiYgc3ZxLT5uZXh0X2d1 ZXN0X2F2YWlsX2VsZW0pIHsKPiArICAgICAgICAgICAgLyoKPiArICAgICAgICAgICAgICogQXZh aWwgcmluZyB3YXMgZnVsbCB3aGVuIHZob3N0X3N2cV9mbHVzaCB3YXMgY2FsbGVkLCBzbyBpdCdz IGEKPiArICAgICAgICAgICAgICogZ29vZCBtb21lbnQgdG8gbWFrZSBtb3JlIGRlc2NyaXB0b3Jz IGF2YWlsYWJsZSBpZiBwb3NzaWJsZQo+ICsgICAgICAgICAgICAgKi8KPiArICAgICAgICAgICAg dmhvc3RfaGFuZGxlX2d1ZXN0X2tpY2soc3ZxKTsKPiArICAgICAgICB9Cj4gKwo+ICsgICAgICAg IHZob3N0X3N2cV9zZXRfbm90aWZpY2F0aW9uKHN2cSwgdHJ1ZSk7Cj4gKyAgICB9IHdoaWxlICh2 aG9zdF9zdnFfbW9yZV91c2VkKHN2cSkpOwoKClNvIHRoaXMgYWN0dWFsbHkgZG9lc24ndCBtYWtl IHN1cmUgYWxsIHRoZSBidWZmZXJzIHdlcmUgcHJvY2Vzc2VkIGJ5IHRoZSAKZGV2aWNlPyBJcyB0 aGlzIGludGVuZGVkIChJIHNlZSBpdCB3YXMgY2FsbGVkIGJ5IHRoZSB2aG9zdF9zdnFfc3RvcCgp KS4KCk5vdGUgdGhhdCBpdCBtZWFucyBzb21lIGJ1ZmZlcnMgbWlnaHQgbm90IGJlIHN1Ym1pdHRl ZCB0byB0aGUgZGV2aWNlIAphZnRlciBtaWdyYXRpb24/CgoKPiArfQo+ICsKPiArLyoqCj4gKyAq IEZvcndhcmQgdXNlZCBidWZmZXJzLgo+ICsgKgo+ICsgKiBAbiBoZGV2IGNhbGwgZXZlbnQgbm90 aWZpZXIsIHRoZSBvbmUgdGhhdCBkZXZpY2Ugc2V0IHRvIG5vdGlmeSBzdnEuCj4gKyAqCj4gKyAq IE5vdGUgdGhhdCB3ZSBhcmUgbm90IG1ha2luZyBhbnkgYnVmZmVycyBhdmFpbGFibGUgaW4gdGhl IGxvb3AsIHRoZXJlIGlzIG5vCj4gKyAqIHdheSB0aGF0IGl0IHJ1bnMgbW9yZSB0aGFuIHZpcnRx dWV1ZSBzaXplIHRpbWVzLgo+ICsgKi8KPiAgIHN0YXRpYyB2b2lkIHZob3N0X3N2cV9oYW5kbGVf Y2FsbChFdmVudE5vdGlmaWVyICpuKQo+ICAgewo+ICAgICAgIFZob3N0U2hhZG93VmlydHF1ZXVl ICpzdnEgPSBjb250YWluZXJfb2YobiwgVmhvc3RTaGFkb3dWaXJ0cXVldWUsCj4gICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBoZGV2X2NhbGwpOwo+ICAgCj4g LSAgICBpZiAodW5saWtlbHkoIWV2ZW50X25vdGlmaWVyX3Rlc3RfYW5kX2NsZWFyKG4pKSkgewo+ IC0gICAgICAgIHJldHVybjsKPiAtICAgIH0KPiArICAgIC8qIENsZWFyIGV2ZW50IG5vdGlmaWVy ICovCj4gKyAgICBldmVudF9ub3RpZmllcl90ZXN0X2FuZF9jbGVhcihuKTsKPiAgIAo+IC0gICAg ZXZlbnRfbm90aWZpZXJfc2V0KCZzdnEtPnN2cV9jYWxsKTsKPiArICAgIHZob3N0X3N2cV9mbHVz aChzdnEsIHRydWUpOwo+ICAgfQo+ICAgCj4gICAvKgo+IEBAIC0xMzIsNiArNDY4LDE0IEBAIHZv aWQgdmhvc3Rfc3ZxX3NldF9ndWVzdF9jYWxsX25vdGlmaWVyKFZob3N0U2hhZG93VmlydHF1ZXVl ICpzdnEsIGludCBjYWxsX2ZkKQo+ICAgICAgIGV2ZW50X25vdGlmaWVyX2luaXRfZmQoJnN2cS0+ c3ZxX2NhbGwsIGNhbGxfZmQpOwo+ICAgfQo+ICAgCj4gKy8qCj4gKyAqIFNldCB0aGUgZGV2aWNl J3MgbWVtb3J5IHJlZ2lvbiBub3RpZmllci4gYWRkciA9IE5VTEwgY2xlYXIgaXQuCj4gKyAqLwo+ ICt2b2lkIHZob3N0X3N2cV9zZXRfaG9zdF9tcl9ub3RpZmllcihWaG9zdFNoYWRvd1ZpcnRxdWV1 ZSAqc3ZxLCB2b2lkICphZGRyKQo+ICt7Cj4gKyAgICBzdnEtPmhvc3Rfbm90aWZpZXJfbXIgPSBh ZGRyOwo+ICt9Cj4gKwo+ICAgLyoKPiAgICAqIEdldCB0aGUgc2hhZG93IHZxIHZyaW5nIGFkZHJl c3MuCj4gICAgKiBAc3ZxIFNoYWRvdyB2aXJ0cXVldWUKPiBAQCAtMTg1LDcgKzUyOSw4IEBAIHN0 YXRpYyB2b2lkIHZob3N0X3N2cV9zZXRfc3ZxX2tpY2tfZmRfaW50ZXJuYWwoVmhvc3RTaGFkb3dW aXJ0cXVldWUgKnN2cSwKPiAgICAgICAgKiBuZWVkIHRvIGV4cGxpY2l0ZWx5IGNoZWNrIGZvciB0 aGVtLgo+ICAgICAgICAqLwo+ICAgICAgIGV2ZW50X25vdGlmaWVyX2luaXRfZmQoJnN2cS0+c3Zx X2tpY2ssIHN2cV9raWNrX2ZkKTsKPiAtICAgIGV2ZW50X25vdGlmaWVyX3NldF9oYW5kbGVyKCZz dnEtPnN2cV9raWNrLCB2aG9zdF9oYW5kbGVfZ3Vlc3Rfa2ljayk7Cj4gKyAgICBldmVudF9ub3Rp Zmllcl9zZXRfaGFuZGxlcigmc3ZxLT5zdnFfa2ljaywKPiArICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgIHZob3N0X2hhbmRsZV9ndWVzdF9raWNrX25vdGlmaWVyKTsKPiAgIAo+ICAgICAg IC8qCj4gICAgICAgICogIWNoZWNrX29sZCBtZWFucyB0aGF0IHdlIGFyZSBzdGFydGluZyBTVlEs IHRha2luZyB0aGUgZGVzY3JpcHRvciBmcm9tCj4gQEAgLTIzMyw3ICs1NzgsMTYgQEAgdm9pZCB2 aG9zdF9zdnFfc3RhcnQoc3RydWN0IHZob3N0X2RldiAqZGV2LCB1bnNpZ25lZCBpZHgsCj4gICB2 b2lkIHZob3N0X3N2cV9zdG9wKHN0cnVjdCB2aG9zdF9kZXYgKmRldiwgdW5zaWduZWQgaWR4LAo+ ICAgICAgICAgICAgICAgICAgICAgICBWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAqc3ZxKQo+ICAgewo+ ICsgICAgdW5zaWduZWQgaTsKPiAgICAgICBldmVudF9ub3RpZmllcl9zZXRfaGFuZGxlcigmc3Zx LT5zdnFfa2ljaywgTlVMTCk7Cj4gKyAgICB2aG9zdF9zdnFfZmx1c2goc3ZxLCBmYWxzZSk7Cj4g Kwo+ICsgICAgZm9yIChpID0gMDsgaSA8IHN2cS0+dnJpbmcubnVtOyArK2kpIHsKPiArICAgICAg ICBnX2F1dG9mcmVlIFZpcnRRdWV1ZUVsZW1lbnQgKmVsZW0gPSBzdnEtPnJpbmdfaWRfbWFwc1tp XTsKPiArICAgICAgICBpZiAoZWxlbSkgewo+ICsgICAgICAgICAgICB2aXJ0cXVldWVfZGV0YWNo X2VsZW1lbnQoc3ZxLT52cSwgZWxlbSwgZWxlbS0+bGVuKTsKPiArICAgICAgICB9Cj4gKyAgICB9 Cj4gICB9Cj4gICAKPiAgIC8qCj4gQEAgLTI0OCw3ICs2MDIsNyBAQCBWaG9zdFNoYWRvd1ZpcnRx dWV1ZSAqdmhvc3Rfc3ZxX25ldyhzdHJ1Y3Qgdmhvc3RfZGV2ICpkZXYsIGludCBpZHgpCj4gICAg ICAgc2l6ZV90IGRyaXZlcl9zaXplOwo+ICAgICAgIHNpemVfdCBkZXZpY2Vfc2l6ZTsKPiAgICAg ICBnX2F1dG9mcmVlIFZob3N0U2hhZG93VmlydHF1ZXVlICpzdnEgPSBnX25ldzAoVmhvc3RTaGFk b3dWaXJ0cXVldWUsIDEpOwo+IC0gICAgaW50IHI7Cj4gKyAgICBpbnQgciwgaTsKPiAgIAo+ICAg ICAgIHIgPSBldmVudF9ub3RpZmllcl9pbml0KCZzdnEtPmhkZXZfa2ljaywgMCk7Cj4gICAgICAg aWYgKHIgIT0gMCkgewo+IEBAIC0yNzQsNiArNjI4LDExIEBAIFZob3N0U2hhZG93VmlydHF1ZXVl ICp2aG9zdF9zdnFfbmV3KHN0cnVjdCB2aG9zdF9kZXYgKmRldiwgaW50IGlkeCkKPiAgICAgICBt ZW1zZXQoc3ZxLT52cmluZy5kZXNjLCAwLCBkcml2ZXJfc2l6ZSk7Cj4gICAgICAgc3ZxLT52cmlu Zy51c2VkID0gcWVtdV9tZW1hbGlnbihxZW11X3JlYWxfaG9zdF9wYWdlX3NpemUsIGRldmljZV9z aXplKTsKPiAgICAgICBtZW1zZXQoc3ZxLT52cmluZy51c2VkLCAwLCBkZXZpY2Vfc2l6ZSk7Cj4g KyAgICBmb3IgKGkgPSAwOyBpIDwgbnVtIC0gMTsgaSsrKSB7Cj4gKyAgICAgICAgc3ZxLT52cmlu Zy5kZXNjW2ldLm5leHQgPSBjcHVfdG9fbGUxNihpICsgMSk7Cj4gKyAgICB9Cj4gKwo+ICsgICAg c3ZxLT5yaW5nX2lkX21hcHMgPSBnX25ldzAoVmlydFF1ZXVlRWxlbWVudCAqLCBudW0pOwo+ICAg ICAgIGV2ZW50X25vdGlmaWVyX3NldF9oYW5kbGVyKCZzdnEtPmhkZXZfY2FsbCwgdmhvc3Rfc3Zx X2hhbmRsZV9jYWxsKTsKPiAgICAgICByZXR1cm4gZ19zdGVhbF9wb2ludGVyKCZzdnEpOwo+ICAg Cj4gQEAgLTI5Miw2ICs2NTEsNyBAQCB2b2lkIHZob3N0X3N2cV9mcmVlKFZob3N0U2hhZG93Vmly dHF1ZXVlICp2cSkKPiAgICAgICBldmVudF9ub3RpZmllcl9jbGVhbnVwKCZ2cS0+aGRldl9raWNr KTsKPiAgICAgICBldmVudF9ub3RpZmllcl9zZXRfaGFuZGxlcigmdnEtPmhkZXZfY2FsbCwgTlVM TCk7Cj4gICAgICAgZXZlbnRfbm90aWZpZXJfY2xlYW51cCgmdnEtPmhkZXZfY2FsbCk7Cj4gKyAg ICBnX2ZyZWUodnEtPnJpbmdfaWRfbWFwcyk7Cj4gICAgICAgcWVtdV92ZnJlZSh2cS0+dnJpbmcu ZGVzYyk7Cj4gICAgICAgcWVtdV92ZnJlZSh2cS0+dnJpbmcudXNlZCk7Cj4gICAgICAgZ19mcmVl KHZxKTsKPiBkaWZmIC0tZ2l0IGEvaHcvdmlydGlvL3Zob3N0LXZkcGEuYyBiL2h3L3ZpcnRpby92 aG9zdC12ZHBhLmMKPiBpbmRleCBmYzgzOTZiYThhLi5lMWM1NWU0M2U3IDEwMDY0NAo+IC0tLSBh L2h3L3ZpcnRpby92aG9zdC12ZHBhLmMKPiArKysgYi9ody92aXJ0aW8vdmhvc3QtdmRwYS5jCj4g QEAgLTE5LDYgKzE5LDcgQEAKPiAgICNpbmNsdWRlICJody92aXJ0aW8vdmlydGlvLW5ldC5oIgo+ ICAgI2luY2x1ZGUgImh3L3ZpcnRpby92aG9zdC1zaGFkb3ctdmlydHF1ZXVlLmgiCj4gICAjaW5j bHVkZSAiaHcvdmlydGlvL3Zob3N0LXZkcGEuaCIKPiArI2luY2x1ZGUgImh3L3ZpcnRpby92aG9z dC1zaGFkb3ctdmlydHF1ZXVlLmgiCj4gICAjaW5jbHVkZSAiZXhlYy9hZGRyZXNzLXNwYWNlcy5o Igo+ICAgI2luY2x1ZGUgInFlbXUvbWFpbi1sb29wLmgiCj4gICAjaW5jbHVkZSAiY3B1LmgiCj4g QEAgLTgyMSw2ICs4MjIsMTkgQEAgc3RhdGljIGJvb2wgIHZob3N0X3ZkcGFfZm9yY2VfaW9tbXUo c3RydWN0IHZob3N0X2RldiAqZGV2KQo+ICAgICAgIHJldHVybiB0cnVlOwo+ICAgfQo+ICAgCj4g K3N0YXRpYyBpbnQgdmhvc3RfdmRwYV92cmluZ19wYXVzZShzdHJ1Y3Qgdmhvc3RfZGV2ICpkZXYp Cj4gK3sKPiArICAgIGludCByOwo+ICsgICAgdWludDhfdCBzdGF0dXM7Cj4gKwo+ICsgICAgdmhv c3RfdmRwYV9hZGRfc3RhdHVzKGRldiwgVklSVElPX0NPTkZJR19TX0RFVklDRV9TVE9QUEVEKTsK PiArICAgIGRvIHsKPiArICAgICAgICByID0gdmhvc3RfdmRwYV9jYWxsKGRldiwgVkhPU1RfVkRQ QV9HRVRfU1RBVFVTLCAmc3RhdHVzKTsKPiArICAgIH0gd2hpbGUgKHIgPT0gMCAmJiAhKHN0YXR1 cyAmIFZJUlRJT19DT05GSUdfU19ERVZJQ0VfU1RPUFBFRCkpOwo+ICsKPiArICAgIHJldHVybiAw Owo+ICt9Cj4gKwo+ICAgLyoKPiAgICAqIFN0YXJ0IG9yIHN0b3AgYSBzaGFkb3cgdmlydHF1ZXVl IGluIGEgdmRwYSBkZXZpY2UKPiAgICAqCj4gQEAgLTg0NCw3ICs4NTgsMTQgQEAgc3RhdGljIGJv b2wgdmhvc3RfdmRwYV9zdnFfc3RhcnRfdnEoc3RydWN0IHZob3N0X2RldiAqZGV2LCB1bnNpZ25l ZCBpZHgsCj4gICAgICAgICAgIC5pbmRleCA9IHZxX2luZGV4LAo+ICAgICAgIH07Cj4gICAgICAg c3RydWN0IHZob3N0X3ZyaW5nX2ZpbGUgdmhvc3RfY2FsbF9maWxlID0gewo+IC0gICAgICAgIC5p bmRleCA9IGlkeCArIGRldi0+dnFfaW5kZXgsCj4gKyAgICAgICAgLmluZGV4ID0gdnFfaW5kZXgs Cj4gKyAgICB9Owo+ICsgICAgc3RydWN0IHZob3N0X3ZyaW5nX2FkZHIgYWRkciA9IHsKPiArICAg ICAgICAuaW5kZXggPSB2cV9pbmRleCwKPiArICAgIH07Cj4gKyAgICBzdHJ1Y3Qgdmhvc3RfdnJp bmdfc3RhdGUgbnVtID0gewo+ICsgICAgICAgIC5pbmRleCA9IHZxX2luZGV4LAo+ICsgICAgICAg IC5udW0gPSB2aXJ0aW9fcXVldWVfZ2V0X251bShkZXYtPnZkZXYsIHZxX2luZGV4KSwKPiAgICAg ICB9Owo+ICAgICAgIGludCByOwo+ICAgCj4gQEAgLTg1Miw2ICs4NzMsNyBAQCBzdGF0aWMgYm9v bCB2aG9zdF92ZHBhX3N2cV9zdGFydF92cShzdHJ1Y3Qgdmhvc3RfZGV2ICpkZXYsIHVuc2lnbmVk IGlkeCwKPiAgICAgICAgICAgY29uc3QgRXZlbnROb3RpZmllciAqdmhvc3Rfa2ljayA9IHZob3N0 X3N2cV9nZXRfZGV2X2tpY2tfbm90aWZpZXIoc3ZxKTsKPiAgICAgICAgICAgY29uc3QgRXZlbnRO b3RpZmllciAqdmhvc3RfY2FsbCA9IHZob3N0X3N2cV9nZXRfc3ZxX2NhbGxfbm90aWZpZXIoc3Zx KTsKPiAgIAo+ICsgICAgICAgIHZob3N0X3N2cV9nZXRfdnJpbmdfYWRkcihzdnEsICZhZGRyKTsK PiAgICAgICAgICAgaWYgKG4tPmFkZHIpIHsKPiAgICAgICAgICAgICAgIHIgPSB2aXJ0aW9fcXVl dWVfc2V0X2hvc3Rfbm90aWZpZXJfbXIoZGV2LT52ZGV2LCBpZHgsICZuLT5tciwKPiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZmFsc2UpOwo+IEBA IC04NzAsOCArODkyLDIwIEBAIHN0YXRpYyBib29sIHZob3N0X3ZkcGFfc3ZxX3N0YXJ0X3ZxKHN0 cnVjdCB2aG9zdF9kZXYgKmRldiwgdW5zaWduZWQgaWR4LAo+ICAgICAgICAgICB2aG9zdF9raWNr X2ZpbGUuZmQgPSBldmVudF9ub3RpZmllcl9nZXRfZmQodmhvc3Rfa2ljayk7Cj4gICAgICAgICAg IHZob3N0X2NhbGxfZmlsZS5mZCA9IGV2ZW50X25vdGlmaWVyX2dldF9mZCh2aG9zdF9jYWxsKTsK PiAgICAgICB9IGVsc2Ugewo+ICsgICAgICAgIHN0cnVjdCB2aG9zdF92cmluZ19zdGF0ZSBzdGF0 ZSA9IHsKPiArICAgICAgICAgICAgLmluZGV4ID0gdnFfaW5kZXgsCj4gKyAgICAgICAgfTsKPiAr Cj4gICAgICAgICAgIHZob3N0X3N2cV9zdG9wKGRldiwgaWR4LCBzdnEpOwo+ICAgCj4gKyAgICAg ICAgc3RhdGUubnVtID0gdmlydGlvX3F1ZXVlX2dldF9sYXN0X2F2YWlsX2lkeChkZXYtPnZkZXYs IGlkeCk7Cj4gKyAgICAgICAgciA9IHZob3N0X3ZkcGFfc2V0X3ZyaW5nX2Jhc2UoZGV2LCAmc3Rh dGUpOwo+ICsgICAgICAgIGlmICh1bmxpa2VseShyKSkgewo+ICsgICAgICAgICAgICBlcnJvcl9z ZXRnX2Vycm5vKGVycnAsIC1yLCAidmhvc3Rfc2V0X3ZyaW5nX2Jhc2UgZmFpbGVkIik7Cj4gKyAg ICAgICAgICAgIHJldHVybiBmYWxzZTsKPiArICAgICAgICB9Cj4gKwo+ICsgICAgICAgIHZob3N0 X3ZkcGFfdnFfZ2V0X2FkZHIoZGV2LCAmYWRkciwgJmRldi0+dnFzW2lkeF0pOwo+ICAgICAgICAg ICBpZiAobi0+YWRkcikgewo+ICAgICAgICAgICAgICAgciA9IHZpcnRpb19xdWV1ZV9zZXRfaG9z dF9ub3RpZmllcl9tcihkZXYtPnZkZXYsIGlkeCwgJm4tPm1yLAo+ICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cnVlKTsKPiBAQCAtODg1LDYgKzkx OSwxNyBAQCBzdGF0aWMgYm9vbCB2aG9zdF92ZHBhX3N2cV9zdGFydF92cShzdHJ1Y3Qgdmhvc3Rf ZGV2ICpkZXYsIHVuc2lnbmVkIGlkeCwKPiAgICAgICAgICAgdmhvc3RfY2FsbF9maWxlLmZkID0g di0+Y2FsbF9mZFtpZHhdOwo+ICAgICAgIH0KPiAgIAo+ICsgICAgciA9IHZob3N0X3ZkcGFfc2V0 X3ZyaW5nX2FkZHIoZGV2LCAmYWRkcik7Cj4gKyAgICBpZiAodW5saWtlbHkocikpIHsKPiArICAg ICAgICBlcnJvcl9zZXRnX2Vycm5vKGVycnAsIC1yLCAidmhvc3Rfc2V0X3ZyaW5nX2FkZHIgZmFp bGVkIik7Cj4gKyAgICAgICAgcmV0dXJuIGZhbHNlOwo+ICsgICAgfQo+ICsgICAgciA9IHZob3N0 X3ZkcGFfc2V0X3ZyaW5nX251bShkZXYsICZudW0pOwo+ICsgICAgaWYgKHVubGlrZWx5KHIpKSB7 Cj4gKyAgICAgICAgZXJyb3Jfc2V0Z19lcnJubyhlcnJwLCAtciwgInZob3N0X3NldF92cmluZ19u dW0gZmFpbGVkIik7Cj4gKyAgICAgICAgcmV0dXJuIGZhbHNlOwo+ICsgICAgfQo+ICsKPiAgICAg ICByID0gdmhvc3RfdmRwYV9zZXRfdnJpbmdfZGV2X2tpY2soZGV2LCAmdmhvc3Rfa2lja19maWxl KTsKPiAgICAgICBpZiAodW5saWtlbHkocikpIHsKPiAgICAgICAgICAgZXJyb3Jfc2V0Z19lcnJu byhlcnJwLCAtciwgInZob3N0X3ZkcGFfc2V0X3ZyaW5nX2tpY2sgZmFpbGVkIik7Cj4gQEAgLTg5 OSw2ICs5NDQsNTAgQEAgc3RhdGljIGJvb2wgdmhvc3RfdmRwYV9zdnFfc3RhcnRfdnEoc3RydWN0 IHZob3N0X2RldiAqZGV2LCB1bnNpZ25lZCBpZHgsCj4gICAgICAgcmV0dXJuIHRydWU7Cj4gICB9 Cj4gICAKPiArc3RhdGljIHZvaWQgdmhvc3RfdmRwYV9nZXRfdnFfc3RhdGUoc3RydWN0IHZob3N0 X2RldiAqZGV2LCB1bnNpZ25lZCBpZHgpCj4gK3sKPiArICAgIHN0cnVjdCBWaXJ0SU9EZXZpY2Ug KnZkZXYgPSBkZXYtPnZkZXY7Cj4gKwo+ICsgICAgdmlydGlvX3F1ZXVlX3Jlc3RvcmVfbGFzdF9h dmFpbF9pZHgodmRldiwgaWR4KTsKPiArICAgIHZpcnRpb19xdWV1ZV9pbnZhbGlkYXRlX3NpZ25h bGxlZF91c2VkKHZkZXYsIGlkeCk7Cj4gKyAgICB2aXJ0aW9fcXVldWVfdXBkYXRlX3VzZWRfaWR4 KHZkZXYsIGlkeCk7Cj4gK30KCgpEbyB3ZSBuZWVkIHRvIGNoYW5nZSB2aG9zdF92ZHBhX2dldF92 cmluZ19iYXNlKCkgdG8gcmV0dXJuIAp2cS0+bGFzdF9hdmFpbF9pZHggYXMgd2VsbD8KClRoYW5r cwoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KVmlydHVh bGl6YXRpb24gbWFpbGluZyBsaXN0ClZpcnR1YWxpemF0aW9uQGxpc3RzLmxpbnV4LWZvdW5kYXRp b24ub3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9uLm9yZy9tYWlsbWFuL2xpc3RpbmZv L3ZpcnR1YWxpemF0aW9u From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB1F4C433F5 for ; Tue, 2 Nov 2021 08:00:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 25E4860F56 for ; Tue, 2 Nov 2021 08:00:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 25E4860F56 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:56926 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhoj0-0004HX-4B for qemu-devel@archiver.kernel.org; Tue, 02 Nov 2021 04:00:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48368) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhohe-0003Wo-DE for qemu-devel@nongnu.org; Tue, 02 Nov 2021 03:59:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:23617) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhohb-0007ID-6k for qemu-devel@nongnu.org; Tue, 02 Nov 2021 03:59:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635839966; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9CvVJ9im3mG5R3xIrXuhU7ug7R+jbBZPgpFPtRunZGA=; b=F3CilOBoa2Rnn1fOek45Rz8zs1CTOW9AyKN0N6vK9tKo0IgNwlMZVdqwljVgAU2CBxfrtH 8NeKqfBuwh0P8X5kE9sxnORv3dk6NLZ5RP3lU20Auej37T38MYAdVdANISs/AMdxIwk5GJ 8DbNyC4NRL6cCqJ4w3Shhu31/pPVUEU= Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-483-BZ1JudBAMWC3d-sZaXhkPA-1; Tue, 02 Nov 2021 03:59:25 -0400 X-MC-Unique: BZ1JudBAMWC3d-sZaXhkPA-1 Received: by mail-pf1-f197.google.com with SMTP id a127-20020a627f85000000b0047feae4a8d9so5775340pfd.19 for ; Tue, 02 Nov 2021 00:59:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=9CvVJ9im3mG5R3xIrXuhU7ug7R+jbBZPgpFPtRunZGA=; b=Lq7iA77KfGJZrWNRiEgAdkFNnVQEianADlCN0hsjtpK+M3Lm2VxBkK9oYN9fzoIKWB RIWANXzrUL7QP/OLfTOu0sakAqaUlOPjOhdfUsLVv2GrR/t8+zOIR667vUh1J4HzxSz8 XbvEOJKy86RBR6CYGz79eh/oyZYUzUiKB1+K7mkfpHicj2Ss5EUfFJkkOZTg+iXKo0af fX/kbB3wqs29gqtDrDcS27bEFM+bXO0dla0+GV4pix0UD1MmjY5/6ujxM2dWNfKW4Frv 8Z+19a0KbxjNNrsavBWRgxb1AokXmHm7fdlDwWVs9oCRWKsSOxDVbSetxtdGKqTInVrJ eM0w== X-Gm-Message-State: AOAM531jnvFXDxJ2a377pRnWFrJDRXDmVGOKB9KSEhQ/y9ogD/UTAc1y faoYOPgNyUx3igBVu4cd3hnyQDx+VDR8T6kfpMbwE3lXrlXwAEUO1j0yHEPmNrbQFIjk74wooEl pGNSEqyXNKORslWw= X-Received: by 2002:a17:90a:7e86:: with SMTP id j6mr4779234pjl.167.1635839963581; Tue, 02 Nov 2021 00:59:23 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw9gC2wIEhXsLE7drd3TjM966xz3oOWtKGRoZPZuFmAxiFvKOoj/qO+ciofZ51gY0dQ+LL1MA== X-Received: by 2002:a17:90a:7e86:: with SMTP id j6mr4779195pjl.167.1635839963141; Tue, 02 Nov 2021 00:59:23 -0700 (PDT) Received: from wangxiaodeMacBook-Air.local ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id m186sm17712178pfb.165.2021.11.02.00.59.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 02 Nov 2021 00:59:22 -0700 (PDT) Subject: Re: [RFC PATCH v5 22/26] vhost: Shadow virtqueue buffers forwarding To: =?UTF-8?Q?Eugenio_P=c3=a9rez?= , qemu-devel@nongnu.org References: <20211029183525.1776416-1-eperezma@redhat.com> <20211029183525.1776416-23-eperezma@redhat.com> From: Jason Wang Message-ID: <5cc9cb09-8b10-d406-25bf-b8de5b510a07@redhat.com> Date: Tue, 2 Nov 2021 15:59:10 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 MIME-Version: 1.0 In-Reply-To: <20211029183525.1776416-23-eperezma@redhat.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US Received-SPF: pass client-ip=216.205.24.124; envelope-from=jasowang@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-1.14, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" 在 2021/10/30 上午2:35, Eugenio Pérez 写道: > Initial version of shadow virtqueue that actually forward buffers. There > are no iommu support at the moment, and that will be addressed in future > patches of this series. Since all vhost-vdpa devices uses forced IOMMU, > this means that SVQ is not usable at this point of the series on any > device. > > For simplicity it only supports modern devices, that expects vring > in little endian, with split ring and no event idx or indirect > descriptors. Support for them will not be added in this series. > > It reuses the VirtQueue code for the device part. The driver part is > based on Linux's virtio_ring driver, but with stripped functionality > and optimizations so it's easier to review. Later commits add simpler > ones. > > However to forwarding buffers have some particular pieces: One of the > most unexpected ones is that a guest's buffer can expand through more > than one descriptor in SVQ. While this is handled gracefully by qemu's > emulated virtio devices, it may cause unexpected SVQ queue full. This > patch also solves it checking for this condition at both guest's kicks > and device's calls. The code may be more elegant in the future if SVQ > code runs in its own iocontext. > > Note that vhost_vdpa_get_vq_state trust the device to write its status > to used_idx at pause(), finishing all in-flight descriptors. This may > not be enough for complex devices, but other development like usage of > inflight_fd on top of this solution may extend the usage in the future. > > In particular, SVQ trust it to recover guest's virtqueue at start, and > to mark as used the latest descriptors used by the device in the > meantime. > > Signed-off-by: Eugenio Pérez > --- > qapi/net.json | 5 +- > hw/virtio/vhost-shadow-virtqueue.c | 400 +++++++++++++++++++++++++++-- > hw/virtio/vhost-vdpa.c | 144 ++++++++++- > 3 files changed, 521 insertions(+), 28 deletions(-) > > diff --git a/qapi/net.json b/qapi/net.json > index fca2f6ebca..1c6d3b2179 100644 > --- a/qapi/net.json > +++ b/qapi/net.json > @@ -84,12 +84,9 @@ > # > # Use vhost shadow virtqueue. > # > -# SVQ can just forward notifications between the device and the guest at this > -# moment. This will expand in future changes. > -# > # @name: the device name of the VirtIO device > # > -# @set: true to use the alternate shadow VQ notifications > +# @set: true to use the alternate shadow VQ > # > # Since: 6.2 > # > diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c > index cb9ffcb015..ad1b2342be 100644 > --- a/hw/virtio/vhost-shadow-virtqueue.c > +++ b/hw/virtio/vhost-shadow-virtqueue.c > @@ -9,6 +9,9 @@ > > #include "qemu/osdep.h" > #include "hw/virtio/vhost-shadow-virtqueue.h" > +#include "hw/virtio/vhost.h" > +#include "hw/virtio/virtio-access.h" > + > #include "standard-headers/linux/vhost_types.h" > > #include "qemu/error-report.h" > @@ -45,6 +48,27 @@ typedef struct VhostShadowVirtqueue { > > /* Virtio device */ > VirtIODevice *vdev; > + > + /* Map for returning guest's descriptors */ > + VirtQueueElement **ring_id_maps; > + > + /* Next VirtQueue element that guest made available */ > + VirtQueueElement *next_guest_avail_elem; > + > + /* Next head to expose to device */ > + uint16_t avail_idx_shadow; > + > + /* Next free descriptor */ > + uint16_t free_head; > + > + /* Last seen used idx */ > + uint16_t shadow_used_idx; > + > + /* Next head to consume from device */ > + uint16_t last_used_idx; > + > + /* Cache for the exposed notification flag */ > + bool notification; > } VhostShadowVirtqueue; > > /** > @@ -56,25 +80,174 @@ const EventNotifier *vhost_svq_get_dev_kick_notifier( > return &svq->hdev_kick; > } > > -/* If the device is using some of these, SVQ cannot communicate */ > +/** > + * VirtIO transport device feature acknowledge > + * > + * @dev_features The device features. If success, the acknowledged features. > + * > + * Returns true if SVQ can go with a subset of these, false otherwise. > + */ > bool vhost_svq_valid_device_features(uint64_t *dev_features) > { > - return true; > + uint64_t b; > + bool r = true; > + > + for (b = VIRTIO_TRANSPORT_F_START; b <= VIRTIO_TRANSPORT_F_END; ++b) { > + switch (b) { > + case VIRTIO_F_NOTIFY_ON_EMPTY: > + case VIRTIO_F_ANY_LAYOUT: > + continue; > + > + case VIRTIO_F_ACCESS_PLATFORM: > + /* SVQ does not know how to translate addresses */ > + if (*dev_features & BIT_ULL(b)) { > + clear_bit(b, dev_features); > + r = false; > + } > + break; > + > + case VIRTIO_F_VERSION_1: > + /* SVQ trust that guest vring is little endian */ > + if (!(*dev_features & BIT_ULL(b))) { > + set_bit(b, dev_features); > + r = false; > + } > + continue; > + > + default: > + if (*dev_features & BIT_ULL(b)) { > + clear_bit(b, dev_features); > + } > + } > + } > + > + return r; > } > > -/* If the guest is using some of these, SVQ cannot communicate */ > +/** > + * Check of guest's acknowledge features. > + * > + * @guest_features The guest's acknowledged features > + * > + * Returns true if SVQ can handle them, false otherwise. > + */ > bool vhost_svq_valid_guest_features(uint64_t *guest_features) > { > - return true; > + static const uint64_t transport = MAKE_64BIT_MASK(VIRTIO_TRANSPORT_F_START, > + VIRTIO_TRANSPORT_F_END - VIRTIO_TRANSPORT_F_START); > + > + /* These transport features are handled by VirtQueue */ > + static const uint64_t valid = (BIT_ULL(VIRTIO_RING_F_INDIRECT_DESC) | > + BIT_ULL(VIRTIO_F_VERSION_1)); > + > + /* We are only interested in transport-related feature bits */ > + uint64_t guest_transport_features = (*guest_features) & transport; > + > + *guest_features &= (valid | ~transport); > + return !(guest_transport_features & (transport ^ valid)); > } > > -/* Forward guest notifications */ > -static void vhost_handle_guest_kick(EventNotifier *n) > +/** > + * Number of descriptors that SVQ can make available from the guest. > + * > + * @svq The svq > + */ > +static uint16_t vhost_svq_available_slots(const VhostShadowVirtqueue *svq) > { > - VhostShadowVirtqueue *svq = container_of(n, VhostShadowVirtqueue, > - svq_kick); > + return svq->vring.num - (svq->avail_idx_shadow - svq->shadow_used_idx); > +} > > - if (unlikely(!event_notifier_test_and_clear(n))) { > +static void vhost_svq_set_notification(VhostShadowVirtqueue *svq, bool enable) > +{ > + uint16_t notification_flag; > + > + if (svq->notification == enable) { > + return; > + } > + > + notification_flag = cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); > + > + svq->notification = enable; > + if (enable) { > + svq->vring.avail->flags &= ~notification_flag; > + } else { > + svq->vring.avail->flags |= notification_flag; > + } > +} > + > +static void vhost_vring_write_descs(VhostShadowVirtqueue *svq, > + const struct iovec *iovec, > + size_t num, bool more_descs, bool write) > +{ > + uint16_t i = svq->free_head, last = svq->free_head; > + unsigned n; > + uint16_t flags = write ? cpu_to_le16(VRING_DESC_F_WRITE) : 0; > + vring_desc_t *descs = svq->vring.desc; > + > + if (num == 0) { > + return; > + } > + > + for (n = 0; n < num; n++) { > + if (more_descs || (n + 1 < num)) { > + descs[i].flags = flags | cpu_to_le16(VRING_DESC_F_NEXT); > + } else { > + descs[i].flags = flags; > + } > + descs[i].addr = cpu_to_le64((hwaddr)iovec[n].iov_base); > + descs[i].len = cpu_to_le32(iovec[n].iov_len); > + > + last = i; > + i = cpu_to_le16(descs[i].next); > + } > + > + svq->free_head = le16_to_cpu(descs[last].next); > +} > + > +static unsigned vhost_svq_add_split(VhostShadowVirtqueue *svq, > + VirtQueueElement *elem) > +{ > + int head; > + unsigned avail_idx; > + vring_avail_t *avail = svq->vring.avail; > + > + head = svq->free_head; > + > + /* We need some descriptors here */ > + assert(elem->out_num || elem->in_num); > + > + vhost_vring_write_descs(svq, elem->out_sg, elem->out_num, > + elem->in_num > 0, false); > + vhost_vring_write_descs(svq, elem->in_sg, elem->in_num, false, true); > + > + /* > + * Put entry in available array (but don't update avail->idx until they > + * do sync). > + */ > + avail_idx = svq->avail_idx_shadow & (svq->vring.num - 1); > + avail->ring[avail_idx] = cpu_to_le16(head); > + svq->avail_idx_shadow++; > + > + /* Update avail index after the descriptor is wrote */ > + smp_wmb(); A question, since we may talk with the real hardware, is smp_wmb() sufficient in this case or do we need to honer VIRTIO_F_ORDER_PLATFORM? > + avail->idx = cpu_to_le16(svq->avail_idx_shadow); > + > + return head; > + > +} > + > +static void vhost_svq_add(VhostShadowVirtqueue *svq, VirtQueueElement *elem) > +{ > + unsigned qemu_head = vhost_svq_add_split(svq, elem); > + > + svq->ring_id_maps[qemu_head] = elem; > +} > + > +static void vhost_svq_kick(VhostShadowVirtqueue *svq) > +{ > + /* We need to expose available array entries before checking used flags */ > + smp_mb(); > + if (svq->vring.used->flags & VRING_USED_F_NO_NOTIFY) { > return; > } > > @@ -86,25 +259,188 @@ static void vhost_handle_guest_kick(EventNotifier *n) > } > } > > -/* > - * Set the device's memory region notifier. addr = NULL clear it. > +/** > + * Forward available buffers. > + * > + * @svq Shadow VirtQueue > + * > + * Note that this function does not guarantee that all guest's available > + * buffers are available to the device in SVQ avail ring. The guest may have > + * exposed a GPA / GIOVA congiuous buffer, but it may not be contiguous in qemu > + * vaddr. > + * > + * If that happens, guest's kick notifications will be disabled until device > + * makes some buffers used. > */ > -void vhost_svq_set_host_mr_notifier(VhostShadowVirtqueue *svq, void *addr) > +static void vhost_handle_guest_kick(VhostShadowVirtqueue *svq) > { > - svq->host_notifier_mr = addr; > + /* Clear event notifier */ > + event_notifier_test_and_clear(&svq->svq_kick); > + > + /* Make available as many buffers as possible */ > + do { > + if (virtio_queue_get_notification(svq->vq)) { > + virtio_queue_set_notification(svq->vq, false); > + } > + > + while (true) { > + VirtQueueElement *elem; > + > + if (svq->next_guest_avail_elem) { > + elem = g_steal_pointer(&svq->next_guest_avail_elem); > + } else { > + elem = virtqueue_pop(svq->vq, sizeof(*elem)); > + } > + > + if (!elem) { > + break; > + } > + > + if (elem->out_num + elem->in_num > > + vhost_svq_available_slots(svq)) { > + /* > + * This condition is possible since a contiguous buffer in GPA > + * does not imply a contiguous buffer in qemu's VA > + * scatter-gather segments. If that happen, the buffer exposed > + * to the device needs to be a chain of descriptors at this > + * moment. > + * > + * SVQ cannot hold more available buffers if we are here: > + * queue the current guest descriptor and ignore further kicks > + * until some elements are used. > + */ I wonder what's the advantage of tracking the pending elem like this. It looks to me we can simply rewind last_avail_idx in this case? > + svq->next_guest_avail_elem = elem; > + return; > + } > + > + vhost_svq_add(svq, elem); > + vhost_svq_kick(svq); > + } > + > + virtio_queue_set_notification(svq->vq, true); > + } while (!virtio_queue_empty(svq->vq)); > +} > + > +/** > + * Handle guest's kick. > + * > + * @n guest kick event notifier, the one that guest set to notify svq. > + */ > +static void vhost_handle_guest_kick_notifier(EventNotifier *n) > +{ > + VhostShadowVirtqueue *svq = container_of(n, VhostShadowVirtqueue, > + svq_kick); > + vhost_handle_guest_kick(svq); > +} > + > +static bool vhost_svq_more_used(VhostShadowVirtqueue *svq) > +{ > + if (svq->last_used_idx != svq->shadow_used_idx) { > + return true; > + } > + > + svq->shadow_used_idx = cpu_to_le16(svq->vring.used->idx); > + > + return svq->last_used_idx != svq->shadow_used_idx; > +} > + > +static VirtQueueElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq) > +{ > + vring_desc_t *descs = svq->vring.desc; > + const vring_used_t *used = svq->vring.used; > + vring_used_elem_t used_elem; > + uint16_t last_used; > + > + if (!vhost_svq_more_used(svq)) { > + return NULL; > + } > + > + /* Only get used array entries after they have been exposed by dev */ > + smp_rmb(); > + last_used = svq->last_used_idx & (svq->vring.num - 1); > + used_elem.id = le32_to_cpu(used->ring[last_used].id); > + used_elem.len = le32_to_cpu(used->ring[last_used].len); > + > + svq->last_used_idx++; > + if (unlikely(used_elem.id >= svq->vring.num)) { > + error_report("Device %s says index %u is used", svq->vdev->name, > + used_elem.id); > + return NULL; > + } > + > + if (unlikely(!svq->ring_id_maps[used_elem.id])) { > + error_report( > + "Device %s says index %u is used, but it was not available", > + svq->vdev->name, used_elem.id); > + return NULL; > + } > + > + descs[used_elem.id].next = svq->free_head; > + svq->free_head = used_elem.id; > + > + svq->ring_id_maps[used_elem.id]->len = used_elem.len; > + return g_steal_pointer(&svq->ring_id_maps[used_elem.id]); > } > > -/* Forward vhost notifications */ > +static void vhost_svq_flush(VhostShadowVirtqueue *svq, > + bool check_for_avail_queue) > +{ > + VirtQueue *vq = svq->vq; > + > + /* Make as many buffers as possible used. */ > + do { > + unsigned i = 0; > + > + vhost_svq_set_notification(svq, false); > + while (true) { > + g_autofree VirtQueueElement *elem = vhost_svq_get_buf(svq); > + if (!elem) { > + break; > + } > + > + if (unlikely(i >= svq->vring.num)) { > + virtio_error(svq->vdev, > + "More than %u used buffers obtained in a %u size SVQ", > + i, svq->vring.num); > + virtqueue_fill(vq, elem, elem->len, i); > + virtqueue_flush(vq, i); > + i = 0; > + } > + virtqueue_fill(vq, elem, elem->len, i++); > + } > + > + virtqueue_flush(vq, i); > + event_notifier_set(&svq->svq_call); > + > + if (check_for_avail_queue && svq->next_guest_avail_elem) { > + /* > + * Avail ring was full when vhost_svq_flush was called, so it's a > + * good moment to make more descriptors available if possible > + */ > + vhost_handle_guest_kick(svq); > + } > + > + vhost_svq_set_notification(svq, true); > + } while (vhost_svq_more_used(svq)); So this actually doesn't make sure all the buffers were processed by the device? Is this intended (I see it was called by the vhost_svq_stop()). Note that it means some buffers might not be submitted to the device after migration? > +} > + > +/** > + * Forward used buffers. > + * > + * @n hdev call event notifier, the one that device set to notify svq. > + * > + * Note that we are not making any buffers available in the loop, there is no > + * way that it runs more than virtqueue size times. > + */ > static void vhost_svq_handle_call(EventNotifier *n) > { > VhostShadowVirtqueue *svq = container_of(n, VhostShadowVirtqueue, > hdev_call); > > - if (unlikely(!event_notifier_test_and_clear(n))) { > - return; > - } > + /* Clear event notifier */ > + event_notifier_test_and_clear(n); > > - event_notifier_set(&svq->svq_call); > + vhost_svq_flush(svq, true); > } > > /* > @@ -132,6 +468,14 @@ void vhost_svq_set_guest_call_notifier(VhostShadowVirtqueue *svq, int call_fd) > event_notifier_init_fd(&svq->svq_call, call_fd); > } > > +/* > + * Set the device's memory region notifier. addr = NULL clear it. > + */ > +void vhost_svq_set_host_mr_notifier(VhostShadowVirtqueue *svq, void *addr) > +{ > + svq->host_notifier_mr = addr; > +} > + > /* > * Get the shadow vq vring address. > * @svq Shadow virtqueue > @@ -185,7 +529,8 @@ static void vhost_svq_set_svq_kick_fd_internal(VhostShadowVirtqueue *svq, > * need to explicitely check for them. > */ > event_notifier_init_fd(&svq->svq_kick, svq_kick_fd); > - event_notifier_set_handler(&svq->svq_kick, vhost_handle_guest_kick); > + event_notifier_set_handler(&svq->svq_kick, > + vhost_handle_guest_kick_notifier); > > /* > * !check_old means that we are starting SVQ, taking the descriptor from > @@ -233,7 +578,16 @@ void vhost_svq_start(struct vhost_dev *dev, unsigned idx, > void vhost_svq_stop(struct vhost_dev *dev, unsigned idx, > VhostShadowVirtqueue *svq) > { > + unsigned i; > event_notifier_set_handler(&svq->svq_kick, NULL); > + vhost_svq_flush(svq, false); > + > + for (i = 0; i < svq->vring.num; ++i) { > + g_autofree VirtQueueElement *elem = svq->ring_id_maps[i]; > + if (elem) { > + virtqueue_detach_element(svq->vq, elem, elem->len); > + } > + } > } > > /* > @@ -248,7 +602,7 @@ VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *dev, int idx) > size_t driver_size; > size_t device_size; > g_autofree VhostShadowVirtqueue *svq = g_new0(VhostShadowVirtqueue, 1); > - int r; > + int r, i; > > r = event_notifier_init(&svq->hdev_kick, 0); > if (r != 0) { > @@ -274,6 +628,11 @@ VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *dev, int idx) > memset(svq->vring.desc, 0, driver_size); > svq->vring.used = qemu_memalign(qemu_real_host_page_size, device_size); > memset(svq->vring.used, 0, device_size); > + for (i = 0; i < num - 1; i++) { > + svq->vring.desc[i].next = cpu_to_le16(i + 1); > + } > + > + svq->ring_id_maps = g_new0(VirtQueueElement *, num); > event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call); > return g_steal_pointer(&svq); > > @@ -292,6 +651,7 @@ void vhost_svq_free(VhostShadowVirtqueue *vq) > event_notifier_cleanup(&vq->hdev_kick); > event_notifier_set_handler(&vq->hdev_call, NULL); > event_notifier_cleanup(&vq->hdev_call); > + g_free(vq->ring_id_maps); > qemu_vfree(vq->vring.desc); > qemu_vfree(vq->vring.used); > g_free(vq); > diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c > index fc8396ba8a..e1c55e43e7 100644 > --- a/hw/virtio/vhost-vdpa.c > +++ b/hw/virtio/vhost-vdpa.c > @@ -19,6 +19,7 @@ > #include "hw/virtio/virtio-net.h" > #include "hw/virtio/vhost-shadow-virtqueue.h" > #include "hw/virtio/vhost-vdpa.h" > +#include "hw/virtio/vhost-shadow-virtqueue.h" > #include "exec/address-spaces.h" > #include "qemu/main-loop.h" > #include "cpu.h" > @@ -821,6 +822,19 @@ static bool vhost_vdpa_force_iommu(struct vhost_dev *dev) > return true; > } > > +static int vhost_vdpa_vring_pause(struct vhost_dev *dev) > +{ > + int r; > + uint8_t status; > + > + vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_DEVICE_STOPPED); > + do { > + r = vhost_vdpa_call(dev, VHOST_VDPA_GET_STATUS, &status); > + } while (r == 0 && !(status & VIRTIO_CONFIG_S_DEVICE_STOPPED)); > + > + return 0; > +} > + > /* > * Start or stop a shadow virtqueue in a vdpa device > * > @@ -844,7 +858,14 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *dev, unsigned idx, > .index = vq_index, > }; > struct vhost_vring_file vhost_call_file = { > - .index = idx + dev->vq_index, > + .index = vq_index, > + }; > + struct vhost_vring_addr addr = { > + .index = vq_index, > + }; > + struct vhost_vring_state num = { > + .index = vq_index, > + .num = virtio_queue_get_num(dev->vdev, vq_index), > }; > int r; > > @@ -852,6 +873,7 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *dev, unsigned idx, > const EventNotifier *vhost_kick = vhost_svq_get_dev_kick_notifier(svq); > const EventNotifier *vhost_call = vhost_svq_get_svq_call_notifier(svq); > > + vhost_svq_get_vring_addr(svq, &addr); > if (n->addr) { > r = virtio_queue_set_host_notifier_mr(dev->vdev, idx, &n->mr, > false); > @@ -870,8 +892,20 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *dev, unsigned idx, > vhost_kick_file.fd = event_notifier_get_fd(vhost_kick); > vhost_call_file.fd = event_notifier_get_fd(vhost_call); > } else { > + struct vhost_vring_state state = { > + .index = vq_index, > + }; > + > vhost_svq_stop(dev, idx, svq); > > + state.num = virtio_queue_get_last_avail_idx(dev->vdev, idx); > + r = vhost_vdpa_set_vring_base(dev, &state); > + if (unlikely(r)) { > + error_setg_errno(errp, -r, "vhost_set_vring_base failed"); > + return false; > + } > + > + vhost_vdpa_vq_get_addr(dev, &addr, &dev->vqs[idx]); > if (n->addr) { > r = virtio_queue_set_host_notifier_mr(dev->vdev, idx, &n->mr, > true); > @@ -885,6 +919,17 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *dev, unsigned idx, > vhost_call_file.fd = v->call_fd[idx]; > } > > + r = vhost_vdpa_set_vring_addr(dev, &addr); > + if (unlikely(r)) { > + error_setg_errno(errp, -r, "vhost_set_vring_addr failed"); > + return false; > + } > + r = vhost_vdpa_set_vring_num(dev, &num); > + if (unlikely(r)) { > + error_setg_errno(errp, -r, "vhost_set_vring_num failed"); > + return false; > + } > + > r = vhost_vdpa_set_vring_dev_kick(dev, &vhost_kick_file); > if (unlikely(r)) { > error_setg_errno(errp, -r, "vhost_vdpa_set_vring_kick failed"); > @@ -899,6 +944,50 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *dev, unsigned idx, > return true; > } > > +static void vhost_vdpa_get_vq_state(struct vhost_dev *dev, unsigned idx) > +{ > + struct VirtIODevice *vdev = dev->vdev; > + > + virtio_queue_restore_last_avail_idx(vdev, idx); > + virtio_queue_invalidate_signalled_used(vdev, idx); > + virtio_queue_update_used_idx(vdev, idx); > +} Do we need to change vhost_vdpa_get_vring_base() to return vq->last_avail_idx as well? Thanks