From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C96C8C433EF for ; Sun, 30 Jan 2022 04:43:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 5279F4018B; Sun, 30 Jan 2022 04:43:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SRvQ6vGcJjpz; Sun, 30 Jan 2022 04:43:08 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 497F7401E9; Sun, 30 Jan 2022 04:43:08 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 07471C0011; Sun, 30 Jan 2022 04:43:08 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 94508C000B for ; Sun, 30 Jan 2022 04:43:06 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 6A44360B48 for ; Sun, 30 Jan 2022 04:43:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DxEMDig9Gczm for ; Sun, 30 Jan 2022 04:43:05 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id D6BF4607D0 for ; Sun, 30 Jan 2022 04:43:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643517783; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b2drNTCNkwJABK5/bzrkE2Qv3Y+XGpam7FKWmJlBdG8=; b=ZVnidb+GyHpZZ1HRMiYP7/jx97khkXHuMD+WhZppOwN6FTgVORRbFzPeF6pBt2ElcbwW3o B7VQuVPnBoSz6HDoFdsodWNiyCTiqbeqct6eHgCNPCj9aNurfYTyjMaQJ+jAgmE2d6uWzR rIk79n/czMotLWRkyD/Ow4LHsxCu11c= Received: from mail-pj1-f69.google.com (mail-pj1-f69.google.com [209.85.216.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-645-SyqPBMTZMhisUz10--4TEw-1; Sat, 29 Jan 2022 23:43:01 -0500 X-MC-Unique: SyqPBMTZMhisUz10--4TEw-1 Received: by mail-pj1-f69.google.com with SMTP id x4-20020a17090ab00400b001b58c484826so8998822pjq.0 for ; Sat, 29 Jan 2022 20:43:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=b2drNTCNkwJABK5/bzrkE2Qv3Y+XGpam7FKWmJlBdG8=; b=WRa9BdHG4GlCTcr5RZexbIQkMrrlZld8n1avPCSvbh9CIy2CXT0YFoy9UJhHaqRcPM w2tiqmv3S8o4i1RqNOQvwfaw9zdGGafRjiXZoZbCse8afO29nQ2qL93gPfBwi3smxuLP BgrpbSbadtp/b008S8ZWs2At/+Au2kVgTkkJrsDSlQSdk1YzSfcYkQX26pfUpGbBBZz8 D5Tyvj99NtvPphlQBiVz4XbKPtwlDo+jAy5GzuGRDmH0fI7empa3ysku4pG1seEGnmTg ppHA9KeYX5HmcR2iX7n62aKy0EZCAEvgc/85HYnKYDFUmyJppRtLyixjRSdZteDwW0DM LwZg== X-Gm-Message-State: AOAM532M2fFv60vT1sKLn/bSP2PlonC3Xo9G+MemAr4AtoSDbX7nB9Zw tohVEoUfXAzG+VoG8uZ0vxdADlr5dKd1bhvLAeoDYQcgP5pk+ABwNpZbMLR/QKivpjW4ik+oZEO 20QqQfXn9wssSyh6MigpTpxvl2tYty/3slc9USp2wEw== X-Received: by 2002:a05:6a00:14cc:: with SMTP id w12mr14774264pfu.47.1643517780584; Sat, 29 Jan 2022 20:43:00 -0800 (PST) X-Google-Smtp-Source: ABdhPJyHwZkhnrIFy+LQWFKt5x7QLQ24RCeeTJpFcEaHvUcHjHXhwWTDpqN3V2x9WpK0HEtWuzkjgw== X-Received: by 2002:a05:6a00:14cc:: with SMTP id w12mr14774228pfu.47.1643517780114; Sat, 29 Jan 2022 20:43:00 -0800 (PST) Received: from [10.72.13.18] ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id q12sm14936965pfk.199.2022.01.29.20.42.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 29 Jan 2022 20:42:59 -0800 (PST) Message-ID: <6c98f4e8-8695-ab83-ae37-2d6293a1fafa@redhat.com> Date: Sun, 30 Jan 2022 12:42:44 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.5.1 Subject: Re: [PATCH 18/31] vhost: Shadow virtqueue buffers forwarding To: =?UTF-8?Q?Eugenio_P=c3=a9rez?= , qemu-devel@nongnu.org References: <20220121202733.404989-1-eperezma@redhat.com> <20220121202733.404989-19-eperezma@redhat.com> From: Jason Wang In-Reply-To: <20220121202733.404989-19-eperezma@redhat.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Cc: Laurent Vivier , Parav Pandit , Cindy Lu , "Michael S. Tsirkin" , Richard Henderson , Markus Armbruster , Gautam Dawar , virtualization@lists.linux-foundation.org, Eduardo Habkost , Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Paolo Bonzini , Zhu Lingshan , Eric Blake X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" CuWcqCAyMDIyLzEvMjIg5LiK5Y2INDoyNywgRXVnZW5pbyBQw6lyZXog5YaZ6YGTOgo+IEluaXRp YWwgdmVyc2lvbiBvZiBzaGFkb3cgdmlydHF1ZXVlIHRoYXQgYWN0dWFsbHkgZm9yd2FyZCBidWZm ZXJzLiBUaGVyZQo+IGlzIG5vIGlvbW11IHN1cHBvcnQgYXQgdGhlIG1vbWVudCwgYW5kIHRoYXQg d2lsbCBiZSBhZGRyZXNzZWQgaW4gZnV0dXJlCj4gcGF0Y2hlcyBvZiB0aGlzIHNlcmllcy4gU2lu Y2UgYWxsIHZob3N0LXZkcGEgZGV2aWNlcyB1c2UgZm9yY2VkIElPTU1VLAo+IHRoaXMgbWVhbnMg dGhhdCBTVlEgaXMgbm90IHVzYWJsZSBhdCB0aGlzIHBvaW50IG9mIHRoZSBzZXJpZXMgb24gYW55 Cj4gZGV2aWNlLgo+Cj4gRm9yIHNpbXBsaWNpdHkgaXQgb25seSBzdXBwb3J0cyBtb2Rlcm4gZGV2 aWNlcywgdGhhdCBleHBlY3RzIHZyaW5nCj4gaW4gbGl0dGxlIGVuZGlhbiwgd2l0aCBzcGxpdCBy aW5nIGFuZCBubyBldmVudCBpZHggb3IgaW5kaXJlY3QKPiBkZXNjcmlwdG9ycy4gU3VwcG9ydCBm b3IgdGhlbSB3aWxsIG5vdCBiZSBhZGRlZCBpbiB0aGlzIHNlcmllcy4KPgo+IEl0IHJldXNlcyB0 aGUgVmlydFF1ZXVlIGNvZGUgZm9yIHRoZSBkZXZpY2UgcGFydC4gVGhlIGRyaXZlciBwYXJ0IGlz Cj4gYmFzZWQgb24gTGludXgncyB2aXJ0aW9fcmluZyBkcml2ZXIsIGJ1dCB3aXRoIHN0cmlwcGVk IGZ1bmN0aW9uYWxpdHkKPiBhbmQgb3B0aW1pemF0aW9ucyBzbyBpdCdzIGVhc2llciB0byByZXZp ZXcuCj4KPiBIb3dldmVyLCBmb3J3YXJkaW5nIGJ1ZmZlcnMgaGF2ZSBzb21lIHBhcnRpY3VsYXIg cGllY2VzOiBPbmUgb2YgdGhlIG1vc3QKPiB1bmV4cGVjdGVkIG9uZXMgaXMgdGhhdCBhIGd1ZXN0 J3MgYnVmZmVyIGNhbiBleHBhbmQgdGhyb3VnaCBtb3JlIHRoYW4KPiBvbmUgZGVzY3JpcHRvciBp biBTVlEuIFdoaWxlIHRoaXMgaXMgaGFuZGxlZCBncmFjZWZ1bGx5IGJ5IHFlbXUncwo+IGVtdWxh dGVkIHZpcnRpbyBkZXZpY2VzLCBpdCBtYXkgY2F1c2UgdW5leHBlY3RlZCBTVlEgcXVldWUgZnVs bC4gVGhpcwo+IHBhdGNoIGFsc28gc29sdmVzIGl0IGJ5IGNoZWNraW5nIGZvciB0aGlzIGNvbmRp dGlvbiBhdCBib3RoIGd1ZXN0J3MKPiBraWNrcyBhbmQgZGV2aWNlJ3MgY2FsbHMuIFRoZSBjb2Rl IG1heSBiZSBtb3JlIGVsZWdhbnQgaW4gdGhlIGZ1dHVyZSBpZgo+IFNWUSBjb2RlIHJ1bnMgaW4g aXRzIG93biBpb2NvbnRleHQuCj4KPiBTaWduZWQtb2ZmLWJ5OiBFdWdlbmlvIFDDqXJleiA8ZXBl cmV6bWFAcmVkaGF0LmNvbT4KPiAtLS0KPiAgIGh3L3ZpcnRpby92aG9zdC1zaGFkb3ctdmlydHF1 ZXVlLmggfCAgIDIgKwo+ICAgaHcvdmlydGlvL3Zob3N0LXNoYWRvdy12aXJ0cXVldWUuYyB8IDM2 NSArKysrKysrKysrKysrKysrKysrKysrKysrKysrLQo+ICAgaHcvdmlydGlvL3Zob3N0LXZkcGEu YyAgICAgICAgICAgICB8IDExMSArKysrKysrKy0KPiAgIDMgZmlsZXMgY2hhbmdlZCwgNDYyIGlu c2VydGlvbnMoKyksIDE2IGRlbGV0aW9ucygtKQo+Cj4gZGlmZiAtLWdpdCBhL2h3L3ZpcnRpby92 aG9zdC1zaGFkb3ctdmlydHF1ZXVlLmggYi9ody92aXJ0aW8vdmhvc3Qtc2hhZG93LXZpcnRxdWV1 ZS5oCj4gaW5kZXggMzlhZWY1ZmZkZi4uMTljOTM0YWY0OSAxMDA2NDQKPiAtLS0gYS9ody92aXJ0 aW8vdmhvc3Qtc2hhZG93LXZpcnRxdWV1ZS5oCj4gKysrIGIvaHcvdmlydGlvL3Zob3N0LXNoYWRv dy12aXJ0cXVldWUuaAo+IEBAIC0zMyw2ICszMyw4IEBAIHVpbnQxNl90IHZob3N0X3N2cV9nZXRf bnVtKGNvbnN0IFZob3N0U2hhZG93VmlydHF1ZXVlICpzdnEpOwo+ICAgc2l6ZV90IHZob3N0X3N2 cV9kcml2ZXJfYXJlYV9zaXplKGNvbnN0IFZob3N0U2hhZG93VmlydHF1ZXVlICpzdnEpOwo+ICAg c2l6ZV90IHZob3N0X3N2cV9kZXZpY2VfYXJlYV9zaXplKGNvbnN0IFZob3N0U2hhZG93VmlydHF1 ZXVlICpzdnEpOwo+ICAgCj4gK3ZvaWQgdmhvc3Rfc3ZxX3N0YXJ0KFZob3N0U2hhZG93VmlydHF1 ZXVlICpzdnEsIFZpcnRJT0RldmljZSAqdmRldiwKPiArICAgICAgICAgICAgICAgICAgICAgVmly dFF1ZXVlICp2cSk7Cj4gICB2b2lkIHZob3N0X3N2cV9zdG9wKFZob3N0U2hhZG93VmlydHF1ZXVl ICpzdnEpOwo+ICAgCj4gICBWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAqdmhvc3Rfc3ZxX25ldyh1aW50 MTZfdCBxc2l6ZSk7Cj4gZGlmZiAtLWdpdCBhL2h3L3ZpcnRpby92aG9zdC1zaGFkb3ctdmlydHF1 ZXVlLmMgYi9ody92aXJ0aW8vdmhvc3Qtc2hhZG93LXZpcnRxdWV1ZS5jCj4gaW5kZXggN2MxNjgw NzVkNy4uYTFhNDA0ZjY4ZiAxMDA2NDQKPiAtLS0gYS9ody92aXJ0aW8vdmhvc3Qtc2hhZG93LXZp cnRxdWV1ZS5jCj4gKysrIGIvaHcvdmlydGlvL3Zob3N0LXNoYWRvdy12aXJ0cXVldWUuYwo+IEBA IC05LDYgKzksOCBAQAo+ICAgCj4gICAjaW5jbHVkZSAicWVtdS9vc2RlcC5oIgo+ICAgI2luY2x1 ZGUgImh3L3ZpcnRpby92aG9zdC1zaGFkb3ctdmlydHF1ZXVlLmgiCj4gKyNpbmNsdWRlICJody92 aXJ0aW8vdmhvc3QuaCIKPiArI2luY2x1ZGUgImh3L3ZpcnRpby92aXJ0aW8tYWNjZXNzLmgiCj4g ICAjaW5jbHVkZSAic3RhbmRhcmQtaGVhZGVycy9saW51eC92aG9zdF90eXBlcy5oIgo+ICAgCj4g ICAjaW5jbHVkZSAicWVtdS9lcnJvci1yZXBvcnQuaCIKPiBAQCAtMzYsNiArMzgsMzMgQEAgdHlw ZWRlZiBzdHJ1Y3QgVmhvc3RTaGFkb3dWaXJ0cXVldWUgewo+ICAgCj4gICAgICAgLyogR3Vlc3Qn cyBjYWxsIG5vdGlmaWVyLCB3aGVyZSBTVlEgY2FsbHMgZ3Vlc3QuICovCj4gICAgICAgRXZlbnRO b3RpZmllciBzdnFfY2FsbDsKPiArCj4gKyAgICAvKiBWaXJ0aW8gcXVldWUgc2hhZG93aW5nICov Cj4gKyAgICBWaXJ0UXVldWUgKnZxOwo+ICsKPiArICAgIC8qIFZpcnRpbyBkZXZpY2UgKi8KPiAr ICAgIFZpcnRJT0RldmljZSAqdmRldjsKPiArCj4gKyAgICAvKiBNYXAgZm9yIHJldHVybmluZyBn dWVzdCdzIGRlc2NyaXB0b3JzICovCj4gKyAgICBWaXJ0UXVldWVFbGVtZW50ICoqcmluZ19pZF9t YXBzOwo+ICsKPiArICAgIC8qIE5leHQgVmlydFF1ZXVlIGVsZW1lbnQgdGhhdCBndWVzdCBtYWRl IGF2YWlsYWJsZSAqLwo+ICsgICAgVmlydFF1ZXVlRWxlbWVudCAqbmV4dF9ndWVzdF9hdmFpbF9l bGVtOwo+ICsKPiArICAgIC8qIE5leHQgaGVhZCB0byBleHBvc2UgdG8gZGV2aWNlICovCj4gKyAg ICB1aW50MTZfdCBhdmFpbF9pZHhfc2hhZG93Owo+ICsKPiArICAgIC8qIE5leHQgZnJlZSBkZXNj cmlwdG9yICovCj4gKyAgICB1aW50MTZfdCBmcmVlX2hlYWQ7Cj4gKwo+ICsgICAgLyogTGFzdCBz ZWVuIHVzZWQgaWR4ICovCj4gKyAgICB1aW50MTZfdCBzaGFkb3dfdXNlZF9pZHg7Cj4gKwo+ICsg ICAgLyogTmV4dCBoZWFkIHRvIGNvbnN1bWUgZnJvbSBkZXZpY2UgKi8KPiArICAgIHVpbnQxNl90 IGxhc3RfdXNlZF9pZHg7Cj4gKwo+ICsgICAgLyogQ2FjaGUgZm9yIHRoZSBleHBvc2VkIG5vdGlm aWNhdGlvbiBmbGFnICovCj4gKyAgICBib29sIG5vdGlmaWNhdGlvbjsKPiAgIH0gVmhvc3RTaGFk b3dWaXJ0cXVldWU7Cj4gICAKPiAgICNkZWZpbmUgSU5WQUxJRF9TVlFfS0lDS19GRCAtMQo+IEBA IC0xNDgsMzAgKzE3NywyOTQgQEAgYm9vbCB2aG9zdF9zdnFfYWNrX2d1ZXN0X2ZlYXR1cmVzKHVp bnQ2NF90IGRldl9mZWF0dXJlcywKPiAgICAgICByZXR1cm4gdHJ1ZTsKPiAgIH0KPiAgIAo+IC0v KiBGb3J3YXJkIGd1ZXN0IG5vdGlmaWNhdGlvbnMgKi8KPiAtc3RhdGljIHZvaWQgdmhvc3RfaGFu ZGxlX2d1ZXN0X2tpY2soRXZlbnROb3RpZmllciAqbikKPiArLyoqCj4gKyAqIE51bWJlciBvZiBk ZXNjcmlwdG9ycyB0aGF0IFNWUSBjYW4gbWFrZSBhdmFpbGFibGUgZnJvbSB0aGUgZ3Vlc3QuCj4g KyAqCj4gKyAqIEBzdnEgICBUaGUgc3ZxCj4gKyAqLwo+ICtzdGF0aWMgdWludDE2X3Qgdmhvc3Rf c3ZxX2F2YWlsYWJsZV9zbG90cyhjb25zdCBWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAqc3ZxKQo+ICAg ewo+IC0gICAgVmhvc3RTaGFkb3dWaXJ0cXVldWUgKnN2cSA9IGNvbnRhaW5lcl9vZihuLCBWaG9z dFNoYWRvd1ZpcnRxdWV1ZSwKPiAtICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgc3ZxX2tpY2spOwo+ICsgICAgcmV0dXJuIHN2cS0+dnJpbmcubnVtIC0gKHN2cS0+ YXZhaWxfaWR4X3NoYWRvdyAtIHN2cS0+c2hhZG93X3VzZWRfaWR4KTsKPiArfQo+ICsKPiArc3Rh dGljIHZvaWQgdmhvc3Rfc3ZxX3NldF9ub3RpZmljYXRpb24oVmhvc3RTaGFkb3dWaXJ0cXVldWUg KnN2cSwgYm9vbCBlbmFibGUpCj4gK3sKPiArICAgIHVpbnQxNl90IG5vdGlmaWNhdGlvbl9mbGFn Owo+ICAgCj4gLSAgICBpZiAodW5saWtlbHkoIWV2ZW50X25vdGlmaWVyX3Rlc3RfYW5kX2NsZWFy KG4pKSkgewo+ICsgICAgaWYgKHN2cS0+bm90aWZpY2F0aW9uID09IGVuYWJsZSkgewo+ICsgICAg ICAgIHJldHVybjsKPiArICAgIH0KPiArCj4gKyAgICBub3RpZmljYXRpb25fZmxhZyA9IGNwdV90 b19sZTE2KFZSSU5HX0FWQUlMX0ZfTk9fSU5URVJSVVBUKTsKPiArCj4gKyAgICBzdnEtPm5vdGlm aWNhdGlvbiA9IGVuYWJsZTsKPiArICAgIGlmIChlbmFibGUpIHsKPiArICAgICAgICBzdnEtPnZy aW5nLmF2YWlsLT5mbGFncyAmPSB+bm90aWZpY2F0aW9uX2ZsYWc7Cj4gKyAgICB9IGVsc2Ugewo+ ICsgICAgICAgIHN2cS0+dnJpbmcuYXZhaWwtPmZsYWdzIHw9IG5vdGlmaWNhdGlvbl9mbGFnOwo+ ICsgICAgfQo+ICt9Cj4gKwo+ICtzdGF0aWMgdm9pZCB2aG9zdF92cmluZ193cml0ZV9kZXNjcyhW aG9zdFNoYWRvd1ZpcnRxdWV1ZSAqc3ZxLAo+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICBjb25zdCBzdHJ1Y3QgaW92ZWMgKmlvdmVjLAo+ICsgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBzaXplX3QgbnVtLCBib29sIG1vcmVfZGVzY3MsIGJvb2wgd3JpdGUp Cj4gK3sKPiArICAgIHVpbnQxNl90IGkgPSBzdnEtPmZyZWVfaGVhZCwgbGFzdCA9IHN2cS0+ZnJl ZV9oZWFkOwo+ICsgICAgdW5zaWduZWQgbjsKPiArICAgIHVpbnQxNl90IGZsYWdzID0gd3JpdGUg PyBjcHVfdG9fbGUxNihWUklOR19ERVNDX0ZfV1JJVEUpIDogMDsKPiArICAgIHZyaW5nX2Rlc2Nf dCAqZGVzY3MgPSBzdnEtPnZyaW5nLmRlc2M7Cj4gKwo+ICsgICAgaWYgKG51bSA9PSAwKSB7Cj4g KyAgICAgICAgcmV0dXJuOwo+ICsgICAgfQo+ICsKPiArICAgIGZvciAobiA9IDA7IG4gPCBudW07 IG4rKykgewo+ICsgICAgICAgIGlmIChtb3JlX2Rlc2NzIHx8IChuICsgMSA8IG51bSkpIHsKPiAr ICAgICAgICAgICAgZGVzY3NbaV0uZmxhZ3MgPSBmbGFncyB8IGNwdV90b19sZTE2KFZSSU5HX0RF U0NfRl9ORVhUKTsKPiArICAgICAgICB9IGVsc2Ugewo+ICsgICAgICAgICAgICBkZXNjc1tpXS5m bGFncyA9IGZsYWdzOwo+ICsgICAgICAgIH0KPiArICAgICAgICBkZXNjc1tpXS5hZGRyID0gY3B1 X3RvX2xlNjQoKGh3YWRkcilpb3ZlY1tuXS5pb3ZfYmFzZSk7Cj4gKyAgICAgICAgZGVzY3NbaV0u bGVuID0gY3B1X3RvX2xlMzIoaW92ZWNbbl0uaW92X2xlbik7Cj4gKwo+ICsgICAgICAgIGxhc3Qg PSBpOwo+ICsgICAgICAgIGkgPSBjcHVfdG9fbGUxNihkZXNjc1tpXS5uZXh0KTsKPiArICAgIH0K PiArCj4gKyAgICBzdnEtPmZyZWVfaGVhZCA9IGxlMTZfdG9fY3B1KGRlc2NzW2xhc3RdLm5leHQp Owo+ICt9Cj4gKwo+ICtzdGF0aWMgdW5zaWduZWQgdmhvc3Rfc3ZxX2FkZF9zcGxpdChWaG9zdFNo YWRvd1ZpcnRxdWV1ZSAqc3ZxLAo+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBWaXJ0UXVldWVFbGVtZW50ICplbGVtKQo+ICt7Cj4gKyAgICBpbnQgaGVhZDsKPiArICAgIHVu c2lnbmVkIGF2YWlsX2lkeDsKPiArICAgIHZyaW5nX2F2YWlsX3QgKmF2YWlsID0gc3ZxLT52cmlu Zy5hdmFpbDsKPiArCj4gKyAgICBoZWFkID0gc3ZxLT5mcmVlX2hlYWQ7Cj4gKwo+ICsgICAgLyog V2UgbmVlZCBzb21lIGRlc2NyaXB0b3JzIGhlcmUgKi8KPiArICAgIGFzc2VydChlbGVtLT5vdXRf bnVtIHx8IGVsZW0tPmluX251bSk7CgoKTG9va3MgbGlrZSB0aGlzIGNvdWxkIGJlIHRyaWdnZXJl ZCBieSBndWVzdCwgd2UgbmVlZCBmYWlsIGluc3RlYWQgYXNzZXJ0IApoZXJlLgoKCj4gKwo+ICsg ICAgdmhvc3RfdnJpbmdfd3JpdGVfZGVzY3Moc3ZxLCBlbGVtLT5vdXRfc2csIGVsZW0tPm91dF9u dW0sCj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICBlbGVtLT5pbl9udW0gPiAwLCBmYWxz ZSk7Cj4gKyAgICB2aG9zdF92cmluZ193cml0ZV9kZXNjcyhzdnEsIGVsZW0tPmluX3NnLCBlbGVt LT5pbl9udW0sIGZhbHNlLCB0cnVlKTsKPiArCj4gKyAgICAvKgo+ICsgICAgICogUHV0IGVudHJ5 IGluIGF2YWlsYWJsZSBhcnJheSAoYnV0IGRvbid0IHVwZGF0ZSBhdmFpbC0+aWR4IHVudGlsIHRo ZXkKPiArICAgICAqIGRvIHN5bmMpLgo+ICsgICAgICovCj4gKyAgICBhdmFpbF9pZHggPSBzdnEt PmF2YWlsX2lkeF9zaGFkb3cgJiAoc3ZxLT52cmluZy5udW0gLSAxKTsKPiArICAgIGF2YWlsLT5y aW5nW2F2YWlsX2lkeF0gPSBjcHVfdG9fbGUxNihoZWFkKTsKPiArICAgIHN2cS0+YXZhaWxfaWR4 X3NoYWRvdysrOwo+ICsKPiArICAgIC8qIFVwZGF0ZSBhdmFpbCBpbmRleCBhZnRlciB0aGUgZGVz Y3JpcHRvciBpcyB3cm90ZSAqLwo+ICsgICAgc21wX3dtYigpOwo+ICsgICAgYXZhaWwtPmlkeCA9 IGNwdV90b19sZTE2KHN2cS0+YXZhaWxfaWR4X3NoYWRvdyk7Cj4gKwo+ICsgICAgcmV0dXJuIGhl YWQ7Cj4gK30KPiArCj4gK3N0YXRpYyB2b2lkIHZob3N0X3N2cV9hZGQoVmhvc3RTaGFkb3dWaXJ0 cXVldWUgKnN2cSwgVmlydFF1ZXVlRWxlbWVudCAqZWxlbSkKPiArewo+ICsgICAgdW5zaWduZWQg cWVtdV9oZWFkID0gdmhvc3Rfc3ZxX2FkZF9zcGxpdChzdnEsIGVsZW0pOwo+ICsKPiArICAgIHN2 cS0+cmluZ19pZF9tYXBzW3FlbXVfaGVhZF0gPSBlbGVtOwo+ICt9Cj4gKwo+ICtzdGF0aWMgdm9p ZCB2aG9zdF9zdnFfa2ljayhWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAqc3ZxKQo+ICt7Cj4gKyAgICAv KiBXZSBuZWVkIHRvIGV4cG9zZSBhdmFpbGFibGUgYXJyYXkgZW50cmllcyBiZWZvcmUgY2hlY2tp bmcgdXNlZCBmbGFncyAqLwo+ICsgICAgc21wX21iKCk7Cj4gKyAgICBpZiAoc3ZxLT52cmluZy51 c2VkLT5mbGFncyAmIFZSSU5HX1VTRURfRl9OT19OT1RJRlkpIHsKPiAgICAgICAgICAgcmV0dXJu Owo+ICAgICAgIH0KPiAgIAo+ICAgICAgIGV2ZW50X25vdGlmaWVyX3NldCgmc3ZxLT5oZGV2X2tp Y2spOwo+ICAgfQo+ICAgCj4gLS8qIEZvcndhcmQgdmhvc3Qgbm90aWZpY2F0aW9ucyAqLwo+ICsv KioKPiArICogRm9yd2FyZCBhdmFpbGFibGUgYnVmZmVycy4KPiArICoKPiArICogQHN2cSBTaGFk b3cgVmlydFF1ZXVlCj4gKyAqCj4gKyAqIE5vdGUgdGhhdCB0aGlzIGZ1bmN0aW9uIGRvZXMgbm90 IGd1YXJhbnRlZSB0aGF0IGFsbCBndWVzdCdzIGF2YWlsYWJsZQo+ICsgKiBidWZmZXJzIGFyZSBh dmFpbGFibGUgdG8gdGhlIGRldmljZSBpbiBTVlEgYXZhaWwgcmluZy4gVGhlIGd1ZXN0IG1heSBo YXZlCj4gKyAqIGV4cG9zZWQgYSBHUEEgLyBHSU9WQSBjb25naXVvdXMgYnVmZmVyLCBidXQgaXQg bWF5IG5vdCBiZSBjb250aWd1b3VzIGluIHFlbXUKPiArICogdmFkZHIuCj4gKyAqCj4gKyAqIElm IHRoYXQgaGFwcGVucywgZ3Vlc3QncyBraWNrIG5vdGlmaWNhdGlvbnMgd2lsbCBiZSBkaXNhYmxl ZCB1bnRpbCBkZXZpY2UKPiArICogbWFrZXMgc29tZSBidWZmZXJzIHVzZWQuCj4gKyAqLwo+ICtz dGF0aWMgdm9pZCB2aG9zdF9oYW5kbGVfZ3Vlc3Rfa2ljayhWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAq c3ZxKQo+ICt7Cj4gKyAgICAvKiBDbGVhciBldmVudCBub3RpZmllciAqLwo+ICsgICAgZXZlbnRf bm90aWZpZXJfdGVzdF9hbmRfY2xlYXIoJnN2cS0+c3ZxX2tpY2spOwo+ICsKPiArICAgIC8qIE1h a2UgYXZhaWxhYmxlIGFzIG1hbnkgYnVmZmVycyBhcyBwb3NzaWJsZSAqLwo+ICsgICAgZG8gewo+ ICsgICAgICAgIGlmICh2aXJ0aW9fcXVldWVfZ2V0X25vdGlmaWNhdGlvbihzdnEtPnZxKSkgewo+ ICsgICAgICAgICAgICB2aXJ0aW9fcXVldWVfc2V0X25vdGlmaWNhdGlvbihzdnEtPnZxLCBmYWxz ZSk7CgoKVGhpcyBsb29rcyBsaWtlIGFuIG9wdGltaXphdGlvbiB0aGUgc2hvdWxkIGJlbG9uZyB0 byAKdmlydGlvX3F1ZXVlX3NldF9ub3RpZmljYXRpb24oKSBpdHNlbGYuCgoKPiArICAgICAgICB9 Cj4gKwo+ICsgICAgICAgIHdoaWxlICh0cnVlKSB7Cj4gKyAgICAgICAgICAgIFZpcnRRdWV1ZUVs ZW1lbnQgKmVsZW07Cj4gKwo+ICsgICAgICAgICAgICBpZiAoc3ZxLT5uZXh0X2d1ZXN0X2F2YWls X2VsZW0pIHsKPiArICAgICAgICAgICAgICAgIGVsZW0gPSBnX3N0ZWFsX3BvaW50ZXIoJnN2cS0+ bmV4dF9ndWVzdF9hdmFpbF9lbGVtKTsKPiArICAgICAgICAgICAgfSBlbHNlIHsKPiArICAgICAg ICAgICAgICAgIGVsZW0gPSB2aXJ0cXVldWVfcG9wKHN2cS0+dnEsIHNpemVvZigqZWxlbSkpOwo+ ICsgICAgICAgICAgICB9Cj4gKwo+ICsgICAgICAgICAgICBpZiAoIWVsZW0pIHsKPiArICAgICAg ICAgICAgICAgIGJyZWFrOwo+ICsgICAgICAgICAgICB9Cj4gKwo+ICsgICAgICAgICAgICBpZiAo ZWxlbS0+b3V0X251bSArIGVsZW0tPmluX251bSA+Cj4gKyAgICAgICAgICAgICAgICB2aG9zdF9z dnFfYXZhaWxhYmxlX3Nsb3RzKHN2cSkpIHsKPiArICAgICAgICAgICAgICAgIC8qCj4gKyAgICAg ICAgICAgICAgICAgKiBUaGlzIGNvbmRpdGlvbiBpcyBwb3NzaWJsZSBzaW5jZSBhIGNvbnRpZ3Vv dXMgYnVmZmVyIGluIEdQQQo+ICsgICAgICAgICAgICAgICAgICogZG9lcyBub3QgaW1wbHkgYSBj b250aWd1b3VzIGJ1ZmZlciBpbiBxZW11J3MgVkEKPiArICAgICAgICAgICAgICAgICAqIHNjYXR0 ZXItZ2F0aGVyIHNlZ21lbnRzLiBJZiB0aGF0IGhhcHBlbiwgdGhlIGJ1ZmZlciBleHBvc2VkCj4g KyAgICAgICAgICAgICAgICAgKiB0byB0aGUgZGV2aWNlIG5lZWRzIHRvIGJlIGEgY2hhaW4gb2Yg ZGVzY3JpcHRvcnMgYXQgdGhpcwo+ICsgICAgICAgICAgICAgICAgICogbW9tZW50Lgo+ICsgICAg ICAgICAgICAgICAgICoKPiArICAgICAgICAgICAgICAgICAqIFNWUSBjYW5ub3QgaG9sZCBtb3Jl IGF2YWlsYWJsZSBidWZmZXJzIGlmIHdlIGFyZSBoZXJlOgo+ICsgICAgICAgICAgICAgICAgICog cXVldWUgdGhlIGN1cnJlbnQgZ3Vlc3QgZGVzY3JpcHRvciBhbmQgaWdub3JlIGZ1cnRoZXIga2lj a3MKPiArICAgICAgICAgICAgICAgICAqIHVudGlsIHNvbWUgZWxlbWVudHMgYXJlIHVzZWQuCj4g KyAgICAgICAgICAgICAgICAgKi8KPiArICAgICAgICAgICAgICAgIHN2cS0+bmV4dF9ndWVzdF9h dmFpbF9lbGVtID0gZWxlbTsKPiArICAgICAgICAgICAgICAgIHJldHVybjsKPiArICAgICAgICAg ICAgfQo+ICsKPiArICAgICAgICAgICAgdmhvc3Rfc3ZxX2FkZChzdnEsIGVsZW0pOwo+ICsgICAg ICAgICAgICB2aG9zdF9zdnFfa2ljayhzdnEpOwo+ICsgICAgICAgIH0KPiArCj4gKyAgICAgICAg dmlydGlvX3F1ZXVlX3NldF9ub3RpZmljYXRpb24oc3ZxLT52cSwgdHJ1ZSk7Cj4gKyAgICB9IHdo aWxlICghdmlydGlvX3F1ZXVlX2VtcHR5KHN2cS0+dnEpKTsKPiArfQo+ICsKPiArLyoqCj4gKyAq IEhhbmRsZSBndWVzdCdzIGtpY2suCj4gKyAqCj4gKyAqIEBuIGd1ZXN0IGtpY2sgZXZlbnQgbm90 aWZpZXIsIHRoZSBvbmUgdGhhdCBndWVzdCBzZXQgdG8gbm90aWZ5IHN2cS4KPiArICovCj4gK3N0 YXRpYyB2b2lkIHZob3N0X2hhbmRsZV9ndWVzdF9raWNrX25vdGlmaWVyKEV2ZW50Tm90aWZpZXIg Km4pCj4gK3sKPiArICAgIFZob3N0U2hhZG93VmlydHF1ZXVlICpzdnEgPSBjb250YWluZXJfb2Yo biwgVmhvc3RTaGFkb3dWaXJ0cXVldWUsCj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIHN2cV9raWNrKTsKPiArICAgIHZob3N0X2hhbmRsZV9ndWVzdF9raWNr KHN2cSk7Cj4gK30KPiArCj4gK3N0YXRpYyBib29sIHZob3N0X3N2cV9tb3JlX3VzZWQoVmhvc3RT aGFkb3dWaXJ0cXVldWUgKnN2cSkKPiArewo+ICsgICAgaWYgKHN2cS0+bGFzdF91c2VkX2lkeCAh PSBzdnEtPnNoYWRvd191c2VkX2lkeCkgewo+ICsgICAgICAgIHJldHVybiB0cnVlOwo+ICsgICAg fQo+ICsKPiArICAgIHN2cS0+c2hhZG93X3VzZWRfaWR4ID0gY3B1X3RvX2xlMTYoc3ZxLT52cmlu Zy51c2VkLT5pZHgpOwo+ICsKPiArICAgIHJldHVybiBzdnEtPmxhc3RfdXNlZF9pZHggIT0gc3Zx LT5zaGFkb3dfdXNlZF9pZHg7Cj4gK30KPiArCj4gK3N0YXRpYyBWaXJ0UXVldWVFbGVtZW50ICp2 aG9zdF9zdnFfZ2V0X2J1ZihWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAqc3ZxKQo+ICt7Cj4gKyAgICB2 cmluZ19kZXNjX3QgKmRlc2NzID0gc3ZxLT52cmluZy5kZXNjOwo+ICsgICAgY29uc3QgdnJpbmdf dXNlZF90ICp1c2VkID0gc3ZxLT52cmluZy51c2VkOwo+ICsgICAgdnJpbmdfdXNlZF9lbGVtX3Qg dXNlZF9lbGVtOwo+ICsgICAgdWludDE2X3QgbGFzdF91c2VkOwo+ICsKPiArICAgIGlmICghdmhv c3Rfc3ZxX21vcmVfdXNlZChzdnEpKSB7Cj4gKyAgICAgICAgcmV0dXJuIE5VTEw7Cj4gKyAgICB9 Cj4gKwo+ICsgICAgLyogT25seSBnZXQgdXNlZCBhcnJheSBlbnRyaWVzIGFmdGVyIHRoZXkgaGF2 ZSBiZWVuIGV4cG9zZWQgYnkgZGV2ICovCj4gKyAgICBzbXBfcm1iKCk7Cj4gKyAgICBsYXN0X3Vz ZWQgPSBzdnEtPmxhc3RfdXNlZF9pZHggJiAoc3ZxLT52cmluZy5udW0gLSAxKTsKPiArICAgIHVz ZWRfZWxlbS5pZCA9IGxlMzJfdG9fY3B1KHVzZWQtPnJpbmdbbGFzdF91c2VkXS5pZCk7Cj4gKyAg ICB1c2VkX2VsZW0ubGVuID0gbGUzMl90b19jcHUodXNlZC0+cmluZ1tsYXN0X3VzZWRdLmxlbik7 Cj4gKwo+ICsgICAgc3ZxLT5sYXN0X3VzZWRfaWR4Kys7Cj4gKyAgICBpZiAodW5saWtlbHkodXNl ZF9lbGVtLmlkID49IHN2cS0+dnJpbmcubnVtKSkgewo+ICsgICAgICAgIGVycm9yX3JlcG9ydCgi RGV2aWNlICVzIHNheXMgaW5kZXggJXUgaXMgdXNlZCIsIHN2cS0+dmRldi0+bmFtZSwKPiArICAg ICAgICAgICAgICAgICAgICAgdXNlZF9lbGVtLmlkKTsKPiArICAgICAgICByZXR1cm4gTlVMTDsK PiArICAgIH0KPiArCj4gKyAgICBpZiAodW5saWtlbHkoIXN2cS0+cmluZ19pZF9tYXBzW3VzZWRf ZWxlbS5pZF0pKSB7Cj4gKyAgICAgICAgZXJyb3JfcmVwb3J0KAo+ICsgICAgICAgICAgICAiRGV2 aWNlICVzIHNheXMgaW5kZXggJXUgaXMgdXNlZCwgYnV0IGl0IHdhcyBub3QgYXZhaWxhYmxlIiwK PiArICAgICAgICAgICAgc3ZxLT52ZGV2LT5uYW1lLCB1c2VkX2VsZW0uaWQpOwo+ICsgICAgICAg IHJldHVybiBOVUxMOwo+ICsgICAgfQo+ICsKPiArICAgIGRlc2NzW3VzZWRfZWxlbS5pZF0ubmV4 dCA9IHN2cS0+ZnJlZV9oZWFkOwo+ICsgICAgc3ZxLT5mcmVlX2hlYWQgPSB1c2VkX2VsZW0uaWQ7 Cj4gKwo+ICsgICAgc3ZxLT5yaW5nX2lkX21hcHNbdXNlZF9lbGVtLmlkXS0+bGVuID0gdXNlZF9l bGVtLmxlbjsKPiArICAgIHJldHVybiBnX3N0ZWFsX3BvaW50ZXIoJnN2cS0+cmluZ19pZF9tYXBz W3VzZWRfZWxlbS5pZF0pOwo+ICt9Cj4gKwo+ICtzdGF0aWMgdm9pZCB2aG9zdF9zdnFfZmx1c2go Vmhvc3RTaGFkb3dWaXJ0cXVldWUgKnN2cSwKPiArICAgICAgICAgICAgICAgICAgICAgICAgICAg IGJvb2wgY2hlY2tfZm9yX2F2YWlsX3F1ZXVlKQo+ICt7Cj4gKyAgICBWaXJ0UXVldWUgKnZxID0g c3ZxLT52cTsKPiArCj4gKyAgICAvKiBNYWtlIGFzIG1hbnkgYnVmZmVycyBhcyBwb3NzaWJsZSB1 c2VkLiAqLwo+ICsgICAgZG8gewo+ICsgICAgICAgIHVuc2lnbmVkIGkgPSAwOwo+ICsKPiArICAg ICAgICB2aG9zdF9zdnFfc2V0X25vdGlmaWNhdGlvbihzdnEsIGZhbHNlKTsKPiArICAgICAgICB3 aGlsZSAodHJ1ZSkgewo+ICsgICAgICAgICAgICBnX2F1dG9mcmVlIFZpcnRRdWV1ZUVsZW1lbnQg KmVsZW0gPSB2aG9zdF9zdnFfZ2V0X2J1ZihzdnEpOwo+ICsgICAgICAgICAgICBpZiAoIWVsZW0p IHsKPiArICAgICAgICAgICAgICAgIGJyZWFrOwo+ICsgICAgICAgICAgICB9Cj4gKwo+ICsgICAg ICAgICAgICBpZiAodW5saWtlbHkoaSA+PSBzdnEtPnZyaW5nLm51bSkpIHsKPiArICAgICAgICAg ICAgICAgIHZpcnRpb19lcnJvcihzdnEtPnZkZXYsCj4gKyAgICAgICAgICAgICAgICAgICAgICAg ICAiTW9yZSB0aGFuICV1IHVzZWQgYnVmZmVycyBvYnRhaW5lZCBpbiBhICV1IHNpemUgU1ZRIiwK PiArICAgICAgICAgICAgICAgICAgICAgICAgIGksIHN2cS0+dnJpbmcubnVtKTsKPiArICAgICAg ICAgICAgICAgIHZpcnRxdWV1ZV9maWxsKHZxLCBlbGVtLCBlbGVtLT5sZW4sIGkpOwo+ICsgICAg ICAgICAgICAgICAgdmlydHF1ZXVlX2ZsdXNoKHZxLCBpKTsKCgpMZXQncyBzaW1wbHkgdXNlIHZp cnRxdWV1ZV9wdXNoKCkgaGVyZT8KCgo+ICsgICAgICAgICAgICAgICAgaSA9IDA7CgoKRG8gd2Ug bmVlZCB0byBiYWlsIG91dCBoZXJlPwoKCj4gKyAgICAgICAgICAgIH0KPiArICAgICAgICAgICAg dmlydHF1ZXVlX2ZpbGwodnEsIGVsZW0sIGVsZW0tPmxlbiwgaSsrKTsKPiArICAgICAgICB9Cj4g Kwo+ICsgICAgICAgIHZpcnRxdWV1ZV9mbHVzaCh2cSwgaSk7Cj4gKyAgICAgICAgZXZlbnRfbm90 aWZpZXJfc2V0KCZzdnEtPnN2cV9jYWxsKTsKPiArCj4gKyAgICAgICAgaWYgKGNoZWNrX2Zvcl9h dmFpbF9xdWV1ZSAmJiBzdnEtPm5leHRfZ3Vlc3RfYXZhaWxfZWxlbSkgewo+ICsgICAgICAgICAg ICAvKgo+ICsgICAgICAgICAgICAgKiBBdmFpbCByaW5nIHdhcyBmdWxsIHdoZW4gdmhvc3Rfc3Zx X2ZsdXNoIHdhcyBjYWxsZWQsIHNvIGl0J3MgYQo+ICsgICAgICAgICAgICAgKiBnb29kIG1vbWVu dCB0byBtYWtlIG1vcmUgZGVzY3JpcHRvcnMgYXZhaWxhYmxlIGlmIHBvc3NpYmxlCj4gKyAgICAg ICAgICAgICAqLwo+ICsgICAgICAgICAgICB2aG9zdF9oYW5kbGVfZ3Vlc3Rfa2ljayhzdnEpOwoK CklzIHRoZXJlIGJldHRlciB0byBoYXZlIGEgc2ltaWxhciBjaGVjayBhcyB2aG9zdF9oYW5kbGVf Z3Vlc3Rfa2ljaygpIGRpZD8KCiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGlmIChlbGVtLT5vdXRf bnVtICsgZWxlbS0+aW5fbnVtID4KIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB2aG9z dF9zdnFfYXZhaWxhYmxlX3Nsb3RzKHN2cSkpIHsKCgo+ICsgICAgICAgIH0KPiArCj4gKyAgICAg ICAgdmhvc3Rfc3ZxX3NldF9ub3RpZmljYXRpb24oc3ZxLCB0cnVlKTsKCgpBIG1iKCkgaXMgbmVl ZGVkIGhlcmU/IE90aGVyd2lzZSB3ZSBtYXkgbG9zdCBhIGNhbGwgaGVyZSAod2hlcmUgCnZob3N0 X3N2cV9tb3JlX3VzZWQoKSBpcyBydW4gYmVmb3JlIHZob3N0X3N2cV9zZXRfbm90aWZpY2F0aW9u KCkpLgoKCj4gKyAgICB9IHdoaWxlICh2aG9zdF9zdnFfbW9yZV91c2VkKHN2cSkpOwo+ICt9Cj4g Kwo+ICsvKioKPiArICogRm9yd2FyZCB1c2VkIGJ1ZmZlcnMuCj4gKyAqCj4gKyAqIEBuIGhkZXYg Y2FsbCBldmVudCBub3RpZmllciwgdGhlIG9uZSB0aGF0IGRldmljZSBzZXQgdG8gbm90aWZ5IHN2 cS4KPiArICoKPiArICogTm90ZSB0aGF0IHdlIGFyZSBub3QgbWFraW5nIGFueSBidWZmZXJzIGF2 YWlsYWJsZSBpbiB0aGUgbG9vcCwgdGhlcmUgaXMgbm8KPiArICogd2F5IHRoYXQgaXQgcnVucyBt b3JlIHRoYW4gdmlydHF1ZXVlIHNpemUgdGltZXMuCj4gKyAqLwo+ICAgc3RhdGljIHZvaWQgdmhv c3Rfc3ZxX2hhbmRsZV9jYWxsKEV2ZW50Tm90aWZpZXIgKm4pCj4gICB7Cj4gICAgICAgVmhvc3RT aGFkb3dWaXJ0cXVldWUgKnN2cSA9IGNvbnRhaW5lcl9vZihuLCBWaG9zdFNoYWRvd1ZpcnRxdWV1 ZSwKPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGhkZXZf Y2FsbCk7Cj4gICAKPiAtICAgIGlmICh1bmxpa2VseSghZXZlbnRfbm90aWZpZXJfdGVzdF9hbmRf Y2xlYXIobikpKSB7Cj4gLSAgICAgICAgcmV0dXJuOwo+IC0gICAgfQo+ICsgICAgLyogQ2xlYXIg ZXZlbnQgbm90aWZpZXIgKi8KPiArICAgIGV2ZW50X25vdGlmaWVyX3Rlc3RfYW5kX2NsZWFyKG4p OwoKCkFueSByZWFzb24gdGhhdCB3ZSByZW1vdmUgdGhlIGFib3ZlIGNoZWNrPwoKCj4gICAKPiAt ICAgIGV2ZW50X25vdGlmaWVyX3NldCgmc3ZxLT5zdnFfY2FsbCk7Cj4gKyAgICB2aG9zdF9zdnFf Zmx1c2goc3ZxLCB0cnVlKTsKPiAgIH0KPiAgIAo+ICAgLyoqCj4gQEAgLTI1OCwxMyArNTUxLDM4 IEBAIHZvaWQgdmhvc3Rfc3ZxX3NldF9zdnFfa2lja19mZChWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAq c3ZxLCBpbnQgc3ZxX2tpY2tfZmQpCj4gICAgICAgICogbmVlZCB0byBleHBsaWNpdGVseSBjaGVj ayBmb3IgdGhlbS4KPiAgICAgICAgKi8KPiAgICAgICBldmVudF9ub3RpZmllcl9pbml0X2ZkKCZz dnEtPnN2cV9raWNrLCBzdnFfa2lja19mZCk7Cj4gLSAgICBldmVudF9ub3RpZmllcl9zZXRfaGFu ZGxlcigmc3ZxLT5zdnFfa2ljaywgdmhvc3RfaGFuZGxlX2d1ZXN0X2tpY2spOwo+ICsgICAgZXZl bnRfbm90aWZpZXJfc2V0X2hhbmRsZXIoJnN2cS0+c3ZxX2tpY2ssCj4gKyAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICB2aG9zdF9oYW5kbGVfZ3Vlc3Rfa2lja19ub3RpZmllcik7Cj4gICAK PiAgICAgICBpZiAoIWNoZWNrX29sZCB8fCBldmVudF9ub3RpZmllcl90ZXN0X2FuZF9jbGVhcigm dG1wKSkgewo+ICAgICAgICAgICBldmVudF9ub3RpZmllcl9zZXQoJnN2cS0+aGRldl9raWNrKTsK PiAgICAgICB9Cj4gICB9Cj4gICAKPiArLyoqCj4gKyAqIFN0YXJ0IHNoYWRvdyB2aXJ0cXVldWUg b3BlcmF0aW9uLgo+ICsgKgo+ICsgKiBAc3ZxIFNoYWRvdyBWaXJ0cXVldWUKPiArICogQHZkZXYg ICAgICAgIFZpcnRJTyBkZXZpY2UKPiArICogQHZxICAgICAgICAgIFZpcnRxdWV1ZSB0byBzaGFk b3cKPiArICovCj4gK3ZvaWQgdmhvc3Rfc3ZxX3N0YXJ0KFZob3N0U2hhZG93VmlydHF1ZXVlICpz dnEsIFZpcnRJT0RldmljZSAqdmRldiwKPiArICAgICAgICAgICAgICAgICAgICAgVmlydFF1ZXVl ICp2cSkKPiArewo+ICsgICAgc3ZxLT5uZXh0X2d1ZXN0X2F2YWlsX2VsZW0gPSBOVUxMOwo+ICsg ICAgc3ZxLT5hdmFpbF9pZHhfc2hhZG93ID0gMDsKPiArICAgIHN2cS0+c2hhZG93X3VzZWRfaWR4 ID0gMDsKPiArICAgIHN2cS0+bGFzdF91c2VkX2lkeCA9IDA7Cj4gKyAgICBzdnEtPnZkZXYgPSB2 ZGV2Owo+ICsgICAgc3ZxLT52cSA9IHZxOwo+ICsKPiArICAgIG1lbXNldChzdnEtPnZyaW5nLmF2 YWlsLCAwLCBzaXplb2YoKnN2cS0+dnJpbmcuYXZhaWwpKTsKPiArICAgIG1lbXNldChzdnEtPnZy aW5nLnVzZWQsIDAsIHNpemVvZigqc3ZxLT52cmluZy5hdmFpbCkpOwo+ICsgICAgZm9yICh1bnNp Z25lZCBpID0gMDsgaSA8IHN2cS0+dnJpbmcubnVtIC0gMTsgaSsrKSB7Cj4gKyAgICAgICAgc3Zx LT52cmluZy5kZXNjW2ldLm5leHQgPSBjcHVfdG9fbGUxNihpICsgMSk7Cj4gKyAgICB9Cj4gK30K PiArCj4gICAvKioKPiAgICAqIFN0b3Agc2hhZG93IHZpcnRxdWV1ZSBvcGVyYXRpb24uCj4gICAg KiBAc3ZxIFNoYWRvdyBWaXJ0cXVldWUKPiBAQCAtMjcyLDYgKzU5MCwyOCBAQCB2b2lkIHZob3N0 X3N2cV9zZXRfc3ZxX2tpY2tfZmQoVmhvc3RTaGFkb3dWaXJ0cXVldWUgKnN2cSwgaW50IHN2cV9r aWNrX2ZkKQo+ICAgdm9pZCB2aG9zdF9zdnFfc3RvcChWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAqc3Zx KQo+ICAgewo+ICAgICAgIGV2ZW50X25vdGlmaWVyX3NldF9oYW5kbGVyKCZzdnEtPnN2cV9raWNr LCBOVUxMKTsKPiArICAgIGdfYXV0b2ZyZWUgVmlydFF1ZXVlRWxlbWVudCAqbmV4dF9hdmFpbF9l bGVtID0gTlVMTDsKPiArCj4gKyAgICBpZiAoIXN2cS0+dnEpIHsKPiArICAgICAgICByZXR1cm47 Cj4gKyAgICB9Cj4gKwo+ICsgICAgLyogU2VuZCBhbGwgcGVuZGluZyB1c2VkIGRlc2NyaXB0b3Jz IHRvIGd1ZXN0ICovCj4gKyAgICB2aG9zdF9zdnFfZmx1c2goc3ZxLCBmYWxzZSk7Cj4gKwo+ICsg ICAgZm9yICh1bnNpZ25lZCBpID0gMDsgaSA8IHN2cS0+dnJpbmcubnVtOyArK2kpIHsKPiArICAg ICAgICBnX2F1dG9mcmVlIFZpcnRRdWV1ZUVsZW1lbnQgKmVsZW0gPSBOVUxMOwo+ICsgICAgICAg IGVsZW0gPSBnX3N0ZWFsX3BvaW50ZXIoJnN2cS0+cmluZ19pZF9tYXBzW2ldKTsKPiArICAgICAg ICBpZiAoZWxlbSkgewo+ICsgICAgICAgICAgICB2aXJ0cXVldWVfZGV0YWNoX2VsZW1lbnQoc3Zx LT52cSwgZWxlbSwgZWxlbS0+bGVuKTsKPiArICAgICAgICB9Cj4gKyAgICB9Cj4gKwo+ICsgICAg bmV4dF9hdmFpbF9lbGVtID0gZ19zdGVhbF9wb2ludGVyKCZzdnEtPm5leHRfZ3Vlc3RfYXZhaWxf ZWxlbSk7Cj4gKyAgICBpZiAobmV4dF9hdmFpbF9lbGVtKSB7Cj4gKyAgICAgICAgdmlydHF1ZXVl X2RldGFjaF9lbGVtZW50KHN2cS0+dnEsIG5leHRfYXZhaWxfZWxlbSwKPiArICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgbmV4dF9hdmFpbF9lbGVtLT5sZW4pOwo+ICsgICAgfQo+ICAg fQo+ICAgCj4gICAvKioKPiBAQCAtMzE2LDcgKzY1Niw3IEBAIFZob3N0U2hhZG93VmlydHF1ZXVl ICp2aG9zdF9zdnFfbmV3KHVpbnQxNl90IHFzaXplKQo+ICAgICAgIG1lbXNldChzdnEtPnZyaW5n LmRlc2MsIDAsIGRyaXZlcl9zaXplKTsKPiAgICAgICBzdnEtPnZyaW5nLnVzZWQgPSBxZW11X21l bWFsaWduKHFlbXVfcmVhbF9ob3N0X3BhZ2Vfc2l6ZSwgZGV2aWNlX3NpemUpOwo+ICAgICAgIG1l bXNldChzdnEtPnZyaW5nLnVzZWQsIDAsIGRldmljZV9zaXplKTsKPiAtCj4gKyAgICBzdnEtPnJp bmdfaWRfbWFwcyA9IGdfbmV3MChWaXJ0UXVldWVFbGVtZW50ICosIHFzaXplKTsKPiAgICAgICBl dmVudF9ub3RpZmllcl9zZXRfaGFuZGxlcigmc3ZxLT5oZGV2X2NhbGwsIHZob3N0X3N2cV9oYW5k bGVfY2FsbCk7Cj4gICAgICAgcmV0dXJuIGdfc3RlYWxfcG9pbnRlcigmc3ZxKTsKPiAgIAo+IEBA IC0zMzUsNiArNjc1LDcgQEAgdm9pZCB2aG9zdF9zdnFfZnJlZShWaG9zdFNoYWRvd1ZpcnRxdWV1 ZSAqdnEpCj4gICAgICAgZXZlbnRfbm90aWZpZXJfY2xlYW51cCgmdnEtPmhkZXZfa2ljayk7Cj4g ICAgICAgZXZlbnRfbm90aWZpZXJfc2V0X2hhbmRsZXIoJnZxLT5oZGV2X2NhbGwsIE5VTEwpOwo+ ICAgICAgIGV2ZW50X25vdGlmaWVyX2NsZWFudXAoJnZxLT5oZGV2X2NhbGwpOwo+ICsgICAgZ19m cmVlKHZxLT5yaW5nX2lkX21hcHMpOwo+ICAgICAgIHFlbXVfdmZyZWUodnEtPnZyaW5nLmRlc2Mp Owo+ICAgICAgIHFlbXVfdmZyZWUodnEtPnZyaW5nLnVzZWQpOwo+ICAgICAgIGdfZnJlZSh2cSk7 Cj4gZGlmZiAtLWdpdCBhL2h3L3ZpcnRpby92aG9zdC12ZHBhLmMgYi9ody92aXJ0aW8vdmhvc3Qt dmRwYS5jCj4gaW5kZXggNTNlMTRiYWZhMC4uMGU1YzAwZWQ3ZSAxMDA2NDQKPiAtLS0gYS9ody92 aXJ0aW8vdmhvc3QtdmRwYS5jCj4gKysrIGIvaHcvdmlydGlvL3Zob3N0LXZkcGEuYwo+IEBAIC03 NTIsOSArNzUyLDkgQEAgc3RhdGljIGludCB2aG9zdF92ZHBhX3NldF92cmluZ19jYWxsKHN0cnVj dCB2aG9zdF9kZXYgKmRldiwKPiAgICAqIE5vdGUgdGhhdCB0aGlzIGZ1bmN0aW9uIGRvZXMgbm90 IHJld2luZCBraWNrIGZpbGUgZGVzY3JpcHRvciBpZiBjYW5ub3Qgc2V0Cj4gICAgKiBjYWxsIG9u ZS4KPiAgICAqLwo+IC1zdGF0aWMgYm9vbCB2aG9zdF92ZHBhX3N2cV9zZXR1cChzdHJ1Y3Qgdmhv c3RfZGV2ICpkZXYsCj4gLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgVmhvc3RTaGFk b3dWaXJ0cXVldWUgKnN2cSwKPiAtICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNp Z25lZCBpZHgpCj4gK3N0YXRpYyBpbnQgdmhvc3RfdmRwYV9zdnFfc2V0X2ZkcyhzdHJ1Y3Qgdmhv c3RfZGV2ICpkZXYsCj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBWaG9zdFNo YWRvd1ZpcnRxdWV1ZSAqc3ZxLAo+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg dW5zaWduZWQgaWR4KQo+ICAgewo+ICAgICAgIHN0cnVjdCB2aG9zdF92cmluZ19maWxlIGZpbGUg PSB7Cj4gICAgICAgICAgIC5pbmRleCA9IGRldi0+dnFfaW5kZXggKyBpZHgsCj4gQEAgLTc2Nyw3 ICs3NjcsNyBAQCBzdGF0aWMgYm9vbCB2aG9zdF92ZHBhX3N2cV9zZXR1cChzdHJ1Y3Qgdmhvc3Rf ZGV2ICpkZXYsCj4gICAgICAgciA9IHZob3N0X3ZkcGFfc2V0X3ZyaW5nX2Rldl9raWNrKGRldiwg JmZpbGUpOwo+ICAgICAgIGlmICh1bmxpa2VseShyICE9IDApKSB7Cj4gICAgICAgICAgIGVycm9y X3JlcG9ydCgiQ2FuJ3Qgc2V0IGRldmljZSBraWNrIGZkICglZCkiLCAtcik7Cj4gLSAgICAgICAg cmV0dXJuIGZhbHNlOwo+ICsgICAgICAgIHJldHVybiByOwo+ICAgICAgIH0KPiAgIAo+ICAgICAg IGV2ZW50X25vdGlmaWVyID0gdmhvc3Rfc3ZxX2dldF9zdnFfY2FsbF9ub3RpZmllcihzdnEpOwo+ IEBAIC03NzcsNiArNzc3LDk5IEBAIHN0YXRpYyBib29sIHZob3N0X3ZkcGFfc3ZxX3NldHVwKHN0 cnVjdCB2aG9zdF9kZXYgKmRldiwKPiAgICAgICAgICAgZXJyb3JfcmVwb3J0KCJDYW4ndCBzZXQg ZGV2aWNlIGNhbGwgZmQgKCVkKSIsIC1yKTsKPiAgICAgICB9Cj4gICAKPiArICAgIHJldHVybiBy Owo+ICt9Cj4gKwo+ICsvKioKPiArICogVW5tYXAgU1ZRIGFyZWEgaW4gdGhlIGRldmljZQo+ICsg Ki8KPiArc3RhdGljIGJvb2wgdmhvc3RfdmRwYV9zdnFfdW5tYXBfcmluZyhzdHJ1Y3Qgdmhvc3Rf dmRwYSAqdiwgaHdhZGRyIGlvdmEsCj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgaHdhZGRyIHNpemUpCj4gK3sKPiArICAgIGludCByOwo+ICsKPiArICAgIHNpemUgPSBS T1VORF9VUChzaXplLCBxZW11X3JlYWxfaG9zdF9wYWdlX3NpemUpOwo+ICsgICAgciA9IHZob3N0 X3ZkcGFfZG1hX3VubWFwKHYsIGlvdmEsIHNpemUpOwo+ICsgICAgcmV0dXJuIHIgPT0gMDsKPiAr fQo+ICsKPiArc3RhdGljIGJvb2wgdmhvc3RfdmRwYV9zdnFfdW5tYXBfcmluZ3Moc3RydWN0IHZo b3N0X2RldiAqZGV2LAo+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBj b25zdCBWaG9zdFNoYWRvd1ZpcnRxdWV1ZSAqc3ZxKQo+ICt7Cj4gKyAgICBzdHJ1Y3Qgdmhvc3Rf dmRwYSAqdiA9IGRldi0+b3BhcXVlOwo+ICsgICAgc3RydWN0IHZob3N0X3ZyaW5nX2FkZHIgc3Zx X2FkZHI7Cj4gKyAgICBzaXplX3QgZGV2aWNlX3NpemUgPSB2aG9zdF9zdnFfZGV2aWNlX2FyZWFf c2l6ZShzdnEpOwo+ICsgICAgc2l6ZV90IGRyaXZlcl9zaXplID0gdmhvc3Rfc3ZxX2RyaXZlcl9h cmVhX3NpemUoc3ZxKTsKPiArICAgIGJvb2wgb2s7Cj4gKwo+ICsgICAgdmhvc3Rfc3ZxX2dldF92 cmluZ19hZGRyKHN2cSwgJnN2cV9hZGRyKTsKPiArCj4gKyAgICBvayA9IHZob3N0X3ZkcGFfc3Zx X3VubWFwX3Jpbmcodiwgc3ZxX2FkZHIuZGVzY191c2VyX2FkZHIsIGRyaXZlcl9zaXplKTsKPiAr ICAgIGlmICh1bmxpa2VseSghb2spKSB7Cj4gKyAgICAgICAgcmV0dXJuIGZhbHNlOwo+ICsgICAg fQo+ICsKPiArICAgIHJldHVybiB2aG9zdF92ZHBhX3N2cV91bm1hcF9yaW5nKHYsIHN2cV9hZGRy LnVzZWRfdXNlcl9hZGRyLCBkZXZpY2Vfc2l6ZSk7Cj4gK30KPiArCj4gKy8qKgo+ICsgKiBNYXAg c2hhZG93IHZpcnRxdWV1ZSByaW5ncyBpbiBkZXZpY2UKPiArICoKPiArICogQGRldiAgIFRoZSB2 aG9zdCBkZXZpY2UKPiArICogQHN2cSAgIFRoZSBzaGFkb3cgdmlydHF1ZXVlCj4gKyAqLwo+ICtz dGF0aWMgYm9vbCB2aG9zdF92ZHBhX3N2cV9tYXBfcmluZ3Moc3RydWN0IHZob3N0X2RldiAqZGV2 LAo+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3QgVmhvc3RTaGFk b3dWaXJ0cXVldWUgKnN2cSkKPiArewo+ICsgICAgc3RydWN0IHZob3N0X3ZkcGEgKnYgPSBkZXYt Pm9wYXF1ZTsKPiArICAgIHN0cnVjdCB2aG9zdF92cmluZ19hZGRyIHN2cV9hZGRyOwo+ICsgICAg c2l6ZV90IGRldmljZV9zaXplID0gdmhvc3Rfc3ZxX2RldmljZV9hcmVhX3NpemUoc3ZxKTsKPiAr ICAgIHNpemVfdCBkcml2ZXJfc2l6ZSA9IHZob3N0X3N2cV9kcml2ZXJfYXJlYV9zaXplKHN2cSk7 Cj4gKyAgICBpbnQgcjsKPiArCj4gKyAgICB2aG9zdF9zdnFfZ2V0X3ZyaW5nX2FkZHIoc3ZxLCAm c3ZxX2FkZHIpOwo+ICsKPiArICAgIHIgPSB2aG9zdF92ZHBhX2RtYV9tYXAodiwgc3ZxX2FkZHIu ZGVzY191c2VyX2FkZHIsIGRyaXZlcl9zaXplLAo+ICsgICAgICAgICAgICAgICAgICAgICAgICAg ICAodm9pZCAqKXN2cV9hZGRyLmRlc2NfdXNlcl9hZGRyLCB0cnVlKTsKPiArICAgIGlmICh1bmxp a2VseShyICE9IDApKSB7Cj4gKyAgICAgICAgcmV0dXJuIGZhbHNlOwo+ICsgICAgfQo+ICsKPiAr ICAgIHIgPSB2aG9zdF92ZHBhX2RtYV9tYXAodiwgc3ZxX2FkZHIudXNlZF91c2VyX2FkZHIsIGRl dmljZV9zaXplLAo+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAodm9pZCAqKXN2cV9hZGRy LnVzZWRfdXNlcl9hZGRyLCBmYWxzZSk7CgoKRG8gd2UgbmVlZCB1bm1hcCB0aGUgZHJpdmVyIGFy ZWEgaWYgd2UgZmFpbCBoZXJlPwoKVGhhbmtzCgoKPiArICAgIHJldHVybiByID09IDA7Cj4gK30K PiArCj4gK3N0YXRpYyBib29sIHZob3N0X3ZkcGFfc3ZxX3NldHVwKHN0cnVjdCB2aG9zdF9kZXYg KmRldiwKPiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBWaG9zdFNoYWRvd1ZpcnRx dWV1ZSAqc3ZxLAo+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlk eCkKPiArewo+ICsgICAgdWludDE2X3QgdnFfaW5kZXggPSBkZXYtPnZxX2luZGV4ICsgaWR4Owo+ ICsgICAgc3RydWN0IHZob3N0X3ZyaW5nX3N0YXRlIHMgPSB7Cj4gKyAgICAgICAgLmluZGV4ID0g dnFfaW5kZXgsCj4gKyAgICB9Owo+ICsgICAgaW50IHI7Cj4gKyAgICBib29sIG9rOwo+ICsKPiAr ICAgIHIgPSB2aG9zdF92ZHBhX3NldF9kZXZfdnJpbmdfYmFzZShkZXYsICZzKTsKPiArICAgIGlm ICh1bmxpa2VseShyKSkgewo+ICsgICAgICAgIGVycm9yX3JlcG9ydCgiQ2FuJ3Qgc2V0IHZyaW5n IGJhc2UgKCVkKSIsIHIpOwo+ICsgICAgICAgIHJldHVybiBmYWxzZTsKPiArICAgIH0KPiArCj4g KyAgICBzLm51bSA9IHZob3N0X3N2cV9nZXRfbnVtKHN2cSk7Cj4gKyAgICByID0gdmhvc3RfdmRw YV9zZXRfZGV2X3ZyaW5nX251bShkZXYsICZzKTsKPiArICAgIGlmICh1bmxpa2VseShyKSkgewo+ ICsgICAgICAgIGVycm9yX3JlcG9ydCgiQ2FuJ3Qgc2V0IHZyaW5nIG51bSAoJWQpIiwgcik7Cj4g KyAgICAgICAgcmV0dXJuIGZhbHNlOwo+ICsgICAgfQo+ICsKPiArICAgIG9rID0gdmhvc3RfdmRw YV9zdnFfbWFwX3JpbmdzKGRldiwgc3ZxKTsKPiArICAgIGlmICh1bmxpa2VseSghb2spKSB7Cj4g KyAgICAgICAgcmV0dXJuIGZhbHNlOwo+ICsgICAgfQo+ICsKPiArICAgIHIgPSB2aG9zdF92ZHBh X3N2cV9zZXRfZmRzKGRldiwgc3ZxLCBpZHgpOwo+ICAgICAgIHJldHVybiByID09IDA7Cj4gICB9 Cj4gICAKPiBAQCAtNzg4LDE0ICs4ODEsMjQgQEAgc3RhdGljIGludCB2aG9zdF92ZHBhX2Rldl9z dGFydChzdHJ1Y3Qgdmhvc3RfZGV2ICpkZXYsIGJvb2wgc3RhcnRlZCkKPiAgICAgICBpZiAoc3Rh cnRlZCkgewo+ICAgICAgICAgICB2aG9zdF92ZHBhX2hvc3Rfbm90aWZpZXJzX2luaXQoZGV2KTsK PiAgICAgICAgICAgZm9yICh1bnNpZ25lZCBpID0gMDsgaSA8IHYtPnNoYWRvd192cXMtPmxlbjsg KytpKSB7Cj4gKyAgICAgICAgICAgIFZpcnRRdWV1ZSAqdnEgPSB2aXJ0aW9fZ2V0X3F1ZXVlKGRl di0+dmRldiwgZGV2LT52cV9pbmRleCArIGkpOwo+ICAgICAgICAgICAgICAgVmhvc3RTaGFkb3dW aXJ0cXVldWUgKnN2cSA9IGdfcHRyX2FycmF5X2luZGV4KHYtPnNoYWRvd192cXMsIGkpOwo+ICAg ICAgICAgICAgICAgYm9vbCBvayA9IHZob3N0X3ZkcGFfc3ZxX3NldHVwKGRldiwgc3ZxLCBpKTsK PiAgICAgICAgICAgICAgIGlmICh1bmxpa2VseSghb2spKSB7Cj4gICAgICAgICAgICAgICAgICAg cmV0dXJuIC0xOwo+ICAgICAgICAgICAgICAgfQo+ICsgICAgICAgICAgICB2aG9zdF9zdnFfc3Rh cnQoc3ZxLCBkZXYtPnZkZXYsIHZxKTsKPiAgICAgICAgICAgfQo+ICAgICAgICAgICB2aG9zdF92 ZHBhX3NldF92cmluZ19yZWFkeShkZXYpOwo+ICAgICAgIH0gZWxzZSB7Cj4gKyAgICAgICAgZm9y ICh1bnNpZ25lZCBpID0gMDsgaSA8IHYtPnNoYWRvd192cXMtPmxlbjsgKytpKSB7Cj4gKyAgICAg ICAgICAgIFZob3N0U2hhZG93VmlydHF1ZXVlICpzdnEgPSBnX3B0cl9hcnJheV9pbmRleCh2LT5z aGFkb3dfdnFzLAo+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgaSk7Cj4gKyAgICAgICAgICAgIGJvb2wgb2sgPSB2aG9zdF92ZHBhX3N2 cV91bm1hcF9yaW5ncyhkZXYsIHN2cSk7Cj4gKyAgICAgICAgICAgIGlmICh1bmxpa2VseSghb2sp KSB7Cj4gKyAgICAgICAgICAgICAgICByZXR1cm4gLTE7Cj4gKyAgICAgICAgICAgIH0KPiArICAg ICAgICB9Cj4gICAgICAgICAgIHZob3N0X3ZkcGFfaG9zdF9ub3RpZmllcnNfdW5pbml0KGRldiwg ZGV2LT5udnFzKTsKPiAgICAgICB9Cj4gICAKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fClZpcnR1YWxpemF0aW9uIG1haWxpbmcgbGlzdApWaXJ0dWFsaXph dGlvbkBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4Zm91bmRh dGlvbi5vcmcvbWFpbG1hbi9saXN0aW5mby92aXJ0dWFsaXphdGlvbg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96B92C433F5 for ; Sun, 30 Jan 2022 04:44:51 +0000 (UTC) Received: from localhost ([::1]:58482 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nE254-0006eU-35 for qemu-devel@archiver.kernel.org; Sat, 29 Jan 2022 23:44:50 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51580) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nE23b-0005xj-2o for qemu-devel@nongnu.org; Sat, 29 Jan 2022 23:43:19 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:28873) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nE23N-0003Ku-0A for qemu-devel@nongnu.org; Sat, 29 Jan 2022 23:43:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643517783; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b2drNTCNkwJABK5/bzrkE2Qv3Y+XGpam7FKWmJlBdG8=; b=ZVnidb+GyHpZZ1HRMiYP7/jx97khkXHuMD+WhZppOwN6FTgVORRbFzPeF6pBt2ElcbwW3o B7VQuVPnBoSz6HDoFdsodWNiyCTiqbeqct6eHgCNPCj9aNurfYTyjMaQJ+jAgmE2d6uWzR rIk79n/czMotLWRkyD/Ow4LHsxCu11c= Received: from mail-pj1-f72.google.com (mail-pj1-f72.google.com [209.85.216.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-58-g_Kq9I2pNjKvr8sFUf-M-Q-1; Sat, 29 Jan 2022 23:43:01 -0500 X-MC-Unique: g_Kq9I2pNjKvr8sFUf-M-Q-1 Received: by mail-pj1-f72.google.com with SMTP id n3-20020a17090a2bc300b001b5bced2cf9so8971216pje.3 for ; Sat, 29 Jan 2022 20:43:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=b2drNTCNkwJABK5/bzrkE2Qv3Y+XGpam7FKWmJlBdG8=; b=foD6prUgFrHpPPLhDwH71zK/FbqDokPnhVqC8GVRSGUJ208IHPHMftbVGv6LJN8dpa e0Aze0IWkTnk2mXLKZ+3HuTvrEMAFTpH2kH2m9Rz0khn2ZBOMjMjgNwRZG5aIYRIffGP FBZuBxzECwpKzvX62xa0Hu1BuRKyOelD+DN5UE54cdWg2fI42aDJv2zlZllwS5ucyx4N o135cbsQO7xgvRbIOnZI3l+p/efxr50A9bw0CFUr0CC5tRIlSANWSR4wbkxeSgUFsW+f ovDaAR6319JboussGB9Czo1K3gfzD3mLv5k7zToAgDb27eR9PbYzu+LJNlaa5o5UXea+ QqxA== X-Gm-Message-State: AOAM531j26C+AbI1N/xjS+aREEEGddZOiZwQ/qfvwas0YklfpSqWlGZj N4DwbPxZ8SDBuB92ouTScyoiWeu7nFXQIBQNNLfbWmmZawe9ilA4M/w3RvJ/ie8/QBaJx8PYTVE 51TFPB53PFKA02H0= X-Received: by 2002:a05:6a00:14cc:: with SMTP id w12mr14774266pfu.47.1643517780588; Sat, 29 Jan 2022 20:43:00 -0800 (PST) X-Google-Smtp-Source: ABdhPJyHwZkhnrIFy+LQWFKt5x7QLQ24RCeeTJpFcEaHvUcHjHXhwWTDpqN3V2x9WpK0HEtWuzkjgw== X-Received: by 2002:a05:6a00:14cc:: with SMTP id w12mr14774228pfu.47.1643517780114; Sat, 29 Jan 2022 20:43:00 -0800 (PST) Received: from [10.72.13.18] ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id q12sm14936965pfk.199.2022.01.29.20.42.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 29 Jan 2022 20:42:59 -0800 (PST) Message-ID: <6c98f4e8-8695-ab83-ae37-2d6293a1fafa@redhat.com> Date: Sun, 30 Jan 2022 12:42:44 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.5.1 Subject: Re: [PATCH 18/31] vhost: Shadow virtqueue buffers forwarding To: =?UTF-8?Q?Eugenio_P=c3=a9rez?= , qemu-devel@nongnu.org References: <20220121202733.404989-1-eperezma@redhat.com> <20220121202733.404989-19-eperezma@redhat.com> From: Jason Wang In-Reply-To: <20220121202733.404989-19-eperezma@redhat.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=170.10.133.124; envelope-from=jasowang@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.167, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , Cindy Lu , "Michael S. Tsirkin" , Juan Quintela , Richard Henderson , Markus Armbruster , Gautam Dawar , virtualization@lists.linux-foundation.org, Eduardo Habkost , Harpreet Singh Anand , Xiao W Wang , Peter Xu , Stefan Hajnoczi , Eli Cohen , Paolo Bonzini , Zhu Lingshan , Eric Blake , Stefano Garzarella Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" 在 2022/1/22 上午4:27, Eugenio Pérez 写道: > Initial version of shadow virtqueue that actually forward buffers. There > is no iommu support at the moment, and that will be addressed in future > patches of this series. Since all vhost-vdpa devices use forced IOMMU, > this means that SVQ is not usable at this point of the series on any > device. > > For simplicity it only supports modern devices, that expects vring > in little endian, with split ring and no event idx or indirect > descriptors. Support for them will not be added in this series. > > It reuses the VirtQueue code for the device part. The driver part is > based on Linux's virtio_ring driver, but with stripped functionality > and optimizations so it's easier to review. > > However, forwarding buffers have some particular pieces: One of the most > unexpected ones is that a guest's buffer can expand through more than > one descriptor in SVQ. While this is handled gracefully by qemu's > emulated virtio devices, it may cause unexpected SVQ queue full. This > patch also solves it by checking for this condition at both guest's > kicks and device's calls. The code may be more elegant in the future if > SVQ code runs in its own iocontext. > > Signed-off-by: Eugenio Pérez > --- > hw/virtio/vhost-shadow-virtqueue.h | 2 + > hw/virtio/vhost-shadow-virtqueue.c | 365 ++++++++++++++++++++++++++++- > hw/virtio/vhost-vdpa.c | 111 ++++++++- > 3 files changed, 462 insertions(+), 16 deletions(-) > > diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h > index 39aef5ffdf..19c934af49 100644 > --- a/hw/virtio/vhost-shadow-virtqueue.h > +++ b/hw/virtio/vhost-shadow-virtqueue.h > @@ -33,6 +33,8 @@ uint16_t vhost_svq_get_num(const VhostShadowVirtqueue *svq); > size_t vhost_svq_driver_area_size(const VhostShadowVirtqueue *svq); > size_t vhost_svq_device_area_size(const VhostShadowVirtqueue *svq); > > +void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev, > + VirtQueue *vq); > void vhost_svq_stop(VhostShadowVirtqueue *svq); > > VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize); > diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c > index 7c168075d7..a1a404f68f 100644 > --- a/hw/virtio/vhost-shadow-virtqueue.c > +++ b/hw/virtio/vhost-shadow-virtqueue.c > @@ -9,6 +9,8 @@ > > #include "qemu/osdep.h" > #include "hw/virtio/vhost-shadow-virtqueue.h" > +#include "hw/virtio/vhost.h" > +#include "hw/virtio/virtio-access.h" > #include "standard-headers/linux/vhost_types.h" > > #include "qemu/error-report.h" > @@ -36,6 +38,33 @@ typedef struct VhostShadowVirtqueue { > > /* Guest's call notifier, where SVQ calls guest. */ > EventNotifier svq_call; > + > + /* Virtio queue shadowing */ > + VirtQueue *vq; > + > + /* Virtio device */ > + VirtIODevice *vdev; > + > + /* Map for returning guest's descriptors */ > + VirtQueueElement **ring_id_maps; > + > + /* Next VirtQueue element that guest made available */ > + VirtQueueElement *next_guest_avail_elem; > + > + /* Next head to expose to device */ > + uint16_t avail_idx_shadow; > + > + /* Next free descriptor */ > + uint16_t free_head; > + > + /* Last seen used idx */ > + uint16_t shadow_used_idx; > + > + /* Next head to consume from device */ > + uint16_t last_used_idx; > + > + /* Cache for the exposed notification flag */ > + bool notification; > } VhostShadowVirtqueue; > > #define INVALID_SVQ_KICK_FD -1 > @@ -148,30 +177,294 @@ bool vhost_svq_ack_guest_features(uint64_t dev_features, > return true; > } > > -/* Forward guest notifications */ > -static void vhost_handle_guest_kick(EventNotifier *n) > +/** > + * Number of descriptors that SVQ can make available from the guest. > + * > + * @svq The svq > + */ > +static uint16_t vhost_svq_available_slots(const VhostShadowVirtqueue *svq) > { > - VhostShadowVirtqueue *svq = container_of(n, VhostShadowVirtqueue, > - svq_kick); > + return svq->vring.num - (svq->avail_idx_shadow - svq->shadow_used_idx); > +} > + > +static void vhost_svq_set_notification(VhostShadowVirtqueue *svq, bool enable) > +{ > + uint16_t notification_flag; > > - if (unlikely(!event_notifier_test_and_clear(n))) { > + if (svq->notification == enable) { > + return; > + } > + > + notification_flag = cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); > + > + svq->notification = enable; > + if (enable) { > + svq->vring.avail->flags &= ~notification_flag; > + } else { > + svq->vring.avail->flags |= notification_flag; > + } > +} > + > +static void vhost_vring_write_descs(VhostShadowVirtqueue *svq, > + const struct iovec *iovec, > + size_t num, bool more_descs, bool write) > +{ > + uint16_t i = svq->free_head, last = svq->free_head; > + unsigned n; > + uint16_t flags = write ? cpu_to_le16(VRING_DESC_F_WRITE) : 0; > + vring_desc_t *descs = svq->vring.desc; > + > + if (num == 0) { > + return; > + } > + > + for (n = 0; n < num; n++) { > + if (more_descs || (n + 1 < num)) { > + descs[i].flags = flags | cpu_to_le16(VRING_DESC_F_NEXT); > + } else { > + descs[i].flags = flags; > + } > + descs[i].addr = cpu_to_le64((hwaddr)iovec[n].iov_base); > + descs[i].len = cpu_to_le32(iovec[n].iov_len); > + > + last = i; > + i = cpu_to_le16(descs[i].next); > + } > + > + svq->free_head = le16_to_cpu(descs[last].next); > +} > + > +static unsigned vhost_svq_add_split(VhostShadowVirtqueue *svq, > + VirtQueueElement *elem) > +{ > + int head; > + unsigned avail_idx; > + vring_avail_t *avail = svq->vring.avail; > + > + head = svq->free_head; > + > + /* We need some descriptors here */ > + assert(elem->out_num || elem->in_num); Looks like this could be triggered by guest, we need fail instead assert here. > + > + vhost_vring_write_descs(svq, elem->out_sg, elem->out_num, > + elem->in_num > 0, false); > + vhost_vring_write_descs(svq, elem->in_sg, elem->in_num, false, true); > + > + /* > + * Put entry in available array (but don't update avail->idx until they > + * do sync). > + */ > + avail_idx = svq->avail_idx_shadow & (svq->vring.num - 1); > + avail->ring[avail_idx] = cpu_to_le16(head); > + svq->avail_idx_shadow++; > + > + /* Update avail index after the descriptor is wrote */ > + smp_wmb(); > + avail->idx = cpu_to_le16(svq->avail_idx_shadow); > + > + return head; > +} > + > +static void vhost_svq_add(VhostShadowVirtqueue *svq, VirtQueueElement *elem) > +{ > + unsigned qemu_head = vhost_svq_add_split(svq, elem); > + > + svq->ring_id_maps[qemu_head] = elem; > +} > + > +static void vhost_svq_kick(VhostShadowVirtqueue *svq) > +{ > + /* We need to expose available array entries before checking used flags */ > + smp_mb(); > + if (svq->vring.used->flags & VRING_USED_F_NO_NOTIFY) { > return; > } > > event_notifier_set(&svq->hdev_kick); > } > > -/* Forward vhost notifications */ > +/** > + * Forward available buffers. > + * > + * @svq Shadow VirtQueue > + * > + * Note that this function does not guarantee that all guest's available > + * buffers are available to the device in SVQ avail ring. The guest may have > + * exposed a GPA / GIOVA congiuous buffer, but it may not be contiguous in qemu > + * vaddr. > + * > + * If that happens, guest's kick notifications will be disabled until device > + * makes some buffers used. > + */ > +static void vhost_handle_guest_kick(VhostShadowVirtqueue *svq) > +{ > + /* Clear event notifier */ > + event_notifier_test_and_clear(&svq->svq_kick); > + > + /* Make available as many buffers as possible */ > + do { > + if (virtio_queue_get_notification(svq->vq)) { > + virtio_queue_set_notification(svq->vq, false); This looks like an optimization the should belong to virtio_queue_set_notification() itself. > + } > + > + while (true) { > + VirtQueueElement *elem; > + > + if (svq->next_guest_avail_elem) { > + elem = g_steal_pointer(&svq->next_guest_avail_elem); > + } else { > + elem = virtqueue_pop(svq->vq, sizeof(*elem)); > + } > + > + if (!elem) { > + break; > + } > + > + if (elem->out_num + elem->in_num > > + vhost_svq_available_slots(svq)) { > + /* > + * This condition is possible since a contiguous buffer in GPA > + * does not imply a contiguous buffer in qemu's VA > + * scatter-gather segments. If that happen, the buffer exposed > + * to the device needs to be a chain of descriptors at this > + * moment. > + * > + * SVQ cannot hold more available buffers if we are here: > + * queue the current guest descriptor and ignore further kicks > + * until some elements are used. > + */ > + svq->next_guest_avail_elem = elem; > + return; > + } > + > + vhost_svq_add(svq, elem); > + vhost_svq_kick(svq); > + } > + > + virtio_queue_set_notification(svq->vq, true); > + } while (!virtio_queue_empty(svq->vq)); > +} > + > +/** > + * Handle guest's kick. > + * > + * @n guest kick event notifier, the one that guest set to notify svq. > + */ > +static void vhost_handle_guest_kick_notifier(EventNotifier *n) > +{ > + VhostShadowVirtqueue *svq = container_of(n, VhostShadowVirtqueue, > + svq_kick); > + vhost_handle_guest_kick(svq); > +} > + > +static bool vhost_svq_more_used(VhostShadowVirtqueue *svq) > +{ > + if (svq->last_used_idx != svq->shadow_used_idx) { > + return true; > + } > + > + svq->shadow_used_idx = cpu_to_le16(svq->vring.used->idx); > + > + return svq->last_used_idx != svq->shadow_used_idx; > +} > + > +static VirtQueueElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq) > +{ > + vring_desc_t *descs = svq->vring.desc; > + const vring_used_t *used = svq->vring.used; > + vring_used_elem_t used_elem; > + uint16_t last_used; > + > + if (!vhost_svq_more_used(svq)) { > + return NULL; > + } > + > + /* Only get used array entries after they have been exposed by dev */ > + smp_rmb(); > + last_used = svq->last_used_idx & (svq->vring.num - 1); > + used_elem.id = le32_to_cpu(used->ring[last_used].id); > + used_elem.len = le32_to_cpu(used->ring[last_used].len); > + > + svq->last_used_idx++; > + if (unlikely(used_elem.id >= svq->vring.num)) { > + error_report("Device %s says index %u is used", svq->vdev->name, > + used_elem.id); > + return NULL; > + } > + > + if (unlikely(!svq->ring_id_maps[used_elem.id])) { > + error_report( > + "Device %s says index %u is used, but it was not available", > + svq->vdev->name, used_elem.id); > + return NULL; > + } > + > + descs[used_elem.id].next = svq->free_head; > + svq->free_head = used_elem.id; > + > + svq->ring_id_maps[used_elem.id]->len = used_elem.len; > + return g_steal_pointer(&svq->ring_id_maps[used_elem.id]); > +} > + > +static void vhost_svq_flush(VhostShadowVirtqueue *svq, > + bool check_for_avail_queue) > +{ > + VirtQueue *vq = svq->vq; > + > + /* Make as many buffers as possible used. */ > + do { > + unsigned i = 0; > + > + vhost_svq_set_notification(svq, false); > + while (true) { > + g_autofree VirtQueueElement *elem = vhost_svq_get_buf(svq); > + if (!elem) { > + break; > + } > + > + if (unlikely(i >= svq->vring.num)) { > + virtio_error(svq->vdev, > + "More than %u used buffers obtained in a %u size SVQ", > + i, svq->vring.num); > + virtqueue_fill(vq, elem, elem->len, i); > + virtqueue_flush(vq, i); Let's simply use virtqueue_push() here? > + i = 0; Do we need to bail out here? > + } > + virtqueue_fill(vq, elem, elem->len, i++); > + } > + > + virtqueue_flush(vq, i); > + event_notifier_set(&svq->svq_call); > + > + if (check_for_avail_queue && svq->next_guest_avail_elem) { > + /* > + * Avail ring was full when vhost_svq_flush was called, so it's a > + * good moment to make more descriptors available if possible > + */ > + vhost_handle_guest_kick(svq); Is there better to have a similar check as vhost_handle_guest_kick() did?             if (elem->out_num + elem->in_num >                 vhost_svq_available_slots(svq)) { > + } > + > + vhost_svq_set_notification(svq, true); A mb() is needed here? Otherwise we may lost a call here (where vhost_svq_more_used() is run before vhost_svq_set_notification()). > + } while (vhost_svq_more_used(svq)); > +} > + > +/** > + * Forward used buffers. > + * > + * @n hdev call event notifier, the one that device set to notify svq. > + * > + * Note that we are not making any buffers available in the loop, there is no > + * way that it runs more than virtqueue size times. > + */ > static void vhost_svq_handle_call(EventNotifier *n) > { > VhostShadowVirtqueue *svq = container_of(n, VhostShadowVirtqueue, > hdev_call); > > - if (unlikely(!event_notifier_test_and_clear(n))) { > - return; > - } > + /* Clear event notifier */ > + event_notifier_test_and_clear(n); Any reason that we remove the above check? > > - event_notifier_set(&svq->svq_call); > + vhost_svq_flush(svq, true); > } > > /** > @@ -258,13 +551,38 @@ void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd) > * need to explicitely check for them. > */ > event_notifier_init_fd(&svq->svq_kick, svq_kick_fd); > - event_notifier_set_handler(&svq->svq_kick, vhost_handle_guest_kick); > + event_notifier_set_handler(&svq->svq_kick, > + vhost_handle_guest_kick_notifier); > > if (!check_old || event_notifier_test_and_clear(&tmp)) { > event_notifier_set(&svq->hdev_kick); > } > } > > +/** > + * Start shadow virtqueue operation. > + * > + * @svq Shadow Virtqueue > + * @vdev VirtIO device > + * @vq Virtqueue to shadow > + */ > +void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev, > + VirtQueue *vq) > +{ > + svq->next_guest_avail_elem = NULL; > + svq->avail_idx_shadow = 0; > + svq->shadow_used_idx = 0; > + svq->last_used_idx = 0; > + svq->vdev = vdev; > + svq->vq = vq; > + > + memset(svq->vring.avail, 0, sizeof(*svq->vring.avail)); > + memset(svq->vring.used, 0, sizeof(*svq->vring.avail)); > + for (unsigned i = 0; i < svq->vring.num - 1; i++) { > + svq->vring.desc[i].next = cpu_to_le16(i + 1); > + } > +} > + > /** > * Stop shadow virtqueue operation. > * @svq Shadow Virtqueue > @@ -272,6 +590,28 @@ void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd) > void vhost_svq_stop(VhostShadowVirtqueue *svq) > { > event_notifier_set_handler(&svq->svq_kick, NULL); > + g_autofree VirtQueueElement *next_avail_elem = NULL; > + > + if (!svq->vq) { > + return; > + } > + > + /* Send all pending used descriptors to guest */ > + vhost_svq_flush(svq, false); > + > + for (unsigned i = 0; i < svq->vring.num; ++i) { > + g_autofree VirtQueueElement *elem = NULL; > + elem = g_steal_pointer(&svq->ring_id_maps[i]); > + if (elem) { > + virtqueue_detach_element(svq->vq, elem, elem->len); > + } > + } > + > + next_avail_elem = g_steal_pointer(&svq->next_guest_avail_elem); > + if (next_avail_elem) { > + virtqueue_detach_element(svq->vq, next_avail_elem, > + next_avail_elem->len); > + } > } > > /** > @@ -316,7 +656,7 @@ VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize) > memset(svq->vring.desc, 0, driver_size); > svq->vring.used = qemu_memalign(qemu_real_host_page_size, device_size); > memset(svq->vring.used, 0, device_size); > - > + svq->ring_id_maps = g_new0(VirtQueueElement *, qsize); > event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call); > return g_steal_pointer(&svq); > > @@ -335,6 +675,7 @@ void vhost_svq_free(VhostShadowVirtqueue *vq) > event_notifier_cleanup(&vq->hdev_kick); > event_notifier_set_handler(&vq->hdev_call, NULL); > event_notifier_cleanup(&vq->hdev_call); > + g_free(vq->ring_id_maps); > qemu_vfree(vq->vring.desc); > qemu_vfree(vq->vring.used); > g_free(vq); > diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c > index 53e14bafa0..0e5c00ed7e 100644 > --- a/hw/virtio/vhost-vdpa.c > +++ b/hw/virtio/vhost-vdpa.c > @@ -752,9 +752,9 @@ static int vhost_vdpa_set_vring_call(struct vhost_dev *dev, > * Note that this function does not rewind kick file descriptor if cannot set > * call one. > */ > -static bool vhost_vdpa_svq_setup(struct vhost_dev *dev, > - VhostShadowVirtqueue *svq, > - unsigned idx) > +static int vhost_vdpa_svq_set_fds(struct vhost_dev *dev, > + VhostShadowVirtqueue *svq, > + unsigned idx) > { > struct vhost_vring_file file = { > .index = dev->vq_index + idx, > @@ -767,7 +767,7 @@ static bool vhost_vdpa_svq_setup(struct vhost_dev *dev, > r = vhost_vdpa_set_vring_dev_kick(dev, &file); > if (unlikely(r != 0)) { > error_report("Can't set device kick fd (%d)", -r); > - return false; > + return r; > } > > event_notifier = vhost_svq_get_svq_call_notifier(svq); > @@ -777,6 +777,99 @@ static bool vhost_vdpa_svq_setup(struct vhost_dev *dev, > error_report("Can't set device call fd (%d)", -r); > } > > + return r; > +} > + > +/** > + * Unmap SVQ area in the device > + */ > +static bool vhost_vdpa_svq_unmap_ring(struct vhost_vdpa *v, hwaddr iova, > + hwaddr size) > +{ > + int r; > + > + size = ROUND_UP(size, qemu_real_host_page_size); > + r = vhost_vdpa_dma_unmap(v, iova, size); > + return r == 0; > +} > + > +static bool vhost_vdpa_svq_unmap_rings(struct vhost_dev *dev, > + const VhostShadowVirtqueue *svq) > +{ > + struct vhost_vdpa *v = dev->opaque; > + struct vhost_vring_addr svq_addr; > + size_t device_size = vhost_svq_device_area_size(svq); > + size_t driver_size = vhost_svq_driver_area_size(svq); > + bool ok; > + > + vhost_svq_get_vring_addr(svq, &svq_addr); > + > + ok = vhost_vdpa_svq_unmap_ring(v, svq_addr.desc_user_addr, driver_size); > + if (unlikely(!ok)) { > + return false; > + } > + > + return vhost_vdpa_svq_unmap_ring(v, svq_addr.used_user_addr, device_size); > +} > + > +/** > + * Map shadow virtqueue rings in device > + * > + * @dev The vhost device > + * @svq The shadow virtqueue > + */ > +static bool vhost_vdpa_svq_map_rings(struct vhost_dev *dev, > + const VhostShadowVirtqueue *svq) > +{ > + struct vhost_vdpa *v = dev->opaque; > + struct vhost_vring_addr svq_addr; > + size_t device_size = vhost_svq_device_area_size(svq); > + size_t driver_size = vhost_svq_driver_area_size(svq); > + int r; > + > + vhost_svq_get_vring_addr(svq, &svq_addr); > + > + r = vhost_vdpa_dma_map(v, svq_addr.desc_user_addr, driver_size, > + (void *)svq_addr.desc_user_addr, true); > + if (unlikely(r != 0)) { > + return false; > + } > + > + r = vhost_vdpa_dma_map(v, svq_addr.used_user_addr, device_size, > + (void *)svq_addr.used_user_addr, false); Do we need unmap the driver area if we fail here? Thanks > + return r == 0; > +} > + > +static bool vhost_vdpa_svq_setup(struct vhost_dev *dev, > + VhostShadowVirtqueue *svq, > + unsigned idx) > +{ > + uint16_t vq_index = dev->vq_index + idx; > + struct vhost_vring_state s = { > + .index = vq_index, > + }; > + int r; > + bool ok; > + > + r = vhost_vdpa_set_dev_vring_base(dev, &s); > + if (unlikely(r)) { > + error_report("Can't set vring base (%d)", r); > + return false; > + } > + > + s.num = vhost_svq_get_num(svq); > + r = vhost_vdpa_set_dev_vring_num(dev, &s); > + if (unlikely(r)) { > + error_report("Can't set vring num (%d)", r); > + return false; > + } > + > + ok = vhost_vdpa_svq_map_rings(dev, svq); > + if (unlikely(!ok)) { > + return false; > + } > + > + r = vhost_vdpa_svq_set_fds(dev, svq, idx); > return r == 0; > } > > @@ -788,14 +881,24 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) > if (started) { > vhost_vdpa_host_notifiers_init(dev); > for (unsigned i = 0; i < v->shadow_vqs->len; ++i) { > + VirtQueue *vq = virtio_get_queue(dev->vdev, dev->vq_index + i); > VhostShadowVirtqueue *svq = g_ptr_array_index(v->shadow_vqs, i); > bool ok = vhost_vdpa_svq_setup(dev, svq, i); > if (unlikely(!ok)) { > return -1; > } > + vhost_svq_start(svq, dev->vdev, vq); > } > vhost_vdpa_set_vring_ready(dev); > } else { > + for (unsigned i = 0; i < v->shadow_vqs->len; ++i) { > + VhostShadowVirtqueue *svq = g_ptr_array_index(v->shadow_vqs, > + i); > + bool ok = vhost_vdpa_svq_unmap_rings(dev, svq); > + if (unlikely(!ok)) { > + return -1; > + } > + } > vhost_vdpa_host_notifiers_uninit(dev, dev->nvqs); > } >