From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932729AbeDXAzC (ORCPT ); Mon, 23 Apr 2018 20:55:02 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:49188 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932635AbeDXAzA (ORCPT ); Mon, 23 Apr 2018 20:55:00 -0400 Subject: Re: [RFC v2] virtio: support packed ring To: Tiwei Bie Cc: mst@redhat.com, wexu@redhat.com, virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, jfreimann@redhat.com References: <20180401141216.8969-1-tiwei.bie@intel.com> <515e635b-bc80-9b8d-72f9-b390ae5103ec@redhat.com> <20180423092908.77rii3gi7dcaf7o6@debian> From: Jason Wang Message-ID: Date: Tue, 24 Apr 2018 08:54:52 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <20180423092908.77rii3gi7dcaf7o6@debian> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2018年04月23日 17:29, Tiwei Bie wrote: > On Mon, Apr 23, 2018 at 01:42:14PM +0800, Jason Wang wrote: >> On 2018年04月01日 22:12, Tiwei Bie wrote: >>> Hello everyone, >>> >>> This RFC implements packed ring support for virtio driver. >>> >>> The code was tested with DPDK vhost (testpmd/vhost-PMD) implemented >>> by Jens at http://dpdk.org/ml/archives/dev/2018-January/089417.html >>> Minor changes are needed for the vhost code, e.g. to kick the guest. >>> >>> TODO: >>> - Refinements and bug fixes; >>> - Split into small patches; >>> - Test indirect descriptor support; >>> - Test/fix event suppression support; >>> - Test devices other than net; >>> >>> RFC v1 -> RFC v2: >>> - Add indirect descriptor support - compile test only; >>> - Add event suppression supprt - compile test only; >>> - Move vring_packed_init() out of uapi (Jason, MST); >>> - Merge two loops into one in virtqueue_add_packed() (Jason); >>> - Split vring_unmap_one() for packed ring and split ring (Jason); >>> - Avoid using '%' operator (Jason); >>> - Rename free_head -> next_avail_idx (Jason); >>> - Add comments for virtio_wmb() in virtqueue_add_packed() (Jason); >>> - Some other refinements and bug fixes; >>> >>> Thanks! >>> >>> Signed-off-by: Tiwei Bie >>> --- >>> drivers/virtio/virtio_ring.c | 1094 +++++++++++++++++++++++++++++------- >>> include/linux/virtio_ring.h | 8 +- >>> include/uapi/linux/virtio_config.h | 12 +- >>> include/uapi/linux/virtio_ring.h | 61 ++ >>> 4 files changed, 980 insertions(+), 195 deletions(-) >>> >>> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c >>> index 71458f493cf8..0515dca34d77 100644 >>> --- a/drivers/virtio/virtio_ring.c >>> +++ b/drivers/virtio/virtio_ring.c >>> @@ -58,14 +58,15 @@ >> [...] >> >>> + >>> + if (vq->indirect) { >>> + u32 len; >>> + >>> + desc = vq->desc_state[head].indir_desc; >>> + /* Free the indirect table, if any, now that it's unmapped. */ >>> + if (!desc) >>> + goto out; >>> + >>> + len = virtio32_to_cpu(vq->vq.vdev, >>> + vq->vring_packed.desc[head].len); >>> + >>> + BUG_ON(!(vq->vring_packed.desc[head].flags & >>> + cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_INDIRECT))); >> It looks to me spec does not force to keep VRING_DESC_F_INDIRECT here. So we >> can safely remove this BUG_ON() here. >> >>> + BUG_ON(len == 0 || len % sizeof(struct vring_packed_desc)); >> Len could be ignored for used descriptor according to the spec, so we need >> remove this BUG_ON() too. > Yeah, you're right! The BUG_ON() isn't right. I'll remove it. > And I think something related to this in the spec isn't very > clear currently. > > In the spec, there are below words: > > https://github.com/oasis-tcs/virtio-spec/blob/d4fec517dfcf/packed-ring.tex#L272 > """ > In descriptors with VIRTQ_DESC_F_INDIRECT set VIRTQ_DESC_F_WRITE > is reserved and is ignored by the device. > """ > > So when device writes back an used descriptor in this case, > device may not set the VIRTQ_DESC_F_WRITE flag as the flag > is reserved and should be ignored. > > https://github.com/oasis-tcs/virtio-spec/blob/d4fec517dfcf/packed-ring.tex#L170 > """ > Element Length is reserved for used descriptors without the > VIRTQ_DESC_F_WRITE flag, and is ignored by drivers. > """ > > And this is the way how driver ignores the `len` in an used > descriptor. > > https://github.com/oasis-tcs/virtio-spec/blob/d4fec517dfcf/packed-ring.tex#L241 > """ > To increase ring capacity the driver can store a (read-only > by the device) table of indirect descriptors anywhere in memory, > and insert a descriptor in the main virtqueue (with \field{Flags} > bit VIRTQ_DESC_F_INDIRECT on) that refers to a buffer element > containing this indirect descriptor table; > """ > > So the indirect descriptors in the table are read-only by > the device. And the only descriptor which is writeable by > the device is the descriptor in the main virtqueue (with > Flags bit VIRTQ_DESC_F_INDIRECT on). So if we ignore the > `len` in this descriptor, we won't be able to get the > length of the data written by the device. > > So I think the `len` in this descriptor will carry the > length of the data written by the device (if the buffers > are writable to the device) even if the VIRTQ_DESC_F_WRITE > isn't set by the device. How do you think? Yes I think so. But we'd better need clarification from Michael. > > >> The reason is we don't touch descriptor ring in the case of split, so >> BUG_ON()s may help there. >> >>> + >>> + for (j = 0; j < len / sizeof(struct vring_packed_desc); j++) >>> + vring_unmap_one_packed(vq, &desc[j]); >>> + >>> + kfree(desc); >>> + vq->desc_state[head].indir_desc = NULL; >>> + } else if (ctx) { >>> + *ctx = vq->desc_state[head].indir_desc; >>> + } >>> + >>> +out: >>> + return vq->desc_state[head].num; >>> +} >>> + >>> +static inline bool more_used_split(const struct vring_virtqueue *vq) >>> { >>> return vq->last_used_idx != virtio16_to_cpu(vq->vq.vdev, vq->vring.used->idx); >>> } >>> +static inline bool more_used_packed(const struct vring_virtqueue *vq) >>> +{ >>> + u16 last_used, flags; >>> + bool avail, used; >>> + >>> + if (vq->vq.num_free == vq->vring_packed.num) >>> + return false; >>> + >>> + last_used = vq->last_used_idx; >>> + flags = virtio16_to_cpu(vq->vq.vdev, >>> + vq->vring_packed.desc[last_used].flags); >>> + avail = flags & VRING_DESC_F_AVAIL(1); >>> + used = flags & VRING_DESC_F_USED(1); >>> + >>> + return avail == used; >>> +} >> This looks interesting, spec said: >> >> " >> Thus VIRTQ_DESC_F_AVAIL and VIRTQ_DESC_F_USED bits are different for an >> available descriptor and >> equal for a used descriptor. >> Note that this observation is mostly useful for sanity-checking as these are >> necessary but not sufficient >> conditions - for example, all descriptors are zero-initialized. To detect >> used and available descriptors it is >> possible for drivers and devices to keep track of the last observed value of >> VIRTQ_DESC_F_USED/VIRTQ_- >> DESC_F_AVAIL. Other techniques to detect >> VIRTQ_DESC_F_AVAIL/VIRTQ_DESC_F_USED bit changes >> might also be possible. >> " >> >> So it looks to me it was not sufficient, looking at the example codes in >> spec, do we need to track last seen used_wrap_counter here? > I don't think we have to track used_wrap_counter in > driver. There was a discussion on this: > > https://lists.oasis-open.org/archives/virtio-dev/201802/msg00177.html > > And after that, below sentence was added (it's also > in the above words you quoted): > > """ > Other techniques to detect > VIRTQ_DESC_F_AVAIL/VIRTQ_DESC_F_USED bit changes > might also be possible. > """ > > Best regards, > Tiwei Bie I see, the extra condition "if (vq->vq.num_free == vq->vring_packed.num)" help in this case. Thanks > >> Thanks From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Wang Subject: Re: [RFC v2] virtio: support packed ring Date: Tue, 24 Apr 2018 08:54:52 +0800 Message-ID: References: <20180401141216.8969-1-tiwei.bie@intel.com> <515e635b-bc80-9b8d-72f9-b390ae5103ec@redhat.com> <20180423092908.77rii3gi7dcaf7o6@debian> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Cc: mst@redhat.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, wexu@redhat.com To: Tiwei Bie Return-path: In-Reply-To: <20180423092908.77rii3gi7dcaf7o6@debian> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org List-Id: netdev.vger.kernel.org CgpPbiAyMDE45bm0MDTmnIgyM+aXpSAxNzoyOSwgVGl3ZWkgQmllIHdyb3RlOgo+IE9uIE1vbiwg QXByIDIzLCAyMDE4IGF0IDAxOjQyOjE0UE0gKzA4MDAsIEphc29uIFdhbmcgd3JvdGU6Cj4+IE9u IDIwMTjlubQwNOaciDAx5pelIDIyOjEyLCBUaXdlaSBCaWUgd3JvdGU6Cj4+PiBIZWxsbyBldmVy eW9uZSwKPj4+Cj4+PiBUaGlzIFJGQyBpbXBsZW1lbnRzIHBhY2tlZCByaW5nIHN1cHBvcnQgZm9y IHZpcnRpbyBkcml2ZXIuCj4+Pgo+Pj4gVGhlIGNvZGUgd2FzIHRlc3RlZCB3aXRoIERQREsgdmhv c3QgKHRlc3RwbWQvdmhvc3QtUE1EKSBpbXBsZW1lbnRlZAo+Pj4gYnkgSmVucyBhdCBodHRwOi8v ZHBkay5vcmcvbWwvYXJjaGl2ZXMvZGV2LzIwMTgtSmFudWFyeS8wODk0MTcuaHRtbAo+Pj4gTWlu b3IgY2hhbmdlcyBhcmUgbmVlZGVkIGZvciB0aGUgdmhvc3QgY29kZSwgZS5nLiB0byBraWNrIHRo ZSBndWVzdC4KPj4+Cj4+PiBUT0RPOgo+Pj4gLSBSZWZpbmVtZW50cyBhbmQgYnVnIGZpeGVzOwo+ Pj4gLSBTcGxpdCBpbnRvIHNtYWxsIHBhdGNoZXM7Cj4+PiAtIFRlc3QgaW5kaXJlY3QgZGVzY3Jp cHRvciBzdXBwb3J0Owo+Pj4gLSBUZXN0L2ZpeCBldmVudCBzdXBwcmVzc2lvbiBzdXBwb3J0Owo+ Pj4gLSBUZXN0IGRldmljZXMgb3RoZXIgdGhhbiBuZXQ7Cj4+Pgo+Pj4gUkZDIHYxIC0+IFJGQyB2 MjoKPj4+IC0gQWRkIGluZGlyZWN0IGRlc2NyaXB0b3Igc3VwcG9ydCAtIGNvbXBpbGUgdGVzdCBv bmx5Owo+Pj4gLSBBZGQgZXZlbnQgc3VwcHJlc3Npb24gc3VwcHJ0IC0gY29tcGlsZSB0ZXN0IG9u bHk7Cj4+PiAtIE1vdmUgdnJpbmdfcGFja2VkX2luaXQoKSBvdXQgb2YgdWFwaSAoSmFzb24sIE1T VCk7Cj4+PiAtIE1lcmdlIHR3byBsb29wcyBpbnRvIG9uZSBpbiB2aXJ0cXVldWVfYWRkX3BhY2tl ZCgpIChKYXNvbik7Cj4+PiAtIFNwbGl0IHZyaW5nX3VubWFwX29uZSgpIGZvciBwYWNrZWQgcmlu ZyBhbmQgc3BsaXQgcmluZyAoSmFzb24pOwo+Pj4gLSBBdm9pZCB1c2luZyAnJScgb3BlcmF0b3Ig KEphc29uKTsKPj4+IC0gUmVuYW1lIGZyZWVfaGVhZCAtPiBuZXh0X2F2YWlsX2lkeCAoSmFzb24p Owo+Pj4gLSBBZGQgY29tbWVudHMgZm9yIHZpcnRpb193bWIoKSBpbiB2aXJ0cXVldWVfYWRkX3Bh Y2tlZCgpIChKYXNvbik7Cj4+PiAtIFNvbWUgb3RoZXIgcmVmaW5lbWVudHMgYW5kIGJ1ZyBmaXhl czsKPj4+Cj4+PiBUaGFua3MhCj4+Pgo+Pj4gU2lnbmVkLW9mZi1ieTogVGl3ZWkgQmllIDx0aXdl aS5iaWVAaW50ZWwuY29tPgo+Pj4gLS0tCj4+PiAgICBkcml2ZXJzL3ZpcnRpby92aXJ0aW9fcmlu Zy5jICAgICAgIHwgMTA5NCArKysrKysrKysrKysrKysrKysrKysrKysrKysrKy0tLS0tLS0KPj4+ ICAgIGluY2x1ZGUvbGludXgvdmlydGlvX3JpbmcuaCAgICAgICAgfCAgICA4ICstCj4+PiAgICBp bmNsdWRlL3VhcGkvbGludXgvdmlydGlvX2NvbmZpZy5oIHwgICAxMiArLQo+Pj4gICAgaW5jbHVk ZS91YXBpL2xpbnV4L3ZpcnRpb19yaW5nLmggICB8ICAgNjEgKysKPj4+ICAgIDQgZmlsZXMgY2hh bmdlZCwgOTgwIGluc2VydGlvbnMoKyksIDE5NSBkZWxldGlvbnMoLSkKPj4+Cj4+PiBkaWZmIC0t Z2l0IGEvZHJpdmVycy92aXJ0aW8vdmlydGlvX3JpbmcuYyBiL2RyaXZlcnMvdmlydGlvL3ZpcnRp b19yaW5nLmMKPj4+IGluZGV4IDcxNDU4ZjQ5M2NmOC4uMDUxNWRjYTM0ZDc3IDEwMDY0NAo+Pj4g LS0tIGEvZHJpdmVycy92aXJ0aW8vdmlydGlvX3JpbmcuYwo+Pj4gKysrIGIvZHJpdmVycy92aXJ0 aW8vdmlydGlvX3JpbmcuYwo+Pj4gQEAgLTU4LDE0ICs1OCwxNSBAQAo+PiBbLi4uXQo+Pgo+Pj4g Kwo+Pj4gKwlpZiAodnEtPmluZGlyZWN0KSB7Cj4+PiArCQl1MzIgbGVuOwo+Pj4gKwo+Pj4gKwkJ ZGVzYyA9IHZxLT5kZXNjX3N0YXRlW2hlYWRdLmluZGlyX2Rlc2M7Cj4+PiArCQkvKiBGcmVlIHRo ZSBpbmRpcmVjdCB0YWJsZSwgaWYgYW55LCBub3cgdGhhdCBpdCdzIHVubWFwcGVkLiAqLwo+Pj4g KwkJaWYgKCFkZXNjKQo+Pj4gKwkJCWdvdG8gb3V0Owo+Pj4gKwo+Pj4gKwkJbGVuID0gdmlydGlv MzJfdG9fY3B1KHZxLT52cS52ZGV2LAo+Pj4gKwkJCQkgICAgICB2cS0+dnJpbmdfcGFja2VkLmRl c2NbaGVhZF0ubGVuKTsKPj4+ICsKPj4+ICsJCUJVR19PTighKHZxLT52cmluZ19wYWNrZWQuZGVz Y1toZWFkXS5mbGFncyAmCj4+PiArCQkJIGNwdV90b192aXJ0aW8xNih2cS0+dnEudmRldiwgVlJJ TkdfREVTQ19GX0lORElSRUNUKSkpOwo+PiBJdCBsb29rcyB0byBtZSBzcGVjIGRvZXMgbm90IGZv cmNlIHRvIGtlZXAgVlJJTkdfREVTQ19GX0lORElSRUNUIGhlcmUuIFNvIHdlCj4+IGNhbiBzYWZl bHkgcmVtb3ZlIHRoaXMgQlVHX09OKCkgaGVyZS4KPj4KPj4+ICsJCUJVR19PTihsZW4gPT0gMCB8 fCBsZW4gJSBzaXplb2Yoc3RydWN0IHZyaW5nX3BhY2tlZF9kZXNjKSk7Cj4+IExlbiBjb3VsZCBi ZSBpZ25vcmVkIGZvciB1c2VkIGRlc2NyaXB0b3IgYWNjb3JkaW5nIHRvIHRoZSBzcGVjLCBzbyB3 ZSBuZWVkCj4+IHJlbW92ZSB0aGlzIEJVR19PTigpIHRvby4KPiBZZWFoLCB5b3UncmUgcmlnaHQh IFRoZSBCVUdfT04oKSBpc24ndCByaWdodC4gSSdsbCByZW1vdmUgaXQuCj4gQW5kIEkgdGhpbmsg c29tZXRoaW5nIHJlbGF0ZWQgdG8gdGhpcyBpbiB0aGUgc3BlYyBpc24ndCB2ZXJ5Cj4gY2xlYXIg Y3VycmVudGx5Lgo+Cj4gSW4gdGhlIHNwZWMsIHRoZXJlIGFyZSBiZWxvdyB3b3JkczoKPgo+IGh0 dHBzOi8vZ2l0aHViLmNvbS9vYXNpcy10Y3MvdmlydGlvLXNwZWMvYmxvYi9kNGZlYzUxN2RmY2Yv cGFja2VkLXJpbmcudGV4I0wyNzIKPiAiIiIKPiBJbiBkZXNjcmlwdG9ycyB3aXRoIFZJUlRRX0RF U0NfRl9JTkRJUkVDVCBzZXQgVklSVFFfREVTQ19GX1dSSVRFCj4gaXMgcmVzZXJ2ZWQgYW5kIGlz IGlnbm9yZWQgYnkgdGhlIGRldmljZS4KPiAiIiIKPgo+IFNvIHdoZW4gZGV2aWNlIHdyaXRlcyBi YWNrIGFuIHVzZWQgZGVzY3JpcHRvciBpbiB0aGlzIGNhc2UsCj4gZGV2aWNlIG1heSBub3Qgc2V0 IHRoZSBWSVJUUV9ERVNDX0ZfV1JJVEUgZmxhZyBhcyB0aGUgZmxhZwo+IGlzIHJlc2VydmVkIGFu ZCBzaG91bGQgYmUgaWdub3JlZC4KPgo+IGh0dHBzOi8vZ2l0aHViLmNvbS9vYXNpcy10Y3Mvdmly dGlvLXNwZWMvYmxvYi9kNGZlYzUxN2RmY2YvcGFja2VkLXJpbmcudGV4I0wxNzAKPiAiIiIKPiBF bGVtZW50IExlbmd0aCBpcyByZXNlcnZlZCBmb3IgdXNlZCBkZXNjcmlwdG9ycyB3aXRob3V0IHRo ZQo+IFZJUlRRX0RFU0NfRl9XUklURSBmbGFnLCBhbmQgaXMgaWdub3JlZCBieSBkcml2ZXJzLgo+ ICIiIgo+Cj4gQW5kIHRoaXMgaXMgdGhlIHdheSBob3cgZHJpdmVyIGlnbm9yZXMgdGhlIGBsZW5g IGluIGFuIHVzZWQKPiBkZXNjcmlwdG9yLgo+Cj4gaHR0cHM6Ly9naXRodWIuY29tL29hc2lzLXRj cy92aXJ0aW8tc3BlYy9ibG9iL2Q0ZmVjNTE3ZGZjZi9wYWNrZWQtcmluZy50ZXgjTDI0MQo+ICIi Igo+IFRvIGluY3JlYXNlIHJpbmcgY2FwYWNpdHkgdGhlIGRyaXZlciBjYW4gc3RvcmUgYSAocmVh ZC1vbmx5Cj4gYnkgdGhlIGRldmljZSkgdGFibGUgb2YgaW5kaXJlY3QgZGVzY3JpcHRvcnMgYW55 d2hlcmUgaW4gbWVtb3J5LAo+IGFuZCBpbnNlcnQgYSBkZXNjcmlwdG9yIGluIHRoZSBtYWluIHZp cnRxdWV1ZSAod2l0aCBcZmllbGR7RmxhZ3N9Cj4gYml0IFZJUlRRX0RFU0NfRl9JTkRJUkVDVCBv bikgdGhhdCByZWZlcnMgdG8gYSBidWZmZXIgZWxlbWVudAo+IGNvbnRhaW5pbmcgdGhpcyBpbmRp cmVjdCBkZXNjcmlwdG9yIHRhYmxlOwo+ICIiIgo+Cj4gU28gdGhlIGluZGlyZWN0IGRlc2NyaXB0 b3JzIGluIHRoZSB0YWJsZSBhcmUgcmVhZC1vbmx5IGJ5Cj4gdGhlIGRldmljZS4gQW5kIHRoZSBv bmx5IGRlc2NyaXB0b3Igd2hpY2ggaXMgd3JpdGVhYmxlIGJ5Cj4gdGhlIGRldmljZSBpcyB0aGUg ZGVzY3JpcHRvciBpbiB0aGUgbWFpbiB2aXJ0cXVldWUgKHdpdGgKPiBGbGFncyBiaXQgVklSVFFf REVTQ19GX0lORElSRUNUIG9uKS4gU28gaWYgd2UgaWdub3JlIHRoZQo+IGBsZW5gIGluIHRoaXMg ZGVzY3JpcHRvciwgd2Ugd29uJ3QgYmUgYWJsZSB0byBnZXQgdGhlCj4gbGVuZ3RoIG9mIHRoZSBk YXRhIHdyaXR0ZW4gYnkgdGhlIGRldmljZS4KPgo+IFNvIEkgdGhpbmsgdGhlIGBsZW5gIGluIHRo aXMgZGVzY3JpcHRvciB3aWxsIGNhcnJ5IHRoZQo+IGxlbmd0aCBvZiB0aGUgZGF0YSB3cml0dGVu IGJ5IHRoZSBkZXZpY2UgKGlmIHRoZSBidWZmZXJzCj4gYXJlIHdyaXRhYmxlIHRvIHRoZSBkZXZp Y2UpIGV2ZW4gaWYgdGhlIFZJUlRRX0RFU0NfRl9XUklURQo+IGlzbid0IHNldCBieSB0aGUgZGV2 aWNlLiBIb3cgZG8geW91IHRoaW5rPwoKWWVzIEkgdGhpbmsgc28uIEJ1dCB3ZSdkIGJldHRlciBu ZWVkIGNsYXJpZmljYXRpb24gZnJvbSBNaWNoYWVsLgoKPgo+Cj4+IFRoZSByZWFzb24gaXMgd2Ug ZG9uJ3QgdG91Y2ggZGVzY3JpcHRvciByaW5nIGluIHRoZSBjYXNlIG9mIHNwbGl0LCBzbwo+PiBC VUdfT04oKXMgbWF5IGhlbHAgdGhlcmUuCj4+Cj4+PiArCj4+PiArCQlmb3IgKGogPSAwOyBqIDwg bGVuIC8gc2l6ZW9mKHN0cnVjdCB2cmluZ19wYWNrZWRfZGVzYyk7IGorKykKPj4+ICsJCQl2cmlu Z191bm1hcF9vbmVfcGFja2VkKHZxLCAmZGVzY1tqXSk7Cj4+PiArCj4+PiArCQlrZnJlZShkZXNj KTsKPj4+ICsJCXZxLT5kZXNjX3N0YXRlW2hlYWRdLmluZGlyX2Rlc2MgPSBOVUxMOwo+Pj4gKwl9 IGVsc2UgaWYgKGN0eCkgewo+Pj4gKwkJKmN0eCA9IHZxLT5kZXNjX3N0YXRlW2hlYWRdLmluZGly X2Rlc2M7Cj4+PiArCX0KPj4+ICsKPj4+ICtvdXQ6Cj4+PiArCXJldHVybiB2cS0+ZGVzY19zdGF0 ZVtoZWFkXS5udW07Cj4+PiArfQo+Pj4gKwo+Pj4gK3N0YXRpYyBpbmxpbmUgYm9vbCBtb3JlX3Vz ZWRfc3BsaXQoY29uc3Qgc3RydWN0IHZyaW5nX3ZpcnRxdWV1ZSAqdnEpCj4+PiAgICB7Cj4+PiAg ICAJcmV0dXJuIHZxLT5sYXN0X3VzZWRfaWR4ICE9IHZpcnRpbzE2X3RvX2NwdSh2cS0+dnEudmRl diwgdnEtPnZyaW5nLnVzZWQtPmlkeCk7Cj4+PiAgICB9Cj4+PiArc3RhdGljIGlubGluZSBib29s IG1vcmVfdXNlZF9wYWNrZWQoY29uc3Qgc3RydWN0IHZyaW5nX3ZpcnRxdWV1ZSAqdnEpCj4+PiAr ewo+Pj4gKwl1MTYgbGFzdF91c2VkLCBmbGFnczsKPj4+ICsJYm9vbCBhdmFpbCwgdXNlZDsKPj4+ ICsKPj4+ICsJaWYgKHZxLT52cS5udW1fZnJlZSA9PSB2cS0+dnJpbmdfcGFja2VkLm51bSkKPj4+ ICsJCXJldHVybiBmYWxzZTsKPj4+ICsKPj4+ICsJbGFzdF91c2VkID0gdnEtPmxhc3RfdXNlZF9p ZHg7Cj4+PiArCWZsYWdzID0gdmlydGlvMTZfdG9fY3B1KHZxLT52cS52ZGV2LAo+Pj4gKwkJCQl2 cS0+dnJpbmdfcGFja2VkLmRlc2NbbGFzdF91c2VkXS5mbGFncyk7Cj4+PiArCWF2YWlsID0gZmxh Z3MgJiBWUklOR19ERVNDX0ZfQVZBSUwoMSk7Cj4+PiArCXVzZWQgPSBmbGFncyAmIFZSSU5HX0RF U0NfRl9VU0VEKDEpOwo+Pj4gKwo+Pj4gKwlyZXR1cm4gYXZhaWwgPT0gdXNlZDsKPj4+ICt9Cj4+ IFRoaXMgbG9va3MgaW50ZXJlc3RpbmcsIHNwZWMgc2FpZDoKPj4KPj4gIgo+PiBUaHVzIFZJUlRR X0RFU0NfRl9BVkFJTCBhbmQgVklSVFFfREVTQ19GX1VTRUQgYml0cyBhcmUgZGlmZmVyZW50IGZv ciBhbgo+PiBhdmFpbGFibGUgZGVzY3JpcHRvciBhbmQKPj4gZXF1YWwgZm9yIGEgdXNlZCBkZXNj cmlwdG9yLgo+PiBOb3RlIHRoYXQgdGhpcyBvYnNlcnZhdGlvbiBpcyBtb3N0bHkgdXNlZnVsIGZv ciBzYW5pdHktY2hlY2tpbmcgYXMgdGhlc2UgYXJlCj4+IG5lY2Vzc2FyeSBidXQgbm90IHN1ZmZp Y2llbnQKPj4gY29uZGl0aW9ucyAtIGZvciBleGFtcGxlLCBhbGwgZGVzY3JpcHRvcnMgYXJlIHpl cm8taW5pdGlhbGl6ZWQuIFRvIGRldGVjdAo+PiB1c2VkIGFuZCBhdmFpbGFibGUgZGVzY3JpcHRv cnMgaXQgaXMKPj4gcG9zc2libGUgZm9yIGRyaXZlcnMgYW5kIGRldmljZXMgdG8ga2VlcCB0cmFj ayBvZiB0aGUgbGFzdCBvYnNlcnZlZCB2YWx1ZSBvZgo+PiBWSVJUUV9ERVNDX0ZfVVNFRC9WSVJU UV8tCj4+IERFU0NfRl9BVkFJTC4gT3RoZXIgdGVjaG5pcXVlcyB0byBkZXRlY3QKPj4gVklSVFFf REVTQ19GX0FWQUlML1ZJUlRRX0RFU0NfRl9VU0VEIGJpdCBjaGFuZ2VzCj4+IG1pZ2h0IGFsc28g YmUgcG9zc2libGUuCj4+ICIKPj4KPj4gU28gaXQgbG9va3MgdG8gbWUgaXQgd2FzIG5vdCBzdWZm aWNpZW50LCBsb29raW5nIGF0IHRoZSBleGFtcGxlIGNvZGVzIGluCj4+IHNwZWMsIGRvIHdlIG5l ZWQgdG8gdHJhY2sgbGFzdCBzZWVuIHVzZWRfd3JhcF9jb3VudGVyIGhlcmU/Cj4gSSBkb24ndCB0 aGluayB3ZSBoYXZlIHRvIHRyYWNrIHVzZWRfd3JhcF9jb3VudGVyIGluCj4gZHJpdmVyLiBUaGVy ZSB3YXMgYSBkaXNjdXNzaW9uIG9uIHRoaXM6Cj4KPiBodHRwczovL2xpc3RzLm9hc2lzLW9wZW4u b3JnL2FyY2hpdmVzL3ZpcnRpby1kZXYvMjAxODAyL21zZzAwMTc3Lmh0bWwKPgo+IEFuZCBhZnRl ciB0aGF0LCBiZWxvdyBzZW50ZW5jZSB3YXMgYWRkZWQgKGl0J3MgYWxzbwo+IGluIHRoZSBhYm92 ZSB3b3JkcyB5b3UgcXVvdGVkKToKPgo+ICIiIgo+IE90aGVyIHRlY2huaXF1ZXMgdG8gZGV0ZWN0 Cj4gVklSVFFfREVTQ19GX0FWQUlML1ZJUlRRX0RFU0NfRl9VU0VEIGJpdCBjaGFuZ2VzCj4gbWln aHQgYWxzbyBiZSBwb3NzaWJsZS4KPiAiIiIKPgo+IEJlc3QgcmVnYXJkcywKPiBUaXdlaSBCaWUK Ckkgc2VlLCB0aGUgZXh0cmEgY29uZGl0aW9uICJpZiAodnEtPnZxLm51bV9mcmVlID09IAp2cS0+ dnJpbmdfcGFja2VkLm51bSkiIGhlbHAgaW4gdGhpcyBjYXNlLgoKVGhhbmtzCgo+Cj4+IFRoYW5r cwoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KVmlydHVh bGl6YXRpb24gbWFpbGluZyBsaXN0ClZpcnR1YWxpemF0aW9uQGxpc3RzLmxpbnV4LWZvdW5kYXRp b24ub3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9uLm9yZy9tYWlsbWFuL2xpc3RpbmZv L3ZpcnR1YWxpemF0aW9u