From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03DC0C4360D for ; Mon, 9 Sep 2019 02:20:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C98BD218AC for ; Mon, 9 Sep 2019 02:20:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732723AbfIICTg (ORCPT ); Sun, 8 Sep 2019 22:19:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40768 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732156AbfIICTg (ORCPT ); Sun, 8 Sep 2019 22:19:36 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 222B0A46C10; Mon, 9 Sep 2019 02:19:35 +0000 (UTC) Received: from [10.72.12.61] (ovpn-12-61.pek2.redhat.com [10.72.12.61]) by smtp.corp.redhat.com (Postfix) with ESMTP id 16CA960925; Mon, 9 Sep 2019 02:19:05 +0000 (UTC) Subject: Re: [PATCH 2/2] vhost: re-introducing metadata acceleration through kernel virtual address To: "Michael S. Tsirkin" Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, jgg@mellanox.com, aarcange@redhat.com, jglisse@redhat.com, linux-mm@kvack.org, James Bottomley , Christoph Hellwig , David Miller , linux-arm-kernel@lists.infradead.org, linux-parisc@vger.kernel.org References: <20190905122736.19768-1-jasowang@redhat.com> <20190905122736.19768-3-jasowang@redhat.com> <20190908063618-mutt-send-email-mst@kernel.org> From: Jason Wang Message-ID: <1cb5aa8d-6213-5fce-5a77-fcada572c882@redhat.com> Date: Mon, 9 Sep 2019 10:18:57 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <20190908063618-mutt-send-email-mst@kernel.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.68]); Mon, 09 Sep 2019 02:19:35 +0000 (UTC) Sender: linux-parisc-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-parisc@vger.kernel.org On 2019/9/8 下午7:05, Michael S. Tsirkin wrote: > On Thu, Sep 05, 2019 at 08:27:36PM +0800, Jason Wang wrote: >> This is a rework on the commit 7f466032dc9e ("vhost: access vq >> metadata through kernel virtual address"). >> >> It was noticed that the copy_to/from_user() friends that was used to >> access virtqueue metdata tends to be very expensive for dataplane >> implementation like vhost since it involves lots of software checks, >> speculation barriers, > So if we drop speculation barrier, > there's a problem here in access will now be speculated. > This effectively disables the defence in depth effect of > b3bbfb3fb5d25776b8e3f361d2eedaabb0b496cd > x86: Introduce __uaccess_begin_nospec() and uaccess_try_nospec > > > So now we need to sprinkle array_index_nospec or barrier_nospec over the > code whenever we use an index we got from userspace. > See below for some examples. > > >> hardware feature toggling (e.g SMAP). The >> extra cost will be more obvious when transferring small packets since >> the time spent on metadata accessing become more significant. >> >> This patch tries to eliminate those overheads by accessing them >> through direct mapping of those pages. Invalidation callbacks is >> implemented for co-operation with general VM management (swap, KSM, >> THP or NUMA balancing). We will try to get the direct mapping of vq >> metadata before each round of packet processing if it doesn't >> exist. If we fail, we will simplely fallback to copy_to/from_user() >> friends. >> >> This invalidation, direct mapping access and set are synchronized >> through spinlock. This takes a step back from the original commit >> 7f466032dc9e ("vhost: access vq metadata through kernel virtual >> address") which tries to RCU which is suspicious and hard to be >> reviewed. This won't perform as well as RCU because of the atomic, >> this could be addressed by the future optimization. >> >> This method might does not work for high mem page which requires >> temporary mapping so we just fallback to normal >> copy_to/from_user() and may not for arch that has virtual tagged cache >> since extra cache flushing is needed to eliminate the alias. This will >> result complex logic and bad performance. For those archs, this patch >> simply go for copy_to/from_user() friends. This is done by ruling out >> kernel mapping codes through ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE. >> >> Note that this is only done when device IOTLB is not enabled. We >> could use similar method to optimize IOTLB in the future. >> >> Tests shows at most about 22% improvement on TX PPS when using >> virtio-user + vhost_net + xdp1 + TAP on 4.0GHz Kaby Lake. >> >> SMAP on | SMAP off >> Before: 4.9Mpps | 6.9Mpps >> After: 6.0Mpps | 7.5Mpps >> >> On a elder CPU Sandy Bridge without SMAP support. TX PPS doesn't see >> any difference. > Why is not Kaby Lake with SMAP off the same as Sandy Bridge? I don't know, I guess it was because the atomic is l > > >> Cc: Andrea Arcangeli >> Cc: James Bottomley >> Cc: Christoph Hellwig >> Cc: David Miller >> Cc: Jerome Glisse >> Cc: Jason Gunthorpe >> Cc: linux-mm@kvack.org >> Cc: linux-arm-kernel@lists.infradead.org >> Cc: linux-parisc@vger.kernel.org >> Signed-off-by: Jason Wang >> Signed-off-by: Michael S. Tsirkin >> --- >> drivers/vhost/vhost.c | 551 +++++++++++++++++++++++++++++++++++++++++- >> drivers/vhost/vhost.h | 41 ++++ >> 2 files changed, 589 insertions(+), 3 deletions(-) >> >> diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c >> index 791562e03fe0..f98155f28f02 100644 >> --- a/drivers/vhost/vhost.c >> +++ b/drivers/vhost/vhost.c >> @@ -298,6 +298,182 @@ static void vhost_vq_meta_reset(struct vhost_dev *d) >> __vhost_vq_meta_reset(d->vqs[i]); >> } >> >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> +static void vhost_map_unprefetch(struct vhost_map *map) >> +{ >> + kfree(map->pages); >> + kfree(map); >> +} >> + >> +static void vhost_set_map_dirty(struct vhost_virtqueue *vq, >> + struct vhost_map *map, int index) >> +{ >> + struct vhost_uaddr *uaddr = &vq->uaddrs[index]; >> + int i; >> + >> + if (uaddr->write) { >> + for (i = 0; i < map->npages; i++) >> + set_page_dirty(map->pages[i]); >> + } >> +} >> + >> +static void vhost_uninit_vq_maps(struct vhost_virtqueue *vq) >> +{ >> + struct vhost_map *map[VHOST_NUM_ADDRS]; >> + int i; >> + >> + spin_lock(&vq->mmu_lock); >> + for (i = 0; i < VHOST_NUM_ADDRS; i++) { >> + map[i] = vq->maps[i]; >> + if (map[i]) { >> + vhost_set_map_dirty(vq, map[i], i); >> + vq->maps[i] = NULL; >> + } >> + } >> + spin_unlock(&vq->mmu_lock); >> + >> + /* No need for synchronization since we are serialized with >> + * memory accessors (e.g vq mutex held). >> + */ >> + >> + for (i = 0; i < VHOST_NUM_ADDRS; i++) >> + if (map[i]) >> + vhost_map_unprefetch(map[i]); >> + >> +} >> + >> +static void vhost_reset_vq_maps(struct vhost_virtqueue *vq) >> +{ >> + int i; >> + >> + vhost_uninit_vq_maps(vq); >> + for (i = 0; i < VHOST_NUM_ADDRS; i++) >> + vq->uaddrs[i].size = 0; >> +} >> + >> +static bool vhost_map_range_overlap(struct vhost_uaddr *uaddr, >> + unsigned long start, >> + unsigned long end) >> +{ >> + if (unlikely(!uaddr->size)) >> + return false; >> + >> + return !(end < uaddr->uaddr || start > uaddr->uaddr - 1 + uaddr->size); >> +} >> + >> +static void inline vhost_vq_access_map_begin(struct vhost_virtqueue *vq) >> +{ >> + spin_lock(&vq->mmu_lock); >> +} >> + >> +static void inline vhost_vq_access_map_end(struct vhost_virtqueue *vq) >> +{ >> + spin_unlock(&vq->mmu_lock); >> +} >> + >> +static int vhost_invalidate_vq_start(struct vhost_virtqueue *vq, >> + int index, >> + unsigned long start, >> + unsigned long end, >> + bool blockable) >> +{ >> + struct vhost_uaddr *uaddr = &vq->uaddrs[index]; >> + struct vhost_map *map; >> + >> + if (!vhost_map_range_overlap(uaddr, start, end)) >> + return 0; >> + else if (!blockable) >> + return -EAGAIN; >> + >> + spin_lock(&vq->mmu_lock); >> + ++vq->invalidate_count; >> + >> + map = vq->maps[index]; >> + if (map) >> + vq->maps[index] = NULL; >> + spin_unlock(&vq->mmu_lock); >> + >> + if (map) { >> + vhost_set_map_dirty(vq, map, index); >> + vhost_map_unprefetch(map); >> + } >> + >> + return 0; >> +} >> + >> +static void vhost_invalidate_vq_end(struct vhost_virtqueue *vq, >> + int index, >> + unsigned long start, >> + unsigned long end) >> +{ >> + if (!vhost_map_range_overlap(&vq->uaddrs[index], start, end)) >> + return; >> + >> + spin_lock(&vq->mmu_lock); >> + --vq->invalidate_count; >> + spin_unlock(&vq->mmu_lock); >> +} >> + >> +static int vhost_invalidate_range_start(struct mmu_notifier *mn, >> + const struct mmu_notifier_range *range) >> +{ >> + struct vhost_dev *dev = container_of(mn, struct vhost_dev, >> + mmu_notifier); >> + bool blockable = mmu_notifier_range_blockable(range); >> + int i, j, ret; >> + >> + for (i = 0; i < dev->nvqs; i++) { >> + struct vhost_virtqueue *vq = dev->vqs[i]; >> + >> + for (j = 0; j < VHOST_NUM_ADDRS; j++) { >> + ret = vhost_invalidate_vq_start(vq, j, >> + range->start, >> + range->end, blockable); >> + if (ret) >> + return ret; >> + } >> + } >> + >> + return 0; >> +} >> + >> +static void vhost_invalidate_range_end(struct mmu_notifier *mn, >> + const struct mmu_notifier_range *range) >> +{ >> + struct vhost_dev *dev = container_of(mn, struct vhost_dev, >> + mmu_notifier); >> + int i, j; >> + >> + for (i = 0; i < dev->nvqs; i++) { >> + struct vhost_virtqueue *vq = dev->vqs[i]; >> + >> + for (j = 0; j < VHOST_NUM_ADDRS; j++) >> + vhost_invalidate_vq_end(vq, j, >> + range->start, >> + range->end); >> + } >> +} >> + >> +static const struct mmu_notifier_ops vhost_mmu_notifier_ops = { >> + .invalidate_range_start = vhost_invalidate_range_start, >> + .invalidate_range_end = vhost_invalidate_range_end, >> +}; >> + >> +static void vhost_init_maps(struct vhost_dev *dev) >> +{ >> + struct vhost_virtqueue *vq; >> + int i, j; >> + >> + dev->mmu_notifier.ops = &vhost_mmu_notifier_ops; >> + >> + for (i = 0; i < dev->nvqs; ++i) { >> + vq = dev->vqs[i]; >> + for (j = 0; j < VHOST_NUM_ADDRS; j++) >> + vq->maps[j] = NULL; >> + } >> +} >> +#endif >> + >> static void vhost_vq_reset(struct vhost_dev *dev, >> struct vhost_virtqueue *vq) >> { >> @@ -326,7 +502,11 @@ static void vhost_vq_reset(struct vhost_dev *dev, >> vq->busyloop_timeout = 0; >> vq->umem = NULL; >> vq->iotlb = NULL; >> + vq->invalidate_count = 0; >> __vhost_vq_meta_reset(vq); >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> + vhost_reset_vq_maps(vq); >> +#endif >> } >> >> static int vhost_worker(void *data) >> @@ -471,12 +651,15 @@ void vhost_dev_init(struct vhost_dev *dev, >> dev->iov_limit = iov_limit; >> dev->weight = weight; >> dev->byte_weight = byte_weight; >> + dev->has_notifier = false; >> init_llist_head(&dev->work_list); >> init_waitqueue_head(&dev->wait); >> INIT_LIST_HEAD(&dev->read_list); >> INIT_LIST_HEAD(&dev->pending_list); >> spin_lock_init(&dev->iotlb_lock); >> - >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> + vhost_init_maps(dev); >> +#endif >> >> for (i = 0; i < dev->nvqs; ++i) { >> vq = dev->vqs[i]; >> @@ -485,6 +668,7 @@ void vhost_dev_init(struct vhost_dev *dev, >> vq->heads = NULL; >> vq->dev = dev; >> mutex_init(&vq->mutex); >> + spin_lock_init(&vq->mmu_lock); >> vhost_vq_reset(dev, vq); >> if (vq->handle_kick) >> vhost_poll_init(&vq->poll, vq->handle_kick, >> @@ -564,7 +748,19 @@ long vhost_dev_set_owner(struct vhost_dev *dev) >> if (err) >> goto err_cgroup; >> >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> + err = mmu_notifier_register(&dev->mmu_notifier, dev->mm); >> + if (err) >> + goto err_mmu_notifier; >> +#endif >> + dev->has_notifier = true; >> + >> return 0; >> + >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> +err_mmu_notifier: >> + vhost_dev_free_iovecs(dev); >> +#endif >> err_cgroup: >> kthread_stop(worker); >> dev->worker = NULL; >> @@ -655,6 +851,107 @@ static void vhost_clear_msg(struct vhost_dev *dev) >> spin_unlock(&dev->iotlb_lock); >> } >> >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> +static void vhost_setup_uaddr(struct vhost_virtqueue *vq, >> + int index, unsigned long uaddr, >> + size_t size, bool write) >> +{ >> + struct vhost_uaddr *addr = &vq->uaddrs[index]; >> + >> + addr->uaddr = uaddr; >> + addr->size = size; >> + addr->write = write; >> +} >> + >> +static void vhost_setup_vq_uaddr(struct vhost_virtqueue *vq) >> +{ >> + vhost_setup_uaddr(vq, VHOST_ADDR_DESC, >> + (unsigned long)vq->desc, >> + vhost_get_desc_size(vq, vq->num), >> + false); >> + vhost_setup_uaddr(vq, VHOST_ADDR_AVAIL, >> + (unsigned long)vq->avail, >> + vhost_get_avail_size(vq, vq->num), >> + false); >> + vhost_setup_uaddr(vq, VHOST_ADDR_USED, >> + (unsigned long)vq->used, >> + vhost_get_used_size(vq, vq->num), >> + true); >> +} >> + >> +static int vhost_map_prefetch(struct vhost_virtqueue *vq, >> + int index) >> +{ >> + struct vhost_map *map; >> + struct vhost_uaddr *uaddr = &vq->uaddrs[index]; >> + struct page **pages; >> + int npages = DIV_ROUND_UP(uaddr->size, PAGE_SIZE); >> + int npinned; >> + void *vaddr, *v; >> + int err; >> + int i; >> + >> + spin_lock(&vq->mmu_lock); >> + >> + err = -EFAULT; >> + if (vq->invalidate_count) >> + goto err; >> + >> + err = -ENOMEM; >> + map = kmalloc(sizeof(*map), GFP_ATOMIC); >> + if (!map) >> + goto err; >> + >> + pages = kmalloc_array(npages, sizeof(struct page *), GFP_ATOMIC); >> + if (!pages) >> + goto err_pages; >> + >> + err = EFAULT; >> + npinned = __get_user_pages_fast(uaddr->uaddr, npages, >> + uaddr->write, pages); >> + if (npinned > 0) >> + release_pages(pages, npinned); >> + if (npinned != npages) >> + goto err_gup; >> + >> + for (i = 0; i < npinned; i++) >> + if (PageHighMem(pages[i])) >> + goto err_gup; >> + >> + vaddr = v = page_address(pages[0]); >> + >> + /* For simplicity, fallback to userspace address if VA is not >> + * contigious. >> + */ >> + for (i = 1; i < npinned; i++) { >> + v += PAGE_SIZE; >> + if (v != page_address(pages[i])) >> + goto err_gup; >> + } >> + >> + map->addr = vaddr + (uaddr->uaddr & (PAGE_SIZE - 1)); >> + map->npages = npages; >> + map->pages = pages; >> + >> + vq->maps[index] = map; >> + /* No need for a synchronize_rcu(). This function should be >> + * called by dev->worker so we are serialized with all >> + * readers. >> + */ >> + spin_unlock(&vq->mmu_lock); >> + >> + return 0; >> + >> +err_gup: >> + kfree(pages); >> +err_pages: >> + kfree(map); >> +err: >> + spin_unlock(&vq->mmu_lock); >> + return err; >> +} >> +#endif >> + >> void vhost_dev_cleanup(struct vhost_dev *dev) >> { >> int i; >> @@ -684,8 +981,20 @@ void vhost_dev_cleanup(struct vhost_dev *dev) >> kthread_stop(dev->worker); >> dev->worker = NULL; >> } >> - if (dev->mm) >> + if (dev->mm) { >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> + if (dev->has_notifier) { >> + mmu_notifier_unregister(&dev->mmu_notifier, >> + dev->mm); >> + dev->has_notifier = false; >> + } >> +#endif >> mmput(dev->mm); >> + } >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> + for (i = 0; i < dev->nvqs; i++) >> + vhost_uninit_vq_maps(dev->vqs[i]); >> +#endif >> dev->mm = NULL; >> } >> EXPORT_SYMBOL_GPL(vhost_dev_cleanup); >> @@ -914,6 +1223,26 @@ static inline void __user *__vhost_get_user(struct vhost_virtqueue *vq, >> >> static inline int vhost_put_avail_event(struct vhost_virtqueue *vq) >> { >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> + struct vhost_map *map; >> + struct vring_used *used; >> + >> + if (!vq->iotlb) { >> + vhost_vq_access_map_begin(vq); >> + >> + map = vq->maps[VHOST_ADDR_USED]; >> + if (likely(map)) { >> + used = map->addr; >> + *((__virtio16 *)&used->ring[vq->num]) = >> + cpu_to_vhost16(vq, vq->avail_idx); >> + vhost_vq_access_map_end(vq); >> + return 0; >> + } >> + >> + vhost_vq_access_map_end(vq); >> + } >> +#endif >> + >> return vhost_put_user(vq, cpu_to_vhost16(vq, vq->avail_idx), >> vhost_avail_event(vq)); >> } >> @@ -922,6 +1251,27 @@ static inline int vhost_put_used(struct vhost_virtqueue *vq, >> struct vring_used_elem *head, int idx, >> int count) >> { >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> + struct vhost_map *map; >> + struct vring_used *used; >> + size_t size; >> + >> + if (!vq->iotlb) { >> + vhost_vq_access_map_begin(vq); >> + >> + map = vq->maps[VHOST_ADDR_USED]; >> + if (likely(map)) { >> + used = map->addr; >> + size = count * sizeof(*head); >> + memcpy(used->ring + idx, head, size); >> + vhost_vq_access_map_end(vq); >> + return 0; >> + } >> + >> + vhost_vq_access_map_end(vq); >> + } >> +#endif >> + >> return vhost_copy_to_user(vq, vq->used->ring + idx, head, >> count * sizeof(*head)); >> } >> @@ -929,6 +1279,25 @@ static inline int vhost_put_used(struct vhost_virtqueue *vq, >> static inline int vhost_put_used_flags(struct vhost_virtqueue *vq) >> >> { >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> + struct vhost_map *map; >> + struct vring_used *used; >> + >> + if (!vq->iotlb) { >> + vhost_vq_access_map_begin(vq); >> + >> + map = vq->maps[VHOST_ADDR_USED]; >> + if (likely(map)) { >> + used = map->addr; >> + used->flags = cpu_to_vhost16(vq, vq->used_flags); >> + vhost_vq_access_map_end(vq); >> + return 0; >> + } >> + >> + vhost_vq_access_map_end(vq); >> + } >> +#endif >> + >> return vhost_put_user(vq, cpu_to_vhost16(vq, vq->used_flags), >> &vq->used->flags); >> } >> @@ -936,6 +1305,25 @@ static inline int vhost_put_used_flags(struct vhost_virtqueue *vq) >> static inline int vhost_put_used_idx(struct vhost_virtqueue *vq) >> >> { >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> + struct vhost_map *map; >> + struct vring_used *used; >> + >> + if (!vq->iotlb) { >> + vhost_vq_access_map_begin(vq); >> + >> + map = vq->maps[VHOST_ADDR_USED]; >> + if (likely(map)) { >> + used = map->addr; >> + used->idx = cpu_to_vhost16(vq, vq->last_used_idx); >> + vhost_vq_access_map_end(vq); >> + return 0; >> + } >> + >> + vhost_vq_access_map_end(vq); >> + } >> +#endif >> + >> return vhost_put_user(vq, cpu_to_vhost16(vq, vq->last_used_idx), >> &vq->used->idx); >> } >> @@ -981,12 +1369,50 @@ static void vhost_dev_unlock_vqs(struct vhost_dev *d) >> static inline int vhost_get_avail_idx(struct vhost_virtqueue *vq, >> __virtio16 *idx) >> { >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> + struct vhost_map *map; >> + struct vring_avail *avail; >> + >> + if (!vq->iotlb) { >> + vhost_vq_access_map_begin(vq); >> + >> + map = vq->maps[VHOST_ADDR_AVAIL]; >> + if (likely(map)) { >> + avail = map->addr; >> + *idx = avail->idx; > index can now be speculated. [...] > + vhost_vq_access_map_begin(vq); > + > + map = vq->maps[VHOST_ADDR_AVAIL]; > + if (likely(map)) { > + avail = map->addr; > + *head = avail->ring[idx & (vq->num - 1)]; > > Since idx can be speculated, I guess we need array_index_nospec here? So we have ACQUIRE(mmu_lock) get idx RELEASE(mmu_lock) ACQUIRE(mmu_lock) read array[idx] RELEASE(mmu_lock) Then I think idx can't be speculated consider we've passed RELEASE + ACQUIRE? > > >> + vhost_vq_access_map_end(vq); >> + return 0; >> + } >> + >> + vhost_vq_access_map_end(vq); >> + } >> +#endif >> + >> return vhost_get_avail(vq, *head, >> &vq->avail->ring[idx & (vq->num - 1)]); >> } >> @@ -994,24 +1420,98 @@ static inline int vhost_get_avail_head(struct vhost_virtqueue *vq, >> static inline int vhost_get_avail_flags(struct vhost_virtqueue *vq, >> __virtio16 *flags) >> { >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> + struct vhost_map *map; >> + struct vring_avail *avail; >> + >> + if (!vq->iotlb) { >> + vhost_vq_access_map_begin(vq); >> + >> + map = vq->maps[VHOST_ADDR_AVAIL]; >> + if (likely(map)) { >> + avail = map->addr; >> + *flags = avail->flags; >> + vhost_vq_access_map_end(vq); >> + return 0; >> + } >> + >> + vhost_vq_access_map_end(vq); >> + } >> +#endif >> + >> return vhost_get_avail(vq, *flags, &vq->avail->flags); >> } >> >> static inline int vhost_get_used_event(struct vhost_virtqueue *vq, >> __virtio16 *event) >> { >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> + struct vhost_map *map; >> + struct vring_avail *avail; >> + >> + if (!vq->iotlb) { >> + vhost_vq_access_map_begin(vq); >> + map = vq->maps[VHOST_ADDR_AVAIL]; >> + if (likely(map)) { >> + avail = map->addr; >> + *event = (__virtio16)avail->ring[vq->num]; >> + vhost_vq_access_map_end(vq); >> + return 0; >> + } >> + vhost_vq_access_map_end(vq); >> + } >> +#endif >> + >> return vhost_get_avail(vq, *event, vhost_used_event(vq)); >> } >> >> static inline int vhost_get_used_idx(struct vhost_virtqueue *vq, >> __virtio16 *idx) >> { >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> + struct vhost_map *map; >> + struct vring_used *used; >> + >> + if (!vq->iotlb) { >> + vhost_vq_access_map_begin(vq); >> + >> + map = vq->maps[VHOST_ADDR_USED]; >> + if (likely(map)) { >> + used = map->addr; >> + *idx = used->idx; >> + vhost_vq_access_map_end(vq); >> + return 0; >> + } >> + >> + vhost_vq_access_map_end(vq); >> + } >> +#endif >> + >> return vhost_get_used(vq, *idx, &vq->used->idx); >> } > > This seems to be used during init. Why do we bother > accelerating this? Ok, I can remove this part in next version. > > >> >> static inline int vhost_get_desc(struct vhost_virtqueue *vq, >> struct vring_desc *desc, int idx) >> { >> +#if VHOST_ARCH_CAN_ACCEL_UACCESS >> + struct vhost_map *map; >> + struct vring_desc *d; >> + >> + if (!vq->iotlb) { >> + vhost_vq_access_map_begin(vq); >> + >> + map = vq->maps[VHOST_ADDR_DESC]; >> + if (likely(map)) { >> + d = map->addr; >> + *desc = *(d + idx); > > Since idx can be speculated, I guess we need array_index_nospec here? This is similar to the above avail idx case. > > >> + vhost_vq_access_map_end(vq); >> + return 0; >> + } >> + >> + vhost_vq_access_map_end(vq); >> + } >> +#endif >> + >> return vhost_copy_from_user(vq, desc, vq->desc + idx, sizeof(*desc)); >> } >> > I also wonder about the userspace address we get eventualy. > It would seem that we need to prevent that from speculating - > and that seems like a good idea even if this patch isn't > applied. As you are playing with micro-benchmarks, maybe > you could the below patch? Let me test it. Thanks > It's unfortunately untested. > Thanks a lot in advance! > > ===> > vhost: block speculation of translated descriptors > > iovec addresses coming from vhost are assumed to be > pre-validated, but in fact can be speculated to a value > out of range. > > Userspace address are later validated with array_index_nospec so we can > be sure kernel info does not leak through these addresses, but vhost > must also not leak userspace info outside the allowed memory table to > guests. > > Following the defence in depth principle, make sure > the address is not validated out of node range. > > Signed-off-by: Michael S. Tsirkin > > --- > > > diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c > index 5dc174ac8cac..863e25011ef6 100644 > --- a/drivers/vhost/vhost.c > +++ b/drivers/vhost/vhost.c > @@ -2072,7 +2076,9 @@ static int translate_desc(struct vhost_virtqueue *vq, u64 addr, u32 len, > size = node->size - addr + node->start; > _iov->iov_len = min((u64)len - s, size); > _iov->iov_base = (void __user *)(unsigned long) > - (node->userspace_addr + addr - node->start); > + (node->userspace_addr + > + array_index_nospec(addr - node->start, > + node->size)); > s += size; > addr += size; > ++ret; From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.3 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B44D2C4360D for ; Mon, 9 Sep 2019 02:19:51 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 84318216C8 for ; Mon, 9 Sep 2019 02:19:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="ArjNywLI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 84318216C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=eee9FJaJDaux2NTS+WQHZHKUoAuDpHGKnoAMKn+LVbs=; b=ArjNywLI6SJeofkQOzQC6bJ1s iCTloigKz5qYJ9k3ZEHWEy8a2/Z11R0ZxaAvkBuazpv01cLqL716NbopPL/iBx3aS8P0VuD3ORGe6 skkB2xSCf8iJHylHtY3bSkL5UhWMjpN83/3mI00FS2S1LeXH3WUV7Dnos0zzMOIFfd5P3+o88W5Fd dEG3HaHXhzy3BupyPYRhr02qdtDeLZkGLQLRynhms5jUgH+tB92ZUvg8hJcvB3M7+6DKRwKlbKycx FNFzcnVhjxUo/L7Y4kD3PGDVUDtNxp63ZdJf7Fld5mmx/Nco0kKmcgvN0Ho8AGVYIlAHu6w+NDR7I Xm+MMnXJw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1i79HJ-00031k-IY; Mon, 09 Sep 2019 02:19:41 +0000 Received: from mx1.redhat.com ([209.132.183.28]) by bombadil.infradead.org with esmtps (Exim 4.92 #3 (Red Hat Linux)) id 1i79HE-000315-Ue for linux-arm-kernel@lists.infradead.org; Mon, 09 Sep 2019 02:19:39 +0000 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 222B0A46C10; Mon, 9 Sep 2019 02:19:35 +0000 (UTC) Received: from [10.72.12.61] (ovpn-12-61.pek2.redhat.com [10.72.12.61]) by smtp.corp.redhat.com (Postfix) with ESMTP id 16CA960925; Mon, 9 Sep 2019 02:19:05 +0000 (UTC) Subject: Re: [PATCH 2/2] vhost: re-introducing metadata acceleration through kernel virtual address To: "Michael S. Tsirkin" References: <20190905122736.19768-1-jasowang@redhat.com> <20190905122736.19768-3-jasowang@redhat.com> <20190908063618-mutt-send-email-mst@kernel.org> From: Jason Wang Message-ID: <1cb5aa8d-6213-5fce-5a77-fcada572c882@redhat.com> Date: Mon, 9 Sep 2019 10:18:57 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <20190908063618-mutt-send-email-mst@kernel.org> Content-Language: en-US X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.68]); Mon, 09 Sep 2019 02:19:35 +0000 (UTC) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190908_191937_042475_64FF4166 X-CRM114-Status: GOOD ( 31.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com, Christoph Hellwig , linux-parisc@vger.kernel.org, kvm@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, James Bottomley , linux-mm@kvack.org, jglisse@redhat.com, jgg@mellanox.com, David Miller , linux-arm-kernel@lists.infradead.org Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org Ck9uIDIwMTkvOS84IOS4i+WNiDc6MDUsIE1pY2hhZWwgUy4gVHNpcmtpbiB3cm90ZToKPiBPbiBU aHUsIFNlcCAwNSwgMjAxOSBhdCAwODoyNzozNlBNICswODAwLCBKYXNvbiBXYW5nIHdyb3RlOgo+ PiBUaGlzIGlzIGEgcmV3b3JrIG9uIHRoZSBjb21taXQgN2Y0NjYwMzJkYzllICgidmhvc3Q6IGFj Y2VzcyB2cQo+PiBtZXRhZGF0YSB0aHJvdWdoIGtlcm5lbCB2aXJ0dWFsIGFkZHJlc3MiKS4KPj4K Pj4gSXQgd2FzIG5vdGljZWQgdGhhdCB0aGUgY29weV90by9mcm9tX3VzZXIoKSBmcmllbmRzIHRo YXQgd2FzIHVzZWQgdG8KPj4gYWNjZXNzIHZpcnRxdWV1ZSBtZXRkYXRhIHRlbmRzIHRvIGJlIHZl cnkgZXhwZW5zaXZlIGZvciBkYXRhcGxhbmUKPj4gaW1wbGVtZW50YXRpb24gbGlrZSB2aG9zdCBz aW5jZSBpdCBpbnZvbHZlcyBsb3RzIG9mIHNvZnR3YXJlIGNoZWNrcywKPj4gc3BlY3VsYXRpb24g YmFycmllcnMsCj4gU28gaWYgd2UgZHJvcCBzcGVjdWxhdGlvbiBiYXJyaWVyLAo+IHRoZXJlJ3Mg YSBwcm9ibGVtIGhlcmUgaW4gYWNjZXNzIHdpbGwgbm93IGJlIHNwZWN1bGF0ZWQuCj4gVGhpcyBl ZmZlY3RpdmVseSBkaXNhYmxlcyB0aGUgZGVmZW5jZSBpbiBkZXB0aCBlZmZlY3Qgb2YKPiBiM2Ji ZmIzZmI1ZDI1Nzc2YjhlM2YzNjFkMmVlZGFhYmIwYjQ5NmNkCj4gICAgICB4ODY6IEludHJvZHVj ZSBfX3VhY2Nlc3NfYmVnaW5fbm9zcGVjKCkgYW5kIHVhY2Nlc3NfdHJ5X25vc3BlYwo+Cj4KPiBT byBub3cgd2UgbmVlZCB0byBzcHJpbmtsZSBhcnJheV9pbmRleF9ub3NwZWMgb3IgYmFycmllcl9u b3NwZWMgb3ZlciB0aGUKPiBjb2RlIHdoZW5ldmVyIHdlIHVzZSBhbiBpbmRleCB3ZSBnb3QgZnJv bSB1c2Vyc3BhY2UuCj4gU2VlIGJlbG93IGZvciBzb21lIGV4YW1wbGVzLgo+Cj4KPj4gaGFyZHdh cmUgZmVhdHVyZSB0b2dnbGluZyAoZS5nIFNNQVApLiBUaGUKPj4gZXh0cmEgY29zdCB3aWxsIGJl IG1vcmUgb2J2aW91cyB3aGVuIHRyYW5zZmVycmluZyBzbWFsbCBwYWNrZXRzIHNpbmNlCj4+IHRo ZSB0aW1lIHNwZW50IG9uIG1ldGFkYXRhIGFjY2Vzc2luZyBiZWNvbWUgbW9yZSBzaWduaWZpY2Fu dC4KPj4KPj4gVGhpcyBwYXRjaCB0cmllcyB0byBlbGltaW5hdGUgdGhvc2Ugb3ZlcmhlYWRzIGJ5 IGFjY2Vzc2luZyB0aGVtCj4+IHRocm91Z2ggZGlyZWN0IG1hcHBpbmcgb2YgdGhvc2UgcGFnZXMu IEludmFsaWRhdGlvbiBjYWxsYmFja3MgaXMKPj4gaW1wbGVtZW50ZWQgZm9yIGNvLW9wZXJhdGlv biB3aXRoIGdlbmVyYWwgVk0gbWFuYWdlbWVudCAoc3dhcCwgS1NNLAo+PiBUSFAgb3IgTlVNQSBi YWxhbmNpbmcpLiBXZSB3aWxsIHRyeSB0byBnZXQgdGhlIGRpcmVjdCBtYXBwaW5nIG9mIHZxCj4+ IG1ldGFkYXRhIGJlZm9yZSBlYWNoIHJvdW5kIG9mIHBhY2tldCBwcm9jZXNzaW5nIGlmIGl0IGRv ZXNuJ3QKPj4gZXhpc3QuIElmIHdlIGZhaWwsIHdlIHdpbGwgc2ltcGxlbHkgZmFsbGJhY2sgdG8g Y29weV90by9mcm9tX3VzZXIoKQo+PiBmcmllbmRzLgo+Pgo+PiBUaGlzIGludmFsaWRhdGlvbiwg ZGlyZWN0IG1hcHBpbmcgYWNjZXNzIGFuZCBzZXQgYXJlIHN5bmNocm9uaXplZAo+PiB0aHJvdWdo IHNwaW5sb2NrLiBUaGlzIHRha2VzIGEgc3RlcCBiYWNrIGZyb20gdGhlIG9yaWdpbmFsIGNvbW1p dAo+PiA3ZjQ2NjAzMmRjOWUgKCJ2aG9zdDogYWNjZXNzIHZxIG1ldGFkYXRhIHRocm91Z2gga2Vy bmVsIHZpcnR1YWwKPj4gYWRkcmVzcyIpIHdoaWNoIHRyaWVzIHRvIFJDVSB3aGljaCBpcyBzdXNw aWNpb3VzIGFuZCBoYXJkIHRvIGJlCj4+IHJldmlld2VkLiBUaGlzIHdvbid0IHBlcmZvcm0gYXMg d2VsbCBhcyBSQ1UgYmVjYXVzZSBvZiB0aGUgYXRvbWljLAo+PiB0aGlzIGNvdWxkIGJlIGFkZHJl c3NlZCBieSB0aGUgZnV0dXJlIG9wdGltaXphdGlvbi4KPj4KPj4gVGhpcyBtZXRob2QgbWlnaHQg ZG9lcyBub3Qgd29yayBmb3IgaGlnaCBtZW0gcGFnZSB3aGljaCByZXF1aXJlcwo+PiB0ZW1wb3Jh cnkgbWFwcGluZyBzbyB3ZSBqdXN0IGZhbGxiYWNrIHRvIG5vcm1hbAo+PiBjb3B5X3RvL2Zyb21f dXNlcigpIGFuZCBtYXkgbm90IGZvciBhcmNoIHRoYXQgaGFzIHZpcnR1YWwgdGFnZ2VkIGNhY2hl Cj4+IHNpbmNlIGV4dHJhIGNhY2hlIGZsdXNoaW5nIGlzIG5lZWRlZCB0byBlbGltaW5hdGUgdGhl IGFsaWFzLiBUaGlzIHdpbGwKPj4gcmVzdWx0IGNvbXBsZXggbG9naWMgYW5kIGJhZCBwZXJmb3Jt YW5jZS4gRm9yIHRob3NlIGFyY2hzLCB0aGlzIHBhdGNoCj4+IHNpbXBseSBnbyBmb3IgY29weV90 by9mcm9tX3VzZXIoKSBmcmllbmRzLiBUaGlzIGlzIGRvbmUgYnkgcnVsaW5nIG91dAo+PiBrZXJu ZWwgbWFwcGluZyBjb2RlcyB0aHJvdWdoIEFSQ0hfSU1QTEVNRU5UU19GTFVTSF9EQ0FDSEVfUEFH RS4KPj4KPj4gTm90ZSB0aGF0IHRoaXMgaXMgb25seSBkb25lIHdoZW4gZGV2aWNlIElPVExCIGlz IG5vdCBlbmFibGVkLiBXZQo+PiBjb3VsZCB1c2Ugc2ltaWxhciBtZXRob2QgdG8gb3B0aW1pemUg SU9UTEIgaW4gdGhlIGZ1dHVyZS4KPj4KPj4gVGVzdHMgc2hvd3MgYXQgbW9zdCBhYm91dCAyMiUg aW1wcm92ZW1lbnQgb24gVFggUFBTIHdoZW4gdXNpbmcKPj4gdmlydGlvLXVzZXIgKyB2aG9zdF9u ZXQgKyB4ZHAxICsgVEFQIG9uIDQuMEdIeiBLYWJ5IExha2UuCj4+Cj4+ICAgICAgICAgIFNNQVAg b24gfCBTTUFQIG9mZgo+PiBCZWZvcmU6IDQuOU1wcHMgfCA2LjlNcHBzCj4+IEFmdGVyOiAgNi4w TXBwcyB8IDcuNU1wcHMKPj4KPj4gT24gYSBlbGRlciBDUFUgU2FuZHkgQnJpZGdlIHdpdGhvdXQg U01BUCBzdXBwb3J0LiBUWCBQUFMgZG9lc24ndCBzZWUKPj4gYW55IGRpZmZlcmVuY2UuCj4gV2h5 IGlzIG5vdCBLYWJ5IExha2Ugd2l0aCBTTUFQIG9mZiB0aGUgc2FtZSBhcyBTYW5keSBCcmlkZ2U/ CgoKSSBkb24ndCBrbm93LCBJIGd1ZXNzIGl0IHdhcyBiZWNhdXNlIHRoZSBhdG9taWMgaXMgbAoK Cj4KPgo+PiBDYzogQW5kcmVhIEFyY2FuZ2VsaSA8YWFyY2FuZ2VAcmVkaGF0LmNvbT4KPj4gQ2M6 IEphbWVzIEJvdHRvbWxleSA8SmFtZXMuQm90dG9tbGV5QGhhbnNlbnBhcnRuZXJzaGlwLmNvbT4K Pj4gQ2M6IENocmlzdG9waCBIZWxsd2lnIDxoY2hAaW5mcmFkZWFkLm9yZz4KPj4gQ2M6IERhdmlk IE1pbGxlciA8ZGF2ZW1AZGF2ZW1sb2Z0Lm5ldD4KPj4gQ2M6IEplcm9tZSBHbGlzc2UgPGpnbGlz c2VAcmVkaGF0LmNvbT4KPj4gQ2M6IEphc29uIEd1bnRob3JwZSA8amdnQG1lbGxhbm94LmNvbT4K Pj4gQ2M6IGxpbnV4LW1tQGt2YWNrLm9yZwo+PiBDYzogbGludXgtYXJtLWtlcm5lbEBsaXN0cy5p bmZyYWRlYWQub3JnCj4+IENjOiBsaW51eC1wYXJpc2NAdmdlci5rZXJuZWwub3JnCj4+IFNpZ25l ZC1vZmYtYnk6IEphc29uIFdhbmcgPGphc293YW5nQHJlZGhhdC5jb20+Cj4+IFNpZ25lZC1vZmYt Ynk6IE1pY2hhZWwgUy4gVHNpcmtpbiA8bXN0QHJlZGhhdC5jb20+Cj4+IC0tLQo+PiAgIGRyaXZl cnMvdmhvc3Qvdmhvc3QuYyB8IDU1MSArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr KysrKysrKy0KPj4gICBkcml2ZXJzL3Zob3N0L3Zob3N0LmggfCAgNDEgKysrKwo+PiAgIDIgZmls ZXMgY2hhbmdlZCwgNTg5IGluc2VydGlvbnMoKyksIDMgZGVsZXRpb25zKC0pCj4+Cj4+IGRpZmYg LS1naXQgYS9kcml2ZXJzL3Zob3N0L3Zob3N0LmMgYi9kcml2ZXJzL3Zob3N0L3Zob3N0LmMKPj4g aW5kZXggNzkxNTYyZTAzZmUwLi5mOTgxNTVmMjhmMDIgMTAwNjQ0Cj4+IC0tLSBhL2RyaXZlcnMv dmhvc3Qvdmhvc3QuYwo+PiArKysgYi9kcml2ZXJzL3Zob3N0L3Zob3N0LmMKPj4gQEAgLTI5OCw2 ICsyOTgsMTgyIEBAIHN0YXRpYyB2b2lkIHZob3N0X3ZxX21ldGFfcmVzZXQoc3RydWN0IHZob3N0 X2RldiAqZCkKPj4gICAJCV9fdmhvc3RfdnFfbWV0YV9yZXNldChkLT52cXNbaV0pOwo+PiAgIH0K Pj4gICAKPj4gKyNpZiBWSE9TVF9BUkNIX0NBTl9BQ0NFTF9VQUNDRVNTCj4+ICtzdGF0aWMgdm9p ZCB2aG9zdF9tYXBfdW5wcmVmZXRjaChzdHJ1Y3Qgdmhvc3RfbWFwICptYXApCj4+ICt7Cj4+ICsJ a2ZyZWUobWFwLT5wYWdlcyk7Cj4+ICsJa2ZyZWUobWFwKTsKPj4gK30KPj4gKwo+PiArc3RhdGlj IHZvaWQgdmhvc3Rfc2V0X21hcF9kaXJ0eShzdHJ1Y3Qgdmhvc3RfdmlydHF1ZXVlICp2cSwKPj4g KwkJCQlzdHJ1Y3Qgdmhvc3RfbWFwICptYXAsIGludCBpbmRleCkKPj4gK3sKPj4gKwlzdHJ1Y3Qg dmhvc3RfdWFkZHIgKnVhZGRyID0gJnZxLT51YWRkcnNbaW5kZXhdOwo+PiArCWludCBpOwo+PiAr Cj4+ICsJaWYgKHVhZGRyLT53cml0ZSkgewo+PiArCQlmb3IgKGkgPSAwOyBpIDwgbWFwLT5ucGFn ZXM7IGkrKykKPj4gKwkJCXNldF9wYWdlX2RpcnR5KG1hcC0+cGFnZXNbaV0pOwo+PiArCX0KPj4g K30KPj4gKwo+PiArc3RhdGljIHZvaWQgdmhvc3RfdW5pbml0X3ZxX21hcHMoc3RydWN0IHZob3N0 X3ZpcnRxdWV1ZSAqdnEpCj4+ICt7Cj4+ICsJc3RydWN0IHZob3N0X21hcCAqbWFwW1ZIT1NUX05V TV9BRERSU107Cj4+ICsJaW50IGk7Cj4+ICsKPj4gKwlzcGluX2xvY2soJnZxLT5tbXVfbG9jayk7 Cj4+ICsJZm9yIChpID0gMDsgaSA8IFZIT1NUX05VTV9BRERSUzsgaSsrKSB7Cj4+ICsJCW1hcFtp XSA9IHZxLT5tYXBzW2ldOwo+PiArCQlpZiAobWFwW2ldKSB7Cj4+ICsJCQl2aG9zdF9zZXRfbWFw X2RpcnR5KHZxLCBtYXBbaV0sIGkpOwo+PiArCQkJdnEtPm1hcHNbaV0gPSBOVUxMOwo+PiArCQl9 Cj4+ICsJfQo+PiArCXNwaW5fdW5sb2NrKCZ2cS0+bW11X2xvY2spOwo+PiArCj4+ICsJLyogTm8g bmVlZCBmb3Igc3luY2hyb25pemF0aW9uIHNpbmNlIHdlIGFyZSBzZXJpYWxpemVkIHdpdGgKPj4g KwkgKiBtZW1vcnkgYWNjZXNzb3JzIChlLmcgdnEgbXV0ZXggaGVsZCkuCj4+ICsJICovCj4+ICsK Pj4gKwlmb3IgKGkgPSAwOyBpIDwgVkhPU1RfTlVNX0FERFJTOyBpKyspCj4+ICsJCWlmIChtYXBb aV0pCj4+ICsJCQl2aG9zdF9tYXBfdW5wcmVmZXRjaChtYXBbaV0pOwo+PiArCj4+ICt9Cj4+ICsK Pj4gK3N0YXRpYyB2b2lkIHZob3N0X3Jlc2V0X3ZxX21hcHMoc3RydWN0IHZob3N0X3ZpcnRxdWV1 ZSAqdnEpCj4+ICt7Cj4+ICsJaW50IGk7Cj4+ICsKPj4gKwl2aG9zdF91bmluaXRfdnFfbWFwcyh2 cSk7Cj4+ICsJZm9yIChpID0gMDsgaSA8IFZIT1NUX05VTV9BRERSUzsgaSsrKQo+PiArCQl2cS0+ dWFkZHJzW2ldLnNpemUgPSAwOwo+PiArfQo+PiArCj4+ICtzdGF0aWMgYm9vbCB2aG9zdF9tYXBf cmFuZ2Vfb3ZlcmxhcChzdHJ1Y3Qgdmhvc3RfdWFkZHIgKnVhZGRyLAo+PiArCQkJCSAgICAgdW5z aWduZWQgbG9uZyBzdGFydCwKPj4gKwkJCQkgICAgIHVuc2lnbmVkIGxvbmcgZW5kKQo+PiArewo+ PiArCWlmICh1bmxpa2VseSghdWFkZHItPnNpemUpKQo+PiArCQlyZXR1cm4gZmFsc2U7Cj4+ICsK Pj4gKwlyZXR1cm4gIShlbmQgPCB1YWRkci0+dWFkZHIgfHwgc3RhcnQgPiB1YWRkci0+dWFkZHIg LSAxICsgdWFkZHItPnNpemUpOwo+PiArfQo+PiArCj4+ICtzdGF0aWMgdm9pZCBpbmxpbmUgdmhv c3RfdnFfYWNjZXNzX21hcF9iZWdpbihzdHJ1Y3Qgdmhvc3RfdmlydHF1ZXVlICp2cSkKPj4gK3sK Pj4gKwlzcGluX2xvY2soJnZxLT5tbXVfbG9jayk7Cj4+ICt9Cj4+ICsKPj4gK3N0YXRpYyB2b2lk IGlubGluZSB2aG9zdF92cV9hY2Nlc3NfbWFwX2VuZChzdHJ1Y3Qgdmhvc3RfdmlydHF1ZXVlICp2 cSkKPj4gK3sKPj4gKwlzcGluX3VubG9jaygmdnEtPm1tdV9sb2NrKTsKPj4gK30KPj4gKwo+PiAr c3RhdGljIGludCB2aG9zdF9pbnZhbGlkYXRlX3ZxX3N0YXJ0KHN0cnVjdCB2aG9zdF92aXJ0cXVl dWUgKnZxLAo+PiArCQkJCSAgICAgaW50IGluZGV4LAo+PiArCQkJCSAgICAgdW5zaWduZWQgbG9u ZyBzdGFydCwKPj4gKwkJCQkgICAgIHVuc2lnbmVkIGxvbmcgZW5kLAo+PiArCQkJCSAgICAgYm9v bCBibG9ja2FibGUpCj4+ICt7Cj4+ICsJc3RydWN0IHZob3N0X3VhZGRyICp1YWRkciA9ICZ2cS0+ dWFkZHJzW2luZGV4XTsKPj4gKwlzdHJ1Y3Qgdmhvc3RfbWFwICptYXA7Cj4+ICsKPj4gKwlpZiAo IXZob3N0X21hcF9yYW5nZV9vdmVybGFwKHVhZGRyLCBzdGFydCwgZW5kKSkKPj4gKwkJcmV0dXJu IDA7Cj4+ICsJZWxzZSBpZiAoIWJsb2NrYWJsZSkKPj4gKwkJcmV0dXJuIC1FQUdBSU47Cj4+ICsK Pj4gKwlzcGluX2xvY2soJnZxLT5tbXVfbG9jayk7Cj4+ICsJKyt2cS0+aW52YWxpZGF0ZV9jb3Vu dDsKPj4gKwo+PiArCW1hcCA9IHZxLT5tYXBzW2luZGV4XTsKPj4gKwlpZiAobWFwKQo+PiArCQl2 cS0+bWFwc1tpbmRleF0gPSBOVUxMOwo+PiArCXNwaW5fdW5sb2NrKCZ2cS0+bW11X2xvY2spOwo+ PiArCj4+ICsJaWYgKG1hcCkgewo+PiArCQl2aG9zdF9zZXRfbWFwX2RpcnR5KHZxLCBtYXAsIGlu ZGV4KTsKPj4gKwkJdmhvc3RfbWFwX3VucHJlZmV0Y2gobWFwKTsKPj4gKwl9Cj4+ICsKPj4gKwly ZXR1cm4gMDsKPj4gK30KPj4gKwo+PiArc3RhdGljIHZvaWQgdmhvc3RfaW52YWxpZGF0ZV92cV9l bmQoc3RydWN0IHZob3N0X3ZpcnRxdWV1ZSAqdnEsCj4+ICsJCQkJICAgIGludCBpbmRleCwKPj4g KwkJCQkgICAgdW5zaWduZWQgbG9uZyBzdGFydCwKPj4gKwkJCQkgICAgdW5zaWduZWQgbG9uZyBl bmQpCj4+ICt7Cj4+ICsJaWYgKCF2aG9zdF9tYXBfcmFuZ2Vfb3ZlcmxhcCgmdnEtPnVhZGRyc1tp bmRleF0sIHN0YXJ0LCBlbmQpKQo+PiArCQlyZXR1cm47Cj4+ICsKPj4gKwlzcGluX2xvY2soJnZx LT5tbXVfbG9jayk7Cj4+ICsJLS12cS0+aW52YWxpZGF0ZV9jb3VudDsKPj4gKwlzcGluX3VubG9j aygmdnEtPm1tdV9sb2NrKTsKPj4gK30KPj4gKwo+PiArc3RhdGljIGludCB2aG9zdF9pbnZhbGlk YXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLAo+PiArCQkJCQljb25zdCBz dHJ1Y3QgbW11X25vdGlmaWVyX3JhbmdlICpyYW5nZSkKPj4gK3sKPj4gKwlzdHJ1Y3Qgdmhvc3Rf ZGV2ICpkZXYgPSBjb250YWluZXJfb2YobW4sIHN0cnVjdCB2aG9zdF9kZXYsCj4+ICsJCQkJCSAg ICAgbW11X25vdGlmaWVyKTsKPj4gKwlib29sIGJsb2NrYWJsZSA9IG1tdV9ub3RpZmllcl9yYW5n ZV9ibG9ja2FibGUocmFuZ2UpOwo+PiArCWludCBpLCBqLCByZXQ7Cj4+ICsKPj4gKwlmb3IgKGkg PSAwOyBpIDwgZGV2LT5udnFzOyBpKyspIHsKPj4gKwkJc3RydWN0IHZob3N0X3ZpcnRxdWV1ZSAq dnEgPSBkZXYtPnZxc1tpXTsKPj4gKwo+PiArCQlmb3IgKGogPSAwOyBqIDwgVkhPU1RfTlVNX0FE RFJTOyBqKyspIHsKPj4gKwkJCXJldCA9IHZob3N0X2ludmFsaWRhdGVfdnFfc3RhcnQodnEsIGos Cj4+ICsJCQkJCQkJcmFuZ2UtPnN0YXJ0LAo+PiArCQkJCQkJCXJhbmdlLT5lbmQsIGJsb2NrYWJs ZSk7Cj4+ICsJCQlpZiAocmV0KQo+PiArCQkJCXJldHVybiByZXQ7Cj4+ICsJCX0KPj4gKwl9Cj4+ ICsKPj4gKwlyZXR1cm4gMDsKPj4gK30KPj4gKwo+PiArc3RhdGljIHZvaWQgdmhvc3RfaW52YWxp ZGF0ZV9yYW5nZV9lbmQoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCj4+ICsJCQkJICAgICAgIGNv bnN0IHN0cnVjdCBtbXVfbm90aWZpZXJfcmFuZ2UgKnJhbmdlKQo+PiArewo+PiArCXN0cnVjdCB2 aG9zdF9kZXYgKmRldiA9IGNvbnRhaW5lcl9vZihtbiwgc3RydWN0IHZob3N0X2RldiwKPj4gKwkJ CQkJICAgICBtbXVfbm90aWZpZXIpOwo+PiArCWludCBpLCBqOwo+PiArCj4+ICsJZm9yIChpID0g MDsgaSA8IGRldi0+bnZxczsgaSsrKSB7Cj4+ICsJCXN0cnVjdCB2aG9zdF92aXJ0cXVldWUgKnZx ID0gZGV2LT52cXNbaV07Cj4+ICsKPj4gKwkJZm9yIChqID0gMDsgaiA8IFZIT1NUX05VTV9BRERS UzsgaisrKQo+PiArCQkJdmhvc3RfaW52YWxpZGF0ZV92cV9lbmQodnEsIGosCj4+ICsJCQkJCQly YW5nZS0+c3RhcnQsCj4+ICsJCQkJCQlyYW5nZS0+ZW5kKTsKPj4gKwl9Cj4+ICt9Cj4+ICsKPj4g K3N0YXRpYyBjb25zdCBzdHJ1Y3QgbW11X25vdGlmaWVyX29wcyB2aG9zdF9tbXVfbm90aWZpZXJf b3BzID0gewo+PiArCS5pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0ID0gdmhvc3RfaW52YWxpZGF0ZV9y YW5nZV9zdGFydCwKPj4gKwkuaW52YWxpZGF0ZV9yYW5nZV9lbmQgPSB2aG9zdF9pbnZhbGlkYXRl X3JhbmdlX2VuZCwKPj4gK307Cj4+ICsKPj4gK3N0YXRpYyB2b2lkIHZob3N0X2luaXRfbWFwcyhz dHJ1Y3Qgdmhvc3RfZGV2ICpkZXYpCj4+ICt7Cj4+ICsJc3RydWN0IHZob3N0X3ZpcnRxdWV1ZSAq dnE7Cj4+ICsJaW50IGksIGo7Cj4+ICsKPj4gKwlkZXYtPm1tdV9ub3RpZmllci5vcHMgPSAmdmhv c3RfbW11X25vdGlmaWVyX29wczsKPj4gKwo+PiArCWZvciAoaSA9IDA7IGkgPCBkZXYtPm52cXM7 ICsraSkgewo+PiArCQl2cSA9IGRldi0+dnFzW2ldOwo+PiArCQlmb3IgKGogPSAwOyBqIDwgVkhP U1RfTlVNX0FERFJTOyBqKyspCj4+ICsJCQl2cS0+bWFwc1tqXSA9IE5VTEw7Cj4+ICsJfQo+PiAr fQo+PiArI2VuZGlmCj4+ICsKPj4gICBzdGF0aWMgdm9pZCB2aG9zdF92cV9yZXNldChzdHJ1Y3Qg dmhvc3RfZGV2ICpkZXYsCj4+ICAgCQkJICAgc3RydWN0IHZob3N0X3ZpcnRxdWV1ZSAqdnEpCj4+ ICAgewo+PiBAQCAtMzI2LDcgKzUwMiwxMSBAQCBzdGF0aWMgdm9pZCB2aG9zdF92cV9yZXNldChz dHJ1Y3Qgdmhvc3RfZGV2ICpkZXYsCj4+ICAgCXZxLT5idXN5bG9vcF90aW1lb3V0ID0gMDsKPj4g ICAJdnEtPnVtZW0gPSBOVUxMOwo+PiAgIAl2cS0+aW90bGIgPSBOVUxMOwo+PiArCXZxLT5pbnZh bGlkYXRlX2NvdW50ID0gMDsKPj4gICAJX192aG9zdF92cV9tZXRhX3Jlc2V0KHZxKTsKPj4gKyNp ZiBWSE9TVF9BUkNIX0NBTl9BQ0NFTF9VQUNDRVNTCj4+ICsJdmhvc3RfcmVzZXRfdnFfbWFwcyh2 cSk7Cj4+ICsjZW5kaWYKPj4gICB9Cj4+ICAgCj4+ICAgc3RhdGljIGludCB2aG9zdF93b3JrZXIo dm9pZCAqZGF0YSkKPj4gQEAgLTQ3MSwxMiArNjUxLDE1IEBAIHZvaWQgdmhvc3RfZGV2X2luaXQo c3RydWN0IHZob3N0X2RldiAqZGV2LAo+PiAgIAlkZXYtPmlvdl9saW1pdCA9IGlvdl9saW1pdDsK Pj4gICAJZGV2LT53ZWlnaHQgPSB3ZWlnaHQ7Cj4+ICAgCWRldi0+Ynl0ZV93ZWlnaHQgPSBieXRl X3dlaWdodDsKPj4gKwlkZXYtPmhhc19ub3RpZmllciA9IGZhbHNlOwo+PiAgIAlpbml0X2xsaXN0 X2hlYWQoJmRldi0+d29ya19saXN0KTsKPj4gICAJaW5pdF93YWl0cXVldWVfaGVhZCgmZGV2LT53 YWl0KTsKPj4gICAJSU5JVF9MSVNUX0hFQUQoJmRldi0+cmVhZF9saXN0KTsKPj4gICAJSU5JVF9M SVNUX0hFQUQoJmRldi0+cGVuZGluZ19saXN0KTsKPj4gICAJc3Bpbl9sb2NrX2luaXQoJmRldi0+ aW90bGJfbG9jayk7Cj4+IC0KPj4gKyNpZiBWSE9TVF9BUkNIX0NBTl9BQ0NFTF9VQUNDRVNTCj4+ ICsJdmhvc3RfaW5pdF9tYXBzKGRldik7Cj4+ICsjZW5kaWYKPj4gICAKPj4gICAJZm9yIChpID0g MDsgaSA8IGRldi0+bnZxczsgKytpKSB7Cj4+ICAgCQl2cSA9IGRldi0+dnFzW2ldOwo+PiBAQCAt NDg1LDYgKzY2OCw3IEBAIHZvaWQgdmhvc3RfZGV2X2luaXQoc3RydWN0IHZob3N0X2RldiAqZGV2 LAo+PiAgIAkJdnEtPmhlYWRzID0gTlVMTDsKPj4gICAJCXZxLT5kZXYgPSBkZXY7Cj4+ICAgCQlt dXRleF9pbml0KCZ2cS0+bXV0ZXgpOwo+PiArCQlzcGluX2xvY2tfaW5pdCgmdnEtPm1tdV9sb2Nr KTsKPj4gICAJCXZob3N0X3ZxX3Jlc2V0KGRldiwgdnEpOwo+PiAgIAkJaWYgKHZxLT5oYW5kbGVf a2ljaykKPj4gICAJCQl2aG9zdF9wb2xsX2luaXQoJnZxLT5wb2xsLCB2cS0+aGFuZGxlX2tpY2ss Cj4+IEBAIC01NjQsNyArNzQ4LDE5IEBAIGxvbmcgdmhvc3RfZGV2X3NldF9vd25lcihzdHJ1Y3Qg dmhvc3RfZGV2ICpkZXYpCj4+ICAgCWlmIChlcnIpCj4+ICAgCQlnb3RvIGVycl9jZ3JvdXA7Cj4+ ICAgCj4+ICsjaWYgVkhPU1RfQVJDSF9DQU5fQUNDRUxfVUFDQ0VTUwo+PiArCWVyciA9IG1tdV9u b3RpZmllcl9yZWdpc3RlcigmZGV2LT5tbXVfbm90aWZpZXIsIGRldi0+bW0pOwo+PiArCWlmIChl cnIpCj4+ICsJCWdvdG8gZXJyX21tdV9ub3RpZmllcjsKPj4gKyNlbmRpZgo+PiArCWRldi0+aGFz X25vdGlmaWVyID0gdHJ1ZTsKPj4gKwo+PiAgIAlyZXR1cm4gMDsKPj4gKwo+PiArI2lmIFZIT1NU X0FSQ0hfQ0FOX0FDQ0VMX1VBQ0NFU1MKPj4gK2Vycl9tbXVfbm90aWZpZXI6Cj4+ICsJdmhvc3Rf ZGV2X2ZyZWVfaW92ZWNzKGRldik7Cj4+ICsjZW5kaWYKPj4gICBlcnJfY2dyb3VwOgo+PiAgIAlr dGhyZWFkX3N0b3Aod29ya2VyKTsKPj4gICAJZGV2LT53b3JrZXIgPSBOVUxMOwo+PiBAQCAtNjU1 LDYgKzg1MSwxMDcgQEAgc3RhdGljIHZvaWQgdmhvc3RfY2xlYXJfbXNnKHN0cnVjdCB2aG9zdF9k ZXYgKmRldikKPj4gICAJc3Bpbl91bmxvY2soJmRldi0+aW90bGJfbG9jayk7Cj4+ICAgfQo+PiAg IAo+PiArI2lmIFZIT1NUX0FSQ0hfQ0FOX0FDQ0VMX1VBQ0NFU1MKPj4gK3N0YXRpYyB2b2lkIHZo b3N0X3NldHVwX3VhZGRyKHN0cnVjdCB2aG9zdF92aXJ0cXVldWUgKnZxLAo+PiArCQkJICAgICAg aW50IGluZGV4LCB1bnNpZ25lZCBsb25nIHVhZGRyLAo+PiArCQkJICAgICAgc2l6ZV90IHNpemUs IGJvb2wgd3JpdGUpCj4+ICt7Cj4+ICsJc3RydWN0IHZob3N0X3VhZGRyICphZGRyID0gJnZxLT51 YWRkcnNbaW5kZXhdOwo+PiArCj4+ICsJYWRkci0+dWFkZHIgPSB1YWRkcjsKPj4gKwlhZGRyLT5z aXplID0gc2l6ZTsKPj4gKwlhZGRyLT53cml0ZSA9IHdyaXRlOwo+PiArfQo+PiArCj4+ICtzdGF0 aWMgdm9pZCB2aG9zdF9zZXR1cF92cV91YWRkcihzdHJ1Y3Qgdmhvc3RfdmlydHF1ZXVlICp2cSkK Pj4gK3sKPj4gKwl2aG9zdF9zZXR1cF91YWRkcih2cSwgVkhPU1RfQUREUl9ERVNDLAo+PiArCQkJ ICAodW5zaWduZWQgbG9uZyl2cS0+ZGVzYywKPj4gKwkJCSAgdmhvc3RfZ2V0X2Rlc2Nfc2l6ZSh2 cSwgdnEtPm51bSksCj4+ICsJCQkgIGZhbHNlKTsKPj4gKwl2aG9zdF9zZXR1cF91YWRkcih2cSwg VkhPU1RfQUREUl9BVkFJTCwKPj4gKwkJCSAgKHVuc2lnbmVkIGxvbmcpdnEtPmF2YWlsLAo+PiAr CQkJICB2aG9zdF9nZXRfYXZhaWxfc2l6ZSh2cSwgdnEtPm51bSksCj4+ICsJCQkgIGZhbHNlKTsK Pj4gKwl2aG9zdF9zZXR1cF91YWRkcih2cSwgVkhPU1RfQUREUl9VU0VELAo+PiArCQkJICAodW5z aWduZWQgbG9uZyl2cS0+dXNlZCwKPj4gKwkJCSAgdmhvc3RfZ2V0X3VzZWRfc2l6ZSh2cSwgdnEt Pm51bSksCj4+ICsJCQkgIHRydWUpOwo+PiArfQo+PiArCj4+ICtzdGF0aWMgaW50IHZob3N0X21h cF9wcmVmZXRjaChzdHJ1Y3Qgdmhvc3RfdmlydHF1ZXVlICp2cSwKPj4gKwkJCSAgICAgICBpbnQg aW5kZXgpCj4+ICt7Cj4+ICsJc3RydWN0IHZob3N0X21hcCAqbWFwOwo+PiArCXN0cnVjdCB2aG9z dF91YWRkciAqdWFkZHIgPSAmdnEtPnVhZGRyc1tpbmRleF07Cj4+ICsJc3RydWN0IHBhZ2UgKipw YWdlczsKPj4gKwlpbnQgbnBhZ2VzID0gRElWX1JPVU5EX1VQKHVhZGRyLT5zaXplLCBQQUdFX1NJ WkUpOwo+PiArCWludCBucGlubmVkOwo+PiArCXZvaWQgKnZhZGRyLCAqdjsKPj4gKwlpbnQgZXJy Owo+PiArCWludCBpOwo+PiArCj4+ICsJc3Bpbl9sb2NrKCZ2cS0+bW11X2xvY2spOwo+PiArCj4+ ICsJZXJyID0gLUVGQVVMVDsKPj4gKwlpZiAodnEtPmludmFsaWRhdGVfY291bnQpCj4+ICsJCWdv dG8gZXJyOwo+PiArCj4+ICsJZXJyID0gLUVOT01FTTsKPj4gKwltYXAgPSBrbWFsbG9jKHNpemVv ZigqbWFwKSwgR0ZQX0FUT01JQyk7Cj4+ICsJaWYgKCFtYXApCj4+ICsJCWdvdG8gZXJyOwo+PiAr Cj4+ICsJcGFnZXMgPSBrbWFsbG9jX2FycmF5KG5wYWdlcywgc2l6ZW9mKHN0cnVjdCBwYWdlICop LCBHRlBfQVRPTUlDKTsKPj4gKwlpZiAoIXBhZ2VzKQo+PiArCQlnb3RvIGVycl9wYWdlczsKPj4g Kwo+PiArCWVyciA9IEVGQVVMVDsKPj4gKwlucGlubmVkID0gX19nZXRfdXNlcl9wYWdlc19mYXN0 KHVhZGRyLT51YWRkciwgbnBhZ2VzLAo+PiArCQkJCQl1YWRkci0+d3JpdGUsIHBhZ2VzKTsKPj4g KwlpZiAobnBpbm5lZCA+IDApCj4+ICsJCXJlbGVhc2VfcGFnZXMocGFnZXMsIG5waW5uZWQpOwo+ PiArCWlmIChucGlubmVkICE9IG5wYWdlcykKPj4gKwkJZ290byBlcnJfZ3VwOwo+PiArCj4+ICsJ Zm9yIChpID0gMDsgaSA8IG5waW5uZWQ7IGkrKykKPj4gKwkJaWYgKFBhZ2VIaWdoTWVtKHBhZ2Vz W2ldKSkKPj4gKwkJCWdvdG8gZXJyX2d1cDsKPj4gKwo+PiArCXZhZGRyID0gdiA9IHBhZ2VfYWRk cmVzcyhwYWdlc1swXSk7Cj4+ICsKPj4gKwkvKiBGb3Igc2ltcGxpY2l0eSwgZmFsbGJhY2sgdG8g dXNlcnNwYWNlIGFkZHJlc3MgaWYgVkEgaXMgbm90Cj4+ICsJICogY29udGlnaW91cy4KPj4gKwkg Ki8KPj4gKwlmb3IgKGkgPSAxOyBpIDwgbnBpbm5lZDsgaSsrKSB7Cj4+ICsJCXYgKz0gUEFHRV9T SVpFOwo+PiArCQlpZiAodiAhPSBwYWdlX2FkZHJlc3MocGFnZXNbaV0pKQo+PiArCQkJZ290byBl cnJfZ3VwOwo+PiArCX0KPj4gKwo+PiArCW1hcC0+YWRkciA9IHZhZGRyICsgKHVhZGRyLT51YWRk ciAmIChQQUdFX1NJWkUgLSAxKSk7Cj4+ICsJbWFwLT5ucGFnZXMgPSBucGFnZXM7Cj4+ICsJbWFw LT5wYWdlcyA9IHBhZ2VzOwo+PiArCj4+ICsJdnEtPm1hcHNbaW5kZXhdID0gbWFwOwo+PiArCS8q IE5vIG5lZWQgZm9yIGEgc3luY2hyb25pemVfcmN1KCkuIFRoaXMgZnVuY3Rpb24gc2hvdWxkIGJl Cj4+ICsJICogY2FsbGVkIGJ5IGRldi0+d29ya2VyIHNvIHdlIGFyZSBzZXJpYWxpemVkIHdpdGgg YWxsCj4+ICsJICogcmVhZGVycy4KPj4gKwkgKi8KPj4gKwlzcGluX3VubG9jaygmdnEtPm1tdV9s b2NrKTsKPj4gKwo+PiArCXJldHVybiAwOwo+PiArCj4+ICtlcnJfZ3VwOgo+PiArCWtmcmVlKHBh Z2VzKTsKPj4gK2Vycl9wYWdlczoKPj4gKwlrZnJlZShtYXApOwo+PiArZXJyOgo+PiArCXNwaW5f dW5sb2NrKCZ2cS0+bW11X2xvY2spOwo+PiArCXJldHVybiBlcnI7Cj4+ICt9Cj4+ICsjZW5kaWYK Pj4gKwo+PiAgIHZvaWQgdmhvc3RfZGV2X2NsZWFudXAoc3RydWN0IHZob3N0X2RldiAqZGV2KQo+ PiAgIHsKPj4gICAJaW50IGk7Cj4+IEBAIC02ODQsOCArOTgxLDIwIEBAIHZvaWQgdmhvc3RfZGV2 X2NsZWFudXAoc3RydWN0IHZob3N0X2RldiAqZGV2KQo+PiAgIAkJa3RocmVhZF9zdG9wKGRldi0+ d29ya2VyKTsKPj4gICAJCWRldi0+d29ya2VyID0gTlVMTDsKPj4gICAJfQo+PiAtCWlmIChkZXYt Pm1tKQo+PiArCWlmIChkZXYtPm1tKSB7Cj4+ICsjaWYgVkhPU1RfQVJDSF9DQU5fQUNDRUxfVUFD Q0VTUwo+PiArCQlpZiAoZGV2LT5oYXNfbm90aWZpZXIpIHsKPj4gKwkJCW1tdV9ub3RpZmllcl91 bnJlZ2lzdGVyKCZkZXYtPm1tdV9ub3RpZmllciwKPj4gKwkJCQkJCWRldi0+bW0pOwo+PiArCQkJ ZGV2LT5oYXNfbm90aWZpZXIgPSBmYWxzZTsKPj4gKwkJfQo+PiArI2VuZGlmCj4+ICAgCQltbXB1 dChkZXYtPm1tKTsKPj4gKwl9Cj4+ICsjaWYgVkhPU1RfQVJDSF9DQU5fQUNDRUxfVUFDQ0VTUwo+ PiArCWZvciAoaSA9IDA7IGkgPCBkZXYtPm52cXM7IGkrKykKPj4gKwkJdmhvc3RfdW5pbml0X3Zx X21hcHMoZGV2LT52cXNbaV0pOwo+PiArI2VuZGlmCj4+ICAgCWRldi0+bW0gPSBOVUxMOwo+PiAg IH0KPj4gICBFWFBPUlRfU1lNQk9MX0dQTCh2aG9zdF9kZXZfY2xlYW51cCk7Cj4+IEBAIC05MTQs NiArMTIyMywyNiBAQCBzdGF0aWMgaW5saW5lIHZvaWQgX191c2VyICpfX3Zob3N0X2dldF91c2Vy KHN0cnVjdCB2aG9zdF92aXJ0cXVldWUgKnZxLAo+PiAgIAo+PiAgIHN0YXRpYyBpbmxpbmUgaW50 IHZob3N0X3B1dF9hdmFpbF9ldmVudChzdHJ1Y3Qgdmhvc3RfdmlydHF1ZXVlICp2cSkKPj4gICB7 Cj4+ICsjaWYgVkhPU1RfQVJDSF9DQU5fQUNDRUxfVUFDQ0VTUwo+PiArCXN0cnVjdCB2aG9zdF9t YXAgKm1hcDsKPj4gKwlzdHJ1Y3QgdnJpbmdfdXNlZCAqdXNlZDsKPj4gKwo+PiArCWlmICghdnEt PmlvdGxiKSB7Cj4+ICsJCXZob3N0X3ZxX2FjY2Vzc19tYXBfYmVnaW4odnEpOwo+PiArCj4+ICsJ CW1hcCA9IHZxLT5tYXBzW1ZIT1NUX0FERFJfVVNFRF07Cj4+ICsJCWlmIChsaWtlbHkobWFwKSkg ewo+PiArCQkJdXNlZCA9IG1hcC0+YWRkcjsKPj4gKwkJCSooKF9fdmlydGlvMTYgKikmdXNlZC0+ cmluZ1t2cS0+bnVtXSkgPQo+PiArCQkJCWNwdV90b192aG9zdDE2KHZxLCB2cS0+YXZhaWxfaWR4 KTsKPj4gKwkJCXZob3N0X3ZxX2FjY2Vzc19tYXBfZW5kKHZxKTsKPj4gKwkJCXJldHVybiAwOwo+ PiArCQl9Cj4+ICsKPj4gKwkJdmhvc3RfdnFfYWNjZXNzX21hcF9lbmQodnEpOwo+PiArCX0KPj4g KyNlbmRpZgo+PiArCj4+ICAgCXJldHVybiB2aG9zdF9wdXRfdXNlcih2cSwgY3B1X3RvX3Zob3N0 MTYodnEsIHZxLT5hdmFpbF9pZHgpLAo+PiAgIAkJCSAgICAgIHZob3N0X2F2YWlsX2V2ZW50KHZx KSk7Cj4+ICAgfQo+PiBAQCAtOTIyLDYgKzEyNTEsMjcgQEAgc3RhdGljIGlubGluZSBpbnQgdmhv c3RfcHV0X3VzZWQoc3RydWN0IHZob3N0X3ZpcnRxdWV1ZSAqdnEsCj4+ICAgCQkJCSBzdHJ1Y3Qg dnJpbmdfdXNlZF9lbGVtICpoZWFkLCBpbnQgaWR4LAo+PiAgIAkJCQkgaW50IGNvdW50KQo+PiAg IHsKPj4gKyNpZiBWSE9TVF9BUkNIX0NBTl9BQ0NFTF9VQUNDRVNTCj4+ICsJc3RydWN0IHZob3N0 X21hcCAqbWFwOwo+PiArCXN0cnVjdCB2cmluZ191c2VkICp1c2VkOwo+PiArCXNpemVfdCBzaXpl Owo+PiArCj4+ICsJaWYgKCF2cS0+aW90bGIpIHsKPj4gKwkJdmhvc3RfdnFfYWNjZXNzX21hcF9i ZWdpbih2cSk7Cj4+ICsKPj4gKwkJbWFwID0gdnEtPm1hcHNbVkhPU1RfQUREUl9VU0VEXTsKPj4g KwkJaWYgKGxpa2VseShtYXApKSB7Cj4+ICsJCQl1c2VkID0gbWFwLT5hZGRyOwo+PiArCQkJc2l6 ZSA9IGNvdW50ICogc2l6ZW9mKCpoZWFkKTsKPj4gKwkJCW1lbWNweSh1c2VkLT5yaW5nICsgaWR4 LCBoZWFkLCBzaXplKTsKPj4gKwkJCXZob3N0X3ZxX2FjY2Vzc19tYXBfZW5kKHZxKTsKPj4gKwkJ CXJldHVybiAwOwo+PiArCQl9Cj4+ICsKPj4gKwkJdmhvc3RfdnFfYWNjZXNzX21hcF9lbmQodnEp Owo+PiArCX0KPj4gKyNlbmRpZgo+PiArCj4+ICAgCXJldHVybiB2aG9zdF9jb3B5X3RvX3VzZXIo dnEsIHZxLT51c2VkLT5yaW5nICsgaWR4LCBoZWFkLAo+PiAgIAkJCQkgIGNvdW50ICogc2l6ZW9m KCpoZWFkKSk7Cj4+ICAgfQo+PiBAQCAtOTI5LDYgKzEyNzksMjUgQEAgc3RhdGljIGlubGluZSBp bnQgdmhvc3RfcHV0X3VzZWQoc3RydWN0IHZob3N0X3ZpcnRxdWV1ZSAqdnEsCj4+ICAgc3RhdGlj IGlubGluZSBpbnQgdmhvc3RfcHV0X3VzZWRfZmxhZ3Moc3RydWN0IHZob3N0X3ZpcnRxdWV1ZSAq dnEpCj4+ICAgCj4+ICAgewo+PiArI2lmIFZIT1NUX0FSQ0hfQ0FOX0FDQ0VMX1VBQ0NFU1MKPj4g KwlzdHJ1Y3Qgdmhvc3RfbWFwICptYXA7Cj4+ICsJc3RydWN0IHZyaW5nX3VzZWQgKnVzZWQ7Cj4+ ICsKPj4gKwlpZiAoIXZxLT5pb3RsYikgewo+PiArCQl2aG9zdF92cV9hY2Nlc3NfbWFwX2JlZ2lu KHZxKTsKPj4gKwo+PiArCQltYXAgPSB2cS0+bWFwc1tWSE9TVF9BRERSX1VTRURdOwo+PiArCQlp ZiAobGlrZWx5KG1hcCkpIHsKPj4gKwkJCXVzZWQgPSBtYXAtPmFkZHI7Cj4+ICsJCQl1c2VkLT5m bGFncyA9IGNwdV90b192aG9zdDE2KHZxLCB2cS0+dXNlZF9mbGFncyk7Cj4+ICsJCQl2aG9zdF92 cV9hY2Nlc3NfbWFwX2VuZCh2cSk7Cj4+ICsJCQlyZXR1cm4gMDsKPj4gKwkJfQo+PiArCj4+ICsJ CXZob3N0X3ZxX2FjY2Vzc19tYXBfZW5kKHZxKTsKPj4gKwl9Cj4+ICsjZW5kaWYKPj4gKwo+PiAg IAlyZXR1cm4gdmhvc3RfcHV0X3VzZXIodnEsIGNwdV90b192aG9zdDE2KHZxLCB2cS0+dXNlZF9m bGFncyksCj4+ICAgCQkJICAgICAgJnZxLT51c2VkLT5mbGFncyk7Cj4+ICAgfQo+PiBAQCAtOTM2 LDYgKzEzMDUsMjUgQEAgc3RhdGljIGlubGluZSBpbnQgdmhvc3RfcHV0X3VzZWRfZmxhZ3Moc3Ry dWN0IHZob3N0X3ZpcnRxdWV1ZSAqdnEpCj4+ICAgc3RhdGljIGlubGluZSBpbnQgdmhvc3RfcHV0 X3VzZWRfaWR4KHN0cnVjdCB2aG9zdF92aXJ0cXVldWUgKnZxKQo+PiAgIAo+PiAgIHsKPj4gKyNp ZiBWSE9TVF9BUkNIX0NBTl9BQ0NFTF9VQUNDRVNTCj4+ICsJc3RydWN0IHZob3N0X21hcCAqbWFw Owo+PiArCXN0cnVjdCB2cmluZ191c2VkICp1c2VkOwo+PiArCj4+ICsJaWYgKCF2cS0+aW90bGIp IHsKPj4gKwkJdmhvc3RfdnFfYWNjZXNzX21hcF9iZWdpbih2cSk7Cj4+ICsKPj4gKwkJbWFwID0g dnEtPm1hcHNbVkhPU1RfQUREUl9VU0VEXTsKPj4gKwkJaWYgKGxpa2VseShtYXApKSB7Cj4+ICsJ CQl1c2VkID0gbWFwLT5hZGRyOwo+PiArCQkJdXNlZC0+aWR4ID0gY3B1X3RvX3Zob3N0MTYodnEs IHZxLT5sYXN0X3VzZWRfaWR4KTsKPj4gKwkJCXZob3N0X3ZxX2FjY2Vzc19tYXBfZW5kKHZxKTsK Pj4gKwkJCXJldHVybiAwOwo+PiArCQl9Cj4+ICsKPj4gKwkJdmhvc3RfdnFfYWNjZXNzX21hcF9l bmQodnEpOwo+PiArCX0KPj4gKyNlbmRpZgo+PiArCj4+ICAgCXJldHVybiB2aG9zdF9wdXRfdXNl cih2cSwgY3B1X3RvX3Zob3N0MTYodnEsIHZxLT5sYXN0X3VzZWRfaWR4KSwKPj4gICAJCQkgICAg ICAmdnEtPnVzZWQtPmlkeCk7Cj4+ICAgfQo+PiBAQCAtOTgxLDEyICsxMzY5LDUwIEBAIHN0YXRp YyB2b2lkIHZob3N0X2Rldl91bmxvY2tfdnFzKHN0cnVjdCB2aG9zdF9kZXYgKmQpCj4+ICAgc3Rh dGljIGlubGluZSBpbnQgdmhvc3RfZ2V0X2F2YWlsX2lkeChzdHJ1Y3Qgdmhvc3RfdmlydHF1ZXVl ICp2cSwKPj4gICAJCQkJICAgICAgX192aXJ0aW8xNiAqaWR4KQo+PiAgIHsKPj4gKyNpZiBWSE9T VF9BUkNIX0NBTl9BQ0NFTF9VQUNDRVNTCj4+ICsJc3RydWN0IHZob3N0X21hcCAqbWFwOwo+PiAr CXN0cnVjdCB2cmluZ19hdmFpbCAqYXZhaWw7Cj4+ICsKPj4gKwlpZiAoIXZxLT5pb3RsYikgewo+ PiArCQl2aG9zdF92cV9hY2Nlc3NfbWFwX2JlZ2luKHZxKTsKPj4gKwo+PiArCQltYXAgPSB2cS0+ bWFwc1tWSE9TVF9BRERSX0FWQUlMXTsKPj4gKwkJaWYgKGxpa2VseShtYXApKSB7Cj4+ICsJCQlh dmFpbCA9IG1hcC0+YWRkcjsKPj4gKwkJCSppZHggPSBhdmFpbC0+aWR4Owo+IGluZGV4IGNhbiBu b3cgYmUgc3BlY3VsYXRlZC4KClsuLi5dCgoKPiArCQl2aG9zdF92cV9hY2Nlc3NfbWFwX2JlZ2lu KHZxKTsKPiArCj4gKwkJbWFwID0gdnEtPm1hcHNbVkhPU1RfQUREUl9BVkFJTF07Cj4gKwkJaWYg KGxpa2VseShtYXApKSB7Cj4gKwkJCWF2YWlsID0gbWFwLT5hZGRyOwo+ICsJCQkqaGVhZCA9IGF2 YWlsLT5yaW5nW2lkeCAmICh2cS0+bnVtIC0gMSldOwo+Cj4gU2luY2UgaWR4IGNhbiBiZSBzcGVj dWxhdGVkLCBJIGd1ZXNzIHdlIG5lZWQgYXJyYXlfaW5kZXhfbm9zcGVjIGhlcmU/CgoKU28gd2Ug aGF2ZQoKQUNRVUlSRShtbXVfbG9jaykKCmdldCBpZHgKClJFTEVBU0UobW11X2xvY2spCgpBQ1FV SVJFKG1tdV9sb2NrKQoKcmVhZCBhcnJheVtpZHhdCgpSRUxFQVNFKG1tdV9sb2NrKQoKVGhlbiBJ IHRoaW5rIGlkeCBjYW4ndCBiZSBzcGVjdWxhdGVkIGNvbnNpZGVyIHdlJ3ZlIHBhc3NlZCBSRUxF QVNFICsgCkFDUVVJUkU/CgoKPgo+Cj4+ICsJCQl2aG9zdF92cV9hY2Nlc3NfbWFwX2VuZCh2cSk7 Cj4+ICsJCQlyZXR1cm4gMDsKPj4gKwkJfQo+PiArCj4+ICsJCXZob3N0X3ZxX2FjY2Vzc19tYXBf ZW5kKHZxKTsKPj4gKwl9Cj4+ICsjZW5kaWYKPj4gKwo+PiAgIAlyZXR1cm4gdmhvc3RfZ2V0X2F2 YWlsKHZxLCAqaGVhZCwKPj4gICAJCQkgICAgICAgJnZxLT5hdmFpbC0+cmluZ1tpZHggJiAodnEt Pm51bSAtIDEpXSk7Cj4+ICAgfQo+PiBAQCAtOTk0LDI0ICsxNDIwLDk4IEBAIHN0YXRpYyBpbmxp bmUgaW50IHZob3N0X2dldF9hdmFpbF9oZWFkKHN0cnVjdCB2aG9zdF92aXJ0cXVldWUgKnZxLAo+ PiAgIHN0YXRpYyBpbmxpbmUgaW50IHZob3N0X2dldF9hdmFpbF9mbGFncyhzdHJ1Y3Qgdmhvc3Rf dmlydHF1ZXVlICp2cSwKPj4gICAJCQkJCV9fdmlydGlvMTYgKmZsYWdzKQo+PiAgIHsKPj4gKyNp ZiBWSE9TVF9BUkNIX0NBTl9BQ0NFTF9VQUNDRVNTCj4+ICsJc3RydWN0IHZob3N0X21hcCAqbWFw Owo+PiArCXN0cnVjdCB2cmluZ19hdmFpbCAqYXZhaWw7Cj4+ICsKPj4gKwlpZiAoIXZxLT5pb3Rs Yikgewo+PiArCQl2aG9zdF92cV9hY2Nlc3NfbWFwX2JlZ2luKHZxKTsKPj4gKwo+PiArCQltYXAg PSB2cS0+bWFwc1tWSE9TVF9BRERSX0FWQUlMXTsKPj4gKwkJaWYgKGxpa2VseShtYXApKSB7Cj4+ ICsJCQlhdmFpbCA9IG1hcC0+YWRkcjsKPj4gKwkJCSpmbGFncyA9IGF2YWlsLT5mbGFnczsKPj4g KwkJCXZob3N0X3ZxX2FjY2Vzc19tYXBfZW5kKHZxKTsKPj4gKwkJCXJldHVybiAwOwo+PiArCQl9 Cj4+ICsKPj4gKwkJdmhvc3RfdnFfYWNjZXNzX21hcF9lbmQodnEpOwo+PiArCX0KPj4gKyNlbmRp Zgo+PiArCj4+ICAgCXJldHVybiB2aG9zdF9nZXRfYXZhaWwodnEsICpmbGFncywgJnZxLT5hdmFp bC0+ZmxhZ3MpOwo+PiAgIH0KPj4gICAKPj4gICBzdGF0aWMgaW5saW5lIGludCB2aG9zdF9nZXRf dXNlZF9ldmVudChzdHJ1Y3Qgdmhvc3RfdmlydHF1ZXVlICp2cSwKPj4gICAJCQkJICAgICAgIF9f dmlydGlvMTYgKmV2ZW50KQo+PiAgIHsKPj4gKyNpZiBWSE9TVF9BUkNIX0NBTl9BQ0NFTF9VQUND RVNTCj4+ICsJc3RydWN0IHZob3N0X21hcCAqbWFwOwo+PiArCXN0cnVjdCB2cmluZ19hdmFpbCAq YXZhaWw7Cj4+ICsKPj4gKwlpZiAoIXZxLT5pb3RsYikgewo+PiArCQl2aG9zdF92cV9hY2Nlc3Nf bWFwX2JlZ2luKHZxKTsKPj4gKwkJbWFwID0gdnEtPm1hcHNbVkhPU1RfQUREUl9BVkFJTF07Cj4+ ICsJCWlmIChsaWtlbHkobWFwKSkgewo+PiArCQkJYXZhaWwgPSBtYXAtPmFkZHI7Cj4+ICsJCQkq ZXZlbnQgPSAoX192aXJ0aW8xNilhdmFpbC0+cmluZ1t2cS0+bnVtXTsKPj4gKwkJCXZob3N0X3Zx X2FjY2Vzc19tYXBfZW5kKHZxKTsKPj4gKwkJCXJldHVybiAwOwo+PiArCQl9Cj4+ICsJCXZob3N0 X3ZxX2FjY2Vzc19tYXBfZW5kKHZxKTsKPj4gKwl9Cj4+ICsjZW5kaWYKPj4gKwo+PiAgIAlyZXR1 cm4gdmhvc3RfZ2V0X2F2YWlsKHZxLCAqZXZlbnQsIHZob3N0X3VzZWRfZXZlbnQodnEpKTsKPj4g ICB9Cj4+ICAgCj4+ICAgc3RhdGljIGlubGluZSBpbnQgdmhvc3RfZ2V0X3VzZWRfaWR4KHN0cnVj dCB2aG9zdF92aXJ0cXVldWUgKnZxLAo+PiAgIAkJCQkgICAgIF9fdmlydGlvMTYgKmlkeCkKPj4g ICB7Cj4+ICsjaWYgVkhPU1RfQVJDSF9DQU5fQUNDRUxfVUFDQ0VTUwo+PiArCXN0cnVjdCB2aG9z dF9tYXAgKm1hcDsKPj4gKwlzdHJ1Y3QgdnJpbmdfdXNlZCAqdXNlZDsKPj4gKwo+PiArCWlmICgh dnEtPmlvdGxiKSB7Cj4+ICsJCXZob3N0X3ZxX2FjY2Vzc19tYXBfYmVnaW4odnEpOwo+PiArCj4+ ICsJCW1hcCA9IHZxLT5tYXBzW1ZIT1NUX0FERFJfVVNFRF07Cj4+ICsJCWlmIChsaWtlbHkobWFw KSkgewo+PiArCQkJdXNlZCA9IG1hcC0+YWRkcjsKPj4gKwkJCSppZHggPSB1c2VkLT5pZHg7Cj4+ ICsJCQl2aG9zdF92cV9hY2Nlc3NfbWFwX2VuZCh2cSk7Cj4+ICsJCQlyZXR1cm4gMDsKPj4gKwkJ fQo+PiArCj4+ICsJCXZob3N0X3ZxX2FjY2Vzc19tYXBfZW5kKHZxKTsKPj4gKwl9Cj4+ICsjZW5k aWYKPj4gKwo+PiAgIAlyZXR1cm4gdmhvc3RfZ2V0X3VzZWQodnEsICppZHgsICZ2cS0+dXNlZC0+ aWR4KTsKPj4gICB9Cj4KPiBUaGlzIHNlZW1zIHRvIGJlIHVzZWQgZHVyaW5nIGluaXQuIFdoeSBk byB3ZSBib3RoZXIKPiBhY2NlbGVyYXRpbmcgdGhpcz8KCgpPaywgSSBjYW4gcmVtb3ZlIHRoaXMg cGFydCBpbiBuZXh0IHZlcnNpb24uCgoKPgo+Cj4+ICAgCj4+ICAgc3RhdGljIGlubGluZSBpbnQg dmhvc3RfZ2V0X2Rlc2Moc3RydWN0IHZob3N0X3ZpcnRxdWV1ZSAqdnEsCj4+ICAgCQkJCSBzdHJ1 Y3QgdnJpbmdfZGVzYyAqZGVzYywgaW50IGlkeCkKPj4gICB7Cj4+ICsjaWYgVkhPU1RfQVJDSF9D QU5fQUNDRUxfVUFDQ0VTUwo+PiArCXN0cnVjdCB2aG9zdF9tYXAgKm1hcDsKPj4gKwlzdHJ1Y3Qg dnJpbmdfZGVzYyAqZDsKPj4gKwo+PiArCWlmICghdnEtPmlvdGxiKSB7Cj4+ICsJCXZob3N0X3Zx X2FjY2Vzc19tYXBfYmVnaW4odnEpOwo+PiArCj4+ICsJCW1hcCA9IHZxLT5tYXBzW1ZIT1NUX0FE RFJfREVTQ107Cj4+ICsJCWlmIChsaWtlbHkobWFwKSkgewo+PiArCQkJZCA9IG1hcC0+YWRkcjsK Pj4gKwkJCSpkZXNjID0gKihkICsgaWR4KTsKPgo+IFNpbmNlIGlkeCBjYW4gYmUgc3BlY3VsYXRl ZCwgSSBndWVzcyB3ZSBuZWVkIGFycmF5X2luZGV4X25vc3BlYyBoZXJlPwoKClRoaXMgaXMgc2lt aWxhciB0byB0aGUgYWJvdmUgYXZhaWwgaWR4IGNhc2UuCgoKPgo+Cj4+ICsJCQl2aG9zdF92cV9h Y2Nlc3NfbWFwX2VuZCh2cSk7Cj4+ICsJCQlyZXR1cm4gMDsKPj4gKwkJfQo+PiArCj4+ICsJCXZo b3N0X3ZxX2FjY2Vzc19tYXBfZW5kKHZxKTsKPj4gKwl9Cj4+ICsjZW5kaWYKPj4gKwo+PiAgIAly ZXR1cm4gdmhvc3RfY29weV9mcm9tX3VzZXIodnEsIGRlc2MsIHZxLT5kZXNjICsgaWR4LCBzaXpl b2YoKmRlc2MpKTsKPj4gICB9Cj4+ICAgCj4gSSBhbHNvIHdvbmRlciBhYm91dCB0aGUgdXNlcnNw YWNlIGFkZHJlc3Mgd2UgZ2V0IGV2ZW50dWFseS4KPiBJdCB3b3VsZCBzZWVtIHRoYXQgd2UgbmVl ZCB0byBwcmV2ZW50IHRoYXQgZnJvbSBzcGVjdWxhdGluZyAtCj4gYW5kIHRoYXQgc2VlbXMgbGlr ZSBhIGdvb2QgaWRlYSBldmVuIGlmIHRoaXMgcGF0Y2ggaXNuJ3QKPiBhcHBsaWVkLiBBcyB5b3Ug YXJlIHBsYXlpbmcgd2l0aCBtaWNyby1iZW5jaG1hcmtzLCBtYXliZQo+IHlvdSBjb3VsZCB0aGUg YmVsb3cgcGF0Y2g/CgoKTGV0IG1lIHRlc3QgaXQuCgpUaGFua3MKCgo+IEl0J3MgdW5mb3J0dW5h dGVseSB1bnRlc3RlZC4KPiBUaGFua3MgYSBsb3QgaW4gYWR2YW5jZSEKPgo+ID09PT4KPiB2aG9z dDogYmxvY2sgc3BlY3VsYXRpb24gb2YgdHJhbnNsYXRlZCBkZXNjcmlwdG9ycwo+Cj4gaW92ZWMg YWRkcmVzc2VzIGNvbWluZyBmcm9tIHZob3N0IGFyZSBhc3N1bWVkIHRvIGJlCj4gcHJlLXZhbGlk YXRlZCwgYnV0IGluIGZhY3QgY2FuIGJlIHNwZWN1bGF0ZWQgdG8gYSB2YWx1ZQo+IG91dCBvZiBy YW5nZS4KPgo+IFVzZXJzcGFjZSBhZGRyZXNzIGFyZSBsYXRlciB2YWxpZGF0ZWQgd2l0aCBhcnJh eV9pbmRleF9ub3NwZWMgc28gd2UgY2FuCj4gYmUgc3VyZSBrZXJuZWwgaW5mbyBkb2VzIG5vdCBs ZWFrIHRocm91Z2ggdGhlc2UgYWRkcmVzc2VzLCBidXQgdmhvc3QKPiBtdXN0IGFsc28gbm90IGxl YWsgdXNlcnNwYWNlIGluZm8gb3V0c2lkZSB0aGUgYWxsb3dlZCBtZW1vcnkgdGFibGUgdG8KPiBn dWVzdHMuCj4KPiBGb2xsb3dpbmcgdGhlIGRlZmVuY2UgaW4gZGVwdGggcHJpbmNpcGxlLCBtYWtl IHN1cmUKPiB0aGUgYWRkcmVzcyBpcyBub3QgdmFsaWRhdGVkIG91dCBvZiBub2RlIHJhbmdlLgo+ Cj4gU2lnbmVkLW9mZi1ieTogTWljaGFlbCBTLiBUc2lya2luIDxtc3RAcmVkaGF0LmNvbT4KPgo+ IC0tLQo+Cj4KPiBkaWZmIC0tZ2l0IGEvZHJpdmVycy92aG9zdC92aG9zdC5jIGIvZHJpdmVycy92 aG9zdC92aG9zdC5jCj4gaW5kZXggNWRjMTc0YWM4Y2FjLi44NjNlMjUwMTFlZjYgMTAwNjQ0Cj4g LS0tIGEvZHJpdmVycy92aG9zdC92aG9zdC5jCj4gKysrIGIvZHJpdmVycy92aG9zdC92aG9zdC5j Cj4gQEAgLTIwNzIsNyArMjA3Niw5IEBAIHN0YXRpYyBpbnQgdHJhbnNsYXRlX2Rlc2Moc3RydWN0 IHZob3N0X3ZpcnRxdWV1ZSAqdnEsIHU2NCBhZGRyLCB1MzIgbGVuLAo+ICAgCQlzaXplID0gbm9k ZS0+c2l6ZSAtIGFkZHIgKyBub2RlLT5zdGFydDsKPiAgIAkJX2lvdi0+aW92X2xlbiA9IG1pbigo dTY0KWxlbiAtIHMsIHNpemUpOwo+ICAgCQlfaW92LT5pb3ZfYmFzZSA9ICh2b2lkIF9fdXNlciAq KSh1bnNpZ25lZCBsb25nKQo+IC0JCQkobm9kZS0+dXNlcnNwYWNlX2FkZHIgKyBhZGRyIC0gbm9k ZS0+c3RhcnQpOwo+ICsJCQkobm9kZS0+dXNlcnNwYWNlX2FkZHIgKwo+ICsJCQkgYXJyYXlfaW5k ZXhfbm9zcGVjKGFkZHIgLSBub2RlLT5zdGFydCwKPiArCQkJCQkgICAgbm9kZS0+c2l6ZSkpOwo+ ICAgCQlzICs9IHNpemU7Cj4gICAJCWFkZHIgKz0gc2l6ZTsKPiAgIAkJKytyZXQ7CgpfX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVs IG1haWxpbmcgbGlzdApsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDov L2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg==