From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A196C433E6 for ; Fri, 5 Mar 2021 03:36:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7163F64FEE for ; Fri, 5 Mar 2021 03:36:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229570AbhCEDgP (ORCPT ); Thu, 4 Mar 2021 22:36:15 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:58598 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229494AbhCEDgO (ORCPT ); Thu, 4 Mar 2021 22:36:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614915374; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+X/Ve1rVomErqHpAogwhV+VVtnjkivbivYSKCQWULaw=; b=ij4NpLQmFxylGUuBylsL6gGFs0ATKoPxrX0qoppaePA3AMBjFiS7/2RDKOBB41NMMYQ7ap 83Z0qdQuFZ7M7YYIwZTxbeWOTh7cJOdP7c0RSieb5NNCgP4L9Qigd093vq299Eef+xEp/w znG9HxdDxq4B1swfRbr7VVmPsZE+yZ4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-284-dWqYCSG1OP-6d8i94gU5Ng-1; Thu, 04 Mar 2021 22:36:10 -0500 X-MC-Unique: dWqYCSG1OP-6d8i94gU5Ng-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1D8EA10059D0; Fri, 5 Mar 2021 03:36:08 +0000 (UTC) Received: from wangxiaodeMacBook-Air.local (ovpn-13-196.pek2.redhat.com [10.72.13.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id C114B694C9; Fri, 5 Mar 2021 03:35:54 +0000 (UTC) Subject: Re: [RFC v4 06/11] vduse: Implement an MMU-based IOMMU driver To: Yongji Xie Cc: "Michael S. Tsirkin" , Stefan Hajnoczi , Stefano Garzarella , Parav Pandit , Bob Liu , Christoph Hellwig , Randy Dunlap , Matthew Wilcox , viro@zeniv.linux.org.uk, Jens Axboe , bcrl@kvack.org, Jonathan Corbet , virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, kvm@vger.kernel.org, linux-aio@kvack.org, linux-fsdevel@vger.kernel.org References: <20210223115048.435-1-xieyongji@bytedance.com> <20210223115048.435-7-xieyongji@bytedance.com> <573ab913-55ce-045a-478f-1200bd78cf7b@redhat.com> From: Jason Wang Message-ID: Date: Fri, 5 Mar 2021 11:35:53 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 2021/3/4 1:12 下午, Yongji Xie wrote: > On Thu, Mar 4, 2021 at 12:21 PM Jason Wang wrote: >> >> On 2021/2/23 7:50 下午, Xie Yongji wrote: >>> This implements a MMU-based IOMMU driver to support mapping >>> kernel dma buffer into userspace. The basic idea behind it is >>> treating MMU (VA->PA) as IOMMU (IOVA->PA). The driver will set >>> up MMU mapping instead of IOMMU mapping for the DMA transfer so >>> that the userspace process is able to use its virtual address to >>> access the dma buffer in kernel. >>> >>> And to avoid security issue, a bounce-buffering mechanism is >>> introduced to prevent userspace accessing the original buffer >>> directly. >>> >>> Signed-off-by: Xie Yongji >>> --- >>> drivers/vdpa/vdpa_user/iova_domain.c | 486 +++++++++++++++++++++++++++++++++++ >>> drivers/vdpa/vdpa_user/iova_domain.h | 61 +++++ >>> 2 files changed, 547 insertions(+) >>> create mode 100644 drivers/vdpa/vdpa_user/iova_domain.c >>> create mode 100644 drivers/vdpa/vdpa_user/iova_domain.h >>> >>> diff --git a/drivers/vdpa/vdpa_user/iova_domain.c b/drivers/vdpa/vdpa_user/iova_domain.c >>> new file mode 100644 >>> index 000000000000..9285d430d486 >>> --- /dev/null >>> +++ b/drivers/vdpa/vdpa_user/iova_domain.c >>> @@ -0,0 +1,486 @@ >>> +// SPDX-License-Identifier: GPL-2.0-only >>> +/* >>> + * MMU-based IOMMU implementation >>> + * >>> + * Copyright (C) 2020 Bytedance Inc. and/or its affiliates. All rights reserved. >>> + * >>> + * Author: Xie Yongji >>> + * >>> + */ >>> + >>> +#include >>> +#include >>> +#include >>> +#include >>> + >>> +#include "iova_domain.h" >>> + >>> +#define IOVA_START_PFN 1 >>> +#define IOVA_ALLOC_ORDER 12 >>> +#define IOVA_ALLOC_SIZE (1 << IOVA_ALLOC_ORDER) >>> + >>> +static inline struct page * >>> +vduse_domain_get_bounce_page(struct vduse_iova_domain *domain, u64 iova) >>> +{ >>> + u64 index = iova >> PAGE_SHIFT; >>> + >>> + return domain->bounce_pages[index]; >>> +} >>> + >>> +static inline void >>> +vduse_domain_set_bounce_page(struct vduse_iova_domain *domain, >>> + u64 iova, struct page *page) >>> +{ >>> + u64 index = iova >> PAGE_SHIFT; >>> + >>> + domain->bounce_pages[index] = page; >>> +} >>> + >>> +static enum dma_data_direction perm_to_dir(int perm) >>> +{ >>> + enum dma_data_direction dir; >>> + >>> + switch (perm) { >>> + case VHOST_MAP_WO: >>> + dir = DMA_FROM_DEVICE; >>> + break; >>> + case VHOST_MAP_RO: >>> + dir = DMA_TO_DEVICE; >>> + break; >>> + case VHOST_MAP_RW: >>> + dir = DMA_BIDIRECTIONAL; >>> + break; >>> + default: >>> + break; >>> + } >>> + >>> + return dir; >>> +} >>> + >>> +static int dir_to_perm(enum dma_data_direction dir) >>> +{ >>> + int perm = -EFAULT; >>> + >>> + switch (dir) { >>> + case DMA_FROM_DEVICE: >>> + perm = VHOST_MAP_WO; >>> + break; >>> + case DMA_TO_DEVICE: >>> + perm = VHOST_MAP_RO; >>> + break; >>> + case DMA_BIDIRECTIONAL: >>> + perm = VHOST_MAP_RW; >>> + break; >>> + default: >>> + break; >>> + } >>> + >>> + return perm; >>> +} >> >> Let's move the above two helpers to vhost_iotlb.h so they could be used >> by other driver e.g (vpda_sim) >> > Sure. > >>> + >>> +static void do_bounce(phys_addr_t orig, void *addr, size_t size, >>> + enum dma_data_direction dir) >>> +{ >>> + unsigned long pfn = PFN_DOWN(orig); >>> + >>> + if (PageHighMem(pfn_to_page(pfn))) { >>> + unsigned int offset = offset_in_page(orig); >>> + char *buffer; >>> + unsigned int sz = 0; >>> + unsigned long flags; >>> + >>> + while (size) { >>> + sz = min_t(size_t, PAGE_SIZE - offset, size); >>> + >>> + local_irq_save(flags); >>> + buffer = kmap_atomic(pfn_to_page(pfn)); >>> + if (dir == DMA_TO_DEVICE) >>> + memcpy(addr, buffer + offset, sz); >>> + else >>> + memcpy(buffer + offset, addr, sz); >>> + kunmap_atomic(buffer); >>> + local_irq_restore(flags); >> >> I wonder why we need to deal with highmem and irq flags explicitly like >> this. Doesn't kmap_atomic() will take care all of those? >> > Yes, irq flags is useless here. Will remove it. > >>> + >>> + size -= sz; >>> + pfn++; >>> + addr += sz; >>> + offset = 0; >>> + } >>> + } else if (dir == DMA_TO_DEVICE) { >>> + memcpy(addr, phys_to_virt(orig), size); >>> + } else { >>> + memcpy(phys_to_virt(orig), addr, size); >>> + } >>> +} >>> + >>> +static struct page * >>> +vduse_domain_get_mapping_page(struct vduse_iova_domain *domain, u64 iova) >>> +{ >>> + u64 start = iova & PAGE_MASK; >>> + u64 last = start + PAGE_SIZE - 1; >>> + struct vhost_iotlb_map *map; >>> + struct page *page = NULL; >>> + >>> + spin_lock(&domain->iotlb_lock); >>> + map = vhost_iotlb_itree_first(domain->iotlb, start, last); >>> + if (!map) >>> + goto out; >>> + >>> + page = pfn_to_page((map->addr + iova - map->start) >> PAGE_SHIFT); >>> + get_page(page); >>> +out: >>> + spin_unlock(&domain->iotlb_lock); >>> + >>> + return page; >>> +} >>> + >>> +static struct page * >>> +vduse_domain_alloc_bounce_page(struct vduse_iova_domain *domain, u64 iova) >>> +{ >>> + u64 start = iova & PAGE_MASK; >>> + u64 last = start + PAGE_SIZE - 1; >>> + struct vhost_iotlb_map *map; >>> + struct page *page = NULL, *new_page = alloc_page(GFP_KERNEL); >>> + >>> + if (!new_page) >>> + return NULL; >>> + >>> + spin_lock(&domain->iotlb_lock); >>> + if (!vhost_iotlb_itree_first(domain->iotlb, start, last)) { >>> + __free_page(new_page); >>> + goto out; >>> + } >>> + page = vduse_domain_get_bounce_page(domain, iova); >>> + if (page) { >>> + get_page(page); >>> + __free_page(new_page); >>> + goto out; >>> + } >>> + vduse_domain_set_bounce_page(domain, iova, new_page); >>> + get_page(new_page); >>> + page = new_page; >>> + >>> + for (map = vhost_iotlb_itree_first(domain->iotlb, start, last); map; >>> + map = vhost_iotlb_itree_next(map, start, last)) { >>> + unsigned int src_offset = 0, dst_offset = 0; >>> + phys_addr_t src; >>> + void *dst; >>> + size_t sz; >>> + >>> + if (perm_to_dir(map->perm) == DMA_FROM_DEVICE) >>> + continue; >>> + >>> + if (start > map->start) >>> + src_offset = start - map->start; >>> + else >>> + dst_offset = map->start - start; >>> + >>> + src = map->addr + src_offset; >>> + dst = page_address(page) + dst_offset; >>> + sz = min_t(size_t, map->size - src_offset, >>> + PAGE_SIZE - dst_offset); >>> + do_bounce(src, dst, sz, DMA_TO_DEVICE); >>> + } >>> +out: >>> + spin_unlock(&domain->iotlb_lock); >>> + >>> + return page; >>> +} >>> + >>> +static void >>> +vduse_domain_free_bounce_pages(struct vduse_iova_domain *domain, >>> + u64 iova, size_t size) >>> +{ >>> + struct page *page; >>> + >>> + spin_lock(&domain->iotlb_lock); >>> + if (WARN_ON(vhost_iotlb_itree_first(domain->iotlb, iova, >>> + iova + size - 1))) >>> + goto out; >>> + >>> + while (size > 0) { >>> + page = vduse_domain_get_bounce_page(domain, iova); >>> + if (page) { >>> + vduse_domain_set_bounce_page(domain, iova, NULL); >>> + __free_page(page); >>> + } >>> + size -= PAGE_SIZE; >>> + iova += PAGE_SIZE; >>> + } >>> +out: >>> + spin_unlock(&domain->iotlb_lock); >>> +} >>> + >>> +static void vduse_domain_bounce(struct vduse_iova_domain *domain, >>> + dma_addr_t iova, phys_addr_t orig, >>> + size_t size, enum dma_data_direction dir) >>> +{ >>> + unsigned int offset = offset_in_page(iova); >>> + >>> + while (size) { >>> + struct page *p = vduse_domain_get_bounce_page(domain, iova); >>> + size_t sz = min_t(size_t, PAGE_SIZE - offset, size); >>> + >>> + WARN_ON(!p && dir == DMA_FROM_DEVICE); >>> + >>> + if (p) >>> + do_bounce(orig, page_address(p) + offset, sz, dir); >>> + >>> + size -= sz; >>> + orig += sz; >>> + iova += sz; >>> + offset = 0; >>> + } >>> +} >>> + >>> +static dma_addr_t vduse_domain_alloc_iova(struct iova_domain *iovad, >>> + unsigned long size, unsigned long limit) >>> +{ >>> + unsigned long shift = iova_shift(iovad); >>> + unsigned long iova_len = iova_align(iovad, size) >> shift; >>> + unsigned long iova_pfn; >>> + >>> + if (iova_len < (1 << (IOVA_RANGE_CACHE_MAX_SIZE - 1))) >>> + iova_len = roundup_pow_of_two(iova_len); >>> + iova_pfn = alloc_iova_fast(iovad, iova_len, limit >> shift, true); >>> + >>> + return iova_pfn << shift; >>> +} >>> + >>> +static void vduse_domain_free_iova(struct iova_domain *iovad, >>> + dma_addr_t iova, size_t size) >>> +{ >>> + unsigned long shift = iova_shift(iovad); >>> + unsigned long iova_len = iova_align(iovad, size) >> shift; >>> + >>> + free_iova_fast(iovad, iova >> shift, iova_len); >>> +} >>> + >>> +dma_addr_t vduse_domain_map_page(struct vduse_iova_domain *domain, >>> + struct page *page, unsigned long offset, >>> + size_t size, enum dma_data_direction dir, >>> + unsigned long attrs) >>> +{ >>> + struct iova_domain *iovad = &domain->stream_iovad; >>> + unsigned long limit = domain->bounce_size - 1; >>> + phys_addr_t pa = page_to_phys(page) + offset; >>> + dma_addr_t iova = vduse_domain_alloc_iova(iovad, size, limit); >>> + int ret; >>> + >>> + if (!iova) >>> + return DMA_MAPPING_ERROR; >>> + >>> + spin_lock(&domain->iotlb_lock); >>> + ret = vhost_iotlb_add_range(domain->iotlb, (u64)iova, >>> + (u64)iova + size - 1, >>> + pa, dir_to_perm(dir)); >>> + spin_unlock(&domain->iotlb_lock); >>> + if (ret) { >>> + vduse_domain_free_iova(iovad, iova, size); >>> + return DMA_MAPPING_ERROR; >>> + } >>> + if (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL) >>> + vduse_domain_bounce(domain, iova, pa, size, DMA_TO_DEVICE); >>> + >>> + return iova; >>> +} >>> + >>> +void vduse_domain_unmap_page(struct vduse_iova_domain *domain, >>> + dma_addr_t dma_addr, size_t size, >>> + enum dma_data_direction dir, unsigned long attrs) >>> +{ >>> + struct iova_domain *iovad = &domain->stream_iovad; >>> + struct vhost_iotlb_map *map; >>> + phys_addr_t pa; >>> + >>> + spin_lock(&domain->iotlb_lock); >>> + map = vhost_iotlb_itree_first(domain->iotlb, (u64)dma_addr, >>> + (u64)dma_addr + size - 1); >>> + if (WARN_ON(!map)) { >>> + spin_unlock(&domain->iotlb_lock); >>> + return; >>> + } >>> + pa = map->addr; >>> + vhost_iotlb_map_free(domain->iotlb, map); >>> + spin_unlock(&domain->iotlb_lock); >>> + >>> + if (dir == DMA_FROM_DEVICE || dir == DMA_BIDIRECTIONAL) >>> + vduse_domain_bounce(domain, dma_addr, pa, >>> + size, DMA_FROM_DEVICE); >>> + >>> + vduse_domain_free_iova(iovad, dma_addr, size); >>> +} >>> + >>> +void *vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, >>> + size_t size, dma_addr_t *dma_addr, >>> + gfp_t flag, unsigned long attrs) >>> +{ >>> + struct iova_domain *iovad = &domain->consistent_iovad; >>> + unsigned long limit = domain->iova_limit; >>> + dma_addr_t iova = vduse_domain_alloc_iova(iovad, size, limit); >>> + void *orig = alloc_pages_exact(size, flag); >>> + int ret; >>> + >>> + if (!iova || !orig) >>> + goto err; >>> + >>> + spin_lock(&domain->iotlb_lock); >>> + ret = vhost_iotlb_add_range(domain->iotlb, (u64)iova, >>> + (u64)iova + size - 1, >>> + virt_to_phys(orig), VHOST_MAP_RW); >>> + spin_unlock(&domain->iotlb_lock); >>> + if (ret) >>> + goto err; >>> + >>> + *dma_addr = iova; >>> + >>> + return orig; >>> +err: >>> + *dma_addr = DMA_MAPPING_ERROR; >>> + if (orig) >>> + free_pages_exact(orig, size); >>> + if (iova) >>> + vduse_domain_free_iova(iovad, iova, size); >>> + >>> + return NULL; >>> +} >>> + >>> +void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size, >>> + void *vaddr, dma_addr_t dma_addr, >>> + unsigned long attrs) >>> +{ >>> + struct iova_domain *iovad = &domain->consistent_iovad; >>> + struct vhost_iotlb_map *map; >>> + phys_addr_t pa; >>> + >>> + spin_lock(&domain->iotlb_lock); >>> + map = vhost_iotlb_itree_first(domain->iotlb, (u64)dma_addr, >>> + (u64)dma_addr + size - 1); >>> + if (WARN_ON(!map)) { >>> + spin_unlock(&domain->iotlb_lock); >>> + return; >>> + } >>> + pa = map->addr; >>> + vhost_iotlb_map_free(domain->iotlb, map); >>> + spin_unlock(&domain->iotlb_lock); >>> + >>> + vduse_domain_free_iova(iovad, dma_addr, size); >>> + free_pages_exact(phys_to_virt(pa), size); >>> +} >>> + >>> +static vm_fault_t vduse_domain_mmap_fault(struct vm_fault *vmf) >>> +{ >>> + struct vduse_iova_domain *domain = vmf->vma->vm_private_data; >>> + unsigned long iova = vmf->pgoff << PAGE_SHIFT; >>> + struct page *page; >>> + >>> + if (!domain) >>> + return VM_FAULT_SIGBUS; >>> + >>> + if (iova < domain->bounce_size) >>> + page = vduse_domain_alloc_bounce_page(domain, iova); >>> + else >>> + page = vduse_domain_get_mapping_page(domain, iova); >>> + >>> + if (!page) >>> + return VM_FAULT_SIGBUS; >>> + >>> + vmf->page = page; >>> + >>> + return 0; >>> +} >>> + >>> +static const struct vm_operations_struct vduse_domain_mmap_ops = { >>> + .fault = vduse_domain_mmap_fault, >>> +}; >>> + >>> +static int vduse_domain_mmap(struct file *file, struct vm_area_struct *vma) >>> +{ >>> + struct vduse_iova_domain *domain = file->private_data; >>> + >>> + vma->vm_flags |= VM_DONTDUMP | VM_DONTEXPAND; >>> + vma->vm_private_data = domain; >>> + vma->vm_ops = &vduse_domain_mmap_ops; >>> + >>> + return 0; >>> +} >>> + >>> +static int vduse_domain_release(struct inode *inode, struct file *file) >>> +{ >>> + struct vduse_iova_domain *domain = file->private_data; >>> + >>> + vduse_domain_free_bounce_pages(domain, 0, domain->bounce_size); >>> + put_iova_domain(&domain->stream_iovad); >>> + put_iova_domain(&domain->consistent_iovad); >>> + vhost_iotlb_free(domain->iotlb); >>> + vfree(domain->bounce_pages); >>> + kfree(domain); >>> + >>> + return 0; >>> +} >>> + >>> +static const struct file_operations vduse_domain_fops = { >>> + .mmap = vduse_domain_mmap, >>> + .release = vduse_domain_release, >>> +}; >>> + >>> +void vduse_domain_destroy(struct vduse_iova_domain *domain) >>> +{ >>> + fput(domain->file); >>> +} >>> + >>> +struct vduse_iova_domain * >>> +vduse_domain_create(unsigned long iova_limit, size_t bounce_size) >>> +{ >>> + struct vduse_iova_domain *domain; >>> + struct file *file; >>> + unsigned long bounce_pfns = PAGE_ALIGN(bounce_size) >> PAGE_SHIFT; >>> + >>> + if (iova_limit <= bounce_size) >>> + return NULL; >>> + >>> + domain = kzalloc(sizeof(*domain), GFP_KERNEL); >>> + if (!domain) >>> + return NULL; >>> + >>> + domain->iotlb = vhost_iotlb_alloc(0, 0); >>> + if (!domain->iotlb) >>> + goto err_iotlb; >>> + >>> + domain->iova_limit = iova_limit; >>> + domain->bounce_size = PAGE_ALIGN(bounce_size); >>> + domain->bounce_pages = vzalloc(bounce_pfns * sizeof(struct page *)); >>> + if (!domain->bounce_pages) >>> + goto err_page; >>> + >>> + file = anon_inode_getfile("[vduse-domain]", &vduse_domain_fops, >>> + domain, O_RDWR); >>> + if (IS_ERR(file)) >>> + goto err_file; >>> + >>> + domain->file = file; >>> + spin_lock_init(&domain->iotlb_lock); >>> + init_iova_domain(&domain->stream_iovad, >>> + IOVA_ALLOC_SIZE, IOVA_START_PFN); >>> + init_iova_domain(&domain->consistent_iovad, >>> + PAGE_SIZE, bounce_pfns); >>> + >>> + return domain; >>> +err_file: >>> + vfree(domain->bounce_pages); >>> +err_page: >>> + vhost_iotlb_free(domain->iotlb); >>> +err_iotlb: >>> + kfree(domain); >>> + return NULL; >>> +} >>> + >>> +int vduse_domain_init(void) >>> +{ >>> + return iova_cache_get(); >>> +} >>> + >>> +void vduse_domain_exit(void) >>> +{ >>> + iova_cache_put(); >>> +} >>> diff --git a/drivers/vdpa/vdpa_user/iova_domain.h b/drivers/vdpa/vdpa_user/iova_domain.h >>> new file mode 100644 >>> index 000000000000..9c85d8346626 >>> --- /dev/null >>> +++ b/drivers/vdpa/vdpa_user/iova_domain.h >>> @@ -0,0 +1,61 @@ >>> +/* SPDX-License-Identifier: GPL-2.0-only */ >>> +/* >>> + * MMU-based IOMMU implementation >>> + * >>> + * Copyright (C) 2020 Bytedance Inc. and/or its affiliates. All rights reserved. >>> + * >>> + * Author: Xie Yongji >>> + * >>> + */ >>> + >>> +#ifndef _VDUSE_IOVA_DOMAIN_H >>> +#define _VDUSE_IOVA_DOMAIN_H >>> + >>> +#include >>> +#include >>> +#include >>> + >>> +struct vduse_iova_domain { >>> + struct iova_domain stream_iovad; >>> + struct iova_domain consistent_iovad; >>> + struct page **bounce_pages; >>> + size_t bounce_size; >>> + unsigned long iova_limit; >>> + struct vhost_iotlb *iotlb; >> >> Sorry if I've asked this before. >> >> But what's the reason for maintaing a dedicated IOTLB here? I think we >> could reuse vduse_dev->iommu since the device can not be used by both >> virtio and vhost in the same time or use vduse_iova_domain->iotlb for >> set_map(). >> > The main difference between domain->iotlb and dev->iotlb is the way to > deal with bounce buffer. In the domain->iotlb case, bounce buffer > needs to be mapped each DMA transfer because we need to get the bounce > pages by an IOVA during DMA unmapping. In the dev->iotlb case, bounce > buffer only needs to be mapped once during initialization, which will > be used to tell userspace how to do mmap(). > >> Also, since vhost IOTLB support per mapping token (opauqe), can we use >> that instead of the bounce_pages *? >> > Sorry, I didn't get you here. Which value do you mean to store in the > opaque pointer？ So I would like to have a way to use a single IOTLB for manage all kinds of mappings. Two possible ideas: 1) map bounce page one by one in vduse_dev_map_page(), in VDUSE_IOTLB_GET_FD, try to merge the result if we had the same fd. Then for bounce pages, userspace still only need to map it once and we can maintain the actual mapping by storing the page or pa in the opaque field of IOTLB entry. 2) map bounce page once in vduse_dev_map_page() and store struct page **bounce_pages in the opaque field of this single IOTLB entry. Does this work? Thanks > > Thanks, > Yongji > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.1 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6240EC433E0 for ; Fri, 5 Mar 2021 03:36:21 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F377764FEF for ; Fri, 5 Mar 2021 03:36:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F377764FEF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=virtualization-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id A85CF4ECA2; Fri, 5 Mar 2021 03:36:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rcZqTWmakAAO; Fri, 5 Mar 2021 03:36:19 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTP id 822E94ECA0; Fri, 5 Mar 2021 03:36:18 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 533CDC000B; Fri, 5 Mar 2021 03:36:18 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3709EC0001 for ; Fri, 5 Mar 2021 03:36:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 1C7234ECA0 for ; Fri, 5 Mar 2021 03:36:17 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jaMM1JKEPEPl for ; Fri, 5 Mar 2021 03:36:15 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 691204BD65 for ; Fri, 5 Mar 2021 03:36:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614915374; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+X/Ve1rVomErqHpAogwhV+VVtnjkivbivYSKCQWULaw=; b=ij4NpLQmFxylGUuBylsL6gGFs0ATKoPxrX0qoppaePA3AMBjFiS7/2RDKOBB41NMMYQ7ap 83Z0qdQuFZ7M7YYIwZTxbeWOTh7cJOdP7c0RSieb5NNCgP4L9Qigd093vq299Eef+xEp/w znG9HxdDxq4B1swfRbr7VVmPsZE+yZ4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-284-dWqYCSG1OP-6d8i94gU5Ng-1; Thu, 04 Mar 2021 22:36:10 -0500 X-MC-Unique: dWqYCSG1OP-6d8i94gU5Ng-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1D8EA10059D0; Fri, 5 Mar 2021 03:36:08 +0000 (UTC) Received: from wangxiaodeMacBook-Air.local (ovpn-13-196.pek2.redhat.com [10.72.13.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id C114B694C9; Fri, 5 Mar 2021 03:35:54 +0000 (UTC) Subject: Re: [RFC v4 06/11] vduse: Implement an MMU-based IOMMU driver To: Yongji Xie References: <20210223115048.435-1-xieyongji@bytedance.com> <20210223115048.435-7-xieyongji@bytedance.com> <573ab913-55ce-045a-478f-1200bd78cf7b@redhat.com> From: Jason Wang Message-ID: Date: Fri, 5 Mar 2021 11:35:53 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-GB X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Cc: Jens Axboe , Jonathan Corbet , kvm@vger.kernel.org, "Michael S. Tsirkin" , linux-aio@kvack.org, netdev@vger.kernel.org, Randy Dunlap , Matthew Wilcox , virtualization@lists.linux-foundation.org, Christoph Hellwig , Bob Liu , bcrl@kvack.org, viro@zeniv.linux.org.uk, Stefan Hajnoczi , linux-fsdevel@vger.kernel.org X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" Ck9uIDIwMjEvMy80IDE6MTIg5LiL5Y2ILCBZb25namkgWGllIHdyb3RlOgo+IE9uIFRodSwgTWFy IDQsIDIwMjEgYXQgMTI6MjEgUE0gSmFzb24gV2FuZyA8amFzb3dhbmdAcmVkaGF0LmNvbT4gd3Jv dGU6Cj4+Cj4+IE9uIDIwMjEvMi8yMyA3OjUwIOS4i+WNiCwgWGllIFlvbmdqaSB3cm90ZToKPj4+ IFRoaXMgaW1wbGVtZW50cyBhIE1NVS1iYXNlZCBJT01NVSBkcml2ZXIgdG8gc3VwcG9ydCBtYXBw aW5nCj4+PiBrZXJuZWwgZG1hIGJ1ZmZlciBpbnRvIHVzZXJzcGFjZS4gVGhlIGJhc2ljIGlkZWEg YmVoaW5kIGl0IGlzCj4+PiB0cmVhdGluZyBNTVUgKFZBLT5QQSkgYXMgSU9NTVUgKElPVkEtPlBB KS4gVGhlIGRyaXZlciB3aWxsIHNldAo+Pj4gdXAgTU1VIG1hcHBpbmcgaW5zdGVhZCBvZiBJT01N VSBtYXBwaW5nIGZvciB0aGUgRE1BIHRyYW5zZmVyIHNvCj4+PiB0aGF0IHRoZSB1c2Vyc3BhY2Ug cHJvY2VzcyBpcyBhYmxlIHRvIHVzZSBpdHMgdmlydHVhbCBhZGRyZXNzIHRvCj4+PiBhY2Nlc3Mg dGhlIGRtYSBidWZmZXIgaW4ga2VybmVsLgo+Pj4KPj4+IEFuZCB0byBhdm9pZCBzZWN1cml0eSBp c3N1ZSwgYSBib3VuY2UtYnVmZmVyaW5nIG1lY2hhbmlzbSBpcwo+Pj4gaW50cm9kdWNlZCB0byBw cmV2ZW50IHVzZXJzcGFjZSBhY2Nlc3NpbmcgdGhlIG9yaWdpbmFsIGJ1ZmZlcgo+Pj4gZGlyZWN0 bHkuCj4+Pgo+Pj4gU2lnbmVkLW9mZi1ieTogWGllIFlvbmdqaSA8eGlleW9uZ2ppQGJ5dGVkYW5j ZS5jb20+Cj4+PiAtLS0KPj4+ICAgIGRyaXZlcnMvdmRwYS92ZHBhX3VzZXIvaW92YV9kb21haW4u YyB8IDQ4NiArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKwo+Pj4gICAgZHJpdmVy cy92ZHBhL3ZkcGFfdXNlci9pb3ZhX2RvbWFpbi5oIHwgIDYxICsrKysrCj4+PiAgICAyIGZpbGVz IGNoYW5nZWQsIDU0NyBpbnNlcnRpb25zKCspCj4+PiAgICBjcmVhdGUgbW9kZSAxMDA2NDQgZHJp dmVycy92ZHBhL3ZkcGFfdXNlci9pb3ZhX2RvbWFpbi5jCj4+PiAgICBjcmVhdGUgbW9kZSAxMDA2 NDQgZHJpdmVycy92ZHBhL3ZkcGFfdXNlci9pb3ZhX2RvbWFpbi5oCj4+Pgo+Pj4gZGlmZiAtLWdp dCBhL2RyaXZlcnMvdmRwYS92ZHBhX3VzZXIvaW92YV9kb21haW4uYyBiL2RyaXZlcnMvdmRwYS92 ZHBhX3VzZXIvaW92YV9kb21haW4uYwo+Pj4gbmV3IGZpbGUgbW9kZSAxMDA2NDQKPj4+IGluZGV4 IDAwMDAwMDAwMDAwMC4uOTI4NWQ0MzBkNDg2Cj4+PiAtLS0gL2Rldi9udWxsCj4+PiArKysgYi9k cml2ZXJzL3ZkcGEvdmRwYV91c2VyL2lvdmFfZG9tYWluLmMKPj4+IEBAIC0wLDAgKzEsNDg2IEBA Cj4+PiArLy8gU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0yLjAtb25seQo+Pj4gKy8qCj4+ PiArICogTU1VLWJhc2VkIElPTU1VIGltcGxlbWVudGF0aW9uCj4+PiArICoKPj4+ICsgKiBDb3B5 cmlnaHQgKEMpIDIwMjAgQnl0ZWRhbmNlIEluYy4gYW5kL29yIGl0cyBhZmZpbGlhdGVzLiBBbGwg cmlnaHRzIHJlc2VydmVkLgo+Pj4gKyAqCj4+PiArICogQXV0aG9yOiBYaWUgWW9uZ2ppIDx4aWV5 b25namlAYnl0ZWRhbmNlLmNvbT4KPj4+ICsgKgo+Pj4gKyAqLwo+Pj4gKwo+Pj4gKyNpbmNsdWRl IDxsaW51eC9zbGFiLmg+Cj4+PiArI2luY2x1ZGUgPGxpbnV4L2ZpbGUuaD4KPj4+ICsjaW5jbHVk ZSA8bGludXgvYW5vbl9pbm9kZXMuaD4KPj4+ICsjaW5jbHVkZSA8bGludXgvaGlnaG1lbS5oPgo+ Pj4gKwo+Pj4gKyNpbmNsdWRlICJpb3ZhX2RvbWFpbi5oIgo+Pj4gKwo+Pj4gKyNkZWZpbmUgSU9W QV9TVEFSVF9QRk4gMQo+Pj4gKyNkZWZpbmUgSU9WQV9BTExPQ19PUkRFUiAxMgo+Pj4gKyNkZWZp bmUgSU9WQV9BTExPQ19TSVpFICgxIDw8IElPVkFfQUxMT0NfT1JERVIpCj4+PiArCj4+PiArc3Rh dGljIGlubGluZSBzdHJ1Y3QgcGFnZSAqCj4+PiArdmR1c2VfZG9tYWluX2dldF9ib3VuY2VfcGFn ZShzdHJ1Y3QgdmR1c2VfaW92YV9kb21haW4gKmRvbWFpbiwgdTY0IGlvdmEpCj4+PiArewo+Pj4g KyAgICAgdTY0IGluZGV4ID0gaW92YSA+PiBQQUdFX1NISUZUOwo+Pj4gKwo+Pj4gKyAgICAgcmV0 dXJuIGRvbWFpbi0+Ym91bmNlX3BhZ2VzW2luZGV4XTsKPj4+ICt9Cj4+PiArCj4+PiArc3RhdGlj IGlubGluZSB2b2lkCj4+PiArdmR1c2VfZG9tYWluX3NldF9ib3VuY2VfcGFnZShzdHJ1Y3QgdmR1 c2VfaW92YV9kb21haW4gKmRvbWFpbiwKPj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAg IHU2NCBpb3ZhLCBzdHJ1Y3QgcGFnZSAqcGFnZSkKPj4+ICt7Cj4+PiArICAgICB1NjQgaW5kZXgg PSBpb3ZhID4+IFBBR0VfU0hJRlQ7Cj4+PiArCj4+PiArICAgICBkb21haW4tPmJvdW5jZV9wYWdl c1tpbmRleF0gPSBwYWdlOwo+Pj4gK30KPj4+ICsKPj4+ICtzdGF0aWMgZW51bSBkbWFfZGF0YV9k aXJlY3Rpb24gcGVybV90b19kaXIoaW50IHBlcm0pCj4+PiArewo+Pj4gKyAgICAgZW51bSBkbWFf ZGF0YV9kaXJlY3Rpb24gZGlyOwo+Pj4gKwo+Pj4gKyAgICAgc3dpdGNoIChwZXJtKSB7Cj4+PiAr ICAgICBjYXNlIFZIT1NUX01BUF9XTzoKPj4+ICsgICAgICAgICAgICAgZGlyID0gRE1BX0ZST01f REVWSUNFOwo+Pj4gKyAgICAgICAgICAgICBicmVhazsKPj4+ICsgICAgIGNhc2UgVkhPU1RfTUFQ X1JPOgo+Pj4gKyAgICAgICAgICAgICBkaXIgPSBETUFfVE9fREVWSUNFOwo+Pj4gKyAgICAgICAg ICAgICBicmVhazsKPj4+ICsgICAgIGNhc2UgVkhPU1RfTUFQX1JXOgo+Pj4gKyAgICAgICAgICAg ICBkaXIgPSBETUFfQklESVJFQ1RJT05BTDsKPj4+ICsgICAgICAgICAgICAgYnJlYWs7Cj4+PiAr ICAgICBkZWZhdWx0Ogo+Pj4gKyAgICAgICAgICAgICBicmVhazsKPj4+ICsgICAgIH0KPj4+ICsK Pj4+ICsgICAgIHJldHVybiBkaXI7Cj4+PiArfQo+Pj4gKwo+Pj4gK3N0YXRpYyBpbnQgZGlyX3Rv X3Blcm0oZW51bSBkbWFfZGF0YV9kaXJlY3Rpb24gZGlyKQo+Pj4gK3sKPj4+ICsgICAgIGludCBw ZXJtID0gLUVGQVVMVDsKPj4+ICsKPj4+ICsgICAgIHN3aXRjaCAoZGlyKSB7Cj4+PiArICAgICBj YXNlIERNQV9GUk9NX0RFVklDRToKPj4+ICsgICAgICAgICAgICAgcGVybSA9IFZIT1NUX01BUF9X TzsKPj4+ICsgICAgICAgICAgICAgYnJlYWs7Cj4+PiArICAgICBjYXNlIERNQV9UT19ERVZJQ0U6 Cj4+PiArICAgICAgICAgICAgIHBlcm0gPSBWSE9TVF9NQVBfUk87Cj4+PiArICAgICAgICAgICAg IGJyZWFrOwo+Pj4gKyAgICAgY2FzZSBETUFfQklESVJFQ1RJT05BTDoKPj4+ICsgICAgICAgICAg ICAgcGVybSA9IFZIT1NUX01BUF9SVzsKPj4+ICsgICAgICAgICAgICAgYnJlYWs7Cj4+PiArICAg ICBkZWZhdWx0Ogo+Pj4gKyAgICAgICAgICAgICBicmVhazsKPj4+ICsgICAgIH0KPj4+ICsKPj4+ ICsgICAgIHJldHVybiBwZXJtOwo+Pj4gK30KPj4KPj4gTGV0J3MgbW92ZSB0aGUgYWJvdmUgdHdv IGhlbHBlcnMgdG8gdmhvc3RfaW90bGIuaCBzbyB0aGV5IGNvdWxkIGJlIHVzZWQKPj4gYnkgb3Ro ZXIgZHJpdmVyIGUuZyAodnBkYV9zaW0pCj4+Cj4gU3VyZS4KPgo+Pj4gKwo+Pj4gK3N0YXRpYyB2 b2lkIGRvX2JvdW5jZShwaHlzX2FkZHJfdCBvcmlnLCB2b2lkICphZGRyLCBzaXplX3Qgc2l6ZSwK Pj4+ICsgICAgICAgICAgICAgICAgICAgICBlbnVtIGRtYV9kYXRhX2RpcmVjdGlvbiBkaXIpCj4+ PiArewo+Pj4gKyAgICAgdW5zaWduZWQgbG9uZyBwZm4gPSBQRk5fRE9XTihvcmlnKTsKPj4+ICsK Pj4+ICsgICAgIGlmIChQYWdlSGlnaE1lbShwZm5fdG9fcGFnZShwZm4pKSkgewo+Pj4gKyAgICAg ICAgICAgICB1bnNpZ25lZCBpbnQgb2Zmc2V0ID0gb2Zmc2V0X2luX3BhZ2Uob3JpZyk7Cj4+PiAr ICAgICAgICAgICAgIGNoYXIgKmJ1ZmZlcjsKPj4+ICsgICAgICAgICAgICAgdW5zaWduZWQgaW50 IHN6ID0gMDsKPj4+ICsgICAgICAgICAgICAgdW5zaWduZWQgbG9uZyBmbGFnczsKPj4+ICsKPj4+ ICsgICAgICAgICAgICAgd2hpbGUgKHNpemUpIHsKPj4+ICsgICAgICAgICAgICAgICAgICAgICBz eiA9IG1pbl90KHNpemVfdCwgUEFHRV9TSVpFIC0gb2Zmc2V0LCBzaXplKTsKPj4+ICsKPj4+ICsg ICAgICAgICAgICAgICAgICAgICBsb2NhbF9pcnFfc2F2ZShmbGFncyk7Cj4+PiArICAgICAgICAg ICAgICAgICAgICAgYnVmZmVyID0ga21hcF9hdG9taWMocGZuX3RvX3BhZ2UocGZuKSk7Cj4+PiAr ICAgICAgICAgICAgICAgICAgICAgaWYgKGRpciA9PSBETUFfVE9fREVWSUNFKQo+Pj4gKyAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgbWVtY3B5KGFkZHIsIGJ1ZmZlciArIG9mZnNldCwgc3op Owo+Pj4gKyAgICAgICAgICAgICAgICAgICAgIGVsc2UKPj4+ICsgICAgICAgICAgICAgICAgICAg ICAgICAgICAgIG1lbWNweShidWZmZXIgKyBvZmZzZXQsIGFkZHIsIHN6KTsKPj4+ICsgICAgICAg ICAgICAgICAgICAgICBrdW5tYXBfYXRvbWljKGJ1ZmZlcik7Cj4+PiArICAgICAgICAgICAgICAg ICAgICAgbG9jYWxfaXJxX3Jlc3RvcmUoZmxhZ3MpOwo+Pgo+PiBJIHdvbmRlciB3aHkgd2UgbmVl ZCB0byBkZWFsIHdpdGggaGlnaG1lbSBhbmQgaXJxIGZsYWdzIGV4cGxpY2l0bHkgbGlrZQo+PiB0 aGlzLiBEb2Vzbid0IGttYXBfYXRvbWljKCkgd2lsbCB0YWtlIGNhcmUgYWxsIG9mIHRob3NlPwo+ Pgo+IFllcywgaXJxIGZsYWdzIGlzIHVzZWxlc3MgaGVyZS4gV2lsbCByZW1vdmUgaXQuCj4KPj4+ ICsKPj4+ICsgICAgICAgICAgICAgICAgICAgICBzaXplIC09IHN6Owo+Pj4gKyAgICAgICAgICAg ICAgICAgICAgIHBmbisrOwo+Pj4gKyAgICAgICAgICAgICAgICAgICAgIGFkZHIgKz0gc3o7Cj4+ PiArICAgICAgICAgICAgICAgICAgICAgb2Zmc2V0ID0gMDsKPj4+ICsgICAgICAgICAgICAgfQo+ Pj4gKyAgICAgfSBlbHNlIGlmIChkaXIgPT0gRE1BX1RPX0RFVklDRSkgewo+Pj4gKyAgICAgICAg ICAgICBtZW1jcHkoYWRkciwgcGh5c190b192aXJ0KG9yaWcpLCBzaXplKTsKPj4+ICsgICAgIH0g ZWxzZSB7Cj4+PiArICAgICAgICAgICAgIG1lbWNweShwaHlzX3RvX3ZpcnQob3JpZyksIGFkZHIs IHNpemUpOwo+Pj4gKyAgICAgfQo+Pj4gK30KPj4+ICsKPj4+ICtzdGF0aWMgc3RydWN0IHBhZ2Ug Kgo+Pj4gK3ZkdXNlX2RvbWFpbl9nZXRfbWFwcGluZ19wYWdlKHN0cnVjdCB2ZHVzZV9pb3ZhX2Rv bWFpbiAqZG9tYWluLCB1NjQgaW92YSkKPj4+ICt7Cj4+PiArICAgICB1NjQgc3RhcnQgPSBpb3Zh ICYgUEFHRV9NQVNLOwo+Pj4gKyAgICAgdTY0IGxhc3QgPSBzdGFydCArIFBBR0VfU0laRSAtIDE7 Cj4+PiArICAgICBzdHJ1Y3Qgdmhvc3RfaW90bGJfbWFwICptYXA7Cj4+PiArICAgICBzdHJ1Y3Qg cGFnZSAqcGFnZSA9IE5VTEw7Cj4+PiArCj4+PiArICAgICBzcGluX2xvY2soJmRvbWFpbi0+aW90 bGJfbG9jayk7Cj4+PiArICAgICBtYXAgPSB2aG9zdF9pb3RsYl9pdHJlZV9maXJzdChkb21haW4t PmlvdGxiLCBzdGFydCwgbGFzdCk7Cj4+PiArICAgICBpZiAoIW1hcCkKPj4+ICsgICAgICAgICAg ICAgZ290byBvdXQ7Cj4+PiArCj4+PiArICAgICBwYWdlID0gcGZuX3RvX3BhZ2UoKG1hcC0+YWRk ciArIGlvdmEgLSBtYXAtPnN0YXJ0KSA+PiBQQUdFX1NISUZUKTsKPj4+ICsgICAgIGdldF9wYWdl KHBhZ2UpOwo+Pj4gK291dDoKPj4+ICsgICAgIHNwaW5fdW5sb2NrKCZkb21haW4tPmlvdGxiX2xv Y2spOwo+Pj4gKwo+Pj4gKyAgICAgcmV0dXJuIHBhZ2U7Cj4+PiArfQo+Pj4gKwo+Pj4gK3N0YXRp YyBzdHJ1Y3QgcGFnZSAqCj4+PiArdmR1c2VfZG9tYWluX2FsbG9jX2JvdW5jZV9wYWdlKHN0cnVj dCB2ZHVzZV9pb3ZhX2RvbWFpbiAqZG9tYWluLCB1NjQgaW92YSkKPj4+ICt7Cj4+PiArICAgICB1 NjQgc3RhcnQgPSBpb3ZhICYgUEFHRV9NQVNLOwo+Pj4gKyAgICAgdTY0IGxhc3QgPSBzdGFydCAr IFBBR0VfU0laRSAtIDE7Cj4+PiArICAgICBzdHJ1Y3Qgdmhvc3RfaW90bGJfbWFwICptYXA7Cj4+ PiArICAgICBzdHJ1Y3QgcGFnZSAqcGFnZSA9IE5VTEwsICpuZXdfcGFnZSA9IGFsbG9jX3BhZ2Uo R0ZQX0tFUk5FTCk7Cj4+PiArCj4+PiArICAgICBpZiAoIW5ld19wYWdlKQo+Pj4gKyAgICAgICAg ICAgICByZXR1cm4gTlVMTDsKPj4+ICsKPj4+ICsgICAgIHNwaW5fbG9jaygmZG9tYWluLT5pb3Rs Yl9sb2NrKTsKPj4+ICsgICAgIGlmICghdmhvc3RfaW90bGJfaXRyZWVfZmlyc3QoZG9tYWluLT5p b3RsYiwgc3RhcnQsIGxhc3QpKSB7Cj4+PiArICAgICAgICAgICAgIF9fZnJlZV9wYWdlKG5ld19w YWdlKTsKPj4+ICsgICAgICAgICAgICAgZ290byBvdXQ7Cj4+PiArICAgICB9Cj4+PiArICAgICBw YWdlID0gdmR1c2VfZG9tYWluX2dldF9ib3VuY2VfcGFnZShkb21haW4sIGlvdmEpOwo+Pj4gKyAg ICAgaWYgKHBhZ2UpIHsKPj4+ICsgICAgICAgICAgICAgZ2V0X3BhZ2UocGFnZSk7Cj4+PiArICAg ICAgICAgICAgIF9fZnJlZV9wYWdlKG5ld19wYWdlKTsKPj4+ICsgICAgICAgICAgICAgZ290byBv dXQ7Cj4+PiArICAgICB9Cj4+PiArICAgICB2ZHVzZV9kb21haW5fc2V0X2JvdW5jZV9wYWdlKGRv bWFpbiwgaW92YSwgbmV3X3BhZ2UpOwo+Pj4gKyAgICAgZ2V0X3BhZ2UobmV3X3BhZ2UpOwo+Pj4g KyAgICAgcGFnZSA9IG5ld19wYWdlOwo+Pj4gKwo+Pj4gKyAgICAgZm9yIChtYXAgPSB2aG9zdF9p b3RsYl9pdHJlZV9maXJzdChkb21haW4tPmlvdGxiLCBzdGFydCwgbGFzdCk7IG1hcDsKPj4+ICsg ICAgICAgICAgbWFwID0gdmhvc3RfaW90bGJfaXRyZWVfbmV4dChtYXAsIHN0YXJ0LCBsYXN0KSkg ewo+Pj4gKyAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc3JjX29mZnNldCA9IDAsIGRzdF9vZmZz ZXQgPSAwOwo+Pj4gKyAgICAgICAgICAgICBwaHlzX2FkZHJfdCBzcmM7Cj4+PiArICAgICAgICAg ICAgIHZvaWQgKmRzdDsKPj4+ICsgICAgICAgICAgICAgc2l6ZV90IHN6Owo+Pj4gKwo+Pj4gKyAg ICAgICAgICAgICBpZiAocGVybV90b19kaXIobWFwLT5wZXJtKSA9PSBETUFfRlJPTV9ERVZJQ0Up Cj4+PiArICAgICAgICAgICAgICAgICAgICAgY29udGludWU7Cj4+PiArCj4+PiArICAgICAgICAg ICAgIGlmIChzdGFydCA+IG1hcC0+c3RhcnQpCj4+PiArICAgICAgICAgICAgICAgICAgICAgc3Jj X29mZnNldCA9IHN0YXJ0IC0gbWFwLT5zdGFydDsKPj4+ICsgICAgICAgICAgICAgZWxzZQo+Pj4g KyAgICAgICAgICAgICAgICAgICAgIGRzdF9vZmZzZXQgPSBtYXAtPnN0YXJ0IC0gc3RhcnQ7Cj4+ PiArCj4+PiArICAgICAgICAgICAgIHNyYyA9IG1hcC0+YWRkciArIHNyY19vZmZzZXQ7Cj4+PiAr ICAgICAgICAgICAgIGRzdCA9IHBhZ2VfYWRkcmVzcyhwYWdlKSArIGRzdF9vZmZzZXQ7Cj4+PiAr ICAgICAgICAgICAgIHN6ID0gbWluX3Qoc2l6ZV90LCBtYXAtPnNpemUgLSBzcmNfb2Zmc2V0LAo+ Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgUEFHRV9TSVpFIC0gZHN0X29mZnNldCk7 Cj4+PiArICAgICAgICAgICAgIGRvX2JvdW5jZShzcmMsIGRzdCwgc3osIERNQV9UT19ERVZJQ0Up Owo+Pj4gKyAgICAgfQo+Pj4gK291dDoKPj4+ICsgICAgIHNwaW5fdW5sb2NrKCZkb21haW4tPmlv dGxiX2xvY2spOwo+Pj4gKwo+Pj4gKyAgICAgcmV0dXJuIHBhZ2U7Cj4+PiArfQo+Pj4gKwo+Pj4g K3N0YXRpYyB2b2lkCj4+PiArdmR1c2VfZG9tYWluX2ZyZWVfYm91bmNlX3BhZ2VzKHN0cnVjdCB2 ZHVzZV9pb3ZhX2RvbWFpbiAqZG9tYWluLAo+Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgdTY0IGlvdmEsIHNpemVfdCBzaXplKQo+Pj4gK3sKPj4+ICsgICAgIHN0cnVjdCBwYWdlICpw YWdlOwo+Pj4gKwo+Pj4gKyAgICAgc3Bpbl9sb2NrKCZkb21haW4tPmlvdGxiX2xvY2spOwo+Pj4g KyAgICAgaWYgKFdBUk5fT04odmhvc3RfaW90bGJfaXRyZWVfZmlyc3QoZG9tYWluLT5pb3RsYiwg aW92YSwKPj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBp b3ZhICsgc2l6ZSAtIDEpKSkKPj4+ICsgICAgICAgICAgICAgZ290byBvdXQ7Cj4+PiArCj4+PiAr ICAgICB3aGlsZSAoc2l6ZSA+IDApIHsKPj4+ICsgICAgICAgICAgICAgcGFnZSA9IHZkdXNlX2Rv bWFpbl9nZXRfYm91bmNlX3BhZ2UoZG9tYWluLCBpb3ZhKTsKPj4+ICsgICAgICAgICAgICAgaWYg KHBhZ2UpIHsKPj4+ICsgICAgICAgICAgICAgICAgICAgICB2ZHVzZV9kb21haW5fc2V0X2JvdW5j ZV9wYWdlKGRvbWFpbiwgaW92YSwgTlVMTCk7Cj4+PiArICAgICAgICAgICAgICAgICAgICAgX19m cmVlX3BhZ2UocGFnZSk7Cj4+PiArICAgICAgICAgICAgIH0KPj4+ICsgICAgICAgICAgICAgc2l6 ZSAtPSBQQUdFX1NJWkU7Cj4+PiArICAgICAgICAgICAgIGlvdmEgKz0gUEFHRV9TSVpFOwo+Pj4g KyAgICAgfQo+Pj4gK291dDoKPj4+ICsgICAgIHNwaW5fdW5sb2NrKCZkb21haW4tPmlvdGxiX2xv Y2spOwo+Pj4gK30KPj4+ICsKPj4+ICtzdGF0aWMgdm9pZCB2ZHVzZV9kb21haW5fYm91bmNlKHN0 cnVjdCB2ZHVzZV9pb3ZhX2RvbWFpbiAqZG9tYWluLAo+Pj4gKyAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgZG1hX2FkZHJfdCBpb3ZhLCBwaHlzX2FkZHJfdCBvcmlnLAo+Pj4gKyAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgc2l6ZV90IHNpemUsIGVudW0gZG1hX2RhdGFfZGlyZWN0aW9u IGRpcikKPj4+ICt7Cj4+PiArICAgICB1bnNpZ25lZCBpbnQgb2Zmc2V0ID0gb2Zmc2V0X2luX3Bh Z2UoaW92YSk7Cj4+PiArCj4+PiArICAgICB3aGlsZSAoc2l6ZSkgewo+Pj4gKyAgICAgICAgICAg ICBzdHJ1Y3QgcGFnZSAqcCA9IHZkdXNlX2RvbWFpbl9nZXRfYm91bmNlX3BhZ2UoZG9tYWluLCBp b3ZhKTsKPj4+ICsgICAgICAgICAgICAgc2l6ZV90IHN6ID0gbWluX3Qoc2l6ZV90LCBQQUdFX1NJ WkUgLSBvZmZzZXQsIHNpemUpOwo+Pj4gKwo+Pj4gKyAgICAgICAgICAgICBXQVJOX09OKCFwICYm IGRpciA9PSBETUFfRlJPTV9ERVZJQ0UpOwo+Pj4gKwo+Pj4gKyAgICAgICAgICAgICBpZiAocCkK Pj4+ICsgICAgICAgICAgICAgICAgICAgICBkb19ib3VuY2Uob3JpZywgcGFnZV9hZGRyZXNzKHAp ICsgb2Zmc2V0LCBzeiwgZGlyKTsKPj4+ICsKPj4+ICsgICAgICAgICAgICAgc2l6ZSAtPSBzejsK Pj4+ICsgICAgICAgICAgICAgb3JpZyArPSBzejsKPj4+ICsgICAgICAgICAgICAgaW92YSArPSBz ejsKPj4+ICsgICAgICAgICAgICAgb2Zmc2V0ID0gMDsKPj4+ICsgICAgIH0KPj4+ICt9Cj4+PiAr Cj4+PiArc3RhdGljIGRtYV9hZGRyX3QgdmR1c2VfZG9tYWluX2FsbG9jX2lvdmEoc3RydWN0IGlv dmFfZG9tYWluICppb3ZhZCwKPj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2ln bmVkIGxvbmcgc2l6ZSwgdW5zaWduZWQgbG9uZyBsaW1pdCkKPj4+ICt7Cj4+PiArICAgICB1bnNp Z25lZCBsb25nIHNoaWZ0ID0gaW92YV9zaGlmdChpb3ZhZCk7Cj4+PiArICAgICB1bnNpZ25lZCBs b25nIGlvdmFfbGVuID0gaW92YV9hbGlnbihpb3ZhZCwgc2l6ZSkgPj4gc2hpZnQ7Cj4+PiArICAg ICB1bnNpZ25lZCBsb25nIGlvdmFfcGZuOwo+Pj4gKwo+Pj4gKyAgICAgaWYgKGlvdmFfbGVuIDwg KDEgPDwgKElPVkFfUkFOR0VfQ0FDSEVfTUFYX1NJWkUgLSAxKSkpCj4+PiArICAgICAgICAgICAg IGlvdmFfbGVuID0gcm91bmR1cF9wb3dfb2ZfdHdvKGlvdmFfbGVuKTsKPj4+ICsgICAgIGlvdmFf cGZuID0gYWxsb2NfaW92YV9mYXN0KGlvdmFkLCBpb3ZhX2xlbiwgbGltaXQgPj4gc2hpZnQsIHRy dWUpOwo+Pj4gKwo+Pj4gKyAgICAgcmV0dXJuIGlvdmFfcGZuIDw8IHNoaWZ0Owo+Pj4gK30KPj4+ ICsKPj4+ICtzdGF0aWMgdm9pZCB2ZHVzZV9kb21haW5fZnJlZV9pb3ZhKHN0cnVjdCBpb3ZhX2Rv bWFpbiAqaW92YWQsCj4+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkbWFfYWRkcl90 IGlvdmEsIHNpemVfdCBzaXplKQo+Pj4gK3sKPj4+ICsgICAgIHVuc2lnbmVkIGxvbmcgc2hpZnQg PSBpb3ZhX3NoaWZ0KGlvdmFkKTsKPj4+ICsgICAgIHVuc2lnbmVkIGxvbmcgaW92YV9sZW4gPSBp b3ZhX2FsaWduKGlvdmFkLCBzaXplKSA+PiBzaGlmdDsKPj4+ICsKPj4+ICsgICAgIGZyZWVfaW92 YV9mYXN0KGlvdmFkLCBpb3ZhID4+IHNoaWZ0LCBpb3ZhX2xlbik7Cj4+PiArfQo+Pj4gKwo+Pj4g K2RtYV9hZGRyX3QgdmR1c2VfZG9tYWluX21hcF9wYWdlKHN0cnVjdCB2ZHVzZV9pb3ZhX2RvbWFp biAqZG9tYWluLAo+Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2Ug KnBhZ2UsIHVuc2lnbmVkIGxvbmcgb2Zmc2V0LAo+Pj4gKyAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgc2l6ZV90IHNpemUsIGVudW0gZG1hX2RhdGFfZGlyZWN0aW9uIGRpciwKPj4+ICsgICAg ICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGxvbmcgYXR0cnMpCj4+PiArewo+Pj4g KyAgICAgc3RydWN0IGlvdmFfZG9tYWluICppb3ZhZCA9ICZkb21haW4tPnN0cmVhbV9pb3ZhZDsK Pj4+ICsgICAgIHVuc2lnbmVkIGxvbmcgbGltaXQgPSBkb21haW4tPmJvdW5jZV9zaXplIC0gMTsK Pj4+ICsgICAgIHBoeXNfYWRkcl90IHBhID0gcGFnZV90b19waHlzKHBhZ2UpICsgb2Zmc2V0Owo+ Pj4gKyAgICAgZG1hX2FkZHJfdCBpb3ZhID0gdmR1c2VfZG9tYWluX2FsbG9jX2lvdmEoaW92YWQs IHNpemUsIGxpbWl0KTsKPj4+ICsgICAgIGludCByZXQ7Cj4+PiArCj4+PiArICAgICBpZiAoIWlv dmEpCj4+PiArICAgICAgICAgICAgIHJldHVybiBETUFfTUFQUElOR19FUlJPUjsKPj4+ICsKPj4+ ICsgICAgIHNwaW5fbG9jaygmZG9tYWluLT5pb3RsYl9sb2NrKTsKPj4+ICsgICAgIHJldCA9IHZo b3N0X2lvdGxiX2FkZF9yYW5nZShkb21haW4tPmlvdGxiLCAodTY0KWlvdmEsCj4+PiArICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHU2NClpb3ZhICsgc2l6ZSAtIDEsCj4+PiArICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcGEsIGRpcl90b19wZXJtKGRpcikpOwo+Pj4g KyAgICAgc3Bpbl91bmxvY2soJmRvbWFpbi0+aW90bGJfbG9jayk7Cj4+PiArICAgICBpZiAocmV0 KSB7Cj4+PiArICAgICAgICAgICAgIHZkdXNlX2RvbWFpbl9mcmVlX2lvdmEoaW92YWQsIGlvdmEs IHNpemUpOwo+Pj4gKyAgICAgICAgICAgICByZXR1cm4gRE1BX01BUFBJTkdfRVJST1I7Cj4+PiAr ICAgICB9Cj4+PiArICAgICBpZiAoZGlyID09IERNQV9UT19ERVZJQ0UgfHwgZGlyID09IERNQV9C SURJUkVDVElPTkFMKQo+Pj4gKyAgICAgICAgICAgICB2ZHVzZV9kb21haW5fYm91bmNlKGRvbWFp biwgaW92YSwgcGEsIHNpemUsIERNQV9UT19ERVZJQ0UpOwo+Pj4gKwo+Pj4gKyAgICAgcmV0dXJu IGlvdmE7Cj4+PiArfQo+Pj4gKwo+Pj4gK3ZvaWQgdmR1c2VfZG9tYWluX3VubWFwX3BhZ2Uoc3Ry dWN0IHZkdXNlX2lvdmFfZG9tYWluICpkb21haW4sCj4+PiArICAgICAgICAgICAgICAgICAgICAg ZG1hX2FkZHJfdCBkbWFfYWRkciwgc2l6ZV90IHNpemUsCj4+PiArICAgICAgICAgICAgICAgICAg ICAgZW51bSBkbWFfZGF0YV9kaXJlY3Rpb24gZGlyLCB1bnNpZ25lZCBsb25nIGF0dHJzKQo+Pj4g K3sKPj4+ICsgICAgIHN0cnVjdCBpb3ZhX2RvbWFpbiAqaW92YWQgPSAmZG9tYWluLT5zdHJlYW1f aW92YWQ7Cj4+PiArICAgICBzdHJ1Y3Qgdmhvc3RfaW90bGJfbWFwICptYXA7Cj4+PiArICAgICBw aHlzX2FkZHJfdCBwYTsKPj4+ICsKPj4+ICsgICAgIHNwaW5fbG9jaygmZG9tYWluLT5pb3RsYl9s b2NrKTsKPj4+ICsgICAgIG1hcCA9IHZob3N0X2lvdGxiX2l0cmVlX2ZpcnN0KGRvbWFpbi0+aW90 bGIsICh1NjQpZG1hX2FkZHIsCj4+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAodTY0KWRtYV9hZGRyICsgc2l6ZSAtIDEpOwo+Pj4gKyAgICAgaWYgKFdBUk5fT04oIW1hcCkp IHsKPj4+ICsgICAgICAgICAgICAgc3Bpbl91bmxvY2soJmRvbWFpbi0+aW90bGJfbG9jayk7Cj4+ PiArICAgICAgICAgICAgIHJldHVybjsKPj4+ICsgICAgIH0KPj4+ICsgICAgIHBhID0gbWFwLT5h ZGRyOwo+Pj4gKyAgICAgdmhvc3RfaW90bGJfbWFwX2ZyZWUoZG9tYWluLT5pb3RsYiwgbWFwKTsK Pj4+ICsgICAgIHNwaW5fdW5sb2NrKCZkb21haW4tPmlvdGxiX2xvY2spOwo+Pj4gKwo+Pj4gKyAg ICAgaWYgKGRpciA9PSBETUFfRlJPTV9ERVZJQ0UgfHwgZGlyID09IERNQV9CSURJUkVDVElPTkFM KQo+Pj4gKyAgICAgICAgICAgICB2ZHVzZV9kb21haW5fYm91bmNlKGRvbWFpbiwgZG1hX2FkZHIs IHBhLAo+Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzaXplLCBETUFf RlJPTV9ERVZJQ0UpOwo+Pj4gKwo+Pj4gKyAgICAgdmR1c2VfZG9tYWluX2ZyZWVfaW92YShpb3Zh ZCwgZG1hX2FkZHIsIHNpemUpOwo+Pj4gK30KPj4+ICsKPj4+ICt2b2lkICp2ZHVzZV9kb21haW5f YWxsb2NfY29oZXJlbnQoc3RydWN0IHZkdXNlX2lvdmFfZG9tYWluICpkb21haW4sCj4+PiArICAg ICAgICAgICAgICAgICAgICAgICAgICAgICBzaXplX3Qgc2l6ZSwgZG1hX2FkZHJfdCAqZG1hX2Fk ZHIsCj4+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICBnZnBfdCBmbGFnLCB1bnNpZ25l ZCBsb25nIGF0dHJzKQo+Pj4gK3sKPj4+ICsgICAgIHN0cnVjdCBpb3ZhX2RvbWFpbiAqaW92YWQg PSAmZG9tYWluLT5jb25zaXN0ZW50X2lvdmFkOwo+Pj4gKyAgICAgdW5zaWduZWQgbG9uZyBsaW1p dCA9IGRvbWFpbi0+aW92YV9saW1pdDsKPj4+ICsgICAgIGRtYV9hZGRyX3QgaW92YSA9IHZkdXNl X2RvbWFpbl9hbGxvY19pb3ZhKGlvdmFkLCBzaXplLCBsaW1pdCk7Cj4+PiArICAgICB2b2lkICpv cmlnID0gYWxsb2NfcGFnZXNfZXhhY3Qoc2l6ZSwgZmxhZyk7Cj4+PiArICAgICBpbnQgcmV0Owo+ Pj4gKwo+Pj4gKyAgICAgaWYgKCFpb3ZhIHx8ICFvcmlnKQo+Pj4gKyAgICAgICAgICAgICBnb3Rv IGVycjsKPj4+ICsKPj4+ICsgICAgIHNwaW5fbG9jaygmZG9tYWluLT5pb3RsYl9sb2NrKTsKPj4+ ICsgICAgIHJldCA9IHZob3N0X2lvdGxiX2FkZF9yYW5nZShkb21haW4tPmlvdGxiLCAodTY0KWlv dmEsCj4+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKHU2NClpb3ZhICsgc2l6 ZSAtIDEsCj4+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdmlydF90b19waHlz KG9yaWcpLCBWSE9TVF9NQVBfUlcpOwo+Pj4gKyAgICAgc3Bpbl91bmxvY2soJmRvbWFpbi0+aW90 bGJfbG9jayk7Cj4+PiArICAgICBpZiAocmV0KQo+Pj4gKyAgICAgICAgICAgICBnb3RvIGVycjsK Pj4+ICsKPj4+ICsgICAgICpkbWFfYWRkciA9IGlvdmE7Cj4+PiArCj4+PiArICAgICByZXR1cm4g b3JpZzsKPj4+ICtlcnI6Cj4+PiArICAgICAqZG1hX2FkZHIgPSBETUFfTUFQUElOR19FUlJPUjsK Pj4+ICsgICAgIGlmIChvcmlnKQo+Pj4gKyAgICAgICAgICAgICBmcmVlX3BhZ2VzX2V4YWN0KG9y aWcsIHNpemUpOwo+Pj4gKyAgICAgaWYgKGlvdmEpCj4+PiArICAgICAgICAgICAgIHZkdXNlX2Rv bWFpbl9mcmVlX2lvdmEoaW92YWQsIGlvdmEsIHNpemUpOwo+Pj4gKwo+Pj4gKyAgICAgcmV0dXJu IE5VTEw7Cj4+PiArfQo+Pj4gKwo+Pj4gK3ZvaWQgdmR1c2VfZG9tYWluX2ZyZWVfY29oZXJlbnQo c3RydWN0IHZkdXNlX2lvdmFfZG9tYWluICpkb21haW4sIHNpemVfdCBzaXplLAo+Pj4gKyAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgdm9pZCAqdmFkZHIsIGRtYV9hZGRyX3QgZG1hX2FkZHIs Cj4+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBsb25nIGF0dHJzKQo+ Pj4gK3sKPj4+ICsgICAgIHN0cnVjdCBpb3ZhX2RvbWFpbiAqaW92YWQgPSAmZG9tYWluLT5jb25z aXN0ZW50X2lvdmFkOwo+Pj4gKyAgICAgc3RydWN0IHZob3N0X2lvdGxiX21hcCAqbWFwOwo+Pj4g KyAgICAgcGh5c19hZGRyX3QgcGE7Cj4+PiArCj4+PiArICAgICBzcGluX2xvY2soJmRvbWFpbi0+ aW90bGJfbG9jayk7Cj4+PiArICAgICBtYXAgPSB2aG9zdF9pb3RsYl9pdHJlZV9maXJzdChkb21h aW4tPmlvdGxiLCAodTY0KWRtYV9hZGRyLAo+Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgKHU2NClkbWFfYWRkciArIHNpemUgLSAxKTsKPj4+ICsgICAgIGlmIChXQVJOX09O KCFtYXApKSB7Cj4+PiArICAgICAgICAgICAgIHNwaW5fdW5sb2NrKCZkb21haW4tPmlvdGxiX2xv Y2spOwo+Pj4gKyAgICAgICAgICAgICByZXR1cm47Cj4+PiArICAgICB9Cj4+PiArICAgICBwYSA9 IG1hcC0+YWRkcjsKPj4+ICsgICAgIHZob3N0X2lvdGxiX21hcF9mcmVlKGRvbWFpbi0+aW90bGIs IG1hcCk7Cj4+PiArICAgICBzcGluX3VubG9jaygmZG9tYWluLT5pb3RsYl9sb2NrKTsKPj4+ICsK Pj4+ICsgICAgIHZkdXNlX2RvbWFpbl9mcmVlX2lvdmEoaW92YWQsIGRtYV9hZGRyLCBzaXplKTsK Pj4+ICsgICAgIGZyZWVfcGFnZXNfZXhhY3QocGh5c190b192aXJ0KHBhKSwgc2l6ZSk7Cj4+PiAr fQo+Pj4gKwo+Pj4gK3N0YXRpYyB2bV9mYXVsdF90IHZkdXNlX2RvbWFpbl9tbWFwX2ZhdWx0KHN0 cnVjdCB2bV9mYXVsdCAqdm1mKQo+Pj4gK3sKPj4+ICsgICAgIHN0cnVjdCB2ZHVzZV9pb3ZhX2Rv bWFpbiAqZG9tYWluID0gdm1mLT52bWEtPnZtX3ByaXZhdGVfZGF0YTsKPj4+ICsgICAgIHVuc2ln bmVkIGxvbmcgaW92YSA9IHZtZi0+cGdvZmYgPDwgUEFHRV9TSElGVDsKPj4+ICsgICAgIHN0cnVj dCBwYWdlICpwYWdlOwo+Pj4gKwo+Pj4gKyAgICAgaWYgKCFkb21haW4pCj4+PiArICAgICAgICAg ICAgIHJldHVybiBWTV9GQVVMVF9TSUdCVVM7Cj4+PiArCj4+PiArICAgICBpZiAoaW92YSA8IGRv bWFpbi0+Ym91bmNlX3NpemUpCj4+PiArICAgICAgICAgICAgIHBhZ2UgPSB2ZHVzZV9kb21haW5f YWxsb2NfYm91bmNlX3BhZ2UoZG9tYWluLCBpb3ZhKTsKPj4+ICsgICAgIGVsc2UKPj4+ICsgICAg ICAgICAgICAgcGFnZSA9IHZkdXNlX2RvbWFpbl9nZXRfbWFwcGluZ19wYWdlKGRvbWFpbiwgaW92 YSk7Cj4+PiArCj4+PiArICAgICBpZiAoIXBhZ2UpCj4+PiArICAgICAgICAgICAgIHJldHVybiBW TV9GQVVMVF9TSUdCVVM7Cj4+PiArCj4+PiArICAgICB2bWYtPnBhZ2UgPSBwYWdlOwo+Pj4gKwo+ Pj4gKyAgICAgcmV0dXJuIDA7Cj4+PiArfQo+Pj4gKwo+Pj4gK3N0YXRpYyBjb25zdCBzdHJ1Y3Qg dm1fb3BlcmF0aW9uc19zdHJ1Y3QgdmR1c2VfZG9tYWluX21tYXBfb3BzID0gewo+Pj4gKyAgICAg LmZhdWx0ID0gdmR1c2VfZG9tYWluX21tYXBfZmF1bHQsCj4+PiArfTsKPj4+ICsKPj4+ICtzdGF0 aWMgaW50IHZkdXNlX2RvbWFpbl9tbWFwKHN0cnVjdCBmaWxlICpmaWxlLCBzdHJ1Y3Qgdm1fYXJl YV9zdHJ1Y3QgKnZtYSkKPj4+ICt7Cj4+PiArICAgICBzdHJ1Y3QgdmR1c2VfaW92YV9kb21haW4g KmRvbWFpbiA9IGZpbGUtPnByaXZhdGVfZGF0YTsKPj4+ICsKPj4+ICsgICAgIHZtYS0+dm1fZmxh Z3MgfD0gVk1fRE9OVERVTVAgfCBWTV9ET05URVhQQU5EOwo+Pj4gKyAgICAgdm1hLT52bV9wcml2 YXRlX2RhdGEgPSBkb21haW47Cj4+PiArICAgICB2bWEtPnZtX29wcyA9ICZ2ZHVzZV9kb21haW5f bW1hcF9vcHM7Cj4+PiArCj4+PiArICAgICByZXR1cm4gMDsKPj4+ICt9Cj4+PiArCj4+PiArc3Rh dGljIGludCB2ZHVzZV9kb21haW5fcmVsZWFzZShzdHJ1Y3QgaW5vZGUgKmlub2RlLCBzdHJ1Y3Qg ZmlsZSAqZmlsZSkKPj4+ICt7Cj4+PiArICAgICBzdHJ1Y3QgdmR1c2VfaW92YV9kb21haW4gKmRv bWFpbiA9IGZpbGUtPnByaXZhdGVfZGF0YTsKPj4+ICsKPj4+ICsgICAgIHZkdXNlX2RvbWFpbl9m cmVlX2JvdW5jZV9wYWdlcyhkb21haW4sIDAsIGRvbWFpbi0+Ym91bmNlX3NpemUpOwo+Pj4gKyAg ICAgcHV0X2lvdmFfZG9tYWluKCZkb21haW4tPnN0cmVhbV9pb3ZhZCk7Cj4+PiArICAgICBwdXRf aW92YV9kb21haW4oJmRvbWFpbi0+Y29uc2lzdGVudF9pb3ZhZCk7Cj4+PiArICAgICB2aG9zdF9p b3RsYl9mcmVlKGRvbWFpbi0+aW90bGIpOwo+Pj4gKyAgICAgdmZyZWUoZG9tYWluLT5ib3VuY2Vf cGFnZXMpOwo+Pj4gKyAgICAga2ZyZWUoZG9tYWluKTsKPj4+ICsKPj4+ICsgICAgIHJldHVybiAw Owo+Pj4gK30KPj4+ICsKPj4+ICtzdGF0aWMgY29uc3Qgc3RydWN0IGZpbGVfb3BlcmF0aW9ucyB2 ZHVzZV9kb21haW5fZm9wcyA9IHsKPj4+ICsgICAgIC5tbWFwID0gdmR1c2VfZG9tYWluX21tYXAs Cj4+PiArICAgICAucmVsZWFzZSA9IHZkdXNlX2RvbWFpbl9yZWxlYXNlLAo+Pj4gK307Cj4+PiAr Cj4+PiArdm9pZCB2ZHVzZV9kb21haW5fZGVzdHJveShzdHJ1Y3QgdmR1c2VfaW92YV9kb21haW4g KmRvbWFpbikKPj4+ICt7Cj4+PiArICAgICBmcHV0KGRvbWFpbi0+ZmlsZSk7Cj4+PiArfQo+Pj4g Kwo+Pj4gK3N0cnVjdCB2ZHVzZV9pb3ZhX2RvbWFpbiAqCj4+PiArdmR1c2VfZG9tYWluX2NyZWF0 ZSh1bnNpZ25lZCBsb25nIGlvdmFfbGltaXQsIHNpemVfdCBib3VuY2Vfc2l6ZSkKPj4+ICt7Cj4+ PiArICAgICBzdHJ1Y3QgdmR1c2VfaW92YV9kb21haW4gKmRvbWFpbjsKPj4+ICsgICAgIHN0cnVj dCBmaWxlICpmaWxlOwo+Pj4gKyAgICAgdW5zaWduZWQgbG9uZyBib3VuY2VfcGZucyA9IFBBR0Vf QUxJR04oYm91bmNlX3NpemUpID4+IFBBR0VfU0hJRlQ7Cj4+PiArCj4+PiArICAgICBpZiAoaW92 YV9saW1pdCA8PSBib3VuY2Vfc2l6ZSkKPj4+ICsgICAgICAgICAgICAgcmV0dXJuIE5VTEw7Cj4+ PiArCj4+PiArICAgICBkb21haW4gPSBremFsbG9jKHNpemVvZigqZG9tYWluKSwgR0ZQX0tFUk5F TCk7Cj4+PiArICAgICBpZiAoIWRvbWFpbikKPj4+ICsgICAgICAgICAgICAgcmV0dXJuIE5VTEw7 Cj4+PiArCj4+PiArICAgICBkb21haW4tPmlvdGxiID0gdmhvc3RfaW90bGJfYWxsb2MoMCwgMCk7 Cj4+PiArICAgICBpZiAoIWRvbWFpbi0+aW90bGIpCj4+PiArICAgICAgICAgICAgIGdvdG8gZXJy X2lvdGxiOwo+Pj4gKwo+Pj4gKyAgICAgZG9tYWluLT5pb3ZhX2xpbWl0ID0gaW92YV9saW1pdDsK Pj4+ICsgICAgIGRvbWFpbi0+Ym91bmNlX3NpemUgPSBQQUdFX0FMSUdOKGJvdW5jZV9zaXplKTsK Pj4+ICsgICAgIGRvbWFpbi0+Ym91bmNlX3BhZ2VzID0gdnphbGxvYyhib3VuY2VfcGZucyAqIHNp emVvZihzdHJ1Y3QgcGFnZSAqKSk7Cj4+PiArICAgICBpZiAoIWRvbWFpbi0+Ym91bmNlX3BhZ2Vz KQo+Pj4gKyAgICAgICAgICAgICBnb3RvIGVycl9wYWdlOwo+Pj4gKwo+Pj4gKyAgICAgZmlsZSA9 IGFub25faW5vZGVfZ2V0ZmlsZSgiW3ZkdXNlLWRvbWFpbl0iLCAmdmR1c2VfZG9tYWluX2ZvcHMs Cj4+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkb21haW4sIE9fUkRXUik7Cj4+PiAr ICAgICBpZiAoSVNfRVJSKGZpbGUpKQo+Pj4gKyAgICAgICAgICAgICBnb3RvIGVycl9maWxlOwo+ Pj4gKwo+Pj4gKyAgICAgZG9tYWluLT5maWxlID0gZmlsZTsKPj4+ICsgICAgIHNwaW5fbG9ja19p bml0KCZkb21haW4tPmlvdGxiX2xvY2spOwo+Pj4gKyAgICAgaW5pdF9pb3ZhX2RvbWFpbigmZG9t YWluLT5zdHJlYW1faW92YWQsCj4+PiArICAgICAgICAgICAgICAgICAgICAgSU9WQV9BTExPQ19T SVpFLCBJT1ZBX1NUQVJUX1BGTik7Cj4+PiArICAgICBpbml0X2lvdmFfZG9tYWluKCZkb21haW4t PmNvbnNpc3RlbnRfaW92YWQsCj4+PiArICAgICAgICAgICAgICAgICAgICAgUEFHRV9TSVpFLCBi b3VuY2VfcGZucyk7Cj4+PiArCj4+PiArICAgICByZXR1cm4gZG9tYWluOwo+Pj4gK2Vycl9maWxl Ogo+Pj4gKyAgICAgdmZyZWUoZG9tYWluLT5ib3VuY2VfcGFnZXMpOwo+Pj4gK2Vycl9wYWdlOgo+ Pj4gKyAgICAgdmhvc3RfaW90bGJfZnJlZShkb21haW4tPmlvdGxiKTsKPj4+ICtlcnJfaW90bGI6 Cj4+PiArICAgICBrZnJlZShkb21haW4pOwo+Pj4gKyAgICAgcmV0dXJuIE5VTEw7Cj4+PiArfQo+ Pj4gKwo+Pj4gK2ludCB2ZHVzZV9kb21haW5faW5pdCh2b2lkKQo+Pj4gK3sKPj4+ICsgICAgIHJl dHVybiBpb3ZhX2NhY2hlX2dldCgpOwo+Pj4gK30KPj4+ICsKPj4+ICt2b2lkIHZkdXNlX2RvbWFp bl9leGl0KHZvaWQpCj4+PiArewo+Pj4gKyAgICAgaW92YV9jYWNoZV9wdXQoKTsKPj4+ICt9Cj4+ PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy92ZHBhL3ZkcGFfdXNlci9pb3ZhX2RvbWFpbi5oIGIvZHJp dmVycy92ZHBhL3ZkcGFfdXNlci9pb3ZhX2RvbWFpbi5oCj4+PiBuZXcgZmlsZSBtb2RlIDEwMDY0 NAo+Pj4gaW5kZXggMDAwMDAwMDAwMDAwLi45Yzg1ZDgzNDY2MjYKPj4+IC0tLSAvZGV2L251bGwK Pj4+ICsrKyBiL2RyaXZlcnMvdmRwYS92ZHBhX3VzZXIvaW92YV9kb21haW4uaAo+Pj4gQEAgLTAs MCArMSw2MSBAQAo+Pj4gKy8qIFNQRFgtTGljZW5zZS1JZGVudGlmaWVyOiBHUEwtMi4wLW9ubHkg Ki8KPj4+ICsvKgo+Pj4gKyAqIE1NVS1iYXNlZCBJT01NVSBpbXBsZW1lbnRhdGlvbgo+Pj4gKyAq Cj4+PiArICogQ29weXJpZ2h0IChDKSAyMDIwIEJ5dGVkYW5jZSBJbmMuIGFuZC9vciBpdHMgYWZm aWxpYXRlcy4gQWxsIHJpZ2h0cyByZXNlcnZlZC4KPj4+ICsgKgo+Pj4gKyAqIEF1dGhvcjogWGll IFlvbmdqaSA8eGlleW9uZ2ppQGJ5dGVkYW5jZS5jb20+Cj4+PiArICoKPj4+ICsgKi8KPj4+ICsK Pj4+ICsjaWZuZGVmIF9WRFVTRV9JT1ZBX0RPTUFJTl9ICj4+PiArI2RlZmluZSBfVkRVU0VfSU9W QV9ET01BSU5fSAo+Pj4gKwo+Pj4gKyNpbmNsdWRlIDxsaW51eC9pb3ZhLmg+Cj4+PiArI2luY2x1 ZGUgPGxpbnV4L2RtYS1tYXBwaW5nLmg+Cj4+PiArI2luY2x1ZGUgPGxpbnV4L3Zob3N0X2lvdGxi Lmg+Cj4+PiArCj4+PiArc3RydWN0IHZkdXNlX2lvdmFfZG9tYWluIHsKPj4+ICsgICAgIHN0cnVj dCBpb3ZhX2RvbWFpbiBzdHJlYW1faW92YWQ7Cj4+PiArICAgICBzdHJ1Y3QgaW92YV9kb21haW4g Y29uc2lzdGVudF9pb3ZhZDsKPj4+ICsgICAgIHN0cnVjdCBwYWdlICoqYm91bmNlX3BhZ2VzOwo+ Pj4gKyAgICAgc2l6ZV90IGJvdW5jZV9zaXplOwo+Pj4gKyAgICAgdW5zaWduZWQgbG9uZyBpb3Zh X2xpbWl0Owo+Pj4gKyAgICAgc3RydWN0IHZob3N0X2lvdGxiICppb3RsYjsKPj4KPj4gU29ycnkg aWYgSSd2ZSBhc2tlZCB0aGlzIGJlZm9yZS4KPj4KPj4gQnV0IHdoYXQncyB0aGUgcmVhc29uIGZv ciBtYWludGFpbmcgYSBkZWRpY2F0ZWQgSU9UTEIgaGVyZT8gSSB0aGluayB3ZQo+PiBjb3VsZCBy ZXVzZSB2ZHVzZV9kZXYtPmlvbW11IHNpbmNlIHRoZSBkZXZpY2UgY2FuIG5vdCBiZSB1c2VkIGJ5 IGJvdGgKPj4gdmlydGlvIGFuZCB2aG9zdCBpbiB0aGUgc2FtZSB0aW1lIG9yIHVzZSB2ZHVzZV9p b3ZhX2RvbWFpbi0+aW90bGIgZm9yCj4+IHNldF9tYXAoKS4KPj4KPiBUaGUgbWFpbiBkaWZmZXJl bmNlIGJldHdlZW4gZG9tYWluLT5pb3RsYiBhbmQgZGV2LT5pb3RsYiBpcyB0aGUgd2F5IHRvCj4g ZGVhbCB3aXRoIGJvdW5jZSBidWZmZXIuIEluIHRoZSBkb21haW4tPmlvdGxiIGNhc2UsIGJvdW5j ZSBidWZmZXIKPiBuZWVkcyB0byBiZSBtYXBwZWQgZWFjaCBETUEgdHJhbnNmZXIgYmVjYXVzZSB3 ZSBuZWVkIHRvIGdldCB0aGUgYm91bmNlCj4gcGFnZXMgYnkgYW4gSU9WQSBkdXJpbmcgRE1BIHVu bWFwcGluZy4gSW4gdGhlIGRldi0+aW90bGIgY2FzZSwgYm91bmNlCj4gYnVmZmVyIG9ubHkgbmVl ZHMgdG8gYmUgbWFwcGVkIG9uY2UgZHVyaW5nIGluaXRpYWxpemF0aW9uLCB3aGljaCB3aWxsCj4g YmUgdXNlZCB0byB0ZWxsIHVzZXJzcGFjZSBob3cgdG8gZG8gbW1hcCgpLgo+Cj4+IEFsc28sIHNp bmNlIHZob3N0IElPVExCIHN1cHBvcnQgcGVyIG1hcHBpbmcgdG9rZW4gKG9wYXVxZSksIGNhbiB3 ZSB1c2UKPj4gdGhhdCBpbnN0ZWFkIG9mIHRoZSBib3VuY2VfcGFnZXMgKj8KPj4KPiBTb3JyeSwg SSBkaWRuJ3QgZ2V0IHlvdSBoZXJlLiBXaGljaCB2YWx1ZSBkbyB5b3UgbWVhbiB0byBzdG9yZSBp biB0aGUKPiBvcGFxdWUgcG9pbnRlcu+8nwoKClNvIEkgd291bGQgbGlrZSB0byBoYXZlIGEgd2F5 IHRvIHVzZSBhIHNpbmdsZSBJT1RMQiBmb3IgbWFuYWdlIGFsbCBraW5kcyAKb2YgbWFwcGluZ3Mu IFR3byBwb3NzaWJsZSBpZGVhczoKCjEpIG1hcCBib3VuY2UgcGFnZSBvbmUgYnkgb25lIGluIHZk dXNlX2Rldl9tYXBfcGFnZSgpLCBpbiAKVkRVU0VfSU9UTEJfR0VUX0ZELCB0cnkgdG8gbWVyZ2Ug dGhlIHJlc3VsdCBpZiB3ZSBoYWQgdGhlIHNhbWUgZmQuIFRoZW4gCmZvciBib3VuY2UgcGFnZXMs IHVzZXJzcGFjZSBzdGlsbCBvbmx5IG5lZWQgdG8gbWFwIGl0IG9uY2UgYW5kIHdlIGNhbiAKbWFp bnRhaW4gdGhlIGFjdHVhbCBtYXBwaW5nIGJ5IHN0b3JpbmcgdGhlIHBhZ2Ugb3IgcGEgaW4gdGhl IG9wYXF1ZSAKZmllbGQgb2YgSU9UTEIgZW50cnkuCjIpIG1hcCBib3VuY2UgcGFnZSBvbmNlIGlu IHZkdXNlX2Rldl9tYXBfcGFnZSgpIGFuZCBzdG9yZSBzdHJ1Y3QgcGFnZSAKKipib3VuY2VfcGFn ZXMgaW4gdGhlIG9wYXF1ZSBmaWVsZCBvZiB0aGlzIHNpbmdsZSBJT1RMQiBlbnRyeS4KCkRvZXMg dGhpcyB3b3JrPwoKVGhhbmtzCgoKPgo+IFRoYW5rcywKPiBZb25namkKPgoKX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KVmlydHVhbGl6YXRpb24gbWFpbGlu ZyBsaXN0ClZpcnR1YWxpemF0aW9uQGxpc3RzLmxpbnV4LWZvdW5kYXRpb24ub3JnCmh0dHBzOi8v bGlzdHMubGludXhmb3VuZGF0aW9uLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3ZpcnR1YWxpemF0aW9u