From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFE6EC4707F for ; Tue, 25 May 2021 09:59:17 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 740A46142A for ; Tue, 25 May 2021 09:59:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 740A46142A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 79D536E9DC; Tue, 25 May 2021 09:59:15 +0000 (UTC) Received: from mail-qt1-x82c.google.com (mail-qt1-x82c.google.com [IPv6:2607:f8b0:4864:20::82c]) by gabe.freedesktop.org (Postfix) with ESMTPS id 421D86E1B8; Tue, 25 May 2021 09:59:14 +0000 (UTC) Received: by mail-qt1-x82c.google.com with SMTP id h21so22688496qtu.5; Tue, 25 May 2021 02:59:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=AOLh2SS/Gc8zZamjOdd0Bg0Az6U4VqWJmXFHFKKKNNc=; b=IZV/m+jELEFyFkM6MQ7qsXFQzjEP7sAEkOqTFePj8YdaSG2Gai0iHTPVo+BILd9OZz kInG7SISsml/L3uPxQEluf3CBDtHV0fBpAuoiVmX+Z/kiZjb7PlP+H/6EBJ5rdonbTvm xxClVL9vMDOmAcBuPfVE4fbjwsRjKl1Qt8vLTu6RI8vDbM8yDjM7dJ+isiVcF9LHyEVo ZDWl22Wz5WLU3SZRbEnMEZDJTNqT6lfcJ2tdTXUwIVZuwxbxFFeQFSjPwRwzj8/QwDTf eqIF93vHTE4zkRih6AFA1Q7iMADtBD2aE5H8SyKePl6LtJqSeLyepJW6cwq1R9mlHP+B oxPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=AOLh2SS/Gc8zZamjOdd0Bg0Az6U4VqWJmXFHFKKKNNc=; b=PQychsOQVzETVno85YApJMcEFivd1hgeQQ02OcJ0zFhayiYfIQNoJfi0arXiQO4L7u abrpGpxaYTjQk/ENtsodz51fFWnKZlvXUJh1+hIvB0fLTnxdqApHE7LrOlYSk4PVaEEf dQipktQMa8UhQJLjZK83mSVt+W5w/5f1lIIw04Q39UJy/16xQAD4WZElDYfrsINtrovA GVpDKCsQm2G/yw9gZxNPKSShvR4l423Zy4MaRgOwoFLpn1B0oG7KwCJGV+HSLHGdvd1t d0cX4ZFTv6d3TsthswjlUYKxzU2hHTW4gSb5qrSaPMDIdhXoKZpkBropn2+ukHgSlTpQ 8rWg== X-Gm-Message-State: AOAM530bBuOcUWeXwhqFirxLPa3V8N3pDbtTwXpqx4EjRSmvwWd70YlS rrnopRxvpJPItBFNQrGaEpHomrt/KzctCarEpEo= X-Google-Smtp-Source: ABdhPJxl2ZO83O91yr4U0yC86m9bQvTGmNz2toG/OJyl8j4WXunhrEIQlZtdEe2geDvRWqOCl8Ez2+AeL6FeVlzHO30= X-Received: by 2002:ac8:60d:: with SMTP id d13mr31849064qth.223.1621936753060; Tue, 25 May 2021 02:59:13 -0700 (PDT) MIME-Version: 1.0 References: <20210521153253.518037-1-thomas.hellstrom@linux.intel.com> <20210521153253.518037-7-thomas.hellstrom@linux.intel.com> <2cc9a60c-4360-40b6-8712-1e50b7bbfd03@linux.intel.com> In-Reply-To: <2cc9a60c-4360-40b6-8712-1e50b7bbfd03@linux.intel.com> From: Matthew Auld Date: Tue, 25 May 2021 10:58:46 +0100 Message-ID: Subject: Re: [Intel-gfx] [PATCH v3 06/12] drm/ttm: Add a generic TTM memcpy move for page-based iomem To: =?UTF-8?Q?Thomas_Hellstr=C3=B6m?= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Intel Graphics Development , =?UTF-8?Q?Christian_K=C3=B6nig?= , ML dri-devel Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Tue, 25 May 2021 at 10:32, Thomas Hellstr=C3=B6m wrote: > > > On 5/25/21 11:18 AM, Matthew Auld wrote: > > On Fri, 21 May 2021 at 16:33, Thomas Hellstr=C3=B6m > > wrote: > >> The internal ttm_bo_util memcpy uses ioremap functionality, and while = it > >> probably might be possible to use it for copying in- and out of > >> sglist represented io memory, using io_mem_reserve() / io_mem_free() > >> callbacks, that would cause problems with fault(). > >> Instead, implement a method mapping page-by-page using kmap_local() > >> semantics. As an additional benefit we then avoid the occasional globa= l > >> TLB flushes of ioremap() and consuming ioremap space, elimination of a > >> critical point of failure and with a slight change of semantics we cou= ld > >> also push the memcpy out async for testing and async driver developmen= t > >> purposes. > >> > >> A special linear iomem iterator is introduced internally to mimic the > >> old ioremap behaviour for code-paths that can't immediately be ported > >> over. This adds to the code size and should be considered a temporary > >> solution. > >> > >> Looking at the code we have a lot of checks for iomap tagged pointers. > >> Ideally we should extend the core memremap functions to also accept > >> uncached memory and kmap_local functionality. Then we could strip a > >> lot of code. > >> > >> Cc: Christian K=C3=B6nig > >> Signed-off-by: Thomas Hellstr=C3=B6m > >> --- > >> v3: > >> - Split up in various TTM files and addressed review comments by > >> Christian K=C3=B6nig. Tested and fixed legacy iomap memcpy path on = i915. > >> --- > >> drivers/gpu/drm/ttm/ttm_bo_util.c | 278 ++++++++++-----------------= -- > >> drivers/gpu/drm/ttm/ttm_module.c | 35 ++++ > >> drivers/gpu/drm/ttm/ttm_resource.c | 166 +++++++++++++++++ > >> drivers/gpu/drm/ttm/ttm_tt.c | 42 +++++ > >> include/drm/ttm/ttm_bo_driver.h | 28 +++ > >> include/drm/ttm/ttm_caching.h | 2 + > >> include/drm/ttm/ttm_kmap_iter.h | 61 +++++++ > >> include/drm/ttm/ttm_resource.h | 61 +++++++ > >> include/drm/ttm/ttm_tt.h | 16 ++ > >> 9 files changed, 508 insertions(+), 181 deletions(-) > >> create mode 100644 include/drm/ttm/ttm_kmap_iter.h > >> > >> diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c b/drivers/gpu/drm/ttm/t= tm_bo_util.c > >> index ae8b61460724..912cbe8e60a2 100644 > >> --- a/drivers/gpu/drm/ttm/ttm_bo_util.c > >> +++ b/drivers/gpu/drm/ttm/ttm_bo_util.c > >> @@ -72,190 +72,126 @@ void ttm_mem_io_free(struct ttm_device *bdev, > >> mem->bus.addr =3D NULL; > >> } > >> > >> -static int ttm_resource_ioremap(struct ttm_device *bdev, > >> - struct ttm_resource *mem, > >> - void **virtual) > >> +/** > >> + * ttm_move_memcpy - Helper to perform a memcpy ttm move operation. > >> + * @bo: The struct ttm_buffer_object. > >> + * @new_mem: The struct ttm_resource we're moving to (copy destinatio= n). > >> + * @new_iter: A struct ttm_kmap_iter representing the destination res= ource. > >> + * @src_iter: A struct ttm_kmap_iter representing the source resource= . > >> + * > >> + * This function is intended to be able to move out async under a > >> + * dma-fence if desired. > >> + */ > >> +void ttm_move_memcpy(struct ttm_buffer_object *bo, > >> + struct ttm_resource *dst_mem, > >> + struct ttm_kmap_iter *dst_iter, > >> + struct ttm_kmap_iter *src_iter) > >> { > >> - int ret; > >> - void *addr; > >> - > >> - *virtual =3D NULL; > >> - ret =3D ttm_mem_io_reserve(bdev, mem); > >> - if (ret || !mem->bus.is_iomem) > >> - return ret; > >> + const struct ttm_kmap_iter_ops *dst_ops =3D dst_iter->ops; > >> + const struct ttm_kmap_iter_ops *src_ops =3D src_iter->ops; > >> + struct ttm_tt *ttm =3D bo->ttm; > >> + struct dma_buf_map src_map, dst_map; > >> + pgoff_t i; > >> > >> - if (mem->bus.addr) { > >> - addr =3D mem->bus.addr; > >> - } else { > >> - size_t bus_size =3D (size_t)mem->num_pages << PAGE_SHI= FT; > >> + /* Single TTM move. NOP */ > >> + if (dst_ops->maps_tt && src_ops->maps_tt) > >> + return; > >> > >> - if (mem->bus.caching =3D=3D ttm_write_combined) > >> - addr =3D ioremap_wc(mem->bus.offset, bus_size)= ; > >> -#ifdef CONFIG_X86 > >> - else if (mem->bus.caching =3D=3D ttm_cached) > >> - addr =3D ioremap_cache(mem->bus.offset, bus_si= ze); > >> -#endif > >> - else > >> - addr =3D ioremap(mem->bus.offset, bus_size); > >> - if (!addr) { > >> - ttm_mem_io_free(bdev, mem); > >> - return -ENOMEM; > >> + /* Don't move nonexistent data. Clear destination instead. */ > >> + if (src_ops->maps_tt && (!ttm || !ttm_tt_is_populated(ttm))) { > >> + if (ttm && !(ttm->page_flags & TTM_PAGE_FLAG_ZERO_ALLO= C)) > >> + return; > >> + > >> + for (i =3D 0; i < dst_mem->num_pages; ++i) { > >> + dst_ops->map_local(dst_iter, &dst_map, i); > >> + if (dst_map.is_iomem) > >> + memset_io(dst_map.vaddr_iomem, 0, PAGE= _SIZE); > >> + else > >> + memset(dst_map.vaddr, 0, PAGE_SIZE); > >> + if (dst_ops->unmap_local) > >> + dst_ops->unmap_local(dst_iter, &dst_ma= p); > >> } > >> + return; > >> } > >> - *virtual =3D addr; > >> - return 0; > >> -} > >> - > >> -static void ttm_resource_iounmap(struct ttm_device *bdev, > >> - struct ttm_resource *mem, > >> - void *virtual) > >> -{ > >> - if (virtual && mem->bus.addr =3D=3D NULL) > >> - iounmap(virtual); > >> - ttm_mem_io_free(bdev, mem); > >> -} > >> - > >> -static int ttm_copy_io_page(void *dst, void *src, unsigned long page) > >> -{ > >> - uint32_t *dstP =3D > >> - (uint32_t *) ((unsigned long)dst + (page << PAGE_SHIFT)); > >> - uint32_t *srcP =3D > >> - (uint32_t *) ((unsigned long)src + (page << PAGE_SHIFT)); > >> - > >> - int i; > >> - for (i =3D 0; i < PAGE_SIZE / sizeof(uint32_t); ++i) > >> - iowrite32(ioread32(srcP++), dstP++); > >> - return 0; > >> -} > >> - > >> -static int ttm_copy_io_ttm_page(struct ttm_tt *ttm, void *src, > >> - unsigned long page, > >> - pgprot_t prot) > >> -{ > >> - struct page *d =3D ttm->pages[page]; > >> - void *dst; > >> - > >> - if (!d) > >> - return -ENOMEM; > >> - > >> - src =3D (void *)((unsigned long)src + (page << PAGE_SHIFT)); > >> - dst =3D kmap_atomic_prot(d, prot); > >> - if (!dst) > >> - return -ENOMEM; > >> - > >> - memcpy_fromio(dst, src, PAGE_SIZE); > >> - > >> - kunmap_atomic(dst); > >> - > >> - return 0; > >> -} > >> - > >> -static int ttm_copy_ttm_io_page(struct ttm_tt *ttm, void *dst, > >> - unsigned long page, > >> - pgprot_t prot) > >> -{ > >> - struct page *s =3D ttm->pages[page]; > >> - void *src; > >> - > >> - if (!s) > >> - return -ENOMEM; > >> - > >> - dst =3D (void *)((unsigned long)dst + (page << PAGE_SHIFT)); > >> - src =3D kmap_atomic_prot(s, prot); > >> - if (!src) > >> - return -ENOMEM; > >> > >> - memcpy_toio(dst, src, PAGE_SIZE); > >> - > >> - kunmap_atomic(src); > >> + for (i =3D 0; i < dst_mem->num_pages; ++i) { > >> + dst_ops->map_local(dst_iter, &dst_map, i); > >> + src_ops->map_local(src_iter, &src_map, i); > >> + > >> + if (!src_map.is_iomem && !dst_map.is_iomem) { > >> + memcpy(dst_map.vaddr, src_map.vaddr, PAGE_SIZE= ); > >> + } else if (!src_map.is_iomem) { > >> + dma_buf_map_memcpy_to(&dst_map, src_map.vaddr, > >> + PAGE_SIZE); > >> + } else if (!dst_map.is_iomem) { > >> + memcpy_fromio(dst_map.vaddr, src_map.vaddr_iom= em, > >> + PAGE_SIZE); > >> + } else { > >> + int j; > >> + u32 __iomem *src =3D src_map.vaddr_iomem; > >> + u32 __iomem *dst =3D dst_map.vaddr_iomem; > >> > >> - return 0; > >> + for (j =3D 0; j < (PAGE_SIZE >> 2); ++j) > > IMO PAGE_SIZE / sizeof(u32) is easier to understand. > > OK, will fix. > > > > > >> + iowrite32(ioread32(src++), dst++); > >> + } > >> + if (src_ops->unmap_local) > >> + src_ops->unmap_local(src_iter, &src_map); > >> + if (dst_ops->unmap_local) > >> + dst_ops->unmap_local(dst_iter, &dst_map); > >> + } > >> } > >> +EXPORT_SYMBOL(ttm_move_memcpy); > >> > >> int ttm_bo_move_memcpy(struct ttm_buffer_object *bo, > >> struct ttm_operation_ctx *ctx, > >> - struct ttm_resource *new_mem) > >> + struct ttm_resource *dst_mem) > >> { > >> struct ttm_device *bdev =3D bo->bdev; > >> - struct ttm_resource_manager *man =3D ttm_manager_type(bdev, ne= w_mem->mem_type); > >> + struct ttm_resource_manager *dst_man =3D > >> + ttm_manager_type(bo->bdev, dst_mem->mem_type); > >> struct ttm_tt *ttm =3D bo->ttm; > >> - struct ttm_resource *old_mem =3D &bo->mem; > >> - struct ttm_resource old_copy =3D *old_mem; > >> - void *old_iomap; > >> - void *new_iomap; > >> + struct ttm_resource *src_mem =3D &bo->mem; > >> + struct ttm_resource_manager *src_man =3D > >> + ttm_manager_type(bdev, src_mem->mem_type); > >> + struct ttm_resource src_copy =3D *src_mem; > >> + union { > >> + struct ttm_kmap_iter_tt tt; > >> + struct ttm_kmap_iter_linear_io io; > >> + } _dst_iter, _src_iter; > >> + struct ttm_kmap_iter *dst_iter, *src_iter; > >> int ret; > >> - unsigned long i; > >> > >> - ret =3D ttm_bo_wait_ctx(bo, ctx); > >> - if (ret) > >> - return ret; > >> - > >> - ret =3D ttm_resource_ioremap(bdev, old_mem, &old_iomap); > >> - if (ret) > >> - return ret; > >> - ret =3D ttm_resource_ioremap(bdev, new_mem, &new_iomap); > >> - if (ret) > >> - goto out; > >> - > >> - /* > >> - * Single TTM move. NOP. > >> - */ > >> - if (old_iomap =3D=3D NULL && new_iomap =3D=3D NULL) > >> - goto out2; > >> - > >> - /* > >> - * Don't move nonexistent data. Clear destination instead. > >> - */ > >> - if (old_iomap =3D=3D NULL && > >> - (ttm =3D=3D NULL || (!ttm_tt_is_populated(ttm) && > >> - !(ttm->page_flags & TTM_PAGE_FLAG_SWAPPED= )))) { > >> - memset_io(new_iomap, 0, new_mem->num_pages*PAGE_SIZE); > >> - goto out2; > >> - } > >> - > >> - /* > >> - * TTM might be null for moves within the same region. > >> - */ > >> - if (ttm) { > >> + if (ttm && ((ttm->page_flags & TTM_PAGE_FLAG_SWAPPED) || > >> + dst_man->use_tt)) { > >> ret =3D ttm_tt_populate(bdev, ttm, ctx); > >> if (ret) > >> - goto out1; > >> + return ret; > >> } > >> > >> - for (i =3D 0; i < new_mem->num_pages; ++i) { > >> - if (old_iomap =3D=3D NULL) { > >> - pgprot_t prot =3D ttm_io_prot(bo, old_mem, PAG= E_KERNEL); > >> - ret =3D ttm_copy_ttm_io_page(ttm, new_iomap, i= , > >> - prot); > >> - } else if (new_iomap =3D=3D NULL) { > >> - pgprot_t prot =3D ttm_io_prot(bo, new_mem, PAG= E_KERNEL); > >> - ret =3D ttm_copy_io_ttm_page(ttm, old_iomap, i= , > >> - prot); > >> - } else { > >> - ret =3D ttm_copy_io_page(new_iomap, old_iomap,= i); > >> - } > >> - if (ret) > >> - goto out1; > >> + dst_iter =3D ttm_kmap_iter_linear_io_init(&_dst_iter.io, bdev,= dst_mem); > >> + if (PTR_ERR(dst_iter) =3D=3D -EINVAL && dst_man->use_tt) > >> + dst_iter =3D ttm_kmap_iter_tt_init(&_dst_iter.tt, bo->= ttm); > >> + if (IS_ERR(dst_iter)) > >> + return PTR_ERR(dst_iter); > >> + > >> + src_iter =3D ttm_kmap_iter_linear_io_init(&_src_iter.io, bdev,= src_mem); > >> + if (PTR_ERR(src_iter) =3D=3D -EINVAL && src_man->use_tt) > >> + src_iter =3D ttm_kmap_iter_tt_init(&_src_iter.tt, bo->= ttm); > >> + if (IS_ERR(src_iter)) { > >> + ret =3D PTR_ERR(src_iter); > >> + goto out_src_iter; > >> } > >> - mb(); > >> -out2: > >> - old_copy =3D *old_mem; > >> > >> - ttm_bo_assign_mem(bo, new_mem); > >> - > >> - if (!man->use_tt) > >> - ttm_bo_tt_destroy(bo); > >> + ttm_move_memcpy(bo, dst_mem, dst_iter, src_iter); > >> + src_copy =3D *src_mem; > >> + ttm_bo_move_sync_cleanup(bo, dst_mem); > >> > >> -out1: > >> - ttm_resource_iounmap(bdev, old_mem, new_iomap); > >> -out: > >> - ttm_resource_iounmap(bdev, &old_copy, old_iomap); > >> + if (!src_iter->ops->maps_tt) > >> + ttm_kmap_iter_linear_io_fini(&_src_iter.io, bdev, &src= _copy); > >> +out_src_iter: > >> + if (!dst_iter->ops->maps_tt) > >> + ttm_kmap_iter_linear_io_fini(&_dst_iter.io, bdev, dst_= mem); > >> > >> - /* > >> - * On error, keep the mm node! > >> - */ > >> - if (!ret) > >> - ttm_resource_free(bo, &old_copy); > >> return ret; > >> } > >> EXPORT_SYMBOL(ttm_bo_move_memcpy); > >> @@ -336,27 +272,7 @@ pgprot_t ttm_io_prot(struct ttm_buffer_object *bo= , struct ttm_resource *res, > >> man =3D ttm_manager_type(bo->bdev, res->mem_type); > >> caching =3D man->use_tt ? bo->ttm->caching : res->bus.caching= ; > >> > >> - /* Cached mappings need no adjustment */ > >> - if (caching =3D=3D ttm_cached) > >> - return tmp; > >> - > >> -#if defined(__i386__) || defined(__x86_64__) > >> - if (caching =3D=3D ttm_write_combined) > >> - tmp =3D pgprot_writecombine(tmp); > >> - else if (boot_cpu_data.x86 > 3) > >> - tmp =3D pgprot_noncached(tmp); > >> -#endif > >> -#if defined(__ia64__) || defined(__arm__) || defined(__aarch64__) || = \ > >> - defined(__powerpc__) || defined(__mips__) > >> - if (caching =3D=3D ttm_write_combined) > >> - tmp =3D pgprot_writecombine(tmp); > >> - else > >> - tmp =3D pgprot_noncached(tmp); > >> -#endif > >> -#if defined(__sparc__) > >> - tmp =3D pgprot_noncached(tmp); > >> -#endif > >> - return tmp; > >> + return ttm_prot_from_caching(caching, tmp); > >> } > >> EXPORT_SYMBOL(ttm_io_prot); > >> > >> diff --git a/drivers/gpu/drm/ttm/ttm_module.c b/drivers/gpu/drm/ttm/tt= m_module.c > >> index 56b0efdba1a9..997c458f68a9 100644 > >> --- a/drivers/gpu/drm/ttm/ttm_module.c > >> +++ b/drivers/gpu/drm/ttm/ttm_module.c > >> @@ -31,12 +31,47 @@ > >> */ > >> #include > >> #include > >> +#include > >> #include > >> #include > >> #include > >> +#include > >> > >> #include "ttm_module.h" > >> > >> +/** > >> + * ttm_prot_from_caching - Modify the page protection according to th= e > >> + * ttm cacing mode > >> + * @caching: The ttm caching mode > >> + * @tmp: The original page protection > >> + * > >> + * Return: The modified page protection > >> + */ > >> +pgprot_t ttm_prot_from_caching(enum ttm_caching caching, pgprot_t tmp= ) > >> +{ > >> + /* Cached mappings need no adjustment */ > >> + if (caching =3D=3D ttm_cached) > >> + return tmp; > >> + > >> +#if defined(__i386__) || defined(__x86_64__) > >> + if (caching =3D=3D ttm_write_combined) > >> + tmp =3D pgprot_writecombine(tmp); > >> + else if (boot_cpu_data.x86 > 3) > >> + tmp =3D pgprot_noncached(tmp); > >> +#endif > >> +#if defined(__ia64__) || defined(__arm__) || defined(__aarch64__) || = \ > >> + defined(__powerpc__) || defined(__mips__) > >> + if (caching =3D=3D ttm_write_combined) > >> + tmp =3D pgprot_writecombine(tmp); > >> + else > >> + tmp =3D pgprot_noncached(tmp); > >> +#endif > >> +#if defined(__sparc__) > >> + tmp =3D pgprot_noncached(tmp); > >> +#endif > >> + return tmp; > >> +} > >> + > >> struct dentry *ttm_debugfs_root; > >> > >> static int __init ttm_init(void) > >> diff --git a/drivers/gpu/drm/ttm/ttm_resource.c b/drivers/gpu/drm/ttm/= ttm_resource.c > >> index 59e2b7157e41..e05ae7e3d477 100644 > >> --- a/drivers/gpu/drm/ttm/ttm_resource.c > >> +++ b/drivers/gpu/drm/ttm/ttm_resource.c > >> @@ -22,6 +22,10 @@ > >> * Authors: Christian K=C3=B6nig > >> */ > >> > >> +#include > >> +#include > >> +#include > >> + > >> #include > >> #include > >> > >> @@ -147,3 +151,165 @@ void ttm_resource_manager_debug(struct ttm_resou= rce_manager *man, > >> man->func->debug(man, p); > >> } > >> EXPORT_SYMBOL(ttm_resource_manager_debug); > >> + > >> +static void ttm_kmap_iter_iomap_map_local(struct ttm_kmap_iter *iter, > >> + struct dma_buf_map *dmap, > >> + pgoff_t i) > >> +{ > >> + struct ttm_kmap_iter_iomap *iter_io =3D > >> + container_of(iter, typeof(*iter_io), base); > >> + void __iomem *addr; > >> + > >> +retry: > >> + while (i >=3D iter_io->cache.end) { > >> + iter_io->cache.sg =3D iter_io->cache.sg ? > >> + sg_next(iter_io->cache.sg) : iter_io->st->sgl; > >> + iter_io->cache.i =3D iter_io->cache.end; > >> + iter_io->cache.end +=3D sg_dma_len(iter_io->cache.sg) = >> > >> + PAGE_SHIFT; > >> + iter_io->cache.offs =3D sg_dma_address(iter_io->cache.= sg) - > >> + iter_io->start; > >> + } > >> + > >> + if (i < iter_io->cache.i) { > >> + iter_io->cache.end =3D 0; > >> + iter_io->cache.sg =3D NULL; > >> + goto retry; > >> + } > >> + > >> + addr =3D io_mapping_map_local_wc(iter_io->iomap, iter_io->cach= e.offs + > >> + (((resource_size_t)i - iter_io-= >cache.i) > >> + << PAGE_SHIFT)); > >> + dma_buf_map_set_vaddr_iomem(dmap, addr); > >> +} > >> + > >> +static void ttm_kmap_iter_iomap_unmap_local(struct ttm_kmap_iter *ite= r, > >> + struct dma_buf_map *map) > >> +{ > >> + io_mapping_unmap_local(map->vaddr_iomem); > >> +} > >> + > >> +static const struct ttm_kmap_iter_ops ttm_kmap_iter_io_ops =3D { > >> + .map_local =3D ttm_kmap_iter_iomap_map_local, > >> + .unmap_local =3D ttm_kmap_iter_iomap_unmap_local, > >> + .maps_tt =3D false, > >> +}; > >> + > >> +/** > >> + * ttm_kmap_iter_iomap_init - Initialize a struct ttm_kmap_iter_iomap > >> + * @iter_io: The struct ttm_kmap_iter_iomap to initialize. > >> + * @iomap: The struct io_mapping representing the underlying linear i= o_memory. > >> + * @st: sg_table into @iomap, representing the memory of the struct > >> + * ttm_resource. > >> + * @start: Offset that needs to be subtracted from @st to make > >> + * sg_dma_address(st->sgl) - @start =3D=3D 0 for @iomap start. > >> + * > >> + * Return: Pointer to the embedded struct ttm_kmap_iter. > >> + */ > >> +struct ttm_kmap_iter * > >> +ttm_kmap_iter_iomap_init(struct ttm_kmap_iter_iomap *iter_io, > >> + struct io_mapping *iomap, > >> + struct sg_table *st, > >> + resource_size_t start) > >> +{ > >> + iter_io->base.ops =3D &ttm_kmap_iter_io_ops; > >> + iter_io->iomap =3D iomap; > >> + iter_io->st =3D st; > >> + iter_io->start =3D start; > >> + memset(&iter_io->cache, 0, sizeof(iter_io->cache)); > >> + > >> + return &iter_io->base; > >> +} > >> +EXPORT_SYMBOL(ttm_kmap_iter_iomap_init); > >> + > >> +/** > >> + * DOC: Linear io iterator > >> + * > >> + * This code should die in the not too near future. Best would be if = we could > >> + * make io-mapping use memremap for all io memory, and have memremap > >> + * implement a kmap_local functionality. We could then strip a huge a= mount of > >> + * code. These linear io iterators are implemented to mimic old funct= ionality, > >> + * and they don't use kmap_local semantics at all internally. Rather = ioremap or > >> + * friends, and at least on 32-bit they add global TLB flushes and po= ints > >> + * of failure. > >> + */ > >> + > >> +static void ttm_kmap_iter_linear_io_map_local(struct ttm_kmap_iter *i= ter, > >> + struct dma_buf_map *dmap= , > >> + pgoff_t i) > >> +{ > >> + struct ttm_kmap_iter_linear_io *iter_io =3D > >> + container_of(iter, typeof(*iter_io), base); > >> + > >> + *dmap =3D iter_io->dmap; > >> + dma_buf_map_incr(dmap, i * PAGE_SIZE); > >> +} > >> + > >> +static const struct ttm_kmap_iter_ops ttm_kmap_iter_linear_io_ops =3D= { > >> + .map_local =3D ttm_kmap_iter_linear_io_map_local, > >> + .maps_tt =3D false, > >> +}; > >> + > >> +struct ttm_kmap_iter * > >> +ttm_kmap_iter_linear_io_init(struct ttm_kmap_iter_linear_io *iter_io, > >> + struct ttm_device *bdev, > >> + struct ttm_resource *mem) > >> +{ > >> + int ret; > >> + > >> + ret =3D ttm_mem_io_reserve(bdev, mem); > >> + if (ret) > >> + goto out_err; > >> + if (!mem->bus.is_iomem) { > >> + ret =3D -EINVAL; > >> + goto out_io_free; > >> + } > >> + > >> + if (mem->bus.addr) { > >> + dma_buf_map_set_vaddr(&iter_io->dmap, mem->bus.addr); > >> + iter_io->needs_unmap =3D false; > >> + } else { > >> + size_t bus_size =3D (size_t)mem->num_pages << PAGE_SHI= FT; > >> + > >> + iter_io->needs_unmap =3D true; > >> + if (mem->bus.caching =3D=3D ttm_write_combined) > >> + dma_buf_map_set_vaddr_iomem(&iter_io->dmap, > >> + ioremap_wc(mem->bu= s.offset, > >> + bus_siz= e)); > >> + else if (mem->bus.caching =3D=3D ttm_cached) > >> + dma_buf_map_set_vaddr(&iter_io->dmap, > >> + memremap(mem->bus.offset= , bus_size, > >> + MEMREMAP_WB)); > > The comments in set_vaddr suggest that this is meant for > > system-memory. Does that actually matter or is it just about not > > losing the __iomem annotation on platforms where it matters? > > Yes, it's the latter. dma_buf_map() is relatively new and the author > probably didn't think about the case of cached iomem, which is used by, > for example, vmwgfx. > > > Apparently cached device local is a thing. Also should this not be > > wrapped in CONFIG_X86? > > Both dma_buf_map() and memremap are generic, I think, I guess memremap > would return NULL if it's not supported. It looks like memremap just wraps ioremap_cache, but since it also discards the __iomem annotation should we be doing that universally? Also not sure if ioremap_cache is universally supported, so wrapping in CONFIG_X86 and falling back to plain ioremap() might be needed? Or at least that looks like roughly what the previous code was doing? Not too sure tbh. > > /Thomas > > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C352C2B9F8 for ; Tue, 25 May 2021 09:59:16 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B1BA76142A for ; Tue, 25 May 2021 09:59:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B1BA76142A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2FFF86E1B8; Tue, 25 May 2021 09:59:15 +0000 (UTC) Received: from mail-qt1-x82c.google.com (mail-qt1-x82c.google.com [IPv6:2607:f8b0:4864:20::82c]) by gabe.freedesktop.org (Postfix) with ESMTPS id 421D86E1B8; Tue, 25 May 2021 09:59:14 +0000 (UTC) Received: by mail-qt1-x82c.google.com with SMTP id h21so22688496qtu.5; Tue, 25 May 2021 02:59:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=AOLh2SS/Gc8zZamjOdd0Bg0Az6U4VqWJmXFHFKKKNNc=; b=IZV/m+jELEFyFkM6MQ7qsXFQzjEP7sAEkOqTFePj8YdaSG2Gai0iHTPVo+BILd9OZz kInG7SISsml/L3uPxQEluf3CBDtHV0fBpAuoiVmX+Z/kiZjb7PlP+H/6EBJ5rdonbTvm xxClVL9vMDOmAcBuPfVE4fbjwsRjKl1Qt8vLTu6RI8vDbM8yDjM7dJ+isiVcF9LHyEVo ZDWl22Wz5WLU3SZRbEnMEZDJTNqT6lfcJ2tdTXUwIVZuwxbxFFeQFSjPwRwzj8/QwDTf eqIF93vHTE4zkRih6AFA1Q7iMADtBD2aE5H8SyKePl6LtJqSeLyepJW6cwq1R9mlHP+B oxPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=AOLh2SS/Gc8zZamjOdd0Bg0Az6U4VqWJmXFHFKKKNNc=; b=PQychsOQVzETVno85YApJMcEFivd1hgeQQ02OcJ0zFhayiYfIQNoJfi0arXiQO4L7u abrpGpxaYTjQk/ENtsodz51fFWnKZlvXUJh1+hIvB0fLTnxdqApHE7LrOlYSk4PVaEEf dQipktQMa8UhQJLjZK83mSVt+W5w/5f1lIIw04Q39UJy/16xQAD4WZElDYfrsINtrovA GVpDKCsQm2G/yw9gZxNPKSShvR4l423Zy4MaRgOwoFLpn1B0oG7KwCJGV+HSLHGdvd1t d0cX4ZFTv6d3TsthswjlUYKxzU2hHTW4gSb5qrSaPMDIdhXoKZpkBropn2+ukHgSlTpQ 8rWg== X-Gm-Message-State: AOAM530bBuOcUWeXwhqFirxLPa3V8N3pDbtTwXpqx4EjRSmvwWd70YlS rrnopRxvpJPItBFNQrGaEpHomrt/KzctCarEpEo= X-Google-Smtp-Source: ABdhPJxl2ZO83O91yr4U0yC86m9bQvTGmNz2toG/OJyl8j4WXunhrEIQlZtdEe2geDvRWqOCl8Ez2+AeL6FeVlzHO30= X-Received: by 2002:ac8:60d:: with SMTP id d13mr31849064qth.223.1621936753060; Tue, 25 May 2021 02:59:13 -0700 (PDT) MIME-Version: 1.0 References: <20210521153253.518037-1-thomas.hellstrom@linux.intel.com> <20210521153253.518037-7-thomas.hellstrom@linux.intel.com> <2cc9a60c-4360-40b6-8712-1e50b7bbfd03@linux.intel.com> In-Reply-To: <2cc9a60c-4360-40b6-8712-1e50b7bbfd03@linux.intel.com> From: Matthew Auld Date: Tue, 25 May 2021 10:58:46 +0100 Message-ID: To: =?UTF-8?Q?Thomas_Hellstr=C3=B6m?= Subject: Re: [Intel-gfx] [PATCH v3 06/12] drm/ttm: Add a generic TTM memcpy move for page-based iomem X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Intel Graphics Development , =?UTF-8?Q?Christian_K=C3=B6nig?= , ML dri-devel Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" T24gVHVlLCAyNSBNYXkgMjAyMSBhdCAxMDozMiwgVGhvbWFzIEhlbGxzdHLDtm0KPHRob21hcy5o ZWxsc3Ryb21AbGludXguaW50ZWwuY29tPiB3cm90ZToKPgo+Cj4gT24gNS8yNS8yMSAxMToxOCBB TSwgTWF0dGhldyBBdWxkIHdyb3RlOgo+ID4gT24gRnJpLCAyMSBNYXkgMjAyMSBhdCAxNjozMywg VGhvbWFzIEhlbGxzdHLDtm0KPiA+IDx0aG9tYXMuaGVsbHN0cm9tQGxpbnV4LmludGVsLmNvbT4g d3JvdGU6Cj4gPj4gVGhlIGludGVybmFsIHR0bV9ib191dGlsIG1lbWNweSB1c2VzIGlvcmVtYXAg ZnVuY3Rpb25hbGl0eSwgYW5kIHdoaWxlIGl0Cj4gPj4gcHJvYmFibHkgbWlnaHQgYmUgcG9zc2li bGUgdG8gdXNlIGl0IGZvciBjb3B5aW5nIGluLSBhbmQgb3V0IG9mCj4gPj4gc2dsaXN0IHJlcHJl c2VudGVkIGlvIG1lbW9yeSwgdXNpbmcgaW9fbWVtX3Jlc2VydmUoKSAvIGlvX21lbV9mcmVlKCkK PiA+PiBjYWxsYmFja3MsIHRoYXQgd291bGQgY2F1c2UgcHJvYmxlbXMgd2l0aCBmYXVsdCgpLgo+ ID4+IEluc3RlYWQsIGltcGxlbWVudCBhIG1ldGhvZCBtYXBwaW5nIHBhZ2UtYnktcGFnZSB1c2lu ZyBrbWFwX2xvY2FsKCkKPiA+PiBzZW1hbnRpY3MuIEFzIGFuIGFkZGl0aW9uYWwgYmVuZWZpdCB3 ZSB0aGVuIGF2b2lkIHRoZSBvY2Nhc2lvbmFsIGdsb2JhbAo+ID4+IFRMQiBmbHVzaGVzIG9mIGlv cmVtYXAoKSBhbmQgY29uc3VtaW5nIGlvcmVtYXAgc3BhY2UsIGVsaW1pbmF0aW9uIG9mIGEKPiA+ PiBjcml0aWNhbCBwb2ludCBvZiBmYWlsdXJlIGFuZCB3aXRoIGEgc2xpZ2h0IGNoYW5nZSBvZiBz ZW1hbnRpY3Mgd2UgY291bGQKPiA+PiBhbHNvIHB1c2ggdGhlIG1lbWNweSBvdXQgYXN5bmMgZm9y IHRlc3RpbmcgYW5kIGFzeW5jIGRyaXZlciBkZXZlbG9wbWVudAo+ID4+IHB1cnBvc2VzLgo+ID4+ Cj4gPj4gQSBzcGVjaWFsIGxpbmVhciBpb21lbSBpdGVyYXRvciBpcyBpbnRyb2R1Y2VkIGludGVy bmFsbHkgdG8gbWltaWMgdGhlCj4gPj4gb2xkIGlvcmVtYXAgYmVoYXZpb3VyIGZvciBjb2RlLXBh dGhzIHRoYXQgY2FuJ3QgaW1tZWRpYXRlbHkgYmUgcG9ydGVkCj4gPj4gb3Zlci4gVGhpcyBhZGRz IHRvIHRoZSBjb2RlIHNpemUgYW5kIHNob3VsZCBiZSBjb25zaWRlcmVkIGEgdGVtcG9yYXJ5Cj4g Pj4gc29sdXRpb24uCj4gPj4KPiA+PiBMb29raW5nIGF0IHRoZSBjb2RlIHdlIGhhdmUgYSBsb3Qg b2YgY2hlY2tzIGZvciBpb21hcCB0YWdnZWQgcG9pbnRlcnMuCj4gPj4gSWRlYWxseSB3ZSBzaG91 bGQgZXh0ZW5kIHRoZSBjb3JlIG1lbXJlbWFwIGZ1bmN0aW9ucyB0byBhbHNvIGFjY2VwdAo+ID4+ IHVuY2FjaGVkIG1lbW9yeSBhbmQga21hcF9sb2NhbCBmdW5jdGlvbmFsaXR5LiBUaGVuIHdlIGNv dWxkIHN0cmlwIGEKPiA+PiBsb3Qgb2YgY29kZS4KPiA+Pgo+ID4+IENjOiBDaHJpc3RpYW4gS8O2 bmlnIDxjaHJpc3RpYW4ua29lbmlnQGFtZC5jb20+Cj4gPj4gU2lnbmVkLW9mZi1ieTogVGhvbWFz IEhlbGxzdHLDtm0gPHRob21hcy5oZWxsc3Ryb21AbGludXguaW50ZWwuY29tPgo+ID4+IC0tLQo+ ID4+IHYzOgo+ID4+IC0gU3BsaXQgdXAgaW4gdmFyaW91cyBUVE0gZmlsZXMgYW5kIGFkZHJlc3Nl ZCByZXZpZXcgY29tbWVudHMgYnkKPiA+PiAgICBDaHJpc3RpYW4gS8O2bmlnLiBUZXN0ZWQgYW5k IGZpeGVkIGxlZ2FjeSBpb21hcCBtZW1jcHkgcGF0aCBvbiBpOTE1Lgo+ID4+IC0tLQo+ID4+ICAg ZHJpdmVycy9ncHUvZHJtL3R0bS90dG1fYm9fdXRpbC5jICB8IDI3OCArKysrKysrKysrLS0tLS0t LS0tLS0tLS0tLS0tLQo+ID4+ICAgZHJpdmVycy9ncHUvZHJtL3R0bS90dG1fbW9kdWxlLmMgICB8 ICAzNSArKysrCj4gPj4gICBkcml2ZXJzL2dwdS9kcm0vdHRtL3R0bV9yZXNvdXJjZS5jIHwgMTY2 ICsrKysrKysrKysrKysrKysrCj4gPj4gICBkcml2ZXJzL2dwdS9kcm0vdHRtL3R0bV90dC5jICAg ICAgIHwgIDQyICsrKysrCj4gPj4gICBpbmNsdWRlL2RybS90dG0vdHRtX2JvX2RyaXZlci5oICAg IHwgIDI4ICsrKwo+ID4+ICAgaW5jbHVkZS9kcm0vdHRtL3R0bV9jYWNoaW5nLmggICAgICB8ICAg MiArCj4gPj4gICBpbmNsdWRlL2RybS90dG0vdHRtX2ttYXBfaXRlci5oICAgIHwgIDYxICsrKysr KysKPiA+PiAgIGluY2x1ZGUvZHJtL3R0bS90dG1fcmVzb3VyY2UuaCAgICAgfCAgNjEgKysrKysr Kwo+ID4+ICAgaW5jbHVkZS9kcm0vdHRtL3R0bV90dC5oICAgICAgICAgICB8ICAxNiArKwo+ID4+ ICAgOSBmaWxlcyBjaGFuZ2VkLCA1MDggaW5zZXJ0aW9ucygrKSwgMTgxIGRlbGV0aW9ucygtKQo+ ID4+ICAgY3JlYXRlIG1vZGUgMTAwNjQ0IGluY2x1ZGUvZHJtL3R0bS90dG1fa21hcF9pdGVyLmgK PiA+Pgo+ID4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vdHRtL3R0bV9ib191dGlsLmMg Yi9kcml2ZXJzL2dwdS9kcm0vdHRtL3R0bV9ib191dGlsLmMKPiA+PiBpbmRleCBhZThiNjE0NjA3 MjQuLjkxMmNiZThlNjBhMiAxMDA2NDQKPiA+PiAtLS0gYS9kcml2ZXJzL2dwdS9kcm0vdHRtL3R0 bV9ib191dGlsLmMKPiA+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vdHRtL3R0bV9ib191dGlsLmMK PiA+PiBAQCAtNzIsMTkwICs3MiwxMjYgQEAgdm9pZCB0dG1fbWVtX2lvX2ZyZWUoc3RydWN0IHR0 bV9kZXZpY2UgKmJkZXYsCj4gPj4gICAgICAgICAgbWVtLT5idXMuYWRkciA9IE5VTEw7Cj4gPj4g ICB9Cj4gPj4KPiA+PiAtc3RhdGljIGludCB0dG1fcmVzb3VyY2VfaW9yZW1hcChzdHJ1Y3QgdHRt X2RldmljZSAqYmRldiwKPiA+PiAtICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0 IHR0bV9yZXNvdXJjZSAqbWVtLAo+ID4+IC0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICB2 b2lkICoqdmlydHVhbCkKPiA+PiArLyoqCj4gPj4gKyAqIHR0bV9tb3ZlX21lbWNweSAtIEhlbHBl ciB0byBwZXJmb3JtIGEgbWVtY3B5IHR0bSBtb3ZlIG9wZXJhdGlvbi4KPiA+PiArICogQGJvOiBU aGUgc3RydWN0IHR0bV9idWZmZXJfb2JqZWN0Lgo+ID4+ICsgKiBAbmV3X21lbTogVGhlIHN0cnVj dCB0dG1fcmVzb3VyY2Ugd2UncmUgbW92aW5nIHRvIChjb3B5IGRlc3RpbmF0aW9uKS4KPiA+PiAr ICogQG5ld19pdGVyOiBBIHN0cnVjdCB0dG1fa21hcF9pdGVyIHJlcHJlc2VudGluZyB0aGUgZGVz dGluYXRpb24gcmVzb3VyY2UuCj4gPj4gKyAqIEBzcmNfaXRlcjogQSBzdHJ1Y3QgdHRtX2ttYXBf aXRlciByZXByZXNlbnRpbmcgdGhlIHNvdXJjZSByZXNvdXJjZS4KPiA+PiArICoKPiA+PiArICog VGhpcyBmdW5jdGlvbiBpcyBpbnRlbmRlZCB0byBiZSBhYmxlIHRvIG1vdmUgb3V0IGFzeW5jIHVu ZGVyIGEKPiA+PiArICogZG1hLWZlbmNlIGlmIGRlc2lyZWQuCj4gPj4gKyAqLwo+ID4+ICt2b2lk IHR0bV9tb3ZlX21lbWNweShzdHJ1Y3QgdHRtX2J1ZmZlcl9vYmplY3QgKmJvLAo+ID4+ICsgICAg ICAgICAgICAgICAgICAgIHN0cnVjdCB0dG1fcmVzb3VyY2UgKmRzdF9tZW0sCj4gPj4gKyAgICAg ICAgICAgICAgICAgICAgc3RydWN0IHR0bV9rbWFwX2l0ZXIgKmRzdF9pdGVyLAo+ID4+ICsgICAg ICAgICAgICAgICAgICAgIHN0cnVjdCB0dG1fa21hcF9pdGVyICpzcmNfaXRlcikKPiA+PiAgIHsK PiA+PiAtICAgICAgIGludCByZXQ7Cj4gPj4gLSAgICAgICB2b2lkICphZGRyOwo+ID4+IC0KPiA+ PiAtICAgICAgICp2aXJ0dWFsID0gTlVMTDsKPiA+PiAtICAgICAgIHJldCA9IHR0bV9tZW1faW9f cmVzZXJ2ZShiZGV2LCBtZW0pOwo+ID4+IC0gICAgICAgaWYgKHJldCB8fCAhbWVtLT5idXMuaXNf aW9tZW0pCj4gPj4gLSAgICAgICAgICAgICAgIHJldHVybiByZXQ7Cj4gPj4gKyAgICAgICBjb25z dCBzdHJ1Y3QgdHRtX2ttYXBfaXRlcl9vcHMgKmRzdF9vcHMgPSBkc3RfaXRlci0+b3BzOwo+ID4+ ICsgICAgICAgY29uc3Qgc3RydWN0IHR0bV9rbWFwX2l0ZXJfb3BzICpzcmNfb3BzID0gc3JjX2l0 ZXItPm9wczsKPiA+PiArICAgICAgIHN0cnVjdCB0dG1fdHQgKnR0bSA9IGJvLT50dG07Cj4gPj4g KyAgICAgICBzdHJ1Y3QgZG1hX2J1Zl9tYXAgc3JjX21hcCwgZHN0X21hcDsKPiA+PiArICAgICAg IHBnb2ZmX3QgaTsKPiA+Pgo+ID4+IC0gICAgICAgaWYgKG1lbS0+YnVzLmFkZHIpIHsKPiA+PiAt ICAgICAgICAgICAgICAgYWRkciA9IG1lbS0+YnVzLmFkZHI7Cj4gPj4gLSAgICAgICB9IGVsc2Ug ewo+ID4+IC0gICAgICAgICAgICAgICBzaXplX3QgYnVzX3NpemUgPSAoc2l6ZV90KW1lbS0+bnVt X3BhZ2VzIDw8IFBBR0VfU0hJRlQ7Cj4gPj4gKyAgICAgICAvKiBTaW5nbGUgVFRNIG1vdmUuIE5P UCAqLwo+ID4+ICsgICAgICAgaWYgKGRzdF9vcHMtPm1hcHNfdHQgJiYgc3JjX29wcy0+bWFwc190 dCkKPiA+PiArICAgICAgICAgICAgICAgcmV0dXJuOwo+ID4+Cj4gPj4gLSAgICAgICAgICAgICAg IGlmIChtZW0tPmJ1cy5jYWNoaW5nID09IHR0bV93cml0ZV9jb21iaW5lZCkKPiA+PiAtICAgICAg ICAgICAgICAgICAgICAgICBhZGRyID0gaW9yZW1hcF93YyhtZW0tPmJ1cy5vZmZzZXQsIGJ1c19z aXplKTsKPiA+PiAtI2lmZGVmIENPTkZJR19YODYKPiA+PiAtICAgICAgICAgICAgICAgZWxzZSBp ZiAobWVtLT5idXMuY2FjaGluZyA9PSB0dG1fY2FjaGVkKQo+ID4+IC0gICAgICAgICAgICAgICAg ICAgICAgIGFkZHIgPSBpb3JlbWFwX2NhY2hlKG1lbS0+YnVzLm9mZnNldCwgYnVzX3NpemUpOwo+ ID4+IC0jZW5kaWYKPiA+PiAtICAgICAgICAgICAgICAgZWxzZQo+ID4+IC0gICAgICAgICAgICAg ICAgICAgICAgIGFkZHIgPSBpb3JlbWFwKG1lbS0+YnVzLm9mZnNldCwgYnVzX3NpemUpOwo+ID4+ IC0gICAgICAgICAgICAgICBpZiAoIWFkZHIpIHsKPiA+PiAtICAgICAgICAgICAgICAgICAgICAg ICB0dG1fbWVtX2lvX2ZyZWUoYmRldiwgbWVtKTsKPiA+PiAtICAgICAgICAgICAgICAgICAgICAg ICByZXR1cm4gLUVOT01FTTsKPiA+PiArICAgICAgIC8qIERvbid0IG1vdmUgbm9uZXhpc3RlbnQg ZGF0YS4gQ2xlYXIgZGVzdGluYXRpb24gaW5zdGVhZC4gKi8KPiA+PiArICAgICAgIGlmIChzcmNf b3BzLT5tYXBzX3R0ICYmICghdHRtIHx8ICF0dG1fdHRfaXNfcG9wdWxhdGVkKHR0bSkpKSB7Cj4g Pj4gKyAgICAgICAgICAgICAgIGlmICh0dG0gJiYgISh0dG0tPnBhZ2VfZmxhZ3MgJiBUVE1fUEFH RV9GTEFHX1pFUk9fQUxMT0MpKQo+ID4+ICsgICAgICAgICAgICAgICAgICAgICAgIHJldHVybjsK PiA+PiArCj4gPj4gKyAgICAgICAgICAgICAgIGZvciAoaSA9IDA7IGkgPCBkc3RfbWVtLT5udW1f cGFnZXM7ICsraSkgewo+ID4+ICsgICAgICAgICAgICAgICAgICAgICAgIGRzdF9vcHMtPm1hcF9s b2NhbChkc3RfaXRlciwgJmRzdF9tYXAsIGkpOwo+ID4+ICsgICAgICAgICAgICAgICAgICAgICAg IGlmIChkc3RfbWFwLmlzX2lvbWVtKQo+ID4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgbWVtc2V0X2lvKGRzdF9tYXAudmFkZHJfaW9tZW0sIDAsIFBBR0VfU0laRSk7Cj4gPj4gKyAg ICAgICAgICAgICAgICAgICAgICAgZWxzZQo+ID4+ICsgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgbWVtc2V0KGRzdF9tYXAudmFkZHIsIDAsIFBBR0VfU0laRSk7Cj4gPj4gKyAgICAgICAg ICAgICAgICAgICAgICAgaWYgKGRzdF9vcHMtPnVubWFwX2xvY2FsKQo+ID4+ICsgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgZHN0X29wcy0+dW5tYXBfbG9jYWwoZHN0X2l0ZXIsICZkc3Rf bWFwKTsKPiA+PiAgICAgICAgICAgICAgICAgIH0KPiA+PiArICAgICAgICAgICAgICAgcmV0dXJu Owo+ID4+ICAgICAgICAgIH0KPiA+PiAtICAgICAgICp2aXJ0dWFsID0gYWRkcjsKPiA+PiAtICAg ICAgIHJldHVybiAwOwo+ID4+IC19Cj4gPj4gLQo+ID4+IC1zdGF0aWMgdm9pZCB0dG1fcmVzb3Vy Y2VfaW91bm1hcChzdHJ1Y3QgdHRtX2RldmljZSAqYmRldiwKPiA+PiAtICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIHN0cnVjdCB0dG1fcmVzb3VyY2UgKm1lbSwKPiA+PiAtICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgIHZvaWQgKnZpcnR1YWwpCj4gPj4gLXsKPiA+PiAtICAgICAg IGlmICh2aXJ0dWFsICYmIG1lbS0+YnVzLmFkZHIgPT0gTlVMTCkKPiA+PiAtICAgICAgICAgICAg ICAgaW91bm1hcCh2aXJ0dWFsKTsKPiA+PiAtICAgICAgIHR0bV9tZW1faW9fZnJlZShiZGV2LCBt ZW0pOwo+ID4+IC19Cj4gPj4gLQo+ID4+IC1zdGF0aWMgaW50IHR0bV9jb3B5X2lvX3BhZ2Uodm9p ZCAqZHN0LCB2b2lkICpzcmMsIHVuc2lnbmVkIGxvbmcgcGFnZSkKPiA+PiAtewo+ID4+IC0gICAg ICAgdWludDMyX3QgKmRzdFAgPQo+ID4+IC0gICAgICAgICAgICh1aW50MzJfdCAqKSAoKHVuc2ln bmVkIGxvbmcpZHN0ICsgKHBhZ2UgPDwgUEFHRV9TSElGVCkpOwo+ID4+IC0gICAgICAgdWludDMy X3QgKnNyY1AgPQo+ID4+IC0gICAgICAgICAgICh1aW50MzJfdCAqKSAoKHVuc2lnbmVkIGxvbmcp c3JjICsgKHBhZ2UgPDwgUEFHRV9TSElGVCkpOwo+ID4+IC0KPiA+PiAtICAgICAgIGludCBpOwo+ ID4+IC0gICAgICAgZm9yIChpID0gMDsgaSA8IFBBR0VfU0laRSAvIHNpemVvZih1aW50MzJfdCk7 ICsraSkKPiA+PiAtICAgICAgICAgICAgICAgaW93cml0ZTMyKGlvcmVhZDMyKHNyY1ArKyksIGRz dFArKyk7Cj4gPj4gLSAgICAgICByZXR1cm4gMDsKPiA+PiAtfQo+ID4+IC0KPiA+PiAtc3RhdGlj IGludCB0dG1fY29weV9pb190dG1fcGFnZShzdHJ1Y3QgdHRtX3R0ICp0dG0sIHZvaWQgKnNyYywK PiA+PiAtICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGxvbmcgcGFnZSwK PiA+PiAtICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBncHJvdF90IHByb3QpCj4gPj4g LXsKPiA+PiAtICAgICAgIHN0cnVjdCBwYWdlICpkID0gdHRtLT5wYWdlc1twYWdlXTsKPiA+PiAt ICAgICAgIHZvaWQgKmRzdDsKPiA+PiAtCj4gPj4gLSAgICAgICBpZiAoIWQpCj4gPj4gLSAgICAg ICAgICAgICAgIHJldHVybiAtRU5PTUVNOwo+ID4+IC0KPiA+PiAtICAgICAgIHNyYyA9ICh2b2lk ICopKCh1bnNpZ25lZCBsb25nKXNyYyArIChwYWdlIDw8IFBBR0VfU0hJRlQpKTsKPiA+PiAtICAg ICAgIGRzdCA9IGttYXBfYXRvbWljX3Byb3QoZCwgcHJvdCk7Cj4gPj4gLSAgICAgICBpZiAoIWRz dCkKPiA+PiAtICAgICAgICAgICAgICAgcmV0dXJuIC1FTk9NRU07Cj4gPj4gLQo+ID4+IC0gICAg ICAgbWVtY3B5X2Zyb21pbyhkc3QsIHNyYywgUEFHRV9TSVpFKTsKPiA+PiAtCj4gPj4gLSAgICAg ICBrdW5tYXBfYXRvbWljKGRzdCk7Cj4gPj4gLQo+ID4+IC0gICAgICAgcmV0dXJuIDA7Cj4gPj4g LX0KPiA+PiAtCj4gPj4gLXN0YXRpYyBpbnQgdHRtX2NvcHlfdHRtX2lvX3BhZ2Uoc3RydWN0IHR0 bV90dCAqdHRtLCB2b2lkICpkc3QsCj4gPj4gLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICB1bnNpZ25lZCBsb25nIHBhZ2UsCj4gPj4gLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBwZ3Byb3RfdCBwcm90KQo+ID4+IC17Cj4gPj4gLSAgICAgICBzdHJ1Y3QgcGFnZSAqcyA9IHR0 bS0+cGFnZXNbcGFnZV07Cj4gPj4gLSAgICAgICB2b2lkICpzcmM7Cj4gPj4gLQo+ID4+IC0gICAg ICAgaWYgKCFzKQo+ID4+IC0gICAgICAgICAgICAgICByZXR1cm4gLUVOT01FTTsKPiA+PiAtCj4g Pj4gLSAgICAgICBkc3QgPSAodm9pZCAqKSgodW5zaWduZWQgbG9uZylkc3QgKyAocGFnZSA8PCBQ QUdFX1NISUZUKSk7Cj4gPj4gLSAgICAgICBzcmMgPSBrbWFwX2F0b21pY19wcm90KHMsIHByb3Qp Owo+ID4+IC0gICAgICAgaWYgKCFzcmMpCj4gPj4gLSAgICAgICAgICAgICAgIHJldHVybiAtRU5P TUVNOwo+ID4+Cj4gPj4gLSAgICAgICBtZW1jcHlfdG9pbyhkc3QsIHNyYywgUEFHRV9TSVpFKTsK PiA+PiAtCj4gPj4gLSAgICAgICBrdW5tYXBfYXRvbWljKHNyYyk7Cj4gPj4gKyAgICAgICBmb3Ig KGkgPSAwOyBpIDwgZHN0X21lbS0+bnVtX3BhZ2VzOyArK2kpIHsKPiA+PiArICAgICAgICAgICAg ICAgZHN0X29wcy0+bWFwX2xvY2FsKGRzdF9pdGVyLCAmZHN0X21hcCwgaSk7Cj4gPj4gKyAgICAg ICAgICAgICAgIHNyY19vcHMtPm1hcF9sb2NhbChzcmNfaXRlciwgJnNyY19tYXAsIGkpOwo+ID4+ ICsKPiA+PiArICAgICAgICAgICAgICAgaWYgKCFzcmNfbWFwLmlzX2lvbWVtICYmICFkc3RfbWFw LmlzX2lvbWVtKSB7Cj4gPj4gKyAgICAgICAgICAgICAgICAgICAgICAgbWVtY3B5KGRzdF9tYXAu dmFkZHIsIHNyY19tYXAudmFkZHIsIFBBR0VfU0laRSk7Cj4gPj4gKyAgICAgICAgICAgICAgIH0g ZWxzZSBpZiAoIXNyY19tYXAuaXNfaW9tZW0pIHsKPiA+PiArICAgICAgICAgICAgICAgICAgICAg ICBkbWFfYnVmX21hcF9tZW1jcHlfdG8oJmRzdF9tYXAsIHNyY19tYXAudmFkZHIsCj4gPj4gKyAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFBBR0VfU0laRSk7Cj4g Pj4gKyAgICAgICAgICAgICAgIH0gZWxzZSBpZiAoIWRzdF9tYXAuaXNfaW9tZW0pIHsKPiA+PiAr ICAgICAgICAgICAgICAgICAgICAgICBtZW1jcHlfZnJvbWlvKGRzdF9tYXAudmFkZHIsIHNyY19t YXAudmFkZHJfaW9tZW0sCj4gPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBQQUdFX1NJWkUpOwo+ID4+ICsgICAgICAgICAgICAgICB9IGVsc2Ugewo+ID4+ICsgICAgICAg ICAgICAgICAgICAgICAgIGludCBqOwo+ID4+ICsgICAgICAgICAgICAgICAgICAgICAgIHUzMiBf X2lvbWVtICpzcmMgPSBzcmNfbWFwLnZhZGRyX2lvbWVtOwo+ID4+ICsgICAgICAgICAgICAgICAg ICAgICAgIHUzMiBfX2lvbWVtICpkc3QgPSBkc3RfbWFwLnZhZGRyX2lvbWVtOwo+ID4+Cj4gPj4g LSAgICAgICByZXR1cm4gMDsKPiA+PiArICAgICAgICAgICAgICAgICAgICAgICBmb3IgKGogPSAw OyBqIDwgKFBBR0VfU0laRSA+PiAyKTsgKytqKQo+ID4gSU1PIFBBR0VfU0laRSAvIHNpemVvZih1 MzIpIGlzIGVhc2llciB0byB1bmRlcnN0YW5kLgo+Cj4gT0ssIHdpbGwgZml4Lgo+Cj4KPiA+Cj4g Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpb3dyaXRlMzIoaW9yZWFkMzIoc3Jj KyspLCBkc3QrKyk7Cj4gPj4gKyAgICAgICAgICAgICAgIH0KPiA+PiArICAgICAgICAgICAgICAg aWYgKHNyY19vcHMtPnVubWFwX2xvY2FsKQo+ID4+ICsgICAgICAgICAgICAgICAgICAgICAgIHNy Y19vcHMtPnVubWFwX2xvY2FsKHNyY19pdGVyLCAmc3JjX21hcCk7Cj4gPj4gKyAgICAgICAgICAg ICAgIGlmIChkc3Rfb3BzLT51bm1hcF9sb2NhbCkKPiA+PiArICAgICAgICAgICAgICAgICAgICAg ICBkc3Rfb3BzLT51bm1hcF9sb2NhbChkc3RfaXRlciwgJmRzdF9tYXApOwo+ID4+ICsgICAgICAg fQo+ID4+ICAgfQo+ID4+ICtFWFBPUlRfU1lNQk9MKHR0bV9tb3ZlX21lbWNweSk7Cj4gPj4KPiA+ PiAgIGludCB0dG1fYm9fbW92ZV9tZW1jcHkoc3RydWN0IHR0bV9idWZmZXJfb2JqZWN0ICpibywK PiA+PiAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgdHRtX29wZXJhdGlvbl9jdHggKmN0 eCwKPiA+PiAtICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCB0dG1fcmVzb3VyY2UgKm5ld19t ZW0pCj4gPj4gKyAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgdHRtX3Jlc291cmNlICpkc3Rf bWVtKQo+ID4+ICAgewo+ID4+ICAgICAgICAgIHN0cnVjdCB0dG1fZGV2aWNlICpiZGV2ID0gYm8t PmJkZXY7Cj4gPj4gLSAgICAgICBzdHJ1Y3QgdHRtX3Jlc291cmNlX21hbmFnZXIgKm1hbiA9IHR0 bV9tYW5hZ2VyX3R5cGUoYmRldiwgbmV3X21lbS0+bWVtX3R5cGUpOwo+ID4+ICsgICAgICAgc3Ry dWN0IHR0bV9yZXNvdXJjZV9tYW5hZ2VyICpkc3RfbWFuID0KPiA+PiArICAgICAgICAgICAgICAg dHRtX21hbmFnZXJfdHlwZShiby0+YmRldiwgZHN0X21lbS0+bWVtX3R5cGUpOwo+ID4+ICAgICAg ICAgIHN0cnVjdCB0dG1fdHQgKnR0bSA9IGJvLT50dG07Cj4gPj4gLSAgICAgICBzdHJ1Y3QgdHRt X3Jlc291cmNlICpvbGRfbWVtID0gJmJvLT5tZW07Cj4gPj4gLSAgICAgICBzdHJ1Y3QgdHRtX3Jl c291cmNlIG9sZF9jb3B5ID0gKm9sZF9tZW07Cj4gPj4gLSAgICAgICB2b2lkICpvbGRfaW9tYXA7 Cj4gPj4gLSAgICAgICB2b2lkICpuZXdfaW9tYXA7Cj4gPj4gKyAgICAgICBzdHJ1Y3QgdHRtX3Jl c291cmNlICpzcmNfbWVtID0gJmJvLT5tZW07Cj4gPj4gKyAgICAgICBzdHJ1Y3QgdHRtX3Jlc291 cmNlX21hbmFnZXIgKnNyY19tYW4gPQo+ID4+ICsgICAgICAgICAgICAgICB0dG1fbWFuYWdlcl90 eXBlKGJkZXYsIHNyY19tZW0tPm1lbV90eXBlKTsKPiA+PiArICAgICAgIHN0cnVjdCB0dG1fcmVz b3VyY2Ugc3JjX2NvcHkgPSAqc3JjX21lbTsKPiA+PiArICAgICAgIHVuaW9uIHsKPiA+PiArICAg ICAgICAgICAgICAgc3RydWN0IHR0bV9rbWFwX2l0ZXJfdHQgdHQ7Cj4gPj4gKyAgICAgICAgICAg ICAgIHN0cnVjdCB0dG1fa21hcF9pdGVyX2xpbmVhcl9pbyBpbzsKPiA+PiArICAgICAgIH0gX2Rz dF9pdGVyLCBfc3JjX2l0ZXI7Cj4gPj4gKyAgICAgICBzdHJ1Y3QgdHRtX2ttYXBfaXRlciAqZHN0 X2l0ZXIsICpzcmNfaXRlcjsKPiA+PiAgICAgICAgICBpbnQgcmV0Owo+ID4+IC0gICAgICAgdW5z aWduZWQgbG9uZyBpOwo+ID4+Cj4gPj4gLSAgICAgICByZXQgPSB0dG1fYm9fd2FpdF9jdHgoYm8s IGN0eCk7Cj4gPj4gLSAgICAgICBpZiAocmV0KQo+ID4+IC0gICAgICAgICAgICAgICByZXR1cm4g cmV0Owo+ID4+IC0KPiA+PiAtICAgICAgIHJldCA9IHR0bV9yZXNvdXJjZV9pb3JlbWFwKGJkZXYs IG9sZF9tZW0sICZvbGRfaW9tYXApOwo+ID4+IC0gICAgICAgaWYgKHJldCkKPiA+PiAtICAgICAg ICAgICAgICAgcmV0dXJuIHJldDsKPiA+PiAtICAgICAgIHJldCA9IHR0bV9yZXNvdXJjZV9pb3Jl bWFwKGJkZXYsIG5ld19tZW0sICZuZXdfaW9tYXApOwo+ID4+IC0gICAgICAgaWYgKHJldCkKPiA+ PiAtICAgICAgICAgICAgICAgZ290byBvdXQ7Cj4gPj4gLQo+ID4+IC0gICAgICAgLyoKPiA+PiAt ICAgICAgICAqIFNpbmdsZSBUVE0gbW92ZS4gTk9QLgo+ID4+IC0gICAgICAgICovCj4gPj4gLSAg ICAgICBpZiAob2xkX2lvbWFwID09IE5VTEwgJiYgbmV3X2lvbWFwID09IE5VTEwpCj4gPj4gLSAg ICAgICAgICAgICAgIGdvdG8gb3V0MjsKPiA+PiAtCj4gPj4gLSAgICAgICAvKgo+ID4+IC0gICAg ICAgICogRG9uJ3QgbW92ZSBub25leGlzdGVudCBkYXRhLiBDbGVhciBkZXN0aW5hdGlvbiBpbnN0 ZWFkLgo+ID4+IC0gICAgICAgICovCj4gPj4gLSAgICAgICBpZiAob2xkX2lvbWFwID09IE5VTEwg JiYKPiA+PiAtICAgICAgICAgICAodHRtID09IE5VTEwgfHwgKCF0dG1fdHRfaXNfcG9wdWxhdGVk KHR0bSkgJiYKPiA+PiAtICAgICAgICAgICAgICAgICAgICAgICAgICAgICEodHRtLT5wYWdlX2Zs YWdzICYgVFRNX1BBR0VfRkxBR19TV0FQUEVEKSkpKSB7Cj4gPj4gLSAgICAgICAgICAgICAgIG1l bXNldF9pbyhuZXdfaW9tYXAsIDAsIG5ld19tZW0tPm51bV9wYWdlcypQQUdFX1NJWkUpOwo+ID4+ IC0gICAgICAgICAgICAgICBnb3RvIG91dDI7Cj4gPj4gLSAgICAgICB9Cj4gPj4gLQo+ID4+IC0g ICAgICAgLyoKPiA+PiAtICAgICAgICAqIFRUTSBtaWdodCBiZSBudWxsIGZvciBtb3ZlcyB3aXRo aW4gdGhlIHNhbWUgcmVnaW9uLgo+ID4+IC0gICAgICAgICovCj4gPj4gLSAgICAgICBpZiAodHRt KSB7Cj4gPj4gKyAgICAgICBpZiAodHRtICYmICgodHRtLT5wYWdlX2ZsYWdzICYgVFRNX1BBR0Vf RkxBR19TV0FQUEVEKSB8fAo+ID4+ICsgICAgICAgICAgICAgICAgICAgZHN0X21hbi0+dXNlX3R0 KSkgewo+ID4+ICAgICAgICAgICAgICAgICAgcmV0ID0gdHRtX3R0X3BvcHVsYXRlKGJkZXYsIHR0 bSwgY3R4KTsKPiA+PiAgICAgICAgICAgICAgICAgIGlmIChyZXQpCj4gPj4gLSAgICAgICAgICAg ICAgICAgICAgICAgZ290byBvdXQxOwo+ID4+ICsgICAgICAgICAgICAgICAgICAgICAgIHJldHVy biByZXQ7Cj4gPj4gICAgICAgICAgfQo+ID4+Cj4gPj4gLSAgICAgICBmb3IgKGkgPSAwOyBpIDwg bmV3X21lbS0+bnVtX3BhZ2VzOyArK2kpIHsKPiA+PiAtICAgICAgICAgICAgICAgaWYgKG9sZF9p b21hcCA9PSBOVUxMKSB7Cj4gPj4gLSAgICAgICAgICAgICAgICAgICAgICAgcGdwcm90X3QgcHJv dCA9IHR0bV9pb19wcm90KGJvLCBvbGRfbWVtLCBQQUdFX0tFUk5FTCk7Cj4gPj4gLSAgICAgICAg ICAgICAgICAgICAgICAgcmV0ID0gdHRtX2NvcHlfdHRtX2lvX3BhZ2UodHRtLCBuZXdfaW9tYXAs IGksCj4gPj4gLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgcHJvdCk7Cj4gPj4gLSAgICAgICAgICAgICAgIH0gZWxzZSBpZiAobmV3X2lvbWFwID09IE5V TEwpIHsKPiA+PiAtICAgICAgICAgICAgICAgICAgICAgICBwZ3Byb3RfdCBwcm90ID0gdHRtX2lv X3Byb3QoYm8sIG5ld19tZW0sIFBBR0VfS0VSTkVMKTsKPiA+PiAtICAgICAgICAgICAgICAgICAg ICAgICByZXQgPSB0dG1fY29weV9pb190dG1fcGFnZSh0dG0sIG9sZF9pb21hcCwgaSwKPiA+PiAt ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwcm90KTsK PiA+PiAtICAgICAgICAgICAgICAgfSBlbHNlIHsKPiA+PiAtICAgICAgICAgICAgICAgICAgICAg ICByZXQgPSB0dG1fY29weV9pb19wYWdlKG5ld19pb21hcCwgb2xkX2lvbWFwLCBpKTsKPiA+PiAt ICAgICAgICAgICAgICAgfQo+ID4+IC0gICAgICAgICAgICAgICBpZiAocmV0KQo+ID4+IC0gICAg ICAgICAgICAgICAgICAgICAgIGdvdG8gb3V0MTsKPiA+PiArICAgICAgIGRzdF9pdGVyID0gdHRt X2ttYXBfaXRlcl9saW5lYXJfaW9faW5pdCgmX2RzdF9pdGVyLmlvLCBiZGV2LCBkc3RfbWVtKTsK PiA+PiArICAgICAgIGlmIChQVFJfRVJSKGRzdF9pdGVyKSA9PSAtRUlOVkFMICYmIGRzdF9tYW4t PnVzZV90dCkKPiA+PiArICAgICAgICAgICAgICAgZHN0X2l0ZXIgPSB0dG1fa21hcF9pdGVyX3R0 X2luaXQoJl9kc3RfaXRlci50dCwgYm8tPnR0bSk7Cj4gPj4gKyAgICAgICBpZiAoSVNfRVJSKGRz dF9pdGVyKSkKPiA+PiArICAgICAgICAgICAgICAgcmV0dXJuIFBUUl9FUlIoZHN0X2l0ZXIpOwo+ ID4+ICsKPiA+PiArICAgICAgIHNyY19pdGVyID0gdHRtX2ttYXBfaXRlcl9saW5lYXJfaW9faW5p dCgmX3NyY19pdGVyLmlvLCBiZGV2LCBzcmNfbWVtKTsKPiA+PiArICAgICAgIGlmIChQVFJfRVJS KHNyY19pdGVyKSA9PSAtRUlOVkFMICYmIHNyY19tYW4tPnVzZV90dCkKPiA+PiArICAgICAgICAg ICAgICAgc3JjX2l0ZXIgPSB0dG1fa21hcF9pdGVyX3R0X2luaXQoJl9zcmNfaXRlci50dCwgYm8t PnR0bSk7Cj4gPj4gKyAgICAgICBpZiAoSVNfRVJSKHNyY19pdGVyKSkgewo+ID4+ICsgICAgICAg ICAgICAgICByZXQgPSBQVFJfRVJSKHNyY19pdGVyKTsKPiA+PiArICAgICAgICAgICAgICAgZ290 byBvdXRfc3JjX2l0ZXI7Cj4gPj4gICAgICAgICAgfQo+ID4+IC0gICAgICAgbWIoKTsKPiA+PiAt b3V0MjoKPiA+PiAtICAgICAgIG9sZF9jb3B5ID0gKm9sZF9tZW07Cj4gPj4KPiA+PiAtICAgICAg IHR0bV9ib19hc3NpZ25fbWVtKGJvLCBuZXdfbWVtKTsKPiA+PiAtCj4gPj4gLSAgICAgICBpZiAo IW1hbi0+dXNlX3R0KQo+ID4+IC0gICAgICAgICAgICAgICB0dG1fYm9fdHRfZGVzdHJveShibyk7 Cj4gPj4gKyAgICAgICB0dG1fbW92ZV9tZW1jcHkoYm8sIGRzdF9tZW0sIGRzdF9pdGVyLCBzcmNf aXRlcik7Cj4gPj4gKyAgICAgICBzcmNfY29weSA9ICpzcmNfbWVtOwo+ID4+ICsgICAgICAgdHRt X2JvX21vdmVfc3luY19jbGVhbnVwKGJvLCBkc3RfbWVtKTsKPiA+Pgo+ID4+IC1vdXQxOgo+ID4+ IC0gICAgICAgdHRtX3Jlc291cmNlX2lvdW5tYXAoYmRldiwgb2xkX21lbSwgbmV3X2lvbWFwKTsK PiA+PiAtb3V0Ogo+ID4+IC0gICAgICAgdHRtX3Jlc291cmNlX2lvdW5tYXAoYmRldiwgJm9sZF9j b3B5LCBvbGRfaW9tYXApOwo+ID4+ICsgICAgICAgaWYgKCFzcmNfaXRlci0+b3BzLT5tYXBzX3R0 KQo+ID4+ICsgICAgICAgICAgICAgICB0dG1fa21hcF9pdGVyX2xpbmVhcl9pb19maW5pKCZfc3Jj X2l0ZXIuaW8sIGJkZXYsICZzcmNfY29weSk7Cj4gPj4gK291dF9zcmNfaXRlcjoKPiA+PiArICAg ICAgIGlmICghZHN0X2l0ZXItPm9wcy0+bWFwc190dCkKPiA+PiArICAgICAgICAgICAgICAgdHRt X2ttYXBfaXRlcl9saW5lYXJfaW9fZmluaSgmX2RzdF9pdGVyLmlvLCBiZGV2LCBkc3RfbWVtKTsK PiA+Pgo+ID4+IC0gICAgICAgLyoKPiA+PiAtICAgICAgICAqIE9uIGVycm9yLCBrZWVwIHRoZSBt bSBub2RlIQo+ID4+IC0gICAgICAgICovCj4gPj4gLSAgICAgICBpZiAoIXJldCkKPiA+PiAtICAg ICAgICAgICAgICAgdHRtX3Jlc291cmNlX2ZyZWUoYm8sICZvbGRfY29weSk7Cj4gPj4gICAgICAg ICAgcmV0dXJuIHJldDsKPiA+PiAgIH0KPiA+PiAgIEVYUE9SVF9TWU1CT0wodHRtX2JvX21vdmVf bWVtY3B5KTsKPiA+PiBAQCAtMzM2LDI3ICsyNzIsNyBAQCBwZ3Byb3RfdCB0dG1faW9fcHJvdChz dHJ1Y3QgdHRtX2J1ZmZlcl9vYmplY3QgKmJvLCBzdHJ1Y3QgdHRtX3Jlc291cmNlICpyZXMsCj4g Pj4gICAgICAgICAgbWFuID0gdHRtX21hbmFnZXJfdHlwZShiby0+YmRldiwgcmVzLT5tZW1fdHlw ZSk7Cj4gPj4gICAgICAgICAgY2FjaGluZyA9IG1hbi0+dXNlX3R0ID8gYm8tPnR0bS0+Y2FjaGlu ZyA6IHJlcy0+YnVzLmNhY2hpbmc7Cj4gPj4KPiA+PiAtICAgICAgIC8qIENhY2hlZCBtYXBwaW5n cyBuZWVkIG5vIGFkanVzdG1lbnQgKi8KPiA+PiAtICAgICAgIGlmIChjYWNoaW5nID09IHR0bV9j YWNoZWQpCj4gPj4gLSAgICAgICAgICAgICAgIHJldHVybiB0bXA7Cj4gPj4gLQo+ID4+IC0jaWYg ZGVmaW5lZChfX2kzODZfXykgfHwgZGVmaW5lZChfX3g4Nl82NF9fKQo+ID4+IC0gICAgICAgaWYg KGNhY2hpbmcgPT0gdHRtX3dyaXRlX2NvbWJpbmVkKQo+ID4+IC0gICAgICAgICAgICAgICB0bXAg PSBwZ3Byb3Rfd3JpdGVjb21iaW5lKHRtcCk7Cj4gPj4gLSAgICAgICBlbHNlIGlmIChib290X2Nw dV9kYXRhLng4NiA+IDMpCj4gPj4gLSAgICAgICAgICAgICAgIHRtcCA9IHBncHJvdF9ub25jYWNo ZWQodG1wKTsKPiA+PiAtI2VuZGlmCj4gPj4gLSNpZiBkZWZpbmVkKF9faWE2NF9fKSB8fCBkZWZp bmVkKF9fYXJtX18pIHx8IGRlZmluZWQoX19hYXJjaDY0X18pIHx8IFwKPiA+PiAtICAgIGRlZmlu ZWQoX19wb3dlcnBjX18pIHx8IGRlZmluZWQoX19taXBzX18pCj4gPj4gLSAgICAgICBpZiAoY2Fj aGluZyA9PSB0dG1fd3JpdGVfY29tYmluZWQpCj4gPj4gLSAgICAgICAgICAgICAgIHRtcCA9IHBn cHJvdF93cml0ZWNvbWJpbmUodG1wKTsKPiA+PiAtICAgICAgIGVsc2UKPiA+PiAtICAgICAgICAg ICAgICAgdG1wID0gcGdwcm90X25vbmNhY2hlZCh0bXApOwo+ID4+IC0jZW5kaWYKPiA+PiAtI2lm IGRlZmluZWQoX19zcGFyY19fKQo+ID4+IC0gICAgICAgdG1wID0gcGdwcm90X25vbmNhY2hlZCh0 bXApOwo+ID4+IC0jZW5kaWYKPiA+PiAtICAgICAgIHJldHVybiB0bXA7Cj4gPj4gKyAgICAgICBy ZXR1cm4gdHRtX3Byb3RfZnJvbV9jYWNoaW5nKGNhY2hpbmcsIHRtcCk7Cj4gPj4gICB9Cj4gPj4g ICBFWFBPUlRfU1lNQk9MKHR0bV9pb19wcm90KTsKPiA+Pgo+ID4+IGRpZmYgLS1naXQgYS9kcml2 ZXJzL2dwdS9kcm0vdHRtL3R0bV9tb2R1bGUuYyBiL2RyaXZlcnMvZ3B1L2RybS90dG0vdHRtX21v ZHVsZS5jCj4gPj4gaW5kZXggNTZiMGVmZGJhMWE5Li45OTdjNDU4ZjY4YTkgMTAwNjQ0Cj4gPj4g LS0tIGEvZHJpdmVycy9ncHUvZHJtL3R0bS90dG1fbW9kdWxlLmMKPiA+PiArKysgYi9kcml2ZXJz L2dwdS9kcm0vdHRtL3R0bV9tb2R1bGUuYwo+ID4+IEBAIC0zMSwxMiArMzEsNDcgQEAKPiA+PiAg ICAqLwo+ID4+ICAgI2luY2x1ZGUgPGxpbnV4L21vZHVsZS5oPgo+ID4+ICAgI2luY2x1ZGUgPGxp bnV4L2RldmljZS5oPgo+ID4+ICsjaW5jbHVkZSA8bGludXgvcGd0YWJsZS5oPgo+ID4+ICAgI2lu Y2x1ZGUgPGxpbnV4L3NjaGVkLmg+Cj4gPj4gICAjaW5jbHVkZSA8bGludXgvZGVidWdmcy5oPgo+ ID4+ICAgI2luY2x1ZGUgPGRybS9kcm1fc3lzZnMuaD4KPiA+PiArI2luY2x1ZGUgPGRybS90dG0v dHRtX2NhY2hpbmcuaD4KPiA+Pgo+ID4+ICAgI2luY2x1ZGUgInR0bV9tb2R1bGUuaCIKPiA+Pgo+ ID4+ICsvKioKPiA+PiArICogdHRtX3Byb3RfZnJvbV9jYWNoaW5nIC0gTW9kaWZ5IHRoZSBwYWdl IHByb3RlY3Rpb24gYWNjb3JkaW5nIHRvIHRoZQo+ID4+ICsgKiB0dG0gY2FjaW5nIG1vZGUKPiA+ PiArICogQGNhY2hpbmc6IFRoZSB0dG0gY2FjaGluZyBtb2RlCj4gPj4gKyAqIEB0bXA6IFRoZSBv cmlnaW5hbCBwYWdlIHByb3RlY3Rpb24KPiA+PiArICoKPiA+PiArICogUmV0dXJuOiBUaGUgbW9k aWZpZWQgcGFnZSBwcm90ZWN0aW9uCj4gPj4gKyAqLwo+ID4+ICtwZ3Byb3RfdCB0dG1fcHJvdF9m cm9tX2NhY2hpbmcoZW51bSB0dG1fY2FjaGluZyBjYWNoaW5nLCBwZ3Byb3RfdCB0bXApCj4gPj4g K3sKPiA+PiArICAgICAgIC8qIENhY2hlZCBtYXBwaW5ncyBuZWVkIG5vIGFkanVzdG1lbnQgKi8K PiA+PiArICAgICAgIGlmIChjYWNoaW5nID09IHR0bV9jYWNoZWQpCj4gPj4gKyAgICAgICAgICAg ICAgIHJldHVybiB0bXA7Cj4gPj4gKwo+ID4+ICsjaWYgZGVmaW5lZChfX2kzODZfXykgfHwgZGVm aW5lZChfX3g4Nl82NF9fKQo+ID4+ICsgICAgICAgaWYgKGNhY2hpbmcgPT0gdHRtX3dyaXRlX2Nv bWJpbmVkKQo+ID4+ICsgICAgICAgICAgICAgICB0bXAgPSBwZ3Byb3Rfd3JpdGVjb21iaW5lKHRt cCk7Cj4gPj4gKyAgICAgICBlbHNlIGlmIChib290X2NwdV9kYXRhLng4NiA+IDMpCj4gPj4gKyAg ICAgICAgICAgICAgIHRtcCA9IHBncHJvdF9ub25jYWNoZWQodG1wKTsKPiA+PiArI2VuZGlmCj4g Pj4gKyNpZiBkZWZpbmVkKF9faWE2NF9fKSB8fCBkZWZpbmVkKF9fYXJtX18pIHx8IGRlZmluZWQo X19hYXJjaDY0X18pIHx8IFwKPiA+PiArICAgICAgIGRlZmluZWQoX19wb3dlcnBjX18pIHx8IGRl ZmluZWQoX19taXBzX18pCj4gPj4gKyAgICAgICBpZiAoY2FjaGluZyA9PSB0dG1fd3JpdGVfY29t YmluZWQpCj4gPj4gKyAgICAgICAgICAgICAgIHRtcCA9IHBncHJvdF93cml0ZWNvbWJpbmUodG1w KTsKPiA+PiArICAgICAgIGVsc2UKPiA+PiArICAgICAgICAgICAgICAgdG1wID0gcGdwcm90X25v bmNhY2hlZCh0bXApOwo+ID4+ICsjZW5kaWYKPiA+PiArI2lmIGRlZmluZWQoX19zcGFyY19fKQo+ ID4+ICsgICAgICAgdG1wID0gcGdwcm90X25vbmNhY2hlZCh0bXApOwo+ID4+ICsjZW5kaWYKPiA+ PiArICAgICAgIHJldHVybiB0bXA7Cj4gPj4gK30KPiA+PiArCj4gPj4gICBzdHJ1Y3QgZGVudHJ5 ICp0dG1fZGVidWdmc19yb290Owo+ID4+Cj4gPj4gICBzdGF0aWMgaW50IF9faW5pdCB0dG1faW5p dCh2b2lkKQo+ID4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vdHRtL3R0bV9yZXNvdXJj ZS5jIGIvZHJpdmVycy9ncHUvZHJtL3R0bS90dG1fcmVzb3VyY2UuYwo+ID4+IGluZGV4IDU5ZTJi NzE1N2U0MS4uZTA1YWU3ZTNkNDc3IDEwMDY0NAo+ID4+IC0tLSBhL2RyaXZlcnMvZ3B1L2RybS90 dG0vdHRtX3Jlc291cmNlLmMKPiA+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vdHRtL3R0bV9yZXNv dXJjZS5jCj4gPj4gQEAgLTIyLDYgKzIyLDEwIEBACj4gPj4gICAgKiBBdXRob3JzOiBDaHJpc3Rp YW4gS8O2bmlnCj4gPj4gICAgKi8KPiA+Pgo+ID4+ICsjaW5jbHVkZSA8bGludXgvZG1hLWJ1Zi1t YXAuaD4KPiA+PiArI2luY2x1ZGUgPGxpbnV4L2lvLW1hcHBpbmcuaD4KPiA+PiArI2luY2x1ZGUg PGxpbnV4L3NjYXR0ZXJsaXN0Lmg+Cj4gPj4gKwo+ID4+ICAgI2luY2x1ZGUgPGRybS90dG0vdHRt X3Jlc291cmNlLmg+Cj4gPj4gICAjaW5jbHVkZSA8ZHJtL3R0bS90dG1fYm9fZHJpdmVyLmg+Cj4g Pj4KPiA+PiBAQCAtMTQ3LDMgKzE1MSwxNjUgQEAgdm9pZCB0dG1fcmVzb3VyY2VfbWFuYWdlcl9k ZWJ1ZyhzdHJ1Y3QgdHRtX3Jlc291cmNlX21hbmFnZXIgKm1hbiwKPiA+PiAgICAgICAgICAgICAg ICAgIG1hbi0+ZnVuYy0+ZGVidWcobWFuLCBwKTsKPiA+PiAgIH0KPiA+PiAgIEVYUE9SVF9TWU1C T0wodHRtX3Jlc291cmNlX21hbmFnZXJfZGVidWcpOwo+ID4+ICsKPiA+PiArc3RhdGljIHZvaWQg dHRtX2ttYXBfaXRlcl9pb21hcF9tYXBfbG9jYWwoc3RydWN0IHR0bV9rbWFwX2l0ZXIgKml0ZXIs Cj4gPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGRt YV9idWZfbWFwICpkbWFwLAo+ID4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIHBnb2ZmX3QgaSkKPiA+PiArewo+ID4+ICsgICAgICAgc3RydWN0IHR0bV9rbWFwX2l0 ZXJfaW9tYXAgKml0ZXJfaW8gPQo+ID4+ICsgICAgICAgICAgICAgICBjb250YWluZXJfb2YoaXRl ciwgdHlwZW9mKCppdGVyX2lvKSwgYmFzZSk7Cj4gPj4gKyAgICAgICB2b2lkIF9faW9tZW0gKmFk ZHI7Cj4gPj4gKwo+ID4+ICtyZXRyeToKPiA+PiArICAgICAgIHdoaWxlIChpID49IGl0ZXJfaW8t PmNhY2hlLmVuZCkgewo+ID4+ICsgICAgICAgICAgICAgICBpdGVyX2lvLT5jYWNoZS5zZyA9IGl0 ZXJfaW8tPmNhY2hlLnNnID8KPiA+PiArICAgICAgICAgICAgICAgICAgICAgICBzZ19uZXh0KGl0 ZXJfaW8tPmNhY2hlLnNnKSA6IGl0ZXJfaW8tPnN0LT5zZ2w7Cj4gPj4gKyAgICAgICAgICAgICAg IGl0ZXJfaW8tPmNhY2hlLmkgPSBpdGVyX2lvLT5jYWNoZS5lbmQ7Cj4gPj4gKyAgICAgICAgICAg ICAgIGl0ZXJfaW8tPmNhY2hlLmVuZCArPSBzZ19kbWFfbGVuKGl0ZXJfaW8tPmNhY2hlLnNnKSA+ Pgo+ID4+ICsgICAgICAgICAgICAgICAgICAgICAgIFBBR0VfU0hJRlQ7Cj4gPj4gKyAgICAgICAg ICAgICAgIGl0ZXJfaW8tPmNhY2hlLm9mZnMgPSBzZ19kbWFfYWRkcmVzcyhpdGVyX2lvLT5jYWNo ZS5zZykgLQo+ID4+ICsgICAgICAgICAgICAgICAgICAgICAgIGl0ZXJfaW8tPnN0YXJ0Owo+ID4+ ICsgICAgICAgfQo+ID4+ICsKPiA+PiArICAgICAgIGlmIChpIDwgaXRlcl9pby0+Y2FjaGUuaSkg ewo+ID4+ICsgICAgICAgICAgICAgICBpdGVyX2lvLT5jYWNoZS5lbmQgPSAwOwo+ID4+ICsgICAg ICAgICAgICAgICBpdGVyX2lvLT5jYWNoZS5zZyA9IE5VTEw7Cj4gPj4gKyAgICAgICAgICAgICAg IGdvdG8gcmV0cnk7Cj4gPj4gKyAgICAgICB9Cj4gPj4gKwo+ID4+ICsgICAgICAgYWRkciA9IGlv X21hcHBpbmdfbWFwX2xvY2FsX3djKGl0ZXJfaW8tPmlvbWFwLCBpdGVyX2lvLT5jYWNoZS5vZmZz ICsKPiA+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAoKChyZXNvdXJj ZV9zaXplX3QpaSAtIGl0ZXJfaW8tPmNhY2hlLmkpCj4gPj4gKyAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgIDw8IFBBR0VfU0hJRlQpKTsKPiA+PiArICAgICAgIGRtYV9idWZf bWFwX3NldF92YWRkcl9pb21lbShkbWFwLCBhZGRyKTsKPiA+PiArfQo+ID4+ICsKPiA+PiArc3Rh dGljIHZvaWQgdHRtX2ttYXBfaXRlcl9pb21hcF91bm1hcF9sb2NhbChzdHJ1Y3QgdHRtX2ttYXBf aXRlciAqaXRlciwKPiA+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIHN0cnVjdCBkbWFfYnVmX21hcCAqbWFwKQo+ID4+ICt7Cj4gPj4gKyAgICAgICBpb19tYXBw aW5nX3VubWFwX2xvY2FsKG1hcC0+dmFkZHJfaW9tZW0pOwo+ID4+ICt9Cj4gPj4gKwo+ID4+ICtz dGF0aWMgY29uc3Qgc3RydWN0IHR0bV9rbWFwX2l0ZXJfb3BzIHR0bV9rbWFwX2l0ZXJfaW9fb3Bz ID0gewo+ID4+ICsgICAgICAgLm1hcF9sb2NhbCA9ICB0dG1fa21hcF9pdGVyX2lvbWFwX21hcF9s b2NhbCwKPiA+PiArICAgICAgIC51bm1hcF9sb2NhbCA9IHR0bV9rbWFwX2l0ZXJfaW9tYXBfdW5t YXBfbG9jYWwsCj4gPj4gKyAgICAgICAubWFwc190dCA9IGZhbHNlLAo+ID4+ICt9Owo+ID4+ICsK PiA+PiArLyoqCj4gPj4gKyAqIHR0bV9rbWFwX2l0ZXJfaW9tYXBfaW5pdCAtIEluaXRpYWxpemUg YSBzdHJ1Y3QgdHRtX2ttYXBfaXRlcl9pb21hcAo+ID4+ICsgKiBAaXRlcl9pbzogVGhlIHN0cnVj dCB0dG1fa21hcF9pdGVyX2lvbWFwIHRvIGluaXRpYWxpemUuCj4gPj4gKyAqIEBpb21hcDogVGhl IHN0cnVjdCBpb19tYXBwaW5nIHJlcHJlc2VudGluZyB0aGUgdW5kZXJseWluZyBsaW5lYXIgaW9f bWVtb3J5Lgo+ID4+ICsgKiBAc3Q6IHNnX3RhYmxlIGludG8gQGlvbWFwLCByZXByZXNlbnRpbmcg dGhlIG1lbW9yeSBvZiB0aGUgc3RydWN0Cj4gPj4gKyAqIHR0bV9yZXNvdXJjZS4KPiA+PiArICog QHN0YXJ0OiBPZmZzZXQgdGhhdCBuZWVkcyB0byBiZSBzdWJ0cmFjdGVkIGZyb20gQHN0IHRvIG1h a2UKPiA+PiArICogc2dfZG1hX2FkZHJlc3Moc3QtPnNnbCkgLSBAc3RhcnQgPT0gMCBmb3IgQGlv bWFwIHN0YXJ0Lgo+ID4+ICsgKgo+ID4+ICsgKiBSZXR1cm46IFBvaW50ZXIgdG8gdGhlIGVtYmVk ZGVkIHN0cnVjdCB0dG1fa21hcF9pdGVyLgo+ID4+ICsgKi8KPiA+PiArc3RydWN0IHR0bV9rbWFw X2l0ZXIgKgo+ID4+ICt0dG1fa21hcF9pdGVyX2lvbWFwX2luaXQoc3RydWN0IHR0bV9rbWFwX2l0 ZXJfaW9tYXAgKml0ZXJfaW8sCj4gPj4gKyAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBp b19tYXBwaW5nICppb21hcCwKPiA+PiArICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IHNn X3RhYmxlICpzdCwKPiA+PiArICAgICAgICAgICAgICAgICAgICAgICAgcmVzb3VyY2Vfc2l6ZV90 IHN0YXJ0KQo+ID4+ICt7Cj4gPj4gKyAgICAgICBpdGVyX2lvLT5iYXNlLm9wcyA9ICZ0dG1fa21h cF9pdGVyX2lvX29wczsKPiA+PiArICAgICAgIGl0ZXJfaW8tPmlvbWFwID0gaW9tYXA7Cj4gPj4g KyAgICAgICBpdGVyX2lvLT5zdCA9IHN0Owo+ID4+ICsgICAgICAgaXRlcl9pby0+c3RhcnQgPSBz dGFydDsKPiA+PiArICAgICAgIG1lbXNldCgmaXRlcl9pby0+Y2FjaGUsIDAsIHNpemVvZihpdGVy X2lvLT5jYWNoZSkpOwo+ID4+ICsKPiA+PiArICAgICAgIHJldHVybiAmaXRlcl9pby0+YmFzZTsK PiA+PiArfQo+ID4+ICtFWFBPUlRfU1lNQk9MKHR0bV9rbWFwX2l0ZXJfaW9tYXBfaW5pdCk7Cj4g Pj4gKwo+ID4+ICsvKioKPiA+PiArICogRE9DOiBMaW5lYXIgaW8gaXRlcmF0b3IKPiA+PiArICoK PiA+PiArICogVGhpcyBjb2RlIHNob3VsZCBkaWUgaW4gdGhlIG5vdCB0b28gbmVhciBmdXR1cmUu IEJlc3Qgd291bGQgYmUgaWYgd2UgY291bGQKPiA+PiArICogbWFrZSBpby1tYXBwaW5nIHVzZSBt ZW1yZW1hcCBmb3IgYWxsIGlvIG1lbW9yeSwgYW5kIGhhdmUgbWVtcmVtYXAKPiA+PiArICogaW1w bGVtZW50IGEga21hcF9sb2NhbCBmdW5jdGlvbmFsaXR5LiBXZSBjb3VsZCB0aGVuIHN0cmlwIGEg aHVnZSBhbW91bnQgb2YKPiA+PiArICogY29kZS4gVGhlc2UgbGluZWFyIGlvIGl0ZXJhdG9ycyBh cmUgaW1wbGVtZW50ZWQgdG8gbWltaWMgb2xkIGZ1bmN0aW9uYWxpdHksCj4gPj4gKyAqIGFuZCB0 aGV5IGRvbid0IHVzZSBrbWFwX2xvY2FsIHNlbWFudGljcyBhdCBhbGwgaW50ZXJuYWxseS4gUmF0 aGVyIGlvcmVtYXAgb3IKPiA+PiArICogZnJpZW5kcywgYW5kIGF0IGxlYXN0IG9uIDMyLWJpdCB0 aGV5IGFkZCBnbG9iYWwgVExCIGZsdXNoZXMgYW5kIHBvaW50cwo+ID4+ICsgKiBvZiBmYWlsdXJl Lgo+ID4+ICsgKi8KPiA+PiArCj4gPj4gK3N0YXRpYyB2b2lkIHR0bV9rbWFwX2l0ZXJfbGluZWFy X2lvX21hcF9sb2NhbChzdHJ1Y3QgdHRtX2ttYXBfaXRlciAqaXRlciwKPiA+PiArICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGRtYV9idWZfbWFwICpk bWFwLAo+ID4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBw Z29mZl90IGkpCj4gPj4gK3sKPiA+PiArICAgICAgIHN0cnVjdCB0dG1fa21hcF9pdGVyX2xpbmVh cl9pbyAqaXRlcl9pbyA9Cj4gPj4gKyAgICAgICAgICAgICAgIGNvbnRhaW5lcl9vZihpdGVyLCB0 eXBlb2YoKml0ZXJfaW8pLCBiYXNlKTsKPiA+PiArCj4gPj4gKyAgICAgICAqZG1hcCA9IGl0ZXJf aW8tPmRtYXA7Cj4gPj4gKyAgICAgICBkbWFfYnVmX21hcF9pbmNyKGRtYXAsIGkgKiBQQUdFX1NJ WkUpOwo+ID4+ICt9Cj4gPj4gKwo+ID4+ICtzdGF0aWMgY29uc3Qgc3RydWN0IHR0bV9rbWFwX2l0 ZXJfb3BzIHR0bV9rbWFwX2l0ZXJfbGluZWFyX2lvX29wcyA9IHsKPiA+PiArICAgICAgIC5tYXBf bG9jYWwgPSAgdHRtX2ttYXBfaXRlcl9saW5lYXJfaW9fbWFwX2xvY2FsLAo+ID4+ICsgICAgICAg Lm1hcHNfdHQgPSBmYWxzZSwKPiA+PiArfTsKPiA+PiArCj4gPj4gK3N0cnVjdCB0dG1fa21hcF9p dGVyICoKPiA+PiArdHRtX2ttYXBfaXRlcl9saW5lYXJfaW9faW5pdChzdHJ1Y3QgdHRtX2ttYXBf aXRlcl9saW5lYXJfaW8gKml0ZXJfaW8sCj4gPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAg ICBzdHJ1Y3QgdHRtX2RldmljZSAqYmRldiwKPiA+PiArICAgICAgICAgICAgICAgICAgICAgICAg ICAgIHN0cnVjdCB0dG1fcmVzb3VyY2UgKm1lbSkKPiA+PiArewo+ID4+ICsgICAgICAgaW50IHJl dDsKPiA+PiArCj4gPj4gKyAgICAgICByZXQgPSB0dG1fbWVtX2lvX3Jlc2VydmUoYmRldiwgbWVt KTsKPiA+PiArICAgICAgIGlmIChyZXQpCj4gPj4gKyAgICAgICAgICAgICAgIGdvdG8gb3V0X2Vy cjsKPiA+PiArICAgICAgIGlmICghbWVtLT5idXMuaXNfaW9tZW0pIHsKPiA+PiArICAgICAgICAg ICAgICAgcmV0ID0gLUVJTlZBTDsKPiA+PiArICAgICAgICAgICAgICAgZ290byBvdXRfaW9fZnJl ZTsKPiA+PiArICAgICAgIH0KPiA+PiArCj4gPj4gKyAgICAgICBpZiAobWVtLT5idXMuYWRkcikg ewo+ID4+ICsgICAgICAgICAgICAgICBkbWFfYnVmX21hcF9zZXRfdmFkZHIoJml0ZXJfaW8tPmRt YXAsIG1lbS0+YnVzLmFkZHIpOwo+ID4+ICsgICAgICAgICAgICAgICBpdGVyX2lvLT5uZWVkc191 bm1hcCA9IGZhbHNlOwo+ID4+ICsgICAgICAgfSBlbHNlIHsKPiA+PiArICAgICAgICAgICAgICAg c2l6ZV90IGJ1c19zaXplID0gKHNpemVfdCltZW0tPm51bV9wYWdlcyA8PCBQQUdFX1NISUZUOwo+ ID4+ICsKPiA+PiArICAgICAgICAgICAgICAgaXRlcl9pby0+bmVlZHNfdW5tYXAgPSB0cnVlOwo+ ID4+ICsgICAgICAgICAgICAgICBpZiAobWVtLT5idXMuY2FjaGluZyA9PSB0dG1fd3JpdGVfY29t YmluZWQpCj4gPj4gKyAgICAgICAgICAgICAgICAgICAgICAgZG1hX2J1Zl9tYXBfc2V0X3ZhZGRy X2lvbWVtKCZpdGVyX2lvLT5kbWFwLAo+ID4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBpb3JlbWFwX3djKG1lbS0+YnVzLm9mZnNldCwKPiA+PiAr ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICBidXNfc2l6ZSkpOwo+ID4+ICsgICAgICAgICAgICAgICBlbHNlIGlmIChtZW0tPmJ1cy5j YWNoaW5nID09IHR0bV9jYWNoZWQpCj4gPj4gKyAgICAgICAgICAgICAgICAgICAgICAgZG1hX2J1 Zl9tYXBfc2V0X3ZhZGRyKCZpdGVyX2lvLT5kbWFwLAo+ID4+ICsgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICBtZW1yZW1hcChtZW0tPmJ1cy5vZmZzZXQsIGJ1c19z aXplLAo+ID4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICBNRU1SRU1BUF9XQikpOwo+ID4gVGhlIGNvbW1lbnRzIGluIHNldF92YWRkciBzdWdn ZXN0IHRoYXQgdGhpcyBpcyBtZWFudCBmb3IKPiA+IHN5c3RlbS1tZW1vcnkuIERvZXMgdGhhdCBh Y3R1YWxseSBtYXR0ZXIgb3IgaXMgaXQganVzdCBhYm91dCBub3QKPiA+IGxvc2luZyB0aGUgX19p b21lbSBhbm5vdGF0aW9uIG9uIHBsYXRmb3JtcyB3aGVyZSBpdCBtYXR0ZXJzPwo+Cj4gWWVzLCBp dCdzIHRoZSBsYXR0ZXIuIGRtYV9idWZfbWFwKCkgaXMgcmVsYXRpdmVseSBuZXcgYW5kIHRoZSBh dXRob3IKPiBwcm9iYWJseSBkaWRuJ3QgdGhpbmsgYWJvdXQgdGhlIGNhc2Ugb2YgY2FjaGVkIGlv bWVtLCB3aGljaCBpcyB1c2VkIGJ5LAo+IGZvciBleGFtcGxlLCB2bXdnZnguCj4KPiA+IEFwcGFy ZW50bHkgY2FjaGVkIGRldmljZSBsb2NhbCBpcyBhIHRoaW5nLiBBbHNvIHNob3VsZCB0aGlzIG5v dCBiZQo+ID4gd3JhcHBlZCBpbiBDT05GSUdfWDg2Pwo+Cj4gQm90aCBkbWFfYnVmX21hcCgpIGFu ZCBtZW1yZW1hcCBhcmUgZ2VuZXJpYywgSSB0aGluaywgSSBndWVzcyBtZW1yZW1hcAo+IHdvdWxk IHJldHVybiBOVUxMIGlmIGl0J3Mgbm90IHN1cHBvcnRlZC4KCkl0IGxvb2tzIGxpa2UgbWVtcmVt YXAganVzdCB3cmFwcyBpb3JlbWFwX2NhY2hlLCBidXQgc2luY2UgaXQgYWxzbwpkaXNjYXJkcyB0 aGUgX19pb21lbSBhbm5vdGF0aW9uIHNob3VsZCB3ZSBiZSBkb2luZyB0aGF0IHVuaXZlcnNhbGx5 PwpBbHNvIG5vdCBzdXJlIGlmIGlvcmVtYXBfY2FjaGUgaXMgdW5pdmVyc2FsbHkgc3VwcG9ydGVk LCBzbyB3cmFwcGluZwppbiBDT05GSUdfWDg2IGFuZCBmYWxsaW5nIGJhY2sgdG8gcGxhaW4gaW9y ZW1hcCgpIG1pZ2h0IGJlIG5lZWRlZD8gT3IKYXQgbGVhc3QgdGhhdCBsb29rcyBsaWtlIHJvdWdo bHkgd2hhdCB0aGUgcHJldmlvdXMgY29kZSB3YXMgZG9pbmc/IE5vdAp0b28gc3VyZSB0YmguCgo+ Cj4gL1Rob21hcwo+Cj4KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX18KSW50ZWwtZ2Z4IG1haWxpbmcgbGlzdApJbnRlbC1nZnhAbGlzdHMuZnJlZWRlc2t0b3Au b3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vaW50ZWwt Z2Z4Cg==