From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BA98C47083 for ; Wed, 2 Jun 2021 17:08:49 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2EB2861CA8 for ; Wed, 2 Jun 2021 17:08:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2EB2861CA8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 918576EE0B; Wed, 2 Jun 2021 17:08:38 +0000 (UTC) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3505C6EDCE; Wed, 2 Jun 2021 17:08:37 +0000 (UTC) IronPort-SDR: OZi4oSi5tAGC0H7N5PkjRkVG01belav7pxCRQH0en+7anUrRDVzF0sGXLVgzAKI+7eQQQ5y9p1 IJVBjwK7vtSA== X-IronPort-AV: E=McAfee;i="6200,9189,10003"; a="225133289" X-IronPort-AV: E=Sophos;i="5.83,242,1616482800"; d="scan'208";a="225133289" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jun 2021 10:07:42 -0700 IronPort-SDR: tijczsWfmzy9DooLrhODkgGbZD2+UYxRoINFfzifWvIq6su+Gh0yOkDKJcDBje+0VXvSaza06t 1twaMiRslL9w== X-IronPort-AV: E=Sophos;i="5.83,242,1616482800"; d="scan'208";a="550220239" Received: from ibanaga-mobl.ger.corp.intel.com (HELO thellst-mobl1.intel.com) ([10.249.254.50]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jun 2021 10:07:41 -0700 From: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 2/5] drm/i915/ttm: Adjust gem flags and caching settings after a move Date: Wed, 2 Jun 2021 19:07:13 +0200 Message-Id: <20210602170716.280491-3-thomas.hellstrom@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210602170716.280491-1-thomas.hellstrom@linux.intel.com> References: <20210602170716.280491-1-thomas.hellstrom@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" After a TTM move we need to update the i915 gem flags and caching settings to reflect the new placement. Also introduce gpu_binds_iomem() and cpu_maps_iomem() to clean up the various ways we previously used to detect this. Finally, initialize the TTM object reserved to be able to update flags and caching before anyone else gets hold of the object. Signed-off-by: Thomas Hellström --- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 112 +++++++++++++++++++----- 1 file changed, 90 insertions(+), 22 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c index ae12a2be11a2..c73c51755c20 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -70,6 +70,17 @@ static struct ttm_placement i915_sys_placement = { .busy_placement = &lmem0_sys_placement_flags[1], }; +static bool gpu_binds_iomem(struct ttm_resource *mem) +{ + return (mem->mem_type != TTM_PL_SYSTEM); +} + +static bool cpu_maps_iomem(struct ttm_resource *mem) +{ + /* Once / if we support GGTT, this is also false for cached ttm_tts */ + return (mem->mem_type != TTM_PL_SYSTEM); +} + static void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj); static struct ttm_tt *i915_ttm_tt_create(struct ttm_buffer_object *bo, @@ -175,6 +186,41 @@ static void i915_ttm_free_cached_io_st(struct drm_i915_gem_object *obj) obj->ttm.cached_io_st = NULL; } +static void +i915_ttm_adjust_domains_after_cpu_move(struct drm_i915_gem_object *obj) +{ + struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); + + if (cpu_maps_iomem(&bo->mem) || bo->ttm->caching != ttm_cached) { + obj->write_domain = I915_GEM_DOMAIN_WC; + obj->read_domains = I915_GEM_DOMAIN_WC; + } else { + obj->write_domain = I915_GEM_DOMAIN_CPU; + obj->read_domains = I915_GEM_DOMAIN_CPU; + } +} + +static void i915_ttm_adjust_gem_after_move(struct drm_i915_gem_object *obj) +{ + struct drm_i915_private *i915 = to_i915(obj->base.dev); + struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); + unsigned int cache_level; + + obj->mem_flags &= ~(I915_BO_FLAG_STRUCT_PAGE | I915_BO_FLAG_IOMEM); + + obj->mem_flags |= cpu_maps_iomem(&bo->mem) ? I915_BO_FLAG_IOMEM : + I915_BO_FLAG_STRUCT_PAGE; + + if ((HAS_LLC(i915) || HAS_SNOOP(i915)) && !gpu_binds_iomem(&bo->mem) && + bo->ttm->caching == ttm_cached) { + cache_level = I915_CACHE_LLC; + } else { + cache_level = I915_CACHE_NONE; + } + + i915_gem_object_set_cache_coherency(obj, cache_level); +} + static void i915_ttm_purge(struct drm_i915_gem_object *obj) { struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); @@ -190,8 +236,10 @@ static void i915_ttm_purge(struct drm_i915_gem_object *obj) /* TTM's purge interface. Note that we might be reentering. */ ret = ttm_bo_validate(bo, &place, &ctx); - if (!ret) { + obj->write_domain = 0; + obj->read_domains = 0; + i915_ttm_adjust_gem_after_move(obj); i915_ttm_free_cached_io_st(obj); obj->mm.madv = __I915_MADV_PURGED; } @@ -273,12 +321,15 @@ i915_ttm_resource_get_st(struct drm_i915_gem_object *obj, struct ttm_resource *res) { struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); - struct ttm_resource_manager *man = - ttm_manager_type(bo->bdev, res->mem_type); - if (man->use_tt) + if (!gpu_binds_iomem(res)) return i915_ttm_tt_get_st(bo->ttm); + /* + * If CPU mapping differs, we need to add the ttm_tt pages to + * the resulting st. Might make sense for GGTT. + */ + GEM_WARN_ON(!cpu_maps_iomem(res)); return intel_region_ttm_node_to_st(obj->mm.region, res->mm_node); } @@ -290,8 +341,6 @@ static int i915_ttm_move(struct ttm_buffer_object *bo, bool evict, struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo); struct ttm_resource_manager *dst_man = ttm_manager_type(bo->bdev, dst_mem->mem_type); - struct ttm_resource_manager *src_man = - ttm_manager_type(bo->bdev, bo->mem.mem_type); struct intel_memory_region *dst_reg, *src_reg; union { struct ttm_kmap_iter_tt tt; @@ -332,34 +381,36 @@ static int i915_ttm_move(struct ttm_buffer_object *bo, bool evict, if (IS_ERR(dst_st)) return PTR_ERR(dst_st); - /* If we start mapping GGTT, we can no longer use man::use_tt here. */ - dst_iter = dst_man->use_tt ? + dst_iter = !cpu_maps_iomem(dst_mem) ? ttm_kmap_iter_tt_init(&_dst_iter.tt, bo->ttm) : ttm_kmap_iter_iomap_init(&_dst_iter.io, &dst_reg->iomap, dst_st, dst_reg->region.start); - src_iter = src_man->use_tt ? + src_iter = !cpu_maps_iomem(&bo->mem) ? ttm_kmap_iter_tt_init(&_src_iter.tt, bo->ttm) : ttm_kmap_iter_iomap_init(&_src_iter.io, &src_reg->iomap, obj->ttm.cached_io_st, src_reg->region.start); ttm_move_memcpy(bo, dst_mem->num_pages, dst_iter, src_iter); + /* Below dst_mem becomes bo->mem. */ ttm_bo_move_sync_cleanup(bo, dst_mem); + i915_ttm_adjust_domains_after_cpu_move(obj); i915_ttm_free_cached_io_st(obj); - if (!dst_man->use_tt) { + if (gpu_binds_iomem(dst_mem) || cpu_maps_iomem(dst_mem)) { obj->ttm.cached_io_st = dst_st; obj->ttm.get_io_page.sg_pos = dst_st->sgl; obj->ttm.get_io_page.sg_idx = 0; } + i915_ttm_adjust_gem_after_move(obj); return 0; } static int i915_ttm_io_mem_reserve(struct ttm_device *bdev, struct ttm_resource *mem) { - if (mem->mem_type < I915_PL_LMEM0) + if (!cpu_maps_iomem(mem)) return 0; mem->bus.caching = ttm_write_combined; @@ -421,6 +472,16 @@ static int i915_ttm_get_pages(struct drm_i915_gem_object *obj) if (ret) return ret == -ENOSPC ? -ENXIO : ret; + i915_ttm_adjust_lru(obj); + if (bo->ttm && !ttm_tt_is_populated(bo->ttm)) { + ret = ttm_tt_populate(bo->bdev, bo->ttm, &ctx); + if (ret) + return ret; + + i915_ttm_adjust_domains_after_cpu_move(obj); + i915_ttm_adjust_gem_after_move(obj); + } + /* Object either has a page vector or is an iomem object */ st = bo->ttm ? i915_ttm_tt_get_st(bo->ttm) : obj->ttm.cached_io_st; if (IS_ERR(st)) @@ -428,8 +489,6 @@ static int i915_ttm_get_pages(struct drm_i915_gem_object *obj) __i915_gem_object_set_pages(obj, st, i915_sg_dma_sizes(st->sgl)); - i915_ttm_adjust_lru(obj); - return ret; } @@ -563,6 +622,7 @@ static u64 i915_ttm_mmap_offset(struct drm_i915_gem_object *obj) const struct drm_i915_gem_object_ops i915_gem_ttm_obj_ops = { .name = "i915_gem_object_ttm", + .flags = I915_GEM_OBJECT_IS_SHRINKABLE, .get_pages = i915_ttm_get_pages, .put_pages = i915_ttm_put_pages, @@ -599,6 +659,10 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem, { static struct lock_class_key lock_class; struct drm_i915_private *i915 = mem->i915; + struct ttm_operation_ctx ctx = { + .interruptible = true, + .no_wait_gpu = false, + }; enum ttm_bo_type bo_type; size_t alignment = 0; int ret; @@ -618,15 +682,14 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem, i915_gem_object_init(obj, &i915_gem_ttm_obj_ops, &lock_class, flags); i915_gem_object_init_memory_region(obj, mem); i915_gem_object_make_unshrinkable(obj); - obj->read_domains = I915_GEM_DOMAIN_WC | I915_GEM_DOMAIN_GTT; - obj->mem_flags |= I915_BO_FLAG_IOMEM; - i915_gem_object_set_cache_coherency(obj, I915_CACHE_NONE); INIT_RADIX_TREE(&obj->ttm.get_io_page.radix, GFP_KERNEL | __GFP_NOWARN); mutex_init(&obj->ttm.get_io_page.lock); bo_type = (obj->flags & I915_BO_ALLOC_USER) ? ttm_bo_type_device : ttm_bo_type_kernel; + obj->base.vma_node.driver_private = i915_gem_to_ttm(obj); + /* * If this function fails, it will call the destructor, but * our caller still owns the object. So no freeing in the @@ -634,14 +697,19 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem, * Similarly, in delayed_destroy, we can't call ttm_bo_put() * until successful initialization. */ - obj->base.vma_node.driver_private = i915_gem_to_ttm(obj); - ret = ttm_bo_init(&i915->bdev, i915_gem_to_ttm(obj), size, - bo_type, &i915_sys_placement, alignment, - true, NULL, NULL, i915_ttm_bo_destroy); + ret = ttm_bo_init_reserved(&i915->bdev, i915_gem_to_ttm(obj), size, + bo_type, &i915_sys_placement, alignment, + &ctx, NULL, NULL, i915_ttm_bo_destroy); + + if (ret) + goto out; - if (!ret) - obj->ttm.created = true; + obj->ttm.created = true; + i915_ttm_adjust_domains_after_cpu_move(obj); + i915_ttm_adjust_gem_after_move(obj); + i915_gem_object_unlock(obj); +out: /* i915 wants -ENXIO when out of memory region space. */ return (ret == -ENOSPC) ? -ENXIO : ret; } -- 2.31.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05262C47092 for ; Wed, 2 Jun 2021 17:08:51 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C852E61CA7 for ; Wed, 2 Jun 2021 17:08:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C852E61CA7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0C3106EE0F; Wed, 2 Jun 2021 17:08:39 +0000 (UTC) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3505C6EDCE; Wed, 2 Jun 2021 17:08:37 +0000 (UTC) IronPort-SDR: OZi4oSi5tAGC0H7N5PkjRkVG01belav7pxCRQH0en+7anUrRDVzF0sGXLVgzAKI+7eQQQ5y9p1 IJVBjwK7vtSA== X-IronPort-AV: E=McAfee;i="6200,9189,10003"; a="225133289" X-IronPort-AV: E=Sophos;i="5.83,242,1616482800"; d="scan'208";a="225133289" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jun 2021 10:07:42 -0700 IronPort-SDR: tijczsWfmzy9DooLrhODkgGbZD2+UYxRoINFfzifWvIq6su+Gh0yOkDKJcDBje+0VXvSaza06t 1twaMiRslL9w== X-IronPort-AV: E=Sophos;i="5.83,242,1616482800"; d="scan'208";a="550220239" Received: from ibanaga-mobl.ger.corp.intel.com (HELO thellst-mobl1.intel.com) ([10.249.254.50]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jun 2021 10:07:41 -0700 From: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Date: Wed, 2 Jun 2021 19:07:13 +0200 Message-Id: <20210602170716.280491-3-thomas.hellstrom@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210602170716.280491-1-thomas.hellstrom@linux.intel.com> References: <20210602170716.280491-1-thomas.hellstrom@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 2/5] drm/i915/ttm: Adjust gem flags and caching settings after a move X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" QWZ0ZXIgYSBUVE0gbW92ZSB3ZSBuZWVkIHRvIHVwZGF0ZSB0aGUgaTkxNSBnZW0gZmxhZ3MgYW5k IGNhY2hpbmcKc2V0dGluZ3MgdG8gcmVmbGVjdCB0aGUgbmV3IHBsYWNlbWVudC4KQWxzbyBpbnRy b2R1Y2UgZ3B1X2JpbmRzX2lvbWVtKCkgYW5kIGNwdV9tYXBzX2lvbWVtKCkgdG8gY2xlYW4gdXAg dGhlCnZhcmlvdXMgd2F5cyB3ZSBwcmV2aW91c2x5IHVzZWQgdG8gZGV0ZWN0IHRoaXMuCkZpbmFs bHksIGluaXRpYWxpemUgdGhlIFRUTSBvYmplY3QgcmVzZXJ2ZWQgdG8gYmUgYWJsZSB0byB1cGRh dGUKZmxhZ3MgYW5kIGNhY2hpbmcgYmVmb3JlIGFueW9uZSBlbHNlIGdldHMgaG9sZCBvZiB0aGUg b2JqZWN0LgoKU2lnbmVkLW9mZi1ieTogVGhvbWFzIEhlbGxzdHLDtm0gPHRob21hcy5oZWxsc3Ry b21AbGludXguaW50ZWwuY29tPgotLS0KIGRyaXZlcnMvZ3B1L2RybS9pOTE1L2dlbS9pOTE1X2dl bV90dG0uYyB8IDExMiArKysrKysrKysrKysrKysrKysrLS0tLS0KIDEgZmlsZSBjaGFuZ2VkLCA5 MCBpbnNlcnRpb25zKCspLCAyMiBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS9kcml2ZXJzL2dw dS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1fdHRtLmMgYi9kcml2ZXJzL2dwdS9kcm0vaTkxNS9nZW0v aTkxNV9nZW1fdHRtLmMKaW5kZXggYWUxMmEyYmUxMWEyLi5jNzNjNTE3NTVjMjAgMTAwNjQ0Ci0t LSBhL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2dlbS9pOTE1X2dlbV90dG0uYworKysgYi9kcml2ZXJz L2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1fdHRtLmMKQEAgLTcwLDYgKzcwLDE3IEBAIHN0YXRp YyBzdHJ1Y3QgdHRtX3BsYWNlbWVudCBpOTE1X3N5c19wbGFjZW1lbnQgPSB7CiAJLmJ1c3lfcGxh Y2VtZW50ID0gJmxtZW0wX3N5c19wbGFjZW1lbnRfZmxhZ3NbMV0sCiB9OwogCitzdGF0aWMgYm9v bCBncHVfYmluZHNfaW9tZW0oc3RydWN0IHR0bV9yZXNvdXJjZSAqbWVtKQoreworCXJldHVybiAo bWVtLT5tZW1fdHlwZSAhPSBUVE1fUExfU1lTVEVNKTsKK30KKworc3RhdGljIGJvb2wgY3B1X21h cHNfaW9tZW0oc3RydWN0IHR0bV9yZXNvdXJjZSAqbWVtKQoreworCS8qIE9uY2UgLyBpZiB3ZSBz dXBwb3J0IEdHVFQsIHRoaXMgaXMgYWxzbyBmYWxzZSBmb3IgY2FjaGVkIHR0bV90dHMgKi8KKwly ZXR1cm4gKG1lbS0+bWVtX3R5cGUgIT0gVFRNX1BMX1NZU1RFTSk7Cit9CisKIHN0YXRpYyB2b2lk IGk5MTVfdHRtX2FkanVzdF9scnUoc3RydWN0IGRybV9pOTE1X2dlbV9vYmplY3QgKm9iaik7CiAK IHN0YXRpYyBzdHJ1Y3QgdHRtX3R0ICppOTE1X3R0bV90dF9jcmVhdGUoc3RydWN0IHR0bV9idWZm ZXJfb2JqZWN0ICpibywKQEAgLTE3NSw2ICsxODYsNDEgQEAgc3RhdGljIHZvaWQgaTkxNV90dG1f ZnJlZV9jYWNoZWRfaW9fc3Qoc3RydWN0IGRybV9pOTE1X2dlbV9vYmplY3QgKm9iaikKIAlvYmot PnR0bS5jYWNoZWRfaW9fc3QgPSBOVUxMOwogfQogCitzdGF0aWMgdm9pZAoraTkxNV90dG1fYWRq dXN0X2RvbWFpbnNfYWZ0ZXJfY3B1X21vdmUoc3RydWN0IGRybV9pOTE1X2dlbV9vYmplY3QgKm9i aikKK3sKKwlzdHJ1Y3QgdHRtX2J1ZmZlcl9vYmplY3QgKmJvID0gaTkxNV9nZW1fdG9fdHRtKG9i aik7CisKKwlpZiAoY3B1X21hcHNfaW9tZW0oJmJvLT5tZW0pIHx8IGJvLT50dG0tPmNhY2hpbmcg IT0gdHRtX2NhY2hlZCkgeworCQlvYmotPndyaXRlX2RvbWFpbiA9IEk5MTVfR0VNX0RPTUFJTl9X QzsKKwkJb2JqLT5yZWFkX2RvbWFpbnMgPSBJOTE1X0dFTV9ET01BSU5fV0M7CisJfSBlbHNlIHsK KwkJb2JqLT53cml0ZV9kb21haW4gPSBJOTE1X0dFTV9ET01BSU5fQ1BVOworCQlvYmotPnJlYWRf ZG9tYWlucyA9IEk5MTVfR0VNX0RPTUFJTl9DUFU7CisJfQorfQorCitzdGF0aWMgdm9pZCBpOTE1 X3R0bV9hZGp1c3RfZ2VtX2FmdGVyX21vdmUoc3RydWN0IGRybV9pOTE1X2dlbV9vYmplY3QgKm9i aikKK3sKKwlzdHJ1Y3QgZHJtX2k5MTVfcHJpdmF0ZSAqaTkxNSA9IHRvX2k5MTUob2JqLT5iYXNl LmRldik7CisJc3RydWN0IHR0bV9idWZmZXJfb2JqZWN0ICpibyA9IGk5MTVfZ2VtX3RvX3R0bShv YmopOworCXVuc2lnbmVkIGludCBjYWNoZV9sZXZlbDsKKworCW9iai0+bWVtX2ZsYWdzICY9IH4o STkxNV9CT19GTEFHX1NUUlVDVF9QQUdFIHwgSTkxNV9CT19GTEFHX0lPTUVNKTsKKworCW9iai0+ bWVtX2ZsYWdzIHw9IGNwdV9tYXBzX2lvbWVtKCZiby0+bWVtKSA/IEk5MTVfQk9fRkxBR19JT01F TSA6CisJCUk5MTVfQk9fRkxBR19TVFJVQ1RfUEFHRTsKKworCWlmICgoSEFTX0xMQyhpOTE1KSB8 fCBIQVNfU05PT1AoaTkxNSkpICYmICFncHVfYmluZHNfaW9tZW0oJmJvLT5tZW0pICYmCisJICAg IGJvLT50dG0tPmNhY2hpbmcgPT0gdHRtX2NhY2hlZCkgeworCQljYWNoZV9sZXZlbCA9IEk5MTVf Q0FDSEVfTExDOworCX0gZWxzZSB7CisJCWNhY2hlX2xldmVsID0gSTkxNV9DQUNIRV9OT05FOwor CX0KKworCWk5MTVfZ2VtX29iamVjdF9zZXRfY2FjaGVfY29oZXJlbmN5KG9iaiwgY2FjaGVfbGV2 ZWwpOworfQorCiBzdGF0aWMgdm9pZCBpOTE1X3R0bV9wdXJnZShzdHJ1Y3QgZHJtX2k5MTVfZ2Vt X29iamVjdCAqb2JqKQogewogCXN0cnVjdCB0dG1fYnVmZmVyX29iamVjdCAqYm8gPSBpOTE1X2dl bV90b190dG0ob2JqKTsKQEAgLTE5MCw4ICsyMzYsMTAgQEAgc3RhdGljIHZvaWQgaTkxNV90dG1f cHVyZ2Uoc3RydWN0IGRybV9pOTE1X2dlbV9vYmplY3QgKm9iaikKIAogCS8qIFRUTSdzIHB1cmdl IGludGVyZmFjZS4gTm90ZSB0aGF0IHdlIG1pZ2h0IGJlIHJlZW50ZXJpbmcuICovCiAJcmV0ID0g dHRtX2JvX3ZhbGlkYXRlKGJvLCAmcGxhY2UsICZjdHgpOwotCiAJaWYgKCFyZXQpIHsKKwkJb2Jq LT53cml0ZV9kb21haW4gPSAwOworCQlvYmotPnJlYWRfZG9tYWlucyA9IDA7CisJCWk5MTVfdHRt X2FkanVzdF9nZW1fYWZ0ZXJfbW92ZShvYmopOwogCQlpOTE1X3R0bV9mcmVlX2NhY2hlZF9pb19z dChvYmopOwogCQlvYmotPm1tLm1hZHYgPSBfX0k5MTVfTUFEVl9QVVJHRUQ7CiAJfQpAQCAtMjcz LDEyICszMjEsMTUgQEAgaTkxNV90dG1fcmVzb3VyY2VfZ2V0X3N0KHN0cnVjdCBkcm1faTkxNV9n ZW1fb2JqZWN0ICpvYmosCiAJCQkgc3RydWN0IHR0bV9yZXNvdXJjZSAqcmVzKQogewogCXN0cnVj dCB0dG1fYnVmZmVyX29iamVjdCAqYm8gPSBpOTE1X2dlbV90b190dG0ob2JqKTsKLQlzdHJ1Y3Qg dHRtX3Jlc291cmNlX21hbmFnZXIgKm1hbiA9Ci0JCXR0bV9tYW5hZ2VyX3R5cGUoYm8tPmJkZXYs IHJlcy0+bWVtX3R5cGUpOwogCi0JaWYgKG1hbi0+dXNlX3R0KQorCWlmICghZ3B1X2JpbmRzX2lv bWVtKHJlcykpCiAJCXJldHVybiBpOTE1X3R0bV90dF9nZXRfc3QoYm8tPnR0bSk7CiAKKwkvKgor CSAqIElmIENQVSBtYXBwaW5nIGRpZmZlcnMsIHdlIG5lZWQgdG8gYWRkIHRoZSB0dG1fdHQgcGFn ZXMgdG8KKwkgKiB0aGUgcmVzdWx0aW5nIHN0LiBNaWdodCBtYWtlIHNlbnNlIGZvciBHR1RULgor CSAqLworCUdFTV9XQVJOX09OKCFjcHVfbWFwc19pb21lbShyZXMpKTsKIAlyZXR1cm4gaW50ZWxf cmVnaW9uX3R0bV9ub2RlX3RvX3N0KG9iai0+bW0ucmVnaW9uLCByZXMtPm1tX25vZGUpOwogfQog CkBAIC0yOTAsOCArMzQxLDYgQEAgc3RhdGljIGludCBpOTE1X3R0bV9tb3ZlKHN0cnVjdCB0dG1f YnVmZmVyX29iamVjdCAqYm8sIGJvb2wgZXZpY3QsCiAJc3RydWN0IGRybV9pOTE1X2dlbV9vYmpl Y3QgKm9iaiA9IGk5MTVfdHRtX3RvX2dlbShibyk7CiAJc3RydWN0IHR0bV9yZXNvdXJjZV9tYW5h Z2VyICpkc3RfbWFuID0KIAkJdHRtX21hbmFnZXJfdHlwZShiby0+YmRldiwgZHN0X21lbS0+bWVt X3R5cGUpOwotCXN0cnVjdCB0dG1fcmVzb3VyY2VfbWFuYWdlciAqc3JjX21hbiA9Ci0JCXR0bV9t YW5hZ2VyX3R5cGUoYm8tPmJkZXYsIGJvLT5tZW0ubWVtX3R5cGUpOwogCXN0cnVjdCBpbnRlbF9t ZW1vcnlfcmVnaW9uICpkc3RfcmVnLCAqc3JjX3JlZzsKIAl1bmlvbiB7CiAJCXN0cnVjdCB0dG1f a21hcF9pdGVyX3R0IHR0OwpAQCAtMzMyLDM0ICszODEsMzYgQEAgc3RhdGljIGludCBpOTE1X3R0 bV9tb3ZlKHN0cnVjdCB0dG1fYnVmZmVyX29iamVjdCAqYm8sIGJvb2wgZXZpY3QsCiAJaWYgKElT X0VSUihkc3Rfc3QpKQogCQlyZXR1cm4gUFRSX0VSUihkc3Rfc3QpOwogCi0JLyogSWYgd2Ugc3Rh cnQgbWFwcGluZyBHR1RULCB3ZSBjYW4gbm8gbG9uZ2VyIHVzZSBtYW46OnVzZV90dCBoZXJlLiAq LwotCWRzdF9pdGVyID0gZHN0X21hbi0+dXNlX3R0ID8KKwlkc3RfaXRlciA9ICFjcHVfbWFwc19p b21lbShkc3RfbWVtKSA/CiAJCXR0bV9rbWFwX2l0ZXJfdHRfaW5pdCgmX2RzdF9pdGVyLnR0LCBi by0+dHRtKSA6CiAJCXR0bV9rbWFwX2l0ZXJfaW9tYXBfaW5pdCgmX2RzdF9pdGVyLmlvLCAmZHN0 X3JlZy0+aW9tYXAsCiAJCQkJCSBkc3Rfc3QsIGRzdF9yZWctPnJlZ2lvbi5zdGFydCk7CiAKLQlz cmNfaXRlciA9IHNyY19tYW4tPnVzZV90dCA/CisJc3JjX2l0ZXIgPSAhY3B1X21hcHNfaW9tZW0o JmJvLT5tZW0pID8KIAkJdHRtX2ttYXBfaXRlcl90dF9pbml0KCZfc3JjX2l0ZXIudHQsIGJvLT50 dG0pIDoKIAkJdHRtX2ttYXBfaXRlcl9pb21hcF9pbml0KCZfc3JjX2l0ZXIuaW8sICZzcmNfcmVn LT5pb21hcCwKIAkJCQkJIG9iai0+dHRtLmNhY2hlZF9pb19zdCwKIAkJCQkJIHNyY19yZWctPnJl Z2lvbi5zdGFydCk7CiAKIAl0dG1fbW92ZV9tZW1jcHkoYm8sIGRzdF9tZW0tPm51bV9wYWdlcywg ZHN0X2l0ZXIsIHNyY19pdGVyKTsKKwkvKiBCZWxvdyBkc3RfbWVtIGJlY29tZXMgYm8tPm1lbS4g Ki8KIAl0dG1fYm9fbW92ZV9zeW5jX2NsZWFudXAoYm8sIGRzdF9tZW0pOworCWk5MTVfdHRtX2Fk anVzdF9kb21haW5zX2FmdGVyX2NwdV9tb3ZlKG9iaik7CiAJaTkxNV90dG1fZnJlZV9jYWNoZWRf aW9fc3Qob2JqKTsKIAotCWlmICghZHN0X21hbi0+dXNlX3R0KSB7CisJaWYgKGdwdV9iaW5kc19p b21lbShkc3RfbWVtKSB8fCBjcHVfbWFwc19pb21lbShkc3RfbWVtKSkgewogCQlvYmotPnR0bS5j YWNoZWRfaW9fc3QgPSBkc3Rfc3Q7CiAJCW9iai0+dHRtLmdldF9pb19wYWdlLnNnX3BvcyA9IGRz dF9zdC0+c2dsOwogCQlvYmotPnR0bS5nZXRfaW9fcGFnZS5zZ19pZHggPSAwOwogCX0KIAorCWk5 MTVfdHRtX2FkanVzdF9nZW1fYWZ0ZXJfbW92ZShvYmopOwogCXJldHVybiAwOwogfQogCiBzdGF0 aWMgaW50IGk5MTVfdHRtX2lvX21lbV9yZXNlcnZlKHN0cnVjdCB0dG1fZGV2aWNlICpiZGV2LCBz dHJ1Y3QgdHRtX3Jlc291cmNlICptZW0pCiB7Ci0JaWYgKG1lbS0+bWVtX3R5cGUgPCBJOTE1X1BM X0xNRU0wKQorCWlmICghY3B1X21hcHNfaW9tZW0obWVtKSkKIAkJcmV0dXJuIDA7CiAKIAltZW0t PmJ1cy5jYWNoaW5nID0gdHRtX3dyaXRlX2NvbWJpbmVkOwpAQCAtNDIxLDYgKzQ3MiwxNiBAQCBz dGF0aWMgaW50IGk5MTVfdHRtX2dldF9wYWdlcyhzdHJ1Y3QgZHJtX2k5MTVfZ2VtX29iamVjdCAq b2JqKQogCWlmIChyZXQpCiAJCXJldHVybiByZXQgPT0gLUVOT1NQQyA/IC1FTlhJTyA6IHJldDsK IAorCWk5MTVfdHRtX2FkanVzdF9scnUob2JqKTsKKwlpZiAoYm8tPnR0bSAmJiAhdHRtX3R0X2lz X3BvcHVsYXRlZChiby0+dHRtKSkgeworCQlyZXQgPSB0dG1fdHRfcG9wdWxhdGUoYm8tPmJkZXYs IGJvLT50dG0sICZjdHgpOworCQlpZiAocmV0KQorCQkJcmV0dXJuIHJldDsKKworCQlpOTE1X3R0 bV9hZGp1c3RfZG9tYWluc19hZnRlcl9jcHVfbW92ZShvYmopOworCQlpOTE1X3R0bV9hZGp1c3Rf Z2VtX2FmdGVyX21vdmUob2JqKTsKKwl9CisKIAkvKiBPYmplY3QgZWl0aGVyIGhhcyBhIHBhZ2Ug dmVjdG9yIG9yIGlzIGFuIGlvbWVtIG9iamVjdCAqLwogCXN0ID0gYm8tPnR0bSA/IGk5MTVfdHRt X3R0X2dldF9zdChiby0+dHRtKSA6IG9iai0+dHRtLmNhY2hlZF9pb19zdDsKIAlpZiAoSVNfRVJS KHN0KSkKQEAgLTQyOCw4ICs0ODksNiBAQCBzdGF0aWMgaW50IGk5MTVfdHRtX2dldF9wYWdlcyhz dHJ1Y3QgZHJtX2k5MTVfZ2VtX29iamVjdCAqb2JqKQogCiAJX19pOTE1X2dlbV9vYmplY3Rfc2V0 X3BhZ2VzKG9iaiwgc3QsIGk5MTVfc2dfZG1hX3NpemVzKHN0LT5zZ2wpKTsKIAotCWk5MTVfdHRt X2FkanVzdF9scnUob2JqKTsKLQogCXJldHVybiByZXQ7CiB9CiAKQEAgLTU2Myw2ICs2MjIsNyBA QCBzdGF0aWMgdTY0IGk5MTVfdHRtX21tYXBfb2Zmc2V0KHN0cnVjdCBkcm1faTkxNV9nZW1fb2Jq ZWN0ICpvYmopCiAKIGNvbnN0IHN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0X29wcyBpOTE1X2dl bV90dG1fb2JqX29wcyA9IHsKIAkubmFtZSA9ICJpOTE1X2dlbV9vYmplY3RfdHRtIiwKKwkuZmxh Z3MgPSBJOTE1X0dFTV9PQkpFQ1RfSVNfU0hSSU5LQUJMRSwKIAogCS5nZXRfcGFnZXMgPSBpOTE1 X3R0bV9nZXRfcGFnZXMsCiAJLnB1dF9wYWdlcyA9IGk5MTVfdHRtX3B1dF9wYWdlcywKQEAgLTU5 OSw2ICs2NTksMTAgQEAgaW50IF9faTkxNV9nZW1fdHRtX29iamVjdF9pbml0KHN0cnVjdCBpbnRl bF9tZW1vcnlfcmVnaW9uICptZW0sCiB7CiAJc3RhdGljIHN0cnVjdCBsb2NrX2NsYXNzX2tleSBs b2NrX2NsYXNzOwogCXN0cnVjdCBkcm1faTkxNV9wcml2YXRlICppOTE1ID0gbWVtLT5pOTE1Owor CXN0cnVjdCB0dG1fb3BlcmF0aW9uX2N0eCBjdHggPSB7CisJCS5pbnRlcnJ1cHRpYmxlID0gdHJ1 ZSwKKwkJLm5vX3dhaXRfZ3B1ID0gZmFsc2UsCisJfTsKIAllbnVtIHR0bV9ib190eXBlIGJvX3R5 cGU7CiAJc2l6ZV90IGFsaWdubWVudCA9IDA7CiAJaW50IHJldDsKQEAgLTYxOCwxNSArNjgyLDE0 IEBAIGludCBfX2k5MTVfZ2VtX3R0bV9vYmplY3RfaW5pdChzdHJ1Y3QgaW50ZWxfbWVtb3J5X3Jl Z2lvbiAqbWVtLAogCWk5MTVfZ2VtX29iamVjdF9pbml0KG9iaiwgJmk5MTVfZ2VtX3R0bV9vYmpf b3BzLCAmbG9ja19jbGFzcywgZmxhZ3MpOwogCWk5MTVfZ2VtX29iamVjdF9pbml0X21lbW9yeV9y ZWdpb24ob2JqLCBtZW0pOwogCWk5MTVfZ2VtX29iamVjdF9tYWtlX3Vuc2hyaW5rYWJsZShvYmop OwotCW9iai0+cmVhZF9kb21haW5zID0gSTkxNV9HRU1fRE9NQUlOX1dDIHwgSTkxNV9HRU1fRE9N QUlOX0dUVDsKLQlvYmotPm1lbV9mbGFncyB8PSBJOTE1X0JPX0ZMQUdfSU9NRU07Ci0JaTkxNV9n ZW1fb2JqZWN0X3NldF9jYWNoZV9jb2hlcmVuY3kob2JqLCBJOTE1X0NBQ0hFX05PTkUpOwogCUlO SVRfUkFESVhfVFJFRSgmb2JqLT50dG0uZ2V0X2lvX3BhZ2UucmFkaXgsIEdGUF9LRVJORUwgfCBf X0dGUF9OT1dBUk4pOwogCW11dGV4X2luaXQoJm9iai0+dHRtLmdldF9pb19wYWdlLmxvY2spOwog CiAJYm9fdHlwZSA9IChvYmotPmZsYWdzICYgSTkxNV9CT19BTExPQ19VU0VSKSA/IHR0bV9ib190 eXBlX2RldmljZSA6CiAJCXR0bV9ib190eXBlX2tlcm5lbDsKIAorCW9iai0+YmFzZS52bWFfbm9k ZS5kcml2ZXJfcHJpdmF0ZSA9IGk5MTVfZ2VtX3RvX3R0bShvYmopOworCiAJLyoKIAkgKiBJZiB0 aGlzIGZ1bmN0aW9uIGZhaWxzLCBpdCB3aWxsIGNhbGwgdGhlIGRlc3RydWN0b3IsIGJ1dAogCSAq IG91ciBjYWxsZXIgc3RpbGwgb3ducyB0aGUgb2JqZWN0LiBTbyBubyBmcmVlaW5nIGluIHRoZQpA QCAtNjM0LDE0ICs2OTcsMTkgQEAgaW50IF9faTkxNV9nZW1fdHRtX29iamVjdF9pbml0KHN0cnVj dCBpbnRlbF9tZW1vcnlfcmVnaW9uICptZW0sCiAJICogU2ltaWxhcmx5LCBpbiBkZWxheWVkX2Rl c3Ryb3ksIHdlIGNhbid0IGNhbGwgdHRtX2JvX3B1dCgpCiAJICogdW50aWwgc3VjY2Vzc2Z1bCBp bml0aWFsaXphdGlvbi4KIAkgKi8KLQlvYmotPmJhc2Uudm1hX25vZGUuZHJpdmVyX3ByaXZhdGUg PSBpOTE1X2dlbV90b190dG0ob2JqKTsKLQlyZXQgPSB0dG1fYm9faW5pdCgmaTkxNS0+YmRldiwg aTkxNV9nZW1fdG9fdHRtKG9iaiksIHNpemUsCi0JCQkgIGJvX3R5cGUsICZpOTE1X3N5c19wbGFj ZW1lbnQsIGFsaWdubWVudCwKLQkJCSAgdHJ1ZSwgTlVMTCwgTlVMTCwgaTkxNV90dG1fYm9fZGVz dHJveSk7CisJcmV0ID0gdHRtX2JvX2luaXRfcmVzZXJ2ZWQoJmk5MTUtPmJkZXYsIGk5MTVfZ2Vt X3RvX3R0bShvYmopLCBzaXplLAorCQkJCSAgIGJvX3R5cGUsICZpOTE1X3N5c19wbGFjZW1lbnQs IGFsaWdubWVudCwKKwkJCQkgICAmY3R4LCBOVUxMLCBOVUxMLCBpOTE1X3R0bV9ib19kZXN0cm95 KTsKKworCWlmIChyZXQpCisJCWdvdG8gb3V0OwogCi0JaWYgKCFyZXQpCi0JCW9iai0+dHRtLmNy ZWF0ZWQgPSB0cnVlOworCW9iai0+dHRtLmNyZWF0ZWQgPSB0cnVlOworCWk5MTVfdHRtX2FkanVz dF9kb21haW5zX2FmdGVyX2NwdV9tb3ZlKG9iaik7CisJaTkxNV90dG1fYWRqdXN0X2dlbV9hZnRl cl9tb3ZlKG9iaik7CisJaTkxNV9nZW1fb2JqZWN0X3VubG9jayhvYmopOwogCitvdXQ6CiAJLyog aTkxNSB3YW50cyAtRU5YSU8gd2hlbiBvdXQgb2YgbWVtb3J5IHJlZ2lvbiBzcGFjZS4gKi8KIAly ZXR1cm4gKHJldCA9PSAtRU5PU1BDKSA/IC1FTlhJTyA6IHJldDsKIH0KLS0gCjIuMzEuMQoKX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KSW50ZWwtZ2Z4IG1h aWxpbmcgbGlzdApJbnRlbC1nZnhAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMu ZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vaW50ZWwtZ2Z4Cg==