From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B191C48BE8 for ; Tue, 15 Jun 2021 10:05:01 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E5FA261428 for ; Tue, 15 Jun 2021 10:05:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E5FA261428 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D23B26E23B; Tue, 15 Jun 2021 10:04:59 +0000 (UTC) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6823F6E22B; Tue, 15 Jun 2021 10:04:58 +0000 (UTC) IronPort-SDR: Bp1Xg9ALpj+aIl4qF06rWzW38OWvYuxBLQoEEHry8hgtham6tXXEmBpH7/0WC/yGMU6c3VwXAy DeQq/w522Zcg== X-IronPort-AV: E=McAfee;i="6200,9189,10015"; a="202936426" X-IronPort-AV: E=Sophos;i="5.83,275,1616482800"; d="scan'208";a="202936426" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Jun 2021 03:04:55 -0700 IronPort-SDR: +UjwLZMwunbmfgGDXoNm+iFrmQijOgNiP1LX8apgWcvQfmvrXHabfvL45l0MT0PkvY3Jihzzrf gFIjaeObAVEw== X-IronPort-AV: E=Sophos;i="5.83,275,1616482800"; d="scan'208";a="421076013" Received: from ramaling-i9x.iind.intel.com ([10.99.66.205]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Jun 2021 03:04:53 -0700 From: Ramalingam C To: intel-gfx , dri-devel Subject: [PATCH v3] drm/i915/ttm: accelerated move implementation Date: Tue, 15 Jun 2021 15:36:21 +0530 Message-Id: <20210615100621.5805-1-ramalingam.c@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <0e309744-4d13-69de-2ad8-ee0df5cd46c1@linux.intel.com> References: <0e309744-4d13-69de-2ad8-ee0df5cd46c1@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Matthew Auld , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Invokes the pipelined page migration through blt, for i915_ttm_move requests of eviction and also obj clear. v2: - subfunction for accel_move (Thomas) - engine_pm_get/put around context_move/clear (Thomas) - Invalidation at accel_clear (Thomas) v3: - Timeout is set for MAX_SCHEDULE_TIMEOUT (Thomas) - s/TTM_PL_PRIV/I915_PL_LMEM0 (Thomas) Signed-off-by: Ramalingam C Reviewed-by: Thomas Hellström --- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 88 +++++++++++++++++++++---- 1 file changed, 75 insertions(+), 13 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c index 3748098b42d5..94571757fb42 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -15,6 +15,9 @@ #include "gem/i915_gem_ttm.h" #include "gem/i915_gem_mman.h" +#include "gt/intel_migrate.h" +#include "gt/intel_engine_pm.h" + #define I915_PL_LMEM0 TTM_PL_PRIV #define I915_PL_SYSTEM TTM_PL_SYSTEM #define I915_PL_STOLEN TTM_PL_VRAM @@ -282,6 +285,62 @@ i915_ttm_resource_get_st(struct drm_i915_gem_object *obj, return intel_region_ttm_node_to_st(obj->mm.region, res->mm_node); } +static int i915_ttm_accel_move(struct ttm_buffer_object *bo, + struct ttm_resource *dst_mem, + struct sg_table *dst_st) +{ + struct drm_i915_private *i915 = container_of(bo->bdev, typeof(*i915), + bdev); + struct ttm_resource_manager *src_man = + ttm_manager_type(bo->bdev, bo->mem.mem_type); + struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo); + struct sg_table *src_st; + struct i915_request *rq; + int ret; + + if (!i915->gt.migrate.context) + return -EINVAL; + + if (!bo->ttm || !ttm_tt_is_populated(bo->ttm)) { + if (bo->type == ttm_bo_type_kernel) + return -EINVAL; + + if (bo->ttm && + !(bo->ttm->page_flags & TTM_PAGE_FLAG_ZERO_ALLOC)) + return 0; + + intel_engine_pm_get(i915->gt.migrate.context->engine); + ret = intel_context_migrate_clear(i915->gt.migrate.context, NULL, + dst_st->sgl, I915_CACHE_NONE, + dst_mem->mem_type >= I915_PL_LMEM0, + 0, &rq); + + if (!ret && rq) { + i915_request_wait(rq, 0, MAX_SCHEDULE_TIMEOUT); + i915_request_put(rq); + } + intel_engine_pm_put(i915->gt.migrate.context->engine); + } else { + src_st = src_man->use_tt ? i915_ttm_tt_get_st(bo->ttm) : + obj->ttm.cached_io_st; + + intel_engine_pm_get(i915->gt.migrate.context->engine); + ret = intel_context_migrate_copy(i915->gt.migrate.context, + NULL, src_st->sgl, I915_CACHE_NONE, + bo->mem.mem_type >= I915_PL_LMEM0, + dst_st->sgl, I915_CACHE_NONE, + dst_mem->mem_type >= I915_PL_LMEM0, + &rq); + if (!ret && rq) { + i915_request_wait(rq, 0, MAX_SCHEDULE_TIMEOUT); + i915_request_put(rq); + } + intel_engine_pm_put(i915->gt.migrate.context->engine); + } + + return ret; +} + static int i915_ttm_move(struct ttm_buffer_object *bo, bool evict, struct ttm_operation_ctx *ctx, struct ttm_resource *dst_mem, @@ -332,19 +391,22 @@ static int i915_ttm_move(struct ttm_buffer_object *bo, bool evict, if (IS_ERR(dst_st)) return PTR_ERR(dst_st); - /* If we start mapping GGTT, we can no longer use man::use_tt here. */ - dst_iter = dst_man->use_tt ? - ttm_kmap_iter_tt_init(&_dst_iter.tt, bo->ttm) : - ttm_kmap_iter_iomap_init(&_dst_iter.io, &dst_reg->iomap, - dst_st, dst_reg->region.start); - - src_iter = src_man->use_tt ? - ttm_kmap_iter_tt_init(&_src_iter.tt, bo->ttm) : - ttm_kmap_iter_iomap_init(&_src_iter.io, &src_reg->iomap, - obj->ttm.cached_io_st, - src_reg->region.start); - - ttm_move_memcpy(bo, dst_mem->num_pages, dst_iter, src_iter); + ret = i915_ttm_accel_move(bo, dst_mem, dst_st); + if (ret) { + /* If we start mapping GGTT, we can no longer use man::use_tt here. */ + dst_iter = dst_man->use_tt ? + ttm_kmap_iter_tt_init(&_dst_iter.tt, bo->ttm) : + ttm_kmap_iter_iomap_init(&_dst_iter.io, &dst_reg->iomap, + dst_st, dst_reg->region.start); + + src_iter = src_man->use_tt ? + ttm_kmap_iter_tt_init(&_src_iter.tt, bo->ttm) : + ttm_kmap_iter_iomap_init(&_src_iter.io, &src_reg->iomap, + obj->ttm.cached_io_st, + src_reg->region.start); + + ttm_move_memcpy(bo, dst_mem->num_pages, dst_iter, src_iter); + } ttm_bo_move_sync_cleanup(bo, dst_mem); i915_ttm_free_cached_io_st(obj); -- 2.20.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35742C48BDF for ; Tue, 15 Jun 2021 10:05:00 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D7B8B61428 for ; Tue, 15 Jun 2021 10:04:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D7B8B61428 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 864A76E22B; Tue, 15 Jun 2021 10:04:59 +0000 (UTC) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6823F6E22B; Tue, 15 Jun 2021 10:04:58 +0000 (UTC) IronPort-SDR: Bp1Xg9ALpj+aIl4qF06rWzW38OWvYuxBLQoEEHry8hgtham6tXXEmBpH7/0WC/yGMU6c3VwXAy DeQq/w522Zcg== X-IronPort-AV: E=McAfee;i="6200,9189,10015"; a="202936426" X-IronPort-AV: E=Sophos;i="5.83,275,1616482800"; d="scan'208";a="202936426" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Jun 2021 03:04:55 -0700 IronPort-SDR: +UjwLZMwunbmfgGDXoNm+iFrmQijOgNiP1LX8apgWcvQfmvrXHabfvL45l0MT0PkvY3Jihzzrf gFIjaeObAVEw== X-IronPort-AV: E=Sophos;i="5.83,275,1616482800"; d="scan'208";a="421076013" Received: from ramaling-i9x.iind.intel.com ([10.99.66.205]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Jun 2021 03:04:53 -0700 From: Ramalingam C To: intel-gfx , dri-devel Date: Tue, 15 Jun 2021 15:36:21 +0530 Message-Id: <20210615100621.5805-1-ramalingam.c@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <0e309744-4d13-69de-2ad8-ee0df5cd46c1@linux.intel.com> References: <0e309744-4d13-69de-2ad8-ee0df5cd46c1@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v3] drm/i915/ttm: accelerated move implementation X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Matthew Auld , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" SW52b2tlcyB0aGUgcGlwZWxpbmVkIHBhZ2UgbWlncmF0aW9uIHRocm91Z2ggYmx0LCBmb3IKaTkx NV90dG1fbW92ZSByZXF1ZXN0cyBvZiBldmljdGlvbiBhbmQgYWxzbyBvYmogY2xlYXIuCgp2MjoK IC0gc3ViZnVuY3Rpb24gZm9yIGFjY2VsX21vdmUgKFRob21hcykKIC0gZW5naW5lX3BtX2dldC9w dXQgYXJvdW5kIGNvbnRleHRfbW92ZS9jbGVhciAoVGhvbWFzKQogLSBJbnZhbGlkYXRpb24gYXQg YWNjZWxfY2xlYXIgKFRob21hcykKCnYzOgogLSBUaW1lb3V0IGlzIHNldCBmb3IgTUFYX1NDSEVE VUxFX1RJTUVPVVQgKFRob21hcykKIC0gcy9UVE1fUExfUFJJVi9JOTE1X1BMX0xNRU0wIChUaG9t YXMpCgpTaWduZWQtb2ZmLWJ5OiBSYW1hbGluZ2FtIEMgPHJhbWFsaW5nYW0uY0BpbnRlbC5jb20+ ClJldmlld2VkLWJ5OiBUaG9tYXMgSGVsbHN0csO2bSA8dGhvbWFzLmhlbGxzdHJvbUBsaW51eC5p bnRlbC5jb20+Ci0tLQogZHJpdmVycy9ncHUvZHJtL2k5MTUvZ2VtL2k5MTVfZ2VtX3R0bS5jIHwg ODggKysrKysrKysrKysrKysrKysrKysrLS0tLQogMSBmaWxlIGNoYW5nZWQsIDc1IGluc2VydGlv bnMoKyksIDEzIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9pOTE1 L2dlbS9pOTE1X2dlbV90dG0uYyBiL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2dlbS9pOTE1X2dlbV90 dG0uYwppbmRleCAzNzQ4MDk4YjQyZDUuLjk0NTcxNzU3ZmI0MiAxMDA2NDQKLS0tIGEvZHJpdmVy cy9ncHUvZHJtL2k5MTUvZ2VtL2k5MTVfZ2VtX3R0bS5jCisrKyBiL2RyaXZlcnMvZ3B1L2RybS9p OTE1L2dlbS9pOTE1X2dlbV90dG0uYwpAQCAtMTUsNiArMTUsOSBAQAogI2luY2x1ZGUgImdlbS9p OTE1X2dlbV90dG0uaCIKICNpbmNsdWRlICJnZW0vaTkxNV9nZW1fbW1hbi5oIgogCisjaW5jbHVk ZSAiZ3QvaW50ZWxfbWlncmF0ZS5oIgorI2luY2x1ZGUgImd0L2ludGVsX2VuZ2luZV9wbS5oIgor CiAjZGVmaW5lIEk5MTVfUExfTE1FTTAgVFRNX1BMX1BSSVYKICNkZWZpbmUgSTkxNV9QTF9TWVNU RU0gVFRNX1BMX1NZU1RFTQogI2RlZmluZSBJOTE1X1BMX1NUT0xFTiBUVE1fUExfVlJBTQpAQCAt MjgyLDYgKzI4NSw2MiBAQCBpOTE1X3R0bV9yZXNvdXJjZV9nZXRfc3Qoc3RydWN0IGRybV9pOTE1 X2dlbV9vYmplY3QgKm9iaiwKIAlyZXR1cm4gaW50ZWxfcmVnaW9uX3R0bV9ub2RlX3RvX3N0KG9i ai0+bW0ucmVnaW9uLCByZXMtPm1tX25vZGUpOwogfQogCitzdGF0aWMgaW50IGk5MTVfdHRtX2Fj Y2VsX21vdmUoc3RydWN0IHR0bV9idWZmZXJfb2JqZWN0ICpibywKKwkJCSAgICAgICBzdHJ1Y3Qg dHRtX3Jlc291cmNlICpkc3RfbWVtLAorCQkJICAgICAgIHN0cnVjdCBzZ190YWJsZSAqZHN0X3N0 KQoreworCXN0cnVjdCBkcm1faTkxNV9wcml2YXRlICppOTE1ID0gY29udGFpbmVyX29mKGJvLT5i ZGV2LCB0eXBlb2YoKmk5MTUpLAorCQkJCQkJICAgICBiZGV2KTsKKwlzdHJ1Y3QgdHRtX3Jlc291 cmNlX21hbmFnZXIgKnNyY19tYW4gPQorCQl0dG1fbWFuYWdlcl90eXBlKGJvLT5iZGV2LCBiby0+ bWVtLm1lbV90eXBlKTsKKwlzdHJ1Y3QgZHJtX2k5MTVfZ2VtX29iamVjdCAqb2JqID0gaTkxNV90 dG1fdG9fZ2VtKGJvKTsKKwlzdHJ1Y3Qgc2dfdGFibGUgKnNyY19zdDsKKwlzdHJ1Y3QgaTkxNV9y ZXF1ZXN0ICpycTsKKwlpbnQgcmV0OworCisJaWYgKCFpOTE1LT5ndC5taWdyYXRlLmNvbnRleHQp CisJCXJldHVybiAtRUlOVkFMOworCisJaWYgKCFiby0+dHRtIHx8ICF0dG1fdHRfaXNfcG9wdWxh dGVkKGJvLT50dG0pKSB7CisJCWlmIChiby0+dHlwZSA9PSB0dG1fYm9fdHlwZV9rZXJuZWwpCisJ CQlyZXR1cm4gLUVJTlZBTDsKKworCQlpZiAoYm8tPnR0bSAmJgorCQkgICAgIShiby0+dHRtLT5w YWdlX2ZsYWdzICYgVFRNX1BBR0VfRkxBR19aRVJPX0FMTE9DKSkKKwkJCXJldHVybiAwOworCisJ CWludGVsX2VuZ2luZV9wbV9nZXQoaTkxNS0+Z3QubWlncmF0ZS5jb250ZXh0LT5lbmdpbmUpOwor CQlyZXQgPSBpbnRlbF9jb250ZXh0X21pZ3JhdGVfY2xlYXIoaTkxNS0+Z3QubWlncmF0ZS5jb250 ZXh0LCBOVUxMLAorCQkJCQkJICBkc3Rfc3QtPnNnbCwgSTkxNV9DQUNIRV9OT05FLAorCQkJCQkJ ICBkc3RfbWVtLT5tZW1fdHlwZSA+PSBJOTE1X1BMX0xNRU0wLAorCQkJCQkJICAwLCAmcnEpOwor CisJCWlmICghcmV0ICYmIHJxKSB7CisJCQlpOTE1X3JlcXVlc3Rfd2FpdChycSwgMCwgTUFYX1ND SEVEVUxFX1RJTUVPVVQpOworCQkJaTkxNV9yZXF1ZXN0X3B1dChycSk7CisJCX0KKwkJaW50ZWxf ZW5naW5lX3BtX3B1dChpOTE1LT5ndC5taWdyYXRlLmNvbnRleHQtPmVuZ2luZSk7CisJfSBlbHNl IHsKKwkJc3JjX3N0ID0gc3JjX21hbi0+dXNlX3R0ID8gaTkxNV90dG1fdHRfZ2V0X3N0KGJvLT50 dG0pIDoKKwkJCQkJCW9iai0+dHRtLmNhY2hlZF9pb19zdDsKKworCQlpbnRlbF9lbmdpbmVfcG1f Z2V0KGk5MTUtPmd0Lm1pZ3JhdGUuY29udGV4dC0+ZW5naW5lKTsKKwkJcmV0ID0gaW50ZWxfY29u dGV4dF9taWdyYXRlX2NvcHkoaTkxNS0+Z3QubWlncmF0ZS5jb250ZXh0LAorCQkJCQkJIE5VTEws IHNyY19zdC0+c2dsLCBJOTE1X0NBQ0hFX05PTkUsCisJCQkJCQkgYm8tPm1lbS5tZW1fdHlwZSA+ PSBJOTE1X1BMX0xNRU0wLAorCQkJCQkJIGRzdF9zdC0+c2dsLCBJOTE1X0NBQ0hFX05PTkUsCisJ CQkJCQkgZHN0X21lbS0+bWVtX3R5cGUgPj0gSTkxNV9QTF9MTUVNMCwKKwkJCQkJCSAmcnEpOwor CQlpZiAoIXJldCAmJiBycSkgeworCQkJaTkxNV9yZXF1ZXN0X3dhaXQocnEsIDAsIE1BWF9TQ0hF RFVMRV9USU1FT1VUKTsKKwkJCWk5MTVfcmVxdWVzdF9wdXQocnEpOworCQl9CisJCWludGVsX2Vu Z2luZV9wbV9wdXQoaTkxNS0+Z3QubWlncmF0ZS5jb250ZXh0LT5lbmdpbmUpOworCX0KKworCXJl dHVybiByZXQ7Cit9CisKIHN0YXRpYyBpbnQgaTkxNV90dG1fbW92ZShzdHJ1Y3QgdHRtX2J1ZmZl cl9vYmplY3QgKmJvLCBib29sIGV2aWN0LAogCQkJIHN0cnVjdCB0dG1fb3BlcmF0aW9uX2N0eCAq Y3R4LAogCQkJIHN0cnVjdCB0dG1fcmVzb3VyY2UgKmRzdF9tZW0sCkBAIC0zMzIsMTkgKzM5MSwy MiBAQCBzdGF0aWMgaW50IGk5MTVfdHRtX21vdmUoc3RydWN0IHR0bV9idWZmZXJfb2JqZWN0ICpi bywgYm9vbCBldmljdCwKIAlpZiAoSVNfRVJSKGRzdF9zdCkpCiAJCXJldHVybiBQVFJfRVJSKGRz dF9zdCk7CiAKLQkvKiBJZiB3ZSBzdGFydCBtYXBwaW5nIEdHVFQsIHdlIGNhbiBubyBsb25nZXIg dXNlIG1hbjo6dXNlX3R0IGhlcmUuICovCi0JZHN0X2l0ZXIgPSBkc3RfbWFuLT51c2VfdHQgPwot CQl0dG1fa21hcF9pdGVyX3R0X2luaXQoJl9kc3RfaXRlci50dCwgYm8tPnR0bSkgOgotCQl0dG1f a21hcF9pdGVyX2lvbWFwX2luaXQoJl9kc3RfaXRlci5pbywgJmRzdF9yZWctPmlvbWFwLAotCQkJ CQkgZHN0X3N0LCBkc3RfcmVnLT5yZWdpb24uc3RhcnQpOwotCi0Jc3JjX2l0ZXIgPSBzcmNfbWFu LT51c2VfdHQgPwotCQl0dG1fa21hcF9pdGVyX3R0X2luaXQoJl9zcmNfaXRlci50dCwgYm8tPnR0 bSkgOgotCQl0dG1fa21hcF9pdGVyX2lvbWFwX2luaXQoJl9zcmNfaXRlci5pbywgJnNyY19yZWct PmlvbWFwLAotCQkJCQkgb2JqLT50dG0uY2FjaGVkX2lvX3N0LAotCQkJCQkgc3JjX3JlZy0+cmVn aW9uLnN0YXJ0KTsKLQotCXR0bV9tb3ZlX21lbWNweShibywgZHN0X21lbS0+bnVtX3BhZ2VzLCBk c3RfaXRlciwgc3JjX2l0ZXIpOworCXJldCA9IGk5MTVfdHRtX2FjY2VsX21vdmUoYm8sIGRzdF9t ZW0sIGRzdF9zdCk7CisJaWYgKHJldCkgeworCQkvKiBJZiB3ZSBzdGFydCBtYXBwaW5nIEdHVFQs IHdlIGNhbiBubyBsb25nZXIgdXNlIG1hbjo6dXNlX3R0IGhlcmUuICovCisJCWRzdF9pdGVyID0g ZHN0X21hbi0+dXNlX3R0ID8KKwkJCXR0bV9rbWFwX2l0ZXJfdHRfaW5pdCgmX2RzdF9pdGVyLnR0 LCBiby0+dHRtKSA6CisJCQl0dG1fa21hcF9pdGVyX2lvbWFwX2luaXQoJl9kc3RfaXRlci5pbywg JmRzdF9yZWctPmlvbWFwLAorCQkJCQkJIGRzdF9zdCwgZHN0X3JlZy0+cmVnaW9uLnN0YXJ0KTsK KworCQlzcmNfaXRlciA9IHNyY19tYW4tPnVzZV90dCA/CisJCQl0dG1fa21hcF9pdGVyX3R0X2lu aXQoJl9zcmNfaXRlci50dCwgYm8tPnR0bSkgOgorCQkJdHRtX2ttYXBfaXRlcl9pb21hcF9pbml0 KCZfc3JjX2l0ZXIuaW8sICZzcmNfcmVnLT5pb21hcCwKKwkJCQkJCSBvYmotPnR0bS5jYWNoZWRf aW9fc3QsCisJCQkJCQkgc3JjX3JlZy0+cmVnaW9uLnN0YXJ0KTsKKworCQl0dG1fbW92ZV9tZW1j cHkoYm8sIGRzdF9tZW0tPm51bV9wYWdlcywgZHN0X2l0ZXIsIHNyY19pdGVyKTsKKwl9CiAJdHRt X2JvX21vdmVfc3luY19jbGVhbnVwKGJvLCBkc3RfbWVtKTsKIAlpOTE1X3R0bV9mcmVlX2NhY2hl ZF9pb19zdChvYmopOwogCi0tIAoyLjIwLjEKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fCkludGVsLWdmeCBtYWlsaW5nIGxpc3QKSW50ZWwtZ2Z4QGxpc3Rz LmZyZWVkZXNrdG9wLm9yZwpodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xp c3RpbmZvL2ludGVsLWdmeAo=