From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 470C5C433B4 for ; Fri, 16 Apr 2021 16:23:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1B46D61002 for ; Fri, 16 Apr 2021 16:23:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235108AbhDPQXr (ORCPT ); Fri, 16 Apr 2021 12:23:47 -0400 Received: from mx2.suse.de ([195.135.220.15]:51150 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234323AbhDPQXq (ORCPT ); Fri, 16 Apr 2021 12:23:46 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 7C2E4B229; Fri, 16 Apr 2021 16:23:20 +0000 (UTC) To: Mike Snitzer Cc: Christoph Hellwig , Jens Axboe , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org References: <20210415231530.95464-1-snitzer@redhat.com> <20210415231530.95464-4-snitzer@redhat.com> <6185100e-89e6-0a7f-8901-9ce86fe8f1ac@suse.de> <20210416150301.GC16047@redhat.com> From: Hannes Reinecke Organization: SUSE Linux GmbH Subject: Re: [PATCH v2 3/4] nvme: introduce FAILUP handling for REQ_FAILFAST_TRANSPORT Message-ID: Date: Fri, 16 Apr 2021 18:23:20 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: <20210416150301.GC16047@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On 4/16/21 5:03 PM, Mike Snitzer wrote: > On Fri, Apr 16 2021 at 10:07am -0400, > Hannes Reinecke wrote: > >> On 4/16/21 1:15 AM, Mike Snitzer wrote: >>> If REQ_FAILFAST_TRANSPORT is set it means the driver should not retry >>> IO that completed with transport errors. REQ_FAILFAST_TRANSPORT is >>> set by multipathing software (e.g. dm-multipath) before it issues IO. >>> >>> Update NVMe to allow failover of requests marked with either >>> REQ_NVME_MPATH or REQ_FAILFAST_TRANSPORT. This allows such requests >>> to be given a disposition of either FAILOVER or FAILUP respectively. >>> FAILUP handling ensures a retryable error is returned up from NVMe. >>> >>> Introduce nvme_failup_req() for use in nvme_complete_rq() if >>> nvme_decide_disposition() returns FAILUP. nvme_failup_req() ensures >>> the request is completed with a retryable IO error when appropriate. >>> __nvme_end_req() was factored out for use by both nvme_end_req() and >>> nvme_failup_req(). >>> >>> Signed-off-by: Mike Snitzer >>> --- >>> drivers/nvme/host/core.c | 31 ++++++++++++++++++++++++++----- >>> 1 file changed, 26 insertions(+), 5 deletions(-) >>> >>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c >>> index 4134cf3c7e48..10375197dd53 100644 >>> --- a/drivers/nvme/host/core.c >>> +++ b/drivers/nvme/host/core.c >>> @@ -299,6 +299,7 @@ enum nvme_disposition { >>> COMPLETE, >>> RETRY, >>> FAILOVER, >>> + FAILUP, >>> }; >>> >>> static inline enum nvme_disposition nvme_decide_disposition(struct request *req) >>> @@ -318,10 +319,11 @@ static inline enum nvme_disposition nvme_decide_disposition(struct request *req) >>> nvme_req(req)->retries >= nvme_max_retries) >>> return COMPLETE; >>> >>> - if (req->cmd_flags & REQ_NVME_MPATH) { >>> + if (req->cmd_flags & (REQ_NVME_MPATH | REQ_FAILFAST_TRANSPORT)) { >>> if (nvme_is_path_error(nvme_req(req)->status) || >>> blk_queue_dying(req->q)) >>> - return FAILOVER; >>> + return (req->cmd_flags & REQ_NVME_MPATH) ? >>> + FAILOVER : FAILUP; >>> } else { >>> if (blk_queue_dying(req->q)) >>> return COMPLETE; >>> @@ -330,10 +332,8 @@ static inline enum nvme_disposition nvme_decide_disposition(struct request *req) >>> return RETRY; >>> } >>> >>> -static inline void nvme_end_req(struct request *req) >>> +static inline void __nvme_end_req(struct request *req, blk_status_t status) >>> { >>> - blk_status_t status = nvme_error_status(nvme_req(req)->status); >>> - >>> if (IS_ENABLED(CONFIG_BLK_DEV_ZONED) && >>> req_op(req) == REQ_OP_ZONE_APPEND) >>> req->__sector = nvme_lba_to_sect(req->q->queuedata, >>> @@ -343,6 +343,24 @@ static inline void nvme_end_req(struct request *req) >>> blk_mq_end_request(req, status); >>> } >>> >>> +static inline void nvme_end_req(struct request *req) >>> +{ >>> + __nvme_end_req(req, nvme_error_status(nvme_req(req)->status)); >>> +} >>> + >>> +static void nvme_failup_req(struct request *req) >>> +{ >>> + blk_status_t status = nvme_error_status(nvme_req(req)->status); >>> + >>> + if (WARN_ON_ONCE(!blk_path_error(status))) { >>> + pr_debug("Request meant for failover but blk_status_t (errno=%d) was not retryable.\n", >>> + blk_status_to_errno(status)); >>> + status = BLK_STS_IOERR; >>> + } >>> + >>> + __nvme_end_req(req, status); >>> +} >>> + >>> void nvme_complete_rq(struct request *req) >>> { >>> trace_nvme_complete_rq(req); >>> @@ -361,6 +379,9 @@ void nvme_complete_rq(struct request *req) >>> case FAILOVER: >>> nvme_failover_req(req); >>> return; >>> + case FAILUP: >>> + nvme_failup_req(req); >>> + return; >>> } >>> } >>> EXPORT_SYMBOL_GPL(nvme_complete_rq); >>> >> >> Hmm. Quite convoluted, methinks. > > Maybe you didn't read the header or patch? > > I'm cool with critical review when it is clear the reviewer fully > understands the patch but... ;) > >> Shouldn't this achieve the same thing? >> >> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c >> index e89ec2522ca6..8c36a2196b66 100644 >> --- a/drivers/nvme/host/core.c >> +++ b/drivers/nvme/host/core.c >> @@ -303,8 +303,10 @@ static inline enum nvme_disposition >> nvme_decide_disposition(struct request *req) >> if (likely(nvme_req(req)->status == 0)) >> return COMPLETE; >> >> - if (blk_noretry_request(req) || >> - (nvme_req(req)->status & NVME_SC_DNR) || >> + if (blk_noretry_request(req)) >> + nvme_req(req)->status |= NVME_SC_DNR; >> + >> + if ((nvme_req(req)->status & NVME_SC_DNR) || >> nvme_req(req)->retries >= nvme_max_retries) >> return COMPLETE; > > Definitely won't achieve the same. And especially not with patch 1/4 > ("nvme: return BLK_STS_DO_NOT_RETRY if the DNR bit is set") that you > gave your Reviewed-by to earlier. > Ah. Right. Sorry. > Instead of "FAILUP", I thought about using "FAILUP_AND_OVER" to convey > that this is a variant of failover. Meaning it takes the same patch as > nvme "FAILOVER" until the very end; where it does REQ_FAILFAST_TRANSPORT > specific work detailed in nvme_failup_req(). > All very intricate; will need to check the patches in their combined version. Not deliberately stalling, mind you, just wanting to figure out what the net result will be. Cheers, Hannes -- Dr. Hannes Reinecke Kernel Storage Architect hare@suse.de +49 911 74053 688 SUSE Software Solutions Germany GmbH, 90409 Nürnberg GF: F. Imendörffer, HRB 36809 (AG Nürnberg) From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D485AC433B4 for ; Fri, 16 Apr 2021 16:23:48 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 314FA611AE for ; Fri, 16 Apr 2021 16:23:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 314FA611AE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:Subject: From:References:Cc:To:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=i4rYfxjy758O0d+e85xTdrWOhxYlAK2b/dGdZtAB7lY=; b=Mtvw0SaJlPvfdTaIPN0pg12ao tndbDphzx5ofCJW5bs8nPMKVGQgJf8hciTwbK3qQ/r2IIQWW6KeOHxoR4d9pT4G8RMIbGPqK/FoUK RkRY0Bvmzjsn9pL8TlFvxD0GcZ4tKILrk44fHXog5MML7B7TGmZsaXDifHyi8bk5AQBX9aZJ0bDmQ WESSqtWdHiE5FIRgkMYbtUPFbpBsagmyGiRmY+dyhYGnpc5ZAB+dchCGjsLNQ6WAN6QAyLddBU051 ni54zHIfh4TZw3ZK5fxyoDaghThyA7goOtkq3M2TzWZrCKY74DaWfr/CV16I39cd24Zqy4HUhwrcH QQQ3HBxDg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lXRFj-002lcp-99; Fri, 16 Apr 2021 16:23:31 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lXRFe-002lcH-6W for linux-nvme@desiato.infradead.org; Fri, 16 Apr 2021 16:23:26 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:Subject:From:References :Cc:To:Sender:Reply-To:Content-ID:Content-Description; bh=5m/5uwegQfHgGuoU3hXo2ZKGt28W9t7YDj/WiXZnKPk=; b=RoB6/3lTp21u50pVT8IzaNRlZC 54eX2hvu3UqnTPX4xqxXctEZz7rxcDkwxTH0cXwSq3maOUBePlDHZYldr2d5vex+JVH6QXxjiXOpP kxM4kfrVSNowC4rrPX1ytmlZ/iqof4wEiJOWBjTJHHlhGbh41EIkyxGHxUzdR9pevaZ4CqGJW0Fyd Oe5147M5m5X8r3gikHVv92PP6HVtKLslXokZ0l7xyeaHAI5FqwHj8aa2oFzuNHz/d0iX76QVThZ4F bfdAbgAec4cn8YzFL9/snKqVdAOFvzY/h2cmYcj5lj3jbs8u+Pyj997WziNExmMN9t2VL7gvJPmIf dfL1CphA==; Received: from mx2.suse.de ([195.135.220.15]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lXRFb-009YEl-4c for linux-nvme@lists.infradead.org; Fri, 16 Apr 2021 16:23:24 +0000 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 7C2E4B229; Fri, 16 Apr 2021 16:23:20 +0000 (UTC) To: Mike Snitzer Cc: Christoph Hellwig , Jens Axboe , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org References: <20210415231530.95464-1-snitzer@redhat.com> <20210415231530.95464-4-snitzer@redhat.com> <6185100e-89e6-0a7f-8901-9ce86fe8f1ac@suse.de> <20210416150301.GC16047@redhat.com> From: Hannes Reinecke Organization: SUSE Linux GmbH Subject: Re: [PATCH v2 3/4] nvme: introduce FAILUP handling for REQ_FAILFAST_TRANSPORT Message-ID: Date: Fri, 16 Apr 2021 18:23:20 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: <20210416150301.GC16047@redhat.com> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210416_092323_501872_40EAAD19 X-CRM114-Status: GOOD ( 26.83 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org T24gNC8xNi8yMSA1OjAzIFBNLCBNaWtlIFNuaXR6ZXIgd3JvdGU6Cj4gT24gRnJpLCBBcHIgMTYg MjAyMSBhdCAxMDowN2FtIC0wNDAwLAo+IEhhbm5lcyBSZWluZWNrZSA8aGFyZUBzdXNlLmRlPiB3 cm90ZToKPiAKPj4gT24gNC8xNi8yMSAxOjE1IEFNLCBNaWtlIFNuaXR6ZXIgd3JvdGU6Cj4+PiBJ ZiBSRVFfRkFJTEZBU1RfVFJBTlNQT1JUIGlzIHNldCBpdCBtZWFucyB0aGUgZHJpdmVyIHNob3Vs ZCBub3QgcmV0cnkKPj4+IElPIHRoYXQgY29tcGxldGVkIHdpdGggdHJhbnNwb3J0IGVycm9ycy4g UkVRX0ZBSUxGQVNUX1RSQU5TUE9SVCBpcwo+Pj4gc2V0IGJ5IG11bHRpcGF0aGluZyBzb2Z0d2Fy ZSAoZS5nLiBkbS1tdWx0aXBhdGgpIGJlZm9yZSBpdCBpc3N1ZXMgSU8uCj4+Pgo+Pj4gVXBkYXRl IE5WTWUgdG8gYWxsb3cgZmFpbG92ZXIgb2YgcmVxdWVzdHMgbWFya2VkIHdpdGggZWl0aGVyCj4+ PiBSRVFfTlZNRV9NUEFUSCBvciBSRVFfRkFJTEZBU1RfVFJBTlNQT1JULiBUaGlzIGFsbG93cyBz dWNoIHJlcXVlc3RzCj4+PiB0byBiZSBnaXZlbiBhIGRpc3Bvc2l0aW9uIG9mIGVpdGhlciBGQUlM T1ZFUiBvciBGQUlMVVAgcmVzcGVjdGl2ZWx5Lgo+Pj4gRkFJTFVQIGhhbmRsaW5nIGVuc3VyZXMg YSByZXRyeWFibGUgZXJyb3IgaXMgcmV0dXJuZWQgdXAgZnJvbSBOVk1lLgo+Pj4KPj4+IEludHJv ZHVjZSBudm1lX2ZhaWx1cF9yZXEoKSBmb3IgdXNlIGluIG52bWVfY29tcGxldGVfcnEoKSBpZgo+ Pj4gbnZtZV9kZWNpZGVfZGlzcG9zaXRpb24oKSByZXR1cm5zIEZBSUxVUC4gbnZtZV9mYWlsdXBf cmVxKCkgZW5zdXJlcwo+Pj4gdGhlIHJlcXVlc3QgaXMgY29tcGxldGVkIHdpdGggYSByZXRyeWFi bGUgSU8gZXJyb3Igd2hlbiBhcHByb3ByaWF0ZS4KPj4+IF9fbnZtZV9lbmRfcmVxKCkgd2FzIGZh Y3RvcmVkIG91dCBmb3IgdXNlIGJ5IGJvdGggbnZtZV9lbmRfcmVxKCkgYW5kCj4+PiBudm1lX2Zh aWx1cF9yZXEoKS4KPj4+Cj4+PiBTaWduZWQtb2ZmLWJ5OiBNaWtlIFNuaXR6ZXIgPHNuaXR6ZXJA cmVkaGF0LmNvbT4KPj4+IC0tLQo+Pj4gIGRyaXZlcnMvbnZtZS9ob3N0L2NvcmUuYyB8IDMxICsr KysrKysrKysrKysrKysrKysrKysrKysrLS0tLS0KPj4+ICAxIGZpbGUgY2hhbmdlZCwgMjYgaW5z ZXJ0aW9ucygrKSwgNSBkZWxldGlvbnMoLSkKPj4+Cj4+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9u dm1lL2hvc3QvY29yZS5jIGIvZHJpdmVycy9udm1lL2hvc3QvY29yZS5jCj4+PiBpbmRleCA0MTM0 Y2YzYzdlNDguLjEwMzc1MTk3ZGQ1MyAxMDA2NDQKPj4+IC0tLSBhL2RyaXZlcnMvbnZtZS9ob3N0 L2NvcmUuYwo+Pj4gKysrIGIvZHJpdmVycy9udm1lL2hvc3QvY29yZS5jCj4+PiBAQCAtMjk5LDYg KzI5OSw3IEBAIGVudW0gbnZtZV9kaXNwb3NpdGlvbiB7Cj4+PiAgCUNPTVBMRVRFLAo+Pj4gIAlS RVRSWSwKPj4+ICAJRkFJTE9WRVIsCj4+PiArCUZBSUxVUCwKPj4+ICB9Owo+Pj4gIAo+Pj4gIHN0 YXRpYyBpbmxpbmUgZW51bSBudm1lX2Rpc3Bvc2l0aW9uIG52bWVfZGVjaWRlX2Rpc3Bvc2l0aW9u KHN0cnVjdCByZXF1ZXN0ICpyZXEpCj4+PiBAQCAtMzE4LDEwICszMTksMTEgQEAgc3RhdGljIGlu bGluZSBlbnVtIG52bWVfZGlzcG9zaXRpb24gbnZtZV9kZWNpZGVfZGlzcG9zaXRpb24oc3RydWN0 IHJlcXVlc3QgKnJlcSkKPj4+ICAJICAgIG52bWVfcmVxKHJlcSktPnJldHJpZXMgPj0gbnZtZV9t YXhfcmV0cmllcykKPj4+ICAJCXJldHVybiBDT01QTEVURTsKPj4+ICAKPj4+IC0JaWYgKHJlcS0+ Y21kX2ZsYWdzICYgUkVRX05WTUVfTVBBVEgpIHsKPj4+ICsJaWYgKHJlcS0+Y21kX2ZsYWdzICYg KFJFUV9OVk1FX01QQVRIIHwgUkVRX0ZBSUxGQVNUX1RSQU5TUE9SVCkpIHsKPj4+ICAJCWlmIChu dm1lX2lzX3BhdGhfZXJyb3IobnZtZV9yZXEocmVxKS0+c3RhdHVzKSB8fAo+Pj4gIAkJICAgIGJs a19xdWV1ZV9keWluZyhyZXEtPnEpKQo+Pj4gLQkJCXJldHVybiBGQUlMT1ZFUjsKPj4+ICsJCQly ZXR1cm4gKHJlcS0+Y21kX2ZsYWdzICYgUkVRX05WTUVfTVBBVEgpID8KPj4+ICsJCQkJRkFJTE9W RVIgOiBGQUlMVVA7Cj4+PiAgCX0gZWxzZSB7Cj4+PiAgCQlpZiAoYmxrX3F1ZXVlX2R5aW5nKHJl cS0+cSkpCj4+PiAgCQkJcmV0dXJuIENPTVBMRVRFOwo+Pj4gQEAgLTMzMCwxMCArMzMyLDggQEAg c3RhdGljIGlubGluZSBlbnVtIG52bWVfZGlzcG9zaXRpb24gbnZtZV9kZWNpZGVfZGlzcG9zaXRp b24oc3RydWN0IHJlcXVlc3QgKnJlcSkKPj4+ICAJcmV0dXJuIFJFVFJZOwo+Pj4gIH0KPj4+ICAK Pj4+IC1zdGF0aWMgaW5saW5lIHZvaWQgbnZtZV9lbmRfcmVxKHN0cnVjdCByZXF1ZXN0ICpyZXEp Cj4+PiArc3RhdGljIGlubGluZSB2b2lkIF9fbnZtZV9lbmRfcmVxKHN0cnVjdCByZXF1ZXN0ICpy ZXEsIGJsa19zdGF0dXNfdCBzdGF0dXMpCj4+PiAgewo+Pj4gLQlibGtfc3RhdHVzX3Qgc3RhdHVz ID0gbnZtZV9lcnJvcl9zdGF0dXMobnZtZV9yZXEocmVxKS0+c3RhdHVzKTsKPj4+IC0KPj4+ICAJ aWYgKElTX0VOQUJMRUQoQ09ORklHX0JMS19ERVZfWk9ORUQpICYmCj4+PiAgCSAgICByZXFfb3Ao cmVxKSA9PSBSRVFfT1BfWk9ORV9BUFBFTkQpCj4+PiAgCQlyZXEtPl9fc2VjdG9yID0gbnZtZV9s YmFfdG9fc2VjdChyZXEtPnEtPnF1ZXVlZGF0YSwKPj4+IEBAIC0zNDMsNiArMzQzLDI0IEBAIHN0 YXRpYyBpbmxpbmUgdm9pZCBudm1lX2VuZF9yZXEoc3RydWN0IHJlcXVlc3QgKnJlcSkKPj4+ICAJ YmxrX21xX2VuZF9yZXF1ZXN0KHJlcSwgc3RhdHVzKTsKPj4+ICB9Cj4+PiAgCj4+PiArc3RhdGlj IGlubGluZSB2b2lkIG52bWVfZW5kX3JlcShzdHJ1Y3QgcmVxdWVzdCAqcmVxKQo+Pj4gK3sKPj4+ ICsJX19udm1lX2VuZF9yZXEocmVxLCBudm1lX2Vycm9yX3N0YXR1cyhudm1lX3JlcShyZXEpLT5z dGF0dXMpKTsKPj4+ICt9Cj4+PiArCj4+PiArc3RhdGljIHZvaWQgbnZtZV9mYWlsdXBfcmVxKHN0 cnVjdCByZXF1ZXN0ICpyZXEpCj4+PiArewo+Pj4gKwlibGtfc3RhdHVzX3Qgc3RhdHVzID0gbnZt ZV9lcnJvcl9zdGF0dXMobnZtZV9yZXEocmVxKS0+c3RhdHVzKTsKPj4+ICsKPj4+ICsJaWYgKFdB Uk5fT05fT05DRSghYmxrX3BhdGhfZXJyb3Ioc3RhdHVzKSkpIHsKPj4+ICsJCXByX2RlYnVnKCJS ZXF1ZXN0IG1lYW50IGZvciBmYWlsb3ZlciBidXQgYmxrX3N0YXR1c190IChlcnJubz0lZCkgd2Fz IG5vdCByZXRyeWFibGUuXG4iLAo+Pj4gKwkJCSBibGtfc3RhdHVzX3RvX2Vycm5vKHN0YXR1cykp Owo+Pj4gKwkJc3RhdHVzID0gQkxLX1NUU19JT0VSUjsKPj4+ICsJfQo+Pj4gKwo+Pj4gKwlfX252 bWVfZW5kX3JlcShyZXEsIHN0YXR1cyk7Cj4+PiArfQo+Pj4gKwo+Pj4gIHZvaWQgbnZtZV9jb21w bGV0ZV9ycShzdHJ1Y3QgcmVxdWVzdCAqcmVxKQo+Pj4gIHsKPj4+ICAJdHJhY2VfbnZtZV9jb21w bGV0ZV9ycShyZXEpOwo+Pj4gQEAgLTM2MSw2ICszNzksOSBAQCB2b2lkIG52bWVfY29tcGxldGVf cnEoc3RydWN0IHJlcXVlc3QgKnJlcSkKPj4+ICAJY2FzZSBGQUlMT1ZFUjoKPj4+ICAJCW52bWVf ZmFpbG92ZXJfcmVxKHJlcSk7Cj4+PiAgCQlyZXR1cm47Cj4+PiArCWNhc2UgRkFJTFVQOgo+Pj4g KwkJbnZtZV9mYWlsdXBfcmVxKHJlcSk7Cj4+PiArCQlyZXR1cm47Cj4+PiAgCX0KPj4+ICB9Cj4+ PiAgRVhQT1JUX1NZTUJPTF9HUEwobnZtZV9jb21wbGV0ZV9ycSk7Cj4+Pgo+Pgo+PiBIbW0uIFF1 aXRlIGNvbnZvbHV0ZWQsIG1ldGhpbmtzLgo+IAo+IE1heWJlIHlvdSBkaWRuJ3QgcmVhZCB0aGUg aGVhZGVyIG9yIHBhdGNoPwo+IAo+IEknbSBjb29sIHdpdGggY3JpdGljYWwgcmV2aWV3IHdoZW4g aXQgaXMgY2xlYXIgdGhlIHJldmlld2VyIGZ1bGx5Cj4gdW5kZXJzdGFuZHMgdGhlIHBhdGNoIGJ1 dC4uLiA7KQo+IAo+PiBTaG91bGRuJ3QgdGhpcyBhY2hpZXZlIHRoZSBzYW1lIHRoaW5nPwo+Pgo+ PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9udm1lL2hvc3QvY29yZS5jIGIvZHJpdmVycy9udm1lL2hv c3QvY29yZS5jCj4+IGluZGV4IGU4OWVjMjUyMmNhNi4uOGMzNmEyMTk2YjY2IDEwMDY0NAo+PiAt LS0gYS9kcml2ZXJzL252bWUvaG9zdC9jb3JlLmMKPj4gKysrIGIvZHJpdmVycy9udm1lL2hvc3Qv Y29yZS5jCj4+IEBAIC0zMDMsOCArMzAzLDEwIEBAIHN0YXRpYyBpbmxpbmUgZW51bSBudm1lX2Rp c3Bvc2l0aW9uCj4+IG52bWVfZGVjaWRlX2Rpc3Bvc2l0aW9uKHN0cnVjdCByZXF1ZXN0ICpyZXEp Cj4+ICAgICAgICAgaWYgKGxpa2VseShudm1lX3JlcShyZXEpLT5zdGF0dXMgPT0gMCkpCj4+ICAg ICAgICAgICAgICAgICByZXR1cm4gQ09NUExFVEU7Cj4+Cj4+IC0gICAgICAgaWYgKGJsa19ub3Jl dHJ5X3JlcXVlc3QocmVxKSB8fAo+PiAtICAgICAgICAgICAobnZtZV9yZXEocmVxKS0+c3RhdHVz ICYgTlZNRV9TQ19ETlIpIHx8Cj4+ICsgICAgICAgaWYgKGJsa19ub3JldHJ5X3JlcXVlc3QocmVx KSkKPj4gKyAgICAgICAgICAgICAgIG52bWVfcmVxKHJlcSktPnN0YXR1cyB8PSBOVk1FX1NDX0RO UjsKPj4gKwo+PiArICAgICAgIGlmICgobnZtZV9yZXEocmVxKS0+c3RhdHVzICYgTlZNRV9TQ19E TlIpIHx8Cj4+ICAgICAgICAgICAgIG52bWVfcmVxKHJlcSktPnJldHJpZXMgPj0gbnZtZV9tYXhf cmV0cmllcykKPj4gICAgICAgICAgICAgICAgIHJldHVybiBDT01QTEVURTsKPiAKPiBEZWZpbml0 ZWx5IHdvbid0IGFjaGlldmUgdGhlIHNhbWUuIEFuZCBlc3BlY2lhbGx5IG5vdCB3aXRoIHBhdGNo IDEvNAo+ICgibnZtZTogcmV0dXJuIEJMS19TVFNfRE9fTk9UX1JFVFJZIGlmIHRoZSBETlIgYml0 IGlzIHNldCIpIHRoYXQgeW91Cj4gZ2F2ZSB5b3VyIFJldmlld2VkLWJ5IHRvIGVhcmxpZXIuCj4g CkFoLiBSaWdodC4gU29ycnkuCgo+IEluc3RlYWQgb2YgIkZBSUxVUCIsIEkgdGhvdWdodCBhYm91 dCB1c2luZyAiRkFJTFVQX0FORF9PVkVSIiB0byBjb252ZXkKPiB0aGF0IHRoaXMgaXMgYSB2YXJp YW50IG9mIGZhaWxvdmVyLiAgTWVhbmluZyBpdCB0YWtlcyB0aGUgc2FtZSBwYXRjaCBhcwo+IG52 bWUgIkZBSUxPVkVSIiB1bnRpbCB0aGUgdmVyeSBlbmQ7IHdoZXJlIGl0IGRvZXMgUkVRX0ZBSUxG QVNUX1RSQU5TUE9SVAo+IHNwZWNpZmljIHdvcmsgZGV0YWlsZWQgaW4gbnZtZV9mYWlsdXBfcmVx KCkuCj4gCkFsbCB2ZXJ5IGludHJpY2F0ZTsgd2lsbCBuZWVkIHRvIGNoZWNrIHRoZSBwYXRjaGVz IGluIHRoZWlyIGNvbWJpbmVkCnZlcnNpb24uCk5vdCBkZWxpYmVyYXRlbHkgc3RhbGxpbmcsIG1p bmQgeW91LCBqdXN0IHdhbnRpbmcgdG8gZmlndXJlIG91dCB3aGF0IHRoZQpuZXQgcmVzdWx0IHdp bGwgYmUuCgpDaGVlcnMsCgpIYW5uZXMKLS0gCkRyLiBIYW5uZXMgUmVpbmVja2UJCSAgICAgICAg S2VybmVsIFN0b3JhZ2UgQXJjaGl0ZWN0CmhhcmVAc3VzZS5kZQkJCSAgICAgICAgICAgICAgICs0 OSA5MTEgNzQwNTMgNjg4ClNVU0UgU29mdHdhcmUgU29sdXRpb25zIEdlcm1hbnkgR21iSCwgOTA0 MDkgTsO8cm5iZXJnCkdGOiBGLiBJbWVuZMO2cmZmZXIsIEhSQiAzNjgwOSAoQUcgTsO8cm5iZXJn KQoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KTGludXgt bnZtZSBtYWlsaW5nIGxpc3QKTGludXgtbnZtZUBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9s aXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtbnZtZQo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF270C433ED for ; Fri, 16 Apr 2021 16:24:40 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 51DE46137D for ; Fri, 16 Apr 2021 16:24:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 51DE46137D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=tempfail smtp.mailfrom=dm-devel-bounces@redhat.com Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-41-qdvDRdd_MjW8RWp9ku8G4g-1; Fri, 16 Apr 2021 12:24:37 -0400 X-MC-Unique: qdvDRdd_MjW8RWp9ku8G4g-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 80BAD80623B; Fri, 16 Apr 2021 16:24:31 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6C99560C16; Fri, 16 Apr 2021 16:24:30 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 9AE1544A5F; Fri, 16 Apr 2021 16:24:27 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 13GGNStD013622 for ; Fri, 16 Apr 2021 12:23:29 -0400 Received: by smtp.corp.redhat.com (Postfix) id A99D62021463; Fri, 16 Apr 2021 16:23:28 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast06.extmail.prod.ext.rdu2.redhat.com [10.11.55.22]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A4135202146B for ; Fri, 16 Apr 2021 16:23:25 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4588A185A79C for ; Fri, 16 Apr 2021 16:23:25 +0000 (UTC) Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-589-S2UtpQhTOSW_TqefS6-Njw-1; Fri, 16 Apr 2021 12:23:23 -0400 X-MC-Unique: S2UtpQhTOSW_TqefS6-Njw-1 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 7C2E4B229; Fri, 16 Apr 2021 16:23:20 +0000 (UTC) To: Mike Snitzer References: <20210415231530.95464-1-snitzer@redhat.com> <20210415231530.95464-4-snitzer@redhat.com> <6185100e-89e6-0a7f-8901-9ce86fe8f1ac@suse.de> <20210416150301.GC16047@redhat.com> From: Hannes Reinecke Organization: SUSE Linux GmbH Message-ID: Date: Fri, 16 Apr 2021 18:23:20 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: <20210416150301.GC16047@redhat.com> X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 13GGNStD013622 X-loop: dm-devel@redhat.com Cc: Jens Axboe , linux-block@vger.kernel.org, dm-devel@redhat.com, Christoph Hellwig , linux-nvme@lists.infradead.org Subject: Re: [dm-devel] [PATCH v2 3/4] nvme: introduce FAILUP handling for REQ_FAILFAST_TRANSPORT X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dm-devel-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 T24gNC8xNi8yMSA1OjAzIFBNLCBNaWtlIFNuaXR6ZXIgd3JvdGU6Cj4gT24gRnJpLCBBcHIgMTYg MjAyMSBhdCAxMDowN2FtIC0wNDAwLAo+IEhhbm5lcyBSZWluZWNrZSA8aGFyZUBzdXNlLmRlPiB3 cm90ZToKPiAKPj4gT24gNC8xNi8yMSAxOjE1IEFNLCBNaWtlIFNuaXR6ZXIgd3JvdGU6Cj4+PiBJ ZiBSRVFfRkFJTEZBU1RfVFJBTlNQT1JUIGlzIHNldCBpdCBtZWFucyB0aGUgZHJpdmVyIHNob3Vs ZCBub3QgcmV0cnkKPj4+IElPIHRoYXQgY29tcGxldGVkIHdpdGggdHJhbnNwb3J0IGVycm9ycy4g UkVRX0ZBSUxGQVNUX1RSQU5TUE9SVCBpcwo+Pj4gc2V0IGJ5IG11bHRpcGF0aGluZyBzb2Z0d2Fy ZSAoZS5nLiBkbS1tdWx0aXBhdGgpIGJlZm9yZSBpdCBpc3N1ZXMgSU8uCj4+Pgo+Pj4gVXBkYXRl IE5WTWUgdG8gYWxsb3cgZmFpbG92ZXIgb2YgcmVxdWVzdHMgbWFya2VkIHdpdGggZWl0aGVyCj4+ PiBSRVFfTlZNRV9NUEFUSCBvciBSRVFfRkFJTEZBU1RfVFJBTlNQT1JULiBUaGlzIGFsbG93cyBz dWNoIHJlcXVlc3RzCj4+PiB0byBiZSBnaXZlbiBhIGRpc3Bvc2l0aW9uIG9mIGVpdGhlciBGQUlM T1ZFUiBvciBGQUlMVVAgcmVzcGVjdGl2ZWx5Lgo+Pj4gRkFJTFVQIGhhbmRsaW5nIGVuc3VyZXMg YSByZXRyeWFibGUgZXJyb3IgaXMgcmV0dXJuZWQgdXAgZnJvbSBOVk1lLgo+Pj4KPj4+IEludHJv ZHVjZSBudm1lX2ZhaWx1cF9yZXEoKSBmb3IgdXNlIGluIG52bWVfY29tcGxldGVfcnEoKSBpZgo+ Pj4gbnZtZV9kZWNpZGVfZGlzcG9zaXRpb24oKSByZXR1cm5zIEZBSUxVUC4gbnZtZV9mYWlsdXBf cmVxKCkgZW5zdXJlcwo+Pj4gdGhlIHJlcXVlc3QgaXMgY29tcGxldGVkIHdpdGggYSByZXRyeWFi bGUgSU8gZXJyb3Igd2hlbiBhcHByb3ByaWF0ZS4KPj4+IF9fbnZtZV9lbmRfcmVxKCkgd2FzIGZh Y3RvcmVkIG91dCBmb3IgdXNlIGJ5IGJvdGggbnZtZV9lbmRfcmVxKCkgYW5kCj4+PiBudm1lX2Zh aWx1cF9yZXEoKS4KPj4+Cj4+PiBTaWduZWQtb2ZmLWJ5OiBNaWtlIFNuaXR6ZXIgPHNuaXR6ZXJA cmVkaGF0LmNvbT4KPj4+IC0tLQo+Pj4gIGRyaXZlcnMvbnZtZS9ob3N0L2NvcmUuYyB8IDMxICsr KysrKysrKysrKysrKysrKysrKysrKysrLS0tLS0KPj4+ICAxIGZpbGUgY2hhbmdlZCwgMjYgaW5z ZXJ0aW9ucygrKSwgNSBkZWxldGlvbnMoLSkKPj4+Cj4+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9u dm1lL2hvc3QvY29yZS5jIGIvZHJpdmVycy9udm1lL2hvc3QvY29yZS5jCj4+PiBpbmRleCA0MTM0 Y2YzYzdlNDguLjEwMzc1MTk3ZGQ1MyAxMDA2NDQKPj4+IC0tLSBhL2RyaXZlcnMvbnZtZS9ob3N0 L2NvcmUuYwo+Pj4gKysrIGIvZHJpdmVycy9udm1lL2hvc3QvY29yZS5jCj4+PiBAQCAtMjk5LDYg KzI5OSw3IEBAIGVudW0gbnZtZV9kaXNwb3NpdGlvbiB7Cj4+PiAgCUNPTVBMRVRFLAo+Pj4gIAlS RVRSWSwKPj4+ICAJRkFJTE9WRVIsCj4+PiArCUZBSUxVUCwKPj4+ICB9Owo+Pj4gIAo+Pj4gIHN0 YXRpYyBpbmxpbmUgZW51bSBudm1lX2Rpc3Bvc2l0aW9uIG52bWVfZGVjaWRlX2Rpc3Bvc2l0aW9u KHN0cnVjdCByZXF1ZXN0ICpyZXEpCj4+PiBAQCAtMzE4LDEwICszMTksMTEgQEAgc3RhdGljIGlu bGluZSBlbnVtIG52bWVfZGlzcG9zaXRpb24gbnZtZV9kZWNpZGVfZGlzcG9zaXRpb24oc3RydWN0 IHJlcXVlc3QgKnJlcSkKPj4+ICAJICAgIG52bWVfcmVxKHJlcSktPnJldHJpZXMgPj0gbnZtZV9t YXhfcmV0cmllcykKPj4+ICAJCXJldHVybiBDT01QTEVURTsKPj4+ICAKPj4+IC0JaWYgKHJlcS0+ Y21kX2ZsYWdzICYgUkVRX05WTUVfTVBBVEgpIHsKPj4+ICsJaWYgKHJlcS0+Y21kX2ZsYWdzICYg KFJFUV9OVk1FX01QQVRIIHwgUkVRX0ZBSUxGQVNUX1RSQU5TUE9SVCkpIHsKPj4+ICAJCWlmIChu dm1lX2lzX3BhdGhfZXJyb3IobnZtZV9yZXEocmVxKS0+c3RhdHVzKSB8fAo+Pj4gIAkJICAgIGJs a19xdWV1ZV9keWluZyhyZXEtPnEpKQo+Pj4gLQkJCXJldHVybiBGQUlMT1ZFUjsKPj4+ICsJCQly ZXR1cm4gKHJlcS0+Y21kX2ZsYWdzICYgUkVRX05WTUVfTVBBVEgpID8KPj4+ICsJCQkJRkFJTE9W RVIgOiBGQUlMVVA7Cj4+PiAgCX0gZWxzZSB7Cj4+PiAgCQlpZiAoYmxrX3F1ZXVlX2R5aW5nKHJl cS0+cSkpCj4+PiAgCQkJcmV0dXJuIENPTVBMRVRFOwo+Pj4gQEAgLTMzMCwxMCArMzMyLDggQEAg c3RhdGljIGlubGluZSBlbnVtIG52bWVfZGlzcG9zaXRpb24gbnZtZV9kZWNpZGVfZGlzcG9zaXRp b24oc3RydWN0IHJlcXVlc3QgKnJlcSkKPj4+ICAJcmV0dXJuIFJFVFJZOwo+Pj4gIH0KPj4+ICAK Pj4+IC1zdGF0aWMgaW5saW5lIHZvaWQgbnZtZV9lbmRfcmVxKHN0cnVjdCByZXF1ZXN0ICpyZXEp Cj4+PiArc3RhdGljIGlubGluZSB2b2lkIF9fbnZtZV9lbmRfcmVxKHN0cnVjdCByZXF1ZXN0ICpy ZXEsIGJsa19zdGF0dXNfdCBzdGF0dXMpCj4+PiAgewo+Pj4gLQlibGtfc3RhdHVzX3Qgc3RhdHVz ID0gbnZtZV9lcnJvcl9zdGF0dXMobnZtZV9yZXEocmVxKS0+c3RhdHVzKTsKPj4+IC0KPj4+ICAJ aWYgKElTX0VOQUJMRUQoQ09ORklHX0JMS19ERVZfWk9ORUQpICYmCj4+PiAgCSAgICByZXFfb3Ao cmVxKSA9PSBSRVFfT1BfWk9ORV9BUFBFTkQpCj4+PiAgCQlyZXEtPl9fc2VjdG9yID0gbnZtZV9s YmFfdG9fc2VjdChyZXEtPnEtPnF1ZXVlZGF0YSwKPj4+IEBAIC0zNDMsNiArMzQzLDI0IEBAIHN0 YXRpYyBpbmxpbmUgdm9pZCBudm1lX2VuZF9yZXEoc3RydWN0IHJlcXVlc3QgKnJlcSkKPj4+ICAJ YmxrX21xX2VuZF9yZXF1ZXN0KHJlcSwgc3RhdHVzKTsKPj4+ICB9Cj4+PiAgCj4+PiArc3RhdGlj IGlubGluZSB2b2lkIG52bWVfZW5kX3JlcShzdHJ1Y3QgcmVxdWVzdCAqcmVxKQo+Pj4gK3sKPj4+ ICsJX19udm1lX2VuZF9yZXEocmVxLCBudm1lX2Vycm9yX3N0YXR1cyhudm1lX3JlcShyZXEpLT5z dGF0dXMpKTsKPj4+ICt9Cj4+PiArCj4+PiArc3RhdGljIHZvaWQgbnZtZV9mYWlsdXBfcmVxKHN0 cnVjdCByZXF1ZXN0ICpyZXEpCj4+PiArewo+Pj4gKwlibGtfc3RhdHVzX3Qgc3RhdHVzID0gbnZt ZV9lcnJvcl9zdGF0dXMobnZtZV9yZXEocmVxKS0+c3RhdHVzKTsKPj4+ICsKPj4+ICsJaWYgKFdB Uk5fT05fT05DRSghYmxrX3BhdGhfZXJyb3Ioc3RhdHVzKSkpIHsKPj4+ICsJCXByX2RlYnVnKCJS ZXF1ZXN0IG1lYW50IGZvciBmYWlsb3ZlciBidXQgYmxrX3N0YXR1c190IChlcnJubz0lZCkgd2Fz IG5vdCByZXRyeWFibGUuXG4iLAo+Pj4gKwkJCSBibGtfc3RhdHVzX3RvX2Vycm5vKHN0YXR1cykp Owo+Pj4gKwkJc3RhdHVzID0gQkxLX1NUU19JT0VSUjsKPj4+ICsJfQo+Pj4gKwo+Pj4gKwlfX252 bWVfZW5kX3JlcShyZXEsIHN0YXR1cyk7Cj4+PiArfQo+Pj4gKwo+Pj4gIHZvaWQgbnZtZV9jb21w bGV0ZV9ycShzdHJ1Y3QgcmVxdWVzdCAqcmVxKQo+Pj4gIHsKPj4+ICAJdHJhY2VfbnZtZV9jb21w bGV0ZV9ycShyZXEpOwo+Pj4gQEAgLTM2MSw2ICszNzksOSBAQCB2b2lkIG52bWVfY29tcGxldGVf cnEoc3RydWN0IHJlcXVlc3QgKnJlcSkKPj4+ICAJY2FzZSBGQUlMT1ZFUjoKPj4+ICAJCW52bWVf ZmFpbG92ZXJfcmVxKHJlcSk7Cj4+PiAgCQlyZXR1cm47Cj4+PiArCWNhc2UgRkFJTFVQOgo+Pj4g KwkJbnZtZV9mYWlsdXBfcmVxKHJlcSk7Cj4+PiArCQlyZXR1cm47Cj4+PiAgCX0KPj4+ICB9Cj4+ PiAgRVhQT1JUX1NZTUJPTF9HUEwobnZtZV9jb21wbGV0ZV9ycSk7Cj4+Pgo+Pgo+PiBIbW0uIFF1 aXRlIGNvbnZvbHV0ZWQsIG1ldGhpbmtzLgo+IAo+IE1heWJlIHlvdSBkaWRuJ3QgcmVhZCB0aGUg aGVhZGVyIG9yIHBhdGNoPwo+IAo+IEknbSBjb29sIHdpdGggY3JpdGljYWwgcmV2aWV3IHdoZW4g aXQgaXMgY2xlYXIgdGhlIHJldmlld2VyIGZ1bGx5Cj4gdW5kZXJzdGFuZHMgdGhlIHBhdGNoIGJ1 dC4uLiA7KQo+IAo+PiBTaG91bGRuJ3QgdGhpcyBhY2hpZXZlIHRoZSBzYW1lIHRoaW5nPwo+Pgo+ PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9udm1lL2hvc3QvY29yZS5jIGIvZHJpdmVycy9udm1lL2hv c3QvY29yZS5jCj4+IGluZGV4IGU4OWVjMjUyMmNhNi4uOGMzNmEyMTk2YjY2IDEwMDY0NAo+PiAt LS0gYS9kcml2ZXJzL252bWUvaG9zdC9jb3JlLmMKPj4gKysrIGIvZHJpdmVycy9udm1lL2hvc3Qv Y29yZS5jCj4+IEBAIC0zMDMsOCArMzAzLDEwIEBAIHN0YXRpYyBpbmxpbmUgZW51bSBudm1lX2Rp c3Bvc2l0aW9uCj4+IG52bWVfZGVjaWRlX2Rpc3Bvc2l0aW9uKHN0cnVjdCByZXF1ZXN0ICpyZXEp Cj4+ICAgICAgICAgaWYgKGxpa2VseShudm1lX3JlcShyZXEpLT5zdGF0dXMgPT0gMCkpCj4+ICAg ICAgICAgICAgICAgICByZXR1cm4gQ09NUExFVEU7Cj4+Cj4+IC0gICAgICAgaWYgKGJsa19ub3Jl dHJ5X3JlcXVlc3QocmVxKSB8fAo+PiAtICAgICAgICAgICAobnZtZV9yZXEocmVxKS0+c3RhdHVz ICYgTlZNRV9TQ19ETlIpIHx8Cj4+ICsgICAgICAgaWYgKGJsa19ub3JldHJ5X3JlcXVlc3QocmVx KSkKPj4gKyAgICAgICAgICAgICAgIG52bWVfcmVxKHJlcSktPnN0YXR1cyB8PSBOVk1FX1NDX0RO UjsKPj4gKwo+PiArICAgICAgIGlmICgobnZtZV9yZXEocmVxKS0+c3RhdHVzICYgTlZNRV9TQ19E TlIpIHx8Cj4+ICAgICAgICAgICAgIG52bWVfcmVxKHJlcSktPnJldHJpZXMgPj0gbnZtZV9tYXhf cmV0cmllcykKPj4gICAgICAgICAgICAgICAgIHJldHVybiBDT01QTEVURTsKPiAKPiBEZWZpbml0 ZWx5IHdvbid0IGFjaGlldmUgdGhlIHNhbWUuIEFuZCBlc3BlY2lhbGx5IG5vdCB3aXRoIHBhdGNo IDEvNAo+ICgibnZtZTogcmV0dXJuIEJMS19TVFNfRE9fTk9UX1JFVFJZIGlmIHRoZSBETlIgYml0 IGlzIHNldCIpIHRoYXQgeW91Cj4gZ2F2ZSB5b3VyIFJldmlld2VkLWJ5IHRvIGVhcmxpZXIuCj4g CkFoLiBSaWdodC4gU29ycnkuCgo+IEluc3RlYWQgb2YgIkZBSUxVUCIsIEkgdGhvdWdodCBhYm91 dCB1c2luZyAiRkFJTFVQX0FORF9PVkVSIiB0byBjb252ZXkKPiB0aGF0IHRoaXMgaXMgYSB2YXJp YW50IG9mIGZhaWxvdmVyLiAgTWVhbmluZyBpdCB0YWtlcyB0aGUgc2FtZSBwYXRjaCBhcwo+IG52 bWUgIkZBSUxPVkVSIiB1bnRpbCB0aGUgdmVyeSBlbmQ7IHdoZXJlIGl0IGRvZXMgUkVRX0ZBSUxG QVNUX1RSQU5TUE9SVAo+IHNwZWNpZmljIHdvcmsgZGV0YWlsZWQgaW4gbnZtZV9mYWlsdXBfcmVx KCkuCj4gCkFsbCB2ZXJ5IGludHJpY2F0ZTsgd2lsbCBuZWVkIHRvIGNoZWNrIHRoZSBwYXRjaGVz IGluIHRoZWlyIGNvbWJpbmVkCnZlcnNpb24uCk5vdCBkZWxpYmVyYXRlbHkgc3RhbGxpbmcsIG1p bmQgeW91LCBqdXN0IHdhbnRpbmcgdG8gZmlndXJlIG91dCB3aGF0IHRoZQpuZXQgcmVzdWx0IHdp bGwgYmUuCgpDaGVlcnMsCgpIYW5uZXMKLS0gCkRyLiBIYW5uZXMgUmVpbmVja2UJCSAgICAgICAg S2VybmVsIFN0b3JhZ2UgQXJjaGl0ZWN0CmhhcmVAc3VzZS5kZQkJCSAgICAgICAgICAgICAgICs0 OSA5MTEgNzQwNTMgNjg4ClNVU0UgU29mdHdhcmUgU29sdXRpb25zIEdlcm1hbnkgR21iSCwgOTA0 MDkgTsO8cm5iZXJnCkdGOiBGLiBJbWVuZMO2cmZmZXIsIEhSQiAzNjgwOSAoQUcgTsO8cm5iZXJn KQoKCi0tCmRtLWRldmVsIG1haWxpbmcgbGlzdApkbS1kZXZlbEByZWRoYXQuY29tCmh0dHBzOi8v bGlzdG1hbi5yZWRoYXQuY29tL21haWxtYW4vbGlzdGluZm8vZG0tZGV2ZWw=