From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D891C433B4 for ; Thu, 8 Apr 2021 08:46:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5A02A61176 for ; Thu, 8 Apr 2021 08:46:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229741AbhDHIqY (ORCPT ); Thu, 8 Apr 2021 04:46:24 -0400 Received: from mx2.suse.de ([195.135.220.15]:43346 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229566AbhDHIqY (ORCPT ); Thu, 8 Apr 2021 04:46:24 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 7A075AE58; Thu, 8 Apr 2021 08:46:12 +0000 (UTC) Subject: Re: [PATCH v9 13/13] lpfc: vmid: Introducing vmid in io path. To: Muneendra , linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, tj@kernel.org, linux-nvme@lists.infradead.org Cc: jsmart2021@gmail.com, emilne@redhat.com, mkumar@redhat.com, Gaurav Srivastava References: <1617750397-26466-1-git-send-email-muneendra.kumar@broadcom.com> <1617750397-26466-14-git-send-email-muneendra.kumar@broadcom.com> From: Hannes Reinecke Message-ID: <91b0c309-6908-8fd9-ac60-a8572500c3ed@suse.de> Date: Thu, 8 Apr 2021 10:46:11 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: <1617750397-26466-14-git-send-email-muneendra.kumar@broadcom.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On 4/7/21 1:06 AM, Muneendra wrote: > From: Gaurav Srivastava > > The patch introduces the vmid in the io path. It checks if the vmid is > enabled and if io belongs to a vm or not and acts accordingly. Other > supporing APIs are also included in the patch. supporting (sp) > > Signed-off-by: Gaurav Srivastava > Signed-off-by: James Smart > > --- > v9: > Added changes related to locking and new hashtable implementation > > v8: > Added proper error codes > updated logic while handling vmid > > v7: > No change > > v6: > No change > > v5: > No change > > v4: > No change > > v3: > Replaced blkcg_get_app_identifier with blkcg_get_fc_appid > > v2: > Ported the patch on top of 5.10/scsi-queue > Added a fix for issuing QFPA command which was not included in the > last submit > --- > drivers/scsi/lpfc/lpfc_scsi.c | 169 ++++++++++++++++++++++++++++++++++ > 1 file changed, 169 insertions(+) > > diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c > index f81178464436..3267c5858238 100644 > --- a/drivers/scsi/lpfc/lpfc_scsi.c > +++ b/drivers/scsi/lpfc/lpfc_scsi.c > @@ -5263,6 +5263,155 @@ static void lpfc_vmid_assign_cs_ctl(struct lpfc_vport *vport, > } > } > > +/* > + * lpfc_vmid_get_appid- get the vmid associated with the uuid > + * @vport: The virtual port for which this call is being executed. > + * @uuid: uuid associated with the VE > + * @cmd: address of scsi cmmd descriptor > + * @tag: VMID tag > + * Returns status of the function > + */ > +static int lpfc_vmid_get_appid(struct lpfc_vport *vport, char *uuid, struct > + scsi_cmnd * cmd, union lpfc_vmid_io_tag *tag) > +{ > + struct lpfc_vmid *vmp = NULL; > + int hash, len, rc, i; > + u8 pending = 0; > + > + /* check if QFPA is complete */ > + if (lpfc_vmid_is_type_priority_tag(vport) && !(vport->vmid_flag & > + LPFC_VMID_QFPA_CMPL)) { > + vport->work_port_events |= WORKER_CHECK_VMID_ISSUE_QFPA; > + return -EAGAIN; > + } > + > + /* search if the uuid has already been mapped to the vmid */ > + len = strlen(uuid); > + hash = lpfc_vmid_hash_fn(uuid, len); > + > + /* search for the VMID in the table */ > + read_lock(&vport->vmid_lock); > + vmp = lpfc_get_vmid_from_hastable(vport, hash, uuid); > + > + /* if found, check if its already registered */ > + if (vmp && vmp->flag & LPFC_VMID_REGISTERED) { > + read_unlock(&vport->vmid_lock); > + lpfc_vmid_update_entry(vport, cmd, vmp, tag); > + rc = 0; > + } else if (vmp && (vmp->flag & LPFC_VMID_REQ_REGISTER || > + vmp->flag & LPFC_VMID_DE_REGISTER)) { > + /* else if register or dereg request has already been sent */ > + /* Hence vmid tag will not be added for this IO */ > + read_unlock(&vport->vmid_lock); > + rc = -EBUSY; > + } else { > + /* The vmid was not found in the hashtable. At this point, */ > + /* drop the read lock first before proceeding further */ > + read_unlock(&vport->vmid_lock); > + /* start the process to obtain one as per the */ > + /* type of the vmid indicated */ > + write_lock(&vport->vmid_lock); > + vmp = lpfc_get_vmid_from_hastable(vport, hash, uuid); > + > + /* while the read lock was released, in case the entry was */ > + /* added by other context or is in process of being added */ > + if (vmp && vmp->flag & LPFC_VMID_REGISTERED) { > + lpfc_vmid_update_entry(vport, cmd, vmp, tag); > + write_unlock(&vport->vmid_lock); > + return 0; > + } else if (vmp && vmp->flag & LPFC_VMID_REQ_REGISTER) { > + write_unlock(&vport->vmid_lock); > + return -EBUSY; > + } > + > + /* else search and allocate a free slot in the hash table */ > + if (vport->cur_vmid_cnt < vport->max_vmid) { > + for (i = 0; i < vport->max_vmid; ++i) { > + vmp = vport->vmid + i; > + if (vmp->flag == LPFC_VMID_SLOT_FREE) { > + vmp = vport->vmid + i; > + break; > + } > + } > + } else { > + write_unlock(&vport->vmid_lock); > + return -ENOMEM; > + } > + > + if (vmp && (vmp->flag == LPFC_VMID_SLOT_FREE)) { > + /* Add the vmid and register */ > + lpfc_put_vmid_in_hashtable(vport, hash, vmp); > + vmp->vmid_len = len; > + memcpy(vmp->host_vmid, uuid, vmp->vmid_len); > + vmp->io_rd_cnt = 0; > + vmp->io_wr_cnt = 0; > + vmp->flag = LPFC_VMID_SLOT_USED; > + > + vmp->delete_inactive = > + vport->vmid_inactivity_timeout ? 1 : 0; > + > + /* if type priority tag, get next available vmid */ > + if (lpfc_vmid_is_type_priority_tag(vport)) > + lpfc_vmid_assign_cs_ctl(vport, vmp); > + > + /* allocate the per cpu variable for holding */ > + /* the last access time stamp only if vmid is enabled */ > + if (!vmp->last_io_time) > + vmp->last_io_time = > + __alloc_percpu(sizeof(u64), > + __alignof__(struct > + lpfc_vmid)); > + > + /* registration pending */ > + pending = 1; > + } else { > + rc = -ENOMEM; > + } > + write_unlock(&vport->vmid_lock); > + > + /* complete transaction with switch */ > + if (pending) { > + if (lpfc_vmid_is_type_priority_tag(vport)) > + rc = lpfc_vmid_uvem(vport, vmp, true); > + else > + rc = lpfc_vmid_cmd(vport, > + SLI_CTAS_RAPP_IDENT, > + vmp); > + if (!rc) { > + write_lock(&vport->vmid_lock); > + vport->cur_vmid_cnt++; > + vmp->flag |= LPFC_VMID_REQ_REGISTER; > + write_unlock(&vport->vmid_lock); > + } > + } > + > + /* finally, enable the idle timer once */ > + if (!(vport->phba->pport->vmid_flag & LPFC_VMID_TIMER_ENBLD)) { > + mod_timer(&vport->phba->inactive_vmid_poll, > + jiffies + > + msecs_to_jiffies(1000 * LPFC_VMID_TIMER)); > + vport->phba->pport->vmid_flag |= LPFC_VMID_TIMER_ENBLD; > + } > + } > + return rc; > +} > + > +/* > + * lpfc_is_command_vm_io - get the uuid from blk cgroup > + * @cmd:Pointer to scsi_cmnd data structure > + * Returns uuid if present if not null > + */ > +static char *lpfc_is_command_vm_io(struct scsi_cmnd *cmd) > +{ > + char *uuid = NULL; > + > + if (cmd->request) { > + if (cmd->request->bio) > + uuid = blkcg_get_fc_appid(cmd->request->bio); > + } > + return uuid; > +} > + > /** > * lpfc_queuecommand - scsi_host_template queuecommand entry point > * @shost: kernel scsi host pointer. > @@ -5288,6 +5437,7 @@ lpfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) > int err, idx; > #ifdef CONFIG_SCSI_LPFC_DEBUG_FS > uint64_t start = 0L; > + u8 *uuid = NULL; > > if (phba->ktime_on) > start = ktime_get_ns(); > @@ -5415,6 +5565,25 @@ lpfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) > } > > > + /* check the necessary and sufficient condition to support VMID */ > + if (lpfc_is_vmid_enabled(phba) && > + (ndlp->vmid_support || > + phba->pport->vmid_priority_tagging == > + LPFC_VMID_PRIO_TAG_ALL_TARGETS)) { > + /* is the IO generated by a VM, get the associated virtual */ > + /* entity id */ > + uuid = lpfc_is_command_vm_io(cmnd); > + > + if (uuid) { > + err = lpfc_vmid_get_appid(vport, uuid, cmnd, > + (union lpfc_vmid_io_tag *) > + &lpfc_cmd->cur_iocbq.vmid_tag); > + if (!err) > + lpfc_cmd->cur_iocbq.iocb_flag |= LPFC_IO_VMID; > + } > + } > + > + atomic_inc(&ndlp->cmd_pending); > #ifdef CONFIG_SCSI_LPFC_DEBUG_FS > if (unlikely(phba->hdwqstat_on & LPFC_CHECK_SCSI_IO)) > this_cpu_inc(phba->sli4_hba.c_stat->xmt_io); > And that's the bit which I don't particular like. Essentially we'll have to inject additional ELS commands _on each I/O_ to get a valid VMID. Where there are _so_ many things which might get wrong, causing an I/O stall. I would have vastly preferred if we could _avoid_ having to do additional ELS commands for VMID registration in the I/O path (ie only allow for I/O with a valid VMID), and reject the I/O otherwise until VMID registration is complete. IE return 'BUSY' (or even a command retry?) when no valid VMID for this particular I/O is found, register the VMID (preferably in another thread), and restart the queue once the VMID is registered. That way we have a clear separation, and the I/O path will always work with valid VMIDs. Hmm? Cheers, Hannes -- Dr. Hannes Reinecke Kernel Storage Architect hare@suse.de +49 911 74053 688 SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E6B8C433B4 for ; Thu, 8 Apr 2021 08:46:36 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CC36061164 for ; Thu, 8 Apr 2021 08:46:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CC36061164 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:Cc:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=kYyTh9YoOtMWi6Eocs8PLt5JpX+0klZSS/0AzJqzoRA=; b=blwyGz24vyVEyvFOHQ+/b5NnP iTTbi7dVLq/aF6F+FmHZQ3lsZemuQcv91N683HCr8BEPoMFr65hzPSRoWXtEbUGeQrib2LE63SyhA WctrkdzjhZZ1GN30G7wSpQCBbaaigmhKkDVK9v9HMr/y0f/3n5nzV9ZF9zhtv54GLLNV44CMrn8K+ lJ8adJcslLFAjWiikxOPQ9v0s4O/GlEAe1B1IdHUsKayMxTfAZ7zx0cJaV4hwtZhEqANJJEfqsR4r xuSWySODp2Hmh6WgZqS83+x85kWbvUE85JjMy7HPjEm7cRiknCK7PHEN5dGJUdKxBqmH+7kXI2/Ly ohSAn6d6w==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUQIy-007NPb-Rx; Thu, 08 Apr 2021 08:46:24 +0000 Received: from mx2.suse.de ([195.135.220.15]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lUQIn-007NOO-0d for linux-nvme@lists.infradead.org; Thu, 08 Apr 2021 08:46:15 +0000 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 7A075AE58; Thu, 8 Apr 2021 08:46:12 +0000 (UTC) Subject: Re: [PATCH v9 13/13] lpfc: vmid: Introducing vmid in io path. To: Muneendra , linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, tj@kernel.org, linux-nvme@lists.infradead.org Cc: jsmart2021@gmail.com, emilne@redhat.com, mkumar@redhat.com, Gaurav Srivastava References: <1617750397-26466-1-git-send-email-muneendra.kumar@broadcom.com> <1617750397-26466-14-git-send-email-muneendra.kumar@broadcom.com> From: Hannes Reinecke Message-ID: <91b0c309-6908-8fd9-ac60-a8572500c3ed@suse.de> Date: Thu, 8 Apr 2021 10:46:11 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: <1617750397-26466-14-git-send-email-muneendra.kumar@broadcom.com> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_094613_505980_91930E84 X-CRM114-Status: GOOD ( 43.31 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org T24gNC83LzIxIDE6MDYgQU0sIE11bmVlbmRyYSB3cm90ZToKPiBGcm9tOiBHYXVyYXYgU3JpdmFz dGF2YSA8Z2F1cmF2LnNyaXZhc3RhdmFAYnJvYWRjb20uY29tPgo+IAo+IFRoZSBwYXRjaCBpbnRy b2R1Y2VzIHRoZSB2bWlkIGluIHRoZSBpbyBwYXRoLiBJdCBjaGVja3MgaWYgdGhlIHZtaWQgaXMK PiBlbmFibGVkIGFuZCBpZiBpbyBiZWxvbmdzIHRvIGEgdm0gb3Igbm90IGFuZCBhY3RzIGFjY29y ZGluZ2x5LiBPdGhlcgo+IHN1cHBvcmluZyBBUElzIGFyZSBhbHNvIGluY2x1ZGVkIGluIHRoZSBw YXRjaC4KCnN1cHBvcnRpbmcgKHNwKQoKPiAKPiBTaWduZWQtb2ZmLWJ5OiBHYXVyYXYgU3JpdmFz dGF2YSAgPGdhdXJhdi5zcml2YXN0YXZhQGJyb2FkY29tLmNvbT4KPiBTaWduZWQtb2ZmLWJ5OiBK YW1lcyBTbWFydCA8anNtYXJ0MjAyMUBnbWFpbC5jb20+Cj4gCj4gLS0tCj4gdjk6Cj4gQWRkZWQg Y2hhbmdlcyByZWxhdGVkIHRvIGxvY2tpbmcgYW5kIG5ldyBoYXNodGFibGUgaW1wbGVtZW50YXRp b24KPiAKPiB2ODoKPiBBZGRlZCBwcm9wZXIgZXJyb3IgY29kZXMKPiB1cGRhdGVkIGxvZ2ljIHdo aWxlIGhhbmRsaW5nIHZtaWQKPiAKPiB2NzoKPiBObyBjaGFuZ2UKPiAKPiB2NjoKPiBObyBjaGFu Z2UKPiAKPiB2NToKPiBObyBjaGFuZ2UKPiAKPiB2NDoKPiBObyBjaGFuZ2UKPiAKPiB2MzoKPiBS ZXBsYWNlZCBibGtjZ19nZXRfYXBwX2lkZW50aWZpZXIgd2l0aCBibGtjZ19nZXRfZmNfYXBwaWQK PiAKPiB2MjoKPiBQb3J0ZWQgdGhlIHBhdGNoIG9uIHRvcCBvZiA1LjEwL3Njc2ktcXVldWUKPiBB ZGRlZCBhIGZpeCBmb3IgaXNzdWluZyBRRlBBIGNvbW1hbmQgd2hpY2ggd2FzIG5vdCBpbmNsdWRl ZCBpbiB0aGUKPiBsYXN0IHN1Ym1pdAo+IC0tLQo+ICAgZHJpdmVycy9zY3NpL2xwZmMvbHBmY19z Y3NpLmMgfCAxNjkgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKwo+ICAgMSBmaWxl IGNoYW5nZWQsIDE2OSBpbnNlcnRpb25zKCspCj4gCj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvc2Nz aS9scGZjL2xwZmNfc2NzaS5jIGIvZHJpdmVycy9zY3NpL2xwZmMvbHBmY19zY3NpLmMKPiBpbmRl eCBmODExNzg0NjQ0MzYuLjMyNjdjNTg1ODIzOCAxMDA2NDQKPiAtLS0gYS9kcml2ZXJzL3Njc2kv bHBmYy9scGZjX3Njc2kuYwo+ICsrKyBiL2RyaXZlcnMvc2NzaS9scGZjL2xwZmNfc2NzaS5jCj4g QEAgLTUyNjMsNiArNTI2MywxNTUgQEAgc3RhdGljIHZvaWQgbHBmY192bWlkX2Fzc2lnbl9jc19j dGwoc3RydWN0IGxwZmNfdnBvcnQgKnZwb3J0LAo+ICAgCX0KPiAgIH0KPiAgIAo+ICsvKgo+ICsg KiBscGZjX3ZtaWRfZ2V0X2FwcGlkLSBnZXQgdGhlIHZtaWQgYXNzb2NpYXRlZCB3aXRoIHRoZSB1 dWlkCj4gKyAqIEB2cG9ydDogVGhlIHZpcnR1YWwgcG9ydCBmb3Igd2hpY2ggdGhpcyBjYWxsIGlz IGJlaW5nIGV4ZWN1dGVkLgo+ICsgKiBAdXVpZDogdXVpZCBhc3NvY2lhdGVkIHdpdGggdGhlIFZF Cj4gKyAqIEBjbWQ6IGFkZHJlc3Mgb2Ygc2NzaSBjbW1kIGRlc2NyaXB0b3IKPiArICogQHRhZzog Vk1JRCB0YWcKPiArICogUmV0dXJucyBzdGF0dXMgb2YgdGhlIGZ1bmN0aW9uCj4gKyAqLwo+ICtz dGF0aWMgaW50IGxwZmNfdm1pZF9nZXRfYXBwaWQoc3RydWN0IGxwZmNfdnBvcnQgKnZwb3J0LCBj aGFyICp1dWlkLCBzdHJ1Y3QKPiArCQkJICAgICAgIHNjc2lfY21uZCAqIGNtZCwgdW5pb24gbHBm Y192bWlkX2lvX3RhZyAqdGFnKQo+ICt7Cj4gKwlzdHJ1Y3QgbHBmY192bWlkICp2bXAgPSBOVUxM Owo+ICsJaW50IGhhc2gsIGxlbiwgcmMsIGk7Cj4gKwl1OCBwZW5kaW5nID0gMDsKPiArCj4gKwkv KiBjaGVjayBpZiBRRlBBIGlzIGNvbXBsZXRlICovCj4gKwlpZiAobHBmY192bWlkX2lzX3R5cGVf cHJpb3JpdHlfdGFnKHZwb3J0KSAmJiAhKHZwb3J0LT52bWlkX2ZsYWcgJgo+ICsJICAgICAgTFBG Q19WTUlEX1FGUEFfQ01QTCkpIHsKPiArCQl2cG9ydC0+d29ya19wb3J0X2V2ZW50cyB8PSBXT1JL RVJfQ0hFQ0tfVk1JRF9JU1NVRV9RRlBBOwo+ICsJCXJldHVybiAtRUFHQUlOOwo+ICsJfQo+ICsK PiArCS8qIHNlYXJjaCBpZiB0aGUgdXVpZCBoYXMgYWxyZWFkeSBiZWVuIG1hcHBlZCB0byB0aGUg dm1pZCAqLwo+ICsJbGVuID0gc3RybGVuKHV1aWQpOwo+ICsJaGFzaCA9IGxwZmNfdm1pZF9oYXNo X2ZuKHV1aWQsIGxlbik7Cj4gKwo+ICsJLyogc2VhcmNoIGZvciB0aGUgVk1JRCBpbiB0aGUgdGFi bGUgKi8KPiArCXJlYWRfbG9jaygmdnBvcnQtPnZtaWRfbG9jayk7Cj4gKwl2bXAgPSBscGZjX2dl dF92bWlkX2Zyb21faGFzdGFibGUodnBvcnQsIGhhc2gsIHV1aWQpOwo+ICsKPiArCS8qIGlmIGZv dW5kLCBjaGVjayBpZiBpdHMgYWxyZWFkeSByZWdpc3RlcmVkICAqLwo+ICsJaWYgKHZtcCAgJiYg dm1wLT5mbGFnICYgTFBGQ19WTUlEX1JFR0lTVEVSRUQpIHsKPiArCQlyZWFkX3VubG9jaygmdnBv cnQtPnZtaWRfbG9jayk7Cj4gKwkJbHBmY192bWlkX3VwZGF0ZV9lbnRyeSh2cG9ydCwgY21kLCB2 bXAsIHRhZyk7Cj4gKwkJcmMgPSAwOwo+ICsJfSBlbHNlIGlmICh2bXAgJiYgKHZtcC0+ZmxhZyAm IExQRkNfVk1JRF9SRVFfUkVHSVNURVIgfHwKPiArCQkJICAgdm1wLT5mbGFnICYgTFBGQ19WTUlE X0RFX1JFR0lTVEVSKSkgewo+ICsJCS8qIGVsc2UgaWYgcmVnaXN0ZXIgb3IgZGVyZWcgcmVxdWVz dCBoYXMgYWxyZWFkeSBiZWVuIHNlbnQgKi8KPiArCQkvKiBIZW5jZSB2bWlkIHRhZyB3aWxsIG5v dCBiZSBhZGRlZCBmb3IgdGhpcyBJTyAqLwo+ICsJCXJlYWRfdW5sb2NrKCZ2cG9ydC0+dm1pZF9s b2NrKTsKPiArCQlyYyA9IC1FQlVTWTsKPiArCX0gZWxzZSB7Cj4gKwkJLyogVGhlIHZtaWQgd2Fz IG5vdCBmb3VuZCBpbiB0aGUgaGFzaHRhYmxlLiBBdCB0aGlzIHBvaW50LCAqLwo+ICsJCS8qIGRy b3AgdGhlIHJlYWQgbG9jayBmaXJzdCBiZWZvcmUgcHJvY2VlZGluZyBmdXJ0aGVyICovCj4gKwkJ cmVhZF91bmxvY2soJnZwb3J0LT52bWlkX2xvY2spOwo+ICsJCS8qIHN0YXJ0IHRoZSBwcm9jZXNz IHRvIG9idGFpbiBvbmUgYXMgcGVyIHRoZSAqLwo+ICsJCS8qIHR5cGUgb2YgdGhlIHZtaWQgaW5k aWNhdGVkICovCj4gKwkJd3JpdGVfbG9jaygmdnBvcnQtPnZtaWRfbG9jayk7Cj4gKwkJdm1wID0g bHBmY19nZXRfdm1pZF9mcm9tX2hhc3RhYmxlKHZwb3J0LCBoYXNoLCB1dWlkKTsKPiArCj4gKwkJ Lyogd2hpbGUgdGhlIHJlYWQgbG9jayB3YXMgcmVsZWFzZWQsIGluIGNhc2UgdGhlIGVudHJ5IHdh cyAqLwo+ICsJCS8qIGFkZGVkIGJ5IG90aGVyIGNvbnRleHQgb3IgaXMgaW4gcHJvY2VzcyBvZiBi ZWluZyBhZGRlZCAqLwo+ICsJCWlmICh2bXAgJiYgdm1wLT5mbGFnICYgTFBGQ19WTUlEX1JFR0lT VEVSRUQpIHsKPiArCQkJbHBmY192bWlkX3VwZGF0ZV9lbnRyeSh2cG9ydCwgY21kLCB2bXAsIHRh Zyk7Cj4gKwkJCXdyaXRlX3VubG9jaygmdnBvcnQtPnZtaWRfbG9jayk7Cj4gKwkJCXJldHVybiAw Owo+ICsJCX0gZWxzZSBpZiAodm1wICYmIHZtcC0+ZmxhZyAmIExQRkNfVk1JRF9SRVFfUkVHSVNU RVIpIHsKPiArCQkJd3JpdGVfdW5sb2NrKCZ2cG9ydC0+dm1pZF9sb2NrKTsKPiArCQkJcmV0dXJu IC1FQlVTWTsKPiArCQl9Cj4gKwo+ICsJCS8qIGVsc2Ugc2VhcmNoIGFuZCBhbGxvY2F0ZSBhIGZy ZWUgc2xvdCBpbiB0aGUgaGFzaCB0YWJsZSAqLwo+ICsJCWlmICh2cG9ydC0+Y3VyX3ZtaWRfY250 IDwgdnBvcnQtPm1heF92bWlkKSB7Cj4gKwkJCWZvciAoaSA9IDA7IGkgPCB2cG9ydC0+bWF4X3Zt aWQ7ICsraSkgewo+ICsJCQkJdm1wID0gdnBvcnQtPnZtaWQgKyBpOwo+ICsJCQkJaWYgKHZtcC0+ ZmxhZyA9PSBMUEZDX1ZNSURfU0xPVF9GUkVFKSB7Cj4gKwkJCQkJdm1wID0gdnBvcnQtPnZtaWQg KyBpOwo+ICsJCQkJCWJyZWFrOwo+ICsJCQkJfQo+ICsJCQl9Cj4gKwkJfSBlbHNlIHsKPiArCQkJ d3JpdGVfdW5sb2NrKCZ2cG9ydC0+dm1pZF9sb2NrKTsKPiArCQkJcmV0dXJuIC1FTk9NRU07Cj4g KwkJfQo+ICsKPiArCQlpZiAodm1wICYmICh2bXAtPmZsYWcgPT0gTFBGQ19WTUlEX1NMT1RfRlJF RSkpIHsKPiArCQkJLyogQWRkIHRoZSB2bWlkIGFuZCByZWdpc3RlciAgKi8KPiArCQkJbHBmY19w dXRfdm1pZF9pbl9oYXNodGFibGUodnBvcnQsIGhhc2gsIHZtcCk7Cj4gKwkJCXZtcC0+dm1pZF9s ZW4gPSBsZW47Cj4gKwkJCW1lbWNweSh2bXAtPmhvc3Rfdm1pZCwgdXVpZCwgdm1wLT52bWlkX2xl bik7Cj4gKwkJCXZtcC0+aW9fcmRfY250ID0gMDsKPiArCQkJdm1wLT5pb193cl9jbnQgPSAwOwo+ ICsJCQl2bXAtPmZsYWcgPSBMUEZDX1ZNSURfU0xPVF9VU0VEOwo+ICsKPiArCQkJdm1wLT5kZWxl dGVfaW5hY3RpdmUgPQo+ICsJCQkgICAgdnBvcnQtPnZtaWRfaW5hY3Rpdml0eV90aW1lb3V0ID8g MSA6IDA7Cj4gKwo+ICsJCQkvKiBpZiB0eXBlIHByaW9yaXR5IHRhZywgZ2V0IG5leHQgYXZhaWxh YmxlIHZtaWQgKi8KPiArCQkJaWYgKGxwZmNfdm1pZF9pc190eXBlX3ByaW9yaXR5X3RhZyh2cG9y dCkpCj4gKwkJCQlscGZjX3ZtaWRfYXNzaWduX2NzX2N0bCh2cG9ydCwgdm1wKTsKPiArCj4gKwkJ CS8qIGFsbG9jYXRlIHRoZSBwZXIgY3B1IHZhcmlhYmxlIGZvciBob2xkaW5nICovCj4gKwkJCS8q IHRoZSBsYXN0IGFjY2VzcyB0aW1lIHN0YW1wIG9ubHkgaWYgdm1pZCBpcyBlbmFibGVkICovCj4g KwkJCWlmICghdm1wLT5sYXN0X2lvX3RpbWUpCj4gKwkJCQl2bXAtPmxhc3RfaW9fdGltZSA9Cj4g KwkJCQkgICAgX19hbGxvY19wZXJjcHUoc2l6ZW9mKHU2NCksCj4gKwkJCQkJCSAgIF9fYWxpZ25v Zl9fKHN0cnVjdAo+ICsJCQkJCQkJICAgICAgIGxwZmNfdm1pZCkpOwo+ICsKPiArCQkJLyogcmVn aXN0cmF0aW9uIHBlbmRpbmcgKi8KPiArCQkJcGVuZGluZyA9IDE7Cj4gKwkJfSBlbHNlIHsKPiAr CQkJcmMgPSAtRU5PTUVNOwo+ICsJCX0KPiArCQl3cml0ZV91bmxvY2soJnZwb3J0LT52bWlkX2xv Y2spOwo+ICsKPiArCQkvKiBjb21wbGV0ZSB0cmFuc2FjdGlvbiB3aXRoIHN3aXRjaCAqLwo+ICsJ CWlmIChwZW5kaW5nKSB7Cj4gKwkJCWlmIChscGZjX3ZtaWRfaXNfdHlwZV9wcmlvcml0eV90YWco dnBvcnQpKQo+ICsJCQkJcmMgPSBscGZjX3ZtaWRfdXZlbSh2cG9ydCwgdm1wLCB0cnVlKTsKPiAr CQkJZWxzZQo+ICsJCQkJcmMgPSBscGZjX3ZtaWRfY21kKHZwb3J0LAo+ICsJCQkJCQkgICBTTElf Q1RBU19SQVBQX0lERU5ULAo+ICsJCQkJCQkgICB2bXApOwo+ICsJCQlpZiAoIXJjKSB7Cj4gKwkJ CQl3cml0ZV9sb2NrKCZ2cG9ydC0+dm1pZF9sb2NrKTsKPiArCQkJCXZwb3J0LT5jdXJfdm1pZF9j bnQrKzsKPiArCQkJCXZtcC0+ZmxhZyB8PSBMUEZDX1ZNSURfUkVRX1JFR0lTVEVSOwo+ICsJCQkJ d3JpdGVfdW5sb2NrKCZ2cG9ydC0+dm1pZF9sb2NrKTsKPiArCQkJfQo+ICsJCX0KPiArCj4gKwkJ LyogZmluYWxseSwgZW5hYmxlIHRoZSBpZGxlIHRpbWVyIG9uY2UgKi8KPiArCQlpZiAoISh2cG9y dC0+cGhiYS0+cHBvcnQtPnZtaWRfZmxhZyAmIExQRkNfVk1JRF9USU1FUl9FTkJMRCkpIHsKPiAr CQkJbW9kX3RpbWVyKCZ2cG9ydC0+cGhiYS0+aW5hY3RpdmVfdm1pZF9wb2xsLAo+ICsJCQkJICBq aWZmaWVzICsKPiArCQkJCSAgbXNlY3NfdG9famlmZmllcygxMDAwICogTFBGQ19WTUlEX1RJTUVS KSk7Cj4gKwkJCXZwb3J0LT5waGJhLT5wcG9ydC0+dm1pZF9mbGFnIHw9IExQRkNfVk1JRF9USU1F Ul9FTkJMRDsKPiArCQl9Cj4gKwl9Cj4gKwlyZXR1cm4gcmM7Cj4gK30KPiArCj4gKy8qCj4gKyAq IGxwZmNfaXNfY29tbWFuZF92bV9pbyAtIGdldCB0aGUgdXVpZCBmcm9tIGJsayBjZ3JvdXAKPiAr ICogQGNtZDpQb2ludGVyIHRvIHNjc2lfY21uZCBkYXRhIHN0cnVjdHVyZQo+ICsgKiBSZXR1cm5z IHV1aWQgaWYgcHJlc2VudCBpZiBub3QgbnVsbAo+ICsgKi8KPiArc3RhdGljIGNoYXIgKmxwZmNf aXNfY29tbWFuZF92bV9pbyhzdHJ1Y3Qgc2NzaV9jbW5kICpjbWQpCj4gK3sKPiArCWNoYXIgKnV1 aWQgPSBOVUxMOwo+ICsKPiArCWlmIChjbWQtPnJlcXVlc3QpIHsKPiArCQlpZiAoY21kLT5yZXF1 ZXN0LT5iaW8pCj4gKwkJCXV1aWQgPSBibGtjZ19nZXRfZmNfYXBwaWQoY21kLT5yZXF1ZXN0LT5i aW8pOwo+ICsJfQo+ICsJcmV0dXJuIHV1aWQ7Cj4gK30KPiArCj4gICAvKioKPiAgICAqIGxwZmNf cXVldWVjb21tYW5kIC0gc2NzaV9ob3N0X3RlbXBsYXRlIHF1ZXVlY29tbWFuZCBlbnRyeSBwb2lu dAo+ICAgICogQHNob3N0OiBrZXJuZWwgc2NzaSBob3N0IHBvaW50ZXIuCj4gQEAgLTUyODgsNiAr NTQzNyw3IEBAIGxwZmNfcXVldWVjb21tYW5kKHN0cnVjdCBTY3NpX0hvc3QgKnNob3N0LCBzdHJ1 Y3Qgc2NzaV9jbW5kICpjbW5kKQo+ICAgCWludCBlcnIsIGlkeDsKPiAgICNpZmRlZiBDT05GSUdf U0NTSV9MUEZDX0RFQlVHX0ZTCj4gICAJdWludDY0X3Qgc3RhcnQgPSAwTDsKPiArCXU4ICp1dWlk ID0gTlVMTDsKPiAgIAo+ICAgCWlmIChwaGJhLT5rdGltZV9vbikKPiAgIAkJc3RhcnQgPSBrdGlt ZV9nZXRfbnMoKTsKPiBAQCAtNTQxNSw2ICs1NTY1LDI1IEBAIGxwZmNfcXVldWVjb21tYW5kKHN0 cnVjdCBTY3NpX0hvc3QgKnNob3N0LCBzdHJ1Y3Qgc2NzaV9jbW5kICpjbW5kKQo+ICAgCX0KPiAg IAo+ICAgCj4gKwkvKiBjaGVjayB0aGUgbmVjZXNzYXJ5IGFuZCBzdWZmaWNpZW50IGNvbmRpdGlv biB0byBzdXBwb3J0IFZNSUQgKi8KPiArCWlmIChscGZjX2lzX3ZtaWRfZW5hYmxlZChwaGJhKSAm Jgo+ICsJICAgIChuZGxwLT52bWlkX3N1cHBvcnQgfHwKPiArCSAgICAgcGhiYS0+cHBvcnQtPnZt aWRfcHJpb3JpdHlfdGFnZ2luZyA9PQo+ICsJICAgICBMUEZDX1ZNSURfUFJJT19UQUdfQUxMX1RB UkdFVFMpKSB7Cj4gKwkJLyogaXMgdGhlIElPIGdlbmVyYXRlZCBieSBhIFZNLCBnZXQgdGhlIGFz c29jaWF0ZWQgdmlydHVhbCAqLwo+ICsJCS8qIGVudGl0eSBpZCAqLwo+ICsJCXV1aWQgPSBscGZj X2lzX2NvbW1hbmRfdm1faW8oY21uZCk7Cj4gKwo+ICsJCWlmICh1dWlkKSB7Cj4gKwkJCWVyciA9 IGxwZmNfdm1pZF9nZXRfYXBwaWQodnBvcnQsIHV1aWQsIGNtbmQsCj4gKwkJCQkodW5pb24gbHBm Y192bWlkX2lvX3RhZyAqKQo+ICsJCQkJCSZscGZjX2NtZC0+Y3VyX2lvY2JxLnZtaWRfdGFnKTsK PiArCQkJaWYgKCFlcnIpCj4gKwkJCQlscGZjX2NtZC0+Y3VyX2lvY2JxLmlvY2JfZmxhZyB8PSBM UEZDX0lPX1ZNSUQ7Cj4gKwkJfQo+ICsJfQo+ICsKPiArCWF0b21pY19pbmMoJm5kbHAtPmNtZF9w ZW5kaW5nKTsKPiAgICNpZmRlZiBDT05GSUdfU0NTSV9MUEZDX0RFQlVHX0ZTCj4gICAJaWYgKHVu bGlrZWx5KHBoYmEtPmhkd3FzdGF0X29uICYgTFBGQ19DSEVDS19TQ1NJX0lPKSkKPiAgIAkJdGhp c19jcHVfaW5jKHBoYmEtPnNsaTRfaGJhLmNfc3RhdC0+eG10X2lvKTsKPiAKQW5kIHRoYXQncyB0 aGUgYml0IHdoaWNoIEkgZG9uJ3QgcGFydGljdWxhciBsaWtlLgoKRXNzZW50aWFsbHkgd2UnbGwg aGF2ZSB0byBpbmplY3QgYWRkaXRpb25hbCBFTFMgY29tbWFuZHMgX29uIGVhY2ggSS9PXyAKdG8g Z2V0IGEgdmFsaWQgVk1JRC4KV2hlcmUgdGhlcmUgYXJlIF9zb18gbWFueSB0aGluZ3Mgd2hpY2gg bWlnaHQgZ2V0IHdyb25nLCBjYXVzaW5nIGFuIEkvTyAKc3RhbGwuCgpJIHdvdWxkIGhhdmUgdmFz dGx5IHByZWZlcnJlZCBpZiB3ZSBjb3VsZCBfYXZvaWRfIGhhdmluZyB0byBkbyAKYWRkaXRpb25h bCBFTFMgY29tbWFuZHMgZm9yIFZNSUQgcmVnaXN0cmF0aW9uIGluIHRoZSBJL08gcGF0aAooaWUg b25seSBhbGxvdyBmb3IgSS9PIHdpdGggYSB2YWxpZCBWTUlEKSwgYW5kIHJlamVjdCB0aGUgSS9P IG90aGVyd2lzZSAKdW50aWwgVk1JRCByZWdpc3RyYXRpb24gaXMgY29tcGxldGUuCgpJRSByZXR1 cm4gJ0JVU1knIChvciBldmVuIGEgY29tbWFuZCByZXRyeT8pIHdoZW4gbm8gdmFsaWQgVk1JRCBm b3IgdGhpcyAKcGFydGljdWxhciBJL08gaXMgZm91bmQsIHJlZ2lzdGVyIHRoZSBWTUlEIChwcmVm ZXJhYmx5IGluIGFub3RoZXIgCnRocmVhZCksIGFuZCByZXN0YXJ0IHRoZSBxdWV1ZSBvbmNlIHRo ZSBWTUlEIGlzIHJlZ2lzdGVyZWQuCgpUaGF0IHdheSB3ZSBoYXZlIGEgY2xlYXIgc2VwYXJhdGlv biwgYW5kIHRoZSBJL08gcGF0aCB3aWxsIGFsd2F5cyB3b3JrIAp3aXRoIHZhbGlkIFZNSURzLgoK SG1tPwoKQ2hlZXJzLAoKSGFubmVzCi0tIApEci4gSGFubmVzIFJlaW5lY2tlICAgICAgICAgICAg ICAgIEtlcm5lbCBTdG9yYWdlIEFyY2hpdGVjdApoYXJlQHN1c2UuZGUgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICArNDkgOTExIDc0MDUzIDY4OApTVVNFIFNvZnR3YXJlIFNvbHV0aW9ucyBH bWJILCBNYXhmZWxkc3RyLiA1LCA5MDQwOSBOw7xybmJlcmcKSFJCIDM2ODA5IChBRyBOw7xybmJl cmcpLCBHZXNjaMOkZnRzZsO8aHJlcjogRmVsaXggSW1lbmTDtnJmZmVyCgpfX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpMaW51eC1udm1lIG1haWxpbmcgbGlz dApMaW51eC1udm1lQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5v cmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1udm1lCg==