From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F691C4708C for ; Fri, 28 May 2021 11:30:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7562061378 for ; Fri, 28 May 2021 11:30:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233262AbhE1Lbh (ORCPT ); Fri, 28 May 2021 07:31:37 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]:40950 "EHLO smtp-out2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230227AbhE1Lbg (ORCPT ); Fri, 28 May 2021 07:31:36 -0400 Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 0DA1A1FD2E; Fri, 28 May 2021 11:30:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1622201401; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YuWgbs4o0pUDWZsyt9VgIqiDbpUBaK7ftHbu223uPyc=; b=B4ktXATz6ZllLWhAHMsAfK5j20VugCm8NUoiB70oQuDu/QseLORYjoGWAP14m4Q7QOosWq qlE2g8Zuw9G5FZAUA8iR6XP1k25gJ/192mfIJok9ulhTSRf3ikYTY3j29PO3WprzosXlzE L9Y+G03qvOb4i4G45Ot2f+zLn+Mx6ZI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1622201401; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YuWgbs4o0pUDWZsyt9VgIqiDbpUBaK7ftHbu223uPyc=; b=ORJKubgfgsOF0TogL8nA1+xUWItBjhBfRDi8QBPHFdpzLs8xusSvSoi4PC96zACGLiHYcw 1/xRJwTakEYr98CQ== Received: from director2.suse.de (director2.suse-dmz.suse.de [192.168.254.72]) by imap.suse.de (Postfix) with ESMTPSA id C0C7B11906; Fri, 28 May 2021 11:30:00 +0000 (UTC) To: Shai Malin , netdev@vger.kernel.org, linux-nvme@lists.infradead.org, davem@davemloft.net, kuba@kernel.org, sagi@grimberg.me, hch@lst.de, axboe@fb.com, kbusch@kernel.org Cc: aelior@marvell.com, mkalderon@marvell.com, okulkarni@marvell.com, pkushwaha@marvell.com, malin1024@gmail.com, Dean Balandin References: <20210527235902.2185-1-smalin@marvell.com> <20210527235902.2185-8-smalin@marvell.com> From: Hannes Reinecke Organization: SUSE Linux GmbH Subject: Re: [RFC PATCH v6 07/27] nvme-tcp-offload: Add queue level implementation Message-ID: <4afc9965-cef6-1bba-9ab0-1272bfa6077f@suse.de> Date: Fri, 28 May 2021 13:29:55 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: <20210527235902.2185-8-smalin@marvell.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 5/28/21 1:58 AM, Shai Malin wrote: > From: Dean Balandin > > In this patch we implement queue level functionality. > The implementation is similar to the nvme-tcp module, the main > difference being that we call the vendor specific create_queue op which > creates the TCP connection, and NVMeTPC connection including > icreq+icresp negotiation. > Once create_queue returns successfully, we can move on to the fabrics > connect. > > Acked-by: Igor Russkikh > Signed-off-by: Dean Balandin > Signed-off-by: Prabhakar Kushwaha > Signed-off-by: Omkar Kulkarni > Signed-off-by: Michal Kalderon > Signed-off-by: Ariel Elior > Signed-off-by: Shai Malin > Reviewed-by: Himanshu Madhani > --- > drivers/nvme/host/tcp-offload.c | 418 +++++++++++++++++++++++++++++--- > drivers/nvme/host/tcp-offload.h | 4 + > 2 files changed, 394 insertions(+), 28 deletions(-) > > diff --git a/drivers/nvme/host/tcp-offload.c b/drivers/nvme/host/tcp-offload.c > index 52d310f7636a..eff10e31f17f 100644 > --- a/drivers/nvme/host/tcp-offload.c > +++ b/drivers/nvme/host/tcp-offload.c > @@ -22,6 +22,11 @@ static inline struct nvme_tcp_ofld_ctrl *to_tcp_ofld_ctrl(struct nvme_ctrl *nctr > return container_of(nctrl, struct nvme_tcp_ofld_ctrl, nctrl); > } > > +static inline int nvme_tcp_ofld_qid(struct nvme_tcp_ofld_queue *queue) > +{ > + return queue - queue->ctrl->queues; > +} > + > /** > * nvme_tcp_ofld_register_dev() - NVMeTCP Offload Library registration > * function. > @@ -182,19 +187,125 @@ nvme_tcp_ofld_alloc_tagset(struct nvme_ctrl *nctrl, bool admin) > return set; > } > > +static void __nvme_tcp_ofld_stop_queue(struct nvme_tcp_ofld_queue *queue) > +{ > + queue->dev->ops->drain_queue(queue); > +} > + > +static void nvme_tcp_ofld_stop_queue(struct nvme_ctrl *nctrl, int qid) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > + struct nvme_tcp_ofld_queue *queue = &ctrl->queues[qid]; > + > + mutex_lock(&queue->queue_lock); > + if (test_and_clear_bit(NVME_TCP_OFLD_Q_LIVE, &queue->flags)) > + __nvme_tcp_ofld_stop_queue(queue); > + mutex_unlock(&queue->queue_lock); > +} > + > +static void nvme_tcp_ofld_stop_io_queues(struct nvme_ctrl *ctrl) > +{ > + int i; > + > + for (i = 1; i < ctrl->queue_count; i++) > + nvme_tcp_ofld_stop_queue(ctrl, i); > +} > + > +static void __nvme_tcp_ofld_free_queue(struct nvme_tcp_ofld_queue *queue) > +{ > + queue->dev->ops->destroy_queue(queue); > +} > + > +static void nvme_tcp_ofld_free_queue(struct nvme_ctrl *nctrl, int qid) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > + struct nvme_tcp_ofld_queue *queue = &ctrl->queues[qid]; > + > + test_and_clear_bit(NVME_TCP_OFLD_Q_ALLOCATED, &queue->flags); You really want to make this an 'if' clause to avoid double free > + > + __nvme_tcp_ofld_free_queue(queue); > + > + mutex_destroy(&queue->queue_lock); > +} > + > +static void > +nvme_tcp_ofld_free_io_queues(struct nvme_ctrl *nctrl) > +{ > + int i; > + > + for (i = 1; i < nctrl->queue_count; i++) > + nvme_tcp_ofld_free_queue(nctrl, i); > +} > + > +static void nvme_tcp_ofld_destroy_io_queues(struct nvme_ctrl *nctrl, bool remove) > +{ > + nvme_tcp_ofld_stop_io_queues(nctrl); > + if (remove) { > + blk_cleanup_queue(nctrl->connect_q); > + blk_mq_free_tag_set(nctrl->tagset); > + } > + nvme_tcp_ofld_free_io_queues(nctrl); > +} > + > +static void nvme_tcp_ofld_destroy_admin_queue(struct nvme_ctrl *nctrl, bool remove) > +{ > + nvme_tcp_ofld_stop_queue(nctrl, 0); > + if (remove) { > + blk_cleanup_queue(nctrl->admin_q); > + blk_cleanup_queue(nctrl->fabrics_q); > + blk_mq_free_tag_set(nctrl->admin_tagset); > + } > + nvme_tcp_ofld_free_queue(nctrl, 0); > +} > + > +static int nvme_tcp_ofld_start_queue(struct nvme_ctrl *nctrl, int qid) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > + struct nvme_tcp_ofld_queue *queue = &ctrl->queues[qid]; > + int rc; > + > + queue = &ctrl->queues[qid]; > + if (qid) { > + queue->cmnd_capsule_len = nctrl->ioccsz * 16; > + rc = nvmf_connect_io_queue(nctrl, qid, false); > + } else { > + queue->cmnd_capsule_len = sizeof(struct nvme_command) + NVME_TCP_ADMIN_CCSZ; > + rc = nvmf_connect_admin_queue(nctrl); > + } > + > + if (!rc) { > + set_bit(NVME_TCP_OFLD_Q_LIVE, &queue->flags); > + } else { > + if (test_bit(NVME_TCP_OFLD_Q_ALLOCATED, &queue->flags)) > + __nvme_tcp_ofld_stop_queue(queue); Why do you need to call 'stop_queue' here? A failure indicates that the queue wasn't started, no? > + dev_err(nctrl->device, > + "failed to connect queue: %d ret=%d\n", qid, rc); > + } > + > + return rc; > +} > + > static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl, > bool new) > { > + struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > + struct nvme_tcp_ofld_queue *queue = &ctrl->queues[0]; > int rc; > > - /* Placeholder - alloc_admin_queue */ > + mutex_init(&queue->queue_lock); > + > + rc = ctrl->dev->ops->create_queue(queue, 0, NVME_AQ_DEPTH); > + if (rc) > + return rc; > + > + set_bit(NVME_TCP_OFLD_Q_ALLOCATED, &queue->flags); > if (new) { > nctrl->admin_tagset = > nvme_tcp_ofld_alloc_tagset(nctrl, true); > if (IS_ERR(nctrl->admin_tagset)) { > rc = PTR_ERR(nctrl->admin_tagset); > nctrl->admin_tagset = NULL; > - goto out_destroy_queue; > + goto out_free_queue; > } > > nctrl->fabrics_q = blk_mq_init_queue(nctrl->admin_tagset); > @@ -212,7 +323,9 @@ static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl, > } > } > > - /* Placeholder - nvme_tcp_ofld_start_queue */ > + rc = nvme_tcp_ofld_start_queue(nctrl, 0); > + if (rc) > + goto out_cleanup_queue; > > rc = nvme_enable_ctrl(nctrl); > if (rc) > @@ -229,19 +342,143 @@ static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl, > out_quiesce_queue: > blk_mq_quiesce_queue(nctrl->admin_q); > blk_sync_queue(nctrl->admin_q); > - > out_stop_queue: > - /* Placeholder - stop offload queue */ > + nvme_tcp_ofld_stop_queue(nctrl, 0); > nvme_cancel_admin_tagset(nctrl); > - > +out_cleanup_queue: > + if (new) > + blk_cleanup_queue(nctrl->admin_q); > out_cleanup_fabrics_q: > if (new) > blk_cleanup_queue(nctrl->fabrics_q); > out_free_tagset: > if (new) > blk_mq_free_tag_set(nctrl->admin_tagset); > -out_destroy_queue: > - /* Placeholder - free admin queue */ > +out_free_queue: > + nvme_tcp_ofld_free_queue(nctrl, 0); > + > + return rc; > +} > + > +static unsigned int nvme_tcp_ofld_nr_io_queues(struct nvme_ctrl *nctrl) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > + struct nvme_tcp_ofld_dev *dev = ctrl->dev; > + u32 hw_vectors = dev->num_hw_vectors; > + u32 nr_write_queues, nr_poll_queues; > + u32 nr_io_queues, nr_total_queues; > + > + nr_io_queues = min3(nctrl->opts->nr_io_queues, num_online_cpus(), > + hw_vectors); > + nr_write_queues = min3(nctrl->opts->nr_write_queues, num_online_cpus(), > + hw_vectors); > + nr_poll_queues = min3(nctrl->opts->nr_poll_queues, num_online_cpus(), > + hw_vectors); > + > + nr_total_queues = nr_io_queues + nr_write_queues + nr_poll_queues; > + > + return nr_total_queues; > +} > + > +static void > +nvme_tcp_ofld_set_io_queues(struct nvme_ctrl *nctrl, unsigned int nr_io_queues) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > + struct nvmf_ctrl_options *opts = nctrl->opts; > + > + if (opts->nr_write_queues && opts->nr_io_queues < nr_io_queues) { > + /* > + * separate read/write queues > + * hand out dedicated default queues only after we have > + * sufficient read queues. > + */ > + ctrl->io_queues[HCTX_TYPE_READ] = opts->nr_io_queues; > + nr_io_queues -= ctrl->io_queues[HCTX_TYPE_READ]; > + ctrl->io_queues[HCTX_TYPE_DEFAULT] = > + min(opts->nr_write_queues, nr_io_queues); > + nr_io_queues -= ctrl->io_queues[HCTX_TYPE_DEFAULT]; > + } else { > + /* > + * shared read/write queues > + * either no write queues were requested, or we don't have > + * sufficient queue count to have dedicated default queues. > + */ > + ctrl->io_queues[HCTX_TYPE_DEFAULT] = > + min(opts->nr_io_queues, nr_io_queues); > + nr_io_queues -= ctrl->io_queues[HCTX_TYPE_DEFAULT]; > + } > + > + if (opts->nr_poll_queues && nr_io_queues) { > + /* map dedicated poll queues only if we have queues left */ > + ctrl->io_queues[HCTX_TYPE_POLL] = > + min(opts->nr_poll_queues, nr_io_queues); > + } > +} > + > +static int nvme_tcp_ofld_create_io_queues(struct nvme_ctrl *nctrl) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > + int i, rc; > + > + for (i = 1; i < nctrl->queue_count; i++) { > + mutex_init(&ctrl->queues[i].queue_lock); > + > + rc = ctrl->dev->ops->create_queue(&ctrl->queues[i], > + i, nctrl->sqsize + 1); > + if (rc) > + goto out_free_queues; > + > + set_bit(NVME_TCP_OFLD_Q_ALLOCATED, &ctrl->queues[i].flags); > + } > + > + return 0; > + > +out_free_queues: > + for (i--; i >= 1; i--) > + nvme_tcp_ofld_free_queue(nctrl, i); > + > + return rc; > +} > + > +static int nvme_tcp_ofld_alloc_io_queues(struct nvme_ctrl *nctrl) > +{ > + unsigned int nr_io_queues; > + int rc; > + > + nr_io_queues = nvme_tcp_ofld_nr_io_queues(nctrl); > + rc = nvme_set_queue_count(nctrl, &nr_io_queues); > + if (rc) > + return rc; > + > + nctrl->queue_count = nr_io_queues + 1; > + if (nctrl->queue_count < 2) { > + dev_err(nctrl->device, > + "unable to set any I/O queues\n"); > + > + return -ENOMEM; > + } > + > + dev_info(nctrl->device, "creating %d I/O queues.\n", nr_io_queues); > + nvme_tcp_ofld_set_io_queues(nctrl, nr_io_queues); > + > + return nvme_tcp_ofld_create_io_queues(nctrl); > +} > + > +static int nvme_tcp_ofld_start_io_queues(struct nvme_ctrl *nctrl) > +{ > + int i, rc = 0; > + > + for (i = 1; i < nctrl->queue_count; i++) { > + rc = nvme_tcp_ofld_start_queue(nctrl, i); > + if (rc) > + goto out_stop_queues; > + } > + > + return 0; > + > +out_stop_queues: > + for (i--; i >= 1; i--) > + nvme_tcp_ofld_stop_queue(nctrl, i); > > return rc; > } > @@ -249,9 +486,10 @@ static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl, > static int > nvme_tcp_ofld_configure_io_queues(struct nvme_ctrl *nctrl, bool new) > { > - int rc; > + int rc = nvme_tcp_ofld_alloc_io_queues(nctrl); > > - /* Placeholder - alloc_io_queues */ > + if (rc) > + return rc; > > if (new) { > nctrl->tagset = nvme_tcp_ofld_alloc_tagset(nctrl, false); > @@ -269,7 +507,9 @@ nvme_tcp_ofld_configure_io_queues(struct nvme_ctrl *nctrl, bool new) > } > } > > - /* Placeholder - start_io_queues */ > + rc = nvme_tcp_ofld_start_io_queues(nctrl); > + if (rc) > + goto out_cleanup_connect_q; > > if (!new) { > nvme_start_queues(nctrl); > @@ -291,16 +531,16 @@ nvme_tcp_ofld_configure_io_queues(struct nvme_ctrl *nctrl, bool new) > out_wait_freeze_timed_out: > nvme_stop_queues(nctrl); > nvme_sync_io_queues(nctrl); > - > - /* Placeholder - Stop IO queues */ > - > + nvme_tcp_ofld_stop_io_queues(nctrl); > +out_cleanup_connect_q: > + nvme_cancel_tagset(nctrl); > if (new) > blk_cleanup_queue(nctrl->connect_q); > out_free_tag_set: > if (new) > blk_mq_free_tag_set(nctrl->tagset); > out_free_io_queues: > - /* Placeholder - free_io_queues */ > + nvme_tcp_ofld_free_io_queues(nctrl); > > return rc; > } > @@ -327,6 +567,17 @@ static void nvme_tcp_ofld_reconnect_or_remove(struct nvme_ctrl *nctrl) > } > } > > +static int > +nvme_tcp_ofld_init_admin_hctx(struct blk_mq_hw_ctx *hctx, void *data, > + unsigned int hctx_idx) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = data; > + > + hctx->driver_data = &ctrl->queues[0]; > + > + return 0; > +} > + > static int nvme_tcp_ofld_setup_ctrl(struct nvme_ctrl *nctrl, bool new) > { > struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > @@ -388,9 +639,19 @@ static int nvme_tcp_ofld_setup_ctrl(struct nvme_ctrl *nctrl, bool new) > return 0; > > destroy_io: > - /* Placeholder - stop and destroy io queues*/ > + if (nctrl->queue_count > 1) { > + nvme_stop_queues(nctrl); > + nvme_sync_io_queues(nctrl); > + nvme_tcp_ofld_stop_io_queues(nctrl); > + nvme_cancel_tagset(nctrl); > + nvme_tcp_ofld_destroy_io_queues(nctrl, new); > + } > destroy_admin: > - /* Placeholder - stop and destroy admin queue*/ > + blk_mq_quiesce_queue(nctrl->admin_q); > + blk_sync_queue(nctrl->admin_q); > + nvme_tcp_ofld_stop_queue(nctrl, 0); > + nvme_cancel_admin_tagset(nctrl); > + nvme_tcp_ofld_destroy_admin_queue(nctrl, new); > out_release_ctrl: > ctrl->dev->ops->release_ctrl(ctrl); > > @@ -439,15 +700,37 @@ static void nvme_tcp_ofld_free_ctrl(struct nvme_ctrl *nctrl) > } > > static void > -nvme_tcp_ofld_teardown_admin_queue(struct nvme_ctrl *ctrl, bool remove) > +nvme_tcp_ofld_teardown_admin_queue(struct nvme_ctrl *nctrl, bool remove) > { > - /* Placeholder - teardown_admin_queue */ > + blk_mq_quiesce_queue(nctrl->admin_q); > + blk_sync_queue(nctrl->admin_q); > + > + nvme_tcp_ofld_stop_queue(nctrl, 0); > + nvme_cancel_admin_tagset(nctrl); > + > + if (remove) > + blk_mq_unquiesce_queue(nctrl->admin_q); > + > + nvme_tcp_ofld_destroy_admin_queue(nctrl, remove); > } > > static void > nvme_tcp_ofld_teardown_io_queues(struct nvme_ctrl *nctrl, bool remove) > { > - /* Placeholder - teardown_io_queues */ > + if (nctrl->queue_count <= 1) > + return; > + > + blk_mq_quiesce_queue(nctrl->admin_q); > + nvme_start_freeze(nctrl); > + nvme_stop_queues(nctrl); > + nvme_sync_io_queues(nctrl); > + nvme_tcp_ofld_stop_io_queues(nctrl); > + nvme_cancel_tagset(nctrl); > + > + if (remove) > + nvme_start_queues(nctrl); That looks odd. Surely all requests need to be flushed even for the not-remove case? > + > + nvme_tcp_ofld_destroy_io_queues(nctrl, remove); > } > > static void nvme_tcp_ofld_reconnect_ctrl_work(struct work_struct *work) > @@ -562,6 +845,12 @@ nvme_tcp_ofld_init_request(struct blk_mq_tag_set *set, > return 0; > } > > +inline size_t nvme_tcp_ofld_inline_data_size(struct nvme_tcp_ofld_queue *queue) > +{ > + return queue->cmnd_capsule_len - sizeof(struct nvme_command); > +} > +EXPORT_SYMBOL_GPL(nvme_tcp_ofld_inline_data_size); > + > static blk_status_t > nvme_tcp_ofld_queue_rq(struct blk_mq_hw_ctx *hctx, > const struct blk_mq_queue_data *bd) > @@ -573,22 +862,95 @@ nvme_tcp_ofld_queue_rq(struct blk_mq_hw_ctx *hctx, > return BLK_STS_OK; > } > > +static void > +nvme_tcp_ofld_exit_request(struct blk_mq_tag_set *set, > + struct request *rq, unsigned int hctx_idx) > +{ > + /* > + * Nothing is allocated in nvme_tcp_ofld_init_request, > + * hence empty. > + */ > +} > + > +static int > +nvme_tcp_ofld_init_hctx(struct blk_mq_hw_ctx *hctx, void *data, > + unsigned int hctx_idx) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = data; > + > + hctx->driver_data = &ctrl->queues[hctx_idx + 1]; > + > + return 0; > +} > + > +static int nvme_tcp_ofld_map_queues(struct blk_mq_tag_set *set) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = set->driver_data; > + struct nvmf_ctrl_options *opts = ctrl->nctrl.opts; > + > + if (opts->nr_write_queues && ctrl->io_queues[HCTX_TYPE_READ]) { > + /* separate read/write queues */ > + set->map[HCTX_TYPE_DEFAULT].nr_queues = > + ctrl->io_queues[HCTX_TYPE_DEFAULT]; > + set->map[HCTX_TYPE_DEFAULT].queue_offset = 0; > + set->map[HCTX_TYPE_READ].nr_queues = > + ctrl->io_queues[HCTX_TYPE_READ]; > + set->map[HCTX_TYPE_READ].queue_offset = > + ctrl->io_queues[HCTX_TYPE_DEFAULT]; > + } else { > + /* shared read/write queues */ > + set->map[HCTX_TYPE_DEFAULT].nr_queues = > + ctrl->io_queues[HCTX_TYPE_DEFAULT]; > + set->map[HCTX_TYPE_DEFAULT].queue_offset = 0; > + set->map[HCTX_TYPE_READ].nr_queues = > + ctrl->io_queues[HCTX_TYPE_DEFAULT]; > + set->map[HCTX_TYPE_READ].queue_offset = 0; > + } > + blk_mq_map_queues(&set->map[HCTX_TYPE_DEFAULT]); > + blk_mq_map_queues(&set->map[HCTX_TYPE_READ]); > + > + if (opts->nr_poll_queues && ctrl->io_queues[HCTX_TYPE_POLL]) { > + /* map dedicated poll queues only if we have queues left */ > + set->map[HCTX_TYPE_POLL].nr_queues = > + ctrl->io_queues[HCTX_TYPE_POLL]; > + set->map[HCTX_TYPE_POLL].queue_offset = > + ctrl->io_queues[HCTX_TYPE_DEFAULT] + > + ctrl->io_queues[HCTX_TYPE_READ]; > + blk_mq_map_queues(&set->map[HCTX_TYPE_POLL]); > + } > + > + dev_info(ctrl->nctrl.device, > + "mapped %d/%d/%d default/read/poll queues.\n", > + ctrl->io_queues[HCTX_TYPE_DEFAULT], > + ctrl->io_queues[HCTX_TYPE_READ], > + ctrl->io_queues[HCTX_TYPE_POLL]); > + > + return 0; > +} > + > +static int nvme_tcp_ofld_poll(struct blk_mq_hw_ctx *hctx) > +{ > + /* Placeholder - Implement polling mechanism */ > + > + return 0; > +} > + > static struct blk_mq_ops nvme_tcp_ofld_mq_ops = { > .queue_rq = nvme_tcp_ofld_queue_rq, > + .complete = nvme_complete_rq, > .init_request = nvme_tcp_ofld_init_request, > - /* > - * All additional ops will be also implemented and registered similar to > - * tcp.c > - */ > + .exit_request = nvme_tcp_ofld_exit_request, > + .init_hctx = nvme_tcp_ofld_init_hctx, > + .map_queues = nvme_tcp_ofld_map_queues, > + .poll = nvme_tcp_ofld_poll, > }; > > static struct blk_mq_ops nvme_tcp_ofld_admin_mq_ops = { > .queue_rq = nvme_tcp_ofld_queue_rq, > + .complete = nvme_complete_rq, > .init_request = nvme_tcp_ofld_init_request, > - /* > - * All additional ops will be also implemented and registered similar to > - * tcp.c > - */ > + .exit_request = nvme_tcp_ofld_exit_request, > + .init_hctx = nvme_tcp_ofld_init_admin_hctx, > }; > > static const struct nvme_ctrl_ops nvme_tcp_ofld_ctrl_ops = { > diff --git a/drivers/nvme/host/tcp-offload.h b/drivers/nvme/host/tcp-offload.h > index b80cdef8511a..fcc377680d9f 100644 > --- a/drivers/nvme/host/tcp-offload.h > +++ b/drivers/nvme/host/tcp-offload.h > @@ -65,6 +65,9 @@ struct nvme_tcp_ofld_queue { > unsigned long flags; > size_t cmnd_capsule_len; > > + /* mutex used during stop_queue */ > + struct mutex queue_lock; > + > u8 hdr_digest; > u8 data_digest; > u8 tos; > @@ -197,3 +200,4 @@ struct nvme_tcp_ofld_ops { > int nvme_tcp_ofld_register_dev(struct nvme_tcp_ofld_dev *dev); > void nvme_tcp_ofld_unregister_dev(struct nvme_tcp_ofld_dev *dev); > void nvme_tcp_ofld_error_recovery(struct nvme_ctrl *nctrl); > +inline size_t nvme_tcp_ofld_inline_data_size(struct nvme_tcp_ofld_queue *queue); > Cheers, Hannes -- Dr. Hannes Reinecke Kernel Storage Architect hare@suse.de +49 911 74053 688 SUSE Software Solutions Germany GmbH, 90409 Nürnberg GF: F. Imendörffer, HRB 36809 (AG Nürnberg) From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C336C47087 for ; Fri, 28 May 2021 11:56:44 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DDCC561157 for ; Fri, 28 May 2021 11:56:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DDCC561157 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date: Message-ID:Subject:From:References:Cc:To:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=W9cBgFpCNMRvYVdyzXkeHFNyBUcnoGijTUpDkmX8uOc=; b=Jx93uPkDmY6SitZ4/IIXXDEYMI CLv/BCNy+fQRJgbqDPE4XF26AdN3MyNROmFwhkvmwXHajsBXP2p02tcVlRXJenIa3YriffSyY7z80 DqH3wqVftlXb0LxHxgYlE8wXerBRUtsVYWYc7PjsbFOXu2R5r61w5nzNDYehT4UDdsBkMtQ38YgyV A/60c43ISb7l045c7H5+gcT5G1GCM1c96CVT1YKC0BTY04fPa2U+weNFTpK9G5ddvgZvuy44DN2LQ 0iAyh2CtLhQVRpiF2DQ72XFE5/NB8+F4BzoF8ssQzZs5kLnUaKQgfgk1Wvci9HkviEXUZBgOJBLsQ yMPHBb5w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmb6J-00F8OQ-W8; Fri, 28 May 2021 11:56:28 +0000 Received: from smtp-out2.suse.de ([195.135.220.29]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmagl-00Erb5-7u for linux-nvme@lists.infradead.org; Fri, 28 May 2021 11:30:06 +0000 Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 0DA1A1FD2E; Fri, 28 May 2021 11:30:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1622201401; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YuWgbs4o0pUDWZsyt9VgIqiDbpUBaK7ftHbu223uPyc=; b=B4ktXATz6ZllLWhAHMsAfK5j20VugCm8NUoiB70oQuDu/QseLORYjoGWAP14m4Q7QOosWq qlE2g8Zuw9G5FZAUA8iR6XP1k25gJ/192mfIJok9ulhTSRf3ikYTY3j29PO3WprzosXlzE L9Y+G03qvOb4i4G45Ot2f+zLn+Mx6ZI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1622201401; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YuWgbs4o0pUDWZsyt9VgIqiDbpUBaK7ftHbu223uPyc=; b=ORJKubgfgsOF0TogL8nA1+xUWItBjhBfRDi8QBPHFdpzLs8xusSvSoi4PC96zACGLiHYcw 1/xRJwTakEYr98CQ== Received: from director2.suse.de (director2.suse-dmz.suse.de [192.168.254.72]) by imap.suse.de (Postfix) with ESMTPSA id C0C7B11906; Fri, 28 May 2021 11:30:00 +0000 (UTC) To: Shai Malin , netdev@vger.kernel.org, linux-nvme@lists.infradead.org, davem@davemloft.net, kuba@kernel.org, sagi@grimberg.me, hch@lst.de, axboe@fb.com, kbusch@kernel.org Cc: aelior@marvell.com, mkalderon@marvell.com, okulkarni@marvell.com, pkushwaha@marvell.com, malin1024@gmail.com, Dean Balandin References: <20210527235902.2185-1-smalin@marvell.com> <20210527235902.2185-8-smalin@marvell.com> From: Hannes Reinecke Organization: SUSE Linux GmbH Subject: Re: [RFC PATCH v6 07/27] nvme-tcp-offload: Add queue level implementation Message-ID: <4afc9965-cef6-1bba-9ab0-1272bfa6077f@suse.de> Date: Fri, 28 May 2021 13:29:55 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: <20210527235902.2185-8-smalin@marvell.com> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210528_043003_642705_DC76F54C X-CRM114-Status: GOOD ( 30.92 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org T24gNS8yOC8yMSAxOjU4IEFNLCBTaGFpIE1hbGluIHdyb3RlOgo+IEZyb206IERlYW4gQmFsYW5k aW4gPGRiYWxhbmRpbkBtYXJ2ZWxsLmNvbT4KPiAKPiBJbiB0aGlzIHBhdGNoIHdlIGltcGxlbWVu dCBxdWV1ZSBsZXZlbCBmdW5jdGlvbmFsaXR5Lgo+IFRoZSBpbXBsZW1lbnRhdGlvbiBpcyBzaW1p bGFyIHRvIHRoZSBudm1lLXRjcCBtb2R1bGUsIHRoZSBtYWluCj4gZGlmZmVyZW5jZSBiZWluZyB0 aGF0IHdlIGNhbGwgdGhlIHZlbmRvciBzcGVjaWZpYyBjcmVhdGVfcXVldWUgb3Agd2hpY2gKPiBj cmVhdGVzIHRoZSBUQ1AgY29ubmVjdGlvbiwgYW5kIE5WTWVUUEMgY29ubmVjdGlvbiBpbmNsdWRp bmcKPiBpY3JlcStpY3Jlc3AgbmVnb3RpYXRpb24uCj4gT25jZSBjcmVhdGVfcXVldWUgcmV0dXJu cyBzdWNjZXNzZnVsbHksIHdlIGNhbiBtb3ZlIG9uIHRvIHRoZSBmYWJyaWNzCj4gY29ubmVjdC4K PiAKPiBBY2tlZC1ieTogSWdvciBSdXNza2lraCA8aXJ1c3NraWtoQG1hcnZlbGwuY29tPgo+IFNp Z25lZC1vZmYtYnk6IERlYW4gQmFsYW5kaW4gPGRiYWxhbmRpbkBtYXJ2ZWxsLmNvbT4KPiBTaWdu ZWQtb2ZmLWJ5OiBQcmFiaGFrYXIgS3VzaHdhaGEgPHBrdXNod2FoYUBtYXJ2ZWxsLmNvbT4KPiBT aWduZWQtb2ZmLWJ5OiBPbWthciBLdWxrYXJuaSA8b2t1bGthcm5pQG1hcnZlbGwuY29tPgo+IFNp Z25lZC1vZmYtYnk6IE1pY2hhbCBLYWxkZXJvbiA8bWthbGRlcm9uQG1hcnZlbGwuY29tPgo+IFNp Z25lZC1vZmYtYnk6IEFyaWVsIEVsaW9yIDxhZWxpb3JAbWFydmVsbC5jb20+Cj4gU2lnbmVkLW9m Zi1ieTogU2hhaSBNYWxpbiA8c21hbGluQG1hcnZlbGwuY29tPgo+IFJldmlld2VkLWJ5OiBIaW1h bnNodSBNYWRoYW5pIDxoaW1hbnNodS5tYWRoYW5pQG9yYWNsZS5jb20+Cj4gLS0tCj4gIGRyaXZl cnMvbnZtZS9ob3N0L3RjcC1vZmZsb2FkLmMgfCA0MTggKysrKysrKysrKysrKysrKysrKysrKysr KysrKystLS0KPiAgZHJpdmVycy9udm1lL2hvc3QvdGNwLW9mZmxvYWQuaCB8ICAgNCArCj4gIDIg ZmlsZXMgY2hhbmdlZCwgMzk0IGluc2VydGlvbnMoKyksIDI4IGRlbGV0aW9ucygtKQo+IAo+IGRp ZmYgLS1naXQgYS9kcml2ZXJzL252bWUvaG9zdC90Y3Atb2ZmbG9hZC5jIGIvZHJpdmVycy9udm1l L2hvc3QvdGNwLW9mZmxvYWQuYwo+IGluZGV4IDUyZDMxMGY3NjM2YS4uZWZmMTBlMzFmMTdmIDEw MDY0NAo+IC0tLSBhL2RyaXZlcnMvbnZtZS9ob3N0L3RjcC1vZmZsb2FkLmMKPiArKysgYi9kcml2 ZXJzL252bWUvaG9zdC90Y3Atb2ZmbG9hZC5jCj4gQEAgLTIyLDYgKzIyLDExIEBAIHN0YXRpYyBp bmxpbmUgc3RydWN0IG52bWVfdGNwX29mbGRfY3RybCAqdG9fdGNwX29mbGRfY3RybChzdHJ1Y3Qg bnZtZV9jdHJsICpuY3RyCj4gIAlyZXR1cm4gY29udGFpbmVyX29mKG5jdHJsLCBzdHJ1Y3QgbnZt ZV90Y3Bfb2ZsZF9jdHJsLCBuY3RybCk7Cj4gIH0KPiAgCj4gK3N0YXRpYyBpbmxpbmUgaW50IG52 bWVfdGNwX29mbGRfcWlkKHN0cnVjdCBudm1lX3RjcF9vZmxkX3F1ZXVlICpxdWV1ZSkKPiArewo+ ICsJcmV0dXJuIHF1ZXVlIC0gcXVldWUtPmN0cmwtPnF1ZXVlczsKPiArfQo+ICsKPiAgLyoqCj4g ICAqIG52bWVfdGNwX29mbGRfcmVnaXN0ZXJfZGV2KCkgLSBOVk1lVENQIE9mZmxvYWQgTGlicmFy eSByZWdpc3RyYXRpb24KPiAgICogZnVuY3Rpb24uCj4gQEAgLTE4MiwxOSArMTg3LDEyNSBAQCBu dm1lX3RjcF9vZmxkX2FsbG9jX3RhZ3NldChzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCwgYm9vbCBh ZG1pbikKPiAgCXJldHVybiBzZXQ7Cj4gIH0KPiAgCj4gK3N0YXRpYyB2b2lkIF9fbnZtZV90Y3Bf b2ZsZF9zdG9wX3F1ZXVlKHN0cnVjdCBudm1lX3RjcF9vZmxkX3F1ZXVlICpxdWV1ZSkKPiArewo+ ICsJcXVldWUtPmRldi0+b3BzLT5kcmFpbl9xdWV1ZShxdWV1ZSk7Cj4gK30KPiArCj4gK3N0YXRp YyB2b2lkIG52bWVfdGNwX29mbGRfc3RvcF9xdWV1ZShzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCwg aW50IHFpZCkKPiArewo+ICsJc3RydWN0IG52bWVfdGNwX29mbGRfY3RybCAqY3RybCA9IHRvX3Rj cF9vZmxkX2N0cmwobmN0cmwpOwo+ICsJc3RydWN0IG52bWVfdGNwX29mbGRfcXVldWUgKnF1ZXVl ID0gJmN0cmwtPnF1ZXVlc1txaWRdOwo+ICsKPiArCW11dGV4X2xvY2soJnF1ZXVlLT5xdWV1ZV9s b2NrKTsKPiArCWlmICh0ZXN0X2FuZF9jbGVhcl9iaXQoTlZNRV9UQ1BfT0ZMRF9RX0xJVkUsICZx dWV1ZS0+ZmxhZ3MpKQo+ICsJCV9fbnZtZV90Y3Bfb2ZsZF9zdG9wX3F1ZXVlKHF1ZXVlKTsKPiAr CW11dGV4X3VubG9jaygmcXVldWUtPnF1ZXVlX2xvY2spOwo+ICt9Cj4gKwo+ICtzdGF0aWMgdm9p ZCBudm1lX3RjcF9vZmxkX3N0b3BfaW9fcXVldWVzKHN0cnVjdCBudm1lX2N0cmwgKmN0cmwpCj4g K3sKPiArCWludCBpOwo+ICsKPiArCWZvciAoaSA9IDE7IGkgPCBjdHJsLT5xdWV1ZV9jb3VudDsg aSsrKQo+ICsJCW52bWVfdGNwX29mbGRfc3RvcF9xdWV1ZShjdHJsLCBpKTsKPiArfQo+ICsKPiAr c3RhdGljIHZvaWQgX19udm1lX3RjcF9vZmxkX2ZyZWVfcXVldWUoc3RydWN0IG52bWVfdGNwX29m bGRfcXVldWUgKnF1ZXVlKQo+ICt7Cj4gKwlxdWV1ZS0+ZGV2LT5vcHMtPmRlc3Ryb3lfcXVldWUo cXVldWUpOwo+ICt9Cj4gKwo+ICtzdGF0aWMgdm9pZCBudm1lX3RjcF9vZmxkX2ZyZWVfcXVldWUo c3RydWN0IG52bWVfY3RybCAqbmN0cmwsIGludCBxaWQpCj4gK3sKPiArCXN0cnVjdCBudm1lX3Rj cF9vZmxkX2N0cmwgKmN0cmwgPSB0b190Y3Bfb2ZsZF9jdHJsKG5jdHJsKTsKPiArCXN0cnVjdCBu dm1lX3RjcF9vZmxkX3F1ZXVlICpxdWV1ZSA9ICZjdHJsLT5xdWV1ZXNbcWlkXTsKPiArCj4gKwl0 ZXN0X2FuZF9jbGVhcl9iaXQoTlZNRV9UQ1BfT0ZMRF9RX0FMTE9DQVRFRCwgJnF1ZXVlLT5mbGFn cyk7CgpZb3UgcmVhbGx5IHdhbnQgdG8gbWFrZSB0aGlzIGFuICdpZicgY2xhdXNlIHRvIGF2b2lk IGRvdWJsZSBmcmVlCgo+ICsKPiArCV9fbnZtZV90Y3Bfb2ZsZF9mcmVlX3F1ZXVlKHF1ZXVlKTsK PiArCj4gKwltdXRleF9kZXN0cm95KCZxdWV1ZS0+cXVldWVfbG9jayk7Cj4gK30KPiArCj4gK3N0 YXRpYyB2b2lkCj4gK252bWVfdGNwX29mbGRfZnJlZV9pb19xdWV1ZXMoc3RydWN0IG52bWVfY3Ry bCAqbmN0cmwpCj4gK3sKPiArCWludCBpOwo+ICsKPiArCWZvciAoaSA9IDE7IGkgPCBuY3RybC0+ cXVldWVfY291bnQ7IGkrKykKPiArCQludm1lX3RjcF9vZmxkX2ZyZWVfcXVldWUobmN0cmwsIGkp Owo+ICt9Cj4gKwo+ICtzdGF0aWMgdm9pZCBudm1lX3RjcF9vZmxkX2Rlc3Ryb3lfaW9fcXVldWVz KHN0cnVjdCBudm1lX2N0cmwgKm5jdHJsLCBib29sIHJlbW92ZSkKPiArewo+ICsJbnZtZV90Y3Bf b2ZsZF9zdG9wX2lvX3F1ZXVlcyhuY3RybCk7Cj4gKwlpZiAocmVtb3ZlKSB7Cj4gKwkJYmxrX2Ns ZWFudXBfcXVldWUobmN0cmwtPmNvbm5lY3RfcSk7Cj4gKwkJYmxrX21xX2ZyZWVfdGFnX3NldChu Y3RybC0+dGFnc2V0KTsKPiArCX0KPiArCW52bWVfdGNwX29mbGRfZnJlZV9pb19xdWV1ZXMobmN0 cmwpOwo+ICt9Cj4gKwo+ICtzdGF0aWMgdm9pZCBudm1lX3RjcF9vZmxkX2Rlc3Ryb3lfYWRtaW5f cXVldWUoc3RydWN0IG52bWVfY3RybCAqbmN0cmwsIGJvb2wgcmVtb3ZlKQo+ICt7Cj4gKwludm1l X3RjcF9vZmxkX3N0b3BfcXVldWUobmN0cmwsIDApOwo+ICsJaWYgKHJlbW92ZSkgewo+ICsJCWJs a19jbGVhbnVwX3F1ZXVlKG5jdHJsLT5hZG1pbl9xKTsKPiArCQlibGtfY2xlYW51cF9xdWV1ZShu Y3RybC0+ZmFicmljc19xKTsKPiArCQlibGtfbXFfZnJlZV90YWdfc2V0KG5jdHJsLT5hZG1pbl90 YWdzZXQpOwo+ICsJfQo+ICsJbnZtZV90Y3Bfb2ZsZF9mcmVlX3F1ZXVlKG5jdHJsLCAwKTsKPiAr fQo+ICsKPiArc3RhdGljIGludCBudm1lX3RjcF9vZmxkX3N0YXJ0X3F1ZXVlKHN0cnVjdCBudm1l X2N0cmwgKm5jdHJsLCBpbnQgcWlkKQo+ICt7Cj4gKwlzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9jdHJs ICpjdHJsID0gdG9fdGNwX29mbGRfY3RybChuY3RybCk7Cj4gKwlzdHJ1Y3QgbnZtZV90Y3Bfb2Zs ZF9xdWV1ZSAqcXVldWUgPSAmY3RybC0+cXVldWVzW3FpZF07Cj4gKwlpbnQgcmM7Cj4gKwo+ICsJ cXVldWUgPSAmY3RybC0+cXVldWVzW3FpZF07Cj4gKwlpZiAocWlkKSB7Cj4gKwkJcXVldWUtPmNt bmRfY2Fwc3VsZV9sZW4gPSBuY3RybC0+aW9jY3N6ICogMTY7Cj4gKwkJcmMgPSBudm1mX2Nvbm5l Y3RfaW9fcXVldWUobmN0cmwsIHFpZCwgZmFsc2UpOwo+ICsJfSBlbHNlIHsKPiArCQlxdWV1ZS0+ Y21uZF9jYXBzdWxlX2xlbiA9IHNpemVvZihzdHJ1Y3QgbnZtZV9jb21tYW5kKSArIE5WTUVfVENQ X0FETUlOX0NDU1o7Cj4gKwkJcmMgPSBudm1mX2Nvbm5lY3RfYWRtaW5fcXVldWUobmN0cmwpOwo+ ICsJfQo+ICsKPiArCWlmICghcmMpIHsKPiArCQlzZXRfYml0KE5WTUVfVENQX09GTERfUV9MSVZF LCAmcXVldWUtPmZsYWdzKTsKPiArCX0gZWxzZSB7Cj4gKwkJaWYgKHRlc3RfYml0KE5WTUVfVENQ X09GTERfUV9BTExPQ0FURUQsICZxdWV1ZS0+ZmxhZ3MpKQo+ICsJCQlfX252bWVfdGNwX29mbGRf c3RvcF9xdWV1ZShxdWV1ZSk7CgpXaHkgZG8geW91IG5lZWQgdG8gY2FsbCAnc3RvcF9xdWV1ZScg aGVyZT8KQSBmYWlsdXJlIGluZGljYXRlcyB0aGF0IHRoZSBxdWV1ZSB3YXNuJ3Qgc3RhcnRlZCwg bm8/Cgo+ICsJCWRldl9lcnIobmN0cmwtPmRldmljZSwKPiArCQkJImZhaWxlZCB0byBjb25uZWN0 IHF1ZXVlOiAlZCByZXQ9JWRcbiIsIHFpZCwgcmMpOwo+ICsJfQo+ICsKPiArCXJldHVybiByYzsK PiArfQo+ICsKPiAgc3RhdGljIGludCBudm1lX3RjcF9vZmxkX2NvbmZpZ3VyZV9hZG1pbl9xdWV1 ZShzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCwKPiAgCQkJCQkgICAgICAgYm9vbCBuZXcpCj4gIHsK PiArCXN0cnVjdCBudm1lX3RjcF9vZmxkX2N0cmwgKmN0cmwgPSB0b190Y3Bfb2ZsZF9jdHJsKG5j dHJsKTsKPiArCXN0cnVjdCBudm1lX3RjcF9vZmxkX3F1ZXVlICpxdWV1ZSA9ICZjdHJsLT5xdWV1 ZXNbMF07Cj4gIAlpbnQgcmM7Cj4gIAo+IC0JLyogUGxhY2Vob2xkZXIgLSBhbGxvY19hZG1pbl9x dWV1ZSAqLwo+ICsJbXV0ZXhfaW5pdCgmcXVldWUtPnF1ZXVlX2xvY2spOwo+ICsKPiArCXJjID0g Y3RybC0+ZGV2LT5vcHMtPmNyZWF0ZV9xdWV1ZShxdWV1ZSwgMCwgTlZNRV9BUV9ERVBUSCk7Cj4g KwlpZiAocmMpCj4gKwkJcmV0dXJuIHJjOwo+ICsKPiArCXNldF9iaXQoTlZNRV9UQ1BfT0ZMRF9R X0FMTE9DQVRFRCwgJnF1ZXVlLT5mbGFncyk7Cj4gIAlpZiAobmV3KSB7Cj4gIAkJbmN0cmwtPmFk bWluX3RhZ3NldCA9Cj4gIAkJCQludm1lX3RjcF9vZmxkX2FsbG9jX3RhZ3NldChuY3RybCwgdHJ1 ZSk7Cj4gIAkJaWYgKElTX0VSUihuY3RybC0+YWRtaW5fdGFnc2V0KSkgewo+ICAJCQlyYyA9IFBU Ul9FUlIobmN0cmwtPmFkbWluX3RhZ3NldCk7Cj4gIAkJCW5jdHJsLT5hZG1pbl90YWdzZXQgPSBO VUxMOwo+IC0JCQlnb3RvIG91dF9kZXN0cm95X3F1ZXVlOwo+ICsJCQlnb3RvIG91dF9mcmVlX3F1 ZXVlOwo+ICAJCX0KPiAgCj4gIAkJbmN0cmwtPmZhYnJpY3NfcSA9IGJsa19tcV9pbml0X3F1ZXVl KG5jdHJsLT5hZG1pbl90YWdzZXQpOwo+IEBAIC0yMTIsNyArMzIzLDkgQEAgc3RhdGljIGludCBu dm1lX3RjcF9vZmxkX2NvbmZpZ3VyZV9hZG1pbl9xdWV1ZShzdHJ1Y3QgbnZtZV9jdHJsICpuY3Ry bCwKPiAgCQl9Cj4gIAl9Cj4gIAo+IC0JLyogUGxhY2Vob2xkZXIgLSBudm1lX3RjcF9vZmxkX3N0 YXJ0X3F1ZXVlICovCj4gKwlyYyA9IG52bWVfdGNwX29mbGRfc3RhcnRfcXVldWUobmN0cmwsIDAp Owo+ICsJaWYgKHJjKQo+ICsJCWdvdG8gb3V0X2NsZWFudXBfcXVldWU7Cj4gIAo+ICAJcmMgPSBu dm1lX2VuYWJsZV9jdHJsKG5jdHJsKTsKPiAgCWlmIChyYykKPiBAQCAtMjI5LDE5ICszNDIsMTQz IEBAIHN0YXRpYyBpbnQgbnZtZV90Y3Bfb2ZsZF9jb25maWd1cmVfYWRtaW5fcXVldWUoc3RydWN0 IG52bWVfY3RybCAqbmN0cmwsCj4gIG91dF9xdWllc2NlX3F1ZXVlOgo+ICAJYmxrX21xX3F1aWVz Y2VfcXVldWUobmN0cmwtPmFkbWluX3EpOwo+ICAJYmxrX3N5bmNfcXVldWUobmN0cmwtPmFkbWlu X3EpOwo+IC0KPiAgb3V0X3N0b3BfcXVldWU6Cj4gLQkvKiBQbGFjZWhvbGRlciAtIHN0b3Agb2Zm bG9hZCBxdWV1ZSAqLwo+ICsJbnZtZV90Y3Bfb2ZsZF9zdG9wX3F1ZXVlKG5jdHJsLCAwKTsKPiAg CW52bWVfY2FuY2VsX2FkbWluX3RhZ3NldChuY3RybCk7Cj4gLQo+ICtvdXRfY2xlYW51cF9xdWV1 ZToKPiArCWlmIChuZXcpCj4gKwkJYmxrX2NsZWFudXBfcXVldWUobmN0cmwtPmFkbWluX3EpOwo+ ICBvdXRfY2xlYW51cF9mYWJyaWNzX3E6Cj4gIAlpZiAobmV3KQo+ICAJCWJsa19jbGVhbnVwX3F1 ZXVlKG5jdHJsLT5mYWJyaWNzX3EpOwo+ICBvdXRfZnJlZV90YWdzZXQ6Cj4gIAlpZiAobmV3KQo+ ICAJCWJsa19tcV9mcmVlX3RhZ19zZXQobmN0cmwtPmFkbWluX3RhZ3NldCk7Cj4gLW91dF9kZXN0 cm95X3F1ZXVlOgo+IC0JLyogUGxhY2Vob2xkZXIgLSBmcmVlIGFkbWluIHF1ZXVlICovCj4gK291 dF9mcmVlX3F1ZXVlOgo+ICsJbnZtZV90Y3Bfb2ZsZF9mcmVlX3F1ZXVlKG5jdHJsLCAwKTsKPiAr Cj4gKwlyZXR1cm4gcmM7Cj4gK30KPiArCj4gK3N0YXRpYyB1bnNpZ25lZCBpbnQgbnZtZV90Y3Bf b2ZsZF9ucl9pb19xdWV1ZXMoc3RydWN0IG52bWVfY3RybCAqbmN0cmwpCj4gK3sKPiArCXN0cnVj dCBudm1lX3RjcF9vZmxkX2N0cmwgKmN0cmwgPSB0b190Y3Bfb2ZsZF9jdHJsKG5jdHJsKTsKPiAr CXN0cnVjdCBudm1lX3RjcF9vZmxkX2RldiAqZGV2ID0gY3RybC0+ZGV2Owo+ICsJdTMyIGh3X3Zl Y3RvcnMgPSBkZXYtPm51bV9od192ZWN0b3JzOwo+ICsJdTMyIG5yX3dyaXRlX3F1ZXVlcywgbnJf cG9sbF9xdWV1ZXM7Cj4gKwl1MzIgbnJfaW9fcXVldWVzLCBucl90b3RhbF9xdWV1ZXM7Cj4gKwo+ ICsJbnJfaW9fcXVldWVzID0gbWluMyhuY3RybC0+b3B0cy0+bnJfaW9fcXVldWVzLCBudW1fb25s aW5lX2NwdXMoKSwKPiArCQkJICAgIGh3X3ZlY3RvcnMpOwo+ICsJbnJfd3JpdGVfcXVldWVzID0g bWluMyhuY3RybC0+b3B0cy0+bnJfd3JpdGVfcXVldWVzLCBudW1fb25saW5lX2NwdXMoKSwKPiAr CQkJICAgICAgIGh3X3ZlY3RvcnMpOwo+ICsJbnJfcG9sbF9xdWV1ZXMgPSBtaW4zKG5jdHJsLT5v cHRzLT5ucl9wb2xsX3F1ZXVlcywgbnVtX29ubGluZV9jcHVzKCksCj4gKwkJCSAgICAgIGh3X3Zl Y3RvcnMpOwo+ICsKPiArCW5yX3RvdGFsX3F1ZXVlcyA9IG5yX2lvX3F1ZXVlcyArIG5yX3dyaXRl X3F1ZXVlcyArIG5yX3BvbGxfcXVldWVzOwo+ICsKPiArCXJldHVybiBucl90b3RhbF9xdWV1ZXM7 Cj4gK30KPiArCj4gK3N0YXRpYyB2b2lkCj4gK252bWVfdGNwX29mbGRfc2V0X2lvX3F1ZXVlcyhz dHJ1Y3QgbnZtZV9jdHJsICpuY3RybCwgdW5zaWduZWQgaW50IG5yX2lvX3F1ZXVlcykKPiArewo+ ICsJc3RydWN0IG52bWVfdGNwX29mbGRfY3RybCAqY3RybCA9IHRvX3RjcF9vZmxkX2N0cmwobmN0 cmwpOwo+ICsJc3RydWN0IG52bWZfY3RybF9vcHRpb25zICpvcHRzID0gbmN0cmwtPm9wdHM7Cj4g Kwo+ICsJaWYgKG9wdHMtPm5yX3dyaXRlX3F1ZXVlcyAmJiBvcHRzLT5ucl9pb19xdWV1ZXMgPCBu cl9pb19xdWV1ZXMpIHsKPiArCQkvKgo+ICsJCSAqIHNlcGFyYXRlIHJlYWQvd3JpdGUgcXVldWVz Cj4gKwkJICogaGFuZCBvdXQgZGVkaWNhdGVkIGRlZmF1bHQgcXVldWVzIG9ubHkgYWZ0ZXIgd2Ug aGF2ZQo+ICsJCSAqIHN1ZmZpY2llbnQgcmVhZCBxdWV1ZXMuCj4gKwkJICovCj4gKwkJY3RybC0+ aW9fcXVldWVzW0hDVFhfVFlQRV9SRUFEXSA9IG9wdHMtPm5yX2lvX3F1ZXVlczsKPiArCQlucl9p b19xdWV1ZXMgLT0gY3RybC0+aW9fcXVldWVzW0hDVFhfVFlQRV9SRUFEXTsKPiArCQljdHJsLT5p b19xdWV1ZXNbSENUWF9UWVBFX0RFRkFVTFRdID0KPiArCQkJbWluKG9wdHMtPm5yX3dyaXRlX3F1 ZXVlcywgbnJfaW9fcXVldWVzKTsKPiArCQlucl9pb19xdWV1ZXMgLT0gY3RybC0+aW9fcXVldWVz W0hDVFhfVFlQRV9ERUZBVUxUXTsKPiArCX0gZWxzZSB7Cj4gKwkJLyoKPiArCQkgKiBzaGFyZWQg cmVhZC93cml0ZSBxdWV1ZXMKPiArCQkgKiBlaXRoZXIgbm8gd3JpdGUgcXVldWVzIHdlcmUgcmVx dWVzdGVkLCBvciB3ZSBkb24ndCBoYXZlCj4gKwkJICogc3VmZmljaWVudCBxdWV1ZSBjb3VudCB0 byBoYXZlIGRlZGljYXRlZCBkZWZhdWx0IHF1ZXVlcy4KPiArCQkgKi8KPiArCQljdHJsLT5pb19x dWV1ZXNbSENUWF9UWVBFX0RFRkFVTFRdID0KPiArCQkJbWluKG9wdHMtPm5yX2lvX3F1ZXVlcywg bnJfaW9fcXVldWVzKTsKPiArCQlucl9pb19xdWV1ZXMgLT0gY3RybC0+aW9fcXVldWVzW0hDVFhf VFlQRV9ERUZBVUxUXTsKPiArCX0KPiArCj4gKwlpZiAob3B0cy0+bnJfcG9sbF9xdWV1ZXMgJiYg bnJfaW9fcXVldWVzKSB7Cj4gKwkJLyogbWFwIGRlZGljYXRlZCBwb2xsIHF1ZXVlcyBvbmx5IGlm IHdlIGhhdmUgcXVldWVzIGxlZnQgKi8KPiArCQljdHJsLT5pb19xdWV1ZXNbSENUWF9UWVBFX1BP TExdID0KPiArCQkJbWluKG9wdHMtPm5yX3BvbGxfcXVldWVzLCBucl9pb19xdWV1ZXMpOwo+ICsJ fQo+ICt9Cj4gKwo+ICtzdGF0aWMgaW50IG52bWVfdGNwX29mbGRfY3JlYXRlX2lvX3F1ZXVlcyhz dHJ1Y3QgbnZtZV9jdHJsICpuY3RybCkKPiArewo+ICsJc3RydWN0IG52bWVfdGNwX29mbGRfY3Ry bCAqY3RybCA9IHRvX3RjcF9vZmxkX2N0cmwobmN0cmwpOwo+ICsJaW50IGksIHJjOwo+ICsKPiAr CWZvciAoaSA9IDE7IGkgPCBuY3RybC0+cXVldWVfY291bnQ7IGkrKykgewo+ICsJCW11dGV4X2lu aXQoJmN0cmwtPnF1ZXVlc1tpXS5xdWV1ZV9sb2NrKTsKPiArCj4gKwkJcmMgPSBjdHJsLT5kZXYt Pm9wcy0+Y3JlYXRlX3F1ZXVlKCZjdHJsLT5xdWV1ZXNbaV0sCj4gKwkJCQkJCSAgaSwgbmN0cmwt PnNxc2l6ZSArIDEpOwo+ICsJCWlmIChyYykKPiArCQkJZ290byBvdXRfZnJlZV9xdWV1ZXM7Cj4g Kwo+ICsJCXNldF9iaXQoTlZNRV9UQ1BfT0ZMRF9RX0FMTE9DQVRFRCwgJmN0cmwtPnF1ZXVlc1tp XS5mbGFncyk7Cj4gKwl9Cj4gKwo+ICsJcmV0dXJuIDA7Cj4gKwo+ICtvdXRfZnJlZV9xdWV1ZXM6 Cj4gKwlmb3IgKGktLTsgaSA+PSAxOyBpLS0pCj4gKwkJbnZtZV90Y3Bfb2ZsZF9mcmVlX3F1ZXVl KG5jdHJsLCBpKTsKPiArCj4gKwlyZXR1cm4gcmM7Cj4gK30KPiArCj4gK3N0YXRpYyBpbnQgbnZt ZV90Y3Bfb2ZsZF9hbGxvY19pb19xdWV1ZXMoc3RydWN0IG52bWVfY3RybCAqbmN0cmwpCj4gK3sK PiArCXVuc2lnbmVkIGludCBucl9pb19xdWV1ZXM7Cj4gKwlpbnQgcmM7Cj4gKwo+ICsJbnJfaW9f cXVldWVzID0gbnZtZV90Y3Bfb2ZsZF9ucl9pb19xdWV1ZXMobmN0cmwpOwo+ICsJcmMgPSBudm1l X3NldF9xdWV1ZV9jb3VudChuY3RybCwgJm5yX2lvX3F1ZXVlcyk7Cj4gKwlpZiAocmMpCj4gKwkJ cmV0dXJuIHJjOwo+ICsKPiArCW5jdHJsLT5xdWV1ZV9jb3VudCA9IG5yX2lvX3F1ZXVlcyArIDE7 Cj4gKwlpZiAobmN0cmwtPnF1ZXVlX2NvdW50IDwgMikgewo+ICsJCWRldl9lcnIobmN0cmwtPmRl dmljZSwKPiArCQkJInVuYWJsZSB0byBzZXQgYW55IEkvTyBxdWV1ZXNcbiIpOwo+ICsKPiArCQly ZXR1cm4gLUVOT01FTTsKPiArCX0KPiArCj4gKwlkZXZfaW5mbyhuY3RybC0+ZGV2aWNlLCAiY3Jl YXRpbmcgJWQgSS9PIHF1ZXVlcy5cbiIsIG5yX2lvX3F1ZXVlcyk7Cj4gKwludm1lX3RjcF9vZmxk X3NldF9pb19xdWV1ZXMobmN0cmwsIG5yX2lvX3F1ZXVlcyk7Cj4gKwo+ICsJcmV0dXJuIG52bWVf dGNwX29mbGRfY3JlYXRlX2lvX3F1ZXVlcyhuY3RybCk7Cj4gK30KPiArCj4gK3N0YXRpYyBpbnQg bnZtZV90Y3Bfb2ZsZF9zdGFydF9pb19xdWV1ZXMoc3RydWN0IG52bWVfY3RybCAqbmN0cmwpCj4g K3sKPiArCWludCBpLCByYyA9IDA7Cj4gKwo+ICsJZm9yIChpID0gMTsgaSA8IG5jdHJsLT5xdWV1 ZV9jb3VudDsgaSsrKSB7Cj4gKwkJcmMgPSBudm1lX3RjcF9vZmxkX3N0YXJ0X3F1ZXVlKG5jdHJs LCBpKTsKPiArCQlpZiAocmMpCj4gKwkJCWdvdG8gb3V0X3N0b3BfcXVldWVzOwo+ICsJfQo+ICsK PiArCXJldHVybiAwOwo+ICsKPiArb3V0X3N0b3BfcXVldWVzOgo+ICsJZm9yIChpLS07IGkgPj0g MTsgaS0tKQo+ICsJCW52bWVfdGNwX29mbGRfc3RvcF9xdWV1ZShuY3RybCwgaSk7Cj4gIAo+ICAJ cmV0dXJuIHJjOwo+ICB9Cj4gQEAgLTI0OSw5ICs0ODYsMTAgQEAgc3RhdGljIGludCBudm1lX3Rj cF9vZmxkX2NvbmZpZ3VyZV9hZG1pbl9xdWV1ZShzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCwKPiAg c3RhdGljIGludAo+ICBudm1lX3RjcF9vZmxkX2NvbmZpZ3VyZV9pb19xdWV1ZXMoc3RydWN0IG52 bWVfY3RybCAqbmN0cmwsIGJvb2wgbmV3KQo+ICB7Cj4gLQlpbnQgcmM7Cj4gKwlpbnQgcmMgPSBu dm1lX3RjcF9vZmxkX2FsbG9jX2lvX3F1ZXVlcyhuY3RybCk7Cj4gIAo+IC0JLyogUGxhY2Vob2xk ZXIgLSBhbGxvY19pb19xdWV1ZXMgKi8KPiArCWlmIChyYykKPiArCQlyZXR1cm4gcmM7Cj4gIAo+ ICAJaWYgKG5ldykgewo+ICAJCW5jdHJsLT50YWdzZXQgPSBudm1lX3RjcF9vZmxkX2FsbG9jX3Rh Z3NldChuY3RybCwgZmFsc2UpOwo+IEBAIC0yNjksNyArNTA3LDkgQEAgbnZtZV90Y3Bfb2ZsZF9j b25maWd1cmVfaW9fcXVldWVzKHN0cnVjdCBudm1lX2N0cmwgKm5jdHJsLCBib29sIG5ldykKPiAg CQl9Cj4gIAl9Cj4gIAo+IC0JLyogUGxhY2Vob2xkZXIgLSBzdGFydF9pb19xdWV1ZXMgKi8KPiAr CXJjID0gbnZtZV90Y3Bfb2ZsZF9zdGFydF9pb19xdWV1ZXMobmN0cmwpOwo+ICsJaWYgKHJjKQo+ ICsJCWdvdG8gb3V0X2NsZWFudXBfY29ubmVjdF9xOwo+ICAKPiAgCWlmICghbmV3KSB7Cj4gIAkJ bnZtZV9zdGFydF9xdWV1ZXMobmN0cmwpOwo+IEBAIC0yOTEsMTYgKzUzMSwxNiBAQCBudm1lX3Rj cF9vZmxkX2NvbmZpZ3VyZV9pb19xdWV1ZXMoc3RydWN0IG52bWVfY3RybCAqbmN0cmwsIGJvb2wg bmV3KQo+ICBvdXRfd2FpdF9mcmVlemVfdGltZWRfb3V0Ogo+ICAJbnZtZV9zdG9wX3F1ZXVlcyhu Y3RybCk7Cj4gIAludm1lX3N5bmNfaW9fcXVldWVzKG5jdHJsKTsKPiAtCj4gLQkvKiBQbGFjZWhv bGRlciAtIFN0b3AgSU8gcXVldWVzICovCj4gLQo+ICsJbnZtZV90Y3Bfb2ZsZF9zdG9wX2lvX3F1 ZXVlcyhuY3RybCk7Cj4gK291dF9jbGVhbnVwX2Nvbm5lY3RfcToKPiArCW52bWVfY2FuY2VsX3Rh Z3NldChuY3RybCk7Cj4gIAlpZiAobmV3KQo+ICAJCWJsa19jbGVhbnVwX3F1ZXVlKG5jdHJsLT5j b25uZWN0X3EpOwo+ICBvdXRfZnJlZV90YWdfc2V0Ogo+ICAJaWYgKG5ldykKPiAgCQlibGtfbXFf ZnJlZV90YWdfc2V0KG5jdHJsLT50YWdzZXQpOwo+ICBvdXRfZnJlZV9pb19xdWV1ZXM6Cj4gLQkv KiBQbGFjZWhvbGRlciAtIGZyZWVfaW9fcXVldWVzICovCj4gKwludm1lX3RjcF9vZmxkX2ZyZWVf aW9fcXVldWVzKG5jdHJsKTsKPiAgCj4gIAlyZXR1cm4gcmM7Cj4gIH0KPiBAQCAtMzI3LDYgKzU2 NywxNyBAQCBzdGF0aWMgdm9pZCBudm1lX3RjcF9vZmxkX3JlY29ubmVjdF9vcl9yZW1vdmUoc3Ry dWN0IG52bWVfY3RybCAqbmN0cmwpCj4gIAl9Cj4gIH0KPiAgCj4gK3N0YXRpYyBpbnQKPiArbnZt ZV90Y3Bfb2ZsZF9pbml0X2FkbWluX2hjdHgoc3RydWN0IGJsa19tcV9od19jdHggKmhjdHgsIHZv aWQgKmRhdGEsCj4gKwkJCSAgICAgIHVuc2lnbmVkIGludCBoY3R4X2lkeCkKPiArewo+ICsJc3Ry dWN0IG52bWVfdGNwX29mbGRfY3RybCAqY3RybCA9IGRhdGE7Cj4gKwo+ICsJaGN0eC0+ZHJpdmVy X2RhdGEgPSAmY3RybC0+cXVldWVzWzBdOwo+ICsKPiArCXJldHVybiAwOwo+ICt9Cj4gKwo+ICBz dGF0aWMgaW50IG52bWVfdGNwX29mbGRfc2V0dXBfY3RybChzdHJ1Y3QgbnZtZV9jdHJsICpuY3Ry bCwgYm9vbCBuZXcpCj4gIHsKPiAgCXN0cnVjdCBudm1lX3RjcF9vZmxkX2N0cmwgKmN0cmwgPSB0 b190Y3Bfb2ZsZF9jdHJsKG5jdHJsKTsKPiBAQCAtMzg4LDkgKzYzOSwxOSBAQCBzdGF0aWMgaW50 IG52bWVfdGNwX29mbGRfc2V0dXBfY3RybChzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCwgYm9vbCBu ZXcpCj4gIAlyZXR1cm4gMDsKPiAgCj4gIGRlc3Ryb3lfaW86Cj4gLQkvKiBQbGFjZWhvbGRlciAt IHN0b3AgYW5kIGRlc3Ryb3kgaW8gcXVldWVzKi8KPiArCWlmIChuY3RybC0+cXVldWVfY291bnQg PiAxKSB7Cj4gKwkJbnZtZV9zdG9wX3F1ZXVlcyhuY3RybCk7Cj4gKwkJbnZtZV9zeW5jX2lvX3F1 ZXVlcyhuY3RybCk7Cj4gKwkJbnZtZV90Y3Bfb2ZsZF9zdG9wX2lvX3F1ZXVlcyhuY3RybCk7Cj4g KwkJbnZtZV9jYW5jZWxfdGFnc2V0KG5jdHJsKTsKPiArCQludm1lX3RjcF9vZmxkX2Rlc3Ryb3lf aW9fcXVldWVzKG5jdHJsLCBuZXcpOwo+ICsJfQo+ICBkZXN0cm95X2FkbWluOgo+IC0JLyogUGxh Y2Vob2xkZXIgLSBzdG9wIGFuZCBkZXN0cm95IGFkbWluIHF1ZXVlKi8KPiArCWJsa19tcV9xdWll c2NlX3F1ZXVlKG5jdHJsLT5hZG1pbl9xKTsKPiArCWJsa19zeW5jX3F1ZXVlKG5jdHJsLT5hZG1p bl9xKTsKPiArCW52bWVfdGNwX29mbGRfc3RvcF9xdWV1ZShuY3RybCwgMCk7Cj4gKwludm1lX2Nh bmNlbF9hZG1pbl90YWdzZXQobmN0cmwpOwo+ICsJbnZtZV90Y3Bfb2ZsZF9kZXN0cm95X2FkbWlu X3F1ZXVlKG5jdHJsLCBuZXcpOwo+ICBvdXRfcmVsZWFzZV9jdHJsOgo+ICAJY3RybC0+ZGV2LT5v cHMtPnJlbGVhc2VfY3RybChjdHJsKTsKPiAgCj4gQEAgLTQzOSwxNSArNzAwLDM3IEBAIHN0YXRp YyB2b2lkIG52bWVfdGNwX29mbGRfZnJlZV9jdHJsKHN0cnVjdCBudm1lX2N0cmwgKm5jdHJsKQo+ ICB9Cj4gIAo+ICBzdGF0aWMgdm9pZAo+IC1udm1lX3RjcF9vZmxkX3RlYXJkb3duX2FkbWluX3F1 ZXVlKHN0cnVjdCBudm1lX2N0cmwgKmN0cmwsIGJvb2wgcmVtb3ZlKQo+ICtudm1lX3RjcF9vZmxk X3RlYXJkb3duX2FkbWluX3F1ZXVlKHN0cnVjdCBudm1lX2N0cmwgKm5jdHJsLCBib29sIHJlbW92 ZSkKPiAgewo+IC0JLyogUGxhY2Vob2xkZXIgLSB0ZWFyZG93bl9hZG1pbl9xdWV1ZSAqLwo+ICsJ YmxrX21xX3F1aWVzY2VfcXVldWUobmN0cmwtPmFkbWluX3EpOwo+ICsJYmxrX3N5bmNfcXVldWUo bmN0cmwtPmFkbWluX3EpOwo+ICsKPiArCW52bWVfdGNwX29mbGRfc3RvcF9xdWV1ZShuY3RybCwg MCk7Cj4gKwludm1lX2NhbmNlbF9hZG1pbl90YWdzZXQobmN0cmwpOwo+ICsKPiArCWlmIChyZW1v dmUpCj4gKwkJYmxrX21xX3VucXVpZXNjZV9xdWV1ZShuY3RybC0+YWRtaW5fcSk7Cj4gKwo+ICsJ bnZtZV90Y3Bfb2ZsZF9kZXN0cm95X2FkbWluX3F1ZXVlKG5jdHJsLCByZW1vdmUpOwo+ICB9Cj4g IAo+ICBzdGF0aWMgdm9pZAo+ICBudm1lX3RjcF9vZmxkX3RlYXJkb3duX2lvX3F1ZXVlcyhzdHJ1 Y3QgbnZtZV9jdHJsICpuY3RybCwgYm9vbCByZW1vdmUpCj4gIHsKPiAtCS8qIFBsYWNlaG9sZGVy IC0gdGVhcmRvd25faW9fcXVldWVzICovCj4gKwlpZiAobmN0cmwtPnF1ZXVlX2NvdW50IDw9IDEp Cj4gKwkJcmV0dXJuOwo+ICsKPiArCWJsa19tcV9xdWllc2NlX3F1ZXVlKG5jdHJsLT5hZG1pbl9x KTsKPiArCW52bWVfc3RhcnRfZnJlZXplKG5jdHJsKTsKPiArCW52bWVfc3RvcF9xdWV1ZXMobmN0 cmwpOwo+ICsJbnZtZV9zeW5jX2lvX3F1ZXVlcyhuY3RybCk7Cj4gKwludm1lX3RjcF9vZmxkX3N0 b3BfaW9fcXVldWVzKG5jdHJsKTsKPiArCW52bWVfY2FuY2VsX3RhZ3NldChuY3RybCk7Cj4gKwo+ ICsJaWYgKHJlbW92ZSkKPiArCQludm1lX3N0YXJ0X3F1ZXVlcyhuY3RybCk7CgpUaGF0IGxvb2tz IG9kZC4KU3VyZWx5IGFsbCByZXF1ZXN0cyBuZWVkIHRvIGJlIGZsdXNoZWQgZXZlbiBmb3IgdGhl IG5vdC1yZW1vdmUgY2FzZT8KCj4gKwo+ICsJbnZtZV90Y3Bfb2ZsZF9kZXN0cm95X2lvX3F1ZXVl cyhuY3RybCwgcmVtb3ZlKTsKPiAgfQo+ICAKPiAgc3RhdGljIHZvaWQgbnZtZV90Y3Bfb2ZsZF9y ZWNvbm5lY3RfY3RybF93b3JrKHN0cnVjdCB3b3JrX3N0cnVjdCAqd29yaykKPiBAQCAtNTYyLDYg Kzg0NSwxMiBAQCBudm1lX3RjcF9vZmxkX2luaXRfcmVxdWVzdChzdHJ1Y3QgYmxrX21xX3RhZ19z ZXQgKnNldCwKPiAgCXJldHVybiAwOwo+ICB9Cj4gIAo+ICtpbmxpbmUgc2l6ZV90IG52bWVfdGNw X29mbGRfaW5saW5lX2RhdGFfc2l6ZShzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9xdWV1ZSAqcXVldWUp Cj4gK3sKPiArCXJldHVybiBxdWV1ZS0+Y21uZF9jYXBzdWxlX2xlbiAtIHNpemVvZihzdHJ1Y3Qg bnZtZV9jb21tYW5kKTsKPiArfQo+ICtFWFBPUlRfU1lNQk9MX0dQTChudm1lX3RjcF9vZmxkX2lu bGluZV9kYXRhX3NpemUpOwo+ICsKPiAgc3RhdGljIGJsa19zdGF0dXNfdAo+ICBudm1lX3RjcF9v ZmxkX3F1ZXVlX3JxKHN0cnVjdCBibGtfbXFfaHdfY3R4ICpoY3R4LAo+ICAJCSAgICAgICBjb25z dCBzdHJ1Y3QgYmxrX21xX3F1ZXVlX2RhdGEgKmJkKQo+IEBAIC01NzMsMjIgKzg2Miw5NSBAQCBu dm1lX3RjcF9vZmxkX3F1ZXVlX3JxKHN0cnVjdCBibGtfbXFfaHdfY3R4ICpoY3R4LAo+ICAJcmV0 dXJuIEJMS19TVFNfT0s7Cj4gIH0KPiAgCj4gK3N0YXRpYyB2b2lkCj4gK252bWVfdGNwX29mbGRf ZXhpdF9yZXF1ZXN0KHN0cnVjdCBibGtfbXFfdGFnX3NldCAqc2V0LAo+ICsJCQkgICBzdHJ1Y3Qg cmVxdWVzdCAqcnEsIHVuc2lnbmVkIGludCBoY3R4X2lkeCkKPiArewo+ICsJLyoKPiArCSAqIE5v dGhpbmcgaXMgYWxsb2NhdGVkIGluIG52bWVfdGNwX29mbGRfaW5pdF9yZXF1ZXN0LAo+ICsJICog aGVuY2UgZW1wdHkuCj4gKwkgKi8KPiArfQo+ICsKPiArc3RhdGljIGludAo+ICtudm1lX3RjcF9v ZmxkX2luaXRfaGN0eChzdHJ1Y3QgYmxrX21xX2h3X2N0eCAqaGN0eCwgdm9pZCAqZGF0YSwKPiAr CQkJdW5zaWduZWQgaW50IGhjdHhfaWR4KQo+ICt7Cj4gKwlzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9j dHJsICpjdHJsID0gZGF0YTsKPiArCj4gKwloY3R4LT5kcml2ZXJfZGF0YSA9ICZjdHJsLT5xdWV1 ZXNbaGN0eF9pZHggKyAxXTsKPiArCj4gKwlyZXR1cm4gMDsKPiArfQo+ICsKPiArc3RhdGljIGlu dCBudm1lX3RjcF9vZmxkX21hcF9xdWV1ZXMoc3RydWN0IGJsa19tcV90YWdfc2V0ICpzZXQpCj4g K3sKPiArCXN0cnVjdCBudm1lX3RjcF9vZmxkX2N0cmwgKmN0cmwgPSBzZXQtPmRyaXZlcl9kYXRh Owo+ICsJc3RydWN0IG52bWZfY3RybF9vcHRpb25zICpvcHRzID0gY3RybC0+bmN0cmwub3B0czsK PiArCj4gKwlpZiAob3B0cy0+bnJfd3JpdGVfcXVldWVzICYmIGN0cmwtPmlvX3F1ZXVlc1tIQ1RY X1RZUEVfUkVBRF0pIHsKPiArCQkvKiBzZXBhcmF0ZSByZWFkL3dyaXRlIHF1ZXVlcyAqLwo+ICsJ CXNldC0+bWFwW0hDVFhfVFlQRV9ERUZBVUxUXS5ucl9xdWV1ZXMgPQo+ICsJCQljdHJsLT5pb19x dWV1ZXNbSENUWF9UWVBFX0RFRkFVTFRdOwo+ICsJCXNldC0+bWFwW0hDVFhfVFlQRV9ERUZBVUxU XS5xdWV1ZV9vZmZzZXQgPSAwOwo+ICsJCXNldC0+bWFwW0hDVFhfVFlQRV9SRUFEXS5ucl9xdWV1 ZXMgPQo+ICsJCQljdHJsLT5pb19xdWV1ZXNbSENUWF9UWVBFX1JFQURdOwo+ICsJCXNldC0+bWFw W0hDVFhfVFlQRV9SRUFEXS5xdWV1ZV9vZmZzZXQgPQo+ICsJCQljdHJsLT5pb19xdWV1ZXNbSENU WF9UWVBFX0RFRkFVTFRdOwo+ICsJfSBlbHNlIHsKPiArCQkvKiBzaGFyZWQgcmVhZC93cml0ZSBx dWV1ZXMgKi8KPiArCQlzZXQtPm1hcFtIQ1RYX1RZUEVfREVGQVVMVF0ubnJfcXVldWVzID0KPiAr CQkJY3RybC0+aW9fcXVldWVzW0hDVFhfVFlQRV9ERUZBVUxUXTsKPiArCQlzZXQtPm1hcFtIQ1RY X1RZUEVfREVGQVVMVF0ucXVldWVfb2Zmc2V0ID0gMDsKPiArCQlzZXQtPm1hcFtIQ1RYX1RZUEVf UkVBRF0ubnJfcXVldWVzID0KPiArCQkJY3RybC0+aW9fcXVldWVzW0hDVFhfVFlQRV9ERUZBVUxU XTsKPiArCQlzZXQtPm1hcFtIQ1RYX1RZUEVfUkVBRF0ucXVldWVfb2Zmc2V0ID0gMDsKPiArCX0K PiArCWJsa19tcV9tYXBfcXVldWVzKCZzZXQtPm1hcFtIQ1RYX1RZUEVfREVGQVVMVF0pOwo+ICsJ YmxrX21xX21hcF9xdWV1ZXMoJnNldC0+bWFwW0hDVFhfVFlQRV9SRUFEXSk7Cj4gKwo+ICsJaWYg KG9wdHMtPm5yX3BvbGxfcXVldWVzICYmIGN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZUEVfUE9MTF0p IHsKPiArCQkvKiBtYXAgZGVkaWNhdGVkIHBvbGwgcXVldWVzIG9ubHkgaWYgd2UgaGF2ZSBxdWV1 ZXMgbGVmdCAqLwo+ICsJCXNldC0+bWFwW0hDVFhfVFlQRV9QT0xMXS5ucl9xdWV1ZXMgPQo+ICsJ CQkJY3RybC0+aW9fcXVldWVzW0hDVFhfVFlQRV9QT0xMXTsKPiArCQlzZXQtPm1hcFtIQ1RYX1RZ UEVfUE9MTF0ucXVldWVfb2Zmc2V0ID0KPiArCQkJY3RybC0+aW9fcXVldWVzW0hDVFhfVFlQRV9E RUZBVUxUXSArCj4gKwkJCWN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZUEVfUkVBRF07Cj4gKwkJYmxr X21xX21hcF9xdWV1ZXMoJnNldC0+bWFwW0hDVFhfVFlQRV9QT0xMXSk7Cj4gKwl9Cj4gKwo+ICsJ ZGV2X2luZm8oY3RybC0+bmN0cmwuZGV2aWNlLAo+ICsJCSAibWFwcGVkICVkLyVkLyVkIGRlZmF1 bHQvcmVhZC9wb2xsIHF1ZXVlcy5cbiIsCj4gKwkJIGN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZUEVf REVGQVVMVF0sCj4gKwkJIGN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZUEVfUkVBRF0sCj4gKwkJIGN0 cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZUEVfUE9MTF0pOwo+ICsKPiArCXJldHVybiAwOwo+ICt9Cj4g Kwo+ICtzdGF0aWMgaW50IG52bWVfdGNwX29mbGRfcG9sbChzdHJ1Y3QgYmxrX21xX2h3X2N0eCAq aGN0eCkKPiArewo+ICsJLyogUGxhY2Vob2xkZXIgLSBJbXBsZW1lbnQgcG9sbGluZyBtZWNoYW5p c20gKi8KPiArCj4gKwlyZXR1cm4gMDsKPiArfQo+ICsKPiAgc3RhdGljIHN0cnVjdCBibGtfbXFf b3BzIG52bWVfdGNwX29mbGRfbXFfb3BzID0gewo+ICAJLnF1ZXVlX3JxCT0gbnZtZV90Y3Bfb2Zs ZF9xdWV1ZV9ycSwKPiArCS5jb21wbGV0ZQk9IG52bWVfY29tcGxldGVfcnEsCj4gIAkuaW5pdF9y ZXF1ZXN0CT0gbnZtZV90Y3Bfb2ZsZF9pbml0X3JlcXVlc3QsCj4gLQkvKgo+IC0JICogQWxsIGFk ZGl0aW9uYWwgb3BzIHdpbGwgYmUgYWxzbyBpbXBsZW1lbnRlZCBhbmQgcmVnaXN0ZXJlZCBzaW1p bGFyIHRvCj4gLQkgKiB0Y3AuYwo+IC0JICovCj4gKwkuZXhpdF9yZXF1ZXN0CT0gbnZtZV90Y3Bf b2ZsZF9leGl0X3JlcXVlc3QsCj4gKwkuaW5pdF9oY3R4CT0gbnZtZV90Y3Bfb2ZsZF9pbml0X2hj dHgsCj4gKwkubWFwX3F1ZXVlcwk9IG52bWVfdGNwX29mbGRfbWFwX3F1ZXVlcywKPiArCS5wb2xs CQk9IG52bWVfdGNwX29mbGRfcG9sbCwKPiAgfTsKPiAgCj4gIHN0YXRpYyBzdHJ1Y3QgYmxrX21x X29wcyBudm1lX3RjcF9vZmxkX2FkbWluX21xX29wcyA9IHsKPiAgCS5xdWV1ZV9ycQk9IG52bWVf dGNwX29mbGRfcXVldWVfcnEsCj4gKwkuY29tcGxldGUJPSBudm1lX2NvbXBsZXRlX3JxLAo+ICAJ LmluaXRfcmVxdWVzdAk9IG52bWVfdGNwX29mbGRfaW5pdF9yZXF1ZXN0LAo+IC0JLyoKPiAtCSAq IEFsbCBhZGRpdGlvbmFsIG9wcyB3aWxsIGJlIGFsc28gaW1wbGVtZW50ZWQgYW5kIHJlZ2lzdGVy ZWQgc2ltaWxhciB0bwo+IC0JICogdGNwLmMKPiAtCSAqLwo+ICsJLmV4aXRfcmVxdWVzdAk9IG52 bWVfdGNwX29mbGRfZXhpdF9yZXF1ZXN0LAo+ICsJLmluaXRfaGN0eAk9IG52bWVfdGNwX29mbGRf aW5pdF9hZG1pbl9oY3R4LAo+ICB9Owo+ICAKPiAgc3RhdGljIGNvbnN0IHN0cnVjdCBudm1lX2N0 cmxfb3BzIG52bWVfdGNwX29mbGRfY3RybF9vcHMgPSB7Cj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMv bnZtZS9ob3N0L3RjcC1vZmZsb2FkLmggYi9kcml2ZXJzL252bWUvaG9zdC90Y3Atb2ZmbG9hZC5o Cj4gaW5kZXggYjgwY2RlZjg1MTFhLi5mY2MzNzc2ODBkOWYgMTAwNjQ0Cj4gLS0tIGEvZHJpdmVy cy9udm1lL2hvc3QvdGNwLW9mZmxvYWQuaAo+ICsrKyBiL2RyaXZlcnMvbnZtZS9ob3N0L3RjcC1v ZmZsb2FkLmgKPiBAQCAtNjUsNiArNjUsOSBAQCBzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9xdWV1ZSB7 Cj4gIAl1bnNpZ25lZCBsb25nIGZsYWdzOwo+ICAJc2l6ZV90IGNtbmRfY2Fwc3VsZV9sZW47Cj4g IAo+ICsJLyogbXV0ZXggdXNlZCBkdXJpbmcgc3RvcF9xdWV1ZSAqLwo+ICsJc3RydWN0IG11dGV4 IHF1ZXVlX2xvY2s7Cj4gKwo+ICAJdTggaGRyX2RpZ2VzdDsKPiAgCXU4IGRhdGFfZGlnZXN0Owo+ ICAJdTggdG9zOwo+IEBAIC0xOTcsMyArMjAwLDQgQEAgc3RydWN0IG52bWVfdGNwX29mbGRfb3Bz IHsKPiAgaW50IG52bWVfdGNwX29mbGRfcmVnaXN0ZXJfZGV2KHN0cnVjdCBudm1lX3RjcF9vZmxk X2RldiAqZGV2KTsKPiAgdm9pZCBudm1lX3RjcF9vZmxkX3VucmVnaXN0ZXJfZGV2KHN0cnVjdCBu dm1lX3RjcF9vZmxkX2RldiAqZGV2KTsKPiAgdm9pZCBudm1lX3RjcF9vZmxkX2Vycm9yX3JlY292 ZXJ5KHN0cnVjdCBudm1lX2N0cmwgKm5jdHJsKTsKPiAraW5saW5lIHNpemVfdCBudm1lX3RjcF9v ZmxkX2lubGluZV9kYXRhX3NpemUoc3RydWN0IG52bWVfdGNwX29mbGRfcXVldWUgKnF1ZXVlKTsK PiAKQ2hlZXJzLAoKSGFubmVzCi0tIApEci4gSGFubmVzIFJlaW5lY2tlCQkgICAgICAgIEtlcm5l bCBTdG9yYWdlIEFyY2hpdGVjdApoYXJlQHN1c2UuZGUJCQkgICAgICAgICAgICAgICArNDkgOTEx IDc0MDUzIDY4OApTVVNFIFNvZnR3YXJlIFNvbHV0aW9ucyBHZXJtYW55IEdtYkgsIDkwNDA5IE7D vHJuYmVyZwpHRjogRi4gSW1lbmTDtnJmZmVyLCBIUkIgMzY4MDkgKEFHIE7DvHJuYmVyZykKCl9f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCkxpbnV4LW52bWUg bWFpbGluZyBsaXN0CkxpbnV4LW52bWVAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMu aW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LW52bWUK