From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58557C433B4 for ; Sat, 1 May 2021 16:36:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2741261458 for ; Sat, 1 May 2021 16:36:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231342AbhEAQg6 (ORCPT ); Sat, 1 May 2021 12:36:58 -0400 Received: from mx2.suse.de ([195.135.220.15]:39632 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230195AbhEAQg5 (ORCPT ); Sat, 1 May 2021 12:36:57 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 4D50BAF4E; Sat, 1 May 2021 16:36:06 +0000 (UTC) Subject: Re: [RFC PATCH v4 13/27] nvme-tcp-offload: Add queue level implementation To: Shai Malin , netdev@vger.kernel.org, linux-nvme@lists.infradead.org, sagi@grimberg.me, hch@lst.de, axboe@fb.com, kbusch@kernel.org Cc: "David S . Miller davem @ davemloft . net --cc=Jakub Kicinski" , aelior@marvell.com, mkalderon@marvell.com, okulkarni@marvell.com, pkushwaha@marvell.com, malin1024@gmail.com, Dean Balandin References: <20210429190926.5086-1-smalin@marvell.com> <20210429190926.5086-14-smalin@marvell.com> From: Hannes Reinecke Message-ID: Date: Sat, 1 May 2021 18:36:04 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: <20210429190926.5086-14-smalin@marvell.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 4/29/21 9:09 PM, Shai Malin wrote: > From: Dean Balandin > > In this patch we implement queue level functionality. > The implementation is similar to the nvme-tcp module, the main > difference being that we call the vendor specific create_queue op which > creates the TCP connection, and NVMeTPC connection including > icreq+icresp negotiation. > Once create_queue returns successfully, we can move on to the fabrics > connect. > > Acked-by: Igor Russkikh > Signed-off-by: Dean Balandin > Signed-off-by: Prabhakar Kushwaha > Signed-off-by: Omkar Kulkarni > Signed-off-by: Michal Kalderon > Signed-off-by: Ariel Elior > Signed-off-by: Shai Malin > --- > drivers/nvme/host/tcp-offload.c | 415 ++++++++++++++++++++++++++++++-- > drivers/nvme/host/tcp-offload.h | 2 +- > 2 files changed, 390 insertions(+), 27 deletions(-) > > diff --git a/drivers/nvme/host/tcp-offload.c b/drivers/nvme/host/tcp-offload.c > index 9082b11c133f..8ddce2257100 100644 > --- a/drivers/nvme/host/tcp-offload.c > +++ b/drivers/nvme/host/tcp-offload.c > @@ -22,6 +22,11 @@ static inline struct nvme_tcp_ofld_ctrl *to_tcp_ofld_ctrl(struct nvme_ctrl *nctr > return container_of(nctrl, struct nvme_tcp_ofld_ctrl, nctrl); > } > > +static inline int nvme_tcp_ofld_qid(struct nvme_tcp_ofld_queue *queue) > +{ > + return queue - queue->ctrl->queues; > +} > + > /** > * nvme_tcp_ofld_register_dev() - NVMeTCP Offload Library registration > * function. > @@ -191,12 +196,94 @@ nvme_tcp_ofld_alloc_tagset(struct nvme_ctrl *nctrl, bool admin) > return set; > } > > +static void __nvme_tcp_ofld_stop_queue(struct nvme_tcp_ofld_queue *queue) > +{ > + queue->dev->ops->drain_queue(queue); > + queue->dev->ops->destroy_queue(queue); > +} > + > +static void nvme_tcp_ofld_stop_queue(struct nvme_ctrl *nctrl, int qid) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > + struct nvme_tcp_ofld_queue *queue = &ctrl->queues[qid]; > + > + if (!test_and_clear_bit(NVME_TCP_OFLD_Q_LIVE, &queue->flags)) > + return; > + > + __nvme_tcp_ofld_stop_queue(queue); > +} > + > +static void nvme_tcp_ofld_stop_io_queues(struct nvme_ctrl *ctrl) > +{ > + int i; > + > + for (i = 1; i < ctrl->queue_count; i++) > + nvme_tcp_ofld_stop_queue(ctrl, i); > +} > + > +static void nvme_tcp_ofld_free_queue(struct nvme_ctrl *nctrl, int qid) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > + struct nvme_tcp_ofld_queue *queue = &ctrl->queues[qid]; > + > + if (!test_and_clear_bit(NVME_TCP_OFLD_Q_ALLOCATED, &queue->flags)) > + return; > + > + queue = &ctrl->queues[qid]; > + queue->ctrl = NULL; > + queue->dev = NULL; > + queue->report_err = NULL; > +} > + > +static void nvme_tcp_ofld_destroy_admin_queue(struct nvme_ctrl *nctrl, bool remove) > +{ > + nvme_tcp_ofld_stop_queue(nctrl, 0); > + if (remove) { > + blk_cleanup_queue(nctrl->admin_q); > + blk_cleanup_queue(nctrl->fabrics_q); > + blk_mq_free_tag_set(nctrl->admin_tagset); > + } > +} > + > +static int nvme_tcp_ofld_start_queue(struct nvme_ctrl *nctrl, int qid) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > + struct nvme_tcp_ofld_queue *queue = &ctrl->queues[qid]; > + int rc; > + > + queue = &ctrl->queues[qid]; > + if (qid) { > + queue->cmnd_capsule_len = nctrl->ioccsz * 16; > + rc = nvmf_connect_io_queue(nctrl, qid, false); > + } else { > + queue->cmnd_capsule_len = sizeof(struct nvme_command) + NVME_TCP_ADMIN_CCSZ; > + rc = nvmf_connect_admin_queue(nctrl); > + } > + > + if (!rc) { > + set_bit(NVME_TCP_OFLD_Q_LIVE, &queue->flags); > + } else { > + if (test_bit(NVME_TCP_OFLD_Q_ALLOCATED, &queue->flags)) > + __nvme_tcp_ofld_stop_queue(queue); > + dev_err(nctrl->device, > + "failed to connect queue: %d ret=%d\n", qid, rc); > + } > + > + return rc; > +} > + > static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl, > bool new) > { > + struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > + struct nvme_tcp_ofld_queue *queue = &ctrl->queues[0]; > int rc; > > - /* Placeholder - alloc_admin_queue */ > + rc = ctrl->dev->ops->create_queue(queue, 0, NVME_AQ_DEPTH); > + if (rc) > + return rc; > + > + set_bit(NVME_TCP_OFLD_Q_ALLOCATED, &queue->flags); > if (new) { > nctrl->admin_tagset = > nvme_tcp_ofld_alloc_tagset(nctrl, true); > @@ -221,7 +308,9 @@ static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl, > } > } > > - /* Placeholder - nvme_tcp_ofld_start_queue */ > + rc = nvme_tcp_ofld_start_queue(nctrl, 0); > + if (rc) > + goto out_cleanup_queue; > > rc = nvme_enable_ctrl(nctrl); > if (rc) > @@ -238,11 +327,12 @@ static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl, > out_quiesce_queue: > blk_mq_quiesce_queue(nctrl->admin_q); > blk_sync_queue(nctrl->admin_q); > - > out_stop_queue: > - /* Placeholder - stop offload queue */ > + nvme_tcp_ofld_stop_queue(nctrl, 0); > nvme_cancel_admin_tagset(nctrl); > - > +out_cleanup_queue: > + if (new) > + blk_cleanup_queue(nctrl->admin_q); > out_cleanup_fabrics_q: > if (new) > blk_cleanup_queue(nctrl->fabrics_q); > @@ -250,7 +340,127 @@ static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl, > if (new) > blk_mq_free_tag_set(nctrl->admin_tagset); > out_free_queue: > - /* Placeholder - free admin queue */ > + nvme_tcp_ofld_free_queue(nctrl, 0); > + > + return rc; > +} > + > +static unsigned int nvme_tcp_ofld_nr_io_queues(struct nvme_ctrl *nctrl) > +{ > + unsigned int nr_io_queues; > + > + nr_io_queues = min(nctrl->opts->nr_io_queues, num_online_cpus()); > + nr_io_queues += min(nctrl->opts->nr_write_queues, num_online_cpus()); > + nr_io_queues += min(nctrl->opts->nr_poll_queues, num_online_cpus()); > + > + return nr_io_queues; > +} > + Really? Isn't this hardware-dependent? I would have expected the hardware to impose some limitations here (# of MSIx interrupts or something). Hmm? > +static void > +nvme_tcp_ofld_set_io_queues(struct nvme_ctrl *nctrl, unsigned int nr_io_queues) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > + struct nvmf_ctrl_options *opts = nctrl->opts; > + > + if (opts->nr_write_queues && opts->nr_io_queues < nr_io_queues) { > + /* > + * separate read/write queues > + * hand out dedicated default queues only after we have > + * sufficient read queues. > + */ > + ctrl->io_queues[HCTX_TYPE_READ] = opts->nr_io_queues; > + nr_io_queues -= ctrl->io_queues[HCTX_TYPE_READ]; > + ctrl->io_queues[HCTX_TYPE_DEFAULT] = > + min(opts->nr_write_queues, nr_io_queues); > + nr_io_queues -= ctrl->io_queues[HCTX_TYPE_DEFAULT]; > + } else { > + /* > + * shared read/write queues > + * either no write queues were requested, or we don't have > + * sufficient queue count to have dedicated default queues. > + */ > + ctrl->io_queues[HCTX_TYPE_DEFAULT] = > + min(opts->nr_io_queues, nr_io_queues); > + nr_io_queues -= ctrl->io_queues[HCTX_TYPE_DEFAULT]; > + } > + > + if (opts->nr_poll_queues && nr_io_queues) { > + /* map dedicated poll queues only if we have queues left */ > + ctrl->io_queues[HCTX_TYPE_POLL] = > + min(opts->nr_poll_queues, nr_io_queues); > + } > +} > + Same here. Poll queues only ever make sense of the hardware can serve specific queue pairs without interrupts. Which again relates to the number of interrupts, and the affinity of those. Or isn't this a concern with your card? > +static void > +nvme_tcp_ofld_terminate_io_queues(struct nvme_ctrl *nctrl, int start_from) > +{ > + int i; > + > + /* admin-q will be ignored because of the loop condition */ > + for (i = start_from; i >= 1; i--) > + nvme_tcp_ofld_stop_queue(nctrl, i); > +} > + Loop condition? Care to elaborate? > +static int nvme_tcp_ofld_create_io_queues(struct nvme_ctrl *nctrl) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > + int i, rc; > + > + for (i = 1; i < nctrl->queue_count; i++) { > + rc = ctrl->dev->ops->create_queue(&ctrl->queues[i], > + i, nctrl->sqsize + 1); > + if (rc) > + goto out_free_queues; > + > + set_bit(NVME_TCP_OFLD_Q_ALLOCATED, &ctrl->queues[i].flags); > + } > + > + return 0; > + > +out_free_queues: > + nvme_tcp_ofld_terminate_io_queues(nctrl, --i); > + > + return rc; > +} > + > +static int nvme_tcp_ofld_alloc_io_queues(struct nvme_ctrl *nctrl) > +{ > + unsigned int nr_io_queues; > + int rc; > + > + nr_io_queues = nvme_tcp_ofld_nr_io_queues(nctrl); > + rc = nvme_set_queue_count(nctrl, &nr_io_queues); > + if (rc) > + return rc; > + > + nctrl->queue_count = nr_io_queues + 1; > + if (nctrl->queue_count < 2) { > + dev_err(nctrl->device, > + "unable to set any I/O queues\n"); > + > + return -ENOMEM; > + } > + > + dev_info(nctrl->device, "creating %d I/O queues.\n", nr_io_queues); > + nvme_tcp_ofld_set_io_queues(nctrl, nr_io_queues); > + > + return nvme_tcp_ofld_create_io_queues(nctrl); > +} > + > +static int nvme_tcp_ofld_start_io_queues(struct nvme_ctrl *nctrl) > +{ > + int i, rc = 0; > + > + for (i = 1; i < nctrl->queue_count; i++) { > + rc = nvme_tcp_ofld_start_queue(nctrl, i); > + if (rc) > + goto terminate_queues; > + } > + > + return 0; > + > +terminate_queues: > + nvme_tcp_ofld_terminate_io_queues(nctrl, --i); > > return rc; > } > @@ -258,9 +468,10 @@ static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl, > static int > nvme_tcp_ofld_configure_io_queues(struct nvme_ctrl *nctrl, bool new) > { > - int rc; > + int rc = nvme_tcp_ofld_alloc_io_queues(nctrl); > > - /* Placeholder - alloc_io_queues */ > + if (rc) > + return rc; > > if (new) { > nctrl->tagset = nvme_tcp_ofld_alloc_tagset(nctrl, false); > @@ -278,7 +489,9 @@ nvme_tcp_ofld_configure_io_queues(struct nvme_ctrl *nctrl, bool new) > } > } > > - /* Placeholder - start_io_queues */ > + rc = nvme_tcp_ofld_start_io_queues(nctrl); > + if (rc) > + goto out_cleanup_connect_q; > > if (!new) { > nvme_start_queues(nctrl); > @@ -300,16 +513,16 @@ nvme_tcp_ofld_configure_io_queues(struct nvme_ctrl *nctrl, bool new) > out_wait_freeze_timed_out: > nvme_stop_queues(nctrl); > nvme_sync_io_queues(nctrl); > - > - /* Placeholder - Stop IO queues */ > - > + nvme_tcp_ofld_stop_io_queues(nctrl); > +out_cleanup_connect_q: > + nvme_cancel_tagset(nctrl); > if (new) > blk_cleanup_queue(nctrl->connect_q); > out_free_tag_set: > if (new) > blk_mq_free_tag_set(nctrl->tagset); > out_free_io_queues: > - /* Placeholder - free_io_queues */ > + nvme_tcp_ofld_terminate_io_queues(nctrl, nctrl->queue_count); > > return rc; > } > @@ -336,6 +549,26 @@ static void nvme_tcp_ofld_reconnect_or_remove(struct nvme_ctrl *nctrl) > } > } > > +static int > +nvme_tcp_ofld_init_admin_hctx(struct blk_mq_hw_ctx *hctx, void *data, > + unsigned int hctx_idx) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = data; > + > + hctx->driver_data = &ctrl->queues[0]; > + > + return 0; > +} > + > +static void nvme_tcp_ofld_destroy_io_queues(struct nvme_ctrl *nctrl, bool remove) > +{ > + nvme_tcp_ofld_stop_io_queues(nctrl); > + if (remove) { > + blk_cleanup_queue(nctrl->connect_q); > + blk_mq_free_tag_set(nctrl->tagset); > + } > +} > + > static int nvme_tcp_ofld_setup_ctrl(struct nvme_ctrl *nctrl, bool new) > { > struct nvmf_ctrl_options *opts = nctrl->opts; > @@ -387,9 +620,19 @@ static int nvme_tcp_ofld_setup_ctrl(struct nvme_ctrl *nctrl, bool new) > return 0; > > destroy_io: > - /* Placeholder - stop and destroy io queues*/ > + if (nctrl->queue_count > 1) { > + nvme_stop_queues(nctrl); > + nvme_sync_io_queues(nctrl); > + nvme_tcp_ofld_stop_io_queues(nctrl); > + nvme_cancel_tagset(nctrl); > + nvme_tcp_ofld_destroy_io_queues(nctrl, new); > + } > destroy_admin: > - /* Placeholder - stop and destroy admin queue*/ > + blk_mq_quiesce_queue(nctrl->admin_q); > + blk_sync_queue(nctrl->admin_q); > + nvme_tcp_ofld_stop_queue(nctrl, 0); > + nvme_cancel_admin_tagset(nctrl); > + nvme_tcp_ofld_destroy_admin_queue(nctrl, new); > > return rc; > } > @@ -410,6 +653,18 @@ nvme_tcp_ofld_check_dev_opts(struct nvmf_ctrl_options *opts, > return 0; > } > > +static void nvme_tcp_ofld_free_ctrl_queues(struct nvme_ctrl *nctrl) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > + int i; > + > + for (i = 0; i < nctrl->queue_count; ++i) > + nvme_tcp_ofld_free_queue(nctrl, i); > + > + kfree(ctrl->queues); > + ctrl->queues = NULL; > +} > + > static void nvme_tcp_ofld_free_ctrl(struct nvme_ctrl *nctrl) > { > struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl); > @@ -419,6 +674,7 @@ static void nvme_tcp_ofld_free_ctrl(struct nvme_ctrl *nctrl) > goto free_ctrl; > > down_write(&nvme_tcp_ofld_ctrl_rwsem); > + nvme_tcp_ofld_free_ctrl_queues(nctrl); > ctrl->dev->ops->release_ctrl(ctrl); > list_del(&ctrl->list); > up_write(&nvme_tcp_ofld_ctrl_rwsem); > @@ -436,15 +692,37 @@ static void nvme_tcp_ofld_submit_async_event(struct nvme_ctrl *arg) > } > > static void > -nvme_tcp_ofld_teardown_admin_queue(struct nvme_ctrl *ctrl, bool remove) > +nvme_tcp_ofld_teardown_admin_queue(struct nvme_ctrl *nctrl, bool remove) > { > - /* Placeholder - teardown_admin_queue */ > + blk_mq_quiesce_queue(nctrl->admin_q); > + blk_sync_queue(nctrl->admin_q); > + > + nvme_tcp_ofld_stop_queue(nctrl, 0); > + nvme_cancel_admin_tagset(nctrl); > + > + if (remove) > + blk_mq_unquiesce_queue(nctrl->admin_q); > + > + nvme_tcp_ofld_destroy_admin_queue(nctrl, remove); > } > > static void > nvme_tcp_ofld_teardown_io_queues(struct nvme_ctrl *nctrl, bool remove) > { > - /* Placeholder - teardown_io_queues */ > + if (nctrl->queue_count <= 1) > + return; > + > + blk_mq_quiesce_queue(nctrl->admin_q); > + nvme_start_freeze(nctrl); > + nvme_stop_queues(nctrl); > + nvme_sync_io_queues(nctrl); > + nvme_tcp_ofld_stop_io_queues(nctrl); > + nvme_cancel_tagset(nctrl); > + > + if (remove) > + nvme_start_queues(nctrl); > + > + nvme_tcp_ofld_destroy_io_queues(nctrl, remove); > } > > static void nvme_tcp_ofld_reconnect_ctrl_work(struct work_struct *work) > @@ -572,6 +850,17 @@ nvme_tcp_ofld_init_request(struct blk_mq_tag_set *set, > return 0; > } > > +inline size_t nvme_tcp_ofld_inline_data_size(struct nvme_tcp_ofld_queue *queue) > +{ > + return queue->cmnd_capsule_len - sizeof(struct nvme_command); > +} > +EXPORT_SYMBOL_GPL(nvme_tcp_ofld_inline_data_size); > + > +static void nvme_tcp_ofld_commit_rqs(struct blk_mq_hw_ctx *hctx) > +{ > + /* Call ops->commit_rqs */ > +} > + > static blk_status_t > nvme_tcp_ofld_queue_rq(struct blk_mq_hw_ctx *hctx, > const struct blk_mq_queue_data *bd) > @@ -583,22 +872,96 @@ nvme_tcp_ofld_queue_rq(struct blk_mq_hw_ctx *hctx, > return BLK_STS_OK; > } > > +static void > +nvme_tcp_ofld_exit_request(struct blk_mq_tag_set *set, > + struct request *rq, unsigned int hctx_idx) > +{ > + /* > + * Nothing is allocated in nvme_tcp_ofld_init_request, > + * hence empty. > + */ > +} > + > +static int > +nvme_tcp_ofld_init_hctx(struct blk_mq_hw_ctx *hctx, void *data, > + unsigned int hctx_idx) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = data; > + > + hctx->driver_data = &ctrl->queues[hctx_idx + 1]; > + > + return 0; > +} > + > +static int nvme_tcp_ofld_map_queues(struct blk_mq_tag_set *set) > +{ > + struct nvme_tcp_ofld_ctrl *ctrl = set->driver_data; > + struct nvmf_ctrl_options *opts = ctrl->nctrl.opts; > + > + if (opts->nr_write_queues && ctrl->io_queues[HCTX_TYPE_READ]) { > + /* separate read/write queues */ > + set->map[HCTX_TYPE_DEFAULT].nr_queues = > + ctrl->io_queues[HCTX_TYPE_DEFAULT]; > + set->map[HCTX_TYPE_DEFAULT].queue_offset = 0; > + set->map[HCTX_TYPE_READ].nr_queues = > + ctrl->io_queues[HCTX_TYPE_READ]; > + set->map[HCTX_TYPE_READ].queue_offset = > + ctrl->io_queues[HCTX_TYPE_DEFAULT]; > + } else { > + /* shared read/write queues */ > + set->map[HCTX_TYPE_DEFAULT].nr_queues = > + ctrl->io_queues[HCTX_TYPE_DEFAULT]; > + set->map[HCTX_TYPE_DEFAULT].queue_offset = 0; > + set->map[HCTX_TYPE_READ].nr_queues = > + ctrl->io_queues[HCTX_TYPE_DEFAULT]; > + set->map[HCTX_TYPE_READ].queue_offset = 0; > + } > + blk_mq_map_queues(&set->map[HCTX_TYPE_DEFAULT]); > + blk_mq_map_queues(&set->map[HCTX_TYPE_READ]); > + > + if (opts->nr_poll_queues && ctrl->io_queues[HCTX_TYPE_POLL]) { > + /* map dedicated poll queues only if we have queues left */ > + set->map[HCTX_TYPE_POLL].nr_queues = > + ctrl->io_queues[HCTX_TYPE_POLL]; > + set->map[HCTX_TYPE_POLL].queue_offset = > + ctrl->io_queues[HCTX_TYPE_DEFAULT] + > + ctrl->io_queues[HCTX_TYPE_READ]; > + blk_mq_map_queues(&set->map[HCTX_TYPE_POLL]); > + } > + > + dev_info(ctrl->nctrl.device, > + "mapped %d/%d/%d default/read/poll queues.\n", > + ctrl->io_queues[HCTX_TYPE_DEFAULT], > + ctrl->io_queues[HCTX_TYPE_READ], > + ctrl->io_queues[HCTX_TYPE_POLL]); > + > + return 0; > +} > + > +static int nvme_tcp_ofld_poll(struct blk_mq_hw_ctx *hctx) > +{ > + /* Placeholder - Implement polling mechanism */ > + > + return 0; > +} > + > static struct blk_mq_ops nvme_tcp_ofld_mq_ops = { > .queue_rq = nvme_tcp_ofld_queue_rq, > + .commit_rqs = nvme_tcp_ofld_commit_rqs, > + .complete = nvme_complete_rq, > .init_request = nvme_tcp_ofld_init_request, > - /* > - * All additional ops will be also implemented and registered similar to > - * tcp.c > - */ > + .exit_request = nvme_tcp_ofld_exit_request, > + .init_hctx = nvme_tcp_ofld_init_hctx, > + .map_queues = nvme_tcp_ofld_map_queues, > + .poll = nvme_tcp_ofld_poll, > }; > > static struct blk_mq_ops nvme_tcp_ofld_admin_mq_ops = { > .queue_rq = nvme_tcp_ofld_queue_rq, > + .complete = nvme_complete_rq, > .init_request = nvme_tcp_ofld_init_request, > - /* > - * All additional ops will be also implemented and registered similar to > - * tcp.c > - */ > + .exit_request = nvme_tcp_ofld_exit_request, > + .init_hctx = nvme_tcp_ofld_init_admin_hctx, > }; > > static const struct nvme_ctrl_ops nvme_tcp_ofld_ctrl_ops = { > diff --git a/drivers/nvme/host/tcp-offload.h b/drivers/nvme/host/tcp-offload.h > index b23b1d7ea6fa..d82645fcf9da 100644 > --- a/drivers/nvme/host/tcp-offload.h > +++ b/drivers/nvme/host/tcp-offload.h > @@ -105,7 +105,6 @@ struct nvme_tcp_ofld_ctrl { > * Each entry in the array indicates the number of queues of > * corresponding type. > */ > - u32 queue_type_mapping[HCTX_MAX_TYPES]; > u32 io_queues[HCTX_MAX_TYPES]; > > /* Connectivity params */ > @@ -205,3 +204,4 @@ struct nvme_tcp_ofld_ops { > int nvme_tcp_ofld_register_dev(struct nvme_tcp_ofld_dev *dev); > void nvme_tcp_ofld_unregister_dev(struct nvme_tcp_ofld_dev *dev); > void nvme_tcp_ofld_error_recovery(struct nvme_ctrl *nctrl); > +inline size_t nvme_tcp_ofld_inline_data_size(struct nvme_tcp_ofld_queue *queue); > Cheers, Hannes -- Dr. Hannes Reinecke Kernel Storage Architect hare@suse.de +49 911 74053 688 SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0287CC433B4 for ; Sat, 1 May 2021 16:36:32 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 32FC7613F0 for ; Sat, 1 May 2021 16:36:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 32FC7613F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:Cc:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=fAlazK0FP+mG1w3qMY9ZmCb/4zUA+bBoZQN5ecA8X0M=; b=qCGy0iX7ZPOhhOLIkuRfDfBep FnMFvK2B2bDJaArJjs+GGoVzw58YTcRvyVnUU7CA1h+UFAkk/eTSchZ0iipKvTrykSjHLXhEGb7X+ 2KvTFMxTiLEbVLKYuYaUmz9dTwZcYU35jJeMvMVHvmSUCnZtpI102bnS9NDQIn5uI+CK5YbOWLeeN LU8Um48cXQwCKymh8zTHt4VuhPUY376PEa/RO5aaA1e3L/csNtH4vsEm+1ZD7/bucuStEIL0KRSO4 9Salt56veR/AEfr9tWvCzVaDSl8li9VzzmRbn4EDUlVQVMHapzTvt8CVPubaEj2rdmzr4SliF1Nj9 g15KcV8eg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lcsbG-00ALhY-IO; Sat, 01 May 2021 16:36:14 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lcsbD-00ALgi-NO for linux-nvme@desiato.infradead.org; Sat, 01 May 2021 16:36:12 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To: Subject:Sender:Reply-To:Content-ID:Content-Description; bh=ujPvxsVA93R0eKYZhOQBP8Cf8vOalz87ULCXOSoooaw=; b=sa7IBSGgxa/p+UHs4bcEsyzsOb xMb+jCS0L+JJi2ZrZ3jY35yoQu8/NtsxhEf2ueMGAejHdLNS74g8F8lITNYxKYKe9EPGOAPWWNw4n SFocrO2rkeczNU5mbwLOO8LUUvGo6+YDqz3VTtkup4smuzl1nsRfotq/if300V7qNB/hLQTN0ikAj LER8e0OmAN38xuLfcVDJ9q6Omn0vkva0CfNkoCIJMbcQSUJNQHaGLqM4RvjATuqvnpY+Vc+LVhwg/ EaPgx2ie5Bd8Ul+iaXUF0WFBADEDlD2EWO393jO84tjIJLly6Ox2dFjSMAcH27N2EMyT6Xc+bgNyL hhGBCHqw==; Received: from mx2.suse.de ([195.135.220.15]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lcsb9-0023vr-Mp for linux-nvme@lists.infradead.org; Sat, 01 May 2021 16:36:10 +0000 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 4D50BAF4E; Sat, 1 May 2021 16:36:06 +0000 (UTC) Subject: Re: [RFC PATCH v4 13/27] nvme-tcp-offload: Add queue level implementation To: Shai Malin , netdev@vger.kernel.org, linux-nvme@lists.infradead.org, sagi@grimberg.me, hch@lst.de, axboe@fb.com, kbusch@kernel.org Cc: "David S . Miller davem @ davemloft . net --cc=Jakub Kicinski" , aelior@marvell.com, mkalderon@marvell.com, okulkarni@marvell.com, pkushwaha@marvell.com, malin1024@gmail.com, Dean Balandin References: <20210429190926.5086-1-smalin@marvell.com> <20210429190926.5086-14-smalin@marvell.com> From: Hannes Reinecke Message-ID: Date: Sat, 1 May 2021 18:36:04 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: <20210429190926.5086-14-smalin@marvell.com> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210501_093608_070472_8958A592 X-CRM114-Status: GOOD ( 31.13 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org T24gNC8yOS8yMSA5OjA5IFBNLCBTaGFpIE1hbGluIHdyb3RlOgo+IEZyb206IERlYW4gQmFsYW5k aW4gPGRiYWxhbmRpbkBtYXJ2ZWxsLmNvbT4KPiAKPiBJbiB0aGlzIHBhdGNoIHdlIGltcGxlbWVu dCBxdWV1ZSBsZXZlbCBmdW5jdGlvbmFsaXR5Lgo+IFRoZSBpbXBsZW1lbnRhdGlvbiBpcyBzaW1p bGFyIHRvIHRoZSBudm1lLXRjcCBtb2R1bGUsIHRoZSBtYWluCj4gZGlmZmVyZW5jZSBiZWluZyB0 aGF0IHdlIGNhbGwgdGhlIHZlbmRvciBzcGVjaWZpYyBjcmVhdGVfcXVldWUgb3Agd2hpY2gKPiBj cmVhdGVzIHRoZSBUQ1AgY29ubmVjdGlvbiwgYW5kIE5WTWVUUEMgY29ubmVjdGlvbiBpbmNsdWRp bmcKPiBpY3JlcStpY3Jlc3AgbmVnb3RpYXRpb24uCj4gT25jZSBjcmVhdGVfcXVldWUgcmV0dXJu cyBzdWNjZXNzZnVsbHksIHdlIGNhbiBtb3ZlIG9uIHRvIHRoZSBmYWJyaWNzCj4gY29ubmVjdC4K PiAKPiBBY2tlZC1ieTogSWdvciBSdXNza2lraCA8aXJ1c3NraWtoQG1hcnZlbGwuY29tPgo+IFNp Z25lZC1vZmYtYnk6IERlYW4gQmFsYW5kaW4gPGRiYWxhbmRpbkBtYXJ2ZWxsLmNvbT4KPiBTaWdu ZWQtb2ZmLWJ5OiBQcmFiaGFrYXIgS3VzaHdhaGEgPHBrdXNod2FoYUBtYXJ2ZWxsLmNvbT4KPiBT aWduZWQtb2ZmLWJ5OiBPbWthciBLdWxrYXJuaSA8b2t1bGthcm5pQG1hcnZlbGwuY29tPgo+IFNp Z25lZC1vZmYtYnk6IE1pY2hhbCBLYWxkZXJvbiA8bWthbGRlcm9uQG1hcnZlbGwuY29tPgo+IFNp Z25lZC1vZmYtYnk6IEFyaWVsIEVsaW9yIDxhZWxpb3JAbWFydmVsbC5jb20+Cj4gU2lnbmVkLW9m Zi1ieTogU2hhaSBNYWxpbiA8c21hbGluQG1hcnZlbGwuY29tPgo+IC0tLQo+ICAgZHJpdmVycy9u dm1lL2hvc3QvdGNwLW9mZmxvYWQuYyB8IDQxNSArKysrKysrKysrKysrKysrKysrKysrKysrKysr KystLQo+ICAgZHJpdmVycy9udm1lL2hvc3QvdGNwLW9mZmxvYWQuaCB8ICAgMiArLQo+ICAgMiBm aWxlcyBjaGFuZ2VkLCAzOTAgaW5zZXJ0aW9ucygrKSwgMjcgZGVsZXRpb25zKC0pCj4gCj4gZGlm ZiAtLWdpdCBhL2RyaXZlcnMvbnZtZS9ob3N0L3RjcC1vZmZsb2FkLmMgYi9kcml2ZXJzL252bWUv aG9zdC90Y3Atb2ZmbG9hZC5jCj4gaW5kZXggOTA4MmIxMWMxMzNmLi44ZGRjZTIyNTcxMDAgMTAw NjQ0Cj4gLS0tIGEvZHJpdmVycy9udm1lL2hvc3QvdGNwLW9mZmxvYWQuYwo+ICsrKyBiL2RyaXZl cnMvbnZtZS9ob3N0L3RjcC1vZmZsb2FkLmMKPiBAQCAtMjIsNiArMjIsMTEgQEAgc3RhdGljIGlu bGluZSBzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9jdHJsICp0b190Y3Bfb2ZsZF9jdHJsKHN0cnVjdCBu dm1lX2N0cmwgKm5jdHIKPiAgIAlyZXR1cm4gY29udGFpbmVyX29mKG5jdHJsLCBzdHJ1Y3QgbnZt ZV90Y3Bfb2ZsZF9jdHJsLCBuY3RybCk7Cj4gICB9Cj4gICAKPiArc3RhdGljIGlubGluZSBpbnQg bnZtZV90Y3Bfb2ZsZF9xaWQoc3RydWN0IG52bWVfdGNwX29mbGRfcXVldWUgKnF1ZXVlKQo+ICt7 Cj4gKwlyZXR1cm4gcXVldWUgLSBxdWV1ZS0+Y3RybC0+cXVldWVzOwo+ICt9Cj4gKwo+ICAgLyoq Cj4gICAgKiBudm1lX3RjcF9vZmxkX3JlZ2lzdGVyX2RldigpIC0gTlZNZVRDUCBPZmZsb2FkIExp YnJhcnkgcmVnaXN0cmF0aW9uCj4gICAgKiBmdW5jdGlvbi4KPiBAQCAtMTkxLDEyICsxOTYsOTQg QEAgbnZtZV90Y3Bfb2ZsZF9hbGxvY190YWdzZXQoc3RydWN0IG52bWVfY3RybCAqbmN0cmwsIGJv b2wgYWRtaW4pCj4gICAJcmV0dXJuIHNldDsKPiAgIH0KPiAgIAo+ICtzdGF0aWMgdm9pZCBfX252 bWVfdGNwX29mbGRfc3RvcF9xdWV1ZShzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9xdWV1ZSAqcXVldWUp Cj4gK3sKPiArCXF1ZXVlLT5kZXYtPm9wcy0+ZHJhaW5fcXVldWUocXVldWUpOwo+ICsJcXVldWUt PmRldi0+b3BzLT5kZXN0cm95X3F1ZXVlKHF1ZXVlKTsKPiArfQo+ICsKPiArc3RhdGljIHZvaWQg bnZtZV90Y3Bfb2ZsZF9zdG9wX3F1ZXVlKHN0cnVjdCBudm1lX2N0cmwgKm5jdHJsLCBpbnQgcWlk KQo+ICt7Cj4gKwlzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9jdHJsICpjdHJsID0gdG9fdGNwX29mbGRf Y3RybChuY3RybCk7Cj4gKwlzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9xdWV1ZSAqcXVldWUgPSAmY3Ry bC0+cXVldWVzW3FpZF07Cj4gKwo+ICsJaWYgKCF0ZXN0X2FuZF9jbGVhcl9iaXQoTlZNRV9UQ1Bf T0ZMRF9RX0xJVkUsICZxdWV1ZS0+ZmxhZ3MpKQo+ICsJCXJldHVybjsKPiArCj4gKwlfX252bWVf dGNwX29mbGRfc3RvcF9xdWV1ZShxdWV1ZSk7Cj4gK30KPiArCj4gK3N0YXRpYyB2b2lkIG52bWVf dGNwX29mbGRfc3RvcF9pb19xdWV1ZXMoc3RydWN0IG52bWVfY3RybCAqY3RybCkKPiArewo+ICsJ aW50IGk7Cj4gKwo+ICsJZm9yIChpID0gMTsgaSA8IGN0cmwtPnF1ZXVlX2NvdW50OyBpKyspCj4g KwkJbnZtZV90Y3Bfb2ZsZF9zdG9wX3F1ZXVlKGN0cmwsIGkpOwo+ICt9Cj4gKwo+ICtzdGF0aWMg dm9pZCBudm1lX3RjcF9vZmxkX2ZyZWVfcXVldWUoc3RydWN0IG52bWVfY3RybCAqbmN0cmwsIGlu dCBxaWQpCj4gK3sKPiArCXN0cnVjdCBudm1lX3RjcF9vZmxkX2N0cmwgKmN0cmwgPSB0b190Y3Bf b2ZsZF9jdHJsKG5jdHJsKTsKPiArCXN0cnVjdCBudm1lX3RjcF9vZmxkX3F1ZXVlICpxdWV1ZSA9 ICZjdHJsLT5xdWV1ZXNbcWlkXTsKPiArCj4gKwlpZiAoIXRlc3RfYW5kX2NsZWFyX2JpdChOVk1F X1RDUF9PRkxEX1FfQUxMT0NBVEVELCAmcXVldWUtPmZsYWdzKSkKPiArCQlyZXR1cm47Cj4gKwo+ ICsJcXVldWUgPSAmY3RybC0+cXVldWVzW3FpZF07Cj4gKwlxdWV1ZS0+Y3RybCA9IE5VTEw7Cj4g KwlxdWV1ZS0+ZGV2ID0gTlVMTDsKPiArCXF1ZXVlLT5yZXBvcnRfZXJyID0gTlVMTDsKPiArfQo+ ICsKPiArc3RhdGljIHZvaWQgbnZtZV90Y3Bfb2ZsZF9kZXN0cm95X2FkbWluX3F1ZXVlKHN0cnVj dCBudm1lX2N0cmwgKm5jdHJsLCBib29sIHJlbW92ZSkKPiArewo+ICsJbnZtZV90Y3Bfb2ZsZF9z dG9wX3F1ZXVlKG5jdHJsLCAwKTsKPiArCWlmIChyZW1vdmUpIHsKPiArCQlibGtfY2xlYW51cF9x dWV1ZShuY3RybC0+YWRtaW5fcSk7Cj4gKwkJYmxrX2NsZWFudXBfcXVldWUobmN0cmwtPmZhYnJp Y3NfcSk7Cj4gKwkJYmxrX21xX2ZyZWVfdGFnX3NldChuY3RybC0+YWRtaW5fdGFnc2V0KTsKPiAr CX0KPiArfQo+ICsKPiArc3RhdGljIGludCBudm1lX3RjcF9vZmxkX3N0YXJ0X3F1ZXVlKHN0cnVj dCBudm1lX2N0cmwgKm5jdHJsLCBpbnQgcWlkKQo+ICt7Cj4gKwlzdHJ1Y3QgbnZtZV90Y3Bfb2Zs ZF9jdHJsICpjdHJsID0gdG9fdGNwX29mbGRfY3RybChuY3RybCk7Cj4gKwlzdHJ1Y3QgbnZtZV90 Y3Bfb2ZsZF9xdWV1ZSAqcXVldWUgPSAmY3RybC0+cXVldWVzW3FpZF07Cj4gKwlpbnQgcmM7Cj4g Kwo+ICsJcXVldWUgPSAmY3RybC0+cXVldWVzW3FpZF07Cj4gKwlpZiAocWlkKSB7Cj4gKwkJcXVl dWUtPmNtbmRfY2Fwc3VsZV9sZW4gPSBuY3RybC0+aW9jY3N6ICogMTY7Cj4gKwkJcmMgPSBudm1m X2Nvbm5lY3RfaW9fcXVldWUobmN0cmwsIHFpZCwgZmFsc2UpOwo+ICsJfSBlbHNlIHsKPiArCQlx dWV1ZS0+Y21uZF9jYXBzdWxlX2xlbiA9IHNpemVvZihzdHJ1Y3QgbnZtZV9jb21tYW5kKSArIE5W TUVfVENQX0FETUlOX0NDU1o7Cj4gKwkJcmMgPSBudm1mX2Nvbm5lY3RfYWRtaW5fcXVldWUobmN0 cmwpOwo+ICsJfQo+ICsKPiArCWlmICghcmMpIHsKPiArCQlzZXRfYml0KE5WTUVfVENQX09GTERf UV9MSVZFLCAmcXVldWUtPmZsYWdzKTsKPiArCX0gZWxzZSB7Cj4gKwkJaWYgKHRlc3RfYml0KE5W TUVfVENQX09GTERfUV9BTExPQ0FURUQsICZxdWV1ZS0+ZmxhZ3MpKQo+ICsJCQlfX252bWVfdGNw X29mbGRfc3RvcF9xdWV1ZShxdWV1ZSk7Cj4gKwkJZGV2X2VycihuY3RybC0+ZGV2aWNlLAo+ICsJ CQkiZmFpbGVkIHRvIGNvbm5lY3QgcXVldWU6ICVkIHJldD0lZFxuIiwgcWlkLCByYyk7Cj4gKwl9 Cj4gKwo+ICsJcmV0dXJuIHJjOwo+ICt9Cj4gKwo+ICAgc3RhdGljIGludCBudm1lX3RjcF9vZmxk X2NvbmZpZ3VyZV9hZG1pbl9xdWV1ZShzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCwKPiAgIAkJCQkJ ICAgICAgIGJvb2wgbmV3KQo+ICAgewo+ICsJc3RydWN0IG52bWVfdGNwX29mbGRfY3RybCAqY3Ry bCA9IHRvX3RjcF9vZmxkX2N0cmwobmN0cmwpOwo+ICsJc3RydWN0IG52bWVfdGNwX29mbGRfcXVl dWUgKnF1ZXVlID0gJmN0cmwtPnF1ZXVlc1swXTsKPiAgIAlpbnQgcmM7Cj4gICAKPiAtCS8qIFBs YWNlaG9sZGVyIC0gYWxsb2NfYWRtaW5fcXVldWUgKi8KPiArCXJjID0gY3RybC0+ZGV2LT5vcHMt PmNyZWF0ZV9xdWV1ZShxdWV1ZSwgMCwgTlZNRV9BUV9ERVBUSCk7Cj4gKwlpZiAocmMpCj4gKwkJ cmV0dXJuIHJjOwo+ICsKPiArCXNldF9iaXQoTlZNRV9UQ1BfT0ZMRF9RX0FMTE9DQVRFRCwgJnF1 ZXVlLT5mbGFncyk7Cj4gICAJaWYgKG5ldykgewo+ICAgCQluY3RybC0+YWRtaW5fdGFnc2V0ID0K PiAgIAkJCQludm1lX3RjcF9vZmxkX2FsbG9jX3RhZ3NldChuY3RybCwgdHJ1ZSk7Cj4gQEAgLTIy MSw3ICszMDgsOSBAQCBzdGF0aWMgaW50IG52bWVfdGNwX29mbGRfY29uZmlndXJlX2FkbWluX3F1 ZXVlKHN0cnVjdCBudm1lX2N0cmwgKm5jdHJsLAo+ICAgCQl9Cj4gICAJfQo+ICAgCj4gLQkvKiBQ bGFjZWhvbGRlciAtIG52bWVfdGNwX29mbGRfc3RhcnRfcXVldWUgKi8KPiArCXJjID0gbnZtZV90 Y3Bfb2ZsZF9zdGFydF9xdWV1ZShuY3RybCwgMCk7Cj4gKwlpZiAocmMpCj4gKwkJZ290byBvdXRf Y2xlYW51cF9xdWV1ZTsKPiAgIAo+ICAgCXJjID0gbnZtZV9lbmFibGVfY3RybChuY3RybCk7Cj4g ICAJaWYgKHJjKQo+IEBAIC0yMzgsMTEgKzMyNywxMiBAQCBzdGF0aWMgaW50IG52bWVfdGNwX29m bGRfY29uZmlndXJlX2FkbWluX3F1ZXVlKHN0cnVjdCBudm1lX2N0cmwgKm5jdHJsLAo+ICAgb3V0 X3F1aWVzY2VfcXVldWU6Cj4gICAJYmxrX21xX3F1aWVzY2VfcXVldWUobmN0cmwtPmFkbWluX3Ep Owo+ICAgCWJsa19zeW5jX3F1ZXVlKG5jdHJsLT5hZG1pbl9xKTsKPiAtCj4gICBvdXRfc3RvcF9x dWV1ZToKPiAtCS8qIFBsYWNlaG9sZGVyIC0gc3RvcCBvZmZsb2FkIHF1ZXVlICovCj4gKwludm1l X3RjcF9vZmxkX3N0b3BfcXVldWUobmN0cmwsIDApOwo+ICAgCW52bWVfY2FuY2VsX2FkbWluX3Rh Z3NldChuY3RybCk7Cj4gLQo+ICtvdXRfY2xlYW51cF9xdWV1ZToKPiArCWlmIChuZXcpCj4gKwkJ YmxrX2NsZWFudXBfcXVldWUobmN0cmwtPmFkbWluX3EpOwo+ICAgb3V0X2NsZWFudXBfZmFicmlj c19xOgo+ICAgCWlmIChuZXcpCj4gICAJCWJsa19jbGVhbnVwX3F1ZXVlKG5jdHJsLT5mYWJyaWNz X3EpOwo+IEBAIC0yNTAsNyArMzQwLDEyNyBAQCBzdGF0aWMgaW50IG52bWVfdGNwX29mbGRfY29u ZmlndXJlX2FkbWluX3F1ZXVlKHN0cnVjdCBudm1lX2N0cmwgKm5jdHJsLAo+ICAgCWlmIChuZXcp Cj4gICAJCWJsa19tcV9mcmVlX3RhZ19zZXQobmN0cmwtPmFkbWluX3RhZ3NldCk7Cj4gICBvdXRf ZnJlZV9xdWV1ZToKPiAtCS8qIFBsYWNlaG9sZGVyIC0gZnJlZSBhZG1pbiBxdWV1ZSAqLwo+ICsJ bnZtZV90Y3Bfb2ZsZF9mcmVlX3F1ZXVlKG5jdHJsLCAwKTsKPiArCj4gKwlyZXR1cm4gcmM7Cj4g K30KPiArCj4gK3N0YXRpYyB1bnNpZ25lZCBpbnQgbnZtZV90Y3Bfb2ZsZF9ucl9pb19xdWV1ZXMo c3RydWN0IG52bWVfY3RybCAqbmN0cmwpCj4gK3sKPiArCXVuc2lnbmVkIGludCBucl9pb19xdWV1 ZXM7Cj4gKwo+ICsJbnJfaW9fcXVldWVzID0gbWluKG5jdHJsLT5vcHRzLT5ucl9pb19xdWV1ZXMs IG51bV9vbmxpbmVfY3B1cygpKTsKPiArCW5yX2lvX3F1ZXVlcyArPSBtaW4obmN0cmwtPm9wdHMt Pm5yX3dyaXRlX3F1ZXVlcywgbnVtX29ubGluZV9jcHVzKCkpOwo+ICsJbnJfaW9fcXVldWVzICs9 IG1pbihuY3RybC0+b3B0cy0+bnJfcG9sbF9xdWV1ZXMsIG51bV9vbmxpbmVfY3B1cygpKTsKPiAr Cj4gKwlyZXR1cm4gbnJfaW9fcXVldWVzOwo+ICt9Cj4gKwoKUmVhbGx5PyBJc24ndCB0aGlzIGhh cmR3YXJlLWRlcGVuZGVudD8KSSB3b3VsZCBoYXZlIGV4cGVjdGVkIHRoZSBoYXJkd2FyZSB0byBp bXBvc2Ugc29tZSBsaW1pdGF0aW9ucyBoZXJlICgjIG9mIApNU0l4IGludGVycnVwdHMgb3Igc29t ZXRoaW5nKS4gSG1tPwoKPiArc3RhdGljIHZvaWQKPiArbnZtZV90Y3Bfb2ZsZF9zZXRfaW9fcXVl dWVzKHN0cnVjdCBudm1lX2N0cmwgKm5jdHJsLCB1bnNpZ25lZCBpbnQgbnJfaW9fcXVldWVzKQo+ ICt7Cj4gKwlzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9jdHJsICpjdHJsID0gdG9fdGNwX29mbGRfY3Ry bChuY3RybCk7Cj4gKwlzdHJ1Y3QgbnZtZl9jdHJsX29wdGlvbnMgKm9wdHMgPSBuY3RybC0+b3B0 czsKPiArCj4gKwlpZiAob3B0cy0+bnJfd3JpdGVfcXVldWVzICYmIG9wdHMtPm5yX2lvX3F1ZXVl cyA8IG5yX2lvX3F1ZXVlcykgewo+ICsJCS8qCj4gKwkJICogc2VwYXJhdGUgcmVhZC93cml0ZSBx dWV1ZXMKPiArCQkgKiBoYW5kIG91dCBkZWRpY2F0ZWQgZGVmYXVsdCBxdWV1ZXMgb25seSBhZnRl ciB3ZSBoYXZlCj4gKwkJICogc3VmZmljaWVudCByZWFkIHF1ZXVlcy4KPiArCQkgKi8KPiArCQlj dHJsLT5pb19xdWV1ZXNbSENUWF9UWVBFX1JFQURdID0gb3B0cy0+bnJfaW9fcXVldWVzOwo+ICsJ CW5yX2lvX3F1ZXVlcyAtPSBjdHJsLT5pb19xdWV1ZXNbSENUWF9UWVBFX1JFQURdOwo+ICsJCWN0 cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZUEVfREVGQVVMVF0gPQo+ICsJCQltaW4ob3B0cy0+bnJfd3Jp dGVfcXVldWVzLCBucl9pb19xdWV1ZXMpOwo+ICsJCW5yX2lvX3F1ZXVlcyAtPSBjdHJsLT5pb19x dWV1ZXNbSENUWF9UWVBFX0RFRkFVTFRdOwo+ICsJfSBlbHNlIHsKPiArCQkvKgo+ICsJCSAqIHNo YXJlZCByZWFkL3dyaXRlIHF1ZXVlcwo+ICsJCSAqIGVpdGhlciBubyB3cml0ZSBxdWV1ZXMgd2Vy ZSByZXF1ZXN0ZWQsIG9yIHdlIGRvbid0IGhhdmUKPiArCQkgKiBzdWZmaWNpZW50IHF1ZXVlIGNv dW50IHRvIGhhdmUgZGVkaWNhdGVkIGRlZmF1bHQgcXVldWVzLgo+ICsJCSAqLwo+ICsJCWN0cmwt PmlvX3F1ZXVlc1tIQ1RYX1RZUEVfREVGQVVMVF0gPQo+ICsJCQltaW4ob3B0cy0+bnJfaW9fcXVl dWVzLCBucl9pb19xdWV1ZXMpOwo+ICsJCW5yX2lvX3F1ZXVlcyAtPSBjdHJsLT5pb19xdWV1ZXNb SENUWF9UWVBFX0RFRkFVTFRdOwo+ICsJfQo+ICsKPiArCWlmIChvcHRzLT5ucl9wb2xsX3F1ZXVl cyAmJiBucl9pb19xdWV1ZXMpIHsKPiArCQkvKiBtYXAgZGVkaWNhdGVkIHBvbGwgcXVldWVzIG9u bHkgaWYgd2UgaGF2ZSBxdWV1ZXMgbGVmdCAqLwo+ICsJCWN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZ UEVfUE9MTF0gPQo+ICsJCQltaW4ob3B0cy0+bnJfcG9sbF9xdWV1ZXMsIG5yX2lvX3F1ZXVlcyk7 Cj4gKwl9Cj4gK30KPiArCgpTYW1lIGhlcmUuClBvbGwgcXVldWVzIG9ubHkgZXZlciBtYWtlIHNl bnNlIG9mIHRoZSBoYXJkd2FyZSBjYW4gc2VydmUgc3BlY2lmaWMgCnF1ZXVlIHBhaXJzIHdpdGhv dXQgaW50ZXJydXB0cy4gV2hpY2ggYWdhaW4gcmVsYXRlcyB0byB0aGUgbnVtYmVyIG9mIAppbnRl cnJ1cHRzLCBhbmQgdGhlIGFmZmluaXR5IG9mIHRob3NlLgpPciBpc24ndCB0aGlzIGEgY29uY2Vy biB3aXRoIHlvdXIgY2FyZD8KCj4gK3N0YXRpYyB2b2lkCj4gK252bWVfdGNwX29mbGRfdGVybWlu YXRlX2lvX3F1ZXVlcyhzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCwgaW50IHN0YXJ0X2Zyb20pCj4g K3sKPiArCWludCBpOwo+ICsKPiArCS8qIGFkbWluLXEgd2lsbCBiZSBpZ25vcmVkIGJlY2F1c2Ug b2YgdGhlIGxvb3AgY29uZGl0aW9uICovCj4gKwlmb3IgKGkgPSBzdGFydF9mcm9tOyBpID49IDE7 IGktLSkKPiArCQludm1lX3RjcF9vZmxkX3N0b3BfcXVldWUobmN0cmwsIGkpOwo+ICt9Cj4gKwoK TG9vcCBjb25kaXRpb24/IENhcmUgdG8gZWxhYm9yYXRlPwoKPiArc3RhdGljIGludCBudm1lX3Rj cF9vZmxkX2NyZWF0ZV9pb19xdWV1ZXMoc3RydWN0IG52bWVfY3RybCAqbmN0cmwpCj4gK3sKPiAr CXN0cnVjdCBudm1lX3RjcF9vZmxkX2N0cmwgKmN0cmwgPSB0b190Y3Bfb2ZsZF9jdHJsKG5jdHJs KTsKPiArCWludCBpLCByYzsKPiArCj4gKwlmb3IgKGkgPSAxOyBpIDwgbmN0cmwtPnF1ZXVlX2Nv dW50OyBpKyspIHsKPiArCQlyYyA9IGN0cmwtPmRldi0+b3BzLT5jcmVhdGVfcXVldWUoJmN0cmwt PnF1ZXVlc1tpXSwKPiArCQkJCQkJICBpLCBuY3RybC0+c3FzaXplICsgMSk7Cj4gKwkJaWYgKHJj KQo+ICsJCQlnb3RvIG91dF9mcmVlX3F1ZXVlczsKPiArCj4gKwkJc2V0X2JpdChOVk1FX1RDUF9P RkxEX1FfQUxMT0NBVEVELCAmY3RybC0+cXVldWVzW2ldLmZsYWdzKTsKPiArCX0KPiArCj4gKwly ZXR1cm4gMDsKPiArCj4gK291dF9mcmVlX3F1ZXVlczoKPiArCW52bWVfdGNwX29mbGRfdGVybWlu YXRlX2lvX3F1ZXVlcyhuY3RybCwgLS1pKTsKPiArCj4gKwlyZXR1cm4gcmM7Cj4gK30KPiArCj4g K3N0YXRpYyBpbnQgbnZtZV90Y3Bfb2ZsZF9hbGxvY19pb19xdWV1ZXMoc3RydWN0IG52bWVfY3Ry bCAqbmN0cmwpCj4gK3sKPiArCXVuc2lnbmVkIGludCBucl9pb19xdWV1ZXM7Cj4gKwlpbnQgcmM7 Cj4gKwo+ICsJbnJfaW9fcXVldWVzID0gbnZtZV90Y3Bfb2ZsZF9ucl9pb19xdWV1ZXMobmN0cmwp Owo+ICsJcmMgPSBudm1lX3NldF9xdWV1ZV9jb3VudChuY3RybCwgJm5yX2lvX3F1ZXVlcyk7Cj4g KwlpZiAocmMpCj4gKwkJcmV0dXJuIHJjOwo+ICsKPiArCW5jdHJsLT5xdWV1ZV9jb3VudCA9IG5y X2lvX3F1ZXVlcyArIDE7Cj4gKwlpZiAobmN0cmwtPnF1ZXVlX2NvdW50IDwgMikgewo+ICsJCWRl dl9lcnIobmN0cmwtPmRldmljZSwKPiArCQkJInVuYWJsZSB0byBzZXQgYW55IEkvTyBxdWV1ZXNc biIpOwo+ICsKPiArCQlyZXR1cm4gLUVOT01FTTsKPiArCX0KPiArCj4gKwlkZXZfaW5mbyhuY3Ry bC0+ZGV2aWNlLCAiY3JlYXRpbmcgJWQgSS9PIHF1ZXVlcy5cbiIsIG5yX2lvX3F1ZXVlcyk7Cj4g Kwludm1lX3RjcF9vZmxkX3NldF9pb19xdWV1ZXMobmN0cmwsIG5yX2lvX3F1ZXVlcyk7Cj4gKwo+ ICsJcmV0dXJuIG52bWVfdGNwX29mbGRfY3JlYXRlX2lvX3F1ZXVlcyhuY3RybCk7Cj4gK30KPiAr Cj4gK3N0YXRpYyBpbnQgbnZtZV90Y3Bfb2ZsZF9zdGFydF9pb19xdWV1ZXMoc3RydWN0IG52bWVf Y3RybCAqbmN0cmwpCj4gK3sKPiArCWludCBpLCByYyA9IDA7Cj4gKwo+ICsJZm9yIChpID0gMTsg aSA8IG5jdHJsLT5xdWV1ZV9jb3VudDsgaSsrKSB7Cj4gKwkJcmMgPSBudm1lX3RjcF9vZmxkX3N0 YXJ0X3F1ZXVlKG5jdHJsLCBpKTsKPiArCQlpZiAocmMpCj4gKwkJCWdvdG8gdGVybWluYXRlX3F1 ZXVlczsKPiArCX0KPiArCj4gKwlyZXR1cm4gMDsKPiArCj4gK3Rlcm1pbmF0ZV9xdWV1ZXM6Cj4g Kwludm1lX3RjcF9vZmxkX3Rlcm1pbmF0ZV9pb19xdWV1ZXMobmN0cmwsIC0taSk7Cj4gICAKPiAg IAlyZXR1cm4gcmM7Cj4gICB9Cj4gQEAgLTI1OCw5ICs0NjgsMTAgQEAgc3RhdGljIGludCBudm1l X3RjcF9vZmxkX2NvbmZpZ3VyZV9hZG1pbl9xdWV1ZShzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCwK PiAgIHN0YXRpYyBpbnQKPiAgIG52bWVfdGNwX29mbGRfY29uZmlndXJlX2lvX3F1ZXVlcyhzdHJ1 Y3QgbnZtZV9jdHJsICpuY3RybCwgYm9vbCBuZXcpCj4gICB7Cj4gLQlpbnQgcmM7Cj4gKwlpbnQg cmMgPSBudm1lX3RjcF9vZmxkX2FsbG9jX2lvX3F1ZXVlcyhuY3RybCk7Cj4gICAKPiAtCS8qIFBs YWNlaG9sZGVyIC0gYWxsb2NfaW9fcXVldWVzICovCj4gKwlpZiAocmMpCj4gKwkJcmV0dXJuIHJj Owo+ICAgCj4gICAJaWYgKG5ldykgewo+ICAgCQluY3RybC0+dGFnc2V0ID0gbnZtZV90Y3Bfb2Zs ZF9hbGxvY190YWdzZXQobmN0cmwsIGZhbHNlKTsKPiBAQCAtMjc4LDcgKzQ4OSw5IEBAIG52bWVf dGNwX29mbGRfY29uZmlndXJlX2lvX3F1ZXVlcyhzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCwgYm9v bCBuZXcpCj4gICAJCX0KPiAgIAl9Cj4gICAKPiAtCS8qIFBsYWNlaG9sZGVyIC0gc3RhcnRfaW9f cXVldWVzICovCj4gKwlyYyA9IG52bWVfdGNwX29mbGRfc3RhcnRfaW9fcXVldWVzKG5jdHJsKTsK PiArCWlmIChyYykKPiArCQlnb3RvIG91dF9jbGVhbnVwX2Nvbm5lY3RfcTsKPiAgIAo+ICAgCWlm ICghbmV3KSB7Cj4gICAJCW52bWVfc3RhcnRfcXVldWVzKG5jdHJsKTsKPiBAQCAtMzAwLDE2ICs1 MTMsMTYgQEAgbnZtZV90Y3Bfb2ZsZF9jb25maWd1cmVfaW9fcXVldWVzKHN0cnVjdCBudm1lX2N0 cmwgKm5jdHJsLCBib29sIG5ldykKPiAgIG91dF93YWl0X2ZyZWV6ZV90aW1lZF9vdXQ6Cj4gICAJ bnZtZV9zdG9wX3F1ZXVlcyhuY3RybCk7Cj4gICAJbnZtZV9zeW5jX2lvX3F1ZXVlcyhuY3RybCk7 Cj4gLQo+IC0JLyogUGxhY2Vob2xkZXIgLSBTdG9wIElPIHF1ZXVlcyAqLwo+IC0KPiArCW52bWVf dGNwX29mbGRfc3RvcF9pb19xdWV1ZXMobmN0cmwpOwo+ICtvdXRfY2xlYW51cF9jb25uZWN0X3E6 Cj4gKwludm1lX2NhbmNlbF90YWdzZXQobmN0cmwpOwo+ICAgCWlmIChuZXcpCj4gICAJCWJsa19j bGVhbnVwX3F1ZXVlKG5jdHJsLT5jb25uZWN0X3EpOwo+ICAgb3V0X2ZyZWVfdGFnX3NldDoKPiAg IAlpZiAobmV3KQo+ICAgCQlibGtfbXFfZnJlZV90YWdfc2V0KG5jdHJsLT50YWdzZXQpOwo+ICAg b3V0X2ZyZWVfaW9fcXVldWVzOgo+IC0JLyogUGxhY2Vob2xkZXIgLSBmcmVlX2lvX3F1ZXVlcyAq Lwo+ICsJbnZtZV90Y3Bfb2ZsZF90ZXJtaW5hdGVfaW9fcXVldWVzKG5jdHJsLCBuY3RybC0+cXVl dWVfY291bnQpOwo+ICAgCj4gICAJcmV0dXJuIHJjOwo+ICAgfQo+IEBAIC0zMzYsNiArNTQ5LDI2 IEBAIHN0YXRpYyB2b2lkIG52bWVfdGNwX29mbGRfcmVjb25uZWN0X29yX3JlbW92ZShzdHJ1Y3Qg bnZtZV9jdHJsICpuY3RybCkKPiAgIAl9Cj4gICB9Cj4gICAKPiArc3RhdGljIGludAo+ICtudm1l X3RjcF9vZmxkX2luaXRfYWRtaW5faGN0eChzdHJ1Y3QgYmxrX21xX2h3X2N0eCAqaGN0eCwgdm9p ZCAqZGF0YSwKPiArCQkJICAgICAgdW5zaWduZWQgaW50IGhjdHhfaWR4KQo+ICt7Cj4gKwlzdHJ1 Y3QgbnZtZV90Y3Bfb2ZsZF9jdHJsICpjdHJsID0gZGF0YTsKPiArCj4gKwloY3R4LT5kcml2ZXJf ZGF0YSA9ICZjdHJsLT5xdWV1ZXNbMF07Cj4gKwo+ICsJcmV0dXJuIDA7Cj4gK30KPiArCj4gK3N0 YXRpYyB2b2lkIG52bWVfdGNwX29mbGRfZGVzdHJveV9pb19xdWV1ZXMoc3RydWN0IG52bWVfY3Ry bCAqbmN0cmwsIGJvb2wgcmVtb3ZlKQo+ICt7Cj4gKwludm1lX3RjcF9vZmxkX3N0b3BfaW9fcXVl dWVzKG5jdHJsKTsKPiArCWlmIChyZW1vdmUpIHsKPiArCQlibGtfY2xlYW51cF9xdWV1ZShuY3Ry bC0+Y29ubmVjdF9xKTsKPiArCQlibGtfbXFfZnJlZV90YWdfc2V0KG5jdHJsLT50YWdzZXQpOwo+ ICsJfQo+ICt9Cj4gKwo+ICAgc3RhdGljIGludCBudm1lX3RjcF9vZmxkX3NldHVwX2N0cmwoc3Ry dWN0IG52bWVfY3RybCAqbmN0cmwsIGJvb2wgbmV3KQo+ICAgewo+ICAgCXN0cnVjdCBudm1mX2N0 cmxfb3B0aW9ucyAqb3B0cyA9IG5jdHJsLT5vcHRzOwo+IEBAIC0zODcsOSArNjIwLDE5IEBAIHN0 YXRpYyBpbnQgbnZtZV90Y3Bfb2ZsZF9zZXR1cF9jdHJsKHN0cnVjdCBudm1lX2N0cmwgKm5jdHJs LCBib29sIG5ldykKPiAgIAlyZXR1cm4gMDsKPiAgIAo+ICAgZGVzdHJveV9pbzoKPiAtCS8qIFBs YWNlaG9sZGVyIC0gc3RvcCBhbmQgZGVzdHJveSBpbyBxdWV1ZXMqLwo+ICsJaWYgKG5jdHJsLT5x dWV1ZV9jb3VudCA+IDEpIHsKPiArCQludm1lX3N0b3BfcXVldWVzKG5jdHJsKTsKPiArCQludm1l X3N5bmNfaW9fcXVldWVzKG5jdHJsKTsKPiArCQludm1lX3RjcF9vZmxkX3N0b3BfaW9fcXVldWVz KG5jdHJsKTsKPiArCQludm1lX2NhbmNlbF90YWdzZXQobmN0cmwpOwo+ICsJCW52bWVfdGNwX29m bGRfZGVzdHJveV9pb19xdWV1ZXMobmN0cmwsIG5ldyk7Cj4gKwl9Cj4gICBkZXN0cm95X2FkbWlu Ogo+IC0JLyogUGxhY2Vob2xkZXIgLSBzdG9wIGFuZCBkZXN0cm95IGFkbWluIHF1ZXVlKi8KPiAr CWJsa19tcV9xdWllc2NlX3F1ZXVlKG5jdHJsLT5hZG1pbl9xKTsKPiArCWJsa19zeW5jX3F1ZXVl KG5jdHJsLT5hZG1pbl9xKTsKPiArCW52bWVfdGNwX29mbGRfc3RvcF9xdWV1ZShuY3RybCwgMCk7 Cj4gKwludm1lX2NhbmNlbF9hZG1pbl90YWdzZXQobmN0cmwpOwo+ICsJbnZtZV90Y3Bfb2ZsZF9k ZXN0cm95X2FkbWluX3F1ZXVlKG5jdHJsLCBuZXcpOwo+ICAgCj4gICAJcmV0dXJuIHJjOwo+ICAg fQo+IEBAIC00MTAsNiArNjUzLDE4IEBAIG52bWVfdGNwX29mbGRfY2hlY2tfZGV2X29wdHMoc3Ry dWN0IG52bWZfY3RybF9vcHRpb25zICpvcHRzLAo+ICAgCXJldHVybiAwOwo+ICAgfQo+ICAgCj4g K3N0YXRpYyB2b2lkIG52bWVfdGNwX29mbGRfZnJlZV9jdHJsX3F1ZXVlcyhzdHJ1Y3QgbnZtZV9j dHJsICpuY3RybCkKPiArewo+ICsJc3RydWN0IG52bWVfdGNwX29mbGRfY3RybCAqY3RybCA9IHRv X3RjcF9vZmxkX2N0cmwobmN0cmwpOwo+ICsJaW50IGk7Cj4gKwo+ICsJZm9yIChpID0gMDsgaSA8 IG5jdHJsLT5xdWV1ZV9jb3VudDsgKytpKQo+ICsJCW52bWVfdGNwX29mbGRfZnJlZV9xdWV1ZShu Y3RybCwgaSk7Cj4gKwo+ICsJa2ZyZWUoY3RybC0+cXVldWVzKTsKPiArCWN0cmwtPnF1ZXVlcyA9 IE5VTEw7Cj4gK30KPiArCj4gICBzdGF0aWMgdm9pZCBudm1lX3RjcF9vZmxkX2ZyZWVfY3RybChz dHJ1Y3QgbnZtZV9jdHJsICpuY3RybCkKPiAgIHsKPiAgIAlzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9j dHJsICpjdHJsID0gdG9fdGNwX29mbGRfY3RybChuY3RybCk7Cj4gQEAgLTQxOSw2ICs2NzQsNyBA QCBzdGF0aWMgdm9pZCBudm1lX3RjcF9vZmxkX2ZyZWVfY3RybChzdHJ1Y3QgbnZtZV9jdHJsICpu Y3RybCkKPiAgIAkJZ290byBmcmVlX2N0cmw7Cj4gICAKPiAgIAlkb3duX3dyaXRlKCZudm1lX3Rj cF9vZmxkX2N0cmxfcndzZW0pOwo+ICsJbnZtZV90Y3Bfb2ZsZF9mcmVlX2N0cmxfcXVldWVzKG5j dHJsKTsKPiAgIAljdHJsLT5kZXYtPm9wcy0+cmVsZWFzZV9jdHJsKGN0cmwpOwo+ICAgCWxpc3Rf ZGVsKCZjdHJsLT5saXN0KTsKPiAgIAl1cF93cml0ZSgmbnZtZV90Y3Bfb2ZsZF9jdHJsX3J3c2Vt KTsKPiBAQCAtNDM2LDE1ICs2OTIsMzcgQEAgc3RhdGljIHZvaWQgbnZtZV90Y3Bfb2ZsZF9zdWJt aXRfYXN5bmNfZXZlbnQoc3RydWN0IG52bWVfY3RybCAqYXJnKQo+ICAgfQo+ICAgCj4gICBzdGF0 aWMgdm9pZAo+IC1udm1lX3RjcF9vZmxkX3RlYXJkb3duX2FkbWluX3F1ZXVlKHN0cnVjdCBudm1l X2N0cmwgKmN0cmwsIGJvb2wgcmVtb3ZlKQo+ICtudm1lX3RjcF9vZmxkX3RlYXJkb3duX2FkbWlu X3F1ZXVlKHN0cnVjdCBudm1lX2N0cmwgKm5jdHJsLCBib29sIHJlbW92ZSkKPiAgIHsKPiAtCS8q IFBsYWNlaG9sZGVyIC0gdGVhcmRvd25fYWRtaW5fcXVldWUgKi8KPiArCWJsa19tcV9xdWllc2Nl X3F1ZXVlKG5jdHJsLT5hZG1pbl9xKTsKPiArCWJsa19zeW5jX3F1ZXVlKG5jdHJsLT5hZG1pbl9x KTsKPiArCj4gKwludm1lX3RjcF9vZmxkX3N0b3BfcXVldWUobmN0cmwsIDApOwo+ICsJbnZtZV9j YW5jZWxfYWRtaW5fdGFnc2V0KG5jdHJsKTsKPiArCj4gKwlpZiAocmVtb3ZlKQo+ICsJCWJsa19t cV91bnF1aWVzY2VfcXVldWUobmN0cmwtPmFkbWluX3EpOwo+ICsKPiArCW52bWVfdGNwX29mbGRf ZGVzdHJveV9hZG1pbl9xdWV1ZShuY3RybCwgcmVtb3ZlKTsKPiAgIH0KPiAgIAo+ICAgc3RhdGlj IHZvaWQKPiAgIG52bWVfdGNwX29mbGRfdGVhcmRvd25faW9fcXVldWVzKHN0cnVjdCBudm1lX2N0 cmwgKm5jdHJsLCBib29sIHJlbW92ZSkKPiAgIHsKPiAtCS8qIFBsYWNlaG9sZGVyIC0gdGVhcmRv d25faW9fcXVldWVzICovCj4gKwlpZiAobmN0cmwtPnF1ZXVlX2NvdW50IDw9IDEpCj4gKwkJcmV0 dXJuOwo+ICsKPiArCWJsa19tcV9xdWllc2NlX3F1ZXVlKG5jdHJsLT5hZG1pbl9xKTsKPiArCW52 bWVfc3RhcnRfZnJlZXplKG5jdHJsKTsKPiArCW52bWVfc3RvcF9xdWV1ZXMobmN0cmwpOwo+ICsJ bnZtZV9zeW5jX2lvX3F1ZXVlcyhuY3RybCk7Cj4gKwludm1lX3RjcF9vZmxkX3N0b3BfaW9fcXVl dWVzKG5jdHJsKTsKPiArCW52bWVfY2FuY2VsX3RhZ3NldChuY3RybCk7Cj4gKwo+ICsJaWYgKHJl bW92ZSkKPiArCQludm1lX3N0YXJ0X3F1ZXVlcyhuY3RybCk7Cj4gKwo+ICsJbnZtZV90Y3Bfb2Zs ZF9kZXN0cm95X2lvX3F1ZXVlcyhuY3RybCwgcmVtb3ZlKTsKPiAgIH0KPiAgIAo+ICAgc3RhdGlj IHZvaWQgbnZtZV90Y3Bfb2ZsZF9yZWNvbm5lY3RfY3RybF93b3JrKHN0cnVjdCB3b3JrX3N0cnVj dCAqd29yaykKPiBAQCAtNTcyLDYgKzg1MCwxNyBAQCBudm1lX3RjcF9vZmxkX2luaXRfcmVxdWVz dChzdHJ1Y3QgYmxrX21xX3RhZ19zZXQgKnNldCwKPiAgIAlyZXR1cm4gMDsKPiAgIH0KPiAgIAo+ ICtpbmxpbmUgc2l6ZV90IG52bWVfdGNwX29mbGRfaW5saW5lX2RhdGFfc2l6ZShzdHJ1Y3QgbnZt ZV90Y3Bfb2ZsZF9xdWV1ZSAqcXVldWUpCj4gK3sKPiArCXJldHVybiBxdWV1ZS0+Y21uZF9jYXBz dWxlX2xlbiAtIHNpemVvZihzdHJ1Y3QgbnZtZV9jb21tYW5kKTsKPiArfQo+ICtFWFBPUlRfU1lN Qk9MX0dQTChudm1lX3RjcF9vZmxkX2lubGluZV9kYXRhX3NpemUpOwo+ICsKPiArc3RhdGljIHZv aWQgbnZtZV90Y3Bfb2ZsZF9jb21taXRfcnFzKHN0cnVjdCBibGtfbXFfaHdfY3R4ICpoY3R4KQo+ ICt7Cj4gKwkvKiBDYWxsIG9wcy0+Y29tbWl0X3JxcyAqLwo+ICt9Cj4gKwo+ICAgc3RhdGljIGJs a19zdGF0dXNfdAo+ICAgbnZtZV90Y3Bfb2ZsZF9xdWV1ZV9ycShzdHJ1Y3QgYmxrX21xX2h3X2N0 eCAqaGN0eCwKPiAgIAkJICAgICAgIGNvbnN0IHN0cnVjdCBibGtfbXFfcXVldWVfZGF0YSAqYmQp Cj4gQEAgLTU4MywyMiArODcyLDk2IEBAIG52bWVfdGNwX29mbGRfcXVldWVfcnEoc3RydWN0IGJs a19tcV9od19jdHggKmhjdHgsCj4gICAJcmV0dXJuIEJMS19TVFNfT0s7Cj4gICB9Cj4gICAKPiAr c3RhdGljIHZvaWQKPiArbnZtZV90Y3Bfb2ZsZF9leGl0X3JlcXVlc3Qoc3RydWN0IGJsa19tcV90 YWdfc2V0ICpzZXQsCj4gKwkJCSAgIHN0cnVjdCByZXF1ZXN0ICpycSwgdW5zaWduZWQgaW50IGhj dHhfaWR4KQo+ICt7Cj4gKwkvKgo+ICsJICogTm90aGluZyBpcyBhbGxvY2F0ZWQgaW4gbnZtZV90 Y3Bfb2ZsZF9pbml0X3JlcXVlc3QsCj4gKwkgKiBoZW5jZSBlbXB0eS4KPiArCSAqLwo+ICt9Cj4g Kwo+ICtzdGF0aWMgaW50Cj4gK252bWVfdGNwX29mbGRfaW5pdF9oY3R4KHN0cnVjdCBibGtfbXFf aHdfY3R4ICpoY3R4LCB2b2lkICpkYXRhLAo+ICsJCQl1bnNpZ25lZCBpbnQgaGN0eF9pZHgpCj4g K3sKPiArCXN0cnVjdCBudm1lX3RjcF9vZmxkX2N0cmwgKmN0cmwgPSBkYXRhOwo+ICsKPiArCWhj dHgtPmRyaXZlcl9kYXRhID0gJmN0cmwtPnF1ZXVlc1toY3R4X2lkeCArIDFdOwo+ICsKPiArCXJl dHVybiAwOwo+ICt9Cj4gKwo+ICtzdGF0aWMgaW50IG52bWVfdGNwX29mbGRfbWFwX3F1ZXVlcyhz dHJ1Y3QgYmxrX21xX3RhZ19zZXQgKnNldCkKPiArewo+ICsJc3RydWN0IG52bWVfdGNwX29mbGRf Y3RybCAqY3RybCA9IHNldC0+ZHJpdmVyX2RhdGE7Cj4gKwlzdHJ1Y3QgbnZtZl9jdHJsX29wdGlv bnMgKm9wdHMgPSBjdHJsLT5uY3RybC5vcHRzOwo+ICsKPiArCWlmIChvcHRzLT5ucl93cml0ZV9x dWV1ZXMgJiYgY3RybC0+aW9fcXVldWVzW0hDVFhfVFlQRV9SRUFEXSkgewo+ICsJCS8qIHNlcGFy YXRlIHJlYWQvd3JpdGUgcXVldWVzICovCj4gKwkJc2V0LT5tYXBbSENUWF9UWVBFX0RFRkFVTFRd Lm5yX3F1ZXVlcyA9Cj4gKwkJCWN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZUEVfREVGQVVMVF07Cj4g KwkJc2V0LT5tYXBbSENUWF9UWVBFX0RFRkFVTFRdLnF1ZXVlX29mZnNldCA9IDA7Cj4gKwkJc2V0 LT5tYXBbSENUWF9UWVBFX1JFQURdLm5yX3F1ZXVlcyA9Cj4gKwkJCWN0cmwtPmlvX3F1ZXVlc1tI Q1RYX1RZUEVfUkVBRF07Cj4gKwkJc2V0LT5tYXBbSENUWF9UWVBFX1JFQURdLnF1ZXVlX29mZnNl dCA9Cj4gKwkJCWN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZUEVfREVGQVVMVF07Cj4gKwl9IGVsc2Ug ewo+ICsJCS8qIHNoYXJlZCByZWFkL3dyaXRlIHF1ZXVlcyAqLwo+ICsJCXNldC0+bWFwW0hDVFhf VFlQRV9ERUZBVUxUXS5ucl9xdWV1ZXMgPQo+ICsJCQljdHJsLT5pb19xdWV1ZXNbSENUWF9UWVBF X0RFRkFVTFRdOwo+ICsJCXNldC0+bWFwW0hDVFhfVFlQRV9ERUZBVUxUXS5xdWV1ZV9vZmZzZXQg PSAwOwo+ICsJCXNldC0+bWFwW0hDVFhfVFlQRV9SRUFEXS5ucl9xdWV1ZXMgPQo+ICsJCQljdHJs LT5pb19xdWV1ZXNbSENUWF9UWVBFX0RFRkFVTFRdOwo+ICsJCXNldC0+bWFwW0hDVFhfVFlQRV9S RUFEXS5xdWV1ZV9vZmZzZXQgPSAwOwo+ICsJfQo+ICsJYmxrX21xX21hcF9xdWV1ZXMoJnNldC0+ bWFwW0hDVFhfVFlQRV9ERUZBVUxUXSk7Cj4gKwlibGtfbXFfbWFwX3F1ZXVlcygmc2V0LT5tYXBb SENUWF9UWVBFX1JFQURdKTsKPiArCj4gKwlpZiAob3B0cy0+bnJfcG9sbF9xdWV1ZXMgJiYgY3Ry bC0+aW9fcXVldWVzW0hDVFhfVFlQRV9QT0xMXSkgewo+ICsJCS8qIG1hcCBkZWRpY2F0ZWQgcG9s bCBxdWV1ZXMgb25seSBpZiB3ZSBoYXZlIHF1ZXVlcyBsZWZ0ICovCj4gKwkJc2V0LT5tYXBbSENU WF9UWVBFX1BPTExdLm5yX3F1ZXVlcyA9Cj4gKwkJCQljdHJsLT5pb19xdWV1ZXNbSENUWF9UWVBF X1BPTExdOwo+ICsJCXNldC0+bWFwW0hDVFhfVFlQRV9QT0xMXS5xdWV1ZV9vZmZzZXQgPQo+ICsJ CQljdHJsLT5pb19xdWV1ZXNbSENUWF9UWVBFX0RFRkFVTFRdICsKPiArCQkJY3RybC0+aW9fcXVl dWVzW0hDVFhfVFlQRV9SRUFEXTsKPiArCQlibGtfbXFfbWFwX3F1ZXVlcygmc2V0LT5tYXBbSENU WF9UWVBFX1BPTExdKTsKPiArCX0KPiArCj4gKwlkZXZfaW5mbyhjdHJsLT5uY3RybC5kZXZpY2Us Cj4gKwkJICJtYXBwZWQgJWQvJWQvJWQgZGVmYXVsdC9yZWFkL3BvbGwgcXVldWVzLlxuIiwKPiAr CQkgY3RybC0+aW9fcXVldWVzW0hDVFhfVFlQRV9ERUZBVUxUXSwKPiArCQkgY3RybC0+aW9fcXVl dWVzW0hDVFhfVFlQRV9SRUFEXSwKPiArCQkgY3RybC0+aW9fcXVldWVzW0hDVFhfVFlQRV9QT0xM XSk7Cj4gKwo+ICsJcmV0dXJuIDA7Cj4gK30KPiArCj4gK3N0YXRpYyBpbnQgbnZtZV90Y3Bfb2Zs ZF9wb2xsKHN0cnVjdCBibGtfbXFfaHdfY3R4ICpoY3R4KQo+ICt7Cj4gKwkvKiBQbGFjZWhvbGRl ciAtIEltcGxlbWVudCBwb2xsaW5nIG1lY2hhbmlzbSAqLwo+ICsKPiArCXJldHVybiAwOwo+ICt9 Cj4gKwo+ICAgc3RhdGljIHN0cnVjdCBibGtfbXFfb3BzIG52bWVfdGNwX29mbGRfbXFfb3BzID0g ewo+ICAgCS5xdWV1ZV9ycQk9IG52bWVfdGNwX29mbGRfcXVldWVfcnEsCj4gKwkuY29tbWl0X3Jx cyAgICAgPSBudm1lX3RjcF9vZmxkX2NvbW1pdF9ycXMsCj4gKwkuY29tcGxldGUJPSBudm1lX2Nv bXBsZXRlX3JxLAo+ICAgCS5pbml0X3JlcXVlc3QJPSBudm1lX3RjcF9vZmxkX2luaXRfcmVxdWVz dCwKPiAtCS8qCj4gLQkgKiBBbGwgYWRkaXRpb25hbCBvcHMgd2lsbCBiZSBhbHNvIGltcGxlbWVu dGVkIGFuZCByZWdpc3RlcmVkIHNpbWlsYXIgdG8KPiAtCSAqIHRjcC5jCj4gLQkgKi8KPiArCS5l eGl0X3JlcXVlc3QJPSBudm1lX3RjcF9vZmxkX2V4aXRfcmVxdWVzdCwKPiArCS5pbml0X2hjdHgJ PSBudm1lX3RjcF9vZmxkX2luaXRfaGN0eCwKPiArCS5tYXBfcXVldWVzCT0gbnZtZV90Y3Bfb2Zs ZF9tYXBfcXVldWVzLAo+ICsJLnBvbGwJCT0gbnZtZV90Y3Bfb2ZsZF9wb2xsLAo+ICAgfTsKPiAg IAo+ICAgc3RhdGljIHN0cnVjdCBibGtfbXFfb3BzIG52bWVfdGNwX29mbGRfYWRtaW5fbXFfb3Bz ID0gewo+ICAgCS5xdWV1ZV9ycQk9IG52bWVfdGNwX29mbGRfcXVldWVfcnEsCj4gKwkuY29tcGxl dGUJPSBudm1lX2NvbXBsZXRlX3JxLAo+ICAgCS5pbml0X3JlcXVlc3QJPSBudm1lX3RjcF9vZmxk X2luaXRfcmVxdWVzdCwKPiAtCS8qCj4gLQkgKiBBbGwgYWRkaXRpb25hbCBvcHMgd2lsbCBiZSBh bHNvIGltcGxlbWVudGVkIGFuZCByZWdpc3RlcmVkIHNpbWlsYXIgdG8KPiAtCSAqIHRjcC5jCj4g LQkgKi8KPiArCS5leGl0X3JlcXVlc3QJPSBudm1lX3RjcF9vZmxkX2V4aXRfcmVxdWVzdCwKPiAr CS5pbml0X2hjdHgJPSBudm1lX3RjcF9vZmxkX2luaXRfYWRtaW5faGN0eCwKPiAgIH07Cj4gICAK PiAgIHN0YXRpYyBjb25zdCBzdHJ1Y3QgbnZtZV9jdHJsX29wcyBudm1lX3RjcF9vZmxkX2N0cmxf b3BzID0gewo+IGRpZmYgLS1naXQgYS9kcml2ZXJzL252bWUvaG9zdC90Y3Atb2ZmbG9hZC5oIGIv ZHJpdmVycy9udm1lL2hvc3QvdGNwLW9mZmxvYWQuaAo+IGluZGV4IGIyM2IxZDdlYTZmYS4uZDgy NjQ1ZmNmOWRhIDEwMDY0NAo+IC0tLSBhL2RyaXZlcnMvbnZtZS9ob3N0L3RjcC1vZmZsb2FkLmgK PiArKysgYi9kcml2ZXJzL252bWUvaG9zdC90Y3Atb2ZmbG9hZC5oCj4gQEAgLTEwNSw3ICsxMDUs NiBAQCBzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9jdHJsIHsKPiAgIAkgKiBFYWNoIGVudHJ5IGluIHRo ZSBhcnJheSBpbmRpY2F0ZXMgdGhlIG51bWJlciBvZiBxdWV1ZXMgb2YKPiAgIAkgKiBjb3JyZXNw b25kaW5nIHR5cGUuCj4gICAJICovCj4gLQl1MzIgcXVldWVfdHlwZV9tYXBwaW5nW0hDVFhfTUFY X1RZUEVTXTsKPiAgIAl1MzIgaW9fcXVldWVzW0hDVFhfTUFYX1RZUEVTXTsKPiAgIAo+ICAgCS8q IENvbm5lY3Rpdml0eSBwYXJhbXMgKi8KPiBAQCAtMjA1LDMgKzIwNCw0IEBAIHN0cnVjdCBudm1l X3RjcF9vZmxkX29wcyB7Cj4gICBpbnQgbnZtZV90Y3Bfb2ZsZF9yZWdpc3Rlcl9kZXYoc3RydWN0 IG52bWVfdGNwX29mbGRfZGV2ICpkZXYpOwo+ICAgdm9pZCBudm1lX3RjcF9vZmxkX3VucmVnaXN0 ZXJfZGV2KHN0cnVjdCBudm1lX3RjcF9vZmxkX2RldiAqZGV2KTsKPiAgIHZvaWQgbnZtZV90Y3Bf b2ZsZF9lcnJvcl9yZWNvdmVyeShzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCk7Cj4gK2lubGluZSBz aXplX3QgbnZtZV90Y3Bfb2ZsZF9pbmxpbmVfZGF0YV9zaXplKHN0cnVjdCBudm1lX3RjcF9vZmxk X3F1ZXVlICpxdWV1ZSk7Cj4gCkNoZWVycywKCkhhbm5lcwotLSAKRHIuIEhhbm5lcyBSZWluZWNr ZSAgICAgICAgICAgICAgICBLZXJuZWwgU3RvcmFnZSBBcmNoaXRlY3QKaGFyZUBzdXNlLmRlICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgKzQ5IDkxMSA3NDA1MyA2ODgKU1VTRSBTb2Z0d2Fy ZSBTb2x1dGlvbnMgR21iSCwgTWF4ZmVsZHN0ci4gNSwgOTA0MDkgTsO8cm5iZXJnCkhSQiAzNjgw OSAoQUcgTsO8cm5iZXJnKSwgR2VzY2jDpGZ0c2bDvGhyZXI6IEZlbGl4IEltZW5kw7ZyZmZlcgoK X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KTGludXgtbnZt ZSBtYWlsaW5nIGxpc3QKTGludXgtbnZtZUBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0 cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtbnZtZQo=