From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F10FC433ED for ; Mon, 3 May 2021 15:57:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E8B7261278 for ; Mon, 3 May 2021 15:57:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231182AbhECP6E (ORCPT ); Mon, 3 May 2021 11:58:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41134 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231149AbhECP57 (ORCPT ); Mon, 3 May 2021 11:57:59 -0400 Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 45574C06138C for ; Mon, 3 May 2021 08:57:05 -0700 (PDT) Received: by mail-ej1-x635.google.com with SMTP id b25so8632361eju.5 for ; Mon, 03 May 2021 08:57:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=XT4mW3ZMsD4PfT/IDSWTua+uPTxHjtks29L8jB2PFzY=; b=ht68pGegIyrA1djGwNVll88RoTAes15ODsJRCSU+H1QidnzYjguT3l+Hk1FgFJ1F1p 3Eqp7a8FVvFQp7hdJNsBv8gKBTyWbVv7HGXxLt2kIps/YAVp0WnyxJyYqn+kPgZx9Cly WLw3n15u8J2reJqKVqQ98N13Xb7P9MgEQph+oiEzJ0uQUVhZl7SeFvHAtzGGubJJSUV+ R5RGQ2Di4tfNbfsPi+f2MGnt4ZcdMnY3UaqrTnXkRswYg1QbpeTC45dG25e9OcGnPLmr O0v+Qr8XU92PhfLp4fJOOnW0dx9nM3PZ7eUuTvcVs0fm95kjZUti3iMzBj3Lley/38cP kxRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=XT4mW3ZMsD4PfT/IDSWTua+uPTxHjtks29L8jB2PFzY=; b=dU04AsX7QWD6elXg5rVM9pFpCs02gHon9aaPTg852D6wtTv/xAN6hkebrbhyefkOMx f95sZ6ZkTO8ptFn9qVsbPmnTSzJTxWaSmLs3J4Kz1g3JR+I3kXTLGERhFBRX41rjViyl cjAxTLYra40jn+51gUIUJCbcNGiEEX+4vp0Xh34rvy0QZK2P61lRIyJpMCf5OMPdkfdF dwBqVpz75lgA9B2AXNkebsH9Hv1ZQZeRX/ztaezSFGz4mj84QxYUKnU+cXmnRg0W2rZ2 bKIJExsnTNC9uiL4jObBYj3QCaMgbRrIg3AlwKK5lSUT8Ree3b1qRnLSJqWlZ+ZCY3he FUNA== X-Gm-Message-State: AOAM530Dg+Nhz7GxlQxyi0OR5aJtgedTDZ/lDAzVhhU98SP8O0aj9uPU pdu81tHPC57JbFE64EIWKYpjz3ZceK7z9Qe5qSM= X-Google-Smtp-Source: ABdhPJxzDYwM7NcWCAOS6DgExxH0j+/6x2+euqiIAmOCwZ6U6dY7pWfHArw9CLEGZK/QrHIpcHk+CIV3u0IncdvbR1k= X-Received: by 2002:a17:906:90b:: with SMTP id i11mr17426011ejd.168.1620057422723; Mon, 03 May 2021 08:57:02 -0700 (PDT) MIME-Version: 1.0 References: <20210429190926.5086-1-smalin@marvell.com> <20210429190926.5086-14-smalin@marvell.com> In-Reply-To: From: Shai Malin Date: Mon, 3 May 2021 18:56:50 +0300 Message-ID: Subject: Re: [RFC PATCH v4 13/27] nvme-tcp-offload: Add queue level implementation To: Hannes Reinecke , linux-nvme@lists.infradead.org Cc: Shai Malin , netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, sagi@grimberg.me, hch@lst.de, axboe@fb.com, kbusch@kernel.org, Ariel Elior , Michal Kalderon , okulkarni@marvell.com, pkushwaha@marvell.com, Dean Balandin Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 5/1/21 7:36 PM, Hannes Reinecke wrote: > On 4/29/21 9:09 PM, Shai Malin wrote: > > From: Dean Balandin > > > > In this patch we implement queue level functionality. > > The implementation is similar to the nvme-tcp module, the main > > difference being that we call the vendor specific create_queue op which > > creates the TCP connection, and NVMeTPC connection including > > icreq+icresp negotiation. > > Once create_queue returns successfully, we can move on to the fabrics > > connect. > > > > Acked-by: Igor Russkikh > > Signed-off-by: Dean Balandin > > Signed-off-by: Prabhakar Kushwaha > > Signed-off-by: Omkar Kulkarni > > Signed-off-by: Michal Kalderon > > Signed-off-by: Ariel Elior > > Signed-off-by: Shai Malin > > --- > > drivers/nvme/host/tcp-offload.c | 415 ++++++++++++++++++++++++++++++-= - > > drivers/nvme/host/tcp-offload.h | 2 +- > > 2 files changed, 390 insertions(+), 27 deletions(-) > > > > diff --git a/drivers/nvme/host/tcp-offload.c b/drivers/nvme/host/tcp-of= fload.c > > index 9082b11c133f..8ddce2257100 100644 > > --- a/drivers/nvme/host/tcp-offload.c > > +++ b/drivers/nvme/host/tcp-offload.c > > @@ -22,6 +22,11 @@ static inline struct nvme_tcp_ofld_ctrl *to_tcp_ofld= _ctrl(struct nvme_ctrl *nctr > > return container_of(nctrl, struct nvme_tcp_ofld_ctrl, nctrl); > > } > > > > +static inline int nvme_tcp_ofld_qid(struct nvme_tcp_ofld_queue *queue) > > +{ > > + return queue - queue->ctrl->queues; > > +} > > + > > /** > > * nvme_tcp_ofld_register_dev() - NVMeTCP Offload Library registratio= n > > * function. > > @@ -191,12 +196,94 @@ nvme_tcp_ofld_alloc_tagset(struct nvme_ctrl *nctr= l, bool admin) > > return set; > > } > > > > +static void __nvme_tcp_ofld_stop_queue(struct nvme_tcp_ofld_queue *que= ue) > > +{ > > + queue->dev->ops->drain_queue(queue); > > + queue->dev->ops->destroy_queue(queue); > > +} > > + > > +static void nvme_tcp_ofld_stop_queue(struct nvme_ctrl *nctrl, int qid) > > +{ > > + struct nvme_tcp_ofld_ctrl *ctrl =3D to_tcp_ofld_ctrl(nctrl); > > + struct nvme_tcp_ofld_queue *queue =3D &ctrl->queues[qid]; > > + > > + if (!test_and_clear_bit(NVME_TCP_OFLD_Q_LIVE, &queue->flags)) > > + return; > > + > > + __nvme_tcp_ofld_stop_queue(queue); > > +} > > + > > +static void nvme_tcp_ofld_stop_io_queues(struct nvme_ctrl *ctrl) > > +{ > > + int i; > > + > > + for (i =3D 1; i < ctrl->queue_count; i++) > > + nvme_tcp_ofld_stop_queue(ctrl, i); > > +} > > + > > +static void nvme_tcp_ofld_free_queue(struct nvme_ctrl *nctrl, int qid) > > +{ > > + struct nvme_tcp_ofld_ctrl *ctrl =3D to_tcp_ofld_ctrl(nctrl); > > + struct nvme_tcp_ofld_queue *queue =3D &ctrl->queues[qid]; > > + > > + if (!test_and_clear_bit(NVME_TCP_OFLD_Q_ALLOCATED, &queue->flags)= ) > > + return; > > + > > + queue =3D &ctrl->queues[qid]; > > + queue->ctrl =3D NULL; > > + queue->dev =3D NULL; > > + queue->report_err =3D NULL; > > +} > > + > > +static void nvme_tcp_ofld_destroy_admin_queue(struct nvme_ctrl *nctrl,= bool remove) > > +{ > > + nvme_tcp_ofld_stop_queue(nctrl, 0); > > + if (remove) { > > + blk_cleanup_queue(nctrl->admin_q); > > + blk_cleanup_queue(nctrl->fabrics_q); > > + blk_mq_free_tag_set(nctrl->admin_tagset); > > + } > > +} > > + > > +static int nvme_tcp_ofld_start_queue(struct nvme_ctrl *nctrl, int qid) > > +{ > > + struct nvme_tcp_ofld_ctrl *ctrl =3D to_tcp_ofld_ctrl(nctrl); > > + struct nvme_tcp_ofld_queue *queue =3D &ctrl->queues[qid]; > > + int rc; > > + > > + queue =3D &ctrl->queues[qid]; > > + if (qid) { > > + queue->cmnd_capsule_len =3D nctrl->ioccsz * 16; > > + rc =3D nvmf_connect_io_queue(nctrl, qid, false); > > + } else { > > + queue->cmnd_capsule_len =3D sizeof(struct nvme_command) += NVME_TCP_ADMIN_CCSZ; > > + rc =3D nvmf_connect_admin_queue(nctrl); > > + } > > + > > + if (!rc) { > > + set_bit(NVME_TCP_OFLD_Q_LIVE, &queue->flags); > > + } else { > > + if (test_bit(NVME_TCP_OFLD_Q_ALLOCATED, &queue->flags)) > > + __nvme_tcp_ofld_stop_queue(queue); > > + dev_err(nctrl->device, > > + "failed to connect queue: %d ret=3D%d\n", qid, rc= ); > > + } > > + > > + return rc; > > +} > > + > > static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctr= l, > > bool new) > > { > > + struct nvme_tcp_ofld_ctrl *ctrl =3D to_tcp_ofld_ctrl(nctrl); > > + struct nvme_tcp_ofld_queue *queue =3D &ctrl->queues[0]; > > int rc; > > > > - /* Placeholder - alloc_admin_queue */ > > + rc =3D ctrl->dev->ops->create_queue(queue, 0, NVME_AQ_DEPTH); > > + if (rc) > > + return rc; > > + > > + set_bit(NVME_TCP_OFLD_Q_ALLOCATED, &queue->flags); > > if (new) { > > nctrl->admin_tagset =3D > > nvme_tcp_ofld_alloc_tagset(nctrl, true); > > @@ -221,7 +308,9 @@ static int nvme_tcp_ofld_configure_admin_queue(stru= ct nvme_ctrl *nctrl, > > } > > } > > > > - /* Placeholder - nvme_tcp_ofld_start_queue */ > > + rc =3D nvme_tcp_ofld_start_queue(nctrl, 0); > > + if (rc) > > + goto out_cleanup_queue; > > > > rc =3D nvme_enable_ctrl(nctrl); > > if (rc) > > @@ -238,11 +327,12 @@ static int nvme_tcp_ofld_configure_admin_queue(st= ruct nvme_ctrl *nctrl, > > out_quiesce_queue: > > blk_mq_quiesce_queue(nctrl->admin_q); > > blk_sync_queue(nctrl->admin_q); > > - > > out_stop_queue: > > - /* Placeholder - stop offload queue */ > > + nvme_tcp_ofld_stop_queue(nctrl, 0); > > nvme_cancel_admin_tagset(nctrl); > > - > > +out_cleanup_queue: > > + if (new) > > + blk_cleanup_queue(nctrl->admin_q); > > out_cleanup_fabrics_q: > > if (new) > > blk_cleanup_queue(nctrl->fabrics_q); > > @@ -250,7 +340,127 @@ static int nvme_tcp_ofld_configure_admin_queue(st= ruct nvme_ctrl *nctrl, > > if (new) > > blk_mq_free_tag_set(nctrl->admin_tagset); > > out_free_queue: > > - /* Placeholder - free admin queue */ > > + nvme_tcp_ofld_free_queue(nctrl, 0); > > + > > + return rc; > > +} > > + > > +static unsigned int nvme_tcp_ofld_nr_io_queues(struct nvme_ctrl *nctrl= ) > > +{ > > + unsigned int nr_io_queues; > > + > > + nr_io_queues =3D min(nctrl->opts->nr_io_queues, num_online_cpus()= ); > > + nr_io_queues +=3D min(nctrl->opts->nr_write_queues, num_online_cp= us()); > > + nr_io_queues +=3D min(nctrl->opts->nr_poll_queues, num_online_cpu= s()); > > + > > + return nr_io_queues; > > +} > > + > > Really? Isn't this hardware-dependent? > I would have expected the hardware to impose some limitations here (# of > MSIx interrupts or something). Hmm? Right! We will be added to nvme_tcp_ofld_ops. > > > +static void > > +nvme_tcp_ofld_set_io_queues(struct nvme_ctrl *nctrl, unsigned int nr_i= o_queues) > > +{ > > + struct nvme_tcp_ofld_ctrl *ctrl =3D to_tcp_ofld_ctrl(nctrl); > > + struct nvmf_ctrl_options *opts =3D nctrl->opts; > > + > > + if (opts->nr_write_queues && opts->nr_io_queues < nr_io_queues) { > > + /* > > + * separate read/write queues > > + * hand out dedicated default queues only after we have > > + * sufficient read queues. > > + */ > > + ctrl->io_queues[HCTX_TYPE_READ] =3D opts->nr_io_queues; > > + nr_io_queues -=3D ctrl->io_queues[HCTX_TYPE_READ]; > > + ctrl->io_queues[HCTX_TYPE_DEFAULT] =3D > > + min(opts->nr_write_queues, nr_io_queues); > > + nr_io_queues -=3D ctrl->io_queues[HCTX_TYPE_DEFAULT]; > > + } else { > > + /* > > + * shared read/write queues > > + * either no write queues were requested, or we don't hav= e > > + * sufficient queue count to have dedicated default queue= s. > > + */ > > + ctrl->io_queues[HCTX_TYPE_DEFAULT] =3D > > + min(opts->nr_io_queues, nr_io_queues); > > + nr_io_queues -=3D ctrl->io_queues[HCTX_TYPE_DEFAULT]; > > + } > > + > > + if (opts->nr_poll_queues && nr_io_queues) { > > + /* map dedicated poll queues only if we have queues left = */ > > + ctrl->io_queues[HCTX_TYPE_POLL] =3D > > + min(opts->nr_poll_queues, nr_io_queues); > > + } > > +} > > + > > Same here. > Poll queues only ever make sense of the hardware can serve specific > queue pairs without interrupts. Which again relates to the number of > interrupts, and the affinity of those. > Or isn't this a concern with your card? Right! We will be added to nvme_tcp_ofld_ops. Our NVMeTCP offload HW supports 256 interrupt lines across the offload devi= ces, meaning 64-256 per offload device, (depends on the number of ports). > > > +static void > > +nvme_tcp_ofld_terminate_io_queues(struct nvme_ctrl *nctrl, int start_f= rom) > > +{ > > + int i; > > + > > + /* admin-q will be ignored because of the loop condition */ > > + for (i =3D start_from; i >=3D 1; i--) > > + nvme_tcp_ofld_stop_queue(nctrl, i); > > +} > > + > > Loop condition? Care to elaborate? Similar code (with the same loop condition) exists in the other transports, e.g. __nvme_tcp_alloc_io_queues(), which calls nvme_tcp_free_queues(). Will rephrase comment to: "Loop condition will stop before index 0 which is the admin queue." > > > +static int nvme_tcp_ofld_create_io_queues(struct nvme_ctrl *nctrl) > > +{ > > + struct nvme_tcp_ofld_ctrl *ctrl =3D to_tcp_ofld_ctrl(nctrl); > > + int i, rc; > > + > > + for (i =3D 1; i < nctrl->queue_count; i++) { > > + rc =3D ctrl->dev->ops->create_queue(&ctrl->queues[i], > > + i, nctrl->sqsize + 1); > > + if (rc) > > + goto out_free_queues; > > + > > + set_bit(NVME_TCP_OFLD_Q_ALLOCATED, &ctrl->queues[i].flags= ); > > + } > > + > > + return 0; > > + > > +out_free_queues: > > + nvme_tcp_ofld_terminate_io_queues(nctrl, --i); > > + > > + return rc; > > +} > > + > > +static int nvme_tcp_ofld_alloc_io_queues(struct nvme_ctrl *nctrl) > > +{ > > + unsigned int nr_io_queues; > > + int rc; > > + > > + nr_io_queues =3D nvme_tcp_ofld_nr_io_queues(nctrl); > > + rc =3D nvme_set_queue_count(nctrl, &nr_io_queues); > > + if (rc) > > + return rc; > > + > > + nctrl->queue_count =3D nr_io_queues + 1; > > + if (nctrl->queue_count < 2) { > > + dev_err(nctrl->device, > > + "unable to set any I/O queues\n"); > > + > > + return -ENOMEM; > > + } > > + > > + dev_info(nctrl->device, "creating %d I/O queues.\n", nr_io_queues= ); > > + nvme_tcp_ofld_set_io_queues(nctrl, nr_io_queues); > > + > > + return nvme_tcp_ofld_create_io_queues(nctrl); > > +} > > + > > +static int nvme_tcp_ofld_start_io_queues(struct nvme_ctrl *nctrl) > > +{ > > + int i, rc =3D 0; > > + > > + for (i =3D 1; i < nctrl->queue_count; i++) { > > + rc =3D nvme_tcp_ofld_start_queue(nctrl, i); > > + if (rc) > > + goto terminate_queues; > > + } > > + > > + return 0; > > + > > +terminate_queues: > > + nvme_tcp_ofld_terminate_io_queues(nctrl, --i); > > > > return rc; > > } > > @@ -258,9 +468,10 @@ static int nvme_tcp_ofld_configure_admin_queue(str= uct nvme_ctrl *nctrl, > > static int > > nvme_tcp_ofld_configure_io_queues(struct nvme_ctrl *nctrl, bool new) > > { > > - int rc; > > + int rc =3D nvme_tcp_ofld_alloc_io_queues(nctrl); > > > > - /* Placeholder - alloc_io_queues */ > > + if (rc) > > + return rc; > > > > if (new) { > > nctrl->tagset =3D nvme_tcp_ofld_alloc_tagset(nctrl, false= ); > > @@ -278,7 +489,9 @@ nvme_tcp_ofld_configure_io_queues(struct nvme_ctrl = *nctrl, bool new) > > } > > } > > > > - /* Placeholder - start_io_queues */ > > + rc =3D nvme_tcp_ofld_start_io_queues(nctrl); > > + if (rc) > > + goto out_cleanup_connect_q; > > > > if (!new) { > > nvme_start_queues(nctrl); > > @@ -300,16 +513,16 @@ nvme_tcp_ofld_configure_io_queues(struct nvme_ctr= l *nctrl, bool new) > > out_wait_freeze_timed_out: > > nvme_stop_queues(nctrl); > > nvme_sync_io_queues(nctrl); > > - > > - /* Placeholder - Stop IO queues */ > > - > > + nvme_tcp_ofld_stop_io_queues(nctrl); > > +out_cleanup_connect_q: > > + nvme_cancel_tagset(nctrl); > > if (new) > > blk_cleanup_queue(nctrl->connect_q); > > out_free_tag_set: > > if (new) > > blk_mq_free_tag_set(nctrl->tagset); > > out_free_io_queues: > > - /* Placeholder - free_io_queues */ > > + nvme_tcp_ofld_terminate_io_queues(nctrl, nctrl->queue_count); > > > > return rc; > > } > > @@ -336,6 +549,26 @@ static void nvme_tcp_ofld_reconnect_or_remove(stru= ct nvme_ctrl *nctrl) > > } > > } > > > > +static int > > +nvme_tcp_ofld_init_admin_hctx(struct blk_mq_hw_ctx *hctx, void *data, > > + unsigned int hctx_idx) > > +{ > > + struct nvme_tcp_ofld_ctrl *ctrl =3D data; > > + > > + hctx->driver_data =3D &ctrl->queues[0]; > > + > > + return 0; > > +} > > + > > +static void nvme_tcp_ofld_destroy_io_queues(struct nvme_ctrl *nctrl, b= ool remove) > > +{ > > + nvme_tcp_ofld_stop_io_queues(nctrl); > > + if (remove) { > > + blk_cleanup_queue(nctrl->connect_q); > > + blk_mq_free_tag_set(nctrl->tagset); > > + } > > +} > > + > > static int nvme_tcp_ofld_setup_ctrl(struct nvme_ctrl *nctrl, bool new= ) > > { > > struct nvmf_ctrl_options *opts =3D nctrl->opts; > > @@ -387,9 +620,19 @@ static int nvme_tcp_ofld_setup_ctrl(struct nvme_ct= rl *nctrl, bool new) > > return 0; > > > > destroy_io: > > - /* Placeholder - stop and destroy io queues*/ > > + if (nctrl->queue_count > 1) { > > + nvme_stop_queues(nctrl); > > + nvme_sync_io_queues(nctrl); > > + nvme_tcp_ofld_stop_io_queues(nctrl); > > + nvme_cancel_tagset(nctrl); > > + nvme_tcp_ofld_destroy_io_queues(nctrl, new); > > + } > > destroy_admin: > > - /* Placeholder - stop and destroy admin queue*/ > > + blk_mq_quiesce_queue(nctrl->admin_q); > > + blk_sync_queue(nctrl->admin_q); > > + nvme_tcp_ofld_stop_queue(nctrl, 0); > > + nvme_cancel_admin_tagset(nctrl); > > + nvme_tcp_ofld_destroy_admin_queue(nctrl, new); > > > > return rc; > > } > > @@ -410,6 +653,18 @@ nvme_tcp_ofld_check_dev_opts(struct nvmf_ctrl_opti= ons *opts, > > return 0; > > } > > > > +static void nvme_tcp_ofld_free_ctrl_queues(struct nvme_ctrl *nctrl) > > +{ > > + struct nvme_tcp_ofld_ctrl *ctrl =3D to_tcp_ofld_ctrl(nctrl); > > + int i; > > + > > + for (i =3D 0; i < nctrl->queue_count; ++i) > > + nvme_tcp_ofld_free_queue(nctrl, i); > > + > > + kfree(ctrl->queues); > > + ctrl->queues =3D NULL; > > +} > > + > > static void nvme_tcp_ofld_free_ctrl(struct nvme_ctrl *nctrl) > > { > > struct nvme_tcp_ofld_ctrl *ctrl =3D to_tcp_ofld_ctrl(nctrl); > > @@ -419,6 +674,7 @@ static void nvme_tcp_ofld_free_ctrl(struct nvme_ctr= l *nctrl) > > goto free_ctrl; > > > > down_write(&nvme_tcp_ofld_ctrl_rwsem); > > + nvme_tcp_ofld_free_ctrl_queues(nctrl); > > ctrl->dev->ops->release_ctrl(ctrl); > > list_del(&ctrl->list); > > up_write(&nvme_tcp_ofld_ctrl_rwsem); > > @@ -436,15 +692,37 @@ static void nvme_tcp_ofld_submit_async_event(stru= ct nvme_ctrl *arg) > > } > > > > static void > > -nvme_tcp_ofld_teardown_admin_queue(struct nvme_ctrl *ctrl, bool remove= ) > > +nvme_tcp_ofld_teardown_admin_queue(struct nvme_ctrl *nctrl, bool remov= e) > > { > > - /* Placeholder - teardown_admin_queue */ > > + blk_mq_quiesce_queue(nctrl->admin_q); > > + blk_sync_queue(nctrl->admin_q); > > + > > + nvme_tcp_ofld_stop_queue(nctrl, 0); > > + nvme_cancel_admin_tagset(nctrl); > > + > > + if (remove) > > + blk_mq_unquiesce_queue(nctrl->admin_q); > > + > > + nvme_tcp_ofld_destroy_admin_queue(nctrl, remove); > > } > > > > static void > > nvme_tcp_ofld_teardown_io_queues(struct nvme_ctrl *nctrl, bool remove= ) > > { > > - /* Placeholder - teardown_io_queues */ > > + if (nctrl->queue_count <=3D 1) > > + return; > > + > > + blk_mq_quiesce_queue(nctrl->admin_q); > > + nvme_start_freeze(nctrl); > > + nvme_stop_queues(nctrl); > > + nvme_sync_io_queues(nctrl); > > + nvme_tcp_ofld_stop_io_queues(nctrl); > > + nvme_cancel_tagset(nctrl); > > + > > + if (remove) > > + nvme_start_queues(nctrl); > > + > > + nvme_tcp_ofld_destroy_io_queues(nctrl, remove); > > } > > > > static void nvme_tcp_ofld_reconnect_ctrl_work(struct work_struct *wor= k) > > @@ -572,6 +850,17 @@ nvme_tcp_ofld_init_request(struct blk_mq_tag_set *= set, > > return 0; > > } > > > > +inline size_t nvme_tcp_ofld_inline_data_size(struct nvme_tcp_ofld_queu= e *queue) > > +{ > > + return queue->cmnd_capsule_len - sizeof(struct nvme_command); > > +} > > +EXPORT_SYMBOL_GPL(nvme_tcp_ofld_inline_data_size); > > + > > +static void nvme_tcp_ofld_commit_rqs(struct blk_mq_hw_ctx *hctx) > > +{ > > + /* Call ops->commit_rqs */ > > +} > > + > > static blk_status_t > > nvme_tcp_ofld_queue_rq(struct blk_mq_hw_ctx *hctx, > > const struct blk_mq_queue_data *bd) > > @@ -583,22 +872,96 @@ nvme_tcp_ofld_queue_rq(struct blk_mq_hw_ctx *hctx= , > > return BLK_STS_OK; > > } > > > > +static void > > +nvme_tcp_ofld_exit_request(struct blk_mq_tag_set *set, > > + struct request *rq, unsigned int hctx_idx) > > +{ > > + /* > > + * Nothing is allocated in nvme_tcp_ofld_init_request, > > + * hence empty. > > + */ > > +} > > + > > +static int > > +nvme_tcp_ofld_init_hctx(struct blk_mq_hw_ctx *hctx, void *data, > > + unsigned int hctx_idx) > > +{ > > + struct nvme_tcp_ofld_ctrl *ctrl =3D data; > > + > > + hctx->driver_data =3D &ctrl->queues[hctx_idx + 1]; > > + > > + return 0; > > +} > > + > > +static int nvme_tcp_ofld_map_queues(struct blk_mq_tag_set *set) > > +{ > > + struct nvme_tcp_ofld_ctrl *ctrl =3D set->driver_data; > > + struct nvmf_ctrl_options *opts =3D ctrl->nctrl.opts; > > + > > + if (opts->nr_write_queues && ctrl->io_queues[HCTX_TYPE_READ]) { > > + /* separate read/write queues */ > > + set->map[HCTX_TYPE_DEFAULT].nr_queues =3D > > + ctrl->io_queues[HCTX_TYPE_DEFAULT]; > > + set->map[HCTX_TYPE_DEFAULT].queue_offset =3D 0; > > + set->map[HCTX_TYPE_READ].nr_queues =3D > > + ctrl->io_queues[HCTX_TYPE_READ]; > > + set->map[HCTX_TYPE_READ].queue_offset =3D > > + ctrl->io_queues[HCTX_TYPE_DEFAULT]; > > + } else { > > + /* shared read/write queues */ > > + set->map[HCTX_TYPE_DEFAULT].nr_queues =3D > > + ctrl->io_queues[HCTX_TYPE_DEFAULT]; > > + set->map[HCTX_TYPE_DEFAULT].queue_offset =3D 0; > > + set->map[HCTX_TYPE_READ].nr_queues =3D > > + ctrl->io_queues[HCTX_TYPE_DEFAULT]; > > + set->map[HCTX_TYPE_READ].queue_offset =3D 0; > > + } > > + blk_mq_map_queues(&set->map[HCTX_TYPE_DEFAULT]); > > + blk_mq_map_queues(&set->map[HCTX_TYPE_READ]); > > + > > + if (opts->nr_poll_queues && ctrl->io_queues[HCTX_TYPE_POLL]) { > > + /* map dedicated poll queues only if we have queues left = */ > > + set->map[HCTX_TYPE_POLL].nr_queues =3D > > + ctrl->io_queues[HCTX_TYPE_POLL]; > > + set->map[HCTX_TYPE_POLL].queue_offset =3D > > + ctrl->io_queues[HCTX_TYPE_DEFAULT] + > > + ctrl->io_queues[HCTX_TYPE_READ]; > > + blk_mq_map_queues(&set->map[HCTX_TYPE_POLL]); > > + } > > + > > + dev_info(ctrl->nctrl.device, > > + "mapped %d/%d/%d default/read/poll queues.\n", > > + ctrl->io_queues[HCTX_TYPE_DEFAULT], > > + ctrl->io_queues[HCTX_TYPE_READ], > > + ctrl->io_queues[HCTX_TYPE_POLL]); > > + > > + return 0; > > +} > > + > > +static int nvme_tcp_ofld_poll(struct blk_mq_hw_ctx *hctx) > > +{ > > + /* Placeholder - Implement polling mechanism */ > > + > > + return 0; > > +} > > + > > static struct blk_mq_ops nvme_tcp_ofld_mq_ops =3D { > > .queue_rq =3D nvme_tcp_ofld_queue_rq, > > + .commit_rqs =3D nvme_tcp_ofld_commit_rqs, > > + .complete =3D nvme_complete_rq, > > .init_request =3D nvme_tcp_ofld_init_request, > > - /* > > - * All additional ops will be also implemented and registered sim= ilar to > > - * tcp.c > > - */ > > + .exit_request =3D nvme_tcp_ofld_exit_request, > > + .init_hctx =3D nvme_tcp_ofld_init_hctx, > > + .map_queues =3D nvme_tcp_ofld_map_queues, > > + .poll =3D nvme_tcp_ofld_poll, > > }; > > > > static struct blk_mq_ops nvme_tcp_ofld_admin_mq_ops =3D { > > .queue_rq =3D nvme_tcp_ofld_queue_rq, > > + .complete =3D nvme_complete_rq, > > .init_request =3D nvme_tcp_ofld_init_request, > > - /* > > - * All additional ops will be also implemented and registered sim= ilar to > > - * tcp.c > > - */ > > + .exit_request =3D nvme_tcp_ofld_exit_request, > > + .init_hctx =3D nvme_tcp_ofld_init_admin_hctx, > > }; > > > > static const struct nvme_ctrl_ops nvme_tcp_ofld_ctrl_ops =3D { > > diff --git a/drivers/nvme/host/tcp-offload.h b/drivers/nvme/host/tcp-of= fload.h > > index b23b1d7ea6fa..d82645fcf9da 100644 > > --- a/drivers/nvme/host/tcp-offload.h > > +++ b/drivers/nvme/host/tcp-offload.h > > @@ -105,7 +105,6 @@ struct nvme_tcp_ofld_ctrl { > > * Each entry in the array indicates the number of queues of > > * corresponding type. > > */ > > - u32 queue_type_mapping[HCTX_MAX_TYPES]; > > u32 io_queues[HCTX_MAX_TYPES]; > > > > /* Connectivity params */ > > @@ -205,3 +204,4 @@ struct nvme_tcp_ofld_ops { > > int nvme_tcp_ofld_register_dev(struct nvme_tcp_ofld_dev *dev); > > void nvme_tcp_ofld_unregister_dev(struct nvme_tcp_ofld_dev *dev); > > void nvme_tcp_ofld_error_recovery(struct nvme_ctrl *nctrl); > > +inline size_t nvme_tcp_ofld_inline_data_size(struct nvme_tcp_ofld_queu= e *queue); > > > Cheers, > > Hannes > -- > Dr. Hannes Reinecke Kernel Storage Architect > hare@suse.de +49 911 74053 688 > SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N=C3=BCrnberg > HRB 36809 (AG N=C3=BCrnberg), Gesch=C3=A4ftsf=C3=BChrer: Felix Imend=C3= =B6rffer From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB644C433B4 for ; Mon, 3 May 2021 15:57:41 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 165F8610C8 for ; Mon, 3 May 2021 15:57:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 165F8610C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Q3b3x3eBHemJNQqEJM9HYSVM83FMe5fv+tcGHkT3vR4=; b=ZUJZoJGGkmcz08wJIzgA+kIB+ t7UW04s6B7DWOHgwRcOOGtHr1VidKS2/SItbETENmoyVmZ/tlGALVrL/Llq26pNoyd/oVC38jH3Fz 0DwV49d7gpdLV5Jox2uOIIJhtECtKDwOx5PMQeHJIY4Sf53gI4Dfkzx2X039czjL+kS6QECAPgT2Y aYWFZxOGKi95qGLKKQ1nv63OnVd2QiO4r0g45nuzcA4h/de49ZuJu6VpmTOMRCraY0hZk9oHnZ0jn jiyC+i7gus5UJg2aA0jQ0aZwlJcyP+OZcSv9QPDtWL7xwPAj0r1rQThgKW56xewP3vpPNKMipzGQs 6Vrt3oL7Q==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1ldawb-00EJbK-ST; Mon, 03 May 2021 15:57:16 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ldawW-00EJaa-TB for linux-nvme@desiato.infradead.org; Mon, 03 May 2021 15:57:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: Content-Type:Cc:To:Subject:Message-ID:Date:From:In-Reply-To:References: MIME-Version:Sender:Reply-To:Content-ID:Content-Description; bh=XT4mW3ZMsD4PfT/IDSWTua+uPTxHjtks29L8jB2PFzY=; b=w3R/5MWsKBnpHlSCG04YNIU//n qg/bA/L1Jmnn40h00f3Ov7cnOKjXGRh3IDjS9NUz6ON/5+UfI6lSbalN1XL3cXE6Q9txmUO8Dc1K6 +8Ce6jFA+PzbFKOXgZqjX0/sig7I2VOc15hEGqJFjvFU57p73Ela+nFNe+pWVyEEb4AWvKxu/PfPX Tl/L7WYUlt0OBBxOT52epmG3jSV1GyQj/+Gm6vy1XL4ByB/7DjGfc93Y56vfKYBWAIqTSuXKO7prp PKyeqCvUZzlYzU59KgY+cXZudzqtvUM9Acg0Yk1RHX2kSkytq4MFuJr3GLWZOTV1Q29FcBNe6MQfk 6nf2tPkg==; Received: from mail-ej1-x62a.google.com ([2a00:1450:4864:20::62a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ldawS-003CvE-Mf for linux-nvme@lists.infradead.org; Mon, 03 May 2021 15:57:07 +0000 Received: by mail-ej1-x62a.google.com with SMTP id gx5so8598888ejb.11 for ; Mon, 03 May 2021 08:57:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=XT4mW3ZMsD4PfT/IDSWTua+uPTxHjtks29L8jB2PFzY=; b=ht68pGegIyrA1djGwNVll88RoTAes15ODsJRCSU+H1QidnzYjguT3l+Hk1FgFJ1F1p 3Eqp7a8FVvFQp7hdJNsBv8gKBTyWbVv7HGXxLt2kIps/YAVp0WnyxJyYqn+kPgZx9Cly WLw3n15u8J2reJqKVqQ98N13Xb7P9MgEQph+oiEzJ0uQUVhZl7SeFvHAtzGGubJJSUV+ R5RGQ2Di4tfNbfsPi+f2MGnt4ZcdMnY3UaqrTnXkRswYg1QbpeTC45dG25e9OcGnPLmr O0v+Qr8XU92PhfLp4fJOOnW0dx9nM3PZ7eUuTvcVs0fm95kjZUti3iMzBj3Lley/38cP kxRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=XT4mW3ZMsD4PfT/IDSWTua+uPTxHjtks29L8jB2PFzY=; b=tr7gToWYlYr2PynWbVGvg+wRO7e8fsujp53PbOpchdoKGyYiEs+xXgJFUR94B9xYFb dYQxGVE3V2+lFXpRq80X/hJrF8sSHQOTli2c8qrjuahw4DSHce8wBy7/l9hrvR4XxM/L dg4DBGHQDyZo8D5Vy8jKN4rjhZUTl9TdL4Iobw1JHdRufgOLmTsX82gOxLBaPdxBOzWF gHk8pGCyAcK5Bl/K+rcNMoRkKnJ/O4cqmaaKaPqgInRyhBkMA2Wn//jaoRIudH4meacb I2rkA8PgB9KP0cV6ybNmV+G3OWMJeGp7/4YCCN+2Vc219Qkia0GNBrD/Or/J7Gm6doFI +E4Q== X-Gm-Message-State: AOAM531hsZpwjrkm3YVJpKvj3hwpU6rinzH9aTV0RQhZ2hPzmX/BKn8R xL76JR4vgai6U2r4seBBSyuy6pye8+cbs+RmS3U= X-Google-Smtp-Source: ABdhPJxzDYwM7NcWCAOS6DgExxH0j+/6x2+euqiIAmOCwZ6U6dY7pWfHArw9CLEGZK/QrHIpcHk+CIV3u0IncdvbR1k= X-Received: by 2002:a17:906:90b:: with SMTP id i11mr17426011ejd.168.1620057422723; Mon, 03 May 2021 08:57:02 -0700 (PDT) MIME-Version: 1.0 References: <20210429190926.5086-1-smalin@marvell.com> <20210429190926.5086-14-smalin@marvell.com> In-Reply-To: From: Shai Malin Date: Mon, 3 May 2021 18:56:50 +0300 Message-ID: Subject: Re: [RFC PATCH v4 13/27] nvme-tcp-offload: Add queue level implementation To: Hannes Reinecke , linux-nvme@lists.infradead.org Cc: Shai Malin , netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, sagi@grimberg.me, hch@lst.de, axboe@fb.com, kbusch@kernel.org, Ariel Elior , Michal Kalderon , okulkarni@marvell.com, pkushwaha@marvell.com, Dean Balandin X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210503_085704_790605_68C83E71 X-CRM114-Status: GOOD ( 29.56 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org T24gNS8xLzIxIDc6MzYgUE0sIEhhbm5lcyBSZWluZWNrZSB3cm90ZToKPiBPbiA0LzI5LzIxIDk6 MDkgUE0sIFNoYWkgTWFsaW4gd3JvdGU6Cj4gPiBGcm9tOiBEZWFuIEJhbGFuZGluIDxkYmFsYW5k aW5AbWFydmVsbC5jb20+Cj4gPgo+ID4gSW4gdGhpcyBwYXRjaCB3ZSBpbXBsZW1lbnQgcXVldWUg bGV2ZWwgZnVuY3Rpb25hbGl0eS4KPiA+IFRoZSBpbXBsZW1lbnRhdGlvbiBpcyBzaW1pbGFyIHRv IHRoZSBudm1lLXRjcCBtb2R1bGUsIHRoZSBtYWluCj4gPiBkaWZmZXJlbmNlIGJlaW5nIHRoYXQg d2UgY2FsbCB0aGUgdmVuZG9yIHNwZWNpZmljIGNyZWF0ZV9xdWV1ZSBvcCB3aGljaAo+ID4gY3Jl YXRlcyB0aGUgVENQIGNvbm5lY3Rpb24sIGFuZCBOVk1lVFBDIGNvbm5lY3Rpb24gaW5jbHVkaW5n Cj4gPiBpY3JlcStpY3Jlc3AgbmVnb3RpYXRpb24uCj4gPiBPbmNlIGNyZWF0ZV9xdWV1ZSByZXR1 cm5zIHN1Y2Nlc3NmdWxseSwgd2UgY2FuIG1vdmUgb24gdG8gdGhlIGZhYnJpY3MKPiA+IGNvbm5l Y3QuCj4gPgo+ID4gQWNrZWQtYnk6IElnb3IgUnVzc2tpa2ggPGlydXNza2lraEBtYXJ2ZWxsLmNv bT4KPiA+IFNpZ25lZC1vZmYtYnk6IERlYW4gQmFsYW5kaW4gPGRiYWxhbmRpbkBtYXJ2ZWxsLmNv bT4KPiA+IFNpZ25lZC1vZmYtYnk6IFByYWJoYWthciBLdXNod2FoYSA8cGt1c2h3YWhhQG1hcnZl bGwuY29tPgo+ID4gU2lnbmVkLW9mZi1ieTogT21rYXIgS3Vsa2FybmkgPG9rdWxrYXJuaUBtYXJ2 ZWxsLmNvbT4KPiA+IFNpZ25lZC1vZmYtYnk6IE1pY2hhbCBLYWxkZXJvbiA8bWthbGRlcm9uQG1h cnZlbGwuY29tPgo+ID4gU2lnbmVkLW9mZi1ieTogQXJpZWwgRWxpb3IgPGFlbGlvckBtYXJ2ZWxs LmNvbT4KPiA+IFNpZ25lZC1vZmYtYnk6IFNoYWkgTWFsaW4gPHNtYWxpbkBtYXJ2ZWxsLmNvbT4K PiA+IC0tLQo+ID4gICBkcml2ZXJzL252bWUvaG9zdC90Y3Atb2ZmbG9hZC5jIHwgNDE1ICsrKysr KysrKysrKysrKysrKysrKysrKysrKysrKy0tCj4gPiAgIGRyaXZlcnMvbnZtZS9ob3N0L3RjcC1v ZmZsb2FkLmggfCAgIDIgKy0KPiA+ICAgMiBmaWxlcyBjaGFuZ2VkLCAzOTAgaW5zZXJ0aW9ucygr KSwgMjcgZGVsZXRpb25zKC0pCj4gPgo+ID4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvbnZtZS9ob3N0 L3RjcC1vZmZsb2FkLmMgYi9kcml2ZXJzL252bWUvaG9zdC90Y3Atb2ZmbG9hZC5jCj4gPiBpbmRl eCA5MDgyYjExYzEzM2YuLjhkZGNlMjI1NzEwMCAxMDA2NDQKPiA+IC0tLSBhL2RyaXZlcnMvbnZt ZS9ob3N0L3RjcC1vZmZsb2FkLmMKPiA+ICsrKyBiL2RyaXZlcnMvbnZtZS9ob3N0L3RjcC1vZmZs b2FkLmMKPiA+IEBAIC0yMiw2ICsyMiwxMSBAQCBzdGF0aWMgaW5saW5lIHN0cnVjdCBudm1lX3Rj cF9vZmxkX2N0cmwgKnRvX3RjcF9vZmxkX2N0cmwoc3RydWN0IG52bWVfY3RybCAqbmN0cgo+ID4g ICAgICAgcmV0dXJuIGNvbnRhaW5lcl9vZihuY3RybCwgc3RydWN0IG52bWVfdGNwX29mbGRfY3Ry bCwgbmN0cmwpOwo+ID4gICB9Cj4gPgo+ID4gK3N0YXRpYyBpbmxpbmUgaW50IG52bWVfdGNwX29m bGRfcWlkKHN0cnVjdCBudm1lX3RjcF9vZmxkX3F1ZXVlICpxdWV1ZSkKPiA+ICt7Cj4gPiArICAg ICByZXR1cm4gcXVldWUgLSBxdWV1ZS0+Y3RybC0+cXVldWVzOwo+ID4gK30KPiA+ICsKPiA+ICAg LyoqCj4gPiAgICAqIG52bWVfdGNwX29mbGRfcmVnaXN0ZXJfZGV2KCkgLSBOVk1lVENQIE9mZmxv YWQgTGlicmFyeSByZWdpc3RyYXRpb24KPiA+ICAgICogZnVuY3Rpb24uCj4gPiBAQCAtMTkxLDEy ICsxOTYsOTQgQEAgbnZtZV90Y3Bfb2ZsZF9hbGxvY190YWdzZXQoc3RydWN0IG52bWVfY3RybCAq bmN0cmwsIGJvb2wgYWRtaW4pCj4gPiAgICAgICByZXR1cm4gc2V0Owo+ID4gICB9Cj4gPgo+ID4g K3N0YXRpYyB2b2lkIF9fbnZtZV90Y3Bfb2ZsZF9zdG9wX3F1ZXVlKHN0cnVjdCBudm1lX3RjcF9v ZmxkX3F1ZXVlICpxdWV1ZSkKPiA+ICt7Cj4gPiArICAgICBxdWV1ZS0+ZGV2LT5vcHMtPmRyYWlu X3F1ZXVlKHF1ZXVlKTsKPiA+ICsgICAgIHF1ZXVlLT5kZXYtPm9wcy0+ZGVzdHJveV9xdWV1ZShx dWV1ZSk7Cj4gPiArfQo+ID4gKwo+ID4gK3N0YXRpYyB2b2lkIG52bWVfdGNwX29mbGRfc3RvcF9x dWV1ZShzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCwgaW50IHFpZCkKPiA+ICt7Cj4gPiArICAgICBz dHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9jdHJsICpjdHJsID0gdG9fdGNwX29mbGRfY3RybChuY3RybCk7 Cj4gPiArICAgICBzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9xdWV1ZSAqcXVldWUgPSAmY3RybC0+cXVl dWVzW3FpZF07Cj4gPiArCj4gPiArICAgICBpZiAoIXRlc3RfYW5kX2NsZWFyX2JpdChOVk1FX1RD UF9PRkxEX1FfTElWRSwgJnF1ZXVlLT5mbGFncykpCj4gPiArICAgICAgICAgICAgIHJldHVybjsK PiA+ICsKPiA+ICsgICAgIF9fbnZtZV90Y3Bfb2ZsZF9zdG9wX3F1ZXVlKHF1ZXVlKTsKPiA+ICt9 Cj4gPiArCj4gPiArc3RhdGljIHZvaWQgbnZtZV90Y3Bfb2ZsZF9zdG9wX2lvX3F1ZXVlcyhzdHJ1 Y3QgbnZtZV9jdHJsICpjdHJsKQo+ID4gK3sKPiA+ICsgICAgIGludCBpOwo+ID4gKwo+ID4gKyAg ICAgZm9yIChpID0gMTsgaSA8IGN0cmwtPnF1ZXVlX2NvdW50OyBpKyspCj4gPiArICAgICAgICAg ICAgIG52bWVfdGNwX29mbGRfc3RvcF9xdWV1ZShjdHJsLCBpKTsKPiA+ICt9Cj4gPiArCj4gPiAr c3RhdGljIHZvaWQgbnZtZV90Y3Bfb2ZsZF9mcmVlX3F1ZXVlKHN0cnVjdCBudm1lX2N0cmwgKm5j dHJsLCBpbnQgcWlkKQo+ID4gK3sKPiA+ICsgICAgIHN0cnVjdCBudm1lX3RjcF9vZmxkX2N0cmwg KmN0cmwgPSB0b190Y3Bfb2ZsZF9jdHJsKG5jdHJsKTsKPiA+ICsgICAgIHN0cnVjdCBudm1lX3Rj cF9vZmxkX3F1ZXVlICpxdWV1ZSA9ICZjdHJsLT5xdWV1ZXNbcWlkXTsKPiA+ICsKPiA+ICsgICAg IGlmICghdGVzdF9hbmRfY2xlYXJfYml0KE5WTUVfVENQX09GTERfUV9BTExPQ0FURUQsICZxdWV1 ZS0+ZmxhZ3MpKQo+ID4gKyAgICAgICAgICAgICByZXR1cm47Cj4gPiArCj4gPiArICAgICBxdWV1 ZSA9ICZjdHJsLT5xdWV1ZXNbcWlkXTsKPiA+ICsgICAgIHF1ZXVlLT5jdHJsID0gTlVMTDsKPiA+ ICsgICAgIHF1ZXVlLT5kZXYgPSBOVUxMOwo+ID4gKyAgICAgcXVldWUtPnJlcG9ydF9lcnIgPSBO VUxMOwo+ID4gK30KPiA+ICsKPiA+ICtzdGF0aWMgdm9pZCBudm1lX3RjcF9vZmxkX2Rlc3Ryb3lf YWRtaW5fcXVldWUoc3RydWN0IG52bWVfY3RybCAqbmN0cmwsIGJvb2wgcmVtb3ZlKQo+ID4gK3sK PiA+ICsgICAgIG52bWVfdGNwX29mbGRfc3RvcF9xdWV1ZShuY3RybCwgMCk7Cj4gPiArICAgICBp ZiAocmVtb3ZlKSB7Cj4gPiArICAgICAgICAgICAgIGJsa19jbGVhbnVwX3F1ZXVlKG5jdHJsLT5h ZG1pbl9xKTsKPiA+ICsgICAgICAgICAgICAgYmxrX2NsZWFudXBfcXVldWUobmN0cmwtPmZhYnJp Y3NfcSk7Cj4gPiArICAgICAgICAgICAgIGJsa19tcV9mcmVlX3RhZ19zZXQobmN0cmwtPmFkbWlu X3RhZ3NldCk7Cj4gPiArICAgICB9Cj4gPiArfQo+ID4gKwo+ID4gK3N0YXRpYyBpbnQgbnZtZV90 Y3Bfb2ZsZF9zdGFydF9xdWV1ZShzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCwgaW50IHFpZCkKPiA+ ICt7Cj4gPiArICAgICBzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9jdHJsICpjdHJsID0gdG9fdGNwX29m bGRfY3RybChuY3RybCk7Cj4gPiArICAgICBzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9xdWV1ZSAqcXVl dWUgPSAmY3RybC0+cXVldWVzW3FpZF07Cj4gPiArICAgICBpbnQgcmM7Cj4gPiArCj4gPiArICAg ICBxdWV1ZSA9ICZjdHJsLT5xdWV1ZXNbcWlkXTsKPiA+ICsgICAgIGlmIChxaWQpIHsKPiA+ICsg ICAgICAgICAgICAgcXVldWUtPmNtbmRfY2Fwc3VsZV9sZW4gPSBuY3RybC0+aW9jY3N6ICogMTY7 Cj4gPiArICAgICAgICAgICAgIHJjID0gbnZtZl9jb25uZWN0X2lvX3F1ZXVlKG5jdHJsLCBxaWQs IGZhbHNlKTsKPiA+ICsgICAgIH0gZWxzZSB7Cj4gPiArICAgICAgICAgICAgIHF1ZXVlLT5jbW5k X2NhcHN1bGVfbGVuID0gc2l6ZW9mKHN0cnVjdCBudm1lX2NvbW1hbmQpICsgTlZNRV9UQ1BfQURN SU5fQ0NTWjsKPiA+ICsgICAgICAgICAgICAgcmMgPSBudm1mX2Nvbm5lY3RfYWRtaW5fcXVldWUo bmN0cmwpOwo+ID4gKyAgICAgfQo+ID4gKwo+ID4gKyAgICAgaWYgKCFyYykgewo+ID4gKyAgICAg ICAgICAgICBzZXRfYml0KE5WTUVfVENQX09GTERfUV9MSVZFLCAmcXVldWUtPmZsYWdzKTsKPiA+ ICsgICAgIH0gZWxzZSB7Cj4gPiArICAgICAgICAgICAgIGlmICh0ZXN0X2JpdChOVk1FX1RDUF9P RkxEX1FfQUxMT0NBVEVELCAmcXVldWUtPmZsYWdzKSkKPiA+ICsgICAgICAgICAgICAgICAgICAg ICBfX252bWVfdGNwX29mbGRfc3RvcF9xdWV1ZShxdWV1ZSk7Cj4gPiArICAgICAgICAgICAgIGRl dl9lcnIobmN0cmwtPmRldmljZSwKPiA+ICsgICAgICAgICAgICAgICAgICAgICAiZmFpbGVkIHRv IGNvbm5lY3QgcXVldWU6ICVkIHJldD0lZFxuIiwgcWlkLCByYyk7Cj4gPiArICAgICB9Cj4gPiAr Cj4gPiArICAgICByZXR1cm4gcmM7Cj4gPiArfQo+ID4gKwo+ID4gICBzdGF0aWMgaW50IG52bWVf dGNwX29mbGRfY29uZmlndXJlX2FkbWluX3F1ZXVlKHN0cnVjdCBudm1lX2N0cmwgKm5jdHJsLAo+ ID4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYm9vbCBuZXcp Cj4gPiAgIHsKPiA+ICsgICAgIHN0cnVjdCBudm1lX3RjcF9vZmxkX2N0cmwgKmN0cmwgPSB0b190 Y3Bfb2ZsZF9jdHJsKG5jdHJsKTsKPiA+ICsgICAgIHN0cnVjdCBudm1lX3RjcF9vZmxkX3F1ZXVl ICpxdWV1ZSA9ICZjdHJsLT5xdWV1ZXNbMF07Cj4gPiAgICAgICBpbnQgcmM7Cj4gPgo+ID4gLSAg ICAgLyogUGxhY2Vob2xkZXIgLSBhbGxvY19hZG1pbl9xdWV1ZSAqLwo+ID4gKyAgICAgcmMgPSBj dHJsLT5kZXYtPm9wcy0+Y3JlYXRlX3F1ZXVlKHF1ZXVlLCAwLCBOVk1FX0FRX0RFUFRIKTsKPiA+ ICsgICAgIGlmIChyYykKPiA+ICsgICAgICAgICAgICAgcmV0dXJuIHJjOwo+ID4gKwo+ID4gKyAg ICAgc2V0X2JpdChOVk1FX1RDUF9PRkxEX1FfQUxMT0NBVEVELCAmcXVldWUtPmZsYWdzKTsKPiA+ ICAgICAgIGlmIChuZXcpIHsKPiA+ICAgICAgICAgICAgICAgbmN0cmwtPmFkbWluX3RhZ3NldCA9 Cj4gPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBudm1lX3RjcF9vZmxkX2FsbG9jX3Rh Z3NldChuY3RybCwgdHJ1ZSk7Cj4gPiBAQCAtMjIxLDcgKzMwOCw5IEBAIHN0YXRpYyBpbnQgbnZt ZV90Y3Bfb2ZsZF9jb25maWd1cmVfYWRtaW5fcXVldWUoc3RydWN0IG52bWVfY3RybCAqbmN0cmws Cj4gPiAgICAgICAgICAgICAgIH0KPiA+ICAgICAgIH0KPiA+Cj4gPiAtICAgICAvKiBQbGFjZWhv bGRlciAtIG52bWVfdGNwX29mbGRfc3RhcnRfcXVldWUgKi8KPiA+ICsgICAgIHJjID0gbnZtZV90 Y3Bfb2ZsZF9zdGFydF9xdWV1ZShuY3RybCwgMCk7Cj4gPiArICAgICBpZiAocmMpCj4gPiArICAg ICAgICAgICAgIGdvdG8gb3V0X2NsZWFudXBfcXVldWU7Cj4gPgo+ID4gICAgICAgcmMgPSBudm1l X2VuYWJsZV9jdHJsKG5jdHJsKTsKPiA+ICAgICAgIGlmIChyYykKPiA+IEBAIC0yMzgsMTEgKzMy NywxMiBAQCBzdGF0aWMgaW50IG52bWVfdGNwX29mbGRfY29uZmlndXJlX2FkbWluX3F1ZXVlKHN0 cnVjdCBudm1lX2N0cmwgKm5jdHJsLAo+ID4gICBvdXRfcXVpZXNjZV9xdWV1ZToKPiA+ICAgICAg IGJsa19tcV9xdWllc2NlX3F1ZXVlKG5jdHJsLT5hZG1pbl9xKTsKPiA+ICAgICAgIGJsa19zeW5j X3F1ZXVlKG5jdHJsLT5hZG1pbl9xKTsKPiA+IC0KPiA+ICAgb3V0X3N0b3BfcXVldWU6Cj4gPiAt ICAgICAvKiBQbGFjZWhvbGRlciAtIHN0b3Agb2ZmbG9hZCBxdWV1ZSAqLwo+ID4gKyAgICAgbnZt ZV90Y3Bfb2ZsZF9zdG9wX3F1ZXVlKG5jdHJsLCAwKTsKPiA+ICAgICAgIG52bWVfY2FuY2VsX2Fk bWluX3RhZ3NldChuY3RybCk7Cj4gPiAtCj4gPiArb3V0X2NsZWFudXBfcXVldWU6Cj4gPiArICAg ICBpZiAobmV3KQo+ID4gKyAgICAgICAgICAgICBibGtfY2xlYW51cF9xdWV1ZShuY3RybC0+YWRt aW5fcSk7Cj4gPiAgIG91dF9jbGVhbnVwX2ZhYnJpY3NfcToKPiA+ICAgICAgIGlmIChuZXcpCj4g PiAgICAgICAgICAgICAgIGJsa19jbGVhbnVwX3F1ZXVlKG5jdHJsLT5mYWJyaWNzX3EpOwo+ID4g QEAgLTI1MCw3ICszNDAsMTI3IEBAIHN0YXRpYyBpbnQgbnZtZV90Y3Bfb2ZsZF9jb25maWd1cmVf YWRtaW5fcXVldWUoc3RydWN0IG52bWVfY3RybCAqbmN0cmwsCj4gPiAgICAgICBpZiAobmV3KQo+ ID4gICAgICAgICAgICAgICBibGtfbXFfZnJlZV90YWdfc2V0KG5jdHJsLT5hZG1pbl90YWdzZXQp Owo+ID4gICBvdXRfZnJlZV9xdWV1ZToKPiA+IC0gICAgIC8qIFBsYWNlaG9sZGVyIC0gZnJlZSBh ZG1pbiBxdWV1ZSAqLwo+ID4gKyAgICAgbnZtZV90Y3Bfb2ZsZF9mcmVlX3F1ZXVlKG5jdHJsLCAw KTsKPiA+ICsKPiA+ICsgICAgIHJldHVybiByYzsKPiA+ICt9Cj4gPiArCj4gPiArc3RhdGljIHVu c2lnbmVkIGludCBudm1lX3RjcF9vZmxkX25yX2lvX3F1ZXVlcyhzdHJ1Y3QgbnZtZV9jdHJsICpu Y3RybCkKPiA+ICt7Cj4gPiArICAgICB1bnNpZ25lZCBpbnQgbnJfaW9fcXVldWVzOwo+ID4gKwo+ ID4gKyAgICAgbnJfaW9fcXVldWVzID0gbWluKG5jdHJsLT5vcHRzLT5ucl9pb19xdWV1ZXMsIG51 bV9vbmxpbmVfY3B1cygpKTsKPiA+ICsgICAgIG5yX2lvX3F1ZXVlcyArPSBtaW4obmN0cmwtPm9w dHMtPm5yX3dyaXRlX3F1ZXVlcywgbnVtX29ubGluZV9jcHVzKCkpOwo+ID4gKyAgICAgbnJfaW9f cXVldWVzICs9IG1pbihuY3RybC0+b3B0cy0+bnJfcG9sbF9xdWV1ZXMsIG51bV9vbmxpbmVfY3B1 cygpKTsKPiA+ICsKPiA+ICsgICAgIHJldHVybiBucl9pb19xdWV1ZXM7Cj4gPiArfQo+ID4gKwo+ Cj4gUmVhbGx5PyBJc24ndCB0aGlzIGhhcmR3YXJlLWRlcGVuZGVudD8KPiBJIHdvdWxkIGhhdmUg ZXhwZWN0ZWQgdGhlIGhhcmR3YXJlIHRvIGltcG9zZSBzb21lIGxpbWl0YXRpb25zIGhlcmUgKCMg b2YKPiBNU0l4IGludGVycnVwdHMgb3Igc29tZXRoaW5nKS4gSG1tPwoKUmlnaHQhICBXZSB3aWxs IGJlIGFkZGVkIHRvIG52bWVfdGNwX29mbGRfb3BzLgoKPgo+ID4gK3N0YXRpYyB2b2lkCj4gPiAr bnZtZV90Y3Bfb2ZsZF9zZXRfaW9fcXVldWVzKHN0cnVjdCBudm1lX2N0cmwgKm5jdHJsLCB1bnNp Z25lZCBpbnQgbnJfaW9fcXVldWVzKQo+ID4gK3sKPiA+ICsgICAgIHN0cnVjdCBudm1lX3RjcF9v ZmxkX2N0cmwgKmN0cmwgPSB0b190Y3Bfb2ZsZF9jdHJsKG5jdHJsKTsKPiA+ICsgICAgIHN0cnVj dCBudm1mX2N0cmxfb3B0aW9ucyAqb3B0cyA9IG5jdHJsLT5vcHRzOwo+ID4gKwo+ID4gKyAgICAg aWYgKG9wdHMtPm5yX3dyaXRlX3F1ZXVlcyAmJiBvcHRzLT5ucl9pb19xdWV1ZXMgPCBucl9pb19x dWV1ZXMpIHsKPiA+ICsgICAgICAgICAgICAgLyoKPiA+ICsgICAgICAgICAgICAgICogc2VwYXJh dGUgcmVhZC93cml0ZSBxdWV1ZXMKPiA+ICsgICAgICAgICAgICAgICogaGFuZCBvdXQgZGVkaWNh dGVkIGRlZmF1bHQgcXVldWVzIG9ubHkgYWZ0ZXIgd2UgaGF2ZQo+ID4gKyAgICAgICAgICAgICAg KiBzdWZmaWNpZW50IHJlYWQgcXVldWVzLgo+ID4gKyAgICAgICAgICAgICAgKi8KPiA+ICsgICAg ICAgICAgICAgY3RybC0+aW9fcXVldWVzW0hDVFhfVFlQRV9SRUFEXSA9IG9wdHMtPm5yX2lvX3F1 ZXVlczsKPiA+ICsgICAgICAgICAgICAgbnJfaW9fcXVldWVzIC09IGN0cmwtPmlvX3F1ZXVlc1tI Q1RYX1RZUEVfUkVBRF07Cj4gPiArICAgICAgICAgICAgIGN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZ UEVfREVGQVVMVF0gPQo+ID4gKyAgICAgICAgICAgICAgICAgICAgIG1pbihvcHRzLT5ucl93cml0 ZV9xdWV1ZXMsIG5yX2lvX3F1ZXVlcyk7Cj4gPiArICAgICAgICAgICAgIG5yX2lvX3F1ZXVlcyAt PSBjdHJsLT5pb19xdWV1ZXNbSENUWF9UWVBFX0RFRkFVTFRdOwo+ID4gKyAgICAgfSBlbHNlIHsK PiA+ICsgICAgICAgICAgICAgLyoKPiA+ICsgICAgICAgICAgICAgICogc2hhcmVkIHJlYWQvd3Jp dGUgcXVldWVzCj4gPiArICAgICAgICAgICAgICAqIGVpdGhlciBubyB3cml0ZSBxdWV1ZXMgd2Vy ZSByZXF1ZXN0ZWQsIG9yIHdlIGRvbid0IGhhdmUKPiA+ICsgICAgICAgICAgICAgICogc3VmZmlj aWVudCBxdWV1ZSBjb3VudCB0byBoYXZlIGRlZGljYXRlZCBkZWZhdWx0IHF1ZXVlcy4KPiA+ICsg ICAgICAgICAgICAgICovCj4gPiArICAgICAgICAgICAgIGN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZ UEVfREVGQVVMVF0gPQo+ID4gKyAgICAgICAgICAgICAgICAgICAgIG1pbihvcHRzLT5ucl9pb19x dWV1ZXMsIG5yX2lvX3F1ZXVlcyk7Cj4gPiArICAgICAgICAgICAgIG5yX2lvX3F1ZXVlcyAtPSBj dHJsLT5pb19xdWV1ZXNbSENUWF9UWVBFX0RFRkFVTFRdOwo+ID4gKyAgICAgfQo+ID4gKwo+ID4g KyAgICAgaWYgKG9wdHMtPm5yX3BvbGxfcXVldWVzICYmIG5yX2lvX3F1ZXVlcykgewo+ID4gKyAg ICAgICAgICAgICAvKiBtYXAgZGVkaWNhdGVkIHBvbGwgcXVldWVzIG9ubHkgaWYgd2UgaGF2ZSBx dWV1ZXMgbGVmdCAqLwo+ID4gKyAgICAgICAgICAgICBjdHJsLT5pb19xdWV1ZXNbSENUWF9UWVBF X1BPTExdID0KPiA+ICsgICAgICAgICAgICAgICAgICAgICBtaW4ob3B0cy0+bnJfcG9sbF9xdWV1 ZXMsIG5yX2lvX3F1ZXVlcyk7Cj4gPiArICAgICB9Cj4gPiArfQo+ID4gKwo+Cj4gU2FtZSBoZXJl Lgo+IFBvbGwgcXVldWVzIG9ubHkgZXZlciBtYWtlIHNlbnNlIG9mIHRoZSBoYXJkd2FyZSBjYW4g c2VydmUgc3BlY2lmaWMKPiBxdWV1ZSBwYWlycyB3aXRob3V0IGludGVycnVwdHMuIFdoaWNoIGFn YWluIHJlbGF0ZXMgdG8gdGhlIG51bWJlciBvZgo+IGludGVycnVwdHMsIGFuZCB0aGUgYWZmaW5p dHkgb2YgdGhvc2UuCj4gT3IgaXNuJ3QgdGhpcyBhIGNvbmNlcm4gd2l0aCB5b3VyIGNhcmQ/CgpS aWdodCEgIFdlIHdpbGwgYmUgYWRkZWQgdG8gbnZtZV90Y3Bfb2ZsZF9vcHMuCk91ciBOVk1lVENQ IG9mZmxvYWQgSFcgc3VwcG9ydHMgMjU2IGludGVycnVwdCBsaW5lcyBhY3Jvc3MgdGhlIG9mZmxv YWQgZGV2aWNlcywKbWVhbmluZyA2NC0yNTYgcGVyIG9mZmxvYWQgZGV2aWNlLCAoZGVwZW5kcyBv biB0aGUgbnVtYmVyIG9mIHBvcnRzKS4KCj4KPiA+ICtzdGF0aWMgdm9pZAo+ID4gK252bWVfdGNw X29mbGRfdGVybWluYXRlX2lvX3F1ZXVlcyhzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCwgaW50IHN0 YXJ0X2Zyb20pCj4gPiArewo+ID4gKyAgICAgaW50IGk7Cj4gPiArCj4gPiArICAgICAvKiBhZG1p bi1xIHdpbGwgYmUgaWdub3JlZCBiZWNhdXNlIG9mIHRoZSBsb29wIGNvbmRpdGlvbiAqLwo+ID4g KyAgICAgZm9yIChpID0gc3RhcnRfZnJvbTsgaSA+PSAxOyBpLS0pCj4gPiArICAgICAgICAgICAg IG52bWVfdGNwX29mbGRfc3RvcF9xdWV1ZShuY3RybCwgaSk7Cj4gPiArfQo+ID4gKwo+Cj4gTG9v cCBjb25kaXRpb24/IENhcmUgdG8gZWxhYm9yYXRlPwoKU2ltaWxhciBjb2RlICh3aXRoIHRoZSBz YW1lIGxvb3AgY29uZGl0aW9uKSBleGlzdHMgaW4gdGhlIG90aGVyIHRyYW5zcG9ydHMsCmUuZy4g X19udm1lX3RjcF9hbGxvY19pb19xdWV1ZXMoKSwgd2hpY2ggY2FsbHMgbnZtZV90Y3BfZnJlZV9x dWV1ZXMoKS4KV2lsbCByZXBocmFzZSBjb21tZW50IHRvOiAiTG9vcCBjb25kaXRpb24gd2lsbCBz dG9wIGJlZm9yZSBpbmRleCAwIHdoaWNoCmlzIHRoZSBhZG1pbiBxdWV1ZS4iCgo+Cj4gPiArc3Rh dGljIGludCBudm1lX3RjcF9vZmxkX2NyZWF0ZV9pb19xdWV1ZXMoc3RydWN0IG52bWVfY3RybCAq bmN0cmwpCj4gPiArewo+ID4gKyAgICAgc3RydWN0IG52bWVfdGNwX29mbGRfY3RybCAqY3RybCA9 IHRvX3RjcF9vZmxkX2N0cmwobmN0cmwpOwo+ID4gKyAgICAgaW50IGksIHJjOwo+ID4gKwo+ID4g KyAgICAgZm9yIChpID0gMTsgaSA8IG5jdHJsLT5xdWV1ZV9jb3VudDsgaSsrKSB7Cj4gPiArICAg ICAgICAgICAgIHJjID0gY3RybC0+ZGV2LT5vcHMtPmNyZWF0ZV9xdWV1ZSgmY3RybC0+cXVldWVz W2ldLAo+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg aSwgbmN0cmwtPnNxc2l6ZSArIDEpOwo+ID4gKyAgICAgICAgICAgICBpZiAocmMpCj4gPiArICAg ICAgICAgICAgICAgICAgICAgZ290byBvdXRfZnJlZV9xdWV1ZXM7Cj4gPiArCj4gPiArICAgICAg ICAgICAgIHNldF9iaXQoTlZNRV9UQ1BfT0ZMRF9RX0FMTE9DQVRFRCwgJmN0cmwtPnF1ZXVlc1tp XS5mbGFncyk7Cj4gPiArICAgICB9Cj4gPiArCj4gPiArICAgICByZXR1cm4gMDsKPiA+ICsKPiA+ ICtvdXRfZnJlZV9xdWV1ZXM6Cj4gPiArICAgICBudm1lX3RjcF9vZmxkX3Rlcm1pbmF0ZV9pb19x dWV1ZXMobmN0cmwsIC0taSk7Cj4gPiArCj4gPiArICAgICByZXR1cm4gcmM7Cj4gPiArfQo+ID4g Kwo+ID4gK3N0YXRpYyBpbnQgbnZtZV90Y3Bfb2ZsZF9hbGxvY19pb19xdWV1ZXMoc3RydWN0IG52 bWVfY3RybCAqbmN0cmwpCj4gPiArewo+ID4gKyAgICAgdW5zaWduZWQgaW50IG5yX2lvX3F1ZXVl czsKPiA+ICsgICAgIGludCByYzsKPiA+ICsKPiA+ICsgICAgIG5yX2lvX3F1ZXVlcyA9IG52bWVf dGNwX29mbGRfbnJfaW9fcXVldWVzKG5jdHJsKTsKPiA+ICsgICAgIHJjID0gbnZtZV9zZXRfcXVl dWVfY291bnQobmN0cmwsICZucl9pb19xdWV1ZXMpOwo+ID4gKyAgICAgaWYgKHJjKQo+ID4gKyAg ICAgICAgICAgICByZXR1cm4gcmM7Cj4gPiArCj4gPiArICAgICBuY3RybC0+cXVldWVfY291bnQg PSBucl9pb19xdWV1ZXMgKyAxOwo+ID4gKyAgICAgaWYgKG5jdHJsLT5xdWV1ZV9jb3VudCA8IDIp IHsKPiA+ICsgICAgICAgICAgICAgZGV2X2VycihuY3RybC0+ZGV2aWNlLAo+ID4gKyAgICAgICAg ICAgICAgICAgICAgICJ1bmFibGUgdG8gc2V0IGFueSBJL08gcXVldWVzXG4iKTsKPiA+ICsKPiA+ ICsgICAgICAgICAgICAgcmV0dXJuIC1FTk9NRU07Cj4gPiArICAgICB9Cj4gPiArCj4gPiArICAg ICBkZXZfaW5mbyhuY3RybC0+ZGV2aWNlLCAiY3JlYXRpbmcgJWQgSS9PIHF1ZXVlcy5cbiIsIG5y X2lvX3F1ZXVlcyk7Cj4gPiArICAgICBudm1lX3RjcF9vZmxkX3NldF9pb19xdWV1ZXMobmN0cmws IG5yX2lvX3F1ZXVlcyk7Cj4gPiArCj4gPiArICAgICByZXR1cm4gbnZtZV90Y3Bfb2ZsZF9jcmVh dGVfaW9fcXVldWVzKG5jdHJsKTsKPiA+ICt9Cj4gPiArCj4gPiArc3RhdGljIGludCBudm1lX3Rj cF9vZmxkX3N0YXJ0X2lvX3F1ZXVlcyhzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCkKPiA+ICt7Cj4g PiArICAgICBpbnQgaSwgcmMgPSAwOwo+ID4gKwo+ID4gKyAgICAgZm9yIChpID0gMTsgaSA8IG5j dHJsLT5xdWV1ZV9jb3VudDsgaSsrKSB7Cj4gPiArICAgICAgICAgICAgIHJjID0gbnZtZV90Y3Bf b2ZsZF9zdGFydF9xdWV1ZShuY3RybCwgaSk7Cj4gPiArICAgICAgICAgICAgIGlmIChyYykKPiA+ ICsgICAgICAgICAgICAgICAgICAgICBnb3RvIHRlcm1pbmF0ZV9xdWV1ZXM7Cj4gPiArICAgICB9 Cj4gPiArCj4gPiArICAgICByZXR1cm4gMDsKPiA+ICsKPiA+ICt0ZXJtaW5hdGVfcXVldWVzOgo+ ID4gKyAgICAgbnZtZV90Y3Bfb2ZsZF90ZXJtaW5hdGVfaW9fcXVldWVzKG5jdHJsLCAtLWkpOwo+ ID4KPiA+ICAgICAgIHJldHVybiByYzsKPiA+ICAgfQo+ID4gQEAgLTI1OCw5ICs0NjgsMTAgQEAg c3RhdGljIGludCBudm1lX3RjcF9vZmxkX2NvbmZpZ3VyZV9hZG1pbl9xdWV1ZShzdHJ1Y3QgbnZt ZV9jdHJsICpuY3RybCwKPiA+ICAgc3RhdGljIGludAo+ID4gICBudm1lX3RjcF9vZmxkX2NvbmZp Z3VyZV9pb19xdWV1ZXMoc3RydWN0IG52bWVfY3RybCAqbmN0cmwsIGJvb2wgbmV3KQo+ID4gICB7 Cj4gPiAtICAgICBpbnQgcmM7Cj4gPiArICAgICBpbnQgcmMgPSBudm1lX3RjcF9vZmxkX2FsbG9j X2lvX3F1ZXVlcyhuY3RybCk7Cj4gPgo+ID4gLSAgICAgLyogUGxhY2Vob2xkZXIgLSBhbGxvY19p b19xdWV1ZXMgKi8KPiA+ICsgICAgIGlmIChyYykKPiA+ICsgICAgICAgICAgICAgcmV0dXJuIHJj Owo+ID4KPiA+ICAgICAgIGlmIChuZXcpIHsKPiA+ICAgICAgICAgICAgICAgbmN0cmwtPnRhZ3Nl dCA9IG52bWVfdGNwX29mbGRfYWxsb2NfdGFnc2V0KG5jdHJsLCBmYWxzZSk7Cj4gPiBAQCAtMjc4 LDcgKzQ4OSw5IEBAIG52bWVfdGNwX29mbGRfY29uZmlndXJlX2lvX3F1ZXVlcyhzdHJ1Y3QgbnZt ZV9jdHJsICpuY3RybCwgYm9vbCBuZXcpCj4gPiAgICAgICAgICAgICAgIH0KPiA+ICAgICAgIH0K PiA+Cj4gPiAtICAgICAvKiBQbGFjZWhvbGRlciAtIHN0YXJ0X2lvX3F1ZXVlcyAqLwo+ID4gKyAg ICAgcmMgPSBudm1lX3RjcF9vZmxkX3N0YXJ0X2lvX3F1ZXVlcyhuY3RybCk7Cj4gPiArICAgICBp ZiAocmMpCj4gPiArICAgICAgICAgICAgIGdvdG8gb3V0X2NsZWFudXBfY29ubmVjdF9xOwo+ID4K PiA+ICAgICAgIGlmICghbmV3KSB7Cj4gPiAgICAgICAgICAgICAgIG52bWVfc3RhcnRfcXVldWVz KG5jdHJsKTsKPiA+IEBAIC0zMDAsMTYgKzUxMywxNiBAQCBudm1lX3RjcF9vZmxkX2NvbmZpZ3Vy ZV9pb19xdWV1ZXMoc3RydWN0IG52bWVfY3RybCAqbmN0cmwsIGJvb2wgbmV3KQo+ID4gICBvdXRf d2FpdF9mcmVlemVfdGltZWRfb3V0Ogo+ID4gICAgICAgbnZtZV9zdG9wX3F1ZXVlcyhuY3RybCk7 Cj4gPiAgICAgICBudm1lX3N5bmNfaW9fcXVldWVzKG5jdHJsKTsKPiA+IC0KPiA+IC0gICAgIC8q IFBsYWNlaG9sZGVyIC0gU3RvcCBJTyBxdWV1ZXMgKi8KPiA+IC0KPiA+ICsgICAgIG52bWVfdGNw X29mbGRfc3RvcF9pb19xdWV1ZXMobmN0cmwpOwo+ID4gK291dF9jbGVhbnVwX2Nvbm5lY3RfcToK PiA+ICsgICAgIG52bWVfY2FuY2VsX3RhZ3NldChuY3RybCk7Cj4gPiAgICAgICBpZiAobmV3KQo+ ID4gICAgICAgICAgICAgICBibGtfY2xlYW51cF9xdWV1ZShuY3RybC0+Y29ubmVjdF9xKTsKPiA+ ICAgb3V0X2ZyZWVfdGFnX3NldDoKPiA+ICAgICAgIGlmIChuZXcpCj4gPiAgICAgICAgICAgICAg IGJsa19tcV9mcmVlX3RhZ19zZXQobmN0cmwtPnRhZ3NldCk7Cj4gPiAgIG91dF9mcmVlX2lvX3F1 ZXVlczoKPiA+IC0gICAgIC8qIFBsYWNlaG9sZGVyIC0gZnJlZV9pb19xdWV1ZXMgKi8KPiA+ICsg ICAgIG52bWVfdGNwX29mbGRfdGVybWluYXRlX2lvX3F1ZXVlcyhuY3RybCwgbmN0cmwtPnF1ZXVl X2NvdW50KTsKPiA+Cj4gPiAgICAgICByZXR1cm4gcmM7Cj4gPiAgIH0KPiA+IEBAIC0zMzYsNiAr NTQ5LDI2IEBAIHN0YXRpYyB2b2lkIG52bWVfdGNwX29mbGRfcmVjb25uZWN0X29yX3JlbW92ZShz dHJ1Y3QgbnZtZV9jdHJsICpuY3RybCkKPiA+ICAgICAgIH0KPiA+ICAgfQo+ID4KPiA+ICtzdGF0 aWMgaW50Cj4gPiArbnZtZV90Y3Bfb2ZsZF9pbml0X2FkbWluX2hjdHgoc3RydWN0IGJsa19tcV9o d19jdHggKmhjdHgsIHZvaWQgKmRhdGEsCj4gPiArICAgICAgICAgICAgICAgICAgICAgICAgICAg dW5zaWduZWQgaW50IGhjdHhfaWR4KQo+ID4gK3sKPiA+ICsgICAgIHN0cnVjdCBudm1lX3RjcF9v ZmxkX2N0cmwgKmN0cmwgPSBkYXRhOwo+ID4gKwo+ID4gKyAgICAgaGN0eC0+ZHJpdmVyX2RhdGEg PSAmY3RybC0+cXVldWVzWzBdOwo+ID4gKwo+ID4gKyAgICAgcmV0dXJuIDA7Cj4gPiArfQo+ID4g Kwo+ID4gK3N0YXRpYyB2b2lkIG52bWVfdGNwX29mbGRfZGVzdHJveV9pb19xdWV1ZXMoc3RydWN0 IG52bWVfY3RybCAqbmN0cmwsIGJvb2wgcmVtb3ZlKQo+ID4gK3sKPiA+ICsgICAgIG52bWVfdGNw X29mbGRfc3RvcF9pb19xdWV1ZXMobmN0cmwpOwo+ID4gKyAgICAgaWYgKHJlbW92ZSkgewo+ID4g KyAgICAgICAgICAgICBibGtfY2xlYW51cF9xdWV1ZShuY3RybC0+Y29ubmVjdF9xKTsKPiA+ICsg ICAgICAgICAgICAgYmxrX21xX2ZyZWVfdGFnX3NldChuY3RybC0+dGFnc2V0KTsKPiA+ICsgICAg IH0KPiA+ICt9Cj4gPiArCj4gPiAgIHN0YXRpYyBpbnQgbnZtZV90Y3Bfb2ZsZF9zZXR1cF9jdHJs KHN0cnVjdCBudm1lX2N0cmwgKm5jdHJsLCBib29sIG5ldykKPiA+ICAgewo+ID4gICAgICAgc3Ry dWN0IG52bWZfY3RybF9vcHRpb25zICpvcHRzID0gbmN0cmwtPm9wdHM7Cj4gPiBAQCAtMzg3LDkg KzYyMCwxOSBAQCBzdGF0aWMgaW50IG52bWVfdGNwX29mbGRfc2V0dXBfY3RybChzdHJ1Y3QgbnZt ZV9jdHJsICpuY3RybCwgYm9vbCBuZXcpCj4gPiAgICAgICByZXR1cm4gMDsKPiA+Cj4gPiAgIGRl c3Ryb3lfaW86Cj4gPiAtICAgICAvKiBQbGFjZWhvbGRlciAtIHN0b3AgYW5kIGRlc3Ryb3kgaW8g cXVldWVzKi8KPiA+ICsgICAgIGlmIChuY3RybC0+cXVldWVfY291bnQgPiAxKSB7Cj4gPiArICAg ICAgICAgICAgIG52bWVfc3RvcF9xdWV1ZXMobmN0cmwpOwo+ID4gKyAgICAgICAgICAgICBudm1l X3N5bmNfaW9fcXVldWVzKG5jdHJsKTsKPiA+ICsgICAgICAgICAgICAgbnZtZV90Y3Bfb2ZsZF9z dG9wX2lvX3F1ZXVlcyhuY3RybCk7Cj4gPiArICAgICAgICAgICAgIG52bWVfY2FuY2VsX3RhZ3Nl dChuY3RybCk7Cj4gPiArICAgICAgICAgICAgIG52bWVfdGNwX29mbGRfZGVzdHJveV9pb19xdWV1 ZXMobmN0cmwsIG5ldyk7Cj4gPiArICAgICB9Cj4gPiAgIGRlc3Ryb3lfYWRtaW46Cj4gPiAtICAg ICAvKiBQbGFjZWhvbGRlciAtIHN0b3AgYW5kIGRlc3Ryb3kgYWRtaW4gcXVldWUqLwo+ID4gKyAg ICAgYmxrX21xX3F1aWVzY2VfcXVldWUobmN0cmwtPmFkbWluX3EpOwo+ID4gKyAgICAgYmxrX3N5 bmNfcXVldWUobmN0cmwtPmFkbWluX3EpOwo+ID4gKyAgICAgbnZtZV90Y3Bfb2ZsZF9zdG9wX3F1 ZXVlKG5jdHJsLCAwKTsKPiA+ICsgICAgIG52bWVfY2FuY2VsX2FkbWluX3RhZ3NldChuY3RybCk7 Cj4gPiArICAgICBudm1lX3RjcF9vZmxkX2Rlc3Ryb3lfYWRtaW5fcXVldWUobmN0cmwsIG5ldyk7 Cj4gPgo+ID4gICAgICAgcmV0dXJuIHJjOwo+ID4gICB9Cj4gPiBAQCAtNDEwLDYgKzY1MywxOCBA QCBudm1lX3RjcF9vZmxkX2NoZWNrX2Rldl9vcHRzKHN0cnVjdCBudm1mX2N0cmxfb3B0aW9ucyAq b3B0cywKPiA+ICAgICAgIHJldHVybiAwOwo+ID4gICB9Cj4gPgo+ID4gK3N0YXRpYyB2b2lkIG52 bWVfdGNwX29mbGRfZnJlZV9jdHJsX3F1ZXVlcyhzdHJ1Y3QgbnZtZV9jdHJsICpuY3RybCkKPiA+ ICt7Cj4gPiArICAgICBzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9jdHJsICpjdHJsID0gdG9fdGNwX29m bGRfY3RybChuY3RybCk7Cj4gPiArICAgICBpbnQgaTsKPiA+ICsKPiA+ICsgICAgIGZvciAoaSA9 IDA7IGkgPCBuY3RybC0+cXVldWVfY291bnQ7ICsraSkKPiA+ICsgICAgICAgICAgICAgbnZtZV90 Y3Bfb2ZsZF9mcmVlX3F1ZXVlKG5jdHJsLCBpKTsKPiA+ICsKPiA+ICsgICAgIGtmcmVlKGN0cmwt PnF1ZXVlcyk7Cj4gPiArICAgICBjdHJsLT5xdWV1ZXMgPSBOVUxMOwo+ID4gK30KPiA+ICsKPiA+ ICAgc3RhdGljIHZvaWQgbnZtZV90Y3Bfb2ZsZF9mcmVlX2N0cmwoc3RydWN0IG52bWVfY3RybCAq bmN0cmwpCj4gPiAgIHsKPiA+ICAgICAgIHN0cnVjdCBudm1lX3RjcF9vZmxkX2N0cmwgKmN0cmwg PSB0b190Y3Bfb2ZsZF9jdHJsKG5jdHJsKTsKPiA+IEBAIC00MTksNiArNjc0LDcgQEAgc3RhdGlj IHZvaWQgbnZtZV90Y3Bfb2ZsZF9mcmVlX2N0cmwoc3RydWN0IG52bWVfY3RybCAqbmN0cmwpCj4g PiAgICAgICAgICAgICAgIGdvdG8gZnJlZV9jdHJsOwo+ID4KPiA+ICAgICAgIGRvd25fd3JpdGUo Jm52bWVfdGNwX29mbGRfY3RybF9yd3NlbSk7Cj4gPiArICAgICBudm1lX3RjcF9vZmxkX2ZyZWVf Y3RybF9xdWV1ZXMobmN0cmwpOwo+ID4gICAgICAgY3RybC0+ZGV2LT5vcHMtPnJlbGVhc2VfY3Ry bChjdHJsKTsKPiA+ICAgICAgIGxpc3RfZGVsKCZjdHJsLT5saXN0KTsKPiA+ICAgICAgIHVwX3dy aXRlKCZudm1lX3RjcF9vZmxkX2N0cmxfcndzZW0pOwo+ID4gQEAgLTQzNiwxNSArNjkyLDM3IEBA IHN0YXRpYyB2b2lkIG52bWVfdGNwX29mbGRfc3VibWl0X2FzeW5jX2V2ZW50KHN0cnVjdCBudm1l X2N0cmwgKmFyZykKPiA+ICAgfQo+ID4KPiA+ICAgc3RhdGljIHZvaWQKPiA+IC1udm1lX3RjcF9v ZmxkX3RlYXJkb3duX2FkbWluX3F1ZXVlKHN0cnVjdCBudm1lX2N0cmwgKmN0cmwsIGJvb2wgcmVt b3ZlKQo+ID4gK252bWVfdGNwX29mbGRfdGVhcmRvd25fYWRtaW5fcXVldWUoc3RydWN0IG52bWVf Y3RybCAqbmN0cmwsIGJvb2wgcmVtb3ZlKQo+ID4gICB7Cj4gPiAtICAgICAvKiBQbGFjZWhvbGRl ciAtIHRlYXJkb3duX2FkbWluX3F1ZXVlICovCj4gPiArICAgICBibGtfbXFfcXVpZXNjZV9xdWV1 ZShuY3RybC0+YWRtaW5fcSk7Cj4gPiArICAgICBibGtfc3luY19xdWV1ZShuY3RybC0+YWRtaW5f cSk7Cj4gPiArCj4gPiArICAgICBudm1lX3RjcF9vZmxkX3N0b3BfcXVldWUobmN0cmwsIDApOwo+ ID4gKyAgICAgbnZtZV9jYW5jZWxfYWRtaW5fdGFnc2V0KG5jdHJsKTsKPiA+ICsKPiA+ICsgICAg IGlmIChyZW1vdmUpCj4gPiArICAgICAgICAgICAgIGJsa19tcV91bnF1aWVzY2VfcXVldWUobmN0 cmwtPmFkbWluX3EpOwo+ID4gKwo+ID4gKyAgICAgbnZtZV90Y3Bfb2ZsZF9kZXN0cm95X2FkbWlu X3F1ZXVlKG5jdHJsLCByZW1vdmUpOwo+ID4gICB9Cj4gPgo+ID4gICBzdGF0aWMgdm9pZAo+ID4g ICBudm1lX3RjcF9vZmxkX3RlYXJkb3duX2lvX3F1ZXVlcyhzdHJ1Y3QgbnZtZV9jdHJsICpuY3Ry bCwgYm9vbCByZW1vdmUpCj4gPiAgIHsKPiA+IC0gICAgIC8qIFBsYWNlaG9sZGVyIC0gdGVhcmRv d25faW9fcXVldWVzICovCj4gPiArICAgICBpZiAobmN0cmwtPnF1ZXVlX2NvdW50IDw9IDEpCj4g PiArICAgICAgICAgICAgIHJldHVybjsKPiA+ICsKPiA+ICsgICAgIGJsa19tcV9xdWllc2NlX3F1 ZXVlKG5jdHJsLT5hZG1pbl9xKTsKPiA+ICsgICAgIG52bWVfc3RhcnRfZnJlZXplKG5jdHJsKTsK PiA+ICsgICAgIG52bWVfc3RvcF9xdWV1ZXMobmN0cmwpOwo+ID4gKyAgICAgbnZtZV9zeW5jX2lv X3F1ZXVlcyhuY3RybCk7Cj4gPiArICAgICBudm1lX3RjcF9vZmxkX3N0b3BfaW9fcXVldWVzKG5j dHJsKTsKPiA+ICsgICAgIG52bWVfY2FuY2VsX3RhZ3NldChuY3RybCk7Cj4gPiArCj4gPiArICAg ICBpZiAocmVtb3ZlKQo+ID4gKyAgICAgICAgICAgICBudm1lX3N0YXJ0X3F1ZXVlcyhuY3RybCk7 Cj4gPiArCj4gPiArICAgICBudm1lX3RjcF9vZmxkX2Rlc3Ryb3lfaW9fcXVldWVzKG5jdHJsLCBy ZW1vdmUpOwo+ID4gICB9Cj4gPgo+ID4gICBzdGF0aWMgdm9pZCBudm1lX3RjcF9vZmxkX3JlY29u bmVjdF9jdHJsX3dvcmsoc3RydWN0IHdvcmtfc3RydWN0ICp3b3JrKQo+ID4gQEAgLTU3Miw2ICs4 NTAsMTcgQEAgbnZtZV90Y3Bfb2ZsZF9pbml0X3JlcXVlc3Qoc3RydWN0IGJsa19tcV90YWdfc2V0 ICpzZXQsCj4gPiAgICAgICByZXR1cm4gMDsKPiA+ICAgfQo+ID4KPiA+ICtpbmxpbmUgc2l6ZV90 IG52bWVfdGNwX29mbGRfaW5saW5lX2RhdGFfc2l6ZShzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9xdWV1 ZSAqcXVldWUpCj4gPiArewo+ID4gKyAgICAgcmV0dXJuIHF1ZXVlLT5jbW5kX2NhcHN1bGVfbGVu IC0gc2l6ZW9mKHN0cnVjdCBudm1lX2NvbW1hbmQpOwo+ID4gK30KPiA+ICtFWFBPUlRfU1lNQk9M X0dQTChudm1lX3RjcF9vZmxkX2lubGluZV9kYXRhX3NpemUpOwo+ID4gKwo+ID4gK3N0YXRpYyB2 b2lkIG52bWVfdGNwX29mbGRfY29tbWl0X3JxcyhzdHJ1Y3QgYmxrX21xX2h3X2N0eCAqaGN0eCkK PiA+ICt7Cj4gPiArICAgICAvKiBDYWxsIG9wcy0+Y29tbWl0X3JxcyAqLwo+ID4gK30KPiA+ICsK PiA+ICAgc3RhdGljIGJsa19zdGF0dXNfdAo+ID4gICBudm1lX3RjcF9vZmxkX3F1ZXVlX3JxKHN0 cnVjdCBibGtfbXFfaHdfY3R4ICpoY3R4LAo+ID4gICAgICAgICAgICAgICAgICAgICAgY29uc3Qg c3RydWN0IGJsa19tcV9xdWV1ZV9kYXRhICpiZCkKPiA+IEBAIC01ODMsMjIgKzg3Miw5NiBAQCBu dm1lX3RjcF9vZmxkX3F1ZXVlX3JxKHN0cnVjdCBibGtfbXFfaHdfY3R4ICpoY3R4LAo+ID4gICAg ICAgcmV0dXJuIEJMS19TVFNfT0s7Cj4gPiAgIH0KPiA+Cj4gPiArc3RhdGljIHZvaWQKPiA+ICtu dm1lX3RjcF9vZmxkX2V4aXRfcmVxdWVzdChzdHJ1Y3QgYmxrX21xX3RhZ19zZXQgKnNldCwKPiA+ ICsgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgcmVxdWVzdCAqcnEsIHVuc2lnbmVkIGlu dCBoY3R4X2lkeCkKPiA+ICt7Cj4gPiArICAgICAvKgo+ID4gKyAgICAgICogTm90aGluZyBpcyBh bGxvY2F0ZWQgaW4gbnZtZV90Y3Bfb2ZsZF9pbml0X3JlcXVlc3QsCj4gPiArICAgICAgKiBoZW5j ZSBlbXB0eS4KPiA+ICsgICAgICAqLwo+ID4gK30KPiA+ICsKPiA+ICtzdGF0aWMgaW50Cj4gPiAr bnZtZV90Y3Bfb2ZsZF9pbml0X2hjdHgoc3RydWN0IGJsa19tcV9od19jdHggKmhjdHgsIHZvaWQg KmRhdGEsCj4gPiArICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IGhjdHhfaWR4KQo+ ID4gK3sKPiA+ICsgICAgIHN0cnVjdCBudm1lX3RjcF9vZmxkX2N0cmwgKmN0cmwgPSBkYXRhOwo+ ID4gKwo+ID4gKyAgICAgaGN0eC0+ZHJpdmVyX2RhdGEgPSAmY3RybC0+cXVldWVzW2hjdHhfaWR4 ICsgMV07Cj4gPiArCj4gPiArICAgICByZXR1cm4gMDsKPiA+ICt9Cj4gPiArCj4gPiArc3RhdGlj IGludCBudm1lX3RjcF9vZmxkX21hcF9xdWV1ZXMoc3RydWN0IGJsa19tcV90YWdfc2V0ICpzZXQp Cj4gPiArewo+ID4gKyAgICAgc3RydWN0IG52bWVfdGNwX29mbGRfY3RybCAqY3RybCA9IHNldC0+ ZHJpdmVyX2RhdGE7Cj4gPiArICAgICBzdHJ1Y3QgbnZtZl9jdHJsX29wdGlvbnMgKm9wdHMgPSBj dHJsLT5uY3RybC5vcHRzOwo+ID4gKwo+ID4gKyAgICAgaWYgKG9wdHMtPm5yX3dyaXRlX3F1ZXVl cyAmJiBjdHJsLT5pb19xdWV1ZXNbSENUWF9UWVBFX1JFQURdKSB7Cj4gPiArICAgICAgICAgICAg IC8qIHNlcGFyYXRlIHJlYWQvd3JpdGUgcXVldWVzICovCj4gPiArICAgICAgICAgICAgIHNldC0+ bWFwW0hDVFhfVFlQRV9ERUZBVUxUXS5ucl9xdWV1ZXMgPQo+ID4gKyAgICAgICAgICAgICAgICAg ICAgIGN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZUEVfREVGQVVMVF07Cj4gPiArICAgICAgICAgICAg IHNldC0+bWFwW0hDVFhfVFlQRV9ERUZBVUxUXS5xdWV1ZV9vZmZzZXQgPSAwOwo+ID4gKyAgICAg ICAgICAgICBzZXQtPm1hcFtIQ1RYX1RZUEVfUkVBRF0ubnJfcXVldWVzID0KPiA+ICsgICAgICAg ICAgICAgICAgICAgICBjdHJsLT5pb19xdWV1ZXNbSENUWF9UWVBFX1JFQURdOwo+ID4gKyAgICAg ICAgICAgICBzZXQtPm1hcFtIQ1RYX1RZUEVfUkVBRF0ucXVldWVfb2Zmc2V0ID0KPiA+ICsgICAg ICAgICAgICAgICAgICAgICBjdHJsLT5pb19xdWV1ZXNbSENUWF9UWVBFX0RFRkFVTFRdOwo+ID4g KyAgICAgfSBlbHNlIHsKPiA+ICsgICAgICAgICAgICAgLyogc2hhcmVkIHJlYWQvd3JpdGUgcXVl dWVzICovCj4gPiArICAgICAgICAgICAgIHNldC0+bWFwW0hDVFhfVFlQRV9ERUZBVUxUXS5ucl9x dWV1ZXMgPQo+ID4gKyAgICAgICAgICAgICAgICAgICAgIGN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZ UEVfREVGQVVMVF07Cj4gPiArICAgICAgICAgICAgIHNldC0+bWFwW0hDVFhfVFlQRV9ERUZBVUxU XS5xdWV1ZV9vZmZzZXQgPSAwOwo+ID4gKyAgICAgICAgICAgICBzZXQtPm1hcFtIQ1RYX1RZUEVf UkVBRF0ubnJfcXVldWVzID0KPiA+ICsgICAgICAgICAgICAgICAgICAgICBjdHJsLT5pb19xdWV1 ZXNbSENUWF9UWVBFX0RFRkFVTFRdOwo+ID4gKyAgICAgICAgICAgICBzZXQtPm1hcFtIQ1RYX1RZ UEVfUkVBRF0ucXVldWVfb2Zmc2V0ID0gMDsKPiA+ICsgICAgIH0KPiA+ICsgICAgIGJsa19tcV9t YXBfcXVldWVzKCZzZXQtPm1hcFtIQ1RYX1RZUEVfREVGQVVMVF0pOwo+ID4gKyAgICAgYmxrX21x X21hcF9xdWV1ZXMoJnNldC0+bWFwW0hDVFhfVFlQRV9SRUFEXSk7Cj4gPiArCj4gPiArICAgICBp ZiAob3B0cy0+bnJfcG9sbF9xdWV1ZXMgJiYgY3RybC0+aW9fcXVldWVzW0hDVFhfVFlQRV9QT0xM XSkgewo+ID4gKyAgICAgICAgICAgICAvKiBtYXAgZGVkaWNhdGVkIHBvbGwgcXVldWVzIG9ubHkg aWYgd2UgaGF2ZSBxdWV1ZXMgbGVmdCAqLwo+ID4gKyAgICAgICAgICAgICBzZXQtPm1hcFtIQ1RY X1RZUEVfUE9MTF0ubnJfcXVldWVzID0KPiA+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAg IGN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZUEVfUE9MTF07Cj4gPiArICAgICAgICAgICAgIHNldC0+ bWFwW0hDVFhfVFlQRV9QT0xMXS5xdWV1ZV9vZmZzZXQgPQo+ID4gKyAgICAgICAgICAgICAgICAg ICAgIGN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZUEVfREVGQVVMVF0gKwo+ID4gKyAgICAgICAgICAg ICAgICAgICAgIGN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZUEVfUkVBRF07Cj4gPiArICAgICAgICAg ICAgIGJsa19tcV9tYXBfcXVldWVzKCZzZXQtPm1hcFtIQ1RYX1RZUEVfUE9MTF0pOwo+ID4gKyAg ICAgfQo+ID4gKwo+ID4gKyAgICAgZGV2X2luZm8oY3RybC0+bmN0cmwuZGV2aWNlLAo+ID4gKyAg ICAgICAgICAgICAgIm1hcHBlZCAlZC8lZC8lZCBkZWZhdWx0L3JlYWQvcG9sbCBxdWV1ZXMuXG4i LAo+ID4gKyAgICAgICAgICAgICAgY3RybC0+aW9fcXVldWVzW0hDVFhfVFlQRV9ERUZBVUxUXSwK PiA+ICsgICAgICAgICAgICAgIGN0cmwtPmlvX3F1ZXVlc1tIQ1RYX1RZUEVfUkVBRF0sCj4gPiAr ICAgICAgICAgICAgICBjdHJsLT5pb19xdWV1ZXNbSENUWF9UWVBFX1BPTExdKTsKPiA+ICsKPiA+ ICsgICAgIHJldHVybiAwOwo+ID4gK30KPiA+ICsKPiA+ICtzdGF0aWMgaW50IG52bWVfdGNwX29m bGRfcG9sbChzdHJ1Y3QgYmxrX21xX2h3X2N0eCAqaGN0eCkKPiA+ICt7Cj4gPiArICAgICAvKiBQ bGFjZWhvbGRlciAtIEltcGxlbWVudCBwb2xsaW5nIG1lY2hhbmlzbSAqLwo+ID4gKwo+ID4gKyAg ICAgcmV0dXJuIDA7Cj4gPiArfQo+ID4gKwo+ID4gICBzdGF0aWMgc3RydWN0IGJsa19tcV9vcHMg bnZtZV90Y3Bfb2ZsZF9tcV9vcHMgPSB7Cj4gPiAgICAgICAucXVldWVfcnEgICAgICAgPSBudm1l X3RjcF9vZmxkX3F1ZXVlX3JxLAo+ID4gKyAgICAgLmNvbW1pdF9ycXMgICAgID0gbnZtZV90Y3Bf b2ZsZF9jb21taXRfcnFzLAo+ID4gKyAgICAgLmNvbXBsZXRlICAgICAgID0gbnZtZV9jb21wbGV0 ZV9ycSwKPiA+ICAgICAgIC5pbml0X3JlcXVlc3QgICA9IG52bWVfdGNwX29mbGRfaW5pdF9yZXF1 ZXN0LAo+ID4gLSAgICAgLyoKPiA+IC0gICAgICAqIEFsbCBhZGRpdGlvbmFsIG9wcyB3aWxsIGJl IGFsc28gaW1wbGVtZW50ZWQgYW5kIHJlZ2lzdGVyZWQgc2ltaWxhciB0bwo+ID4gLSAgICAgICog dGNwLmMKPiA+IC0gICAgICAqLwo+ID4gKyAgICAgLmV4aXRfcmVxdWVzdCAgID0gbnZtZV90Y3Bf b2ZsZF9leGl0X3JlcXVlc3QsCj4gPiArICAgICAuaW5pdF9oY3R4ICAgICAgPSBudm1lX3RjcF9v ZmxkX2luaXRfaGN0eCwKPiA+ICsgICAgIC5tYXBfcXVldWVzICAgICA9IG52bWVfdGNwX29mbGRf bWFwX3F1ZXVlcywKPiA+ICsgICAgIC5wb2xsICAgICAgICAgICA9IG52bWVfdGNwX29mbGRfcG9s bCwKPiA+ICAgfTsKPiA+Cj4gPiAgIHN0YXRpYyBzdHJ1Y3QgYmxrX21xX29wcyBudm1lX3RjcF9v ZmxkX2FkbWluX21xX29wcyA9IHsKPiA+ICAgICAgIC5xdWV1ZV9ycSAgICAgICA9IG52bWVfdGNw X29mbGRfcXVldWVfcnEsCj4gPiArICAgICAuY29tcGxldGUgICAgICAgPSBudm1lX2NvbXBsZXRl X3JxLAo+ID4gICAgICAgLmluaXRfcmVxdWVzdCAgID0gbnZtZV90Y3Bfb2ZsZF9pbml0X3JlcXVl c3QsCj4gPiAtICAgICAvKgo+ID4gLSAgICAgICogQWxsIGFkZGl0aW9uYWwgb3BzIHdpbGwgYmUg YWxzbyBpbXBsZW1lbnRlZCBhbmQgcmVnaXN0ZXJlZCBzaW1pbGFyIHRvCj4gPiAtICAgICAgKiB0 Y3AuYwo+ID4gLSAgICAgICovCj4gPiArICAgICAuZXhpdF9yZXF1ZXN0ICAgPSBudm1lX3RjcF9v ZmxkX2V4aXRfcmVxdWVzdCwKPiA+ICsgICAgIC5pbml0X2hjdHggICAgICA9IG52bWVfdGNwX29m bGRfaW5pdF9hZG1pbl9oY3R4LAo+ID4gICB9Owo+ID4KPiA+ICAgc3RhdGljIGNvbnN0IHN0cnVj dCBudm1lX2N0cmxfb3BzIG52bWVfdGNwX29mbGRfY3RybF9vcHMgPSB7Cj4gPiBkaWZmIC0tZ2l0 IGEvZHJpdmVycy9udm1lL2hvc3QvdGNwLW9mZmxvYWQuaCBiL2RyaXZlcnMvbnZtZS9ob3N0L3Rj cC1vZmZsb2FkLmgKPiA+IGluZGV4IGIyM2IxZDdlYTZmYS4uZDgyNjQ1ZmNmOWRhIDEwMDY0NAo+ ID4gLS0tIGEvZHJpdmVycy9udm1lL2hvc3QvdGNwLW9mZmxvYWQuaAo+ID4gKysrIGIvZHJpdmVy cy9udm1lL2hvc3QvdGNwLW9mZmxvYWQuaAo+ID4gQEAgLTEwNSw3ICsxMDUsNiBAQCBzdHJ1Y3Qg bnZtZV90Y3Bfb2ZsZF9jdHJsIHsKPiA+ICAgICAgICAqIEVhY2ggZW50cnkgaW4gdGhlIGFycmF5 IGluZGljYXRlcyB0aGUgbnVtYmVyIG9mIHF1ZXVlcyBvZgo+ID4gICAgICAgICogY29ycmVzcG9u ZGluZyB0eXBlLgo+ID4gICAgICAgICovCj4gPiAtICAgICB1MzIgcXVldWVfdHlwZV9tYXBwaW5n W0hDVFhfTUFYX1RZUEVTXTsKPiA+ICAgICAgIHUzMiBpb19xdWV1ZXNbSENUWF9NQVhfVFlQRVNd Owo+ID4KPiA+ICAgICAgIC8qIENvbm5lY3Rpdml0eSBwYXJhbXMgKi8KPiA+IEBAIC0yMDUsMyAr MjA0LDQgQEAgc3RydWN0IG52bWVfdGNwX29mbGRfb3BzIHsKPiA+ICAgaW50IG52bWVfdGNwX29m bGRfcmVnaXN0ZXJfZGV2KHN0cnVjdCBudm1lX3RjcF9vZmxkX2RldiAqZGV2KTsKPiA+ICAgdm9p ZCBudm1lX3RjcF9vZmxkX3VucmVnaXN0ZXJfZGV2KHN0cnVjdCBudm1lX3RjcF9vZmxkX2RldiAq ZGV2KTsKPiA+ICAgdm9pZCBudm1lX3RjcF9vZmxkX2Vycm9yX3JlY292ZXJ5KHN0cnVjdCBudm1l X2N0cmwgKm5jdHJsKTsKPiA+ICtpbmxpbmUgc2l6ZV90IG52bWVfdGNwX29mbGRfaW5saW5lX2Rh dGFfc2l6ZShzdHJ1Y3QgbnZtZV90Y3Bfb2ZsZF9xdWV1ZSAqcXVldWUpOwo+ID4KPiBDaGVlcnMs Cj4KPiBIYW5uZXMKPiAtLQo+IERyLiBIYW5uZXMgUmVpbmVja2UgICAgICAgICAgICAgICAgS2Vy bmVsIFN0b3JhZ2UgQXJjaGl0ZWN0Cj4gaGFyZUBzdXNlLmRlICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgKzQ5IDkxMSA3NDA1MyA2ODgKPiBTVVNFIFNvZnR3YXJlIFNvbHV0aW9ucyBHbWJI LCBNYXhmZWxkc3RyLiA1LCA5MDQwOSBOw7xybmJlcmcKPiBIUkIgMzY4MDkgKEFHIE7DvHJuYmVy ZyksIEdlc2Now6RmdHNmw7xocmVyOiBGZWxpeCBJbWVuZMO2cmZmZXIKCl9fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCkxpbnV4LW52bWUgbWFpbGluZyBsaXN0 CkxpbnV4LW52bWVAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9y Zy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LW52bWUK