From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2567C282C4 for ; Mon, 4 Feb 2019 20:33:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 85DA52184E for ; Mon, 4 Feb 2019 20:33:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Klc8gqFG" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729453AbfBDUdM (ORCPT ); Mon, 4 Feb 2019 15:33:12 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:36449 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729415AbfBDUdM (ORCPT ); Mon, 4 Feb 2019 15:33:12 -0500 Received: by mail-wm1-f67.google.com with SMTP id p6so1382709wmc.1 for ; Mon, 04 Feb 2019 12:33:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=2m17gXOtkMOSTAOTsEyOmwhxXNSBBegRnLZuu6UoDfs=; b=Klc8gqFGy55SX2riC7NHRrgMI02ooXo1hW+P+1TX/ZbPLlEKg5soK98l/63E+wgjQi tHMd+u/oPBKOZ4cvLcp8HT0DnBoGtYdEYsd586ObI0kvPYRNJHSgoY+exUAihY3wbi8c d5x6mENa4acaAoiRUtk+NvggTN33QuoACk1QcqYVbQBLTe2YiWP0hLwaI1X9yvV0pjoD y9VKS+JMpfvjdEhvfbqZCCWCeqVIvmNIKyv84M5ic5J9k7zc3q/plE2BKHO50Y+HUdjX 7S4ZShPAC4Q/+m6fX6NTNMfKn61ys/m1nSYDZmzY4S6MAKb7h1EChFqWjkOvr2/c+hGm aRzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=2m17gXOtkMOSTAOTsEyOmwhxXNSBBegRnLZuu6UoDfs=; b=sy6wq7Qd03MQD4+84m4REfmsospT5YyD150YAh8LFufMkB+V6KiCqZ9X1NL0wzZNNU IfzQ9aZnYWmkNHlefu+1HzklMpO2L52a2qmWwRaIcw5hR674kdx8jlPQ1sNKlpbJga/y m2eg/ulY64xfg58NzE5Ow9UvBGeKG+uGAlniONK3h4Ofv0/FKOPEl8D2WJepsBATfKCp iYnoaIFqbg5fvpkku4oWxxRCnfEx2MpkxWO6ZPpuXltlXO7fmcEgiUqqDGoWAeRr0qza l26uRWYAJ/zvZ6eReSwp+LaIhzTfhWUlQ3EO+3C87afT4Ortv9mvuygPPJV0zdxKPHmO Unwg== X-Gm-Message-State: AHQUAuYrAzeFtrMhgVQS4rTcA1OvmvWy02IW6fB4eInmyUBoCbgLTlnl RYg8lhs8b/HZj7nnqHDLPKw= X-Google-Smtp-Source: AHgI3IaO9S0IcIbdSmmvTLOZ5/Utm4rxIKGm2mlIlUqoaEwVurqZYpJ7yUdIk3xfB9tFM3rJj8wATA== X-Received: by 2002:a1c:b70a:: with SMTP id h10mr807932wmf.125.1549312388078; Mon, 04 Feb 2019 12:33:08 -0800 (PST) Received: from ogabbay-VM.habana-labs.com ([31.154.190.6]) by smtp.gmail.com with ESMTPSA id j24sm19990792wrd.86.2019.02.04.12.33.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 Feb 2019 12:33:07 -0800 (PST) From: Oded Gabbay To: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, olof@lixom.net, rppt@linux.ibm.com Cc: ogabbay@habana.ai, arnd@arndb.de, joe@perches.com Subject: [PATCH v3 04/15] habanalabs: add context and ASID modules Date: Mon, 4 Feb 2019 22:32:43 +0200 Message-Id: <20190204203254.4026-5-oded.gabbay@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190204203254.4026-1-oded.gabbay@gmail.com> References: <20190204203254.4026-1-oded.gabbay@gmail.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch adds two modules - ASID and context. Each user process that opens a device's file must have at least one context before it is able to "work" with the device. Each context has its own device address-space and contains information about its runtime state (its active command submissions). To have address-space separation between contexts, each context is assigned a unique ASID, which stands for "address-space id". Goya supports up to 1024 ASIDs. Currently, the driver doesn't support multiple contexts. Therefore, the user doesn't need to actively create a context. A "primary context" is created automatically when the user opens the device's file. Signed-off-by: Oded Gabbay --- drivers/misc/habanalabs/Makefile | 2 +- drivers/misc/habanalabs/asid.c | 58 +++++++++ drivers/misc/habanalabs/context.c | 155 +++++++++++++++++++++++ drivers/misc/habanalabs/device.c | 47 +++++++ drivers/misc/habanalabs/habanalabs.h | 71 +++++++++++ drivers/misc/habanalabs/habanalabs_drv.c | 46 ++++++- 6 files changed, 376 insertions(+), 3 deletions(-) create mode 100644 drivers/misc/habanalabs/asid.c create mode 100644 drivers/misc/habanalabs/context.c diff --git a/drivers/misc/habanalabs/Makefile b/drivers/misc/habanalabs/Makefile index 64bad3ca21c6..4944d6ccf8fe 100644 --- a/drivers/misc/habanalabs/Makefile +++ b/drivers/misc/habanalabs/Makefile @@ -4,7 +4,7 @@ obj-m := habanalabs.o -habanalabs-y := habanalabs_drv.o device.o +habanalabs-y := habanalabs_drv.o device.o context.o asid.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES) diff --git a/drivers/misc/habanalabs/asid.c b/drivers/misc/habanalabs/asid.c new file mode 100644 index 000000000000..0ce84c8f5a47 --- /dev/null +++ b/drivers/misc/habanalabs/asid.c @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Copyright 2016-2018 HabanaLabs, Ltd. + * All Rights Reserved. + */ + +#include "habanalabs.h" + +#include +#include + +int hl_asid_init(struct hl_device *hdev) +{ + hdev->asid_bitmap = kcalloc(BITS_TO_LONGS(hdev->asic_prop.max_asid), + sizeof(*hdev->asid_bitmap), GFP_KERNEL); + if (!hdev->asid_bitmap) + return -ENOMEM; + + mutex_init(&hdev->asid_mutex); + + /* ASID 0 is reserved for KMD */ + set_bit(0, hdev->asid_bitmap); + + return 0; +} + +void hl_asid_fini(struct hl_device *hdev) +{ + mutex_destroy(&hdev->asid_mutex); + kfree(hdev->asid_bitmap); +} + +unsigned long hl_asid_alloc(struct hl_device *hdev) +{ + unsigned long found; + + mutex_lock(&hdev->asid_mutex); + + found = find_first_zero_bit(hdev->asid_bitmap, + hdev->asic_prop.max_asid); + if (found == hdev->asic_prop.max_asid) + found = 0; + else + set_bit(found, hdev->asid_bitmap); + + mutex_unlock(&hdev->asid_mutex); + + return found; +} + +void hl_asid_free(struct hl_device *hdev, unsigned long asid) +{ + if (WARN((asid == 0 || asid >= hdev->asic_prop.max_asid), + "Invalid ASID %lu", asid)) + return; + clear_bit(asid, hdev->asid_bitmap); +} diff --git a/drivers/misc/habanalabs/context.c b/drivers/misc/habanalabs/context.c new file mode 100644 index 000000000000..4d46e51e1a82 --- /dev/null +++ b/drivers/misc/habanalabs/context.c @@ -0,0 +1,155 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Copyright 2016-2018 HabanaLabs, Ltd. + * All Rights Reserved. + */ + +#include "habanalabs.h" + +#include +#include + +static void hl_ctx_fini(struct hl_ctx *ctx) +{ + struct hl_device *hdev = ctx->hdev; + + if (ctx->asid != HL_KERNEL_ASID_ID) + hl_asid_free(hdev, ctx->asid); +} + +void hl_ctx_do_release(struct kref *ref) +{ + struct hl_ctx *ctx; + + ctx = container_of(ref, struct hl_ctx, refcount); + + dev_dbg(ctx->hdev->dev, "Now really releasing context %d\n", ctx->asid); + + hl_ctx_fini(ctx); + + if (ctx->hpriv) + hl_hpriv_put(ctx->hpriv); + + kfree(ctx); +} + +int hl_ctx_create(struct hl_device *hdev, struct hl_fpriv *hpriv) +{ + struct hl_ctx_mgr *mgr = &hpriv->ctx_mgr; + struct hl_ctx *ctx; + int rc; + + ctx = kzalloc(sizeof(*ctx), GFP_KERNEL); + if (!ctx) { + rc = -ENOMEM; + goto out_err; + } + + rc = hl_ctx_init(hdev, ctx, false); + if (rc) + goto free_ctx; + + hl_hpriv_get(hpriv); + ctx->hpriv = hpriv; + + /* TODO: remove for multiple contexts */ + hpriv->ctx = ctx; + hdev->user_ctx = ctx; + + mutex_lock(&mgr->ctx_lock); + rc = idr_alloc(&mgr->ctx_handles, ctx, 1, 0, GFP_KERNEL); + mutex_unlock(&mgr->ctx_lock); + + if (rc < 0) { + dev_err(hdev->dev, "Failed to allocate IDR for a new CTX\n"); + hl_ctx_free(hdev, ctx); + goto out_err; + } + + return 0; + +free_ctx: + kfree(ctx); +out_err: + return rc; +} + +void hl_ctx_free(struct hl_device *hdev, struct hl_ctx *ctx) +{ + if (kref_put(&ctx->refcount, hl_ctx_do_release) == 1) + return; + + dev_warn(hdev->dev, + "Context %d closed or terminated but its CS are executing\n", + ctx->asid); +} + +int hl_ctx_init(struct hl_device *hdev, struct hl_ctx *ctx, bool is_kernel_ctx) +{ + ctx->hdev = hdev; + + kref_init(&ctx->refcount); + + if (is_kernel_ctx) { + ctx->asid = HL_KERNEL_ASID_ID; /* KMD gets ASID 0 */ + } else { + ctx->asid = hl_asid_alloc(hdev); + if (!ctx->asid) { + dev_err(hdev->dev, "No free ASID, failed to create context\n"); + return -ENOMEM; + } + } + + dev_dbg(hdev->dev, "Created context with ASID %u\n", ctx->asid); + + return 0; +} + +void hl_ctx_get(struct hl_device *hdev, struct hl_ctx *ctx) +{ + kref_get(&ctx->refcount); +} + +int hl_ctx_put(struct hl_ctx *ctx) +{ + return kref_put(&ctx->refcount, hl_ctx_do_release); +} + +/* + * hl_ctx_mgr_init - initialize the context manager + * + * @mgr: pointer to context manager structure + * + * This manager is an object inside the hpriv object of the user process. + * The function is called when a user process opens the FD. + */ +void hl_ctx_mgr_init(struct hl_ctx_mgr *mgr) +{ + mutex_init(&mgr->ctx_lock); + idr_init(&mgr->ctx_handles); +} + +/* + * hl_ctx_mgr_fini - finalize the context manager + * + * @hdev: pointer to device structure + * @mgr: pointer to context manager structure + * + * This function goes over all the contexts in the manager and frees them. + * It is called when a process closes the FD. + */ +void hl_ctx_mgr_fini(struct hl_device *hdev, struct hl_ctx_mgr *mgr) +{ + struct hl_ctx *ctx; + struct idr *idp; + u32 id; + + idp = &mgr->ctx_handles; + + idr_for_each_entry(idp, ctx, id) + hl_ctx_free(hdev, ctx); + + idr_destroy(&mgr->ctx_handles); + mutex_destroy(&mgr->ctx_lock); +} diff --git a/drivers/misc/habanalabs/device.c b/drivers/misc/habanalabs/device.c index d77e070ae389..81acb1de9c7e 100644 --- a/drivers/misc/habanalabs/device.c +++ b/drivers/misc/habanalabs/device.c @@ -23,6 +23,12 @@ static void hpriv_release(struct kref *ref) put_pid(hpriv->taskpid); kfree(hpriv); + + /* Now the FD is really closed */ + atomic_dec(&hdev->fd_open_cnt); + + /* This allows a new user context to open the device */ + hdev->user_ctx = NULL; } void hl_hpriv_get(struct hl_fpriv *hpriv) @@ -47,6 +53,8 @@ static int hl_device_release(struct inode *inode, struct file *filp) { struct hl_fpriv *hpriv = filp->private_data; + hl_ctx_mgr_fini(hpriv->hdev, &hpriv->ctx_mgr); + filp->private_data = NULL; hl_hpriv_put(hpriv); @@ -138,7 +146,20 @@ static int device_early_init(struct hl_device *hdev) if (rc) return rc; + rc = hl_asid_init(hdev); + if (rc) + goto early_fini; + + mutex_init(&hdev->fd_open_cnt_lock); + atomic_set(&hdev->fd_open_cnt, 0); + return 0; + +early_fini: + if (hdev->asic_funcs->early_fini) + hdev->asic_funcs->early_fini(hdev); + + return rc; } /* @@ -150,9 +171,12 @@ static int device_early_init(struct hl_device *hdev) static void device_early_fini(struct hl_device *hdev) { + hl_asid_fini(hdev); + if (hdev->asic_funcs->early_fini) hdev->asic_funcs->early_fini(hdev); + mutex_destroy(&hdev->fd_open_cnt_lock); } /* @@ -246,11 +270,30 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) if (rc) goto early_fini; + /* Allocate the kernel context */ + hdev->kernel_ctx = kzalloc(sizeof(*hdev->kernel_ctx), GFP_KERNEL); + if (!hdev->kernel_ctx) { + rc = -ENOMEM; + goto sw_fini; + } + + hdev->user_ctx = NULL; + + rc = hl_ctx_init(hdev, hdev->kernel_ctx, true); + if (rc) { + dev_err(hdev->dev, "failed to initialize kernel context\n"); + goto free_ctx; + } + dev_notice(hdev->dev, "Successfully added device to habanalabs driver\n"); return 0; +free_ctx: + kfree(hdev->kernel_ctx); +sw_fini: + hdev->asic_funcs->sw_fini(hdev); early_fini: device_early_fini(hdev); release_device: @@ -283,6 +326,10 @@ void hl_device_fini(struct hl_device *hdev) /* Mark device as disabled */ hdev->disabled = true; + /* Release kernel context */ + if ((hdev->kernel_ctx) && (hl_ctx_put(hdev->kernel_ctx) != 1)) + dev_err(hdev->dev, "kernel ctx is still alive\n"); + /* Call ASIC S/W finalize function */ hdev->asic_funcs->sw_fini(hdev); diff --git a/drivers/misc/habanalabs/habanalabs.h b/drivers/misc/habanalabs/habanalabs.h index 644624fa378b..9f76373b3e2c 100644 --- a/drivers/misc/habanalabs/habanalabs.h +++ b/drivers/misc/habanalabs/habanalabs.h @@ -77,6 +77,8 @@ struct asic_fixed_properties { #define HL_QUEUE_LENGTH 256 + + /* * ASICs */ @@ -124,6 +126,39 @@ struct hl_asic_funcs { void *cpu_addr, dma_addr_t dma_handle); }; + +/* + * CONTEXTS + */ + +#define HL_KERNEL_ASID_ID 0 + +/** + * struct hl_ctx - user/kernel context. + * @hpriv: pointer to the private (KMD) data of the process (fd). + * @hdev: pointer to the device structure. + * @refcount: reference counter for the context. Context is released only when + * this hits 0l. It is incremented on CS and CS_WAIT. + * @asid: context's unique address space ID in the device's MMU. + */ +struct hl_ctx { + struct hl_fpriv *hpriv; + struct hl_device *hdev; + struct kref refcount; + u32 asid; +}; + +/** + * struct hl_ctx_mgr - for handling multiple contexts. + * @ctx_lock: protects ctx_handles. + * @ctx_handles: idr to hold all ctx handles. + */ +struct hl_ctx_mgr { + struct mutex ctx_lock; + struct idr ctx_handles; +}; + + /* * FILE PRIVATE STRUCTURE */ @@ -133,12 +168,16 @@ struct hl_asic_funcs { * @hdev: habanalabs device structure. * @filp: pointer to the given file structure. * @taskpid: current process ID. + * @ctx: current executing context. + * @ctx_mgr: context manager to handle multiple context for this FD. * @refcount: number of related contexts. */ struct hl_fpriv { struct hl_device *hdev; struct file *filp; struct pid *taskpid; + struct hl_ctx *ctx; /* TODO: remove for multiple ctx */ + struct hl_ctx_mgr ctx_mgr; struct kref refcount; }; @@ -192,13 +231,24 @@ void hl_wreg(struct hl_device *hdev, u32 reg, u32 val); * @dev: realted kernel basic device structure. * @asic_name: ASIC specific nmae. * @asic_type: ASIC specific type. + * @kernel_ctx: KMD context structure. * @dma_pool: DMA pool for small allocations. * @cpu_accessible_dma_mem: KMD <-> ArmCP shared memory CPU address. * @cpu_accessible_dma_address: KMD <-> ArmCP shared memory DMA address. * @cpu_accessible_dma_pool: KMD <-> ArmCP shared memory pool. + * @asid_bitmap: holds used/available ASIDs. + * @asid_mutex: protects asid_bitmap. + * @fd_open_cnt_lock: lock for updating fd_open_cnt in hl_device_open. Although + * fd_open_cnt is atomic, we need this lock to serialize + * the open function because the driver currently supports + * only a single process at a time. In addition, we need a + * lock here so we can flush user processes which are opening + * the device while we are trying to hard reset it * @asic_prop: ASIC specific immutable properties. * @asic_funcs: ASIC specific functions. * @asic_specific: ASIC specific information to use only from ASIC files. + * @user_ctx: current user context executing. + * @fd_open_cnt: number of open user processes. * @major: habanalabs KMD major. * @id: device minor. * @disabled: is device disabled. @@ -211,13 +261,21 @@ struct hl_device { struct device *dev; char asic_name[16]; enum hl_asic_type asic_type; + struct hl_ctx *kernel_ctx; struct dma_pool *dma_pool; void *cpu_accessible_dma_mem; dma_addr_t cpu_accessible_dma_address; struct gen_pool *cpu_accessible_dma_pool; + unsigned long *asid_bitmap; + struct mutex asid_mutex; + /* TODO: remove fd_open_cnt_lock for multiple process support */ + struct mutex fd_open_cnt_lock; struct asic_fixed_properties asic_prop; const struct hl_asic_funcs *asic_funcs; void *asic_specific; + /* TODO: remove user_ctx for multiple process support */ + struct hl_ctx *user_ctx; + atomic_t fd_open_cnt; u32 major; u16 id; u8 disabled; @@ -265,10 +323,23 @@ int hl_poll_timeout_memory(struct hl_device *hdev, u64 addr, u32 timeout_us, int hl_poll_timeout_device_memory(struct hl_device *hdev, void __iomem *addr, u32 timeout_us, u32 *val); +int hl_asid_init(struct hl_device *hdev); +void hl_asid_fini(struct hl_device *hdev); +unsigned long hl_asid_alloc(struct hl_device *hdev); +void hl_asid_free(struct hl_device *hdev, unsigned long asid); + +int hl_ctx_create(struct hl_device *hdev, struct hl_fpriv *hpriv); +void hl_ctx_free(struct hl_device *hdev, struct hl_ctx *ctx); +int hl_ctx_init(struct hl_device *hdev, struct hl_ctx *ctx, bool is_kernel_ctx); +int hl_ctx_put(struct hl_ctx *ctx); +void hl_ctx_mgr_init(struct hl_ctx_mgr *mgr); +void hl_ctx_mgr_fini(struct hl_device *hdev, struct hl_ctx_mgr *mgr); int hl_device_init(struct hl_device *hdev, struct class *hclass); void hl_device_fini(struct hl_device *hdev); int hl_device_suspend(struct hl_device *hdev); int hl_device_resume(struct hl_device *hdev); +void hl_hpriv_get(struct hl_fpriv *hpriv); +void hl_hpriv_put(struct hl_fpriv *hpriv); void goya_set_asic_funcs(struct hl_device *hdev); diff --git a/drivers/misc/habanalabs/habanalabs_drv.c b/drivers/misc/habanalabs/habanalabs_drv.c index 1596cf38f6c0..8ecb51fbb133 100644 --- a/drivers/misc/habanalabs/habanalabs_drv.c +++ b/drivers/misc/habanalabs/habanalabs_drv.c @@ -74,6 +74,7 @@ int hl_device_open(struct inode *inode, struct file *filp) { struct hl_device *hdev; struct hl_fpriv *hpriv; + int rc; mutex_lock(&hl_devs_idr_lock); hdev = idr_find(&hl_devs_idr, iminor(inode)); @@ -85,9 +86,33 @@ int hl_device_open(struct inode *inode, struct file *filp) return -ENXIO; } + mutex_lock(&hdev->fd_open_cnt_lock); + + if (hdev->disabled) { + dev_err_ratelimited(hdev->dev, + "Can't open %s because it is disabled\n", + dev_name(hdev->dev)); + mutex_unlock(&hdev->fd_open_cnt_lock); + return -EPERM; + } + + if (atomic_read(&hdev->fd_open_cnt)) { + dev_info_ratelimited(hdev->dev, + "Device %s is already attached to application\n", + dev_name(hdev->dev)); + mutex_unlock(&hdev->fd_open_cnt_lock); + return -EBUSY; + } + + atomic_inc(&hdev->fd_open_cnt); + + mutex_unlock(&hdev->fd_open_cnt_lock); + hpriv = kzalloc(sizeof(*hpriv), GFP_KERNEL); - if (!hpriv) - return -ENOMEM; + if (!hpriv) { + rc = -ENOMEM; + goto close_device; + } hpriv->hdev = hdev; filp->private_data = hpriv; @@ -95,9 +120,26 @@ int hl_device_open(struct inode *inode, struct file *filp) kref_init(&hpriv->refcount); nonseekable_open(inode, filp); + hl_ctx_mgr_init(&hpriv->ctx_mgr); + + rc = hl_ctx_create(hdev, hpriv); + if (rc) { + dev_err(hdev->dev, "Failed to open FD (CTX fail)\n"); + goto out_err; + } + hpriv->taskpid = find_get_pid(current->pid); return 0; + +out_err: + filp->private_data = NULL; + hl_ctx_mgr_fini(hpriv->hdev, &hpriv->ctx_mgr); + kfree(hpriv); + +close_device: + atomic_dec(&hdev->fd_open_cnt); + return rc; } /* -- 2.17.1