From mboxrd@z Thu Jan 1 00:00:00 1970 From: Adrien Mazarguil Subject: [PATCH v1 4/4] net/mlx: make rdma-core glue path configurable Date: Fri, 2 Feb 2018 16:16:41 +0100 Message-ID: <20180202144736.8239-5-adrien.mazarguil@6wind.com> References: <20180202144736.8239-1-adrien.mazarguil@6wind.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Nelio Laranjeiro , dev@dpdk.org, Marcelo Ricardo Leitner To: Shahaf Shuler Return-path: Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by dpdk.org (Postfix) with ESMTP id 9A6EB1B1AC for ; Fri, 2 Feb 2018 16:16:54 +0100 (CET) Received: by mail-wm0-f67.google.com with SMTP id v123so13069638wmd.5 for ; Fri, 02 Feb 2018 07:16:54 -0800 (PST) Content-Disposition: inline In-Reply-To: <20180202144736.8239-1-adrien.mazarguil@6wind.com> List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Since rdma-core glue libraries are intrinsically tied to their respective PMDs and used as internal plug-ins, their presence in the default search path among other system libraries for the dynamic linker is not necessarily desired. This commit enables their installation and subsequent look-up at run time in RTE_EAL_PMD_PATH if configured to a nonempty string. This path can also be overridden by environment variables MLX[45]_GLUE_PATH. Signed-off-by: Adrien Mazarguil --- doc/guides/nics/mlx4.rst | 17 +++++++++++++++++ doc/guides/nics/mlx5.rst | 14 ++++++++++++++ drivers/net/mlx4/mlx4.c | 43 ++++++++++++++++++++++++++++++++++++++++++- drivers/net/mlx5/mlx5.c | 43 ++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 115 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/mlx4.rst b/doc/guides/nics/mlx4.rst index 88161781c..9e4fbf692 100644 --- a/doc/guides/nics/mlx4.rst +++ b/doc/guides/nics/mlx4.rst @@ -97,6 +97,11 @@ These options can be modified in the ``.config`` file. ``CONFIG_RTE_BUILD_SHARED_LIB`` disabled) and they won't show up as missing with ``ldd(1)``. + It works by moving these dependencies to a purpose-built rdma-core "glue" + plug-in, which must either be installed in ``CONFIG_RTE_EAL_PMD_PATH`` if + set, or in a standard location for the dynamic linker (e.g. ``/lib``) if + left to the default empty string (``""``). + This option has no performance impact. - ``CONFIG_RTE_LIBRTE_MLX4_DEBUG`` (default **n**) @@ -113,6 +118,18 @@ These options can be modified in the ``.config`` file. This value is always 1 for RX queues since they use a single MP. +Environment variables +~~~~~~~~~~~~~~~~~~~~~ + +- ``MLX4_GLUE_PATH`` + + A list of directories in which to search for the rdma-core "glue" plug-in, + separated by colons or semi-colons. + + Only matters when compiled with ``CONFIG_RTE_LIBRTE_MLX4_DLOPEN_DEPS`` + enabled and most useful when ``CONFIG_RTE_EAL_PMD_PATH`` is also set, + since ``LD_LIBRARY_PATH`` has no effect in this case. + Run-time configuration ~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index a9e4bf51a..1635dff2b 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -170,6 +170,11 @@ These options can be modified in the ``.config`` file. ``CONFIG_RTE_BUILD_SHARED_LIB`` disabled) and they won't show up as missing with ``ldd(1)``. + It works by moving these dependencies to a purpose-built rdma-core "glue" + plug-in, which must either be installed in ``CONFIG_RTE_EAL_PMD_PATH`` if + set, or in a standard location for the dynamic linker (e.g. ``/lib``) if + left to the default empty string (``""``). + This option has no performance impact. - ``CONFIG_RTE_LIBRTE_MLX5_DEBUG`` (default **n**) @@ -189,6 +194,15 @@ These options can be modified in the ``.config`` file. Environment variables ~~~~~~~~~~~~~~~~~~~~~ +- ``MLX5_GLUE_PATH`` + + A list of directories in which to search for the rdma-core "glue" plug-in, + separated by colons or semi-colons. + + Only matters when compiled with ``CONFIG_RTE_LIBRTE_MLX5_DLOPEN_DEPS`` + enabled and most useful when ``CONFIG_RTE_EAL_PMD_PATH`` is also set, + since ``LD_LIBRARY_PATH`` has no effect in this case. + - ``MLX5_PMD_ENABLE_PADDING`` Enables HW packet padding in PCI bus transactions. diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c index 61a852fb9..4266cb1bb 100644 --- a/drivers/net/mlx4/mlx4.c +++ b/drivers/net/mlx4/mlx4.c @@ -741,11 +741,52 @@ static struct rte_pci_driver mlx4_driver = { static int mlx4_glue_init(void) { + const char *path[] = { + /* + * A basic security check is necessary before trusting + * MLX4_GLUE_PATH, which may override RTE_EAL_PMD_PATH. + */ + (geteuid() == getuid() && getegid() == getgid() ? + getenv("MLX4_GLUE_PATH") : NULL), + RTE_EAL_PMD_PATH, + }; + unsigned int i = 0; void *handle = NULL; void **sym; const char *dlmsg; - handle = dlopen(MLX4_GLUE, RTLD_LAZY); + while (!handle && i != RTE_DIM(path)) { + const char *end; + size_t len; + int ret; + + if (!path[i]) { + ++i; + continue; + } + end = strpbrk(path[i], ":;"); + if (!end) + end = path[i] + strlen(path[i]); + len = end - path[i]; + ret = 0; + do { + char name[ret + 1]; + + ret = snprintf(name, ret, "%.*s%s" MLX4_GLUE "\n", + (int)len, path[i], + (!len || *(end - 1) == '/') ? "" : "/"); + if (ret == -1) + break; + if (sizeof(name) != (size_t)ret + 1) + continue; + DEBUG("looking for rdma-core glue as \"%s\"", name); + handle = dlopen(name, RTLD_LAZY); + break; + } while (1); + path[i] = end + 1; + if (!*end) + ++i; + } if (!handle) { rte_errno = EINVAL; dlmsg = dlerror(); diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 341230d2b..caa60339e 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1085,11 +1085,52 @@ static struct rte_pci_driver mlx5_driver = { static int mlx5_glue_init(void) { + const char *path[] = { + /* + * A basic security check is necessary before trusting + * MLX5_GLUE_PATH, which may override RTE_EAL_PMD_PATH. + */ + (geteuid() == getuid() && getegid() == getgid() ? + getenv("MLX5_GLUE_PATH") : NULL), + RTE_EAL_PMD_PATH, + }; + unsigned int i = 0; void *handle = NULL; void **sym; const char *dlmsg; - handle = dlopen(MLX5_GLUE, RTLD_LAZY); + while (!handle && i != RTE_DIM(path)) { + const char *end; + size_t len; + int ret; + + if (!path[i]) { + ++i; + continue; + } + end = strpbrk(path[i], ":;"); + if (!end) + end = path[i] + strlen(path[i]); + len = end - path[i]; + ret = 0; + do { + char name[ret + 1]; + + ret = snprintf(name, ret, "%.*s%s" MLX5_GLUE "\n", + (int)len, path[i], + (!len || *(end - 1) == '/') ? "" : "/"); + if (ret == -1) + break; + if (sizeof(name) != (size_t)ret + 1) + continue; + DEBUG("looking for rdma-core glue as \"%s\"", name); + handle = dlopen(name, RTLD_LAZY); + break; + } while (1); + path[i] = end + 1; + if (!*end) + ++i; + } if (!handle) { rte_errno = EINVAL; dlmsg = dlerror(); -- 2.11.0