* [ndctl PATCH v5 01/16] ndctl: add .clang-format
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 02/16] cxl: add a cxl utility and libcxl library Vishal Verma
` (14 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Copy the Linux kernel's .clang-format and modify it for ndctl. Only the
'ForEachMacros' section has been modified from the original kernel copy.
Cc: Dan Williams <dan.j.williams@intel.com>
Reported-by: Ben Widawsky <ben.widawsky@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
.clang-format | 161 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 161 insertions(+)
create mode 100644 .clang-format
diff --git a/.clang-format b/.clang-format
new file mode 100644
index 0000000..4e00fff
--- /dev/null
+++ b/.clang-format
@@ -0,0 +1,161 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# clang-format configuration file. Intended for clang-format >= 4.
+# Copied from Linux's .clang-format
+#
+# For more information, see:
+#
+# https://clang.llvm.org/docs/ClangFormat.html
+# https://clang.llvm.org/docs/ClangFormatStyleOptions.html
+#
+---
+AccessModifierOffset: -4
+AlignAfterOpenBracket: Align
+AlignConsecutiveAssignments: false
+AlignConsecutiveDeclarations: false
+#AlignEscapedNewlines: Left # Unknown to clang-format-4.0
+AlignOperands: true
+AlignTrailingComments: false
+AllowAllParametersOfDeclarationOnNextLine: false
+AllowShortBlocksOnASingleLine: false
+AllowShortCaseLabelsOnASingleLine: false
+AllowShortFunctionsOnASingleLine: None
+AllowShortIfStatementsOnASingleLine: false
+AllowShortLoopsOnASingleLine: false
+AlwaysBreakAfterDefinitionReturnType: None
+AlwaysBreakAfterReturnType: None
+AlwaysBreakBeforeMultilineStrings: false
+AlwaysBreakTemplateDeclarations: false
+BinPackArguments: true
+BinPackParameters: true
+BraceWrapping:
+ AfterClass: false
+ AfterControlStatement: false
+ AfterEnum: false
+ AfterFunction: true
+ AfterNamespace: true
+ AfterObjCDeclaration: false
+ AfterStruct: false
+ AfterUnion: false
+ #AfterExternBlock: false # Unknown to clang-format-5.0
+ BeforeCatch: false
+ BeforeElse: false
+ IndentBraces: false
+ #SplitEmptyFunction: true # Unknown to clang-format-4.0
+ #SplitEmptyRecord: true # Unknown to clang-format-4.0
+ #SplitEmptyNamespace: true # Unknown to clang-format-4.0
+BreakBeforeBinaryOperators: None
+BreakBeforeBraces: Custom
+#BreakBeforeInheritanceComma: false # Unknown to clang-format-4.0
+BreakBeforeTernaryOperators: false
+BreakConstructorInitializersBeforeComma: false
+#BreakConstructorInitializers: BeforeComma # Unknown to clang-format-4.0
+BreakAfterJavaFieldAnnotations: false
+BreakStringLiterals: false
+ColumnLimit: 80
+CommentPragmas: '^ IWYU pragma:'
+#CompactNamespaces: false # Unknown to clang-format-4.0
+ConstructorInitializerAllOnOneLineOrOnePerLine: false
+ConstructorInitializerIndentWidth: 8
+ContinuationIndentWidth: 8
+Cpp11BracedListStyle: false
+DerivePointerAlignment: false
+DisableFormat: false
+ExperimentalAutoDetectBinPacking: false
+#FixNamespaceComments: false # Unknown to clang-format-4.0
+
+# Taken from:
+# while read -r sym; do
+# printf " - '%s'\n" "$sym";
+# done < \
+# <(cscope -dL6 "foreach|for_each" \
+# | awk '{ print $4 $5 }' | grep -E 'foreach|for_each' \
+# | sed -e 's/#define//' \
+# -e 's/*//' \
+# -e 's/://' \
+# -e 's/\(.*for_each.*\)(.*/\1/' \
+# -e 's/\(.*foreach.*\)(.*/\1/' \
+# | sort -u)
+ForEachMacros:
+ - 'daxctl_dev_foreach'
+ - 'daxctl_mapping_foreach'
+ - 'daxctl_region_foreach'
+ - 'kmod_list_foreach'
+ - 'kmod_list_foreach_reverse'
+ - 'list_for_each'
+ - 'list_for_each_off'
+ - 'list_for_each_rev'
+ - 'list_for_each_safe'
+ - 'list_for_each_safe_off'
+ - 'ndctl_btt_foreach'
+ - 'ndctl_btt_foreach_safe'
+ - 'ndctl_bus_foreach'
+ - 'ndctl_dax_foreach'
+ - 'ndctl_dax_foreach_safe'
+ - 'ndctl_dimm_foreach'
+ - 'ndctl_dimm_foreach_in_interleave_set'
+ - 'ndctl_dimm_foreach_in_region'
+ - 'ndctl_interleave_set_foreach'
+ - 'ndctl_mapping_foreach'
+ - 'ndctl_namespace_badblock_foreach'
+ - 'ndctl_namespace_bb_foreach'
+ - 'ndctl_namespace_foreach'
+ - 'ndctl_namespace_foreach_safe'
+ - 'ndctl_pfn_foreach'
+ - 'ndctl_pfn_foreach_safe'
+ - 'ndctl_region_badblock_foreach'
+ - 'ndctl_region_foreach'
+ - 'udev_list_entry_foreach'
+
+#IncludeBlocks: Preserve # Unknown to clang-format-5.0
+IncludeCategories:
+ - Regex: '.*'
+ Priority: 1
+IncludeIsMainRegex: '(Test)?$'
+IndentCaseLabels: false
+#IndentPPDirectives: None # Unknown to clang-format-5.0
+IndentWidth: 8
+IndentWrappedFunctionNames: false
+JavaScriptQuotes: Leave
+JavaScriptWrapImports: true
+KeepEmptyLinesAtTheStartOfBlocks: false
+MacroBlockBegin: ''
+MacroBlockEnd: ''
+MaxEmptyLinesToKeep: 1
+NamespaceIndentation: None
+#ObjCBinPackProtocolList: Auto # Unknown to clang-format-5.0
+ObjCBlockIndentWidth: 8
+ObjCSpaceAfterProperty: true
+ObjCSpaceBeforeProtocolList: true
+
+# Taken from git's rules
+#PenaltyBreakAssignment: 10 # Unknown to clang-format-4.0
+PenaltyBreakBeforeFirstCallParameter: 30
+PenaltyBreakComment: 10
+PenaltyBreakFirstLessLess: 0
+PenaltyBreakString: 10
+PenaltyExcessCharacter: 100
+PenaltyReturnTypeOnItsOwnLine: 60
+
+PointerAlignment: Right
+ReflowComments: false
+SortIncludes: false
+#SortUsingDeclarations: false # Unknown to clang-format-4.0
+SpaceAfterCStyleCast: false
+SpaceAfterTemplateKeyword: true
+SpaceBeforeAssignmentOperators: true
+#SpaceBeforeCtorInitializerColon: true # Unknown to clang-format-5.0
+#SpaceBeforeInheritanceColon: true # Unknown to clang-format-5.0
+SpaceBeforeParens: ControlStatements
+#SpaceBeforeRangeBasedForLoopColon: true # Unknown to clang-format-5.0
+SpaceInEmptyParentheses: false
+SpacesBeforeTrailingComments: 1
+SpacesInAngles: false
+SpacesInContainerLiterals: false
+SpacesInCStyleCastParentheses: false
+SpacesInParentheses: false
+SpacesInSquareBrackets: false
+Standard: Cpp03
+TabWidth: 8
+UseTab: Always
+...
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* [ndctl PATCH v5 02/16] cxl: add a cxl utility and libcxl library
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 01/16] ndctl: add .clang-format Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 21:59 ` [ndctl PATCH v6 " Vishal Verma
2021-12-08 5:12 ` [ndctl PATCH v5 " Dan Williams
2021-11-11 20:44 ` [ndctl PATCH v5 03/16] cxl: add a local copy of the cxl_mem UAPI header Vishal Verma
` (13 subsequent siblings)
15 siblings, 2 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
CXL - or Compute eXpress Link - is a new interconnect that extends PCIe
to support a wide range of devices, including cache coherent memory
expanders. As such, these devices can be new sources of 'persistent
memory', and the 'ndctl' umbrella of tools and libraries needs to be able
to interact with them.
Add a new utility and library for managing these CXL memory devices. This
is an initial bring-up for interacting with CXL devices, and only includes
adding the utility and library infrastructure, parsing device information
from sysfs for CXL devices, and providing a 'cxl-list' command to
display this information in JSON formatted output.
Cc: Ben Widawsky <ben.widawsky@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
Documentation/cxl/cxl-list.txt | 64 +++++
Documentation/cxl/cxl.txt | 34 +++
Documentation/cxl/human-option.txt | 8 +
Documentation/cxl/verbose-option.txt | 5 +
configure.ac | 3 +
Makefile.am | 8 +-
Makefile.am.in | 4 +
cxl/lib/private.h | 29 +++
cxl/lib/libcxl.c | 345 +++++++++++++++++++++++++++
cxl/builtin.h | 8 +
cxl/libcxl.h | 55 +++++
util/filter.h | 2 +
util/json.h | 3 +
util/main.h | 3 +
cxl/cxl.c | 96 ++++++++
cxl/list.c | 113 +++++++++
util/filter.c | 20 ++
util/json.c | 26 ++
.clang-format | 1 +
.gitignore | 4 +-
Documentation/cxl/Makefile.am | 58 +++++
cxl/Makefile.am | 21 ++
cxl/lib/Makefile.am | 32 +++
cxl/lib/libcxl.pc.in | 11 +
cxl/lib/libcxl.sym | 25 ++
25 files changed, 974 insertions(+), 4 deletions(-)
create mode 100644 Documentation/cxl/cxl-list.txt
create mode 100644 Documentation/cxl/cxl.txt
create mode 100644 Documentation/cxl/human-option.txt
create mode 100644 Documentation/cxl/verbose-option.txt
create mode 100644 cxl/lib/private.h
create mode 100644 cxl/lib/libcxl.c
create mode 100644 cxl/builtin.h
create mode 100644 cxl/libcxl.h
create mode 100644 cxl/cxl.c
create mode 100644 cxl/list.c
create mode 100644 Documentation/cxl/Makefile.am
create mode 100644 cxl/Makefile.am
create mode 100644 cxl/lib/Makefile.am
create mode 100644 cxl/lib/libcxl.pc.in
create mode 100644 cxl/lib/libcxl.sym
diff --git a/Documentation/cxl/cxl-list.txt b/Documentation/cxl/cxl-list.txt
new file mode 100644
index 0000000..bd377b3
--- /dev/null
+++ b/Documentation/cxl/cxl-list.txt
@@ -0,0 +1,64 @@
+// SPDX-License-Identifier: GPL-2.0
+
+cxl-list(1)
+===========
+
+NAME
+----
+cxl-list - List CXL capable memory devices, and their attributes in json.
+
+SYNOPSIS
+--------
+[verse]
+'cxl list' [<options>]
+
+Walk the CXL capable device hierarchy in the system and list all device
+instances along with some of their major attributes.
+
+Options can be specified to limit the output to specific memdevs.
+By default, 'cxl list' with no options is equivalent to:
+[verse]
+cxl list --memdevs
+
+EXAMPLE
+-------
+----
+# cxl list --memdevs
+{
+ "memdev":"mem0",
+ "pmem_size":268435456,
+ "ram_size":0,
+}
+----
+
+OPTIONS
+-------
+-d::
+--memdev=::
+ Specify a cxl memory device name to filter the listing. For example:
+----
+# cxl list --memdev=mem0
+{
+ "memdev":"mem0",
+ "pmem_size":268435456,
+ "ram_size":0,
+}
+----
+
+-D::
+--memdevs::
+ Include all CXL memory devices in the listing
+
+-i::
+--idle::
+ Include idle (not enabled / zero-sized) devices in the listing
+
+include::human-option.txt[]
+
+include::verbose-option.txt[]
+
+include::../copyright.txt[]
+
+SEE ALSO
+--------
+linkcxl:ndctl-list[1]
diff --git a/Documentation/cxl/cxl.txt b/Documentation/cxl/cxl.txt
new file mode 100644
index 0000000..41a51c7
--- /dev/null
+++ b/Documentation/cxl/cxl.txt
@@ -0,0 +1,34 @@
+// SPDX-License-Identifier: GPL-2.0
+
+cxl(1)
+======
+
+NAME
+----
+cxl - Provides enumeration and provisioning commands for CXL platforms
+
+SYNOPSIS
+--------
+[verse]
+'cxl' [--version] [--help] COMMAND [ARGS]
+
+OPTIONS
+-------
+-v::
+--version::
+ Display the version of the 'cxl' utility.
+
+-h::
+--help::
+ Run the 'cxl help' command.
+
+DESCRIPTION
+-----------
+The cxl utility provides enumeration and provisioning commands for
+the CXL devices managed by the Linux kernel.
+
+include::../copyright.txt[]
+
+SEE ALSO
+--------
+linkcxl:ndctl[1]
diff --git a/Documentation/cxl/human-option.txt b/Documentation/cxl/human-option.txt
new file mode 100644
index 0000000..2f4de7a
--- /dev/null
+++ b/Documentation/cxl/human-option.txt
@@ -0,0 +1,8 @@
+// SPDX-License-Identifier: GPL-2.0
+
+-u::
+--human::
+ By default the command will output machine-friendly raw-integer
+ data. Instead, with this flag, numbers representing storage size
+ will be formatted as human readable strings with units, other
+ fields are converted to hexadecimal strings.
diff --git a/Documentation/cxl/verbose-option.txt b/Documentation/cxl/verbose-option.txt
new file mode 100644
index 0000000..cb62c8e
--- /dev/null
+++ b/Documentation/cxl/verbose-option.txt
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+-v::
+--verbose::
+ Emit more debug messages
diff --git a/configure.ac b/configure.ac
index dc39dbe..dadae0a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -222,12 +222,15 @@ AC_CONFIG_HEADERS(config.h)
AC_CONFIG_FILES([
Makefile
daxctl/lib/Makefile
+ cxl/lib/Makefile
ndctl/lib/Makefile
ndctl/Makefile
daxctl/Makefile
+ cxl/Makefile
test/Makefile
Documentation/ndctl/Makefile
Documentation/daxctl/Makefile
+ Documentation/cxl/Makefile
])
AC_OUTPUT
diff --git a/Makefile.am b/Makefile.am
index 60a1998..428fd40 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,9 +1,9 @@
include Makefile.am.in
ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS}
-SUBDIRS = . daxctl/lib ndctl/lib ndctl daxctl
+SUBDIRS = . cxl/lib daxctl/lib ndctl/lib cxl ndctl daxctl
if ENABLE_DOCS
-SUBDIRS += Documentation/ndctl Documentation/daxctl
+SUBDIRS += Documentation/ndctl Documentation/daxctl Documentation/cxl
endif
SUBDIRS += test
@@ -87,4 +87,6 @@ libutil_a_SOURCES = \
util/filter.h \
util/bitmap.h
-nobase_include_HEADERS = daxctl/libdaxctl.h
+nobase_include_HEADERS = \
+ daxctl/libdaxctl.h \
+ cxl/libcxl.h
diff --git a/Makefile.am.in b/Makefile.am.in
index bdceda9..aaeee53 100644
--- a/Makefile.am.in
+++ b/Makefile.am.in
@@ -42,3 +42,7 @@ LIBNDCTL_AGE=19
LIBDAXCTL_CURRENT=6
LIBDAXCTL_REVISION=0
LIBDAXCTL_AGE=5
+
+LIBCXL_CURRENT=1
+LIBCXL_REVISION=0
+LIBCXL_AGE=0
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
new file mode 100644
index 0000000..fc88fa1
--- /dev/null
+++ b/cxl/lib/private.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+/* Copyright (C) 2020-2021, Intel Corporation. All rights reserved. */
+#ifndef _LIBCXL_PRIVATE_H_
+#define _LIBCXL_PRIVATE_H_
+
+#include <libkmod.h>
+
+#define CXL_EXPORT __attribute__ ((visibility("default")))
+
+struct cxl_memdev {
+ int id, major, minor;
+ void *dev_buf;
+ size_t buf_len;
+ char *dev_path;
+ char *firmware_version;
+ struct cxl_ctx *ctx;
+ struct list_node list;
+ unsigned long long pmem_size;
+ unsigned long long ram_size;
+ int payload_max;
+ struct kmod_module *module;
+};
+
+static inline int check_kmod(struct kmod_ctx *kmod_ctx)
+{
+ return kmod_ctx ? 0 : -ENXIO;
+}
+
+#endif /* _LIBCXL_PRIVATE_H_ */
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
new file mode 100644
index 0000000..c15e987
--- /dev/null
+++ b/cxl/lib/libcxl.c
@@ -0,0 +1,345 @@
+// SPDX-License-Identifier: LGPL-2.1
+// Copyright (C) 2020-2021, Intel Corporation. All rights reserved.
+#include <stdio.h>
+#include <errno.h>
+#include <limits.h>
+#include <libgen.h>
+#include <stdlib.h>
+#include <dirent.h>
+#include <unistd.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/sysmacros.h>
+#include <uuid/uuid.h>
+#include <ccan/list/list.h>
+#include <ccan/array_size/array_size.h>
+
+#include <util/log.h>
+#include <util/sysfs.h>
+#include <util/bitmap.h>
+#include <cxl/libcxl.h>
+#include "private.h"
+
+/**
+ * struct cxl_ctx - library user context to find "nd" instances
+ *
+ * Instantiate with cxl_new(), which takes an initial reference. Free
+ * the context by dropping the reference count to zero with
+ * cxl_unref(), or take additional references with cxl_ref()
+ * @timeout: default library timeout in milliseconds
+ */
+struct cxl_ctx {
+ /* log_ctx must be first member for cxl_set_log_fn compat */
+ struct log_ctx ctx;
+ int refcount;
+ void *userdata;
+ int memdevs_init;
+ struct list_head memdevs;
+ struct kmod_ctx *kmod_ctx;
+ void *private_data;
+};
+
+static void free_memdev(struct cxl_memdev *memdev, struct list_head *head)
+{
+ if (head)
+ list_del_from(head, &memdev->list);
+ kmod_module_unref(memdev->module);
+ free(memdev->firmware_version);
+ free(memdev->dev_buf);
+ free(memdev->dev_path);
+ free(memdev);
+}
+
+/**
+ * cxl_get_userdata - retrieve stored data pointer from library context
+ * @ctx: cxl library context
+ *
+ * This might be useful to access from callbacks like a custom logging
+ * function.
+ */
+CXL_EXPORT void *cxl_get_userdata(struct cxl_ctx *ctx)
+{
+ if (ctx == NULL)
+ return NULL;
+ return ctx->userdata;
+}
+
+/**
+ * cxl_set_userdata - store custom @userdata in the library context
+ * @ctx: cxl library context
+ * @userdata: data pointer
+ */
+CXL_EXPORT void cxl_set_userdata(struct cxl_ctx *ctx, void *userdata)
+{
+ if (ctx == NULL)
+ return;
+ ctx->userdata = userdata;
+}
+
+CXL_EXPORT void cxl_set_private_data(struct cxl_ctx *ctx, void *data)
+{
+ ctx->private_data = data;
+}
+
+CXL_EXPORT void *cxl_get_private_data(struct cxl_ctx *ctx)
+{
+ return ctx->private_data;
+}
+
+/**
+ * cxl_new - instantiate a new library context
+ * @ctx: context to establish
+ *
+ * Returns zero on success and stores an opaque pointer in ctx. The
+ * context is freed by cxl_unref(), i.e. cxl_new() implies an
+ * internal cxl_ref().
+ */
+CXL_EXPORT int cxl_new(struct cxl_ctx **ctx)
+{
+ struct kmod_ctx *kmod_ctx;
+ struct cxl_ctx *c;
+ int rc = 0;
+
+ c = calloc(1, sizeof(struct cxl_ctx));
+ if (!c)
+ return -ENOMEM;
+
+ kmod_ctx = kmod_new(NULL, NULL);
+ if (check_kmod(kmod_ctx) != 0) {
+ rc = -ENXIO;
+ goto out;
+ }
+
+ c->refcount = 1;
+ log_init(&c->ctx, "libcxl", "CXL_LOG");
+ info(c, "ctx %p created\n", c);
+ dbg(c, "log_priority=%d\n", c->ctx.log_priority);
+ *ctx = c;
+ list_head_init(&c->memdevs);
+ c->kmod_ctx = kmod_ctx;
+
+ return 0;
+out:
+ free(c);
+ return rc;
+}
+
+/**
+ * cxl_ref - take an additional reference on the context
+ * @ctx: context established by cxl_new()
+ */
+CXL_EXPORT struct cxl_ctx *cxl_ref(struct cxl_ctx *ctx)
+{
+ if (ctx == NULL)
+ return NULL;
+ ctx->refcount++;
+ return ctx;
+}
+
+/**
+ * cxl_unref - drop a context reference count
+ * @ctx: context established by cxl_new()
+ *
+ * Drop a reference and if the resulting reference count is 0 destroy
+ * the context.
+ */
+CXL_EXPORT void cxl_unref(struct cxl_ctx *ctx)
+{
+ struct cxl_memdev *memdev, *_d;
+
+ if (ctx == NULL)
+ return;
+ ctx->refcount--;
+ if (ctx->refcount > 0)
+ return;
+
+ list_for_each_safe(&ctx->memdevs, memdev, _d, list)
+ free_memdev(memdev, &ctx->memdevs);
+
+ kmod_unref(ctx->kmod_ctx);
+ info(ctx, "context %p released\n", ctx);
+ free(ctx);
+}
+
+/**
+ * cxl_set_log_fn - override default log routine
+ * @ctx: cxl library context
+ * @log_fn: function to be called for logging messages
+ *
+ * The built-in logging writes to stderr. It can be overridden by a
+ * custom function, to plug log messages into the user's logging
+ * functionality.
+ */
+CXL_EXPORT void cxl_set_log_fn(struct cxl_ctx *ctx,
+ void (*cxl_log_fn)(struct cxl_ctx *ctx, int priority,
+ const char *file, int line, const char *fn,
+ const char *format, va_list args))
+{
+ ctx->ctx.log_fn = (log_fn) cxl_log_fn;
+ info(ctx, "custom logging function %p registered\n", cxl_log_fn);
+}
+
+/**
+ * cxl_get_log_priority - retrieve current library loglevel (syslog)
+ * @ctx: cxl library context
+ */
+CXL_EXPORT int cxl_get_log_priority(struct cxl_ctx *ctx)
+{
+ return ctx->ctx.log_priority;
+}
+
+/**
+ * cxl_set_log_priority - set log verbosity
+ * @priority: from syslog.h, LOG_ERR, LOG_INFO, LOG_DEBUG
+ *
+ * Note: LOG_DEBUG requires library be built with "configure --enable-debug"
+ */
+CXL_EXPORT void cxl_set_log_priority(struct cxl_ctx *ctx, int priority)
+{
+ ctx->ctx.log_priority = priority;
+}
+
+static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
+{
+ const char *devname = devpath_to_devname(cxlmem_base);
+ char *path = calloc(1, strlen(cxlmem_base) + 100);
+ struct cxl_ctx *ctx = parent;
+ struct cxl_memdev *memdev, *memdev_dup;
+ char buf[SYSFS_ATTR_SIZE];
+ struct stat st;
+
+ if (!path)
+ return NULL;
+ dbg(ctx, "%s: base: \'%s\'\n", devname, cxlmem_base);
+
+ memdev = calloc(1, sizeof(*memdev));
+ if (!memdev)
+ goto err_dev;
+ memdev->id = id;
+ memdev->ctx = ctx;
+
+ sprintf(path, "/dev/cxl/%s", devname);
+ if (stat(path, &st) < 0)
+ goto err_read;
+ memdev->major = major(st.st_rdev);
+ memdev->minor = minor(st.st_rdev);
+
+ sprintf(path, "%s/pmem/size", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+ memdev->pmem_size = strtoull(buf, NULL, 0);
+
+ sprintf(path, "%s/ram/size", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+ memdev->ram_size = strtoull(buf, NULL, 0);
+
+ sprintf(path, "%s/payload_max", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+ memdev->payload_max = strtoull(buf, NULL, 0);
+ if (memdev->payload_max < 0)
+ goto err_read;
+
+ memdev->dev_path = strdup(cxlmem_base);
+ if (!memdev->dev_path)
+ goto err_read;
+
+ sprintf(path, "%s/firmware_version", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+
+ memdev->firmware_version = strdup(buf);
+ if (!memdev->firmware_version)
+ goto err_read;
+
+ memdev->dev_buf = calloc(1, strlen(cxlmem_base) + 50);
+ if (!memdev->dev_buf)
+ goto err_read;
+ memdev->buf_len = strlen(cxlmem_base) + 50;
+
+ cxl_memdev_foreach(ctx, memdev_dup)
+ if (memdev_dup->id == memdev->id) {
+ free_memdev(memdev, NULL);
+ free(path);
+ return memdev_dup;
+ }
+
+ list_add(&ctx->memdevs, &memdev->list);
+ free(path);
+ return memdev;
+
+ err_read:
+ free(memdev->firmware_version);
+ free(memdev->dev_buf);
+ free(memdev->dev_path);
+ free(memdev);
+ err_dev:
+ free(path);
+ return NULL;
+}
+
+static void cxl_memdevs_init(struct cxl_ctx *ctx)
+{
+ if (ctx->memdevs_init)
+ return;
+
+ ctx->memdevs_init = 1;
+
+ sysfs_device_parse(ctx, "/sys/bus/cxl/devices", "mem", ctx,
+ add_cxl_memdev);
+}
+
+CXL_EXPORT struct cxl_ctx *cxl_memdev_get_ctx(struct cxl_memdev *memdev)
+{
+ return memdev->ctx;
+}
+
+CXL_EXPORT struct cxl_memdev *cxl_memdev_get_first(struct cxl_ctx *ctx)
+{
+ cxl_memdevs_init(ctx);
+
+ return list_top(&ctx->memdevs, struct cxl_memdev, list);
+}
+
+CXL_EXPORT struct cxl_memdev *cxl_memdev_get_next(struct cxl_memdev *memdev)
+{
+ struct cxl_ctx *ctx = memdev->ctx;
+
+ return list_next(&ctx->memdevs, memdev, list);
+}
+
+CXL_EXPORT int cxl_memdev_get_id(struct cxl_memdev *memdev)
+{
+ return memdev->id;
+}
+
+CXL_EXPORT const char *cxl_memdev_get_devname(struct cxl_memdev *memdev)
+{
+ return devpath_to_devname(memdev->dev_path);
+}
+
+CXL_EXPORT int cxl_memdev_get_major(struct cxl_memdev *memdev)
+{
+ return memdev->major;
+}
+
+CXL_EXPORT int cxl_memdev_get_minor(struct cxl_memdev *memdev)
+{
+ return memdev->minor;
+}
+
+CXL_EXPORT unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev)
+{
+ return memdev->pmem_size;
+}
+
+CXL_EXPORT unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev)
+{
+ return memdev->ram_size;
+}
+
+CXL_EXPORT const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev)
+{
+ return memdev->firmware_version;
+}
diff --git a/cxl/builtin.h b/cxl/builtin.h
new file mode 100644
index 0000000..3797f98
--- /dev/null
+++ b/cxl/builtin.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
+#ifndef _CXL_BUILTIN_H_
+#define _CXL_BUILTIN_H_
+
+struct cxl_ctx;
+int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx);
+#endif /* _CXL_BUILTIN_H_ */
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
new file mode 100644
index 0000000..fd06790
--- /dev/null
+++ b/cxl/libcxl.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+/* Copyright (C) 2020-2021, Intel Corporation. All rights reserved. */
+#ifndef _LIBCXL_H_
+#define _LIBCXL_H_
+
+#include <stdarg.h>
+#include <unistd.h>
+
+#ifdef HAVE_UUID
+#include <uuid/uuid.h>
+#else
+typedef unsigned char uuid_t[16];
+#endif
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct cxl_ctx;
+struct cxl_ctx *cxl_ref(struct cxl_ctx *ctx);
+void cxl_unref(struct cxl_ctx *ctx);
+int cxl_new(struct cxl_ctx **ctx);
+void cxl_set_log_fn(struct cxl_ctx *ctx,
+ void (*log_fn)(struct cxl_ctx *ctx, int priority,
+ const char *file, int line, const char *fn,
+ const char *format, va_list args));
+int cxl_get_log_priority(struct cxl_ctx *ctx);
+void cxl_set_log_priority(struct cxl_ctx *ctx, int priority);
+void cxl_set_userdata(struct cxl_ctx *ctx, void *userdata);
+void *cxl_get_userdata(struct cxl_ctx *ctx);
+void cxl_set_private_data(struct cxl_ctx *ctx, void *data);
+void *cxl_get_private_data(struct cxl_ctx *ctx);
+
+struct cxl_memdev;
+struct cxl_memdev *cxl_memdev_get_first(struct cxl_ctx *ctx);
+struct cxl_memdev *cxl_memdev_get_next(struct cxl_memdev *memdev);
+int cxl_memdev_get_id(struct cxl_memdev *memdev);
+const char *cxl_memdev_get_devname(struct cxl_memdev *memdev);
+int cxl_memdev_get_major(struct cxl_memdev *memdev);
+int cxl_memdev_get_minor(struct cxl_memdev *memdev);
+struct cxl_ctx *cxl_memdev_get_ctx(struct cxl_memdev *memdev);
+unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev);
+unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev);
+const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev);
+
+#define cxl_memdev_foreach(ctx, memdev) \
+ for (memdev = cxl_memdev_get_first(ctx); \
+ memdev != NULL; \
+ memdev = cxl_memdev_get_next(memdev))
+
+#ifdef __cplusplus
+} /* extern "C" */
+#endif
+
+#endif
diff --git a/util/filter.h b/util/filter.h
index 1e1a41c..9a80d65 100644
--- a/util/filter.h
+++ b/util/filter.h
@@ -29,6 +29,8 @@ struct daxctl_dev *util_daxctl_dev_filter(struct daxctl_dev *dev,
const char *ident);
struct daxctl_region *util_daxctl_region_filter(struct daxctl_region *region,
const char *ident);
+struct cxl_memdev *util_cxl_memdev_filter(struct cxl_memdev *memdev,
+ const char *ident);
enum ndctl_namespace_mode util_nsmode(const char *mode);
const char *util_nsmode_name(enum ndctl_namespace_mode mode);
diff --git a/util/json.h b/util/json.h
index 0f09e36..91918c8 100644
--- a/util/json.h
+++ b/util/json.h
@@ -55,4 +55,7 @@ struct json_object *util_dimm_health_to_json(struct ndctl_dimm *dimm);
struct json_object *util_dimm_firmware_to_json(struct ndctl_dimm *dimm,
unsigned long flags);
struct json_object *util_region_capabilities_to_json(struct ndctl_region *region);
+struct cxl_memdev;
+struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
+ unsigned long flags);
#endif /* __NDCTL_JSON_H__ */
diff --git a/util/main.h b/util/main.h
index c89a843..80b55c4 100644
--- a/util/main.h
+++ b/util/main.h
@@ -10,16 +10,19 @@
enum program {
PROG_NDCTL,
PROG_DAXCTL,
+ PROG_CXL,
};
struct ndctl_ctx;
struct daxctl_ctx;
+struct cxl_ctx;
struct cmd_struct {
const char *cmd;
union {
int (*n_fn)(int, const char **, struct ndctl_ctx *ctx);
int (*d_fn)(int, const char **, struct daxctl_ctx *ctx);
+ int (*c_fn)(int, const char **, struct cxl_ctx *ctx);
};
};
diff --git a/cxl/cxl.c b/cxl/cxl.c
new file mode 100644
index 0000000..a7725f8
--- /dev/null
+++ b/cxl/cxl.c
@@ -0,0 +1,96 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
+/* Copyright (C) 2005 Andreas Ericsson. All rights reserved. */
+
+/* originally copied from perf and git */
+
+#include <stdio.h>
+#include <errno.h>
+#include <string.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <cxl/libcxl.h>
+#include <util/parse-options.h>
+#include <ccan/array_size/array_size.h>
+
+#include <util/strbuf.h>
+#include <util/util.h>
+#include <util/main.h>
+#include <cxl/builtin.h>
+
+const char cxl_usage_string[] = "cxl [--version] [--help] COMMAND [ARGS]";
+const char cxl_more_info_string[] =
+ "See 'cxl help COMMAND' for more information on a specific command.\n"
+ " cxl --list-cmds to see all available commands";
+
+static int cmd_version(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ printf("%s\n", VERSION);
+ return 0;
+}
+
+static int cmd_help(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ const char * const builtin_help_subcommands[] = {
+ "list",
+ NULL,
+ };
+ struct option builtin_help_options[] = {
+ OPT_END(),
+ };
+ const char *builtin_help_usage[] = {
+ "cxl help [command]",
+ NULL
+ };
+
+ argc = parse_options_subcommand(argc, argv, builtin_help_options,
+ builtin_help_subcommands, builtin_help_usage, 0);
+
+ if (!argv[0]) {
+ printf("\n usage: %s\n\n", cxl_usage_string);
+ printf("\n %s\n\n", cxl_more_info_string);
+ return 0;
+ }
+
+ return help_show_man_page(argv[0], "cxl", "CXL_MAN_VIEWER");
+}
+
+static struct cmd_struct commands[] = {
+ { "version", .c_fn = cmd_version },
+ { "list", .c_fn = cmd_list },
+ { "help", .c_fn = cmd_help },
+};
+
+int main(int argc, const char **argv)
+{
+ struct cxl_ctx *ctx;
+ int rc;
+
+ /* Look for flags.. */
+ argv++;
+ argc--;
+ main_handle_options(&argv, &argc, cxl_usage_string, commands,
+ ARRAY_SIZE(commands));
+
+ if (argc > 0) {
+ if (!prefixcmp(argv[0], "--"))
+ argv[0] += 2;
+ } else {
+ /* The user didn't specify a command; give them help */
+ printf("\n usage: %s\n\n", cxl_usage_string);
+ printf("\n %s\n\n", cxl_more_info_string);
+ goto out;
+ }
+
+ rc = cxl_new(&ctx);
+ if (rc)
+ goto out;
+ main_handle_internal_command(argc, argv, ctx, commands,
+ ARRAY_SIZE(commands), PROG_CXL);
+ cxl_unref(ctx);
+ fprintf(stderr, "Unknown command: '%s'\n", argv[0]);
+out:
+ return 1;
+}
diff --git a/cxl/list.c b/cxl/list.c
new file mode 100644
index 0000000..3dea73f
--- /dev/null
+++ b/cxl/list.c
@@ -0,0 +1,113 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
+#include <stdio.h>
+#include <errno.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <limits.h>
+#include <util/json.h>
+#include <util/filter.h>
+#include <json-c/json.h>
+#include <cxl/libcxl.h>
+#include <util/parse-options.h>
+#include <ccan/array_size/array_size.h>
+
+static struct {
+ bool memdevs;
+ bool idle;
+ bool human;
+} list;
+
+static unsigned long listopts_to_flags(void)
+{
+ unsigned long flags = 0;
+
+ if (list.idle)
+ flags |= UTIL_JSON_IDLE;
+ if (list.human)
+ flags |= UTIL_JSON_HUMAN;
+ return flags;
+}
+
+static struct {
+ const char *memdev;
+} param;
+
+static int did_fail;
+
+#define fail(fmt, ...) \
+do { \
+ did_fail = 1; \
+ fprintf(stderr, "cxl-%s:%s:%d: " fmt, \
+ VERSION, __func__, __LINE__, ##__VA_ARGS__); \
+} while (0)
+
+static int num_list_flags(void)
+{
+ return list.memdevs;
+}
+
+int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ const struct option options[] = {
+ OPT_STRING('d', "memdev", ¶m.memdev, "memory device name",
+ "filter by CXL memory device name"),
+ OPT_BOOLEAN('D', "memdevs", &list.memdevs,
+ "include CXL memory device info"),
+ OPT_BOOLEAN('i', "idle", &list.idle, "include idle devices"),
+ OPT_BOOLEAN('u', "human", &list.human,
+ "use human friendly number formats "),
+ OPT_END(),
+ };
+ const char * const u[] = {
+ "cxl list [<options>]",
+ NULL
+ };
+ struct json_object *jdevs = NULL;
+ unsigned long list_flags;
+ struct cxl_memdev *memdev;
+ int i;
+
+ argc = parse_options(argc, argv, options, u, 0);
+ for (i = 0; i < argc; i++)
+ error("unknown parameter \"%s\"\n", argv[i]);
+
+ if (argc)
+ usage_with_options(u, options);
+
+ if (num_list_flags() == 0)
+ list.memdevs = true;
+
+ list_flags = listopts_to_flags();
+
+ cxl_memdev_foreach(ctx, memdev) {
+ struct json_object *jdev = NULL;
+
+ if (!util_cxl_memdev_filter(memdev, param.memdev))
+ continue;
+
+ if (list.memdevs) {
+ if (!jdevs) {
+ jdevs = json_object_new_array();
+ if (!jdevs) {
+ fail("\n");
+ continue;
+ }
+ }
+
+ jdev = util_cxl_memdev_to_json(memdev, list_flags);
+ if (!jdev) {
+ fail("\n");
+ continue;
+ }
+ json_object_array_add(jdevs, jdev);
+ }
+ }
+
+ if (jdevs)
+ util_display_json_array(stdout, jdevs, list_flags);
+
+ if (did_fail)
+ return -ENOMEM;
+ return 0;
+}
diff --git a/util/filter.c b/util/filter.c
index 8b4aad3..d81dade 100644
--- a/util/filter.c
+++ b/util/filter.c
@@ -12,6 +12,7 @@
#include <util/filter.h>
#include <ndctl/libndctl.h>
#include <daxctl/libdaxctl.h>
+#include <cxl/libcxl.h>
struct ndctl_bus *util_bus_filter(struct ndctl_bus *bus, const char *__ident)
{
@@ -339,6 +340,25 @@ struct daxctl_region *util_daxctl_region_filter(struct daxctl_region *region,
return NULL;
}
+struct cxl_memdev *util_cxl_memdev_filter(struct cxl_memdev *memdev,
+ const char *ident)
+{
+ int memdev_id;
+
+ if (!ident || strcmp(ident, "all") == 0)
+ return memdev;
+
+ if (strcmp(ident, cxl_memdev_get_devname(memdev)) == 0)
+ return memdev;
+
+ if ((sscanf(ident, "%d", &memdev_id) == 1
+ || sscanf(ident, "mem%d", &memdev_id) == 1)
+ && cxl_memdev_get_id(memdev) == memdev_id)
+ return memdev;
+
+ return NULL;
+}
+
enum ndctl_namespace_mode util_nsmode(const char *mode)
{
if (!mode)
diff --git a/util/json.c b/util/json.c
index a8d2412..3be3a92 100644
--- a/util/json.c
+++ b/util/json.c
@@ -9,6 +9,7 @@
#include <json-c/printbuf.h>
#include <ndctl/libndctl.h>
#include <daxctl/libdaxctl.h>
+#include <cxl/libcxl.h>
#include <ccan/array_size/array_size.h>
#include <ccan/short_types/short_types.h>
#include <ndctl.h>
@@ -1440,3 +1441,28 @@ struct json_object *util_badblock_rec_to_json(u64 block, u64 count,
json_object_put(jerr);
return NULL;
}
+
+struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
+ unsigned long flags)
+{
+ const char *devname = cxl_memdev_get_devname(memdev);
+ struct json_object *jdev, *jobj;
+
+ jdev = json_object_new_object();
+ if (!devname || !jdev)
+ return NULL;
+
+ jobj = json_object_new_string(devname);
+ if (jobj)
+ json_object_object_add(jdev, "memdev", jobj);
+
+ jobj = util_json_object_size(cxl_memdev_get_pmem_size(memdev), flags);
+ if (jobj)
+ json_object_object_add(jdev, "pmem_size", jobj);
+
+ jobj = util_json_object_size(cxl_memdev_get_ram_size(memdev), flags);
+ if (jobj)
+ json_object_object_add(jdev, "ram_size", jobj);
+
+ return jdev;
+}
diff --git a/.clang-format b/.clang-format
index 4e00fff..d2e77d0 100644
--- a/.clang-format
+++ b/.clang-format
@@ -77,6 +77,7 @@ ExperimentalAutoDetectBinPacking: false
# -e 's/\(.*foreach.*\)(.*/\1/' \
# | sort -u)
ForEachMacros:
+ - 'cxl_memdev_foreach'
- 'daxctl_dev_foreach'
- 'daxctl_mapping_foreach'
- 'daxctl_region_foreach'
diff --git a/.gitignore b/.gitignore
index 53512b2..6a97b92 100644
--- a/.gitignore
+++ b/.gitignore
@@ -16,9 +16,11 @@ Makefile.in
*.1
Documentation/daxctl/asciidoc.conf
Documentation/ndctl/asciidoc.conf
-Documentation/ndctl/attrs.adoc
+Documentation/cxl/asciidoc.conf
Documentation/daxctl/asciidoctor-extensions.rb
Documentation/ndctl/asciidoctor-extensions.rb
+Documentation/cxl/asciidoctor-extensions.rb
+Documentation/ndctl/attrs.adoc
.dirstamp
daxctl/config.h
daxctl/daxctl
diff --git a/Documentation/cxl/Makefile.am b/Documentation/cxl/Makefile.am
new file mode 100644
index 0000000..db98dd7
--- /dev/null
+++ b/Documentation/cxl/Makefile.am
@@ -0,0 +1,58 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020-2021 Intel Corporation. All rights reserved.
+
+if USE_ASCIIDOCTOR
+
+do_subst = sed -e 's,@Utility@,Cxl,g' -e's,@utility@,cxl,g'
+CONFFILE = asciidoctor-extensions.rb
+asciidoctor-extensions.rb: ../asciidoctor-extensions.rb.in
+ $(AM_V_GEN) $(do_subst) < $< > $@
+
+else
+
+do_subst = sed -e 's,UTILITY,cxl,g'
+CONFFILE = asciidoc.conf
+asciidoc.conf: ../asciidoc.conf.in
+ $(AM_V_GEN) $(do_subst) < $< > $@
+
+endif
+
+man1_MANS = \
+ cxl.1 \
+ cxl-list.1
+
+EXTRA_DIST = $(man1_MANS)
+
+CLEANFILES = $(man1_MANS)
+
+XML_DEPS = \
+ ../../version.m4 \
+ ../copyright.txt \
+ Makefile \
+ $(CONFFILE)
+
+RM ?= rm -f
+
+if USE_ASCIIDOCTOR
+
+%.1: %.txt $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@+ $@ && \
+ $(ASCIIDOC) -b manpage -d manpage -acompat-mode \
+ -I. -rasciidoctor-extensions \
+ -amansource=cxl -amanmanual="cxl Manual" \
+ -andctl_version=$(VERSION) -o $@+ $< && \
+ mv $@+ $@
+
+else
+
+%.xml: %.txt $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@+ $@ && \
+ $(ASCIIDOC) -b docbook -d manpage -f asciidoc.conf \
+ --unsafe -acxl_version=$(VERSION) -o $@+ $< && \
+ mv $@+ $@
+
+%.1: %.xml $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@ && \
+ $(XMLTO) -o . -m ../manpage-normal.xsl man $<
+
+endif
diff --git a/cxl/Makefile.am b/cxl/Makefile.am
new file mode 100644
index 0000000..98606b9
--- /dev/null
+++ b/cxl/Makefile.am
@@ -0,0 +1,21 @@
+include $(top_srcdir)/Makefile.am.in
+
+bin_PROGRAMS = cxl
+
+DISTCLEANFILES = config.h
+BUILT_SOURCES = config.h
+config.h: $(srcdir)/Makefile.am
+ $(AM_V_GEN) echo "/* Autogenerated by cxl/Makefile.am */" >$@
+
+cxl_SOURCES =\
+ cxl.c \
+ list.c \
+ ../util/json.c \
+ builtin.h
+
+cxl_LDADD =\
+ lib/libcxl.la \
+ ../libutil.a \
+ $(UUID_LIBS) \
+ $(KMOD_LIBS) \
+ $(JSON_LIBS)
diff --git a/cxl/lib/Makefile.am b/cxl/lib/Makefile.am
new file mode 100644
index 0000000..277f0cd
--- /dev/null
+++ b/cxl/lib/Makefile.am
@@ -0,0 +1,32 @@
+include $(top_srcdir)/Makefile.am.in
+
+%.pc: %.pc.in Makefile
+ $(SED_PROCESS)
+
+pkginclude_HEADERS = ../libcxl.h
+lib_LTLIBRARIES = libcxl.la
+
+libcxl_la_SOURCES =\
+ ../libcxl.h \
+ private.h \
+ ../../util/sysfs.c \
+ ../../util/sysfs.h \
+ ../../util/log.c \
+ ../../util/log.h \
+ libcxl.c
+
+libcxl_la_LIBADD =\
+ $(UUID_LIBS) \
+ $(KMOD_LIBS)
+
+EXTRA_DIST += libcxl.sym
+
+libcxl_la_LDFLAGS = $(AM_LDFLAGS) \
+ -version-info $(LIBCXL_CURRENT):$(LIBCXL_REVISION):$(LIBCXL_AGE) \
+ -Wl,--version-script=$(top_srcdir)/cxl/lib/libcxl.sym
+libcxl_la_DEPENDENCIES = libcxl.sym
+
+pkgconfigdir = $(libdir)/pkgconfig
+pkgconfig_DATA = libcxl.pc
+EXTRA_DIST += libcxl.pc.in
+CLEANFILES += libcxl.pc
diff --git a/cxl/lib/libcxl.pc.in b/cxl/lib/libcxl.pc.in
new file mode 100644
index 0000000..949fcdc
--- /dev/null
+++ b/cxl/lib/libcxl.pc.in
@@ -0,0 +1,11 @@
+prefix=@prefix@
+exec_prefix=@exec_prefix@
+libdir=@libdir@
+includedir=@includedir@
+
+Name: libcxl
+Description: Manage CXL devices
+Version: @VERSION@
+Libs: -L${libdir} -lcxl
+Libs.private:
+Cflags: -I${includedir}
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
new file mode 100644
index 0000000..2616e5c
--- /dev/null
+++ b/cxl/lib/libcxl.sym
@@ -0,0 +1,25 @@
+LIBCXL_1 {
+global:
+ cxl_get_userdata;
+ cxl_set_userdata;
+ cxl_get_private_data;
+ cxl_set_private_data;
+ cxl_ref;
+ cxl_get_log_priority;
+ cxl_set_log_fn;
+ cxl_unref;
+ cxl_set_log_priority;
+ cxl_new;
+ cxl_memdev_get_first;
+ cxl_memdev_get_next;
+ cxl_memdev_get_id;
+ cxl_memdev_get_devname;
+ cxl_memdev_get_major;
+ cxl_memdev_get_minor;
+ cxl_memdev_get_ctx;
+ cxl_memdev_get_pmem_size;
+ cxl_memdev_get_ram_size;
+ cxl_memdev_get_firmware_verison;
+local:
+ *;
+};
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* [ndctl PATCH v6 02/16] cxl: add a cxl utility and libcxl library
2021-11-11 20:44 ` [ndctl PATCH v5 02/16] cxl: add a cxl utility and libcxl library Vishal Verma
@ 2021-11-11 21:59 ` Vishal Verma
2021-12-08 5:12 ` [ndctl PATCH v5 " Dan Williams
1 sibling, 0 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 21:59 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
CXL - or Compute eXpress Link - is a new interconnect that extends PCIe
to support a wide range of devices, including cache coherent memory
expanders. As such, these devices can be new sources of 'persistent
memory', and the 'ndctl' umbrella of tools and libraries needs to be able
to interact with them.
Add a new utility and library for managing these CXL memory devices. This
is an initial bring-up for interacting with CXL devices, and only includes
adding the utility and library infrastructure, parsing device information
from sysfs for CXL devices, and providing a 'cxl-list' command to
display this information in JSON formatted output.
Cc: Ben Widawsky <ben.widawsky@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
Changes in v6:
- Use -m/-M as the short options for listing memdevs (Dan)
Documentation/cxl/cxl-list.txt | 64 +++++
Documentation/cxl/cxl.txt | 34 +++
Documentation/cxl/human-option.txt | 8 +
Documentation/cxl/verbose-option.txt | 5 +
configure.ac | 3 +
Makefile.am | 8 +-
Makefile.am.in | 4 +
cxl/lib/private.h | 29 +++
cxl/lib/libcxl.c | 345 +++++++++++++++++++++++++++
cxl/builtin.h | 8 +
cxl/libcxl.h | 55 +++++
util/filter.h | 2 +
util/json.h | 3 +
util/main.h | 3 +
cxl/cxl.c | 96 ++++++++
cxl/list.c | 113 +++++++++
util/filter.c | 20 ++
util/json.c | 26 ++
.clang-format | 1 +
.gitignore | 4 +-
Documentation/cxl/Makefile.am | 58 +++++
cxl/Makefile.am | 21 ++
cxl/lib/Makefile.am | 32 +++
cxl/lib/libcxl.pc.in | 11 +
cxl/lib/libcxl.sym | 25 ++
25 files changed, 974 insertions(+), 4 deletions(-)
create mode 100644 Documentation/cxl/cxl-list.txt
create mode 100644 Documentation/cxl/cxl.txt
create mode 100644 Documentation/cxl/human-option.txt
create mode 100644 Documentation/cxl/verbose-option.txt
create mode 100644 cxl/lib/private.h
create mode 100644 cxl/lib/libcxl.c
create mode 100644 cxl/builtin.h
create mode 100644 cxl/libcxl.h
create mode 100644 cxl/cxl.c
create mode 100644 cxl/list.c
create mode 100644 Documentation/cxl/Makefile.am
create mode 100644 cxl/Makefile.am
create mode 100644 cxl/lib/Makefile.am
create mode 100644 cxl/lib/libcxl.pc.in
create mode 100644 cxl/lib/libcxl.sym
diff --git a/Documentation/cxl/cxl-list.txt b/Documentation/cxl/cxl-list.txt
new file mode 100644
index 0000000..e761cfa
--- /dev/null
+++ b/Documentation/cxl/cxl-list.txt
@@ -0,0 +1,64 @@
+// SPDX-License-Identifier: GPL-2.0
+
+cxl-list(1)
+===========
+
+NAME
+----
+cxl-list - List CXL capable memory devices, and their attributes in json.
+
+SYNOPSIS
+--------
+[verse]
+'cxl list' [<options>]
+
+Walk the CXL capable device hierarchy in the system and list all device
+instances along with some of their major attributes.
+
+Options can be specified to limit the output to specific memdevs.
+By default, 'cxl list' with no options is equivalent to:
+[verse]
+cxl list --memdevs
+
+EXAMPLE
+-------
+----
+# cxl list --memdevs
+{
+ "memdev":"mem0",
+ "pmem_size":268435456,
+ "ram_size":0,
+}
+----
+
+OPTIONS
+-------
+-m::
+--memdev=::
+ Specify a cxl memory device name to filter the listing. For example:
+----
+# cxl list --memdev=mem0
+{
+ "memdev":"mem0",
+ "pmem_size":268435456,
+ "ram_size":0,
+}
+----
+
+-M::
+--memdevs::
+ Include all CXL memory devices in the listing
+
+-i::
+--idle::
+ Include idle (not enabled / zero-sized) devices in the listing
+
+include::human-option.txt[]
+
+include::verbose-option.txt[]
+
+include::../copyright.txt[]
+
+SEE ALSO
+--------
+linkcxl:ndctl-list[1]
diff --git a/Documentation/cxl/cxl.txt b/Documentation/cxl/cxl.txt
new file mode 100644
index 0000000..41a51c7
--- /dev/null
+++ b/Documentation/cxl/cxl.txt
@@ -0,0 +1,34 @@
+// SPDX-License-Identifier: GPL-2.0
+
+cxl(1)
+======
+
+NAME
+----
+cxl - Provides enumeration and provisioning commands for CXL platforms
+
+SYNOPSIS
+--------
+[verse]
+'cxl' [--version] [--help] COMMAND [ARGS]
+
+OPTIONS
+-------
+-v::
+--version::
+ Display the version of the 'cxl' utility.
+
+-h::
+--help::
+ Run the 'cxl help' command.
+
+DESCRIPTION
+-----------
+The cxl utility provides enumeration and provisioning commands for
+the CXL devices managed by the Linux kernel.
+
+include::../copyright.txt[]
+
+SEE ALSO
+--------
+linkcxl:ndctl[1]
diff --git a/Documentation/cxl/human-option.txt b/Documentation/cxl/human-option.txt
new file mode 100644
index 0000000..2f4de7a
--- /dev/null
+++ b/Documentation/cxl/human-option.txt
@@ -0,0 +1,8 @@
+// SPDX-License-Identifier: GPL-2.0
+
+-u::
+--human::
+ By default the command will output machine-friendly raw-integer
+ data. Instead, with this flag, numbers representing storage size
+ will be formatted as human readable strings with units, other
+ fields are converted to hexadecimal strings.
diff --git a/Documentation/cxl/verbose-option.txt b/Documentation/cxl/verbose-option.txt
new file mode 100644
index 0000000..cb62c8e
--- /dev/null
+++ b/Documentation/cxl/verbose-option.txt
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+-v::
+--verbose::
+ Emit more debug messages
diff --git a/configure.ac b/configure.ac
index dc39dbe..dadae0a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -222,12 +222,15 @@ AC_CONFIG_HEADERS(config.h)
AC_CONFIG_FILES([
Makefile
daxctl/lib/Makefile
+ cxl/lib/Makefile
ndctl/lib/Makefile
ndctl/Makefile
daxctl/Makefile
+ cxl/Makefile
test/Makefile
Documentation/ndctl/Makefile
Documentation/daxctl/Makefile
+ Documentation/cxl/Makefile
])
AC_OUTPUT
diff --git a/Makefile.am b/Makefile.am
index 60a1998..428fd40 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,9 +1,9 @@
include Makefile.am.in
ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS}
-SUBDIRS = . daxctl/lib ndctl/lib ndctl daxctl
+SUBDIRS = . cxl/lib daxctl/lib ndctl/lib cxl ndctl daxctl
if ENABLE_DOCS
-SUBDIRS += Documentation/ndctl Documentation/daxctl
+SUBDIRS += Documentation/ndctl Documentation/daxctl Documentation/cxl
endif
SUBDIRS += test
@@ -87,4 +87,6 @@ libutil_a_SOURCES = \
util/filter.h \
util/bitmap.h
-nobase_include_HEADERS = daxctl/libdaxctl.h
+nobase_include_HEADERS = \
+ daxctl/libdaxctl.h \
+ cxl/libcxl.h
diff --git a/Makefile.am.in b/Makefile.am.in
index bdceda9..aaeee53 100644
--- a/Makefile.am.in
+++ b/Makefile.am.in
@@ -42,3 +42,7 @@ LIBNDCTL_AGE=19
LIBDAXCTL_CURRENT=6
LIBDAXCTL_REVISION=0
LIBDAXCTL_AGE=5
+
+LIBCXL_CURRENT=1
+LIBCXL_REVISION=0
+LIBCXL_AGE=0
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
new file mode 100644
index 0000000..fc88fa1
--- /dev/null
+++ b/cxl/lib/private.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+/* Copyright (C) 2020-2021, Intel Corporation. All rights reserved. */
+#ifndef _LIBCXL_PRIVATE_H_
+#define _LIBCXL_PRIVATE_H_
+
+#include <libkmod.h>
+
+#define CXL_EXPORT __attribute__ ((visibility("default")))
+
+struct cxl_memdev {
+ int id, major, minor;
+ void *dev_buf;
+ size_t buf_len;
+ char *dev_path;
+ char *firmware_version;
+ struct cxl_ctx *ctx;
+ struct list_node list;
+ unsigned long long pmem_size;
+ unsigned long long ram_size;
+ int payload_max;
+ struct kmod_module *module;
+};
+
+static inline int check_kmod(struct kmod_ctx *kmod_ctx)
+{
+ return kmod_ctx ? 0 : -ENXIO;
+}
+
+#endif /* _LIBCXL_PRIVATE_H_ */
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
new file mode 100644
index 0000000..c15e987
--- /dev/null
+++ b/cxl/lib/libcxl.c
@@ -0,0 +1,345 @@
+// SPDX-License-Identifier: LGPL-2.1
+// Copyright (C) 2020-2021, Intel Corporation. All rights reserved.
+#include <stdio.h>
+#include <errno.h>
+#include <limits.h>
+#include <libgen.h>
+#include <stdlib.h>
+#include <dirent.h>
+#include <unistd.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/sysmacros.h>
+#include <uuid/uuid.h>
+#include <ccan/list/list.h>
+#include <ccan/array_size/array_size.h>
+
+#include <util/log.h>
+#include <util/sysfs.h>
+#include <util/bitmap.h>
+#include <cxl/libcxl.h>
+#include "private.h"
+
+/**
+ * struct cxl_ctx - library user context to find "nd" instances
+ *
+ * Instantiate with cxl_new(), which takes an initial reference. Free
+ * the context by dropping the reference count to zero with
+ * cxl_unref(), or take additional references with cxl_ref()
+ * @timeout: default library timeout in milliseconds
+ */
+struct cxl_ctx {
+ /* log_ctx must be first member for cxl_set_log_fn compat */
+ struct log_ctx ctx;
+ int refcount;
+ void *userdata;
+ int memdevs_init;
+ struct list_head memdevs;
+ struct kmod_ctx *kmod_ctx;
+ void *private_data;
+};
+
+static void free_memdev(struct cxl_memdev *memdev, struct list_head *head)
+{
+ if (head)
+ list_del_from(head, &memdev->list);
+ kmod_module_unref(memdev->module);
+ free(memdev->firmware_version);
+ free(memdev->dev_buf);
+ free(memdev->dev_path);
+ free(memdev);
+}
+
+/**
+ * cxl_get_userdata - retrieve stored data pointer from library context
+ * @ctx: cxl library context
+ *
+ * This might be useful to access from callbacks like a custom logging
+ * function.
+ */
+CXL_EXPORT void *cxl_get_userdata(struct cxl_ctx *ctx)
+{
+ if (ctx == NULL)
+ return NULL;
+ return ctx->userdata;
+}
+
+/**
+ * cxl_set_userdata - store custom @userdata in the library context
+ * @ctx: cxl library context
+ * @userdata: data pointer
+ */
+CXL_EXPORT void cxl_set_userdata(struct cxl_ctx *ctx, void *userdata)
+{
+ if (ctx == NULL)
+ return;
+ ctx->userdata = userdata;
+}
+
+CXL_EXPORT void cxl_set_private_data(struct cxl_ctx *ctx, void *data)
+{
+ ctx->private_data = data;
+}
+
+CXL_EXPORT void *cxl_get_private_data(struct cxl_ctx *ctx)
+{
+ return ctx->private_data;
+}
+
+/**
+ * cxl_new - instantiate a new library context
+ * @ctx: context to establish
+ *
+ * Returns zero on success and stores an opaque pointer in ctx. The
+ * context is freed by cxl_unref(), i.e. cxl_new() implies an
+ * internal cxl_ref().
+ */
+CXL_EXPORT int cxl_new(struct cxl_ctx **ctx)
+{
+ struct kmod_ctx *kmod_ctx;
+ struct cxl_ctx *c;
+ int rc = 0;
+
+ c = calloc(1, sizeof(struct cxl_ctx));
+ if (!c)
+ return -ENOMEM;
+
+ kmod_ctx = kmod_new(NULL, NULL);
+ if (check_kmod(kmod_ctx) != 0) {
+ rc = -ENXIO;
+ goto out;
+ }
+
+ c->refcount = 1;
+ log_init(&c->ctx, "libcxl", "CXL_LOG");
+ info(c, "ctx %p created\n", c);
+ dbg(c, "log_priority=%d\n", c->ctx.log_priority);
+ *ctx = c;
+ list_head_init(&c->memdevs);
+ c->kmod_ctx = kmod_ctx;
+
+ return 0;
+out:
+ free(c);
+ return rc;
+}
+
+/**
+ * cxl_ref - take an additional reference on the context
+ * @ctx: context established by cxl_new()
+ */
+CXL_EXPORT struct cxl_ctx *cxl_ref(struct cxl_ctx *ctx)
+{
+ if (ctx == NULL)
+ return NULL;
+ ctx->refcount++;
+ return ctx;
+}
+
+/**
+ * cxl_unref - drop a context reference count
+ * @ctx: context established by cxl_new()
+ *
+ * Drop a reference and if the resulting reference count is 0 destroy
+ * the context.
+ */
+CXL_EXPORT void cxl_unref(struct cxl_ctx *ctx)
+{
+ struct cxl_memdev *memdev, *_d;
+
+ if (ctx == NULL)
+ return;
+ ctx->refcount--;
+ if (ctx->refcount > 0)
+ return;
+
+ list_for_each_safe(&ctx->memdevs, memdev, _d, list)
+ free_memdev(memdev, &ctx->memdevs);
+
+ kmod_unref(ctx->kmod_ctx);
+ info(ctx, "context %p released\n", ctx);
+ free(ctx);
+}
+
+/**
+ * cxl_set_log_fn - override default log routine
+ * @ctx: cxl library context
+ * @log_fn: function to be called for logging messages
+ *
+ * The built-in logging writes to stderr. It can be overridden by a
+ * custom function, to plug log messages into the user's logging
+ * functionality.
+ */
+CXL_EXPORT void cxl_set_log_fn(struct cxl_ctx *ctx,
+ void (*cxl_log_fn)(struct cxl_ctx *ctx, int priority,
+ const char *file, int line, const char *fn,
+ const char *format, va_list args))
+{
+ ctx->ctx.log_fn = (log_fn) cxl_log_fn;
+ info(ctx, "custom logging function %p registered\n", cxl_log_fn);
+}
+
+/**
+ * cxl_get_log_priority - retrieve current library loglevel (syslog)
+ * @ctx: cxl library context
+ */
+CXL_EXPORT int cxl_get_log_priority(struct cxl_ctx *ctx)
+{
+ return ctx->ctx.log_priority;
+}
+
+/**
+ * cxl_set_log_priority - set log verbosity
+ * @priority: from syslog.h, LOG_ERR, LOG_INFO, LOG_DEBUG
+ *
+ * Note: LOG_DEBUG requires library be built with "configure --enable-debug"
+ */
+CXL_EXPORT void cxl_set_log_priority(struct cxl_ctx *ctx, int priority)
+{
+ ctx->ctx.log_priority = priority;
+}
+
+static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
+{
+ const char *devname = devpath_to_devname(cxlmem_base);
+ char *path = calloc(1, strlen(cxlmem_base) + 100);
+ struct cxl_ctx *ctx = parent;
+ struct cxl_memdev *memdev, *memdev_dup;
+ char buf[SYSFS_ATTR_SIZE];
+ struct stat st;
+
+ if (!path)
+ return NULL;
+ dbg(ctx, "%s: base: \'%s\'\n", devname, cxlmem_base);
+
+ memdev = calloc(1, sizeof(*memdev));
+ if (!memdev)
+ goto err_dev;
+ memdev->id = id;
+ memdev->ctx = ctx;
+
+ sprintf(path, "/dev/cxl/%s", devname);
+ if (stat(path, &st) < 0)
+ goto err_read;
+ memdev->major = major(st.st_rdev);
+ memdev->minor = minor(st.st_rdev);
+
+ sprintf(path, "%s/pmem/size", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+ memdev->pmem_size = strtoull(buf, NULL, 0);
+
+ sprintf(path, "%s/ram/size", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+ memdev->ram_size = strtoull(buf, NULL, 0);
+
+ sprintf(path, "%s/payload_max", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+ memdev->payload_max = strtoull(buf, NULL, 0);
+ if (memdev->payload_max < 0)
+ goto err_read;
+
+ memdev->dev_path = strdup(cxlmem_base);
+ if (!memdev->dev_path)
+ goto err_read;
+
+ sprintf(path, "%s/firmware_version", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+
+ memdev->firmware_version = strdup(buf);
+ if (!memdev->firmware_version)
+ goto err_read;
+
+ memdev->dev_buf = calloc(1, strlen(cxlmem_base) + 50);
+ if (!memdev->dev_buf)
+ goto err_read;
+ memdev->buf_len = strlen(cxlmem_base) + 50;
+
+ cxl_memdev_foreach(ctx, memdev_dup)
+ if (memdev_dup->id == memdev->id) {
+ free_memdev(memdev, NULL);
+ free(path);
+ return memdev_dup;
+ }
+
+ list_add(&ctx->memdevs, &memdev->list);
+ free(path);
+ return memdev;
+
+ err_read:
+ free(memdev->firmware_version);
+ free(memdev->dev_buf);
+ free(memdev->dev_path);
+ free(memdev);
+ err_dev:
+ free(path);
+ return NULL;
+}
+
+static void cxl_memdevs_init(struct cxl_ctx *ctx)
+{
+ if (ctx->memdevs_init)
+ return;
+
+ ctx->memdevs_init = 1;
+
+ sysfs_device_parse(ctx, "/sys/bus/cxl/devices", "mem", ctx,
+ add_cxl_memdev);
+}
+
+CXL_EXPORT struct cxl_ctx *cxl_memdev_get_ctx(struct cxl_memdev *memdev)
+{
+ return memdev->ctx;
+}
+
+CXL_EXPORT struct cxl_memdev *cxl_memdev_get_first(struct cxl_ctx *ctx)
+{
+ cxl_memdevs_init(ctx);
+
+ return list_top(&ctx->memdevs, struct cxl_memdev, list);
+}
+
+CXL_EXPORT struct cxl_memdev *cxl_memdev_get_next(struct cxl_memdev *memdev)
+{
+ struct cxl_ctx *ctx = memdev->ctx;
+
+ return list_next(&ctx->memdevs, memdev, list);
+}
+
+CXL_EXPORT int cxl_memdev_get_id(struct cxl_memdev *memdev)
+{
+ return memdev->id;
+}
+
+CXL_EXPORT const char *cxl_memdev_get_devname(struct cxl_memdev *memdev)
+{
+ return devpath_to_devname(memdev->dev_path);
+}
+
+CXL_EXPORT int cxl_memdev_get_major(struct cxl_memdev *memdev)
+{
+ return memdev->major;
+}
+
+CXL_EXPORT int cxl_memdev_get_minor(struct cxl_memdev *memdev)
+{
+ return memdev->minor;
+}
+
+CXL_EXPORT unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev)
+{
+ return memdev->pmem_size;
+}
+
+CXL_EXPORT unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev)
+{
+ return memdev->ram_size;
+}
+
+CXL_EXPORT const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev)
+{
+ return memdev->firmware_version;
+}
diff --git a/cxl/builtin.h b/cxl/builtin.h
new file mode 100644
index 0000000..3797f98
--- /dev/null
+++ b/cxl/builtin.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
+#ifndef _CXL_BUILTIN_H_
+#define _CXL_BUILTIN_H_
+
+struct cxl_ctx;
+int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx);
+#endif /* _CXL_BUILTIN_H_ */
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
new file mode 100644
index 0000000..fd06790
--- /dev/null
+++ b/cxl/libcxl.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+/* Copyright (C) 2020-2021, Intel Corporation. All rights reserved. */
+#ifndef _LIBCXL_H_
+#define _LIBCXL_H_
+
+#include <stdarg.h>
+#include <unistd.h>
+
+#ifdef HAVE_UUID
+#include <uuid/uuid.h>
+#else
+typedef unsigned char uuid_t[16];
+#endif
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct cxl_ctx;
+struct cxl_ctx *cxl_ref(struct cxl_ctx *ctx);
+void cxl_unref(struct cxl_ctx *ctx);
+int cxl_new(struct cxl_ctx **ctx);
+void cxl_set_log_fn(struct cxl_ctx *ctx,
+ void (*log_fn)(struct cxl_ctx *ctx, int priority,
+ const char *file, int line, const char *fn,
+ const char *format, va_list args));
+int cxl_get_log_priority(struct cxl_ctx *ctx);
+void cxl_set_log_priority(struct cxl_ctx *ctx, int priority);
+void cxl_set_userdata(struct cxl_ctx *ctx, void *userdata);
+void *cxl_get_userdata(struct cxl_ctx *ctx);
+void cxl_set_private_data(struct cxl_ctx *ctx, void *data);
+void *cxl_get_private_data(struct cxl_ctx *ctx);
+
+struct cxl_memdev;
+struct cxl_memdev *cxl_memdev_get_first(struct cxl_ctx *ctx);
+struct cxl_memdev *cxl_memdev_get_next(struct cxl_memdev *memdev);
+int cxl_memdev_get_id(struct cxl_memdev *memdev);
+const char *cxl_memdev_get_devname(struct cxl_memdev *memdev);
+int cxl_memdev_get_major(struct cxl_memdev *memdev);
+int cxl_memdev_get_minor(struct cxl_memdev *memdev);
+struct cxl_ctx *cxl_memdev_get_ctx(struct cxl_memdev *memdev);
+unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev);
+unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev);
+const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev);
+
+#define cxl_memdev_foreach(ctx, memdev) \
+ for (memdev = cxl_memdev_get_first(ctx); \
+ memdev != NULL; \
+ memdev = cxl_memdev_get_next(memdev))
+
+#ifdef __cplusplus
+} /* extern "C" */
+#endif
+
+#endif
diff --git a/util/filter.h b/util/filter.h
index 1e1a41c..9a80d65 100644
--- a/util/filter.h
+++ b/util/filter.h
@@ -29,6 +29,8 @@ struct daxctl_dev *util_daxctl_dev_filter(struct daxctl_dev *dev,
const char *ident);
struct daxctl_region *util_daxctl_region_filter(struct daxctl_region *region,
const char *ident);
+struct cxl_memdev *util_cxl_memdev_filter(struct cxl_memdev *memdev,
+ const char *ident);
enum ndctl_namespace_mode util_nsmode(const char *mode);
const char *util_nsmode_name(enum ndctl_namespace_mode mode);
diff --git a/util/json.h b/util/json.h
index 0f09e36..91918c8 100644
--- a/util/json.h
+++ b/util/json.h
@@ -55,4 +55,7 @@ struct json_object *util_dimm_health_to_json(struct ndctl_dimm *dimm);
struct json_object *util_dimm_firmware_to_json(struct ndctl_dimm *dimm,
unsigned long flags);
struct json_object *util_region_capabilities_to_json(struct ndctl_region *region);
+struct cxl_memdev;
+struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
+ unsigned long flags);
#endif /* __NDCTL_JSON_H__ */
diff --git a/util/main.h b/util/main.h
index c89a843..80b55c4 100644
--- a/util/main.h
+++ b/util/main.h
@@ -10,16 +10,19 @@
enum program {
PROG_NDCTL,
PROG_DAXCTL,
+ PROG_CXL,
};
struct ndctl_ctx;
struct daxctl_ctx;
+struct cxl_ctx;
struct cmd_struct {
const char *cmd;
union {
int (*n_fn)(int, const char **, struct ndctl_ctx *ctx);
int (*d_fn)(int, const char **, struct daxctl_ctx *ctx);
+ int (*c_fn)(int, const char **, struct cxl_ctx *ctx);
};
};
diff --git a/cxl/cxl.c b/cxl/cxl.c
new file mode 100644
index 0000000..a7725f8
--- /dev/null
+++ b/cxl/cxl.c
@@ -0,0 +1,96 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
+/* Copyright (C) 2005 Andreas Ericsson. All rights reserved. */
+
+/* originally copied from perf and git */
+
+#include <stdio.h>
+#include <errno.h>
+#include <string.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <cxl/libcxl.h>
+#include <util/parse-options.h>
+#include <ccan/array_size/array_size.h>
+
+#include <util/strbuf.h>
+#include <util/util.h>
+#include <util/main.h>
+#include <cxl/builtin.h>
+
+const char cxl_usage_string[] = "cxl [--version] [--help] COMMAND [ARGS]";
+const char cxl_more_info_string[] =
+ "See 'cxl help COMMAND' for more information on a specific command.\n"
+ " cxl --list-cmds to see all available commands";
+
+static int cmd_version(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ printf("%s\n", VERSION);
+ return 0;
+}
+
+static int cmd_help(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ const char * const builtin_help_subcommands[] = {
+ "list",
+ NULL,
+ };
+ struct option builtin_help_options[] = {
+ OPT_END(),
+ };
+ const char *builtin_help_usage[] = {
+ "cxl help [command]",
+ NULL
+ };
+
+ argc = parse_options_subcommand(argc, argv, builtin_help_options,
+ builtin_help_subcommands, builtin_help_usage, 0);
+
+ if (!argv[0]) {
+ printf("\n usage: %s\n\n", cxl_usage_string);
+ printf("\n %s\n\n", cxl_more_info_string);
+ return 0;
+ }
+
+ return help_show_man_page(argv[0], "cxl", "CXL_MAN_VIEWER");
+}
+
+static struct cmd_struct commands[] = {
+ { "version", .c_fn = cmd_version },
+ { "list", .c_fn = cmd_list },
+ { "help", .c_fn = cmd_help },
+};
+
+int main(int argc, const char **argv)
+{
+ struct cxl_ctx *ctx;
+ int rc;
+
+ /* Look for flags.. */
+ argv++;
+ argc--;
+ main_handle_options(&argv, &argc, cxl_usage_string, commands,
+ ARRAY_SIZE(commands));
+
+ if (argc > 0) {
+ if (!prefixcmp(argv[0], "--"))
+ argv[0] += 2;
+ } else {
+ /* The user didn't specify a command; give them help */
+ printf("\n usage: %s\n\n", cxl_usage_string);
+ printf("\n %s\n\n", cxl_more_info_string);
+ goto out;
+ }
+
+ rc = cxl_new(&ctx);
+ if (rc)
+ goto out;
+ main_handle_internal_command(argc, argv, ctx, commands,
+ ARRAY_SIZE(commands), PROG_CXL);
+ cxl_unref(ctx);
+ fprintf(stderr, "Unknown command: '%s'\n", argv[0]);
+out:
+ return 1;
+}
diff --git a/cxl/list.c b/cxl/list.c
new file mode 100644
index 0000000..7c1d2eb
--- /dev/null
+++ b/cxl/list.c
@@ -0,0 +1,113 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
+#include <stdio.h>
+#include <errno.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <limits.h>
+#include <util/json.h>
+#include <util/filter.h>
+#include <json-c/json.h>
+#include <cxl/libcxl.h>
+#include <util/parse-options.h>
+#include <ccan/array_size/array_size.h>
+
+static struct {
+ bool memdevs;
+ bool idle;
+ bool human;
+} list;
+
+static unsigned long listopts_to_flags(void)
+{
+ unsigned long flags = 0;
+
+ if (list.idle)
+ flags |= UTIL_JSON_IDLE;
+ if (list.human)
+ flags |= UTIL_JSON_HUMAN;
+ return flags;
+}
+
+static struct {
+ const char *memdev;
+} param;
+
+static int did_fail;
+
+#define fail(fmt, ...) \
+do { \
+ did_fail = 1; \
+ fprintf(stderr, "cxl-%s:%s:%d: " fmt, \
+ VERSION, __func__, __LINE__, ##__VA_ARGS__); \
+} while (0)
+
+static int num_list_flags(void)
+{
+ return list.memdevs;
+}
+
+int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ const struct option options[] = {
+ OPT_STRING('m', "memdev", ¶m.memdev, "memory device name",
+ "filter by CXL memory device name"),
+ OPT_BOOLEAN('M', "memdevs", &list.memdevs,
+ "include CXL memory device info"),
+ OPT_BOOLEAN('i', "idle", &list.idle, "include idle devices"),
+ OPT_BOOLEAN('u', "human", &list.human,
+ "use human friendly number formats "),
+ OPT_END(),
+ };
+ const char * const u[] = {
+ "cxl list [<options>]",
+ NULL
+ };
+ struct json_object *jdevs = NULL;
+ unsigned long list_flags;
+ struct cxl_memdev *memdev;
+ int i;
+
+ argc = parse_options(argc, argv, options, u, 0);
+ for (i = 0; i < argc; i++)
+ error("unknown parameter \"%s\"\n", argv[i]);
+
+ if (argc)
+ usage_with_options(u, options);
+
+ if (num_list_flags() == 0)
+ list.memdevs = true;
+
+ list_flags = listopts_to_flags();
+
+ cxl_memdev_foreach(ctx, memdev) {
+ struct json_object *jdev = NULL;
+
+ if (!util_cxl_memdev_filter(memdev, param.memdev))
+ continue;
+
+ if (list.memdevs) {
+ if (!jdevs) {
+ jdevs = json_object_new_array();
+ if (!jdevs) {
+ fail("\n");
+ continue;
+ }
+ }
+
+ jdev = util_cxl_memdev_to_json(memdev, list_flags);
+ if (!jdev) {
+ fail("\n");
+ continue;
+ }
+ json_object_array_add(jdevs, jdev);
+ }
+ }
+
+ if (jdevs)
+ util_display_json_array(stdout, jdevs, list_flags);
+
+ if (did_fail)
+ return -ENOMEM;
+ return 0;
+}
diff --git a/util/filter.c b/util/filter.c
index 8b4aad3..d81dade 100644
--- a/util/filter.c
+++ b/util/filter.c
@@ -12,6 +12,7 @@
#include <util/filter.h>
#include <ndctl/libndctl.h>
#include <daxctl/libdaxctl.h>
+#include <cxl/libcxl.h>
struct ndctl_bus *util_bus_filter(struct ndctl_bus *bus, const char *__ident)
{
@@ -339,6 +340,25 @@ struct daxctl_region *util_daxctl_region_filter(struct daxctl_region *region,
return NULL;
}
+struct cxl_memdev *util_cxl_memdev_filter(struct cxl_memdev *memdev,
+ const char *ident)
+{
+ int memdev_id;
+
+ if (!ident || strcmp(ident, "all") == 0)
+ return memdev;
+
+ if (strcmp(ident, cxl_memdev_get_devname(memdev)) == 0)
+ return memdev;
+
+ if ((sscanf(ident, "%d", &memdev_id) == 1
+ || sscanf(ident, "mem%d", &memdev_id) == 1)
+ && cxl_memdev_get_id(memdev) == memdev_id)
+ return memdev;
+
+ return NULL;
+}
+
enum ndctl_namespace_mode util_nsmode(const char *mode)
{
if (!mode)
diff --git a/util/json.c b/util/json.c
index a8d2412..3be3a92 100644
--- a/util/json.c
+++ b/util/json.c
@@ -9,6 +9,7 @@
#include <json-c/printbuf.h>
#include <ndctl/libndctl.h>
#include <daxctl/libdaxctl.h>
+#include <cxl/libcxl.h>
#include <ccan/array_size/array_size.h>
#include <ccan/short_types/short_types.h>
#include <ndctl.h>
@@ -1440,3 +1441,28 @@ struct json_object *util_badblock_rec_to_json(u64 block, u64 count,
json_object_put(jerr);
return NULL;
}
+
+struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
+ unsigned long flags)
+{
+ const char *devname = cxl_memdev_get_devname(memdev);
+ struct json_object *jdev, *jobj;
+
+ jdev = json_object_new_object();
+ if (!devname || !jdev)
+ return NULL;
+
+ jobj = json_object_new_string(devname);
+ if (jobj)
+ json_object_object_add(jdev, "memdev", jobj);
+
+ jobj = util_json_object_size(cxl_memdev_get_pmem_size(memdev), flags);
+ if (jobj)
+ json_object_object_add(jdev, "pmem_size", jobj);
+
+ jobj = util_json_object_size(cxl_memdev_get_ram_size(memdev), flags);
+ if (jobj)
+ json_object_object_add(jdev, "ram_size", jobj);
+
+ return jdev;
+}
diff --git a/.clang-format b/.clang-format
index 4e00fff..d2e77d0 100644
--- a/.clang-format
+++ b/.clang-format
@@ -77,6 +77,7 @@ ExperimentalAutoDetectBinPacking: false
# -e 's/\(.*foreach.*\)(.*/\1/' \
# | sort -u)
ForEachMacros:
+ - 'cxl_memdev_foreach'
- 'daxctl_dev_foreach'
- 'daxctl_mapping_foreach'
- 'daxctl_region_foreach'
diff --git a/.gitignore b/.gitignore
index 53512b2..6a97b92 100644
--- a/.gitignore
+++ b/.gitignore
@@ -16,9 +16,11 @@ Makefile.in
*.1
Documentation/daxctl/asciidoc.conf
Documentation/ndctl/asciidoc.conf
-Documentation/ndctl/attrs.adoc
+Documentation/cxl/asciidoc.conf
Documentation/daxctl/asciidoctor-extensions.rb
Documentation/ndctl/asciidoctor-extensions.rb
+Documentation/cxl/asciidoctor-extensions.rb
+Documentation/ndctl/attrs.adoc
.dirstamp
daxctl/config.h
daxctl/daxctl
diff --git a/Documentation/cxl/Makefile.am b/Documentation/cxl/Makefile.am
new file mode 100644
index 0000000..db98dd7
--- /dev/null
+++ b/Documentation/cxl/Makefile.am
@@ -0,0 +1,58 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020-2021 Intel Corporation. All rights reserved.
+
+if USE_ASCIIDOCTOR
+
+do_subst = sed -e 's,@Utility@,Cxl,g' -e's,@utility@,cxl,g'
+CONFFILE = asciidoctor-extensions.rb
+asciidoctor-extensions.rb: ../asciidoctor-extensions.rb.in
+ $(AM_V_GEN) $(do_subst) < $< > $@
+
+else
+
+do_subst = sed -e 's,UTILITY,cxl,g'
+CONFFILE = asciidoc.conf
+asciidoc.conf: ../asciidoc.conf.in
+ $(AM_V_GEN) $(do_subst) < $< > $@
+
+endif
+
+man1_MANS = \
+ cxl.1 \
+ cxl-list.1
+
+EXTRA_DIST = $(man1_MANS)
+
+CLEANFILES = $(man1_MANS)
+
+XML_DEPS = \
+ ../../version.m4 \
+ ../copyright.txt \
+ Makefile \
+ $(CONFFILE)
+
+RM ?= rm -f
+
+if USE_ASCIIDOCTOR
+
+%.1: %.txt $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@+ $@ && \
+ $(ASCIIDOC) -b manpage -d manpage -acompat-mode \
+ -I. -rasciidoctor-extensions \
+ -amansource=cxl -amanmanual="cxl Manual" \
+ -andctl_version=$(VERSION) -o $@+ $< && \
+ mv $@+ $@
+
+else
+
+%.xml: %.txt $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@+ $@ && \
+ $(ASCIIDOC) -b docbook -d manpage -f asciidoc.conf \
+ --unsafe -acxl_version=$(VERSION) -o $@+ $< && \
+ mv $@+ $@
+
+%.1: %.xml $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@ && \
+ $(XMLTO) -o . -m ../manpage-normal.xsl man $<
+
+endif
diff --git a/cxl/Makefile.am b/cxl/Makefile.am
new file mode 100644
index 0000000..98606b9
--- /dev/null
+++ b/cxl/Makefile.am
@@ -0,0 +1,21 @@
+include $(top_srcdir)/Makefile.am.in
+
+bin_PROGRAMS = cxl
+
+DISTCLEANFILES = config.h
+BUILT_SOURCES = config.h
+config.h: $(srcdir)/Makefile.am
+ $(AM_V_GEN) echo "/* Autogenerated by cxl/Makefile.am */" >$@
+
+cxl_SOURCES =\
+ cxl.c \
+ list.c \
+ ../util/json.c \
+ builtin.h
+
+cxl_LDADD =\
+ lib/libcxl.la \
+ ../libutil.a \
+ $(UUID_LIBS) \
+ $(KMOD_LIBS) \
+ $(JSON_LIBS)
diff --git a/cxl/lib/Makefile.am b/cxl/lib/Makefile.am
new file mode 100644
index 0000000..277f0cd
--- /dev/null
+++ b/cxl/lib/Makefile.am
@@ -0,0 +1,32 @@
+include $(top_srcdir)/Makefile.am.in
+
+%.pc: %.pc.in Makefile
+ $(SED_PROCESS)
+
+pkginclude_HEADERS = ../libcxl.h
+lib_LTLIBRARIES = libcxl.la
+
+libcxl_la_SOURCES =\
+ ../libcxl.h \
+ private.h \
+ ../../util/sysfs.c \
+ ../../util/sysfs.h \
+ ../../util/log.c \
+ ../../util/log.h \
+ libcxl.c
+
+libcxl_la_LIBADD =\
+ $(UUID_LIBS) \
+ $(KMOD_LIBS)
+
+EXTRA_DIST += libcxl.sym
+
+libcxl_la_LDFLAGS = $(AM_LDFLAGS) \
+ -version-info $(LIBCXL_CURRENT):$(LIBCXL_REVISION):$(LIBCXL_AGE) \
+ -Wl,--version-script=$(top_srcdir)/cxl/lib/libcxl.sym
+libcxl_la_DEPENDENCIES = libcxl.sym
+
+pkgconfigdir = $(libdir)/pkgconfig
+pkgconfig_DATA = libcxl.pc
+EXTRA_DIST += libcxl.pc.in
+CLEANFILES += libcxl.pc
diff --git a/cxl/lib/libcxl.pc.in b/cxl/lib/libcxl.pc.in
new file mode 100644
index 0000000..949fcdc
--- /dev/null
+++ b/cxl/lib/libcxl.pc.in
@@ -0,0 +1,11 @@
+prefix=@prefix@
+exec_prefix=@exec_prefix@
+libdir=@libdir@
+includedir=@includedir@
+
+Name: libcxl
+Description: Manage CXL devices
+Version: @VERSION@
+Libs: -L${libdir} -lcxl
+Libs.private:
+Cflags: -I${includedir}
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
new file mode 100644
index 0000000..2616e5c
--- /dev/null
+++ b/cxl/lib/libcxl.sym
@@ -0,0 +1,25 @@
+LIBCXL_1 {
+global:
+ cxl_get_userdata;
+ cxl_set_userdata;
+ cxl_get_private_data;
+ cxl_set_private_data;
+ cxl_ref;
+ cxl_get_log_priority;
+ cxl_set_log_fn;
+ cxl_unref;
+ cxl_set_log_priority;
+ cxl_new;
+ cxl_memdev_get_first;
+ cxl_memdev_get_next;
+ cxl_memdev_get_id;
+ cxl_memdev_get_devname;
+ cxl_memdev_get_major;
+ cxl_memdev_get_minor;
+ cxl_memdev_get_ctx;
+ cxl_memdev_get_pmem_size;
+ cxl_memdev_get_ram_size;
+ cxl_memdev_get_firmware_verison;
+local:
+ *;
+};
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* Re: [ndctl PATCH v5 02/16] cxl: add a cxl utility and libcxl library
2021-11-11 20:44 ` [ndctl PATCH v5 02/16] cxl: add a cxl utility and libcxl library Vishal Verma
2021-11-11 21:59 ` [ndctl PATCH v6 " Vishal Verma
@ 2021-12-08 5:12 ` Dan Williams
2021-12-09 20:23 ` [ndctl PATCH v6] " Vishal Verma
1 sibling, 1 reply; 31+ messages in thread
From: Dan Williams @ 2021-12-08 5:12 UTC (permalink / raw)
To: Vishal Verma; +Cc: linux-cxl, Ben Widawsky, Linux NVDIMM
On Thu, Nov 11, 2021 at 12:45 PM Vishal Verma <vishal.l.verma@intel.com> wrote:
>
> CXL - or Compute eXpress Link - is a new interconnect that extends PCIe
> to support a wide range of devices, including cache coherent memory
> expanders. As such, these devices can be new sources of 'persistent
> memory', and the 'ndctl' umbrella of tools and libraries needs to be able
> to interact with them.
>
> Add a new utility and library for managing these CXL memory devices. This
> is an initial bring-up for interacting with CXL devices, and only includes
> adding the utility and library infrastructure, parsing device information
> from sysfs for CXL devices, and providing a 'cxl-list' command to
> display this information in JSON formatted output.
>
> Cc: Ben Widawsky <ben.widawsky@intel.com>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Reviewed-by: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
[..]
> +int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx)
> +{
> + const struct option options[] = {
> + OPT_STRING('d', "memdev", ¶m.memdev, "memory device name",
> + "filter by CXL memory device name"),
> + OPT_BOOLEAN('D', "memdevs", &list.memdevs,
> + "include CXL memory device info"),
> + OPT_BOOLEAN('i', "idle", &list.idle, "include idle devices"),
> + OPT_BOOLEAN('u', "human", &list.human,
> + "use human friendly number formats "),
> + OPT_END(),
> + };
> + const char * const u[] = {
> + "cxl list [<options>]",
> + NULL
> + };
> + struct json_object *jdevs = NULL;
> + unsigned long list_flags;
> + struct cxl_memdev *memdev;
> + int i;
> +
> + argc = parse_options(argc, argv, options, u, 0);
> + for (i = 0; i < argc; i++)
> + error("unknown parameter \"%s\"\n", argv[i]);
> +
> + if (argc)
> + usage_with_options(u, options);
> +
> + if (num_list_flags() == 0)
> + list.memdevs = true;
It strikes me that we may regret this default. In the interest of
giving us flexibility to pick a new default later (i.e. to not grow
any maintenance dependencies on this default), let's fail for now if
no list options are specified. I expect "cxl list" to enumerate the
active and most relevant end user devices by default. For ndctl its
namespaces, but for cxl-cli I expect it should be regions by default
to show all the CXL address ranges. Likely it should be "--regions
--namespaces" by default since regions are the primary user relevant
devices for volatile memory and namespaces are the end user relevant
devices for persistent memory, but the "--regions" vs "--regions
--namespaces" decision can be made later.
^ permalink raw reply [flat|nested] 31+ messages in thread* [ndctl PATCH v6] cxl: add a cxl utility and libcxl library
2021-12-08 5:12 ` [ndctl PATCH v5 " Dan Williams
@ 2021-12-09 20:23 ` Vishal Verma
2021-12-09 20:40 ` Dan Williams
0 siblings, 1 reply; 31+ messages in thread
From: Vishal Verma @ 2021-12-09 20:23 UTC (permalink / raw)
To: linux-cxl; +Cc: nvdimm, Dan Williams, Ben Widawsky, Vishal Verma
CXL - or Compute eXpress Link - is a new interconnect that extends PCIe
to support a wide range of devices, including cache coherent memory
expanders. As such, these devices can be new sources of 'persistent
memory', and the 'ndctl' umbrella of tools and libraries needs to be able
to interact with them.
Add a new utility and library for managing these CXL memory devices. This
is an initial bring-up for interacting with CXL devices, and only includes
adding the utility and library infrastructure, parsing device information
from sysfs for CXL devices, and providing a 'cxl-list' command to
display this information in JSON formatted output.
Cc: Ben Widawsky <ben.widawsky@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
Changes in v6:
- Remove the cxl-list default of listing memdevs as we may want to list
regions by default, once that support is available. For now just error
out if memdevs are not explicitly asked for. (Dan)
Documentation/cxl/cxl-list.txt | 64 +++++
Documentation/cxl/cxl.txt | 34 +++
Documentation/cxl/human-option.txt | 8 +
Documentation/cxl/verbose-option.txt | 5 +
configure.ac | 3 +
Makefile.am | 8 +-
Makefile.am.in | 4 +
cxl/lib/private.h | 29 +++
cxl/lib/libcxl.c | 345 +++++++++++++++++++++++++++
cxl/builtin.h | 8 +
cxl/libcxl.h | 55 +++++
util/filter.h | 2 +
util/json.h | 3 +
util/main.h | 3 +
cxl/cxl.c | 96 ++++++++
cxl/list.c | 121 ++++++++++
util/filter.c | 20 ++
util/json.c | 26 ++
.clang-format | 1 +
.gitignore | 4 +-
Documentation/cxl/Makefile.am | 58 +++++
cxl/Makefile.am | 21 ++
cxl/lib/Makefile.am | 32 +++
cxl/lib/libcxl.pc.in | 11 +
cxl/lib/libcxl.sym | 25 ++
25 files changed, 982 insertions(+), 4 deletions(-)
create mode 100644 Documentation/cxl/cxl-list.txt
create mode 100644 Documentation/cxl/cxl.txt
create mode 100644 Documentation/cxl/human-option.txt
create mode 100644 Documentation/cxl/verbose-option.txt
create mode 100644 cxl/lib/private.h
create mode 100644 cxl/lib/libcxl.c
create mode 100644 cxl/builtin.h
create mode 100644 cxl/libcxl.h
create mode 100644 cxl/cxl.c
create mode 100644 cxl/list.c
create mode 100644 Documentation/cxl/Makefile.am
create mode 100644 cxl/Makefile.am
create mode 100644 cxl/lib/Makefile.am
create mode 100644 cxl/lib/libcxl.pc.in
create mode 100644 cxl/lib/libcxl.sym
diff --git a/Documentation/cxl/cxl-list.txt b/Documentation/cxl/cxl-list.txt
new file mode 100644
index 0000000..e761cfa
--- /dev/null
+++ b/Documentation/cxl/cxl-list.txt
@@ -0,0 +1,64 @@
+// SPDX-License-Identifier: GPL-2.0
+
+cxl-list(1)
+===========
+
+NAME
+----
+cxl-list - List CXL capable memory devices, and their attributes in json.
+
+SYNOPSIS
+--------
+[verse]
+'cxl list' [<options>]
+
+Walk the CXL capable device hierarchy in the system and list all device
+instances along with some of their major attributes.
+
+Options can be specified to limit the output to specific memdevs.
+By default, 'cxl list' with no options is equivalent to:
+[verse]
+cxl list --memdevs
+
+EXAMPLE
+-------
+----
+# cxl list --memdevs
+{
+ "memdev":"mem0",
+ "pmem_size":268435456,
+ "ram_size":0,
+}
+----
+
+OPTIONS
+-------
+-m::
+--memdev=::
+ Specify a cxl memory device name to filter the listing. For example:
+----
+# cxl list --memdev=mem0
+{
+ "memdev":"mem0",
+ "pmem_size":268435456,
+ "ram_size":0,
+}
+----
+
+-M::
+--memdevs::
+ Include all CXL memory devices in the listing
+
+-i::
+--idle::
+ Include idle (not enabled / zero-sized) devices in the listing
+
+include::human-option.txt[]
+
+include::verbose-option.txt[]
+
+include::../copyright.txt[]
+
+SEE ALSO
+--------
+linkcxl:ndctl-list[1]
diff --git a/Documentation/cxl/cxl.txt b/Documentation/cxl/cxl.txt
new file mode 100644
index 0000000..41a51c7
--- /dev/null
+++ b/Documentation/cxl/cxl.txt
@@ -0,0 +1,34 @@
+// SPDX-License-Identifier: GPL-2.0
+
+cxl(1)
+======
+
+NAME
+----
+cxl - Provides enumeration and provisioning commands for CXL platforms
+
+SYNOPSIS
+--------
+[verse]
+'cxl' [--version] [--help] COMMAND [ARGS]
+
+OPTIONS
+-------
+-v::
+--version::
+ Display the version of the 'cxl' utility.
+
+-h::
+--help::
+ Run the 'cxl help' command.
+
+DESCRIPTION
+-----------
+The cxl utility provides enumeration and provisioning commands for
+the CXL devices managed by the Linux kernel.
+
+include::../copyright.txt[]
+
+SEE ALSO
+--------
+linkcxl:ndctl[1]
diff --git a/Documentation/cxl/human-option.txt b/Documentation/cxl/human-option.txt
new file mode 100644
index 0000000..2f4de7a
--- /dev/null
+++ b/Documentation/cxl/human-option.txt
@@ -0,0 +1,8 @@
+// SPDX-License-Identifier: GPL-2.0
+
+-u::
+--human::
+ By default the command will output machine-friendly raw-integer
+ data. Instead, with this flag, numbers representing storage size
+ will be formatted as human readable strings with units, other
+ fields are converted to hexadecimal strings.
diff --git a/Documentation/cxl/verbose-option.txt b/Documentation/cxl/verbose-option.txt
new file mode 100644
index 0000000..cb62c8e
--- /dev/null
+++ b/Documentation/cxl/verbose-option.txt
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+-v::
+--verbose::
+ Emit more debug messages
diff --git a/configure.ac b/configure.ac
index dc39dbe..dadae0a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -222,12 +222,15 @@ AC_CONFIG_HEADERS(config.h)
AC_CONFIG_FILES([
Makefile
daxctl/lib/Makefile
+ cxl/lib/Makefile
ndctl/lib/Makefile
ndctl/Makefile
daxctl/Makefile
+ cxl/Makefile
test/Makefile
Documentation/ndctl/Makefile
Documentation/daxctl/Makefile
+ Documentation/cxl/Makefile
])
AC_OUTPUT
diff --git a/Makefile.am b/Makefile.am
index 60a1998..428fd40 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,9 +1,9 @@
include Makefile.am.in
ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS}
-SUBDIRS = . daxctl/lib ndctl/lib ndctl daxctl
+SUBDIRS = . cxl/lib daxctl/lib ndctl/lib cxl ndctl daxctl
if ENABLE_DOCS
-SUBDIRS += Documentation/ndctl Documentation/daxctl
+SUBDIRS += Documentation/ndctl Documentation/daxctl Documentation/cxl
endif
SUBDIRS += test
@@ -87,4 +87,6 @@ libutil_a_SOURCES = \
util/filter.h \
util/bitmap.h
-nobase_include_HEADERS = daxctl/libdaxctl.h
+nobase_include_HEADERS = \
+ daxctl/libdaxctl.h \
+ cxl/libcxl.h
diff --git a/Makefile.am.in b/Makefile.am.in
index bdceda9..aaeee53 100644
--- a/Makefile.am.in
+++ b/Makefile.am.in
@@ -42,3 +42,7 @@ LIBNDCTL_AGE=19
LIBDAXCTL_CURRENT=6
LIBDAXCTL_REVISION=0
LIBDAXCTL_AGE=5
+
+LIBCXL_CURRENT=1
+LIBCXL_REVISION=0
+LIBCXL_AGE=0
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
new file mode 100644
index 0000000..fc88fa1
--- /dev/null
+++ b/cxl/lib/private.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+/* Copyright (C) 2020-2021, Intel Corporation. All rights reserved. */
+#ifndef _LIBCXL_PRIVATE_H_
+#define _LIBCXL_PRIVATE_H_
+
+#include <libkmod.h>
+
+#define CXL_EXPORT __attribute__ ((visibility("default")))
+
+struct cxl_memdev {
+ int id, major, minor;
+ void *dev_buf;
+ size_t buf_len;
+ char *dev_path;
+ char *firmware_version;
+ struct cxl_ctx *ctx;
+ struct list_node list;
+ unsigned long long pmem_size;
+ unsigned long long ram_size;
+ int payload_max;
+ struct kmod_module *module;
+};
+
+static inline int check_kmod(struct kmod_ctx *kmod_ctx)
+{
+ return kmod_ctx ? 0 : -ENXIO;
+}
+
+#endif /* _LIBCXL_PRIVATE_H_ */
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
new file mode 100644
index 0000000..c15e987
--- /dev/null
+++ b/cxl/lib/libcxl.c
@@ -0,0 +1,345 @@
+// SPDX-License-Identifier: LGPL-2.1
+// Copyright (C) 2020-2021, Intel Corporation. All rights reserved.
+#include <stdio.h>
+#include <errno.h>
+#include <limits.h>
+#include <libgen.h>
+#include <stdlib.h>
+#include <dirent.h>
+#include <unistd.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/sysmacros.h>
+#include <uuid/uuid.h>
+#include <ccan/list/list.h>
+#include <ccan/array_size/array_size.h>
+
+#include <util/log.h>
+#include <util/sysfs.h>
+#include <util/bitmap.h>
+#include <cxl/libcxl.h>
+#include "private.h"
+
+/**
+ * struct cxl_ctx - library user context to find "nd" instances
+ *
+ * Instantiate with cxl_new(), which takes an initial reference. Free
+ * the context by dropping the reference count to zero with
+ * cxl_unref(), or take additional references with cxl_ref()
+ * @timeout: default library timeout in milliseconds
+ */
+struct cxl_ctx {
+ /* log_ctx must be first member for cxl_set_log_fn compat */
+ struct log_ctx ctx;
+ int refcount;
+ void *userdata;
+ int memdevs_init;
+ struct list_head memdevs;
+ struct kmod_ctx *kmod_ctx;
+ void *private_data;
+};
+
+static void free_memdev(struct cxl_memdev *memdev, struct list_head *head)
+{
+ if (head)
+ list_del_from(head, &memdev->list);
+ kmod_module_unref(memdev->module);
+ free(memdev->firmware_version);
+ free(memdev->dev_buf);
+ free(memdev->dev_path);
+ free(memdev);
+}
+
+/**
+ * cxl_get_userdata - retrieve stored data pointer from library context
+ * @ctx: cxl library context
+ *
+ * This might be useful to access from callbacks like a custom logging
+ * function.
+ */
+CXL_EXPORT void *cxl_get_userdata(struct cxl_ctx *ctx)
+{
+ if (ctx == NULL)
+ return NULL;
+ return ctx->userdata;
+}
+
+/**
+ * cxl_set_userdata - store custom @userdata in the library context
+ * @ctx: cxl library context
+ * @userdata: data pointer
+ */
+CXL_EXPORT void cxl_set_userdata(struct cxl_ctx *ctx, void *userdata)
+{
+ if (ctx == NULL)
+ return;
+ ctx->userdata = userdata;
+}
+
+CXL_EXPORT void cxl_set_private_data(struct cxl_ctx *ctx, void *data)
+{
+ ctx->private_data = data;
+}
+
+CXL_EXPORT void *cxl_get_private_data(struct cxl_ctx *ctx)
+{
+ return ctx->private_data;
+}
+
+/**
+ * cxl_new - instantiate a new library context
+ * @ctx: context to establish
+ *
+ * Returns zero on success and stores an opaque pointer in ctx. The
+ * context is freed by cxl_unref(), i.e. cxl_new() implies an
+ * internal cxl_ref().
+ */
+CXL_EXPORT int cxl_new(struct cxl_ctx **ctx)
+{
+ struct kmod_ctx *kmod_ctx;
+ struct cxl_ctx *c;
+ int rc = 0;
+
+ c = calloc(1, sizeof(struct cxl_ctx));
+ if (!c)
+ return -ENOMEM;
+
+ kmod_ctx = kmod_new(NULL, NULL);
+ if (check_kmod(kmod_ctx) != 0) {
+ rc = -ENXIO;
+ goto out;
+ }
+
+ c->refcount = 1;
+ log_init(&c->ctx, "libcxl", "CXL_LOG");
+ info(c, "ctx %p created\n", c);
+ dbg(c, "log_priority=%d\n", c->ctx.log_priority);
+ *ctx = c;
+ list_head_init(&c->memdevs);
+ c->kmod_ctx = kmod_ctx;
+
+ return 0;
+out:
+ free(c);
+ return rc;
+}
+
+/**
+ * cxl_ref - take an additional reference on the context
+ * @ctx: context established by cxl_new()
+ */
+CXL_EXPORT struct cxl_ctx *cxl_ref(struct cxl_ctx *ctx)
+{
+ if (ctx == NULL)
+ return NULL;
+ ctx->refcount++;
+ return ctx;
+}
+
+/**
+ * cxl_unref - drop a context reference count
+ * @ctx: context established by cxl_new()
+ *
+ * Drop a reference and if the resulting reference count is 0 destroy
+ * the context.
+ */
+CXL_EXPORT void cxl_unref(struct cxl_ctx *ctx)
+{
+ struct cxl_memdev *memdev, *_d;
+
+ if (ctx == NULL)
+ return;
+ ctx->refcount--;
+ if (ctx->refcount > 0)
+ return;
+
+ list_for_each_safe(&ctx->memdevs, memdev, _d, list)
+ free_memdev(memdev, &ctx->memdevs);
+
+ kmod_unref(ctx->kmod_ctx);
+ info(ctx, "context %p released\n", ctx);
+ free(ctx);
+}
+
+/**
+ * cxl_set_log_fn - override default log routine
+ * @ctx: cxl library context
+ * @log_fn: function to be called for logging messages
+ *
+ * The built-in logging writes to stderr. It can be overridden by a
+ * custom function, to plug log messages into the user's logging
+ * functionality.
+ */
+CXL_EXPORT void cxl_set_log_fn(struct cxl_ctx *ctx,
+ void (*cxl_log_fn)(struct cxl_ctx *ctx, int priority,
+ const char *file, int line, const char *fn,
+ const char *format, va_list args))
+{
+ ctx->ctx.log_fn = (log_fn) cxl_log_fn;
+ info(ctx, "custom logging function %p registered\n", cxl_log_fn);
+}
+
+/**
+ * cxl_get_log_priority - retrieve current library loglevel (syslog)
+ * @ctx: cxl library context
+ */
+CXL_EXPORT int cxl_get_log_priority(struct cxl_ctx *ctx)
+{
+ return ctx->ctx.log_priority;
+}
+
+/**
+ * cxl_set_log_priority - set log verbosity
+ * @priority: from syslog.h, LOG_ERR, LOG_INFO, LOG_DEBUG
+ *
+ * Note: LOG_DEBUG requires library be built with "configure --enable-debug"
+ */
+CXL_EXPORT void cxl_set_log_priority(struct cxl_ctx *ctx, int priority)
+{
+ ctx->ctx.log_priority = priority;
+}
+
+static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
+{
+ const char *devname = devpath_to_devname(cxlmem_base);
+ char *path = calloc(1, strlen(cxlmem_base) + 100);
+ struct cxl_ctx *ctx = parent;
+ struct cxl_memdev *memdev, *memdev_dup;
+ char buf[SYSFS_ATTR_SIZE];
+ struct stat st;
+
+ if (!path)
+ return NULL;
+ dbg(ctx, "%s: base: \'%s\'\n", devname, cxlmem_base);
+
+ memdev = calloc(1, sizeof(*memdev));
+ if (!memdev)
+ goto err_dev;
+ memdev->id = id;
+ memdev->ctx = ctx;
+
+ sprintf(path, "/dev/cxl/%s", devname);
+ if (stat(path, &st) < 0)
+ goto err_read;
+ memdev->major = major(st.st_rdev);
+ memdev->minor = minor(st.st_rdev);
+
+ sprintf(path, "%s/pmem/size", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+ memdev->pmem_size = strtoull(buf, NULL, 0);
+
+ sprintf(path, "%s/ram/size", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+ memdev->ram_size = strtoull(buf, NULL, 0);
+
+ sprintf(path, "%s/payload_max", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+ memdev->payload_max = strtoull(buf, NULL, 0);
+ if (memdev->payload_max < 0)
+ goto err_read;
+
+ memdev->dev_path = strdup(cxlmem_base);
+ if (!memdev->dev_path)
+ goto err_read;
+
+ sprintf(path, "%s/firmware_version", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+
+ memdev->firmware_version = strdup(buf);
+ if (!memdev->firmware_version)
+ goto err_read;
+
+ memdev->dev_buf = calloc(1, strlen(cxlmem_base) + 50);
+ if (!memdev->dev_buf)
+ goto err_read;
+ memdev->buf_len = strlen(cxlmem_base) + 50;
+
+ cxl_memdev_foreach(ctx, memdev_dup)
+ if (memdev_dup->id == memdev->id) {
+ free_memdev(memdev, NULL);
+ free(path);
+ return memdev_dup;
+ }
+
+ list_add(&ctx->memdevs, &memdev->list);
+ free(path);
+ return memdev;
+
+ err_read:
+ free(memdev->firmware_version);
+ free(memdev->dev_buf);
+ free(memdev->dev_path);
+ free(memdev);
+ err_dev:
+ free(path);
+ return NULL;
+}
+
+static void cxl_memdevs_init(struct cxl_ctx *ctx)
+{
+ if (ctx->memdevs_init)
+ return;
+
+ ctx->memdevs_init = 1;
+
+ sysfs_device_parse(ctx, "/sys/bus/cxl/devices", "mem", ctx,
+ add_cxl_memdev);
+}
+
+CXL_EXPORT struct cxl_ctx *cxl_memdev_get_ctx(struct cxl_memdev *memdev)
+{
+ return memdev->ctx;
+}
+
+CXL_EXPORT struct cxl_memdev *cxl_memdev_get_first(struct cxl_ctx *ctx)
+{
+ cxl_memdevs_init(ctx);
+
+ return list_top(&ctx->memdevs, struct cxl_memdev, list);
+}
+
+CXL_EXPORT struct cxl_memdev *cxl_memdev_get_next(struct cxl_memdev *memdev)
+{
+ struct cxl_ctx *ctx = memdev->ctx;
+
+ return list_next(&ctx->memdevs, memdev, list);
+}
+
+CXL_EXPORT int cxl_memdev_get_id(struct cxl_memdev *memdev)
+{
+ return memdev->id;
+}
+
+CXL_EXPORT const char *cxl_memdev_get_devname(struct cxl_memdev *memdev)
+{
+ return devpath_to_devname(memdev->dev_path);
+}
+
+CXL_EXPORT int cxl_memdev_get_major(struct cxl_memdev *memdev)
+{
+ return memdev->major;
+}
+
+CXL_EXPORT int cxl_memdev_get_minor(struct cxl_memdev *memdev)
+{
+ return memdev->minor;
+}
+
+CXL_EXPORT unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev)
+{
+ return memdev->pmem_size;
+}
+
+CXL_EXPORT unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev)
+{
+ return memdev->ram_size;
+}
+
+CXL_EXPORT const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev)
+{
+ return memdev->firmware_version;
+}
diff --git a/cxl/builtin.h b/cxl/builtin.h
new file mode 100644
index 0000000..3797f98
--- /dev/null
+++ b/cxl/builtin.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
+#ifndef _CXL_BUILTIN_H_
+#define _CXL_BUILTIN_H_
+
+struct cxl_ctx;
+int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx);
+#endif /* _CXL_BUILTIN_H_ */
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
new file mode 100644
index 0000000..fd06790
--- /dev/null
+++ b/cxl/libcxl.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+/* Copyright (C) 2020-2021, Intel Corporation. All rights reserved. */
+#ifndef _LIBCXL_H_
+#define _LIBCXL_H_
+
+#include <stdarg.h>
+#include <unistd.h>
+
+#ifdef HAVE_UUID
+#include <uuid/uuid.h>
+#else
+typedef unsigned char uuid_t[16];
+#endif
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct cxl_ctx;
+struct cxl_ctx *cxl_ref(struct cxl_ctx *ctx);
+void cxl_unref(struct cxl_ctx *ctx);
+int cxl_new(struct cxl_ctx **ctx);
+void cxl_set_log_fn(struct cxl_ctx *ctx,
+ void (*log_fn)(struct cxl_ctx *ctx, int priority,
+ const char *file, int line, const char *fn,
+ const char *format, va_list args));
+int cxl_get_log_priority(struct cxl_ctx *ctx);
+void cxl_set_log_priority(struct cxl_ctx *ctx, int priority);
+void cxl_set_userdata(struct cxl_ctx *ctx, void *userdata);
+void *cxl_get_userdata(struct cxl_ctx *ctx);
+void cxl_set_private_data(struct cxl_ctx *ctx, void *data);
+void *cxl_get_private_data(struct cxl_ctx *ctx);
+
+struct cxl_memdev;
+struct cxl_memdev *cxl_memdev_get_first(struct cxl_ctx *ctx);
+struct cxl_memdev *cxl_memdev_get_next(struct cxl_memdev *memdev);
+int cxl_memdev_get_id(struct cxl_memdev *memdev);
+const char *cxl_memdev_get_devname(struct cxl_memdev *memdev);
+int cxl_memdev_get_major(struct cxl_memdev *memdev);
+int cxl_memdev_get_minor(struct cxl_memdev *memdev);
+struct cxl_ctx *cxl_memdev_get_ctx(struct cxl_memdev *memdev);
+unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev);
+unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev);
+const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev);
+
+#define cxl_memdev_foreach(ctx, memdev) \
+ for (memdev = cxl_memdev_get_first(ctx); \
+ memdev != NULL; \
+ memdev = cxl_memdev_get_next(memdev))
+
+#ifdef __cplusplus
+} /* extern "C" */
+#endif
+
+#endif
diff --git a/util/filter.h b/util/filter.h
index 1e1a41c..9a80d65 100644
--- a/util/filter.h
+++ b/util/filter.h
@@ -29,6 +29,8 @@ struct daxctl_dev *util_daxctl_dev_filter(struct daxctl_dev *dev,
const char *ident);
struct daxctl_region *util_daxctl_region_filter(struct daxctl_region *region,
const char *ident);
+struct cxl_memdev *util_cxl_memdev_filter(struct cxl_memdev *memdev,
+ const char *ident);
enum ndctl_namespace_mode util_nsmode(const char *mode);
const char *util_nsmode_name(enum ndctl_namespace_mode mode);
diff --git a/util/json.h b/util/json.h
index 0f09e36..91918c8 100644
--- a/util/json.h
+++ b/util/json.h
@@ -55,4 +55,7 @@ struct json_object *util_dimm_health_to_json(struct ndctl_dimm *dimm);
struct json_object *util_dimm_firmware_to_json(struct ndctl_dimm *dimm,
unsigned long flags);
struct json_object *util_region_capabilities_to_json(struct ndctl_region *region);
+struct cxl_memdev;
+struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
+ unsigned long flags);
#endif /* __NDCTL_JSON_H__ */
diff --git a/util/main.h b/util/main.h
index c89a843..80b55c4 100644
--- a/util/main.h
+++ b/util/main.h
@@ -10,16 +10,19 @@
enum program {
PROG_NDCTL,
PROG_DAXCTL,
+ PROG_CXL,
};
struct ndctl_ctx;
struct daxctl_ctx;
+struct cxl_ctx;
struct cmd_struct {
const char *cmd;
union {
int (*n_fn)(int, const char **, struct ndctl_ctx *ctx);
int (*d_fn)(int, const char **, struct daxctl_ctx *ctx);
+ int (*c_fn)(int, const char **, struct cxl_ctx *ctx);
};
};
diff --git a/cxl/cxl.c b/cxl/cxl.c
new file mode 100644
index 0000000..a7725f8
--- /dev/null
+++ b/cxl/cxl.c
@@ -0,0 +1,96 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
+/* Copyright (C) 2005 Andreas Ericsson. All rights reserved. */
+
+/* originally copied from perf and git */
+
+#include <stdio.h>
+#include <errno.h>
+#include <string.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <cxl/libcxl.h>
+#include <util/parse-options.h>
+#include <ccan/array_size/array_size.h>
+
+#include <util/strbuf.h>
+#include <util/util.h>
+#include <util/main.h>
+#include <cxl/builtin.h>
+
+const char cxl_usage_string[] = "cxl [--version] [--help] COMMAND [ARGS]";
+const char cxl_more_info_string[] =
+ "See 'cxl help COMMAND' for more information on a specific command.\n"
+ " cxl --list-cmds to see all available commands";
+
+static int cmd_version(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ printf("%s\n", VERSION);
+ return 0;
+}
+
+static int cmd_help(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ const char * const builtin_help_subcommands[] = {
+ "list",
+ NULL,
+ };
+ struct option builtin_help_options[] = {
+ OPT_END(),
+ };
+ const char *builtin_help_usage[] = {
+ "cxl help [command]",
+ NULL
+ };
+
+ argc = parse_options_subcommand(argc, argv, builtin_help_options,
+ builtin_help_subcommands, builtin_help_usage, 0);
+
+ if (!argv[0]) {
+ printf("\n usage: %s\n\n", cxl_usage_string);
+ printf("\n %s\n\n", cxl_more_info_string);
+ return 0;
+ }
+
+ return help_show_man_page(argv[0], "cxl", "CXL_MAN_VIEWER");
+}
+
+static struct cmd_struct commands[] = {
+ { "version", .c_fn = cmd_version },
+ { "list", .c_fn = cmd_list },
+ { "help", .c_fn = cmd_help },
+};
+
+int main(int argc, const char **argv)
+{
+ struct cxl_ctx *ctx;
+ int rc;
+
+ /* Look for flags.. */
+ argv++;
+ argc--;
+ main_handle_options(&argv, &argc, cxl_usage_string, commands,
+ ARRAY_SIZE(commands));
+
+ if (argc > 0) {
+ if (!prefixcmp(argv[0], "--"))
+ argv[0] += 2;
+ } else {
+ /* The user didn't specify a command; give them help */
+ printf("\n usage: %s\n\n", cxl_usage_string);
+ printf("\n %s\n\n", cxl_more_info_string);
+ goto out;
+ }
+
+ rc = cxl_new(&ctx);
+ if (rc)
+ goto out;
+ main_handle_internal_command(argc, argv, ctx, commands,
+ ARRAY_SIZE(commands), PROG_CXL);
+ cxl_unref(ctx);
+ fprintf(stderr, "Unknown command: '%s'\n", argv[0]);
+out:
+ return 1;
+}
diff --git a/cxl/list.c b/cxl/list.c
new file mode 100644
index 0000000..69201d9
--- /dev/null
+++ b/cxl/list.c
@@ -0,0 +1,121 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
+#include <stdio.h>
+#include <errno.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <limits.h>
+#include <util/json.h>
+#include <util/filter.h>
+#include <json-c/json.h>
+#include <cxl/libcxl.h>
+#include <util/parse-options.h>
+#include <ccan/array_size/array_size.h>
+
+static struct {
+ bool memdevs;
+ bool idle;
+ bool human;
+} list;
+
+static unsigned long listopts_to_flags(void)
+{
+ unsigned long flags = 0;
+
+ if (list.idle)
+ flags |= UTIL_JSON_IDLE;
+ if (list.human)
+ flags |= UTIL_JSON_HUMAN;
+ return flags;
+}
+
+static struct {
+ const char *memdev;
+} param;
+
+static int did_fail;
+
+#define fail(fmt, ...) \
+do { \
+ did_fail = 1; \
+ fprintf(stderr, "cxl-%s:%s:%d: " fmt, \
+ VERSION, __func__, __LINE__, ##__VA_ARGS__); \
+} while (0)
+
+static int num_list_flags(void)
+{
+ return list.memdevs;
+}
+
+int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ const struct option options[] = {
+ OPT_STRING('m', "memdev", ¶m.memdev, "memory device name",
+ "filter by CXL memory device name"),
+ OPT_BOOLEAN('M', "memdevs", &list.memdevs,
+ "include CXL memory device info"),
+ OPT_BOOLEAN('i', "idle", &list.idle, "include idle devices"),
+ OPT_BOOLEAN('u', "human", &list.human,
+ "use human friendly number formats "),
+ OPT_END(),
+ };
+ const char * const u[] = {
+ "cxl list [<options>]",
+ NULL
+ };
+ struct json_object *jdevs = NULL;
+ unsigned long list_flags;
+ struct cxl_memdev *memdev;
+ int i;
+
+ argc = parse_options(argc, argv, options, u, 0);
+ for (i = 0; i < argc; i++)
+ error("unknown parameter \"%s\"\n", argv[i]);
+
+ if (argc)
+ usage_with_options(u, options);
+
+ if (num_list_flags() == 0) {
+ /*
+ * TODO: We likely want to list regions by default if nothing
+ * was explicitly asked for. But until we have region support,
+ * print this error asking for devices explicitly.
+ * Once region support is added, this TODO can be removed.
+ */
+ fprintf(stderr,
+ "please specify entities to list, e.g. using -m/-M\n");
+ }
+
+ list_flags = listopts_to_flags();
+
+ cxl_memdev_foreach(ctx, memdev) {
+ struct json_object *jdev = NULL;
+
+ if (!util_cxl_memdev_filter(memdev, param.memdev))
+ continue;
+
+ if (list.memdevs) {
+ if (!jdevs) {
+ jdevs = json_object_new_array();
+ if (!jdevs) {
+ fail("\n");
+ continue;
+ }
+ }
+
+ jdev = util_cxl_memdev_to_json(memdev, list_flags);
+ if (!jdev) {
+ fail("\n");
+ continue;
+ }
+ json_object_array_add(jdevs, jdev);
+ }
+ }
+
+ if (jdevs)
+ util_display_json_array(stdout, jdevs, list_flags);
+
+ if (did_fail)
+ return -ENOMEM;
+ return 0;
+}
diff --git a/util/filter.c b/util/filter.c
index 8b4aad3..d81dade 100644
--- a/util/filter.c
+++ b/util/filter.c
@@ -12,6 +12,7 @@
#include <util/filter.h>
#include <ndctl/libndctl.h>
#include <daxctl/libdaxctl.h>
+#include <cxl/libcxl.h>
struct ndctl_bus *util_bus_filter(struct ndctl_bus *bus, const char *__ident)
{
@@ -339,6 +340,25 @@ struct daxctl_region *util_daxctl_region_filter(struct daxctl_region *region,
return NULL;
}
+struct cxl_memdev *util_cxl_memdev_filter(struct cxl_memdev *memdev,
+ const char *ident)
+{
+ int memdev_id;
+
+ if (!ident || strcmp(ident, "all") == 0)
+ return memdev;
+
+ if (strcmp(ident, cxl_memdev_get_devname(memdev)) == 0)
+ return memdev;
+
+ if ((sscanf(ident, "%d", &memdev_id) == 1
+ || sscanf(ident, "mem%d", &memdev_id) == 1)
+ && cxl_memdev_get_id(memdev) == memdev_id)
+ return memdev;
+
+ return NULL;
+}
+
enum ndctl_namespace_mode util_nsmode(const char *mode)
{
if (!mode)
diff --git a/util/json.c b/util/json.c
index a8d2412..3be3a92 100644
--- a/util/json.c
+++ b/util/json.c
@@ -9,6 +9,7 @@
#include <json-c/printbuf.h>
#include <ndctl/libndctl.h>
#include <daxctl/libdaxctl.h>
+#include <cxl/libcxl.h>
#include <ccan/array_size/array_size.h>
#include <ccan/short_types/short_types.h>
#include <ndctl.h>
@@ -1440,3 +1441,28 @@ struct json_object *util_badblock_rec_to_json(u64 block, u64 count,
json_object_put(jerr);
return NULL;
}
+
+struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
+ unsigned long flags)
+{
+ const char *devname = cxl_memdev_get_devname(memdev);
+ struct json_object *jdev, *jobj;
+
+ jdev = json_object_new_object();
+ if (!devname || !jdev)
+ return NULL;
+
+ jobj = json_object_new_string(devname);
+ if (jobj)
+ json_object_object_add(jdev, "memdev", jobj);
+
+ jobj = util_json_object_size(cxl_memdev_get_pmem_size(memdev), flags);
+ if (jobj)
+ json_object_object_add(jdev, "pmem_size", jobj);
+
+ jobj = util_json_object_size(cxl_memdev_get_ram_size(memdev), flags);
+ if (jobj)
+ json_object_object_add(jdev, "ram_size", jobj);
+
+ return jdev;
+}
diff --git a/.clang-format b/.clang-format
index 4e00fff..d2e77d0 100644
--- a/.clang-format
+++ b/.clang-format
@@ -77,6 +77,7 @@ ExperimentalAutoDetectBinPacking: false
# -e 's/\(.*foreach.*\)(.*/\1/' \
# | sort -u)
ForEachMacros:
+ - 'cxl_memdev_foreach'
- 'daxctl_dev_foreach'
- 'daxctl_mapping_foreach'
- 'daxctl_region_foreach'
diff --git a/.gitignore b/.gitignore
index 53512b2..6a97b92 100644
--- a/.gitignore
+++ b/.gitignore
@@ -16,9 +16,11 @@ Makefile.in
*.1
Documentation/daxctl/asciidoc.conf
Documentation/ndctl/asciidoc.conf
-Documentation/ndctl/attrs.adoc
+Documentation/cxl/asciidoc.conf
Documentation/daxctl/asciidoctor-extensions.rb
Documentation/ndctl/asciidoctor-extensions.rb
+Documentation/cxl/asciidoctor-extensions.rb
+Documentation/ndctl/attrs.adoc
.dirstamp
daxctl/config.h
daxctl/daxctl
diff --git a/Documentation/cxl/Makefile.am b/Documentation/cxl/Makefile.am
new file mode 100644
index 0000000..db98dd7
--- /dev/null
+++ b/Documentation/cxl/Makefile.am
@@ -0,0 +1,58 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020-2021 Intel Corporation. All rights reserved.
+
+if USE_ASCIIDOCTOR
+
+do_subst = sed -e 's,@Utility@,Cxl,g' -e's,@utility@,cxl,g'
+CONFFILE = asciidoctor-extensions.rb
+asciidoctor-extensions.rb: ../asciidoctor-extensions.rb.in
+ $(AM_V_GEN) $(do_subst) < $< > $@
+
+else
+
+do_subst = sed -e 's,UTILITY,cxl,g'
+CONFFILE = asciidoc.conf
+asciidoc.conf: ../asciidoc.conf.in
+ $(AM_V_GEN) $(do_subst) < $< > $@
+
+endif
+
+man1_MANS = \
+ cxl.1 \
+ cxl-list.1
+
+EXTRA_DIST = $(man1_MANS)
+
+CLEANFILES = $(man1_MANS)
+
+XML_DEPS = \
+ ../../version.m4 \
+ ../copyright.txt \
+ Makefile \
+ $(CONFFILE)
+
+RM ?= rm -f
+
+if USE_ASCIIDOCTOR
+
+%.1: %.txt $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@+ $@ && \
+ $(ASCIIDOC) -b manpage -d manpage -acompat-mode \
+ -I. -rasciidoctor-extensions \
+ -amansource=cxl -amanmanual="cxl Manual" \
+ -andctl_version=$(VERSION) -o $@+ $< && \
+ mv $@+ $@
+
+else
+
+%.xml: %.txt $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@+ $@ && \
+ $(ASCIIDOC) -b docbook -d manpage -f asciidoc.conf \
+ --unsafe -acxl_version=$(VERSION) -o $@+ $< && \
+ mv $@+ $@
+
+%.1: %.xml $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@ && \
+ $(XMLTO) -o . -m ../manpage-normal.xsl man $<
+
+endif
diff --git a/cxl/Makefile.am b/cxl/Makefile.am
new file mode 100644
index 0000000..98606b9
--- /dev/null
+++ b/cxl/Makefile.am
@@ -0,0 +1,21 @@
+include $(top_srcdir)/Makefile.am.in
+
+bin_PROGRAMS = cxl
+
+DISTCLEANFILES = config.h
+BUILT_SOURCES = config.h
+config.h: $(srcdir)/Makefile.am
+ $(AM_V_GEN) echo "/* Autogenerated by cxl/Makefile.am */" >$@
+
+cxl_SOURCES =\
+ cxl.c \
+ list.c \
+ ../util/json.c \
+ builtin.h
+
+cxl_LDADD =\
+ lib/libcxl.la \
+ ../libutil.a \
+ $(UUID_LIBS) \
+ $(KMOD_LIBS) \
+ $(JSON_LIBS)
diff --git a/cxl/lib/Makefile.am b/cxl/lib/Makefile.am
new file mode 100644
index 0000000..277f0cd
--- /dev/null
+++ b/cxl/lib/Makefile.am
@@ -0,0 +1,32 @@
+include $(top_srcdir)/Makefile.am.in
+
+%.pc: %.pc.in Makefile
+ $(SED_PROCESS)
+
+pkginclude_HEADERS = ../libcxl.h
+lib_LTLIBRARIES = libcxl.la
+
+libcxl_la_SOURCES =\
+ ../libcxl.h \
+ private.h \
+ ../../util/sysfs.c \
+ ../../util/sysfs.h \
+ ../../util/log.c \
+ ../../util/log.h \
+ libcxl.c
+
+libcxl_la_LIBADD =\
+ $(UUID_LIBS) \
+ $(KMOD_LIBS)
+
+EXTRA_DIST += libcxl.sym
+
+libcxl_la_LDFLAGS = $(AM_LDFLAGS) \
+ -version-info $(LIBCXL_CURRENT):$(LIBCXL_REVISION):$(LIBCXL_AGE) \
+ -Wl,--version-script=$(top_srcdir)/cxl/lib/libcxl.sym
+libcxl_la_DEPENDENCIES = libcxl.sym
+
+pkgconfigdir = $(libdir)/pkgconfig
+pkgconfig_DATA = libcxl.pc
+EXTRA_DIST += libcxl.pc.in
+CLEANFILES += libcxl.pc
diff --git a/cxl/lib/libcxl.pc.in b/cxl/lib/libcxl.pc.in
new file mode 100644
index 0000000..949fcdc
--- /dev/null
+++ b/cxl/lib/libcxl.pc.in
@@ -0,0 +1,11 @@
+prefix=@prefix@
+exec_prefix=@exec_prefix@
+libdir=@libdir@
+includedir=@includedir@
+
+Name: libcxl
+Description: Manage CXL devices
+Version: @VERSION@
+Libs: -L${libdir} -lcxl
+Libs.private:
+Cflags: -I${includedir}
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
new file mode 100644
index 0000000..2616e5c
--- /dev/null
+++ b/cxl/lib/libcxl.sym
@@ -0,0 +1,25 @@
+LIBCXL_1 {
+global:
+ cxl_get_userdata;
+ cxl_set_userdata;
+ cxl_get_private_data;
+ cxl_set_private_data;
+ cxl_ref;
+ cxl_get_log_priority;
+ cxl_set_log_fn;
+ cxl_unref;
+ cxl_set_log_priority;
+ cxl_new;
+ cxl_memdev_get_first;
+ cxl_memdev_get_next;
+ cxl_memdev_get_id;
+ cxl_memdev_get_devname;
+ cxl_memdev_get_major;
+ cxl_memdev_get_minor;
+ cxl_memdev_get_ctx;
+ cxl_memdev_get_pmem_size;
+ cxl_memdev_get_ram_size;
+ cxl_memdev_get_firmware_verison;
+local:
+ *;
+};
--
2.33.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* Re: [ndctl PATCH v6] cxl: add a cxl utility and libcxl library
2021-12-09 20:23 ` [ndctl PATCH v6] " Vishal Verma
@ 2021-12-09 20:40 ` Dan Williams
2021-12-09 21:09 ` [ndctl PATCH v7] " Vishal Verma
0 siblings, 1 reply; 31+ messages in thread
From: Dan Williams @ 2021-12-09 20:40 UTC (permalink / raw)
To: Vishal Verma; +Cc: linux-cxl, Linux NVDIMM, Ben Widawsky
On Thu, Dec 9, 2021 at 12:23 PM Vishal Verma <vishal.l.verma@intel.com> wrote:
>
> CXL - or Compute eXpress Link - is a new interconnect that extends PCIe
> to support a wide range of devices, including cache coherent memory
> expanders. As such, these devices can be new sources of 'persistent
> memory', and the 'ndctl' umbrella of tools and libraries needs to be able
> to interact with them.
>
> Add a new utility and library for managing these CXL memory devices. This
> is an initial bring-up for interacting with CXL devices, and only includes
> adding the utility and library infrastructure, parsing device information
> from sysfs for CXL devices, and providing a 'cxl-list' command to
> display this information in JSON formatted output.
>
> Cc: Ben Widawsky <ben.widawsky@intel.com>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Reviewed-by: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
> ---
>
> Changes in v6:
> - Remove the cxl-list default of listing memdevs as we may want to list
> regions by default, once that support is available. For now just error
> out if memdevs are not explicitly asked for. (Dan)
>
> Documentation/cxl/cxl-list.txt | 64 +++++
> Documentation/cxl/cxl.txt | 34 +++
> Documentation/cxl/human-option.txt | 8 +
> Documentation/cxl/verbose-option.txt | 5 +
> configure.ac | 3 +
> Makefile.am | 8 +-
> Makefile.am.in | 4 +
> cxl/lib/private.h | 29 +++
> cxl/lib/libcxl.c | 345 +++++++++++++++++++++++++++
> cxl/builtin.h | 8 +
> cxl/libcxl.h | 55 +++++
> util/filter.h | 2 +
> util/json.h | 3 +
> util/main.h | 3 +
> cxl/cxl.c | 96 ++++++++
> cxl/list.c | 121 ++++++++++
> util/filter.c | 20 ++
> util/json.c | 26 ++
> .clang-format | 1 +
> .gitignore | 4 +-
> Documentation/cxl/Makefile.am | 58 +++++
> cxl/Makefile.am | 21 ++
> cxl/lib/Makefile.am | 32 +++
> cxl/lib/libcxl.pc.in | 11 +
> cxl/lib/libcxl.sym | 25 ++
> 25 files changed, 982 insertions(+), 4 deletions(-)
> create mode 100644 Documentation/cxl/cxl-list.txt
> create mode 100644 Documentation/cxl/cxl.txt
> create mode 100644 Documentation/cxl/human-option.txt
> create mode 100644 Documentation/cxl/verbose-option.txt
> create mode 100644 cxl/lib/private.h
> create mode 100644 cxl/lib/libcxl.c
> create mode 100644 cxl/builtin.h
> create mode 100644 cxl/libcxl.h
> create mode 100644 cxl/cxl.c
> create mode 100644 cxl/list.c
> create mode 100644 Documentation/cxl/Makefile.am
> create mode 100644 cxl/Makefile.am
> create mode 100644 cxl/lib/Makefile.am
> create mode 100644 cxl/lib/libcxl.pc.in
> create mode 100644 cxl/lib/libcxl.sym
>
> diff --git a/Documentation/cxl/cxl-list.txt b/Documentation/cxl/cxl-list.txt
> new file mode 100644
> index 0000000..e761cfa
> --- /dev/null
> +++ b/Documentation/cxl/cxl-list.txt
> @@ -0,0 +1,64 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +cxl-list(1)
> +===========
> +
> +NAME
> +----
> +cxl-list - List CXL capable memory devices, and their attributes in json.
> +
> +SYNOPSIS
> +--------
> +[verse]
> +'cxl list' [<options>]
> +
> +Walk the CXL capable device hierarchy in the system and list all device
> +instances along with some of their major attributes.
> +
> +Options can be specified to limit the output to specific memdevs.
> +By default, 'cxl list' with no options is equivalent to:
> +[verse]
> +cxl list --memdevs
With the default change these last 4 lines can be trimmed.
> +
> +EXAMPLE
> +-------
> +----
> +# cxl list --memdevs
> +{
> + "memdev":"mem0",
> + "pmem_size":268435456,
> + "ram_size":0,
> +}
> +----
> +
> +OPTIONS
> +-------
> +-m::
> +--memdev=::
> + Specify a cxl memory device name to filter the listing. For example:
> +----
> +# cxl list --memdev=mem0
> +{
> + "memdev":"mem0",
> + "pmem_size":268435456,
> + "ram_size":0,
> +}
> +----
> +
> +-M::
> +--memdevs::
> + Include all CXL memory devices in the listing
The -m is a filter so I think this can be: s/all//
> +
> +-i::
> +--idle::
> + Include idle (not enabled / zero-sized) devices in the listing
> +
> +include::human-option.txt[]
> +
> +include::verbose-option.txt[]
> +
> +include::../copyright.txt[]
> +
> +SEE ALSO
> +--------
> +linkcxl:ndctl-list[1]
[..]
> diff --git a/cxl/list.c b/cxl/list.c
> new file mode 100644
> index 0000000..69201d9
> --- /dev/null
> +++ b/cxl/list.c
> @@ -0,0 +1,121 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
> +#include <stdio.h>
> +#include <errno.h>
> +#include <stdlib.h>
> +#include <unistd.h>
> +#include <limits.h>
> +#include <util/json.h>
> +#include <util/filter.h>
> +#include <json-c/json.h>
> +#include <cxl/libcxl.h>
> +#include <util/parse-options.h>
> +#include <ccan/array_size/array_size.h>
> +
> +static struct {
> + bool memdevs;
> + bool idle;
> + bool human;
> +} list;
> +
> +static unsigned long listopts_to_flags(void)
> +{
> + unsigned long flags = 0;
> +
> + if (list.idle)
> + flags |= UTIL_JSON_IDLE;
> + if (list.human)
> + flags |= UTIL_JSON_HUMAN;
> + return flags;
> +}
> +
> +static struct {
> + const char *memdev;
> +} param;
> +
> +static int did_fail;
> +
> +#define fail(fmt, ...) \
> +do { \
> + did_fail = 1; \
> + fprintf(stderr, "cxl-%s:%s:%d: " fmt, \
> + VERSION, __func__, __LINE__, ##__VA_ARGS__); \
> +} while (0)
> +
> +static int num_list_flags(void)
> +{
> + return list.memdevs;
> +}
> +
> +int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx)
> +{
> + const struct option options[] = {
> + OPT_STRING('m', "memdev", ¶m.memdev, "memory device name",
> + "filter by CXL memory device name"),
> + OPT_BOOLEAN('M', "memdevs", &list.memdevs,
> + "include CXL memory device info"),
> + OPT_BOOLEAN('i', "idle", &list.idle, "include idle devices"),
> + OPT_BOOLEAN('u', "human", &list.human,
> + "use human friendly number formats "),
> + OPT_END(),
> + };
> + const char * const u[] = {
> + "cxl list [<options>]",
> + NULL
> + };
> + struct json_object *jdevs = NULL;
> + unsigned long list_flags;
> + struct cxl_memdev *memdev;
> + int i;
> +
> + argc = parse_options(argc, argv, options, u, 0);
> + for (i = 0; i < argc; i++)
> + error("unknown parameter \"%s\"\n", argv[i]);
> +
> + if (argc)
> + usage_with_options(u, options);
> +
> + if (num_list_flags() == 0) {
> + /*
> + * TODO: We likely want to list regions by default if nothing
> + * was explicitly asked for. But until we have region support,
> + * print this error asking for devices explicitly.
> + * Once region support is added, this TODO can be removed.
> + */
> + fprintf(stderr,
Should this be error() instead like the "unknown parameter" print above?
> + "please specify entities to list, e.g. using -m/-M\n");
Add a usage_with_options() call here to abort?
^ permalink raw reply [flat|nested] 31+ messages in thread* [ndctl PATCH v7] cxl: add a cxl utility and libcxl library
2021-12-09 20:40 ` Dan Williams
@ 2021-12-09 21:09 ` Vishal Verma
2021-12-09 21:23 ` Dan Williams
0 siblings, 1 reply; 31+ messages in thread
From: Vishal Verma @ 2021-12-09 21:09 UTC (permalink / raw)
To: linux-cxl; +Cc: nvdimm, Dan Williams, Ben Widawsky, Vishal Verma
CXL - or Compute eXpress Link - is a new interconnect that extends PCIe
to support a wide range of devices, including cache coherent memory
expanders. As such, these devices can be new sources of 'persistent
memory', and the 'ndctl' umbrella of tools and libraries needs to be able
to interact with them.
Add a new utility and library for managing these CXL memory devices. This
is an initial bring-up for interacting with CXL devices, and only includes
adding the utility and library infrastructure, parsing device information
from sysfs for CXL devices, and providing a 'cxl-list' command to
display this information in JSON formatted output.
Cc: Ben Widawsky <ben.widawsky@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
Changes in v7:
- Update the man page w.r.t the new default behavior (Dan)
- Use error() and usage_with_options() to fail when no list entities are
specified (Dan)
Documentation/cxl/cxl-list.txt | 59 +++++
Documentation/cxl/cxl.txt | 34 +++
Documentation/cxl/human-option.txt | 8 +
Documentation/cxl/verbose-option.txt | 5 +
configure.ac | 3 +
Makefile.am | 8 +-
Makefile.am.in | 4 +
cxl/lib/private.h | 29 +++
cxl/lib/libcxl.c | 345 +++++++++++++++++++++++++++
cxl/builtin.h | 8 +
cxl/libcxl.h | 55 +++++
util/filter.h | 2 +
util/json.h | 3 +
util/main.h | 3 +
cxl/cxl.c | 96 ++++++++
cxl/list.c | 121 ++++++++++
util/filter.c | 20 ++
util/json.c | 26 ++
.clang-format | 1 +
.gitignore | 4 +-
Documentation/cxl/Makefile.am | 58 +++++
cxl/Makefile.am | 21 ++
cxl/lib/Makefile.am | 32 +++
cxl/lib/libcxl.pc.in | 11 +
cxl/lib/libcxl.sym | 25 ++
25 files changed, 977 insertions(+), 4 deletions(-)
create mode 100644 Documentation/cxl/cxl-list.txt
create mode 100644 Documentation/cxl/cxl.txt
create mode 100644 Documentation/cxl/human-option.txt
create mode 100644 Documentation/cxl/verbose-option.txt
create mode 100644 cxl/lib/private.h
create mode 100644 cxl/lib/libcxl.c
create mode 100644 cxl/builtin.h
create mode 100644 cxl/libcxl.h
create mode 100644 cxl/cxl.c
create mode 100644 cxl/list.c
create mode 100644 Documentation/cxl/Makefile.am
create mode 100644 cxl/Makefile.am
create mode 100644 cxl/lib/Makefile.am
create mode 100644 cxl/lib/libcxl.pc.in
create mode 100644 cxl/lib/libcxl.sym
diff --git a/Documentation/cxl/cxl-list.txt b/Documentation/cxl/cxl-list.txt
new file mode 100644
index 0000000..370d5b8
--- /dev/null
+++ b/Documentation/cxl/cxl-list.txt
@@ -0,0 +1,59 @@
+// SPDX-License-Identifier: GPL-2.0
+
+cxl-list(1)
+===========
+
+NAME
+----
+cxl-list - List CXL capable memory devices, and their attributes in json.
+
+SYNOPSIS
+--------
+[verse]
+'cxl list' [<options>]
+
+Walk the CXL capable device hierarchy in the system and list all device
+instances along with some of their major attributes.
+
+EXAMPLE
+-------
+----
+# cxl list --memdevs
+{
+ "memdev":"mem0",
+ "pmem_size":268435456,
+ "ram_size":0,
+}
+----
+
+OPTIONS
+-------
+-m::
+--memdev=::
+ Specify a cxl memory device name to filter the listing. For example:
+----
+# cxl list --memdev=mem0
+{
+ "memdev":"mem0",
+ "pmem_size":268435456,
+ "ram_size":0,
+}
+----
+
+-M::
+--memdevs::
+ Include CXL memory devices in the listing
+
+-i::
+--idle::
+ Include idle (not enabled / zero-sized) devices in the listing
+
+include::human-option.txt[]
+
+include::verbose-option.txt[]
+
+include::../copyright.txt[]
+
+SEE ALSO
+--------
+linkcxl:ndctl-list[1]
diff --git a/Documentation/cxl/cxl.txt b/Documentation/cxl/cxl.txt
new file mode 100644
index 0000000..41a51c7
--- /dev/null
+++ b/Documentation/cxl/cxl.txt
@@ -0,0 +1,34 @@
+// SPDX-License-Identifier: GPL-2.0
+
+cxl(1)
+======
+
+NAME
+----
+cxl - Provides enumeration and provisioning commands for CXL platforms
+
+SYNOPSIS
+--------
+[verse]
+'cxl' [--version] [--help] COMMAND [ARGS]
+
+OPTIONS
+-------
+-v::
+--version::
+ Display the version of the 'cxl' utility.
+
+-h::
+--help::
+ Run the 'cxl help' command.
+
+DESCRIPTION
+-----------
+The cxl utility provides enumeration and provisioning commands for
+the CXL devices managed by the Linux kernel.
+
+include::../copyright.txt[]
+
+SEE ALSO
+--------
+linkcxl:ndctl[1]
diff --git a/Documentation/cxl/human-option.txt b/Documentation/cxl/human-option.txt
new file mode 100644
index 0000000..2f4de7a
--- /dev/null
+++ b/Documentation/cxl/human-option.txt
@@ -0,0 +1,8 @@
+// SPDX-License-Identifier: GPL-2.0
+
+-u::
+--human::
+ By default the command will output machine-friendly raw-integer
+ data. Instead, with this flag, numbers representing storage size
+ will be formatted as human readable strings with units, other
+ fields are converted to hexadecimal strings.
diff --git a/Documentation/cxl/verbose-option.txt b/Documentation/cxl/verbose-option.txt
new file mode 100644
index 0000000..cb62c8e
--- /dev/null
+++ b/Documentation/cxl/verbose-option.txt
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+-v::
+--verbose::
+ Emit more debug messages
diff --git a/configure.ac b/configure.ac
index dc39dbe..dadae0a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -222,12 +222,15 @@ AC_CONFIG_HEADERS(config.h)
AC_CONFIG_FILES([
Makefile
daxctl/lib/Makefile
+ cxl/lib/Makefile
ndctl/lib/Makefile
ndctl/Makefile
daxctl/Makefile
+ cxl/Makefile
test/Makefile
Documentation/ndctl/Makefile
Documentation/daxctl/Makefile
+ Documentation/cxl/Makefile
])
AC_OUTPUT
diff --git a/Makefile.am b/Makefile.am
index 60a1998..428fd40 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,9 +1,9 @@
include Makefile.am.in
ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS}
-SUBDIRS = . daxctl/lib ndctl/lib ndctl daxctl
+SUBDIRS = . cxl/lib daxctl/lib ndctl/lib cxl ndctl daxctl
if ENABLE_DOCS
-SUBDIRS += Documentation/ndctl Documentation/daxctl
+SUBDIRS += Documentation/ndctl Documentation/daxctl Documentation/cxl
endif
SUBDIRS += test
@@ -87,4 +87,6 @@ libutil_a_SOURCES = \
util/filter.h \
util/bitmap.h
-nobase_include_HEADERS = daxctl/libdaxctl.h
+nobase_include_HEADERS = \
+ daxctl/libdaxctl.h \
+ cxl/libcxl.h
diff --git a/Makefile.am.in b/Makefile.am.in
index bdceda9..aaeee53 100644
--- a/Makefile.am.in
+++ b/Makefile.am.in
@@ -42,3 +42,7 @@ LIBNDCTL_AGE=19
LIBDAXCTL_CURRENT=6
LIBDAXCTL_REVISION=0
LIBDAXCTL_AGE=5
+
+LIBCXL_CURRENT=1
+LIBCXL_REVISION=0
+LIBCXL_AGE=0
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
new file mode 100644
index 0000000..fc88fa1
--- /dev/null
+++ b/cxl/lib/private.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+/* Copyright (C) 2020-2021, Intel Corporation. All rights reserved. */
+#ifndef _LIBCXL_PRIVATE_H_
+#define _LIBCXL_PRIVATE_H_
+
+#include <libkmod.h>
+
+#define CXL_EXPORT __attribute__ ((visibility("default")))
+
+struct cxl_memdev {
+ int id, major, minor;
+ void *dev_buf;
+ size_t buf_len;
+ char *dev_path;
+ char *firmware_version;
+ struct cxl_ctx *ctx;
+ struct list_node list;
+ unsigned long long pmem_size;
+ unsigned long long ram_size;
+ int payload_max;
+ struct kmod_module *module;
+};
+
+static inline int check_kmod(struct kmod_ctx *kmod_ctx)
+{
+ return kmod_ctx ? 0 : -ENXIO;
+}
+
+#endif /* _LIBCXL_PRIVATE_H_ */
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
new file mode 100644
index 0000000..c15e987
--- /dev/null
+++ b/cxl/lib/libcxl.c
@@ -0,0 +1,345 @@
+// SPDX-License-Identifier: LGPL-2.1
+// Copyright (C) 2020-2021, Intel Corporation. All rights reserved.
+#include <stdio.h>
+#include <errno.h>
+#include <limits.h>
+#include <libgen.h>
+#include <stdlib.h>
+#include <dirent.h>
+#include <unistd.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/sysmacros.h>
+#include <uuid/uuid.h>
+#include <ccan/list/list.h>
+#include <ccan/array_size/array_size.h>
+
+#include <util/log.h>
+#include <util/sysfs.h>
+#include <util/bitmap.h>
+#include <cxl/libcxl.h>
+#include "private.h"
+
+/**
+ * struct cxl_ctx - library user context to find "nd" instances
+ *
+ * Instantiate with cxl_new(), which takes an initial reference. Free
+ * the context by dropping the reference count to zero with
+ * cxl_unref(), or take additional references with cxl_ref()
+ * @timeout: default library timeout in milliseconds
+ */
+struct cxl_ctx {
+ /* log_ctx must be first member for cxl_set_log_fn compat */
+ struct log_ctx ctx;
+ int refcount;
+ void *userdata;
+ int memdevs_init;
+ struct list_head memdevs;
+ struct kmod_ctx *kmod_ctx;
+ void *private_data;
+};
+
+static void free_memdev(struct cxl_memdev *memdev, struct list_head *head)
+{
+ if (head)
+ list_del_from(head, &memdev->list);
+ kmod_module_unref(memdev->module);
+ free(memdev->firmware_version);
+ free(memdev->dev_buf);
+ free(memdev->dev_path);
+ free(memdev);
+}
+
+/**
+ * cxl_get_userdata - retrieve stored data pointer from library context
+ * @ctx: cxl library context
+ *
+ * This might be useful to access from callbacks like a custom logging
+ * function.
+ */
+CXL_EXPORT void *cxl_get_userdata(struct cxl_ctx *ctx)
+{
+ if (ctx == NULL)
+ return NULL;
+ return ctx->userdata;
+}
+
+/**
+ * cxl_set_userdata - store custom @userdata in the library context
+ * @ctx: cxl library context
+ * @userdata: data pointer
+ */
+CXL_EXPORT void cxl_set_userdata(struct cxl_ctx *ctx, void *userdata)
+{
+ if (ctx == NULL)
+ return;
+ ctx->userdata = userdata;
+}
+
+CXL_EXPORT void cxl_set_private_data(struct cxl_ctx *ctx, void *data)
+{
+ ctx->private_data = data;
+}
+
+CXL_EXPORT void *cxl_get_private_data(struct cxl_ctx *ctx)
+{
+ return ctx->private_data;
+}
+
+/**
+ * cxl_new - instantiate a new library context
+ * @ctx: context to establish
+ *
+ * Returns zero on success and stores an opaque pointer in ctx. The
+ * context is freed by cxl_unref(), i.e. cxl_new() implies an
+ * internal cxl_ref().
+ */
+CXL_EXPORT int cxl_new(struct cxl_ctx **ctx)
+{
+ struct kmod_ctx *kmod_ctx;
+ struct cxl_ctx *c;
+ int rc = 0;
+
+ c = calloc(1, sizeof(struct cxl_ctx));
+ if (!c)
+ return -ENOMEM;
+
+ kmod_ctx = kmod_new(NULL, NULL);
+ if (check_kmod(kmod_ctx) != 0) {
+ rc = -ENXIO;
+ goto out;
+ }
+
+ c->refcount = 1;
+ log_init(&c->ctx, "libcxl", "CXL_LOG");
+ info(c, "ctx %p created\n", c);
+ dbg(c, "log_priority=%d\n", c->ctx.log_priority);
+ *ctx = c;
+ list_head_init(&c->memdevs);
+ c->kmod_ctx = kmod_ctx;
+
+ return 0;
+out:
+ free(c);
+ return rc;
+}
+
+/**
+ * cxl_ref - take an additional reference on the context
+ * @ctx: context established by cxl_new()
+ */
+CXL_EXPORT struct cxl_ctx *cxl_ref(struct cxl_ctx *ctx)
+{
+ if (ctx == NULL)
+ return NULL;
+ ctx->refcount++;
+ return ctx;
+}
+
+/**
+ * cxl_unref - drop a context reference count
+ * @ctx: context established by cxl_new()
+ *
+ * Drop a reference and if the resulting reference count is 0 destroy
+ * the context.
+ */
+CXL_EXPORT void cxl_unref(struct cxl_ctx *ctx)
+{
+ struct cxl_memdev *memdev, *_d;
+
+ if (ctx == NULL)
+ return;
+ ctx->refcount--;
+ if (ctx->refcount > 0)
+ return;
+
+ list_for_each_safe(&ctx->memdevs, memdev, _d, list)
+ free_memdev(memdev, &ctx->memdevs);
+
+ kmod_unref(ctx->kmod_ctx);
+ info(ctx, "context %p released\n", ctx);
+ free(ctx);
+}
+
+/**
+ * cxl_set_log_fn - override default log routine
+ * @ctx: cxl library context
+ * @log_fn: function to be called for logging messages
+ *
+ * The built-in logging writes to stderr. It can be overridden by a
+ * custom function, to plug log messages into the user's logging
+ * functionality.
+ */
+CXL_EXPORT void cxl_set_log_fn(struct cxl_ctx *ctx,
+ void (*cxl_log_fn)(struct cxl_ctx *ctx, int priority,
+ const char *file, int line, const char *fn,
+ const char *format, va_list args))
+{
+ ctx->ctx.log_fn = (log_fn) cxl_log_fn;
+ info(ctx, "custom logging function %p registered\n", cxl_log_fn);
+}
+
+/**
+ * cxl_get_log_priority - retrieve current library loglevel (syslog)
+ * @ctx: cxl library context
+ */
+CXL_EXPORT int cxl_get_log_priority(struct cxl_ctx *ctx)
+{
+ return ctx->ctx.log_priority;
+}
+
+/**
+ * cxl_set_log_priority - set log verbosity
+ * @priority: from syslog.h, LOG_ERR, LOG_INFO, LOG_DEBUG
+ *
+ * Note: LOG_DEBUG requires library be built with "configure --enable-debug"
+ */
+CXL_EXPORT void cxl_set_log_priority(struct cxl_ctx *ctx, int priority)
+{
+ ctx->ctx.log_priority = priority;
+}
+
+static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
+{
+ const char *devname = devpath_to_devname(cxlmem_base);
+ char *path = calloc(1, strlen(cxlmem_base) + 100);
+ struct cxl_ctx *ctx = parent;
+ struct cxl_memdev *memdev, *memdev_dup;
+ char buf[SYSFS_ATTR_SIZE];
+ struct stat st;
+
+ if (!path)
+ return NULL;
+ dbg(ctx, "%s: base: \'%s\'\n", devname, cxlmem_base);
+
+ memdev = calloc(1, sizeof(*memdev));
+ if (!memdev)
+ goto err_dev;
+ memdev->id = id;
+ memdev->ctx = ctx;
+
+ sprintf(path, "/dev/cxl/%s", devname);
+ if (stat(path, &st) < 0)
+ goto err_read;
+ memdev->major = major(st.st_rdev);
+ memdev->minor = minor(st.st_rdev);
+
+ sprintf(path, "%s/pmem/size", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+ memdev->pmem_size = strtoull(buf, NULL, 0);
+
+ sprintf(path, "%s/ram/size", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+ memdev->ram_size = strtoull(buf, NULL, 0);
+
+ sprintf(path, "%s/payload_max", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+ memdev->payload_max = strtoull(buf, NULL, 0);
+ if (memdev->payload_max < 0)
+ goto err_read;
+
+ memdev->dev_path = strdup(cxlmem_base);
+ if (!memdev->dev_path)
+ goto err_read;
+
+ sprintf(path, "%s/firmware_version", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+
+ memdev->firmware_version = strdup(buf);
+ if (!memdev->firmware_version)
+ goto err_read;
+
+ memdev->dev_buf = calloc(1, strlen(cxlmem_base) + 50);
+ if (!memdev->dev_buf)
+ goto err_read;
+ memdev->buf_len = strlen(cxlmem_base) + 50;
+
+ cxl_memdev_foreach(ctx, memdev_dup)
+ if (memdev_dup->id == memdev->id) {
+ free_memdev(memdev, NULL);
+ free(path);
+ return memdev_dup;
+ }
+
+ list_add(&ctx->memdevs, &memdev->list);
+ free(path);
+ return memdev;
+
+ err_read:
+ free(memdev->firmware_version);
+ free(memdev->dev_buf);
+ free(memdev->dev_path);
+ free(memdev);
+ err_dev:
+ free(path);
+ return NULL;
+}
+
+static void cxl_memdevs_init(struct cxl_ctx *ctx)
+{
+ if (ctx->memdevs_init)
+ return;
+
+ ctx->memdevs_init = 1;
+
+ sysfs_device_parse(ctx, "/sys/bus/cxl/devices", "mem", ctx,
+ add_cxl_memdev);
+}
+
+CXL_EXPORT struct cxl_ctx *cxl_memdev_get_ctx(struct cxl_memdev *memdev)
+{
+ return memdev->ctx;
+}
+
+CXL_EXPORT struct cxl_memdev *cxl_memdev_get_first(struct cxl_ctx *ctx)
+{
+ cxl_memdevs_init(ctx);
+
+ return list_top(&ctx->memdevs, struct cxl_memdev, list);
+}
+
+CXL_EXPORT struct cxl_memdev *cxl_memdev_get_next(struct cxl_memdev *memdev)
+{
+ struct cxl_ctx *ctx = memdev->ctx;
+
+ return list_next(&ctx->memdevs, memdev, list);
+}
+
+CXL_EXPORT int cxl_memdev_get_id(struct cxl_memdev *memdev)
+{
+ return memdev->id;
+}
+
+CXL_EXPORT const char *cxl_memdev_get_devname(struct cxl_memdev *memdev)
+{
+ return devpath_to_devname(memdev->dev_path);
+}
+
+CXL_EXPORT int cxl_memdev_get_major(struct cxl_memdev *memdev)
+{
+ return memdev->major;
+}
+
+CXL_EXPORT int cxl_memdev_get_minor(struct cxl_memdev *memdev)
+{
+ return memdev->minor;
+}
+
+CXL_EXPORT unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev)
+{
+ return memdev->pmem_size;
+}
+
+CXL_EXPORT unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev)
+{
+ return memdev->ram_size;
+}
+
+CXL_EXPORT const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev)
+{
+ return memdev->firmware_version;
+}
diff --git a/cxl/builtin.h b/cxl/builtin.h
new file mode 100644
index 0000000..3797f98
--- /dev/null
+++ b/cxl/builtin.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
+#ifndef _CXL_BUILTIN_H_
+#define _CXL_BUILTIN_H_
+
+struct cxl_ctx;
+int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx);
+#endif /* _CXL_BUILTIN_H_ */
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
new file mode 100644
index 0000000..fd06790
--- /dev/null
+++ b/cxl/libcxl.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+/* Copyright (C) 2020-2021, Intel Corporation. All rights reserved. */
+#ifndef _LIBCXL_H_
+#define _LIBCXL_H_
+
+#include <stdarg.h>
+#include <unistd.h>
+
+#ifdef HAVE_UUID
+#include <uuid/uuid.h>
+#else
+typedef unsigned char uuid_t[16];
+#endif
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct cxl_ctx;
+struct cxl_ctx *cxl_ref(struct cxl_ctx *ctx);
+void cxl_unref(struct cxl_ctx *ctx);
+int cxl_new(struct cxl_ctx **ctx);
+void cxl_set_log_fn(struct cxl_ctx *ctx,
+ void (*log_fn)(struct cxl_ctx *ctx, int priority,
+ const char *file, int line, const char *fn,
+ const char *format, va_list args));
+int cxl_get_log_priority(struct cxl_ctx *ctx);
+void cxl_set_log_priority(struct cxl_ctx *ctx, int priority);
+void cxl_set_userdata(struct cxl_ctx *ctx, void *userdata);
+void *cxl_get_userdata(struct cxl_ctx *ctx);
+void cxl_set_private_data(struct cxl_ctx *ctx, void *data);
+void *cxl_get_private_data(struct cxl_ctx *ctx);
+
+struct cxl_memdev;
+struct cxl_memdev *cxl_memdev_get_first(struct cxl_ctx *ctx);
+struct cxl_memdev *cxl_memdev_get_next(struct cxl_memdev *memdev);
+int cxl_memdev_get_id(struct cxl_memdev *memdev);
+const char *cxl_memdev_get_devname(struct cxl_memdev *memdev);
+int cxl_memdev_get_major(struct cxl_memdev *memdev);
+int cxl_memdev_get_minor(struct cxl_memdev *memdev);
+struct cxl_ctx *cxl_memdev_get_ctx(struct cxl_memdev *memdev);
+unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev);
+unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev);
+const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev);
+
+#define cxl_memdev_foreach(ctx, memdev) \
+ for (memdev = cxl_memdev_get_first(ctx); \
+ memdev != NULL; \
+ memdev = cxl_memdev_get_next(memdev))
+
+#ifdef __cplusplus
+} /* extern "C" */
+#endif
+
+#endif
diff --git a/util/filter.h b/util/filter.h
index 1e1a41c..9a80d65 100644
--- a/util/filter.h
+++ b/util/filter.h
@@ -29,6 +29,8 @@ struct daxctl_dev *util_daxctl_dev_filter(struct daxctl_dev *dev,
const char *ident);
struct daxctl_region *util_daxctl_region_filter(struct daxctl_region *region,
const char *ident);
+struct cxl_memdev *util_cxl_memdev_filter(struct cxl_memdev *memdev,
+ const char *ident);
enum ndctl_namespace_mode util_nsmode(const char *mode);
const char *util_nsmode_name(enum ndctl_namespace_mode mode);
diff --git a/util/json.h b/util/json.h
index 0f09e36..91918c8 100644
--- a/util/json.h
+++ b/util/json.h
@@ -55,4 +55,7 @@ struct json_object *util_dimm_health_to_json(struct ndctl_dimm *dimm);
struct json_object *util_dimm_firmware_to_json(struct ndctl_dimm *dimm,
unsigned long flags);
struct json_object *util_region_capabilities_to_json(struct ndctl_region *region);
+struct cxl_memdev;
+struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
+ unsigned long flags);
#endif /* __NDCTL_JSON_H__ */
diff --git a/util/main.h b/util/main.h
index c89a843..80b55c4 100644
--- a/util/main.h
+++ b/util/main.h
@@ -10,16 +10,19 @@
enum program {
PROG_NDCTL,
PROG_DAXCTL,
+ PROG_CXL,
};
struct ndctl_ctx;
struct daxctl_ctx;
+struct cxl_ctx;
struct cmd_struct {
const char *cmd;
union {
int (*n_fn)(int, const char **, struct ndctl_ctx *ctx);
int (*d_fn)(int, const char **, struct daxctl_ctx *ctx);
+ int (*c_fn)(int, const char **, struct cxl_ctx *ctx);
};
};
diff --git a/cxl/cxl.c b/cxl/cxl.c
new file mode 100644
index 0000000..a7725f8
--- /dev/null
+++ b/cxl/cxl.c
@@ -0,0 +1,96 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
+/* Copyright (C) 2005 Andreas Ericsson. All rights reserved. */
+
+/* originally copied from perf and git */
+
+#include <stdio.h>
+#include <errno.h>
+#include <string.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <cxl/libcxl.h>
+#include <util/parse-options.h>
+#include <ccan/array_size/array_size.h>
+
+#include <util/strbuf.h>
+#include <util/util.h>
+#include <util/main.h>
+#include <cxl/builtin.h>
+
+const char cxl_usage_string[] = "cxl [--version] [--help] COMMAND [ARGS]";
+const char cxl_more_info_string[] =
+ "See 'cxl help COMMAND' for more information on a specific command.\n"
+ " cxl --list-cmds to see all available commands";
+
+static int cmd_version(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ printf("%s\n", VERSION);
+ return 0;
+}
+
+static int cmd_help(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ const char * const builtin_help_subcommands[] = {
+ "list",
+ NULL,
+ };
+ struct option builtin_help_options[] = {
+ OPT_END(),
+ };
+ const char *builtin_help_usage[] = {
+ "cxl help [command]",
+ NULL
+ };
+
+ argc = parse_options_subcommand(argc, argv, builtin_help_options,
+ builtin_help_subcommands, builtin_help_usage, 0);
+
+ if (!argv[0]) {
+ printf("\n usage: %s\n\n", cxl_usage_string);
+ printf("\n %s\n\n", cxl_more_info_string);
+ return 0;
+ }
+
+ return help_show_man_page(argv[0], "cxl", "CXL_MAN_VIEWER");
+}
+
+static struct cmd_struct commands[] = {
+ { "version", .c_fn = cmd_version },
+ { "list", .c_fn = cmd_list },
+ { "help", .c_fn = cmd_help },
+};
+
+int main(int argc, const char **argv)
+{
+ struct cxl_ctx *ctx;
+ int rc;
+
+ /* Look for flags.. */
+ argv++;
+ argc--;
+ main_handle_options(&argv, &argc, cxl_usage_string, commands,
+ ARRAY_SIZE(commands));
+
+ if (argc > 0) {
+ if (!prefixcmp(argv[0], "--"))
+ argv[0] += 2;
+ } else {
+ /* The user didn't specify a command; give them help */
+ printf("\n usage: %s\n\n", cxl_usage_string);
+ printf("\n %s\n\n", cxl_more_info_string);
+ goto out;
+ }
+
+ rc = cxl_new(&ctx);
+ if (rc)
+ goto out;
+ main_handle_internal_command(argc, argv, ctx, commands,
+ ARRAY_SIZE(commands), PROG_CXL);
+ cxl_unref(ctx);
+ fprintf(stderr, "Unknown command: '%s'\n", argv[0]);
+out:
+ return 1;
+}
diff --git a/cxl/list.c b/cxl/list.c
new file mode 100644
index 0000000..043d20c
--- /dev/null
+++ b/cxl/list.c
@@ -0,0 +1,121 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
+#include <stdio.h>
+#include <errno.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <limits.h>
+#include <util/json.h>
+#include <util/filter.h>
+#include <json-c/json.h>
+#include <cxl/libcxl.h>
+#include <util/parse-options.h>
+#include <ccan/array_size/array_size.h>
+
+static struct {
+ bool memdevs;
+ bool idle;
+ bool human;
+} list;
+
+static unsigned long listopts_to_flags(void)
+{
+ unsigned long flags = 0;
+
+ if (list.idle)
+ flags |= UTIL_JSON_IDLE;
+ if (list.human)
+ flags |= UTIL_JSON_HUMAN;
+ return flags;
+}
+
+static struct {
+ const char *memdev;
+} param;
+
+static int did_fail;
+
+#define fail(fmt, ...) \
+do { \
+ did_fail = 1; \
+ fprintf(stderr, "cxl-%s:%s:%d: " fmt, \
+ VERSION, __func__, __LINE__, ##__VA_ARGS__); \
+} while (0)
+
+static int num_list_flags(void)
+{
+ return list.memdevs;
+}
+
+int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ const struct option options[] = {
+ OPT_STRING('m', "memdev", ¶m.memdev, "memory device name",
+ "filter by CXL memory device name"),
+ OPT_BOOLEAN('M', "memdevs", &list.memdevs,
+ "include CXL memory device info"),
+ OPT_BOOLEAN('i', "idle", &list.idle, "include idle devices"),
+ OPT_BOOLEAN('u', "human", &list.human,
+ "use human friendly number formats "),
+ OPT_END(),
+ };
+ const char * const u[] = {
+ "cxl list [<options>]",
+ NULL
+ };
+ struct json_object *jdevs = NULL;
+ unsigned long list_flags;
+ struct cxl_memdev *memdev;
+ int i;
+
+ argc = parse_options(argc, argv, options, u, 0);
+ for (i = 0; i < argc; i++)
+ error("unknown parameter \"%s\"\n", argv[i]);
+
+ if (argc)
+ usage_with_options(u, options);
+
+ if (num_list_flags() == 0) {
+ /*
+ * TODO: We likely want to list regions by default if nothing
+ * was explicitly asked for. But until we have region support,
+ * print this error asking for devices explicitly.
+ * Once region support is added, this TODO can be removed.
+ */
+ error("please specify entities to list, e.g. using -m/-M\n");
+ usage_with_options(u, options);
+ }
+
+ list_flags = listopts_to_flags();
+
+ cxl_memdev_foreach(ctx, memdev) {
+ struct json_object *jdev = NULL;
+
+ if (!util_cxl_memdev_filter(memdev, param.memdev))
+ continue;
+
+ if (list.memdevs) {
+ if (!jdevs) {
+ jdevs = json_object_new_array();
+ if (!jdevs) {
+ fail("\n");
+ continue;
+ }
+ }
+
+ jdev = util_cxl_memdev_to_json(memdev, list_flags);
+ if (!jdev) {
+ fail("\n");
+ continue;
+ }
+ json_object_array_add(jdevs, jdev);
+ }
+ }
+
+ if (jdevs)
+ util_display_json_array(stdout, jdevs, list_flags);
+
+ if (did_fail)
+ return -ENOMEM;
+ return 0;
+}
diff --git a/util/filter.c b/util/filter.c
index 8b4aad3..d81dade 100644
--- a/util/filter.c
+++ b/util/filter.c
@@ -12,6 +12,7 @@
#include <util/filter.h>
#include <ndctl/libndctl.h>
#include <daxctl/libdaxctl.h>
+#include <cxl/libcxl.h>
struct ndctl_bus *util_bus_filter(struct ndctl_bus *bus, const char *__ident)
{
@@ -339,6 +340,25 @@ struct daxctl_region *util_daxctl_region_filter(struct daxctl_region *region,
return NULL;
}
+struct cxl_memdev *util_cxl_memdev_filter(struct cxl_memdev *memdev,
+ const char *ident)
+{
+ int memdev_id;
+
+ if (!ident || strcmp(ident, "all") == 0)
+ return memdev;
+
+ if (strcmp(ident, cxl_memdev_get_devname(memdev)) == 0)
+ return memdev;
+
+ if ((sscanf(ident, "%d", &memdev_id) == 1
+ || sscanf(ident, "mem%d", &memdev_id) == 1)
+ && cxl_memdev_get_id(memdev) == memdev_id)
+ return memdev;
+
+ return NULL;
+}
+
enum ndctl_namespace_mode util_nsmode(const char *mode)
{
if (!mode)
diff --git a/util/json.c b/util/json.c
index a8d2412..3be3a92 100644
--- a/util/json.c
+++ b/util/json.c
@@ -9,6 +9,7 @@
#include <json-c/printbuf.h>
#include <ndctl/libndctl.h>
#include <daxctl/libdaxctl.h>
+#include <cxl/libcxl.h>
#include <ccan/array_size/array_size.h>
#include <ccan/short_types/short_types.h>
#include <ndctl.h>
@@ -1440,3 +1441,28 @@ struct json_object *util_badblock_rec_to_json(u64 block, u64 count,
json_object_put(jerr);
return NULL;
}
+
+struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
+ unsigned long flags)
+{
+ const char *devname = cxl_memdev_get_devname(memdev);
+ struct json_object *jdev, *jobj;
+
+ jdev = json_object_new_object();
+ if (!devname || !jdev)
+ return NULL;
+
+ jobj = json_object_new_string(devname);
+ if (jobj)
+ json_object_object_add(jdev, "memdev", jobj);
+
+ jobj = util_json_object_size(cxl_memdev_get_pmem_size(memdev), flags);
+ if (jobj)
+ json_object_object_add(jdev, "pmem_size", jobj);
+
+ jobj = util_json_object_size(cxl_memdev_get_ram_size(memdev), flags);
+ if (jobj)
+ json_object_object_add(jdev, "ram_size", jobj);
+
+ return jdev;
+}
diff --git a/.clang-format b/.clang-format
index 4e00fff..d2e77d0 100644
--- a/.clang-format
+++ b/.clang-format
@@ -77,6 +77,7 @@ ExperimentalAutoDetectBinPacking: false
# -e 's/\(.*foreach.*\)(.*/\1/' \
# | sort -u)
ForEachMacros:
+ - 'cxl_memdev_foreach'
- 'daxctl_dev_foreach'
- 'daxctl_mapping_foreach'
- 'daxctl_region_foreach'
diff --git a/.gitignore b/.gitignore
index 53512b2..6a97b92 100644
--- a/.gitignore
+++ b/.gitignore
@@ -16,9 +16,11 @@ Makefile.in
*.1
Documentation/daxctl/asciidoc.conf
Documentation/ndctl/asciidoc.conf
-Documentation/ndctl/attrs.adoc
+Documentation/cxl/asciidoc.conf
Documentation/daxctl/asciidoctor-extensions.rb
Documentation/ndctl/asciidoctor-extensions.rb
+Documentation/cxl/asciidoctor-extensions.rb
+Documentation/ndctl/attrs.adoc
.dirstamp
daxctl/config.h
daxctl/daxctl
diff --git a/Documentation/cxl/Makefile.am b/Documentation/cxl/Makefile.am
new file mode 100644
index 0000000..db98dd7
--- /dev/null
+++ b/Documentation/cxl/Makefile.am
@@ -0,0 +1,58 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020-2021 Intel Corporation. All rights reserved.
+
+if USE_ASCIIDOCTOR
+
+do_subst = sed -e 's,@Utility@,Cxl,g' -e's,@utility@,cxl,g'
+CONFFILE = asciidoctor-extensions.rb
+asciidoctor-extensions.rb: ../asciidoctor-extensions.rb.in
+ $(AM_V_GEN) $(do_subst) < $< > $@
+
+else
+
+do_subst = sed -e 's,UTILITY,cxl,g'
+CONFFILE = asciidoc.conf
+asciidoc.conf: ../asciidoc.conf.in
+ $(AM_V_GEN) $(do_subst) < $< > $@
+
+endif
+
+man1_MANS = \
+ cxl.1 \
+ cxl-list.1
+
+EXTRA_DIST = $(man1_MANS)
+
+CLEANFILES = $(man1_MANS)
+
+XML_DEPS = \
+ ../../version.m4 \
+ ../copyright.txt \
+ Makefile \
+ $(CONFFILE)
+
+RM ?= rm -f
+
+if USE_ASCIIDOCTOR
+
+%.1: %.txt $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@+ $@ && \
+ $(ASCIIDOC) -b manpage -d manpage -acompat-mode \
+ -I. -rasciidoctor-extensions \
+ -amansource=cxl -amanmanual="cxl Manual" \
+ -andctl_version=$(VERSION) -o $@+ $< && \
+ mv $@+ $@
+
+else
+
+%.xml: %.txt $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@+ $@ && \
+ $(ASCIIDOC) -b docbook -d manpage -f asciidoc.conf \
+ --unsafe -acxl_version=$(VERSION) -o $@+ $< && \
+ mv $@+ $@
+
+%.1: %.xml $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@ && \
+ $(XMLTO) -o . -m ../manpage-normal.xsl man $<
+
+endif
diff --git a/cxl/Makefile.am b/cxl/Makefile.am
new file mode 100644
index 0000000..98606b9
--- /dev/null
+++ b/cxl/Makefile.am
@@ -0,0 +1,21 @@
+include $(top_srcdir)/Makefile.am.in
+
+bin_PROGRAMS = cxl
+
+DISTCLEANFILES = config.h
+BUILT_SOURCES = config.h
+config.h: $(srcdir)/Makefile.am
+ $(AM_V_GEN) echo "/* Autogenerated by cxl/Makefile.am */" >$@
+
+cxl_SOURCES =\
+ cxl.c \
+ list.c \
+ ../util/json.c \
+ builtin.h
+
+cxl_LDADD =\
+ lib/libcxl.la \
+ ../libutil.a \
+ $(UUID_LIBS) \
+ $(KMOD_LIBS) \
+ $(JSON_LIBS)
diff --git a/cxl/lib/Makefile.am b/cxl/lib/Makefile.am
new file mode 100644
index 0000000..277f0cd
--- /dev/null
+++ b/cxl/lib/Makefile.am
@@ -0,0 +1,32 @@
+include $(top_srcdir)/Makefile.am.in
+
+%.pc: %.pc.in Makefile
+ $(SED_PROCESS)
+
+pkginclude_HEADERS = ../libcxl.h
+lib_LTLIBRARIES = libcxl.la
+
+libcxl_la_SOURCES =\
+ ../libcxl.h \
+ private.h \
+ ../../util/sysfs.c \
+ ../../util/sysfs.h \
+ ../../util/log.c \
+ ../../util/log.h \
+ libcxl.c
+
+libcxl_la_LIBADD =\
+ $(UUID_LIBS) \
+ $(KMOD_LIBS)
+
+EXTRA_DIST += libcxl.sym
+
+libcxl_la_LDFLAGS = $(AM_LDFLAGS) \
+ -version-info $(LIBCXL_CURRENT):$(LIBCXL_REVISION):$(LIBCXL_AGE) \
+ -Wl,--version-script=$(top_srcdir)/cxl/lib/libcxl.sym
+libcxl_la_DEPENDENCIES = libcxl.sym
+
+pkgconfigdir = $(libdir)/pkgconfig
+pkgconfig_DATA = libcxl.pc
+EXTRA_DIST += libcxl.pc.in
+CLEANFILES += libcxl.pc
diff --git a/cxl/lib/libcxl.pc.in b/cxl/lib/libcxl.pc.in
new file mode 100644
index 0000000..949fcdc
--- /dev/null
+++ b/cxl/lib/libcxl.pc.in
@@ -0,0 +1,11 @@
+prefix=@prefix@
+exec_prefix=@exec_prefix@
+libdir=@libdir@
+includedir=@includedir@
+
+Name: libcxl
+Description: Manage CXL devices
+Version: @VERSION@
+Libs: -L${libdir} -lcxl
+Libs.private:
+Cflags: -I${includedir}
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
new file mode 100644
index 0000000..2616e5c
--- /dev/null
+++ b/cxl/lib/libcxl.sym
@@ -0,0 +1,25 @@
+LIBCXL_1 {
+global:
+ cxl_get_userdata;
+ cxl_set_userdata;
+ cxl_get_private_data;
+ cxl_set_private_data;
+ cxl_ref;
+ cxl_get_log_priority;
+ cxl_set_log_fn;
+ cxl_unref;
+ cxl_set_log_priority;
+ cxl_new;
+ cxl_memdev_get_first;
+ cxl_memdev_get_next;
+ cxl_memdev_get_id;
+ cxl_memdev_get_devname;
+ cxl_memdev_get_major;
+ cxl_memdev_get_minor;
+ cxl_memdev_get_ctx;
+ cxl_memdev_get_pmem_size;
+ cxl_memdev_get_ram_size;
+ cxl_memdev_get_firmware_verison;
+local:
+ *;
+};
--
2.33.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* Re: [ndctl PATCH v7] cxl: add a cxl utility and libcxl library
2021-12-09 21:09 ` [ndctl PATCH v7] " Vishal Verma
@ 2021-12-09 21:23 ` Dan Williams
0 siblings, 0 replies; 31+ messages in thread
From: Dan Williams @ 2021-12-09 21:23 UTC (permalink / raw)
To: Vishal Verma; +Cc: linux-cxl, Linux NVDIMM, Ben Widawsky
On Thu, Dec 9, 2021 at 1:09 PM Vishal Verma <vishal.l.verma@intel.com> wrote:
>
> CXL - or Compute eXpress Link - is a new interconnect that extends PCIe
> to support a wide range of devices, including cache coherent memory
> expanders. As such, these devices can be new sources of 'persistent
> memory', and the 'ndctl' umbrella of tools and libraries needs to be able
> to interact with them.
>
> Add a new utility and library for managing these CXL memory devices. This
> is an initial bring-up for interacting with CXL devices, and only includes
> adding the utility and library infrastructure, parsing device information
> from sysfs for CXL devices, and providing a 'cxl-list' command to
> display this information in JSON formatted output.
>
> Cc: Ben Widawsky <ben.widawsky@intel.com>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Reviewed-by: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
> ---
>
> Changes in v7:
> - Update the man page w.r.t the new default behavior (Dan)
> - Use error() and usage_with_options() to fail when no list entities are
> specified (Dan)
Looks good.
^ permalink raw reply [flat|nested] 31+ messages in thread
* [ndctl PATCH v5 03/16] cxl: add a local copy of the cxl_mem UAPI header
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 01/16] ndctl: add .clang-format Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 02/16] cxl: add a cxl utility and libcxl library Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 04/16] util: add the struct_size() helper from the kernel Vishal Verma
` (12 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
While CXL functionality is under development, it is useful to have a
local copy of the UAPI header for cxl_mem definitions. This allows
building cxl and libcxl on systems where the appropriate kernel headers
are not installed in the usual locations.
Cc: Ben Widawsky <ben.widawsky@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
Makefile.am | 3 +-
Makefile.am.in | 1 +
cxl/cxl_mem.h | 189 ++++++++++++++++++++++++++++++++++++++++++++
cxl/lib/Makefile.am | 2 +-
4 files changed, 193 insertions(+), 2 deletions(-)
create mode 100644 cxl/cxl_mem.h
diff --git a/Makefile.am b/Makefile.am
index 428fd40..4904ee7 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -89,4 +89,5 @@ libutil_a_SOURCES = \
nobase_include_HEADERS = \
daxctl/libdaxctl.h \
- cxl/libcxl.h
+ cxl/libcxl.h \
+ cxl/cxl_mem.h
diff --git a/Makefile.am.in b/Makefile.am.in
index aaeee53..a748128 100644
--- a/Makefile.am.in
+++ b/Makefile.am.in
@@ -11,6 +11,7 @@ AM_CPPFLAGS = \
-DNDCTL_MAN_PATH=\""$(mandir)"\" \
-I${top_srcdir}/ndctl/lib \
-I${top_srcdir}/ndctl \
+ -I${top_srcdir}/cxl \
-I${top_srcdir}/ \
$(KMOD_CFLAGS) \
$(UDEV_CFLAGS) \
diff --git a/cxl/cxl_mem.h b/cxl/cxl_mem.h
new file mode 100644
index 0000000..d38cc9c
--- /dev/null
+++ b/cxl/cxl_mem.h
@@ -0,0 +1,189 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+/* Copyright (C) 2020-2021, Intel Corporation. All rights reserved. */
+/*
+ * CXL IOCTLs for Memory Devices
+ */
+
+#ifndef _UAPI_CXL_MEM_H_
+#define _UAPI_CXL_MEM_H_
+
+#include <linux/types.h>
+#include <sys/user.h>
+#include <unistd.h>
+
+#define __user
+
+/**
+ * DOC: UAPI
+ *
+ * Not all of all commands that the driver supports are always available for use
+ * by userspace. Userspace must check the results from the QUERY command in
+ * order to determine the live set of commands.
+ */
+
+#define CXL_MEM_QUERY_COMMANDS _IOR(0xCE, 1, struct cxl_mem_query_commands)
+#define CXL_MEM_SEND_COMMAND _IOWR(0xCE, 2, struct cxl_send_command)
+
+#define CXL_CMDS \
+ ___C(INVALID, "Invalid Command"), \
+ ___C(IDENTIFY, "Identify Command"), \
+ ___C(RAW, "Raw device command"), \
+ ___C(GET_SUPPORTED_LOGS, "Get Supported Logs"), \
+ ___C(GET_FW_INFO, "Get FW Info"), \
+ ___C(GET_PARTITION_INFO, "Get Partition Information"), \
+ ___C(GET_LSA, "Get Label Storage Area"), \
+ ___C(GET_HEALTH_INFO, "Get Health Info"), \
+ ___C(GET_LOG, "Get Log"), \
+ ___C(SET_PARTITION_INFO, "Set Partition Information"), \
+ ___C(SET_LSA, "Set Label Storage Area"), \
+ ___C(GET_ALERT_CONFIG, "Get Alert Configuration"), \
+ ___C(SET_ALERT_CONFIG, "Set Alert Configuration"), \
+ ___C(GET_SHUTDOWN_STATE, "Get Shutdown State"), \
+ ___C(SET_SHUTDOWN_STATE, "Set Shutdown State"), \
+ ___C(GET_POISON, "Get Poison List"), \
+ ___C(INJECT_POISON, "Inject Poison"), \
+ ___C(CLEAR_POISON, "Clear Poison"), \
+ ___C(GET_SCAN_MEDIA_CAPS, "Get Scan Media Capabilities"), \
+ ___C(SCAN_MEDIA, "Scan Media"), \
+ ___C(GET_SCAN_MEDIA, "Get Scan Media Results"), \
+ ___C(MAX, "invalid / last command")
+
+#define ___C(a, b) CXL_MEM_COMMAND_ID_##a
+enum { CXL_CMDS };
+
+#undef ___C
+#define ___C(a, b) { b }
+static const struct {
+ const char *name;
+} cxl_command_names[] = { CXL_CMDS };
+
+/*
+ * Here's how this actually breaks out:
+ * cxl_command_names[] = {
+ * [CXL_MEM_COMMAND_ID_INVALID] = { "Invalid Command" },
+ * [CXL_MEM_COMMAND_ID_IDENTIFY] = { "Identify Command" },
+ * ...
+ * [CXL_MEM_COMMAND_ID_MAX] = { "invalid / last command" },
+ * };
+ */
+
+#undef ___C
+
+/**
+ * struct cxl_command_info - Command information returned from a query.
+ * @id: ID number for the command.
+ * @flags: Flags that specify command behavior.
+ * @size_in: Expected input size, or -1 if variable length.
+ * @size_out: Expected output size, or -1 if variable length.
+ *
+ * Represents a single command that is supported by both the driver and the
+ * hardware. This is returned as part of an array from the query ioctl. The
+ * following would be a command that takes a variable length input and returns 0
+ * bytes of output.
+ *
+ * - @id = 10
+ * - @flags = 0
+ * - @size_in = -1
+ * - @size_out = 0
+ *
+ * See struct cxl_mem_query_commands.
+ */
+struct cxl_command_info {
+ __u32 id;
+
+ __u32 flags;
+#define CXL_MEM_COMMAND_FLAG_MASK GENMASK(0, 0)
+
+ __s32 size_in;
+ __s32 size_out;
+};
+
+/**
+ * struct cxl_mem_query_commands - Query supported commands.
+ * @n_commands: In/out parameter. When @n_commands is > 0, the driver will
+ * return min(num_support_commands, n_commands). When @n_commands
+ * is 0, driver will return the number of total supported commands.
+ * @rsvd: Reserved for future use.
+ * @commands: Output array of supported commands. This array must be allocated
+ * by userspace to be at least min(num_support_commands, @n_commands)
+ *
+ * Allow userspace to query the available commands supported by both the driver,
+ * and the hardware. Commands that aren't supported by either the driver, or the
+ * hardware are not returned in the query.
+ *
+ * Examples:
+ *
+ * - { .n_commands = 0 } // Get number of supported commands
+ * - { .n_commands = 15, .commands = buf } // Return first 15 (or less)
+ * supported commands
+ *
+ * See struct cxl_command_info.
+ */
+struct cxl_mem_query_commands {
+ /*
+ * Input: Number of commands to return (space allocated by user)
+ * Output: Number of commands supported by the driver/hardware
+ *
+ * If n_commands is 0, kernel will only return number of commands and
+ * not try to populate commands[], thus allowing userspace to know how
+ * much space to allocate
+ */
+ __u32 n_commands;
+ __u32 rsvd;
+
+ struct cxl_command_info __user commands[]; /* out: supported commands */
+};
+
+/**
+ * struct cxl_send_command - Send a command to a memory device.
+ * @id: The command to send to the memory device. This must be one of the
+ * commands returned by the query command.
+ * @flags: Flags for the command (input).
+ * @raw: Special fields for raw commands
+ * @raw.opcode: Opcode passed to hardware when using the RAW command.
+ * @raw.rsvd: Must be zero.
+ * @rsvd: Must be zero.
+ * @retval: Return value from the memory device (output).
+ * @in: Parameters associated with input payload.
+ * @in.size: Size of the payload to provide to the device (input).
+ * @in.rsvd: Must be zero.
+ * @in.payload: Pointer to memory for payload input, payload is little endian.
+ * @out: Parameters associated with output payload.
+ * @out.size: Size of the payload received from the device (input/output). This
+ * field is filled in by userspace to let the driver know how much
+ * space was allocated for output. It is populated by the driver to
+ * let userspace know how large the output payload actually was.
+ * @out.rsvd: Must be zero.
+ * @out.payload: Pointer to memory for payload output, payload is little endian.
+ *
+ * Mechanism for userspace to send a command to the hardware for processing. The
+ * driver will do basic validation on the command sizes. In some cases even the
+ * payload may be introspected. Userspace is required to allocate large enough
+ * buffers for size_out which can be variable length in certain situations.
+ */
+struct cxl_send_command {
+ __u32 id;
+ __u32 flags;
+ union {
+ struct {
+ __u16 opcode;
+ __u16 rsvd;
+ } raw;
+ __u32 rsvd;
+ };
+ __u32 retval;
+
+ struct {
+ __s32 size;
+ __u32 rsvd;
+ __u64 payload;
+ } in;
+
+ struct {
+ __s32 size;
+ __u32 rsvd;
+ __u64 payload;
+ } out;
+};
+
+#endif
diff --git a/cxl/lib/Makefile.am b/cxl/lib/Makefile.am
index 277f0cd..72c9ccd 100644
--- a/cxl/lib/Makefile.am
+++ b/cxl/lib/Makefile.am
@@ -3,7 +3,7 @@ include $(top_srcdir)/Makefile.am.in
%.pc: %.pc.in Makefile
$(SED_PROCESS)
-pkginclude_HEADERS = ../libcxl.h
+pkginclude_HEADERS = ../libcxl.h ../cxl_mem.h
lib_LTLIBRARIES = libcxl.la
libcxl_la_SOURCES =\
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* [ndctl PATCH v5 04/16] util: add the struct_size() helper from the kernel
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
` (2 preceding siblings ...)
2021-11-11 20:44 ` [ndctl PATCH v5 03/16] cxl: add a local copy of the cxl_mem UAPI header Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2022-01-31 11:47 ` Joao Martins
2021-11-11 20:44 ` [ndctl PATCH v5 05/16] libcxl: add support for command query and submission Vishal Verma
` (11 subsequent siblings)
15 siblings, 1 reply; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Add struct_size() from include/linux/overflow.h which calculates the
size of a struct with a trailing variable length array.
Suggested-by: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
util/size.h | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++++
util/util.h | 6 ++++++
2 files changed, 68 insertions(+)
diff --git a/util/size.h b/util/size.h
index 646edae..a0f3593 100644
--- a/util/size.h
+++ b/util/size.h
@@ -4,6 +4,8 @@
#ifndef _NDCTL_SIZE_H_
#define _NDCTL_SIZE_H_
#include <stdbool.h>
+#include <stdint.h>
+#include <util/util.h>
#define SZ_1K 0x00000400
#define SZ_4K 0x00001000
@@ -30,4 +32,64 @@ static inline bool is_power_of_2(unsigned long long v)
#define BITS_PER_LONG (sizeof(unsigned long) * 8)
#define HPAGE_SIZE (2 << 20)
+/*
+ * Helpers for struct_size() copied from include/linux/overflow.h (GPL-2.0)
+ *
+ * For simplicity and code hygiene, the fallback code below insists on
+ * a, b and *d having the same type (similar to the min() and max()
+ * macros), whereas gcc's type-generic overflow checkers accept
+ * different types. Hence we don't just make check_add_overflow an
+ * alias for __builtin_add_overflow, but add type checks similar to
+ * below.
+ */
+#define check_add_overflow(a, b, d) (({ \
+ typeof(a) __a = (a); \
+ typeof(b) __b = (b); \
+ typeof(d) __d = (d); \
+ (void) (&__a == &__b); \
+ (void) (&__a == __d); \
+ __builtin_add_overflow(__a, __b, __d); \
+}))
+
+#define check_mul_overflow(a, b, d) (({ \
+ typeof(a) __a = (a); \
+ typeof(b) __b = (b); \
+ typeof(d) __d = (d); \
+ (void) (&__a == &__b); \
+ (void) (&__a == __d); \
+ __builtin_mul_overflow(__a, __b, __d); \
+}))
+
+/*
+ * Compute a*b+c, returning SIZE_MAX on overflow. Internal helper for
+ * struct_size() below.
+ */
+static inline size_t __ab_c_size(size_t a, size_t b, size_t c)
+{
+ size_t bytes;
+
+ if (check_mul_overflow(a, b, &bytes))
+ return SIZE_MAX;
+ if (check_add_overflow(bytes, c, &bytes))
+ return SIZE_MAX;
+
+ return bytes;
+}
+
+/**
+ * struct_size() - Calculate size of structure with trailing array.
+ * @p: Pointer to the structure.
+ * @member: Name of the array member.
+ * @count: Number of elements in the array.
+ *
+ * Calculates size of memory needed for structure @p followed by an
+ * array of @count number of @member elements.
+ *
+ * Return: number of bytes needed or SIZE_MAX on overflow.
+ */
+#define struct_size(p, member, count) \
+ __ab_c_size(count, \
+ sizeof(*(p)->member) + __must_be_array((p)->member),\
+ sizeof(*(p)))
+
#endif /* _NDCTL_SIZE_H_ */
diff --git a/util/util.h b/util/util.h
index ae0e4e1..b2b4ae6 100644
--- a/util/util.h
+++ b/util/util.h
@@ -63,6 +63,12 @@
#define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:-!!(e); }))
#define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)]))
+/* Are two types/vars the same type (ignoring qualifiers)? */
+#define __same_type(a, b) __builtin_types_compatible_p(typeof(a), typeof(b))
+
+/* &a[0] degrades to a pointer: a different type from an array */
+#define __must_be_array(a) BUILD_BUG_ON_ZERO(__same_type((a), &(a)[0]))
+
enum {
READ, WRITE,
};
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* Re: [ndctl PATCH v5 04/16] util: add the struct_size() helper from the kernel
2021-11-11 20:44 ` [ndctl PATCH v5 04/16] util: add the struct_size() helper from the kernel Vishal Verma
@ 2022-01-31 11:47 ` Joao Martins
0 siblings, 0 replies; 31+ messages in thread
From: Joao Martins @ 2022-01-31 11:47 UTC (permalink / raw)
To: Vishal Verma; +Cc: Dan Williams, Ben Widawsky, nvdimm, linux-cxl
Hey Vishal,
On 11/11/21 20:44, Vishal Verma wrote:
> Add struct_size() from include/linux/overflow.h which calculates the
> size of a struct with a trailing variable length array.
[...]
> +/*
> + * Helpers for struct_size() copied from include/linux/overflow.h (GPL-2.0)
> + *
> + * For simplicity and code hygiene, the fallback code below insists on
> + * a, b and *d having the same type (similar to the min() and max()
> + * macros), whereas gcc's type-generic overflow checkers accept
> + * different types. Hence we don't just make check_add_overflow an
> + * alias for __builtin_add_overflow, but add type checks similar to
> + * below.
> + */
> +#define check_add_overflow(a, b, d) (({ \
> + typeof(a) __a = (a); \
> + typeof(b) __b = (b); \
> + typeof(d) __d = (d); \
> + (void) (&__a == &__b); \
> + (void) (&__a == __d); \
> + __builtin_add_overflow(__a, __b, __d); \
> +}))
> +
> +#define check_mul_overflow(a, b, d) (({ \
> + typeof(a) __a = (a); \
> + typeof(b) __b = (b); \
> + typeof(d) __d = (d); \
> + (void) (&__a == &__b); \
> + (void) (&__a == __d); \
> + __builtin_mul_overflow(__a, __b, __d); \
> +}))
> +
The introduction of this two macro helpers broke compilation against < gcc 5.1.0 (think
CentOS/OL/RHEL 7 or anything resembling that) and clang 3.4. Particularly because of the
lack of these overflow builtins __builtin_mul_overflow() / __buitin_add_overflow() giving
errors like:
BUILDSTDERR: lib/.libs/libcxl.so: undefined reference to `__builtin_mul_overflow'
BUILDSTDERR: lib/.libs/libcxl.so: undefined reference to `__builtin_add_overflow'
Since you pulled this helper from the kernel, you might wanna pull the compat code too
that was there in commit f0907827a8a9 ("compiler.h: enable builtin overflow checkers and
add fallback code"), particularly something like below.
#if GCC_VERSION >= 50100
#define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1
#endif
#if __clang__ && \
__has_builtin(__builtin_mul_overflow) && \
__has_builtin(__builtin_add_overflow)
#define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1
#endif
#if COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW
#define check_add_overflow(a, b, d) (({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
__builtin_add_overflow(__a, __b, __d); \
}))
#define check_mul_overflow(a, b, d) (({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
__builtin_mul_overflow(__a, __b, __d); \
}))
#else /* !COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW */
/* Checking for unsigned overflow is relatively easy without causing UB. */
#define __unsigned_add_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
*__d = __a + __b; \
*__d < __a; \
})
#define __unsigned_sub_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
*__d = __a - __b; \
__a < __b; \
})
/*
* If one of a or b is a compile-time constant, this avoids a division.
*/
#define __unsigned_mul_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
*__d = __a * __b; \
__builtin_constant_p(__b) ? \
__b > 0 && __a > type_max(typeof(__a)) / __b : \
__a > 0 && __b > type_max(typeof(__b)) / __a; \
})
/*
* For signed types, detecting overflow is much harder, especially if
* we want to avoid UB. But the interface of these macros is such that
* we must provide a result in *d, and in fact we must produce the
* result promised by gcc's builtins, which is simply the possibly
* wrapped-around value. Fortunately, we can just formally do the
* operations in the widest relevant unsigned type (u64) and then
* truncate the result - gcc is smart enough to generate the same code
* with and without the (u64) casts.
*/
/*
* Adding two signed integers can overflow only if they have the same
* sign, and overflow has happened iff the result has the opposite
* sign.
*/
#define __signed_add_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
*__d = (u64)__a + (u64)__b; \
(((~(__a ^ __b)) & (*__d ^ __a)) \
& type_min(typeof(__a))) != 0; \
})
/*
* Subtraction is similar, except that overflow can now happen only
* when the signs are opposite. In this case, overflow has happened if
* the result has the opposite sign of a.
*/
#define __signed_sub_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
*__d = (u64)__a - (u64)__b; \
((((__a ^ __b)) & (*__d ^ __a)) \
& type_min(typeof(__a))) != 0; \
})
/*
* Signed multiplication is rather hard. gcc always follows C99, so
* division is truncated towards 0. This means that we can write the
* overflow check like this:
*
* (a > 0 && (b > MAX/a || b < MIN/a)) ||
* (a < -1 && (b > MIN/a || b < MAX/a) ||
* (a == -1 && b == MIN)
*
* The redundant casts of -1 are to silence an annoying -Wtype-limits
* (included in -Wextra) warning: When the type is u8 or u16, the
* __b_c_e in check_mul_overflow obviously selects
* __unsigned_mul_overflow, but unfortunately gcc still parses this
* code and warns about the limited range of __b.
*/
#define __signed_mul_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
typeof(a) __tmax = type_max(typeof(a)); \
typeof(a) __tmin = type_min(typeof(a)); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
*__d = (u64)__a * (u64)__b; \
(__b > 0 && (__a > __tmax/__b || __a < __tmin/__b)) || \
(__b < (typeof(__b))-1 && (__a > __tmin/__b || __a < __tmax/__b)) || \
(__b == (typeof(__b))-1 && __a == __tmin); \
})
#define check_add_overflow(a, b, d) \
__builtin_choose_expr(is_signed_type(typeof(a)), \
__signed_add_overflow(a, b, d), \
__unsigned_add_overflow(a, b, d))
#define check_sub_overflow(a, b, d) \
__builtin_choose_expr(is_signed_type(typeof(a)), \
__signed_sub_overflow(a, b, d), \
__unsigned_sub_overflow(a, b, d))
#define check_mul_overflow(a, b, d) \
__builtin_choose_expr(is_signed_type(typeof(a)), \
__signed_mul_overflow(a, b, d), \
__unsigned_mul_overflow(a, b, d))
#endif
^ permalink raw reply [flat|nested] 31+ messages in thread
* [ndctl PATCH v5 05/16] libcxl: add support for command query and submission
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
` (3 preceding siblings ...)
2021-11-11 20:44 ` [ndctl PATCH v5 04/16] util: add the struct_size() helper from the kernel Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 06/16] libcxl: add support for the 'Identify Device' command Vishal Verma
` (10 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Add a set of APIs around 'cxl_cmd' for querying the kernel for supported
commands, allocating and validating command structures against the
supported set, and submitting the commands.
'Query Commands' and 'Send Command' are implemented as IOCTLs in the
kernel. 'Query Commands' returns information about each supported
command, such as flags governing its use, or input and output payload
sizes. This information is used to validate command support, as well as
set up input and output buffers for command submission.
Cc: Ben Widawsky <ben.widawsky@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
cxl/lib/private.h | 33 ++++
cxl/lib/libcxl.c | 390 +++++++++++++++++++++++++++++++++++++++++++++
cxl/libcxl.h | 11 ++
cxl/lib/libcxl.sym | 9 ++
4 files changed, 443 insertions(+)
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
index fc88fa1..87ca17e 100644
--- a/cxl/lib/private.h
+++ b/cxl/lib/private.h
@@ -4,6 +4,9 @@
#define _LIBCXL_PRIVATE_H_
#include <libkmod.h>
+#include <cxl/cxl_mem.h>
+#include <ccan/endian/endian.h>
+#include <ccan/short_types/short_types.h>
#define CXL_EXPORT __attribute__ ((visibility("default")))
@@ -21,6 +24,36 @@ struct cxl_memdev {
struct kmod_module *module;
};
+enum cxl_cmd_query_status {
+ CXL_CMD_QUERY_NOT_RUN = 0,
+ CXL_CMD_QUERY_OK,
+ CXL_CMD_QUERY_UNSUPPORTED,
+};
+
+/**
+ * struct cxl_cmd - CXL memdev command
+ * @memdev: the memory device to which the command is being sent
+ * @query_cmd: structure for the Linux 'Query commands' ioctl
+ * @send_cmd: structure for the Linux 'Send command' ioctl
+ * @input_payload: buffer for input payload managed by libcxl
+ * @output_payload: buffer for output payload managed by libcxl
+ * @refcount: reference for passing command buffer around
+ * @query_status: status from query_commands
+ * @query_idx: index of 'this' command in the query_commands array
+ * @status: command return status from the device
+ */
+struct cxl_cmd {
+ struct cxl_memdev *memdev;
+ struct cxl_mem_query_commands *query_cmd;
+ struct cxl_send_command *send_cmd;
+ void *input_payload;
+ void *output_payload;
+ int refcount;
+ int query_status;
+ int query_idx;
+ int status;
+};
+
static inline int check_kmod(struct kmod_ctx *kmod_ctx)
{
return kmod_ctx ? 0 : -ENXIO;
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
index c15e987..727d599 100644
--- a/cxl/lib/libcxl.c
+++ b/cxl/lib/libcxl.c
@@ -9,14 +9,17 @@
#include <unistd.h>
#include <sys/stat.h>
#include <sys/types.h>
+#include <sys/ioctl.h>
#include <sys/sysmacros.h>
#include <uuid/uuid.h>
#include <ccan/list/list.h>
#include <ccan/array_size/array_size.h>
#include <util/log.h>
+#include <util/size.h>
#include <util/sysfs.h>
#include <util/bitmap.h>
+#include <cxl/cxl_mem.h>
#include <cxl/libcxl.h>
#include "private.h"
@@ -343,3 +346,390 @@ CXL_EXPORT const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev
{
return memdev->firmware_version;
}
+
+CXL_EXPORT void cxl_cmd_unref(struct cxl_cmd *cmd)
+{
+ if (!cmd)
+ return;
+ if (--cmd->refcount == 0) {
+ free(cmd->query_cmd);
+ free(cmd->send_cmd);
+ free(cmd->input_payload);
+ free(cmd->output_payload);
+ free(cmd);
+ }
+}
+
+CXL_EXPORT void cxl_cmd_ref(struct cxl_cmd *cmd)
+{
+ cmd->refcount++;
+}
+
+static int cxl_cmd_alloc_query(struct cxl_cmd *cmd, int num_cmds)
+{
+ size_t size;
+
+ if (!cmd)
+ return -EINVAL;
+
+ if (cmd->query_cmd != NULL)
+ free(cmd->query_cmd);
+
+ size = struct_size(cmd->query_cmd, commands, num_cmds);
+ if (size == SIZE_MAX)
+ return -EOVERFLOW;
+
+ cmd->query_cmd = calloc(1, size);
+ if (!cmd->query_cmd)
+ return -ENOMEM;
+
+ cmd->query_cmd->n_commands = num_cmds;
+
+ return 0;
+}
+
+static struct cxl_cmd *cxl_cmd_new(struct cxl_memdev *memdev)
+{
+ struct cxl_cmd *cmd;
+ size_t size;
+
+ size = sizeof(*cmd);
+ cmd = calloc(1, size);
+ if (!cmd)
+ return NULL;
+
+ cxl_cmd_ref(cmd);
+ cmd->memdev = memdev;
+
+ return cmd;
+}
+
+static int __do_cmd(struct cxl_cmd *cmd, int ioctl_cmd, int fd)
+{
+ void *cmd_buf;
+ int rc;
+
+ switch (ioctl_cmd) {
+ case CXL_MEM_QUERY_COMMANDS:
+ cmd_buf = cmd->query_cmd;
+ break;
+ case CXL_MEM_SEND_COMMAND:
+ cmd_buf = cmd->send_cmd;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ rc = ioctl(fd, ioctl_cmd, cmd_buf);
+ if (rc < 0)
+ rc = -errno;
+
+ return rc;
+}
+
+static int do_cmd(struct cxl_cmd *cmd, int ioctl_cmd)
+{
+ char *path;
+ struct stat st;
+ unsigned int major, minor;
+ int rc = 0, fd;
+ struct cxl_memdev *memdev = cmd->memdev;
+ struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+ const char *devname = cxl_memdev_get_devname(memdev);
+
+ major = cxl_memdev_get_major(memdev);
+ minor = cxl_memdev_get_minor(memdev);
+
+ if (asprintf(&path, "/dev/cxl/%s", devname) < 0)
+ return -ENOMEM;
+
+ fd = open(path, O_RDWR);
+ if (fd < 0) {
+ err(ctx, "failed to open %s: %s\n", path, strerror(errno));
+ rc = -errno;
+ goto out;
+ }
+
+ if (fstat(fd, &st) >= 0 && S_ISCHR(st.st_mode)
+ && major(st.st_rdev) == major
+ && minor(st.st_rdev) == minor) {
+ rc = __do_cmd(cmd, ioctl_cmd, fd);
+ } else {
+ err(ctx, "failed to validate %s as a CXL memdev node\n", path);
+ rc = -ENXIO;
+ }
+ close(fd);
+out:
+ free(path);
+ return rc;
+}
+
+static int alloc_do_query(struct cxl_cmd *cmd, int num_cmds)
+{
+ struct cxl_ctx *ctx = cxl_memdev_get_ctx(cmd->memdev);
+ int rc;
+
+ rc = cxl_cmd_alloc_query(cmd, num_cmds);
+ if (rc)
+ return rc;
+
+ rc = do_cmd(cmd, CXL_MEM_QUERY_COMMANDS);
+ if (rc < 0)
+ err(ctx, "%s: query commands failed: %s\n",
+ cxl_memdev_get_devname(cmd->memdev),
+ strerror(-rc));
+ return rc;
+}
+
+static int cxl_cmd_do_query(struct cxl_cmd *cmd)
+{
+ struct cxl_memdev *memdev = cmd->memdev;
+ struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+ const char *devname = cxl_memdev_get_devname(memdev);
+ int rc, n_commands;
+
+ switch (cmd->query_status) {
+ case CXL_CMD_QUERY_OK:
+ return 0;
+ case CXL_CMD_QUERY_UNSUPPORTED:
+ return -EOPNOTSUPP;
+ case CXL_CMD_QUERY_NOT_RUN:
+ break;
+ default:
+ err(ctx, "%s: Unknown query_status %d\n",
+ devname, cmd->query_status);
+ return -EINVAL;
+ }
+
+ rc = alloc_do_query(cmd, 0);
+ if (rc)
+ return rc;
+
+ n_commands = cmd->query_cmd->n_commands;
+ dbg(ctx, "%s: supports %d commands\n", devname, n_commands);
+
+ return alloc_do_query(cmd, n_commands);
+}
+
+static int cxl_cmd_validate(struct cxl_cmd *cmd, u32 cmd_id)
+{
+ struct cxl_memdev *memdev = cmd->memdev;
+ struct cxl_mem_query_commands *query = cmd->query_cmd;
+ const char *devname = cxl_memdev_get_devname(memdev);
+ struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+ u32 i;
+
+ for (i = 0; i < query->n_commands; i++) {
+ struct cxl_command_info *cinfo = &query->commands[i];
+ const char *cmd_name = cxl_command_names[cinfo->id].name;
+
+ if (cinfo->id != cmd_id)
+ continue;
+
+ dbg(ctx, "%s: %s: in: %d, out %d, flags: %#08x\n",
+ devname, cmd_name, cinfo->size_in,
+ cinfo->size_out, cinfo->flags);
+
+ cmd->query_idx = i;
+ cmd->query_status = CXL_CMD_QUERY_OK;
+ return 0;
+ }
+ cmd->query_status = CXL_CMD_QUERY_UNSUPPORTED;
+ return -EOPNOTSUPP;
+}
+
+CXL_EXPORT int cxl_cmd_set_input_payload(struct cxl_cmd *cmd, void *buf,
+ int size)
+{
+ struct cxl_memdev *memdev = cmd->memdev;
+
+ if (size > memdev->payload_max || size < 0)
+ return -EINVAL;
+
+ if (!buf) {
+
+ /* If the user didn't supply a buffer, allocate it */
+ cmd->input_payload = calloc(1, size);
+ if (!cmd->input_payload)
+ return -ENOMEM;
+ cmd->send_cmd->in.payload = (u64)cmd->input_payload;
+ } else {
+ /*
+ * Use user-buffer as is. If an automatic allocation was
+ * previously made (based on a fixed size from query),
+ * it will get freed during unref.
+ */
+ cmd->send_cmd->in.payload = (u64)buf;
+ }
+ cmd->send_cmd->in.size = size;
+
+ return 0;
+}
+
+CXL_EXPORT int cxl_cmd_set_output_payload(struct cxl_cmd *cmd, void *buf,
+ int size)
+{
+ struct cxl_memdev *memdev = cmd->memdev;
+
+ if (size > memdev->payload_max || size < 0)
+ return -EINVAL;
+
+ if (!buf) {
+
+ /* If the user didn't supply a buffer, allocate it */
+ cmd->output_payload = calloc(1, size);
+ if (!cmd->output_payload)
+ return -ENOMEM;
+ cmd->send_cmd->out.payload = (u64)cmd->output_payload;
+ } else {
+ /*
+ * Use user-buffer as is. If an automatic allocation was
+ * previously made (based on a fixed size from query),
+ * it will get freed during unref.
+ */
+ cmd->send_cmd->out.payload = (u64)buf;
+ }
+ cmd->send_cmd->out.size = size;
+
+ return 0;
+}
+
+static int cxl_cmd_alloc_send(struct cxl_cmd *cmd, u32 cmd_id)
+{
+ struct cxl_mem_query_commands *query = cmd->query_cmd;
+ struct cxl_command_info *cinfo = &query->commands[cmd->query_idx];
+ size_t size;
+
+ if (!query)
+ return -EINVAL;
+
+ size = sizeof(struct cxl_send_command);
+ cmd->send_cmd = calloc(1, size);
+ if (!cmd->send_cmd)
+ return -ENOMEM;
+
+ if (cinfo->id != cmd_id)
+ return -EINVAL;
+
+ cmd->send_cmd->id = cmd_id;
+
+ if (cinfo->size_in > 0) {
+ cmd->input_payload = calloc(1, cinfo->size_in);
+ if (!cmd->input_payload)
+ return -ENOMEM;
+ cmd->send_cmd->in.payload = (u64)cmd->input_payload;
+ cmd->send_cmd->in.size = cinfo->size_in;
+ }
+ if (cinfo->size_out > 0) {
+ cmd->output_payload = calloc(1, cinfo->size_out);
+ if (!cmd->output_payload)
+ return -ENOMEM;
+ cmd->send_cmd->out.payload = (u64)cmd->output_payload;
+ cmd->send_cmd->out.size = cinfo->size_out;
+ }
+
+ return 0;
+}
+
+static struct cxl_cmd *cxl_cmd_new_generic(struct cxl_memdev *memdev,
+ u32 cmd_id)
+{
+ const char *devname = cxl_memdev_get_devname(memdev);
+ struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+ struct cxl_cmd *cmd;
+ int rc;
+
+ cmd = cxl_cmd_new(memdev);
+ if (!cmd)
+ return NULL;
+
+ rc = cxl_cmd_do_query(cmd);
+ if (rc) {
+ err(ctx, "%s: query returned: %s\n", devname, strerror(-rc));
+ goto fail;
+ }
+
+ rc = cxl_cmd_validate(cmd, cmd_id);
+ if (rc) {
+ errno = -rc;
+ goto fail;
+ }
+
+ rc = cxl_cmd_alloc_send(cmd, cmd_id);
+ if (rc) {
+ errno = -rc;
+ goto fail;
+ }
+
+ cmd->status = 1;
+ return cmd;
+
+fail:
+ cxl_cmd_unref(cmd);
+ return NULL;
+}
+
+CXL_EXPORT const char *cxl_cmd_get_devname(struct cxl_cmd *cmd)
+{
+ return cxl_memdev_get_devname(cmd->memdev);
+}
+
+CXL_EXPORT struct cxl_cmd *cxl_cmd_new_raw(struct cxl_memdev *memdev,
+ int opcode)
+{
+ struct cxl_cmd *cmd;
+
+ /* opcode '0' is reserved */
+ if (opcode <= 0) {
+ errno = EINVAL;
+ return NULL;
+ }
+
+ cmd = cxl_cmd_new_generic(memdev, CXL_MEM_COMMAND_ID_RAW);
+ if (!cmd)
+ return NULL;
+
+ cmd->send_cmd->raw.opcode = opcode;
+ return cmd;
+}
+
+CXL_EXPORT int cxl_cmd_submit(struct cxl_cmd *cmd)
+{
+ struct cxl_memdev *memdev = cmd->memdev;
+ const char *devname = cxl_memdev_get_devname(memdev);
+ struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+ int rc;
+
+ switch (cmd->query_status) {
+ case CXL_CMD_QUERY_OK:
+ break;
+ case CXL_CMD_QUERY_UNSUPPORTED:
+ return -EOPNOTSUPP;
+ case CXL_CMD_QUERY_NOT_RUN:
+ return -EINVAL;
+ default:
+ err(ctx, "%s: Unknown query_status %d\n",
+ devname, cmd->query_status);
+ return -EINVAL;
+ }
+
+ dbg(ctx, "%s: submitting SEND cmd: in: %d, out: %d\n", devname,
+ cmd->send_cmd->in.size, cmd->send_cmd->out.size);
+ rc = do_cmd(cmd, CXL_MEM_SEND_COMMAND);
+ cmd->status = cmd->send_cmd->retval;
+ dbg(ctx, "%s: got SEND cmd: in: %d, out: %d, retval: %d, status: %d\n",
+ devname, cmd->send_cmd->in.size, cmd->send_cmd->out.size,
+ rc, cmd->status);
+
+ return rc;
+}
+
+CXL_EXPORT int cxl_cmd_get_mbox_status(struct cxl_cmd *cmd)
+{
+ return cmd->status;
+}
+
+CXL_EXPORT int cxl_cmd_get_out_size(struct cxl_cmd *cmd)
+{
+ return cmd->send_cmd->out.size;
+}
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
index fd06790..6e87b80 100644
--- a/cxl/libcxl.h
+++ b/cxl/libcxl.h
@@ -48,6 +48,17 @@ const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev);
memdev != NULL; \
memdev = cxl_memdev_get_next(memdev))
+struct cxl_cmd;
+const char *cxl_cmd_get_devname(struct cxl_cmd *cmd);
+struct cxl_cmd *cxl_cmd_new_raw(struct cxl_memdev *memdev, int opcode);
+int cxl_cmd_set_input_payload(struct cxl_cmd *cmd, void *in, int size);
+int cxl_cmd_set_output_payload(struct cxl_cmd *cmd, void *out, int size);
+void cxl_cmd_ref(struct cxl_cmd *cmd);
+void cxl_cmd_unref(struct cxl_cmd *cmd);
+int cxl_cmd_submit(struct cxl_cmd *cmd);
+int cxl_cmd_get_mbox_status(struct cxl_cmd *cmd);
+int cxl_cmd_get_out_size(struct cxl_cmd *cmd);
+
#ifdef __cplusplus
} /* extern "C" */
#endif
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
index 2616e5c..3900f90 100644
--- a/cxl/lib/libcxl.sym
+++ b/cxl/lib/libcxl.sym
@@ -20,6 +20,15 @@ global:
cxl_memdev_get_pmem_size;
cxl_memdev_get_ram_size;
cxl_memdev_get_firmware_verison;
+ cxl_cmd_get_devname;
+ cxl_cmd_new_raw;
+ cxl_cmd_set_input_payload;
+ cxl_cmd_set_output_payload;
+ cxl_cmd_ref;
+ cxl_cmd_unref;
+ cxl_cmd_submit;
+ cxl_cmd_get_mbox_status;
+ cxl_cmd_get_out_size;
local:
*;
};
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* [ndctl PATCH v5 06/16] libcxl: add support for the 'Identify Device' command
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
` (4 preceding siblings ...)
2021-11-11 20:44 ` [ndctl PATCH v5 05/16] libcxl: add support for command query and submission Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 07/16] libcxl: add GET_HEALTH_INFO mailbox command and accessors Vishal Verma
` (9 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Add APIs to allocate and send an 'Identify Device' command, and
accessors to retrieve some of the fields from the resulting data.
Only add a handful accessor functions; more can be added as the need
arises. The fields added are fw_revision, partition_align, and
lsa_size.
Cc: Ben Widawsky <ben.widawsky@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
cxl/lib/private.h | 19 +++++++++++++++++
cxl/lib/libcxl.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++
cxl/libcxl.h | 4 ++++
cxl/lib/libcxl.sym | 4 ++++
4 files changed, 79 insertions(+)
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
index 87ca17e..3273f21 100644
--- a/cxl/lib/private.h
+++ b/cxl/lib/private.h
@@ -54,6 +54,25 @@ struct cxl_cmd {
int status;
};
+#define CXL_CMD_IDENTIFY_FW_REV_LENGTH 0x10
+
+struct cxl_cmd_identify {
+ char fw_revision[CXL_CMD_IDENTIFY_FW_REV_LENGTH];
+ le64 total_capacity;
+ le64 volatile_capacity;
+ le64 persistent_capacity;
+ le64 partition_align;
+ le16 info_event_log_size;
+ le16 warning_event_log_size;
+ le16 failure_event_log_size;
+ le16 fatal_event_log_size;
+ le32 lsa_size;
+ u8 poison_list_max_mer[3];
+ le16 inject_poison_limit;
+ u8 poison_caps;
+ u8 qos_telemetry_caps;
+} __attribute__((packed));
+
static inline int check_kmod(struct kmod_ctx *kmod_ctx)
{
return kmod_ctx ? 0 : -ENXIO;
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
index 727d599..ed21670 100644
--- a/cxl/lib/libcxl.c
+++ b/cxl/lib/libcxl.c
@@ -13,7 +13,10 @@
#include <sys/sysmacros.h>
#include <uuid/uuid.h>
#include <ccan/list/list.h>
+#include <ccan/endian/endian.h>
+#include <ccan/minmax/minmax.h>
#include <ccan/array_size/array_size.h>
+#include <ccan/short_types/short_types.h>
#include <util/log.h>
#include <util/size.h>
@@ -674,6 +677,55 @@ CXL_EXPORT const char *cxl_cmd_get_devname(struct cxl_cmd *cmd)
return cxl_memdev_get_devname(cmd->memdev);
}
+CXL_EXPORT struct cxl_cmd *cxl_cmd_new_identify(struct cxl_memdev *memdev)
+{
+ return cxl_cmd_new_generic(memdev, CXL_MEM_COMMAND_ID_IDENTIFY);
+}
+
+CXL_EXPORT int cxl_cmd_identify_get_fw_rev(struct cxl_cmd *cmd, char *fw_rev,
+ int fw_len)
+{
+ struct cxl_cmd_identify *id =
+ (struct cxl_cmd_identify *)cmd->send_cmd->out.payload;
+
+ if (cmd->send_cmd->id != CXL_MEM_COMMAND_ID_IDENTIFY)
+ return -EINVAL;
+ if (cmd->status < 0)
+ return cmd->status;
+
+ if (fw_len > 0)
+ memcpy(fw_rev, id->fw_revision,
+ min(fw_len, CXL_CMD_IDENTIFY_FW_REV_LENGTH));
+ return 0;
+}
+
+CXL_EXPORT unsigned long long cxl_cmd_identify_get_partition_align(
+ struct cxl_cmd *cmd)
+{
+ struct cxl_cmd_identify *id =
+ (struct cxl_cmd_identify *)cmd->send_cmd->out.payload;
+
+ if (cmd->send_cmd->id != CXL_MEM_COMMAND_ID_IDENTIFY)
+ return -EINVAL;
+ if (cmd->status < 0)
+ return cmd->status;
+
+ return le64_to_cpu(id->partition_align);
+}
+
+CXL_EXPORT unsigned int cxl_cmd_identify_get_label_size(struct cxl_cmd *cmd)
+{
+ struct cxl_cmd_identify *id =
+ (struct cxl_cmd_identify *)cmd->send_cmd->out.payload;
+
+ if (cmd->send_cmd->id != CXL_MEM_COMMAND_ID_IDENTIFY)
+ return -EINVAL;
+ if (cmd->status < 0)
+ return cmd->status;
+
+ return le32_to_cpu(id->lsa_size);
+}
+
CXL_EXPORT struct cxl_cmd *cxl_cmd_new_raw(struct cxl_memdev *memdev,
int opcode)
{
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
index 6e87b80..0f2d5e9 100644
--- a/cxl/libcxl.h
+++ b/cxl/libcxl.h
@@ -58,6 +58,10 @@ void cxl_cmd_unref(struct cxl_cmd *cmd);
int cxl_cmd_submit(struct cxl_cmd *cmd);
int cxl_cmd_get_mbox_status(struct cxl_cmd *cmd);
int cxl_cmd_get_out_size(struct cxl_cmd *cmd);
+struct cxl_cmd *cxl_cmd_new_identify(struct cxl_memdev *memdev);
+int cxl_cmd_identify_get_fw_rev(struct cxl_cmd *cmd, char *fw_rev, int fw_len);
+unsigned long long cxl_cmd_identify_get_partition_align(struct cxl_cmd *cmd);
+unsigned int cxl_cmd_identify_get_label_size(struct cxl_cmd *cmd);
#ifdef __cplusplus
} /* extern "C" */
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
index 3900f90..1dc45f4 100644
--- a/cxl/lib/libcxl.sym
+++ b/cxl/lib/libcxl.sym
@@ -29,6 +29,10 @@ global:
cxl_cmd_submit;
cxl_cmd_get_mbox_status;
cxl_cmd_get_out_size;
+ cxl_cmd_new_identify;
+ cxl_cmd_identify_get_fw_rev;
+ cxl_cmd_identify_get_partition_align;
+ cxl_cmd_identify_get_label_size;
local:
*;
};
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* [ndctl PATCH v5 07/16] libcxl: add GET_HEALTH_INFO mailbox command and accessors
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
` (5 preceding siblings ...)
2021-11-11 20:44 ` [ndctl PATCH v5 06/16] libcxl: add support for the 'Identify Device' command Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 23:17 ` Dan Williams
2021-11-11 20:44 ` [ndctl PATCH v5 08/16] libcxl: add support for the 'GET_LSA' command Vishal Verma
` (8 subsequent siblings)
15 siblings, 1 reply; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Add libcxl APIs to create a new GET_HEALTH_INFO mailbox command, the
command output data structure (privately), and accessor APIs to return
the different fields in the health info output.
Cc: Ben Widawsky <ben.widawsky@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
cxl/lib/private.h | 47 ++++++++
cxl/lib/libcxl.c | 291 +++++++++++++++++++++++++++++++++++++++++++++
cxl/libcxl.h | 33 +++++
util/bitmap.h | 85 +++++++++++++
cxl/lib/libcxl.sym | 29 +++++
5 files changed, 485 insertions(+)
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
index 3273f21..885553a 100644
--- a/cxl/lib/private.h
+++ b/cxl/lib/private.h
@@ -73,6 +73,53 @@ struct cxl_cmd_identify {
u8 qos_telemetry_caps;
} __attribute__((packed));
+struct cxl_cmd_get_health_info {
+ u8 health_status;
+ u8 media_status;
+ u8 ext_status;
+ u8 life_used;
+ le16 temperature;
+ le32 dirty_shutdowns;
+ le32 volatile_errors;
+ le32 pmem_errors;
+} __attribute__((packed));
+
+/* CXL 2.0 8.2.9.5.3 Byte 0 Health Status */
+#define CXL_CMD_HEALTH_INFO_STATUS_MAINTENANCE_NEEDED_MASK BIT(0)
+#define CXL_CMD_HEALTH_INFO_STATUS_PERFORMANCE_DEGRADED_MASK BIT(1)
+#define CXL_CMD_HEALTH_INFO_STATUS_HW_REPLACEMENT_NEEDED_MASK BIT(2)
+
+/* CXL 2.0 8.2.9.5.3 Byte 1 Media Status */
+#define CXL_CMD_HEALTH_INFO_MEDIA_STATUS_NORMAL 0x0
+#define CXL_CMD_HEALTH_INFO_MEDIA_STATUS_NOT_READY 0x1
+#define CXL_CMD_HEALTH_INFO_MEDIA_STATUS_PERSISTENCE_LOST 0x2
+#define CXL_CMD_HEALTH_INFO_MEDIA_STATUS_DATA_LOST 0x3
+#define CXL_CMD_HEALTH_INFO_MEDIA_STATUS_POWERLOSS_PERSISTENCE_LOSS 0x4
+#define CXL_CMD_HEALTH_INFO_MEDIA_STATUS_SHUTDOWN_PERSISTENCE_LOSS 0x5
+#define CXL_CMD_HEALTH_INFO_MEDIA_STATUS_PERSISTENCE_LOSS_IMMINENT 0x6
+#define CXL_CMD_HEALTH_INFO_MEDIA_STATUS_POWERLOSS_DATA_LOSS 0x7
+#define CXL_CMD_HEALTH_INFO_MEDIA_STATUS_SHUTDOWN_DATA_LOSS 0x8
+#define CXL_CMD_HEALTH_INFO_MEDIA_STATUS_DATA_LOSS_IMMINENT 0x9
+
+/* CXL 2.0 8.2.9.5.3 Byte 2 Additional Status */
+#define CXL_CMD_HEALTH_INFO_EXT_LIFE_USED_MASK GENMASK(1, 0)
+#define CXL_CMD_HEALTH_INFO_EXT_LIFE_USED_NORMAL (0)
+#define CXL_CMD_HEALTH_INFO_EXT_LIFE_USED_WARNING (1)
+#define CXL_CMD_HEALTH_INFO_EXT_LIFE_USED_CRITICAL (2)
+#define CXL_CMD_HEALTH_INFO_EXT_TEMPERATURE_MASK GENMASK(3, 2)
+#define CXL_CMD_HEALTH_INFO_EXT_TEMPERATURE_NORMAL (0)
+#define CXL_CMD_HEALTH_INFO_EXT_TEMPERATURE_WARNING (1)
+#define CXL_CMD_HEALTH_INFO_EXT_TEMPERATURE_CRITICAL (2)
+#define CXL_CMD_HEALTH_INFO_EXT_CORRECTED_VOLATILE_MASK BIT(4)
+#define CXL_CMD_HEALTH_INFO_EXT_CORRECTED_VOLATILE_NORMAL (0)
+#define CXL_CMD_HEALTH_INFO_EXT_CORRECTED_VOLATILE_WARNING (1)
+#define CXL_CMD_HEALTH_INFO_EXT_CORRECTED_PERSISTENT_MASK BIT(5)
+#define CXL_CMD_HEALTH_INFO_EXT_CORRECTED_PERSISTENT_NORMAL (0)
+#define CXL_CMD_HEALTH_INFO_EXT_CORRECTED_PERSISTENT_WARNING (1)
+
+#define CXL_CMD_HEALTH_INFO_LIFE_USED_NOT_IMPL 0xff
+#define CXL_CMD_HEALTH_INFO_TEMPERATURE_NOT_IMPL 0xffff
+
static inline int check_kmod(struct kmod_ctx *kmod_ctx)
{
return kmod_ctx ? 0 : -ENXIO;
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
index ed21670..065824d 100644
--- a/cxl/lib/libcxl.c
+++ b/cxl/lib/libcxl.c
@@ -677,6 +677,297 @@ CXL_EXPORT const char *cxl_cmd_get_devname(struct cxl_cmd *cmd)
return cxl_memdev_get_devname(cmd->memdev);
}
+static int cxl_cmd_validate_status(struct cxl_cmd *cmd, u32 id)
+{
+ if (cmd->send_cmd->id != id)
+ return -EINVAL;
+ if (cmd->status < 0)
+ return cmd->status;
+ return 0;
+}
+
+/* Helpers for health_info fields (no endian conversion) */
+#define cmd_get_field_u8(cmd, n, N, field) \
+do { \
+ struct cxl_cmd_##n *c = \
+ (struct cxl_cmd_##n *)cmd->send_cmd->out.payload; \
+ int rc = cxl_cmd_validate_status(cmd, CXL_MEM_COMMAND_ID_##N); \
+ if (rc) \
+ return rc; \
+ return c->field; \
+} while(0)
+
+#define cmd_get_field_u16(cmd, n, N, field) \
+do { \
+ struct cxl_cmd_##n *c = \
+ (struct cxl_cmd_##n *)cmd->send_cmd->out.payload; \
+ int rc = cxl_cmd_validate_status(cmd, CXL_MEM_COMMAND_ID_##N); \
+ if (rc) \
+ return rc; \
+ return le16_to_cpu(c->field); \
+} while(0)
+
+
+#define cmd_get_field_u32(cmd, n, N, field) \
+do { \
+ struct cxl_cmd_##n *c = \
+ (struct cxl_cmd_##n *)cmd->send_cmd->out.payload; \
+ int rc = cxl_cmd_validate_status(cmd, CXL_MEM_COMMAND_ID_##N); \
+ if (rc) \
+ return rc; \
+ return le32_to_cpu(c->field); \
+} while(0)
+
+
+#define cmd_get_field_u8_mask(cmd, n, N, field, mask) \
+do { \
+ struct cxl_cmd_##n *c = \
+ (struct cxl_cmd_##n *)cmd->send_cmd->out.payload; \
+ int rc = cxl_cmd_validate_status(cmd, CXL_MEM_COMMAND_ID_##N); \
+ if (rc) \
+ return rc; \
+ return !!(c->field & mask); \
+} while(0)
+
+CXL_EXPORT struct cxl_cmd *cxl_cmd_new_get_health_info(
+ struct cxl_memdev *memdev)
+{
+ return cxl_cmd_new_generic(memdev, CXL_MEM_COMMAND_ID_GET_HEALTH_INFO);
+}
+
+#define cmd_health_get_status_field(c, m) \
+ cmd_get_field_u8_mask(c, get_health_info, GET_HEALTH_INFO, health_status, m)
+
+CXL_EXPORT int cxl_cmd_health_info_get_maintenance_needed(struct cxl_cmd *cmd)
+{
+ cmd_health_get_status_field(cmd,
+ CXL_CMD_HEALTH_INFO_STATUS_MAINTENANCE_NEEDED_MASK);
+}
+
+CXL_EXPORT int cxl_cmd_health_info_get_performance_degraded(struct cxl_cmd *cmd)
+{
+ cmd_health_get_status_field(cmd,
+ CXL_CMD_HEALTH_INFO_STATUS_PERFORMANCE_DEGRADED_MASK);
+}
+
+CXL_EXPORT int cxl_cmd_health_info_get_hw_replacement_needed(struct cxl_cmd *cmd)
+{
+ cmd_health_get_status_field(cmd,
+ CXL_CMD_HEALTH_INFO_STATUS_HW_REPLACEMENT_NEEDED_MASK);
+}
+
+#define cmd_health_check_media_field(cmd, f) \
+do { \
+ struct cxl_cmd_get_health_info *c = \
+ (struct cxl_cmd_get_health_info *)cmd->send_cmd->out.payload; \
+ int rc = cxl_cmd_validate_status(cmd, \
+ CXL_MEM_COMMAND_ID_GET_HEALTH_INFO); \
+ if (rc) \
+ return rc; \
+ return (c->media_status == f); \
+} while(0)
+
+CXL_EXPORT int cxl_cmd_health_info_get_media_normal(struct cxl_cmd *cmd)
+{
+ cmd_health_check_media_field(cmd,
+ CXL_CMD_HEALTH_INFO_MEDIA_STATUS_NORMAL);
+}
+
+CXL_EXPORT int cxl_cmd_health_info_get_media_not_ready(struct cxl_cmd *cmd)
+{
+ cmd_health_check_media_field(cmd,
+ CXL_CMD_HEALTH_INFO_MEDIA_STATUS_NOT_READY);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_media_persistence_lost(struct cxl_cmd *cmd)
+{
+ cmd_health_check_media_field(cmd,
+ CXL_CMD_HEALTH_INFO_MEDIA_STATUS_PERSISTENCE_LOST);
+}
+
+CXL_EXPORT int cxl_cmd_health_info_get_media_data_lost(struct cxl_cmd *cmd)
+{
+ cmd_health_check_media_field(cmd,
+ CXL_CMD_HEALTH_INFO_MEDIA_STATUS_DATA_LOST);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_media_powerloss_persistence_loss(struct cxl_cmd *cmd)
+{
+ cmd_health_check_media_field(cmd,
+ CXL_CMD_HEALTH_INFO_MEDIA_STATUS_POWERLOSS_PERSISTENCE_LOSS);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_media_shutdown_persistence_loss(struct cxl_cmd *cmd)
+{
+ cmd_health_check_media_field(cmd,
+ CXL_CMD_HEALTH_INFO_MEDIA_STATUS_SHUTDOWN_PERSISTENCE_LOSS);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_media_persistence_loss_imminent(struct cxl_cmd *cmd)
+{
+ cmd_health_check_media_field(cmd,
+ CXL_CMD_HEALTH_INFO_MEDIA_STATUS_PERSISTENCE_LOSS_IMMINENT);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_media_powerloss_data_loss(struct cxl_cmd *cmd)
+{
+ cmd_health_check_media_field(cmd,
+ CXL_CMD_HEALTH_INFO_MEDIA_STATUS_POWERLOSS_DATA_LOSS);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_media_shutdown_data_loss(struct cxl_cmd *cmd)
+{
+ cmd_health_check_media_field(cmd,
+ CXL_CMD_HEALTH_INFO_MEDIA_STATUS_SHUTDOWN_DATA_LOSS);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_media_data_loss_imminent(struct cxl_cmd *cmd)
+{
+ cmd_health_check_media_field(cmd,
+ CXL_CMD_HEALTH_INFO_MEDIA_STATUS_DATA_LOSS_IMMINENT);
+}
+
+#define cmd_health_check_ext_field(cmd, fname, type) \
+do { \
+ struct cxl_cmd_get_health_info *c = \
+ (struct cxl_cmd_get_health_info *)cmd->send_cmd->out.payload; \
+ int rc = cxl_cmd_validate_status(cmd, \
+ CXL_MEM_COMMAND_ID_GET_HEALTH_INFO); \
+ if (rc) \
+ return rc; \
+ return (FIELD_GET(fname##_MASK, c->ext_status) == \
+ fname##_##type); \
+} while(0)
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_ext_life_used_normal(struct cxl_cmd *cmd)
+{
+ cmd_health_check_ext_field(cmd,
+ CXL_CMD_HEALTH_INFO_EXT_LIFE_USED, NORMAL);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_ext_life_used_warning(struct cxl_cmd *cmd)
+{
+ cmd_health_check_ext_field(cmd,
+ CXL_CMD_HEALTH_INFO_EXT_LIFE_USED, WARNING);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_ext_life_used_critical(struct cxl_cmd *cmd)
+{
+ cmd_health_check_ext_field(cmd,
+ CXL_CMD_HEALTH_INFO_EXT_LIFE_USED, CRITICAL);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_ext_temperature_normal(struct cxl_cmd *cmd)
+{
+ cmd_health_check_ext_field(cmd,
+ CXL_CMD_HEALTH_INFO_EXT_TEMPERATURE, NORMAL);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_ext_temperature_warning(struct cxl_cmd *cmd)
+{
+ cmd_health_check_ext_field(cmd,
+ CXL_CMD_HEALTH_INFO_EXT_TEMPERATURE, WARNING);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_ext_temperature_critical(struct cxl_cmd *cmd)
+{
+ cmd_health_check_ext_field(cmd,
+ CXL_CMD_HEALTH_INFO_EXT_TEMPERATURE, CRITICAL);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_ext_corrected_volatile_normal(struct cxl_cmd *cmd)
+{
+ cmd_health_check_ext_field(cmd,
+ CXL_CMD_HEALTH_INFO_EXT_CORRECTED_VOLATILE, NORMAL);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_ext_corrected_volatile_warning(struct cxl_cmd *cmd)
+{
+ cmd_health_check_ext_field(cmd,
+ CXL_CMD_HEALTH_INFO_EXT_CORRECTED_VOLATILE, WARNING);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_ext_corrected_persistent_normal(struct cxl_cmd *cmd)
+{
+ cmd_health_check_ext_field(cmd,
+ CXL_CMD_HEALTH_INFO_EXT_CORRECTED_PERSISTENT, NORMAL);
+}
+
+CXL_EXPORT int
+cxl_cmd_health_info_get_ext_corrected_persistent_warning(struct cxl_cmd *cmd)
+{
+ cmd_health_check_ext_field(cmd,
+ CXL_CMD_HEALTH_INFO_EXT_CORRECTED_PERSISTENT, WARNING);
+}
+
+static int health_info_get_life_used_raw(struct cxl_cmd *cmd)
+{
+ cmd_get_field_u8(cmd, get_health_info, GET_HEALTH_INFO,
+ life_used);
+}
+
+CXL_EXPORT int cxl_cmd_health_info_get_life_used(struct cxl_cmd *cmd)
+{
+ int rc = health_info_get_life_used_raw(cmd);
+
+ if (rc < 0)
+ return rc;
+ if (rc == CXL_CMD_HEALTH_INFO_LIFE_USED_NOT_IMPL)
+ return -EOPNOTSUPP;
+ return rc;
+}
+
+static int health_info_get_temperature_raw(struct cxl_cmd *cmd)
+{
+ cmd_get_field_u16(cmd, get_health_info, GET_HEALTH_INFO,
+ temperature);
+}
+
+CXL_EXPORT int cxl_cmd_health_info_get_temperature(struct cxl_cmd *cmd)
+{
+ int rc = health_info_get_temperature_raw(cmd);
+
+ if (rc < 0)
+ return rc;
+ if (rc == CXL_CMD_HEALTH_INFO_TEMPERATURE_NOT_IMPL)
+ return -EOPNOTSUPP;
+ return rc;
+}
+
+CXL_EXPORT int cxl_cmd_health_info_get_dirty_shutdowns(struct cxl_cmd *cmd)
+{
+ cmd_get_field_u32(cmd, get_health_info, GET_HEALTH_INFO,
+ dirty_shutdowns);
+}
+
+CXL_EXPORT int cxl_cmd_health_info_get_volatile_errors(struct cxl_cmd *cmd)
+{
+ cmd_get_field_u32(cmd, get_health_info, GET_HEALTH_INFO,
+ volatile_errors);
+}
+
+CXL_EXPORT int cxl_cmd_health_info_get_pmem_errors(struct cxl_cmd *cmd)
+{
+ cmd_get_field_u32(cmd, get_health_info, GET_HEALTH_INFO,
+ pmem_errors);
+}
+
CXL_EXPORT struct cxl_cmd *cxl_cmd_new_identify(struct cxl_memdev *memdev)
{
return cxl_cmd_new_generic(memdev, CXL_MEM_COMMAND_ID_IDENTIFY);
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
index 0f2d5e9..eae2db8 100644
--- a/cxl/libcxl.h
+++ b/cxl/libcxl.h
@@ -62,6 +62,39 @@ struct cxl_cmd *cxl_cmd_new_identify(struct cxl_memdev *memdev);
int cxl_cmd_identify_get_fw_rev(struct cxl_cmd *cmd, char *fw_rev, int fw_len);
unsigned long long cxl_cmd_identify_get_partition_align(struct cxl_cmd *cmd);
unsigned int cxl_cmd_identify_get_label_size(struct cxl_cmd *cmd);
+struct cxl_cmd *cxl_cmd_new_get_health_info(struct cxl_memdev *memdev);
+int cxl_cmd_health_info_get_maintenance_needed(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_performance_degraded(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_hw_replacement_needed(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_media_normal(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_media_not_ready(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_media_persistence_lost(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_media_data_lost(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_media_normal(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_media_not_ready(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_media_persistence_lost(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_media_data_lost(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_media_powerloss_persistence_loss(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_media_shutdown_persistence_loss(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_media_persistence_loss_imminent(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_media_powerloss_data_loss(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_media_shutdown_data_loss(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_media_data_loss_imminent(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_ext_life_used_normal(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_ext_life_used_warning(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_ext_life_used_critical(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_ext_temperature_normal(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_ext_temperature_warning(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_ext_temperature_critical(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_ext_corrected_volatile_normal(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_ext_corrected_volatile_warning(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_ext_corrected_persistent_normal(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_ext_corrected_persistent_warning(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_life_used(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_temperature(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_dirty_shutdowns(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_volatile_errors(struct cxl_cmd *cmd);
+int cxl_cmd_health_info_get_pmem_errors(struct cxl_cmd *cmd);
#ifdef __cplusplus
} /* extern "C" */
diff --git a/util/bitmap.h b/util/bitmap.h
index 490f3f0..04b3429 100644
--- a/util/bitmap.h
+++ b/util/bitmap.h
@@ -3,10 +3,33 @@
#ifndef _NDCTL_BITMAP_H_
#define _NDCTL_BITMAP_H_
+#include <linux/const.h>
#include <util/size.h>
+#include <util/util.h>
#include <ccan/short_types/short_types.h>
+#ifndef _UL
+#define _UL(x) (_AC(x, UL))
+#endif
+#ifndef _ULL
+#define _ULL(x) (_AC(x, ULL))
+#endif
+
#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define UL(x) (_UL(x))
+#define ULL(x) (_ULL(x))
+
+/* GENMASK() and its dependencies copied from include/linux/{bits.h, const.h} */
+#define __is_constexpr(x) \
+ (sizeof(int) == sizeof(*(8 ? ((void *)((long)(x) * 0l)) : (int *)8)))
+#define GENMASK_INPUT_CHECK(h, l) \
+ (BUILD_BUG_ON_ZERO(__builtin_choose_expr( \
+ __is_constexpr((l) > (h)), (l) > (h), 0)))
+#define __GENMASK(h, l) \
+ (((~UL(0)) - (UL(1) << (l)) + 1) & \
+ (~UL(0) >> (BITS_PER_LONG - 1 - (h))))
+#define GENMASK(h, l) \
+ (GENMASK_INPUT_CHECK(h, l) + __GENMASK(h, l))
#define BIT(nr) (1UL << (nr))
#define BIT_MASK(nr) (1UL << ((nr) % BITS_PER_LONG))
@@ -30,5 +53,67 @@ unsigned long find_next_zero_bit(const unsigned long *addr, unsigned long size,
unsigned long offset);
int bitmap_full(const unsigned long *src, unsigned int nbits);
+/*
+ * Bitfield access macros
+ * (Copied from Linux's include/linux/bitfield.h)
+ *
+ * FIELD_{GET,PREP} macros take as first parameter shifted mask
+ * from which they extract the base mask and shift amount.
+ * Mask must be a compilation time constant.
+ *
+ * Example:
+ *
+ * #define REG_FIELD_A GENMASK(6, 0)
+ * #define REG_FIELD_B BIT(7)
+ * #define REG_FIELD_C GENMASK(15, 8)
+ * #define REG_FIELD_D GENMASK(31, 16)
+ *
+ * Get:
+ * a = FIELD_GET(REG_FIELD_A, reg);
+ * b = FIELD_GET(REG_FIELD_B, reg);
+ *
+ * Set:
+ * reg = FIELD_PREP(REG_FIELD_A, 1) |
+ * FIELD_PREP(REG_FIELD_B, 0) |
+ * FIELD_PREP(REG_FIELD_C, c) |
+ * FIELD_PREP(REG_FIELD_D, 0x40);
+ *
+ * Modify:
+ * reg &= ~REG_FIELD_C;
+ * reg |= FIELD_PREP(REG_FIELD_C, c);
+ */
+
+/* Force a compilation error if a constant expression is not a power of 2 */
+#define __BUILD_BUG_ON_NOT_POWER_OF_2(n) \
+ BUILD_BUG_ON(((n) & ((n) - 1)) != 0)
+#define BUILD_BUG_ON_NOT_POWER_OF_2(n) \
+ BUILD_BUG_ON((n) == 0 || (((n) & ((n) - 1)) != 0))
+
+#define __bf_shf(x) (__builtin_ffsll(x) - 1)
+
+#define __BF_FIELD_CHECK(_mask, _reg, _val) \
+ ({ \
+ BUILD_BUG_ON(!__builtin_constant_p(_mask)); \
+ BUILD_BUG_ON((_mask) == 0); \
+ BUILD_BUG_ON(__builtin_constant_p(_val) ? \
+ ~((_mask) >> __bf_shf(_mask)) & (_val) : 0); \
+ BUILD_BUG_ON((_mask) > (typeof(_reg))~0ull); \
+ __BUILD_BUG_ON_NOT_POWER_OF_2((_mask) + \
+ (1ULL << __bf_shf(_mask))); \
+ })
+
+/**
+ * FIELD_GET() - extract a bitfield element
+ * @_mask: shifted mask defining the field's length and position
+ * @_reg: value of entire bitfield
+ *
+ * FIELD_GET() extracts the field specified by @_mask from the
+ * bitfield passed in as @_reg by masking and shifting it down.
+ */
+#define FIELD_GET(_mask, _reg) \
+ ({ \
+ __BF_FIELD_CHECK(_mask, _reg, 0U); \
+ (typeof(_mask))(((_reg) & (_mask)) >> __bf_shf(_mask)); \
+ })
#endif /* _NDCTL_BITMAP_H_ */
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
index 1dc45f4..c83bc28 100644
--- a/cxl/lib/libcxl.sym
+++ b/cxl/lib/libcxl.sym
@@ -33,6 +33,35 @@ global:
cxl_cmd_identify_get_fw_rev;
cxl_cmd_identify_get_partition_align;
cxl_cmd_identify_get_label_size;
+ cxl_cmd_new_get_health_info;
+ cxl_cmd_health_info_get_maintenance_needed;
+ cxl_cmd_health_info_get_performance_degraded;
+ cxl_cmd_health_info_get_hw_replacement_needed;
+ cxl_cmd_health_info_get_media_normal;
+ cxl_cmd_health_info_get_media_not_ready;
+ cxl_cmd_health_info_get_media_persistence_lost;
+ cxl_cmd_health_info_get_media_data_lost;
+ cxl_cmd_health_info_get_media_powerloss_persistence_loss;
+ cxl_cmd_health_info_get_media_shutdown_persistence_loss;
+ cxl_cmd_health_info_get_media_persistence_loss_imminent;
+ cxl_cmd_health_info_get_media_powerloss_data_loss;
+ cxl_cmd_health_info_get_media_shutdown_data_loss;
+ cxl_cmd_health_info_get_media_data_loss_imminent;
+ cxl_cmd_health_info_get_ext_life_used_normal;
+ cxl_cmd_health_info_get_ext_life_used_warning;
+ cxl_cmd_health_info_get_ext_life_used_critical;
+ cxl_cmd_health_info_get_ext_temperature_normal;
+ cxl_cmd_health_info_get_ext_temperature_warning;
+ cxl_cmd_health_info_get_ext_temperature_critical;
+ cxl_cmd_health_info_get_ext_corrected_volatile_normal;
+ cxl_cmd_health_info_get_ext_corrected_volatile_warning;
+ cxl_cmd_health_info_get_ext_corrected_persistent_normal;
+ cxl_cmd_health_info_get_ext_corrected_persistent_warning;
+ cxl_cmd_health_info_get_life_used;
+ cxl_cmd_health_info_get_temperature;
+ cxl_cmd_health_info_get_dirty_shutdowns;
+ cxl_cmd_health_info_get_volatile_errors;
+ cxl_cmd_health_info_get_pmem_errors;
local:
*;
};
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* Re: [ndctl PATCH v5 07/16] libcxl: add GET_HEALTH_INFO mailbox command and accessors
2021-11-11 20:44 ` [ndctl PATCH v5 07/16] libcxl: add GET_HEALTH_INFO mailbox command and accessors Vishal Verma
@ 2021-11-11 23:17 ` Dan Williams
0 siblings, 0 replies; 31+ messages in thread
From: Dan Williams @ 2021-11-11 23:17 UTC (permalink / raw)
To: Vishal Verma; +Cc: linux-cxl, Ben Widawsky, Linux NVDIMM
On Thu, Nov 11, 2021 at 12:45 PM Vishal Verma <vishal.l.verma@intel.com> wrote:
>
> Add libcxl APIs to create a new GET_HEALTH_INFO mailbox command, the
> command output data structure (privately), and accessor APIs to return
> the different fields in the health info output.
>
> Cc: Ben Widawsky <ben.widawsky@intel.com>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
Looks good to me:
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
^ permalink raw reply [flat|nested] 31+ messages in thread
* [ndctl PATCH v5 08/16] libcxl: add support for the 'GET_LSA' command
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
` (6 preceding siblings ...)
2021-11-11 20:44 ` [ndctl PATCH v5 07/16] libcxl: add GET_HEALTH_INFO mailbox command and accessors Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 09/16] libcxl: add label_size to cxl_memdev, and an API to retrieve it Vishal Verma
` (7 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Add a command allocator and accessor APIs for the 'GET_LSA' mailbox
command.
Cc: Ben Widawsky <ben.widawsky@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
cxl/lib/private.h | 5 +++++
cxl/lib/libcxl.c | 36 ++++++++++++++++++++++++++++++++++++
cxl/libcxl.h | 4 ++++
cxl/lib/libcxl.sym | 2 ++
4 files changed, 47 insertions(+)
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
index 885553a..bf3a897 100644
--- a/cxl/lib/private.h
+++ b/cxl/lib/private.h
@@ -73,6 +73,11 @@ struct cxl_cmd_identify {
u8 qos_telemetry_caps;
} __attribute__((packed));
+struct cxl_cmd_get_lsa_in {
+ le32 offset;
+ le32 length;
+} __attribute__((packed));
+
struct cxl_cmd_get_health_info {
u8 health_status;
u8 media_status;
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
index 065824d..76913a2 100644
--- a/cxl/lib/libcxl.c
+++ b/cxl/lib/libcxl.c
@@ -1036,6 +1036,42 @@ CXL_EXPORT struct cxl_cmd *cxl_cmd_new_raw(struct cxl_memdev *memdev,
return cmd;
}
+CXL_EXPORT struct cxl_cmd *cxl_cmd_new_read_label(struct cxl_memdev *memdev,
+ unsigned int offset, unsigned int length)
+{
+ struct cxl_cmd_get_lsa_in *get_lsa;
+ struct cxl_cmd *cmd;
+
+ cmd = cxl_cmd_new_generic(memdev, CXL_MEM_COMMAND_ID_GET_LSA);
+ if (!cmd)
+ return NULL;
+
+ get_lsa = (struct cxl_cmd_get_lsa_in *)cmd->send_cmd->in.payload;
+ get_lsa->offset = cpu_to_le32(offset);
+ get_lsa->length = cpu_to_le32(length);
+ return cmd;
+}
+
+CXL_EXPORT ssize_t cxl_cmd_read_label_get_payload(struct cxl_cmd *cmd,
+ void *buf, unsigned int length)
+{
+ struct cxl_cmd_get_lsa_in *get_lsa;
+ void *payload;
+ int rc;
+
+ rc = cxl_cmd_validate_status(cmd, CXL_MEM_COMMAND_ID_GET_LSA);
+ if (rc)
+ return rc;
+
+ get_lsa = (struct cxl_cmd_get_lsa_in *)cmd->send_cmd->in.payload;
+ if (length > le32_to_cpu(get_lsa->length))
+ return -EINVAL;
+
+ payload = (void *)cmd->send_cmd->out.payload;
+ memcpy(buf, payload, length);
+ return length;
+}
+
CXL_EXPORT int cxl_cmd_submit(struct cxl_cmd *cmd)
{
struct cxl_memdev *memdev = cmd->memdev;
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
index eae2db8..7408745 100644
--- a/cxl/libcxl.h
+++ b/cxl/libcxl.h
@@ -95,6 +95,10 @@ int cxl_cmd_health_info_get_temperature(struct cxl_cmd *cmd);
int cxl_cmd_health_info_get_dirty_shutdowns(struct cxl_cmd *cmd);
int cxl_cmd_health_info_get_volatile_errors(struct cxl_cmd *cmd);
int cxl_cmd_health_info_get_pmem_errors(struct cxl_cmd *cmd);
+struct cxl_cmd *cxl_cmd_new_read_label(struct cxl_memdev *memdev,
+ unsigned int offset, unsigned int length);
+ssize_t cxl_cmd_read_label_get_payload(struct cxl_cmd *cmd, void *buf,
+ unsigned int length);
#ifdef __cplusplus
} /* extern "C" */
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
index c83bc28..629322c 100644
--- a/cxl/lib/libcxl.sym
+++ b/cxl/lib/libcxl.sym
@@ -62,6 +62,8 @@ global:
cxl_cmd_health_info_get_dirty_shutdowns;
cxl_cmd_health_info_get_volatile_errors;
cxl_cmd_health_info_get_pmem_errors;
+ cxl_cmd_new_read_label;
+ cxl_cmd_read_label_get_payload;
local:
*;
};
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* [ndctl PATCH v5 09/16] libcxl: add label_size to cxl_memdev, and an API to retrieve it
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
` (7 preceding siblings ...)
2021-11-11 20:44 ` [ndctl PATCH v5 08/16] libcxl: add support for the 'GET_LSA' command Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 10/16] libcxl: add representation for an nvdimm bridge object Vishal Verma
` (6 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Size of the Label Storage Area (LSA) is available as a sysfs attribute
called 'label_storage_size'. Add that to libcxl's memdev so that it is available
for label related commands.
Cc: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
cxl/lib/private.h | 1 +
cxl/lib/libcxl.c | 12 ++++++++++++
cxl/libcxl.h | 1 +
cxl/lib/libcxl.sym | 1 +
4 files changed, 15 insertions(+)
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
index bf3a897..c4ed741 100644
--- a/cxl/lib/private.h
+++ b/cxl/lib/private.h
@@ -21,6 +21,7 @@ struct cxl_memdev {
unsigned long long pmem_size;
unsigned long long ram_size;
int payload_max;
+ size_t lsa_size;
struct kmod_module *module;
};
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
index 76913a2..def3a97 100644
--- a/cxl/lib/libcxl.c
+++ b/cxl/lib/libcxl.c
@@ -247,6 +247,13 @@ static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
if (memdev->payload_max < 0)
goto err_read;
+ sprintf(path, "%s/label_storage_size", cxlmem_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto err_read;
+ memdev->lsa_size = strtoull(buf, NULL, 0);
+ if (memdev->lsa_size == ULLONG_MAX)
+ goto err_read;
+
memdev->dev_path = strdup(cxlmem_base);
if (!memdev->dev_path)
goto err_read;
@@ -350,6 +357,11 @@ CXL_EXPORT const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev
return memdev->firmware_version;
}
+CXL_EXPORT size_t cxl_memdev_get_label_size(struct cxl_memdev *memdev)
+{
+ return memdev->lsa_size;
+}
+
CXL_EXPORT void cxl_cmd_unref(struct cxl_cmd *cmd)
{
if (!cmd)
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
index 7408745..d3b97a1 100644
--- a/cxl/libcxl.h
+++ b/cxl/libcxl.h
@@ -42,6 +42,7 @@ struct cxl_ctx *cxl_memdev_get_ctx(struct cxl_memdev *memdev);
unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev);
unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev);
const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev);
+size_t cxl_memdev_get_label_size(struct cxl_memdev *memdev);
#define cxl_memdev_foreach(ctx, memdev) \
for (memdev = cxl_memdev_get_first(ctx); \
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
index 629322c..858e953 100644
--- a/cxl/lib/libcxl.sym
+++ b/cxl/lib/libcxl.sym
@@ -64,6 +64,7 @@ global:
cxl_cmd_health_info_get_pmem_errors;
cxl_cmd_new_read_label;
cxl_cmd_read_label_get_payload;
+ cxl_memdev_get_label_size;
local:
*;
};
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* [ndctl PATCH v5 10/16] libcxl: add representation for an nvdimm bridge object
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
` (8 preceding siblings ...)
2021-11-11 20:44 ` [ndctl PATCH v5 09/16] libcxl: add label_size to cxl_memdev, and an API to retrieve it Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 23:49 ` Dan Williams
2021-11-12 21:52 ` [ndctl PATCH v6 " Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 11/16] libcxl: add interfaces for label operations Vishal Verma
` (5 subsequent siblings)
15 siblings, 2 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Add an nvdimm bridge object representation internal to libcxl. A bridge
object is ited to its parent memdev object, and this patch adds its
first interface, which checks whether a bridge is 'active' - i.e.
implying the label space on the memdev is owned by the kernel.
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
cxl/lib/private.h | 9 ++++++
cxl/lib/libcxl.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++
cxl/libcxl.h | 1 +
cxl/lib/libcxl.sym | 1 +
4 files changed, 84 insertions(+)
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
index c4ed741..2f0b6ea 100644
--- a/cxl/lib/private.h
+++ b/cxl/lib/private.h
@@ -10,6 +10,14 @@
#define CXL_EXPORT __attribute__ ((visibility("default")))
+
+struct cxl_nvdimm_br {
+ int id;
+ void *dev_buf;
+ size_t buf_len;
+ char *dev_path;
+};
+
struct cxl_memdev {
int id, major, minor;
void *dev_buf;
@@ -23,6 +31,7 @@ struct cxl_memdev {
int payload_max;
size_t lsa_size;
struct kmod_module *module;
+ struct cxl_nvdimm_br *bridge;
};
enum cxl_cmd_query_status {
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
index def3a97..7bc0696 100644
--- a/cxl/lib/libcxl.c
+++ b/cxl/lib/libcxl.c
@@ -45,11 +45,19 @@ struct cxl_ctx {
void *private_data;
};
+static void free_bridge(struct cxl_nvdimm_br *bridge)
+{
+ free(bridge->dev_buf);
+ free(bridge->dev_path);
+ free(bridge);
+}
+
static void free_memdev(struct cxl_memdev *memdev, struct list_head *head)
{
if (head)
list_del_from(head, &memdev->list);
kmod_module_unref(memdev->module);
+ free_bridge(memdev->bridge);
free(memdev->firmware_version);
free(memdev->dev_buf);
free(memdev->dev_path);
@@ -205,6 +213,40 @@ CXL_EXPORT void cxl_set_log_priority(struct cxl_ctx *ctx, int priority)
ctx->ctx.log_priority = priority;
}
+static void *add_cxl_bridge(void *parent, int id, const char *br_base)
+{
+ const char *devname = devpath_to_devname(br_base);
+ struct cxl_memdev *memdev = parent;
+ struct cxl_ctx *ctx = memdev->ctx;
+ struct cxl_nvdimm_br *bridge;
+
+ dbg(ctx, "%s: bridge_base: \'%s\'\n", devname, br_base);
+
+ bridge = calloc(1, sizeof(*bridge));
+ if (!bridge)
+ goto err_dev;
+ bridge->id = id;
+
+ bridge->dev_path = strdup(br_base);
+ if (!bridge->dev_path)
+ goto err_read;
+
+ bridge->dev_buf = calloc(1, strlen(br_base) + 50);
+ if (!bridge->dev_buf)
+ goto err_read;
+ bridge->buf_len = strlen(br_base) + 50;
+
+ memdev->bridge = bridge;
+ return bridge;
+
+ err_read:
+ free(bridge->dev_buf);
+ free(bridge->dev_path);
+ free(bridge);
+ err_dev:
+ return NULL;
+}
+
static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
{
const char *devname = devpath_to_devname(cxlmem_base);
@@ -271,6 +313,8 @@ static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
goto err_read;
memdev->buf_len = strlen(cxlmem_base) + 50;
+ sysfs_device_parse(ctx, cxlmem_base, "pmem", memdev, add_cxl_bridge);
+
cxl_memdev_foreach(ctx, memdev_dup)
if (memdev_dup->id == memdev->id) {
free_memdev(memdev, NULL);
@@ -362,6 +406,35 @@ CXL_EXPORT size_t cxl_memdev_get_label_size(struct cxl_memdev *memdev)
return memdev->lsa_size;
}
+static int is_enabled(const char *drvpath)
+{
+ struct stat st;
+
+ if (lstat(drvpath, &st) < 0 || !S_ISLNK(st.st_mode))
+ return 0;
+ else
+ return 1;
+}
+
+CXL_EXPORT int cxl_memdev_nvdimm_bridge_active(struct cxl_memdev *memdev)
+{
+ struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+ struct cxl_nvdimm_br *bridge = memdev->bridge;
+ char *path = memdev->dev_buf;
+ int len = memdev->buf_len;
+
+ if (!bridge)
+ return 0;
+
+ if (snprintf(path, len, "%s/driver", bridge->dev_path) >= len) {
+ err(ctx, "%s: nvdimm bridge buffer too small!\n",
+ cxl_memdev_get_devname(memdev));
+ return 0;
+ }
+
+ return is_enabled(path);
+}
+
CXL_EXPORT void cxl_cmd_unref(struct cxl_cmd *cmd)
{
if (!cmd)
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
index d3b97a1..535e349 100644
--- a/cxl/libcxl.h
+++ b/cxl/libcxl.h
@@ -43,6 +43,7 @@ unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev);
unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev);
const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev);
size_t cxl_memdev_get_label_size(struct cxl_memdev *memdev);
+int cxl_memdev_nvdimm_bridge_active(struct cxl_memdev *memdev);
#define cxl_memdev_foreach(ctx, memdev) \
for (memdev = cxl_memdev_get_first(ctx); \
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
index 858e953..f3b0c63 100644
--- a/cxl/lib/libcxl.sym
+++ b/cxl/lib/libcxl.sym
@@ -65,6 +65,7 @@ global:
cxl_cmd_new_read_label;
cxl_cmd_read_label_get_payload;
cxl_memdev_get_label_size;
+ cxl_memdev_nvdimm_bridge_active;
local:
*;
};
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* Re: [ndctl PATCH v5 10/16] libcxl: add representation for an nvdimm bridge object
2021-11-11 20:44 ` [ndctl PATCH v5 10/16] libcxl: add representation for an nvdimm bridge object Vishal Verma
@ 2021-11-11 23:49 ` Dan Williams
2021-11-12 21:53 ` Verma, Vishal L
2021-11-12 21:52 ` [ndctl PATCH v6 " Vishal Verma
1 sibling, 1 reply; 31+ messages in thread
From: Dan Williams @ 2021-11-11 23:49 UTC (permalink / raw)
To: Vishal Verma; +Cc: linux-cxl, Ben Widawsky, Linux NVDIMM
On Thu, Nov 11, 2021 at 12:45 PM Vishal Verma <vishal.l.verma@intel.com> wrote:
>
> Add an nvdimm bridge object representation internal to libcxl. A bridge
> object is ited to its parent memdev object, and this patch adds its
s/ited/tied/
> first interface, which checks whether a bridge is 'active' - i.e.
> implying the label space on the memdev is owned by the kernel.
Just some minor fixups below and you can add:
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
> ---
> cxl/lib/private.h | 9 ++++++
> cxl/lib/libcxl.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++
> cxl/libcxl.h | 1 +
> cxl/lib/libcxl.sym | 1 +
> 4 files changed, 84 insertions(+)
>
> diff --git a/cxl/lib/private.h b/cxl/lib/private.h
> index c4ed741..2f0b6ea 100644
> --- a/cxl/lib/private.h
> +++ b/cxl/lib/private.h
> @@ -10,6 +10,14 @@
>
> #define CXL_EXPORT __attribute__ ((visibility("default")))
>
> +
> +struct cxl_nvdimm_br {
Might as well spell out "bridge".
> + int id;
> + void *dev_buf;
> + size_t buf_len;
> + char *dev_path;
> +};
> +
> struct cxl_memdev {
> int id, major, minor;
> void *dev_buf;
> @@ -23,6 +31,7 @@ struct cxl_memdev {
> int payload_max;
> size_t lsa_size;
> struct kmod_module *module;
> + struct cxl_nvdimm_br *bridge;
> };
>
> enum cxl_cmd_query_status {
> diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
> index def3a97..7bc0696 100644
> --- a/cxl/lib/libcxl.c
> +++ b/cxl/lib/libcxl.c
> @@ -45,11 +45,19 @@ struct cxl_ctx {
> void *private_data;
> };
>
> +static void free_bridge(struct cxl_nvdimm_br *bridge)
> +{
> + free(bridge->dev_buf);
> + free(bridge->dev_path);
> + free(bridge);
> +}
> +
> static void free_memdev(struct cxl_memdev *memdev, struct list_head *head)
> {
> if (head)
> list_del_from(head, &memdev->list);
> kmod_module_unref(memdev->module);
> + free_bridge(memdev->bridge);
> free(memdev->firmware_version);
> free(memdev->dev_buf);
> free(memdev->dev_path);
> @@ -205,6 +213,40 @@ CXL_EXPORT void cxl_set_log_priority(struct cxl_ctx *ctx, int priority)
> ctx->ctx.log_priority = priority;
> }
>
> +static void *add_cxl_bridge(void *parent, int id, const char *br_base)
> +{
> + const char *devname = devpath_to_devname(br_base);
> + struct cxl_memdev *memdev = parent;
> + struct cxl_ctx *ctx = memdev->ctx;
> + struct cxl_nvdimm_br *bridge;
> +
> + dbg(ctx, "%s: bridge_base: \'%s\'\n", devname, br_base);
> +
> + bridge = calloc(1, sizeof(*bridge));
> + if (!bridge)
> + goto err_dev;
> + bridge->id = id;
> +
> + bridge->dev_path = strdup(br_base);
> + if (!bridge->dev_path)
> + goto err_read;
> +
> + bridge->dev_buf = calloc(1, strlen(br_base) + 50);
> + if (!bridge->dev_buf)
> + goto err_read;
> + bridge->buf_len = strlen(br_base) + 50;
> +
> + memdev->bridge = bridge;
> + return bridge;
> +
> + err_read:
> + free(bridge->dev_buf);
> + free(bridge->dev_path);
> + free(bridge);
> + err_dev:
> + return NULL;
> +}
> +
> static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
> {
> const char *devname = devpath_to_devname(cxlmem_base);
> @@ -271,6 +313,8 @@ static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
> goto err_read;
> memdev->buf_len = strlen(cxlmem_base) + 50;
>
> + sysfs_device_parse(ctx, cxlmem_base, "pmem", memdev, add_cxl_bridge);
> +
> cxl_memdev_foreach(ctx, memdev_dup)
> if (memdev_dup->id == memdev->id) {
> free_memdev(memdev, NULL);
> @@ -362,6 +406,35 @@ CXL_EXPORT size_t cxl_memdev_get_label_size(struct cxl_memdev *memdev)
> return memdev->lsa_size;
> }
>
> +static int is_enabled(const char *drvpath)
> +{
> + struct stat st;
> +
> + if (lstat(drvpath, &st) < 0 || !S_ISLNK(st.st_mode))
> + return 0;
> + else
> + return 1;
> +}
> +
> +CXL_EXPORT int cxl_memdev_nvdimm_bridge_active(struct cxl_memdev *memdev)
> +{
> + struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
> + struct cxl_nvdimm_br *bridge = memdev->bridge;
> + char *path = memdev->dev_buf;
Should this be: bridge->dev_buf?
> + int len = memdev->buf_len;
...and this should bridge->buf_len?
Not strictly a bug, but might as well use the 'bridge' version of
these attributes, right?
> +
> + if (!bridge)
> + return 0;
> +
> + if (snprintf(path, len, "%s/driver", bridge->dev_path) >= len) {
> + err(ctx, "%s: nvdimm bridge buffer too small!\n",
> + cxl_memdev_get_devname(memdev));
> + return 0;
> + }
> +
> + return is_enabled(path);
> +}
> +
> CXL_EXPORT void cxl_cmd_unref(struct cxl_cmd *cmd)
> {
> if (!cmd)
> diff --git a/cxl/libcxl.h b/cxl/libcxl.h
> index d3b97a1..535e349 100644
> --- a/cxl/libcxl.h
> +++ b/cxl/libcxl.h
> @@ -43,6 +43,7 @@ unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev);
> unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev);
> const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev);
> size_t cxl_memdev_get_label_size(struct cxl_memdev *memdev);
> +int cxl_memdev_nvdimm_bridge_active(struct cxl_memdev *memdev);
>
> #define cxl_memdev_foreach(ctx, memdev) \
> for (memdev = cxl_memdev_get_first(ctx); \
> diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
> index 858e953..f3b0c63 100644
> --- a/cxl/lib/libcxl.sym
> +++ b/cxl/lib/libcxl.sym
> @@ -65,6 +65,7 @@ global:
> cxl_cmd_new_read_label;
> cxl_cmd_read_label_get_payload;
> cxl_memdev_get_label_size;
> + cxl_memdev_nvdimm_bridge_active;
> local:
> *;
> };
> --
> 2.31.1
>
^ permalink raw reply [flat|nested] 31+ messages in thread* Re: [ndctl PATCH v5 10/16] libcxl: add representation for an nvdimm bridge object
2021-11-11 23:49 ` Dan Williams
@ 2021-11-12 21:53 ` Verma, Vishal L
0 siblings, 0 replies; 31+ messages in thread
From: Verma, Vishal L @ 2021-11-12 21:53 UTC (permalink / raw)
To: Williams, Dan J; +Cc: Widawsky, Ben, linux-cxl, nvdimm
On Thu, 2021-11-11 at 15:49 -0800, Dan Williams wrote:
> On Thu, Nov 11, 2021 at 12:45 PM Vishal Verma <vishal.l.verma@intel.com> wrote:
> >
> > Add an nvdimm bridge object representation internal to libcxl. A bridge
> > object is ited to its parent memdev object, and this patch adds its
>
> s/ited/tied/
>
> > first interface, which checks whether a bridge is 'active' - i.e.
> > implying the label space on the memdev is owned by the kernel.
>
> Just some minor fixups below and you can add:
Thanks Dan - fixed all these up in v6
>
> Reviewed-by: Dan Williams <dan.j.williams@intel.com>
>
> >
> > Cc: Dan Williams <dan.j.williams@intel.com>
> > Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
> > ---
> > cxl/lib/private.h | 9 ++++++
> > cxl/lib/libcxl.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++
> > cxl/libcxl.h | 1 +
> > cxl/lib/libcxl.sym | 1 +
> > 4 files changed, 84 insertions(+)
> >
> > diff --git a/cxl/lib/private.h b/cxl/lib/private.h
> > index c4ed741..2f0b6ea 100644
> > --- a/cxl/lib/private.h
> > +++ b/cxl/lib/private.h
> > @@ -10,6 +10,14 @@
> >
> > #define CXL_EXPORT __attribute__ ((visibility("default")))
> >
> > +
> > +struct cxl_nvdimm_br {
>
> Might as well spell out "bridge".
>
> > + int id;
> > + void *dev_buf;
> > + size_t buf_len;
> > + char *dev_path;
> > +};
> > +
> > struct cxl_memdev {
> > int id, major, minor;
> > void *dev_buf;
> > @@ -23,6 +31,7 @@ struct cxl_memdev {
> > int payload_max;
> > size_t lsa_size;
> > struct kmod_module *module;
> > + struct cxl_nvdimm_br *bridge;
> > };
> >
> > enum cxl_cmd_query_status {
> > diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
> > index def3a97..7bc0696 100644
> > --- a/cxl/lib/libcxl.c
> > +++ b/cxl/lib/libcxl.c
> > @@ -45,11 +45,19 @@ struct cxl_ctx {
> > void *private_data;
> > };
> >
> > +static void free_bridge(struct cxl_nvdimm_br *bridge)
> > +{
> > + free(bridge->dev_buf);
> > + free(bridge->dev_path);
> > + free(bridge);
> > +}
> > +
> > static void free_memdev(struct cxl_memdev *memdev, struct list_head *head)
> > {
> > if (head)
> > list_del_from(head, &memdev->list);
> > kmod_module_unref(memdev->module);
> > + free_bridge(memdev->bridge);
> > free(memdev->firmware_version);
> > free(memdev->dev_buf);
> > free(memdev->dev_path);
> > @@ -205,6 +213,40 @@ CXL_EXPORT void cxl_set_log_priority(struct cxl_ctx *ctx, int priority)
> > ctx->ctx.log_priority = priority;
> > }
> >
> > +static void *add_cxl_bridge(void *parent, int id, const char *br_base)
> > +{
> > + const char *devname = devpath_to_devname(br_base);
> > + struct cxl_memdev *memdev = parent;
> > + struct cxl_ctx *ctx = memdev->ctx;
> > + struct cxl_nvdimm_br *bridge;
> > +
> > + dbg(ctx, "%s: bridge_base: \'%s\'\n", devname, br_base);
> > +
> > + bridge = calloc(1, sizeof(*bridge));
> > + if (!bridge)
> > + goto err_dev;
> > + bridge->id = id;
> > +
> > + bridge->dev_path = strdup(br_base);
> > + if (!bridge->dev_path)
> > + goto err_read;
> > +
> > + bridge->dev_buf = calloc(1, strlen(br_base) + 50);
> > + if (!bridge->dev_buf)
> > + goto err_read;
> > + bridge->buf_len = strlen(br_base) + 50;
> > +
> > + memdev->bridge = bridge;
> > + return bridge;
> > +
> > + err_read:
> > + free(bridge->dev_buf);
> > + free(bridge->dev_path);
> > + free(bridge);
> > + err_dev:
> > + return NULL;
> > +}
> > +
> > static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
> > {
> > const char *devname = devpath_to_devname(cxlmem_base);
> > @@ -271,6 +313,8 @@ static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
> > goto err_read;
> > memdev->buf_len = strlen(cxlmem_base) + 50;
> >
> > + sysfs_device_parse(ctx, cxlmem_base, "pmem", memdev, add_cxl_bridge);
> > +
> > cxl_memdev_foreach(ctx, memdev_dup)
> > if (memdev_dup->id == memdev->id) {
> > free_memdev(memdev, NULL);
> > @@ -362,6 +406,35 @@ CXL_EXPORT size_t cxl_memdev_get_label_size(struct cxl_memdev *memdev)
> > return memdev->lsa_size;
> > }
> >
> > +static int is_enabled(const char *drvpath)
> > +{
> > + struct stat st;
> > +
> > + if (lstat(drvpath, &st) < 0 || !S_ISLNK(st.st_mode))
> > + return 0;
> > + else
> > + return 1;
> > +}
> > +
> > +CXL_EXPORT int cxl_memdev_nvdimm_bridge_active(struct cxl_memdev *memdev)
> > +{
> > + struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
> > + struct cxl_nvdimm_br *bridge = memdev->bridge;
> > + char *path = memdev->dev_buf;
>
> Should this be: bridge->dev_buf?
>
> > + int len = memdev->buf_len;
>
> ...and this should bridge->buf_len?
>
> Not strictly a bug, but might as well use the 'bridge' version of
> these attributes, right?
>
> > +
> > + if (!bridge)
> > + return 0;
> > +
> > + if (snprintf(path, len, "%s/driver", bridge->dev_path) >= len) {
> > + err(ctx, "%s: nvdimm bridge buffer too small!\n",
> > + cxl_memdev_get_devname(memdev));
> > + return 0;
> > + }
> > +
> > + return is_enabled(path);
> > +}
> > +
> > CXL_EXPORT void cxl_cmd_unref(struct cxl_cmd *cmd)
> > {
> > if (!cmd)
> > diff --git a/cxl/libcxl.h b/cxl/libcxl.h
> > index d3b97a1..535e349 100644
> > --- a/cxl/libcxl.h
> > +++ b/cxl/libcxl.h
> > @@ -43,6 +43,7 @@ unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev);
> > unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev);
> > const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev);
> > size_t cxl_memdev_get_label_size(struct cxl_memdev *memdev);
> > +int cxl_memdev_nvdimm_bridge_active(struct cxl_memdev *memdev);
> >
> > #define cxl_memdev_foreach(ctx, memdev) \
> > for (memdev = cxl_memdev_get_first(ctx); \
> > diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
> > index 858e953..f3b0c63 100644
> > --- a/cxl/lib/libcxl.sym
> > +++ b/cxl/lib/libcxl.sym
> > @@ -65,6 +65,7 @@ global:
> > cxl_cmd_new_read_label;
> > cxl_cmd_read_label_get_payload;
> > cxl_memdev_get_label_size;
> > + cxl_memdev_nvdimm_bridge_active;
> > local:
> > *;
> > };
> > --
> > 2.31.1
> >
^ permalink raw reply [flat|nested] 31+ messages in thread
* [ndctl PATCH v6 10/16] libcxl: add representation for an nvdimm bridge object
2021-11-11 20:44 ` [ndctl PATCH v5 10/16] libcxl: add representation for an nvdimm bridge object Vishal Verma
2021-11-11 23:49 ` Dan Williams
@ 2021-11-12 21:52 ` Vishal Verma
1 sibling, 0 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-12 21:52 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Add an nvdimm bridge object representation internal to libcxl. A bridge
object is tied to its parent memdev object, and this patch adds its
first interface, which checks whether a bridge is 'active' - i.e.
implying the label space on the memdev is owned by the kernel.
Cc: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
cxl/lib/private.h | 8 +++++
cxl/lib/libcxl.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++
cxl/libcxl.h | 1 +
cxl/lib/libcxl.sym | 1 +
4 files changed, 83 insertions(+)
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
index c4ed741..525c41e 100644
--- a/cxl/lib/private.h
+++ b/cxl/lib/private.h
@@ -10,6 +10,13 @@
#define CXL_EXPORT __attribute__ ((visibility("default")))
+struct cxl_nvdimm_bridge {
+ int id;
+ void *dev_buf;
+ size_t buf_len;
+ char *dev_path;
+};
+
struct cxl_memdev {
int id, major, minor;
void *dev_buf;
@@ -23,6 +30,7 @@ struct cxl_memdev {
int payload_max;
size_t lsa_size;
struct kmod_module *module;
+ struct cxl_nvdimm_bridge *bridge;
};
enum cxl_cmd_query_status {
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
index def3a97..60ed646 100644
--- a/cxl/lib/libcxl.c
+++ b/cxl/lib/libcxl.c
@@ -45,11 +45,19 @@ struct cxl_ctx {
void *private_data;
};
+static void free_bridge(struct cxl_nvdimm_bridge *bridge)
+{
+ free(bridge->dev_buf);
+ free(bridge->dev_path);
+ free(bridge);
+}
+
static void free_memdev(struct cxl_memdev *memdev, struct list_head *head)
{
if (head)
list_del_from(head, &memdev->list);
kmod_module_unref(memdev->module);
+ free_bridge(memdev->bridge);
free(memdev->firmware_version);
free(memdev->dev_buf);
free(memdev->dev_path);
@@ -205,6 +213,40 @@ CXL_EXPORT void cxl_set_log_priority(struct cxl_ctx *ctx, int priority)
ctx->ctx.log_priority = priority;
}
+static void *add_cxl_bridge(void *parent, int id, const char *br_base)
+{
+ const char *devname = devpath_to_devname(br_base);
+ struct cxl_memdev *memdev = parent;
+ struct cxl_ctx *ctx = memdev->ctx;
+ struct cxl_nvdimm_bridge *bridge;
+
+ dbg(ctx, "%s: bridge_base: \'%s\'\n", devname, br_base);
+
+ bridge = calloc(1, sizeof(*bridge));
+ if (!bridge)
+ goto err_dev;
+ bridge->id = id;
+
+ bridge->dev_path = strdup(br_base);
+ if (!bridge->dev_path)
+ goto err_read;
+
+ bridge->dev_buf = calloc(1, strlen(br_base) + 50);
+ if (!bridge->dev_buf)
+ goto err_read;
+ bridge->buf_len = strlen(br_base) + 50;
+
+ memdev->bridge = bridge;
+ return bridge;
+
+ err_read:
+ free(bridge->dev_buf);
+ free(bridge->dev_path);
+ free(bridge);
+ err_dev:
+ return NULL;
+}
+
static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
{
const char *devname = devpath_to_devname(cxlmem_base);
@@ -271,6 +313,8 @@ static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
goto err_read;
memdev->buf_len = strlen(cxlmem_base) + 50;
+ sysfs_device_parse(ctx, cxlmem_base, "pmem", memdev, add_cxl_bridge);
+
cxl_memdev_foreach(ctx, memdev_dup)
if (memdev_dup->id == memdev->id) {
free_memdev(memdev, NULL);
@@ -362,6 +406,35 @@ CXL_EXPORT size_t cxl_memdev_get_label_size(struct cxl_memdev *memdev)
return memdev->lsa_size;
}
+static int is_enabled(const char *drvpath)
+{
+ struct stat st;
+
+ if (lstat(drvpath, &st) < 0 || !S_ISLNK(st.st_mode))
+ return 0;
+ else
+ return 1;
+}
+
+CXL_EXPORT int cxl_memdev_nvdimm_bridge_active(struct cxl_memdev *memdev)
+{
+ struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+ struct cxl_nvdimm_bridge *bridge = memdev->bridge;
+ char *path = bridge->dev_buf;
+ int len = bridge->buf_len;
+
+ if (!bridge)
+ return 0;
+
+ if (snprintf(path, len, "%s/driver", bridge->dev_path) >= len) {
+ err(ctx, "%s: nvdimm bridge buffer too small!\n",
+ cxl_memdev_get_devname(memdev));
+ return 0;
+ }
+
+ return is_enabled(path);
+}
+
CXL_EXPORT void cxl_cmd_unref(struct cxl_cmd *cmd)
{
if (!cmd)
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
index d3b97a1..535e349 100644
--- a/cxl/libcxl.h
+++ b/cxl/libcxl.h
@@ -43,6 +43,7 @@ unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev);
unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev);
const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev);
size_t cxl_memdev_get_label_size(struct cxl_memdev *memdev);
+int cxl_memdev_nvdimm_bridge_active(struct cxl_memdev *memdev);
#define cxl_memdev_foreach(ctx, memdev) \
for (memdev = cxl_memdev_get_first(ctx); \
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
index 858e953..f3b0c63 100644
--- a/cxl/lib/libcxl.sym
+++ b/cxl/lib/libcxl.sym
@@ -65,6 +65,7 @@ global:
cxl_cmd_new_read_label;
cxl_cmd_read_label_get_payload;
cxl_memdev_get_label_size;
+ cxl_memdev_nvdimm_bridge_active;
local:
*;
};
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [ndctl PATCH v5 11/16] libcxl: add interfaces for label operations
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
` (9 preceding siblings ...)
2021-11-11 20:44 ` [ndctl PATCH v5 10/16] libcxl: add representation for an nvdimm bridge object Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 12/16] cxl: add commands to read, write, and zero labels Vishal Verma
` (4 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Add libcxl interfaces to allow performinfg label (LSA) manipulations.
Add a 'cxl_cmd_new_set_lsa' interface to create a 'Set LSA' mailbox
command payload, and interfaces to read, write, and zero the LSA area on
a memdev.
Cc: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
cxl/lib/private.h | 6 ++
cxl/lib/libcxl.c | 158 +++++++++++++++++++++++++++++++++++++++++++++
cxl/libcxl.h | 8 +++
cxl/lib/libcxl.sym | 4 ++
4 files changed, 176 insertions(+)
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
index 2f0b6ea..375ba22 100644
--- a/cxl/lib/private.h
+++ b/cxl/lib/private.h
@@ -88,6 +88,12 @@ struct cxl_cmd_get_lsa_in {
le32 length;
} __attribute__((packed));
+struct cxl_cmd_set_lsa {
+ le32 offset;
+ le32 rsvd;
+ unsigned char lsa_data[0];
+} __attribute__ ((packed));
+
struct cxl_cmd_get_health_info {
u8 health_status;
u8 media_status;
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
index 7bc0696..a63e18f 100644
--- a/cxl/lib/libcxl.c
+++ b/cxl/lib/libcxl.c
@@ -1197,3 +1197,161 @@ CXL_EXPORT int cxl_cmd_get_out_size(struct cxl_cmd *cmd)
{
return cmd->send_cmd->out.size;
}
+
+CXL_EXPORT struct cxl_cmd *cxl_cmd_new_write_label(struct cxl_memdev *memdev,
+ void *lsa_buf, unsigned int offset, unsigned int length)
+{
+ struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+ struct cxl_cmd_set_lsa *set_lsa;
+ struct cxl_cmd *cmd;
+ int rc;
+
+ cmd = cxl_cmd_new_generic(memdev, CXL_MEM_COMMAND_ID_SET_LSA);
+ if (!cmd)
+ return NULL;
+
+ /* this will allocate 'in.payload' */
+ rc = cxl_cmd_set_input_payload(cmd, NULL, sizeof(*set_lsa) + length);
+ if (rc) {
+ err(ctx, "%s: cmd setup failed: %s\n",
+ cxl_memdev_get_devname(memdev), strerror(-rc));
+ goto out_fail;
+ }
+ set_lsa = (struct cxl_cmd_set_lsa *)cmd->send_cmd->in.payload;
+ set_lsa->offset = cpu_to_le32(offset);
+ memcpy(set_lsa->lsa_data, lsa_buf, length);
+
+ return cmd;
+
+out_fail:
+ cxl_cmd_unref(cmd);
+ return NULL;
+}
+
+enum lsa_op {
+ LSA_OP_GET,
+ LSA_OP_SET,
+ LSA_OP_ZERO,
+};
+
+static int __lsa_op(struct cxl_memdev *memdev, int op, void *buf,
+ size_t length, size_t offset)
+{
+ const char *devname = cxl_memdev_get_devname(memdev);
+ struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+ void *zero_buf = NULL;
+ struct cxl_cmd *cmd;
+ ssize_t ret_len;
+ int rc = 0;
+
+ switch (op) {
+ case LSA_OP_GET:
+ cmd = cxl_cmd_new_read_label(memdev, offset, length);
+ if (!cmd)
+ return -ENOMEM;
+ rc = cxl_cmd_set_output_payload(cmd, buf, length);
+ if (rc) {
+ err(ctx, "%s: cmd setup failed: %s\n",
+ cxl_memdev_get_devname(memdev), strerror(-rc));
+ goto out;
+ }
+ break;
+ case LSA_OP_ZERO:
+ zero_buf = calloc(1, length);
+ if (!zero_buf)
+ return -ENOMEM;
+ buf = zero_buf;
+ /* fall through */
+ case LSA_OP_SET:
+ cmd = cxl_cmd_new_write_label(memdev, buf, offset, length);
+ if (!cmd) {
+ rc = -ENOMEM;
+ goto out_free;
+ }
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ rc = cxl_cmd_submit(cmd);
+ if (rc < 0) {
+ err(ctx, "%s: cmd submission failed: %s\n",
+ devname, strerror(-rc));
+ goto out;
+ }
+
+ rc = cxl_cmd_get_mbox_status(cmd);
+ if (rc != 0) {
+ err(ctx, "%s: firmware status: %d\n",
+ devname, rc);
+ rc = -ENXIO;
+ goto out;
+ }
+
+ if (op == LSA_OP_GET) {
+ ret_len = cxl_cmd_read_label_get_payload(cmd, buf, length);
+ if (ret_len < 0) {
+ rc = ret_len;
+ goto out;
+ }
+ }
+
+out:
+ cxl_cmd_unref(cmd);
+out_free:
+ free(zero_buf);
+ return rc;
+
+}
+
+static int lsa_op(struct cxl_memdev *memdev, int op, void *buf,
+ size_t length, size_t offset)
+{
+ const char *devname = cxl_memdev_get_devname(memdev);
+ struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+ size_t remaining = length, cur_len, cur_off = 0;
+ int label_iter_max, rc = 0;
+
+ if (op != LSA_OP_ZERO && buf == NULL) {
+ err(ctx, "%s: LSA buffer cannot be NULL\n", devname);
+ return -EINVAL;
+ }
+
+ if (length == 0)
+ return 0;
+
+ label_iter_max = memdev->payload_max - sizeof(struct cxl_cmd_set_lsa);
+ while (remaining) {
+ cur_len = min((size_t)label_iter_max, remaining);
+ rc = __lsa_op(memdev, op, buf + cur_off,
+ cur_len, offset + cur_off);
+ if (rc)
+ break;
+
+ remaining -= cur_len;
+ cur_off += cur_len;
+ }
+
+ if (rc && (op == LSA_OP_SET))
+ err(ctx, "%s: labels may be in an inconsistent state\n",
+ devname);
+ return rc;
+}
+
+CXL_EXPORT int cxl_memdev_zero_label(struct cxl_memdev *memdev, size_t length,
+ size_t offset)
+{
+ return lsa_op(memdev, LSA_OP_ZERO, NULL, length, offset);
+}
+
+CXL_EXPORT int cxl_memdev_write_label(struct cxl_memdev *memdev, void *buf,
+ size_t length, size_t offset)
+{
+ return lsa_op(memdev, LSA_OP_SET, buf, length, offset);
+}
+
+CXL_EXPORT int cxl_memdev_read_label(struct cxl_memdev *memdev, void *buf,
+ size_t length, size_t offset)
+{
+ return lsa_op(memdev, LSA_OP_GET, buf, length, offset);
+}
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
index 535e349..89d35ba 100644
--- a/cxl/libcxl.h
+++ b/cxl/libcxl.h
@@ -44,6 +44,12 @@ unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev);
const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev);
size_t cxl_memdev_get_label_size(struct cxl_memdev *memdev);
int cxl_memdev_nvdimm_bridge_active(struct cxl_memdev *memdev);
+int cxl_memdev_zero_label(struct cxl_memdev *memdev, size_t length,
+ size_t offset);
+int cxl_memdev_read_label(struct cxl_memdev *memdev, void *buf, size_t length,
+ size_t offset);
+int cxl_memdev_write_label(struct cxl_memdev *memdev, void *buf, size_t length,
+ size_t offset);
#define cxl_memdev_foreach(ctx, memdev) \
for (memdev = cxl_memdev_get_first(ctx); \
@@ -101,6 +107,8 @@ struct cxl_cmd *cxl_cmd_new_read_label(struct cxl_memdev *memdev,
unsigned int offset, unsigned int length);
ssize_t cxl_cmd_read_label_get_payload(struct cxl_cmd *cmd, void *buf,
unsigned int length);
+struct cxl_cmd *cxl_cmd_new_write_label(struct cxl_memdev *memdev,
+ void *buf, unsigned int offset, unsigned int length);
#ifdef __cplusplus
} /* extern "C" */
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
index f3b0c63..077d104 100644
--- a/cxl/lib/libcxl.sym
+++ b/cxl/lib/libcxl.sym
@@ -66,6 +66,10 @@ global:
cxl_cmd_read_label_get_payload;
cxl_memdev_get_label_size;
cxl_memdev_nvdimm_bridge_active;
+ cxl_cmd_new_write_label;
+ cxl_memdev_zero_label;
+ cxl_memdev_write_label;
+ cxl_memdev_read_label;
local:
*;
};
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* [ndctl PATCH v5 12/16] cxl: add commands to read, write, and zero labels
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
` (10 preceding siblings ...)
2021-11-11 20:44 ` [ndctl PATCH v5 11/16] libcxl: add interfaces for label operations Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 13/16] Documentation/cxl: add library API documentation Vishal Verma
` (3 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Add the following cxl-cli commands: read-labels, write-labels,
zero-labels. They operate on a CXL memdev, or a set of memdevs, and
allow interacting with the label storage area (LSA) on the device.
Add man pages for the above cxl-cli commands.
Cc: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
Documentation/cxl/cxl-read-labels.txt | 33 +++
Documentation/cxl/cxl-write-labels.txt | 32 +++
Documentation/cxl/cxl-zero-labels.txt | 29 ++
Documentation/cxl/labels-description.txt | 8 +
Documentation/cxl/labels-options.txt | 17 ++
Documentation/cxl/memdev-option.txt | 4 +
cxl/builtin.h | 5 +
cxl/cxl.c | 3 +
cxl/memdev.c | 324 +++++++++++++++++++++++
Documentation/cxl/Makefile.am | 5 +-
cxl/Makefile.am | 1 +
11 files changed, 460 insertions(+), 1 deletion(-)
create mode 100644 Documentation/cxl/cxl-read-labels.txt
create mode 100644 Documentation/cxl/cxl-write-labels.txt
create mode 100644 Documentation/cxl/cxl-zero-labels.txt
create mode 100644 Documentation/cxl/labels-description.txt
create mode 100644 Documentation/cxl/labels-options.txt
create mode 100644 Documentation/cxl/memdev-option.txt
create mode 100644 cxl/memdev.c
diff --git a/Documentation/cxl/cxl-read-labels.txt b/Documentation/cxl/cxl-read-labels.txt
new file mode 100644
index 0000000..143f296
--- /dev/null
+++ b/Documentation/cxl/cxl-read-labels.txt
@@ -0,0 +1,33 @@
+// SPDX-License-Identifier: GPL-2.0
+
+cxl-read-labels(1)
+==================
+
+NAME
+----
+cxl-read-labels - read out the label area on a CXL memdev
+
+SYNOPSIS
+--------
+[verse]
+'cxl read-labels' <mem0> [<mem1>..<memN>] [<options>]
+
+include::labels-description.txt[]
+This command dumps the raw binary data in a memdev's label area to stdout or a
+file. In the multi-memdev case the data is concatenated.
+
+OPTIONS
+-------
+include::labels-options.txt[]
+
+-o::
+--output::
+ output file
+
+include::../copyright.txt[]
+
+SEE ALSO
+--------
+linkcxl:cxl-write-labels[1],
+linkcxl:cxl-zero-labels[1],
+CXL-2.0 9.13.2
diff --git a/Documentation/cxl/cxl-write-labels.txt b/Documentation/cxl/cxl-write-labels.txt
new file mode 100644
index 0000000..75f42a5
--- /dev/null
+++ b/Documentation/cxl/cxl-write-labels.txt
@@ -0,0 +1,32 @@
+// SPDX-License-Identifier: GPL-2.0
+
+cxl-write-labels(1)
+===================
+
+NAME
+----
+cxl-write-labels - write data to the label area on a memdev
+
+SYNOPSIS
+--------
+[verse]
+'cxl write-labels <mem> [-i <filename>]'
+
+include::labels-description.txt[]
+Read data from the input filename, or stdin, and write it to the given
+<mem> device. Note that the device must not be active in any region, or
+actively registered with the nvdimm subsystem. If it is, the kernel will
+not allow write access to the device's label data area.
+
+OPTIONS
+-------
+include::labels-options.txt[]
+-i::
+--input::
+ input file
+
+SEE ALSO
+--------
+linkcxl:cxl-read-labels[1],
+linkcxl:cxl-zero-labels[1],
+CXL-2.0 9.13.2
diff --git a/Documentation/cxl/cxl-zero-labels.txt b/Documentation/cxl/cxl-zero-labels.txt
new file mode 100644
index 0000000..bf95b24
--- /dev/null
+++ b/Documentation/cxl/cxl-zero-labels.txt
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+
+cxl-zero-labels(1)
+==================
+
+NAME
+----
+cxl-zero-labels - zero out the label area on a set of memdevs
+
+SYNOPSIS
+--------
+[verse]
+'cxl zero-labels' <mem0> [<mem1>..<memN>] [<options>]
+
+include::labels-description.txt[]
+This command resets the device to its default state by
+deleting all labels.
+
+OPTIONS
+-------
+include::labels-options.txt[]
+
+include::../copyright.txt[]
+
+SEE ALSO
+--------
+linkcxl:cxl-read-labels[1],
+linkcxl:cxl-write-labels[1],
+CXL-2.0 9.13.2
diff --git a/Documentation/cxl/labels-description.txt b/Documentation/cxl/labels-description.txt
new file mode 100644
index 0000000..f60bd5d
--- /dev/null
+++ b/Documentation/cxl/labels-description.txt
@@ -0,0 +1,8 @@
+// SPDX-License-Identifier: GPL-2.0
+
+DESCRIPTION
+-----------
+The region label area is a small persistent partition of capacity
+available on some CXL memory devices. The label area is used to
+and configure or determine the set of memory devices participating
+in different interleave sets.
diff --git a/Documentation/cxl/labels-options.txt b/Documentation/cxl/labels-options.txt
new file mode 100644
index 0000000..06fbac3
--- /dev/null
+++ b/Documentation/cxl/labels-options.txt
@@ -0,0 +1,17 @@
+// SPDX-License-Identifier: GPL-2.0
+
+<memory device(s)>::
+include::memdev-option.txt[]
+
+-s::
+--size=::
+ Limit the operation to the given number of bytes. A size of 0
+ indicates to operate over the entire label capacity.
+
+-O::
+--offset=::
+ Begin the operation at the given offset into the label area.
+
+-v::
+ Turn on verbose debug messages in the library (if libcxl was built with
+ logging and debug enabled).
diff --git a/Documentation/cxl/memdev-option.txt b/Documentation/cxl/memdev-option.txt
new file mode 100644
index 0000000..e778582
--- /dev/null
+++ b/Documentation/cxl/memdev-option.txt
@@ -0,0 +1,4 @@
+// SPDX-License-Identifier: GPL-2.0
+A 'memX' device name, or a memdev id number. Restrict the operation to
+the specified memdev(s). The keyword 'all' can be specified to indicate
+the lack of any restriction.
diff --git a/cxl/builtin.h b/cxl/builtin.h
index 3797f98..78eca6e 100644
--- a/cxl/builtin.h
+++ b/cxl/builtin.h
@@ -5,4 +5,9 @@
struct cxl_ctx;
int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx);
+int cmd_write_labels(int argc, const char **argv, struct cxl_ctx *ctx);
+int cmd_read_labels(int argc, const char **argv, struct cxl_ctx *ctx);
+int cmd_zero_labels(int argc, const char **argv, struct cxl_ctx *ctx);
+int cmd_init_labels(int argc, const char **argv, struct cxl_ctx *ctx);
+int cmd_check_labels(int argc, const char **argv, struct cxl_ctx *ctx);
#endif /* _CXL_BUILTIN_H_ */
diff --git a/cxl/cxl.c b/cxl/cxl.c
index a7725f8..4b1661d 100644
--- a/cxl/cxl.c
+++ b/cxl/cxl.c
@@ -61,6 +61,9 @@ static struct cmd_struct commands[] = {
{ "version", .c_fn = cmd_version },
{ "list", .c_fn = cmd_list },
{ "help", .c_fn = cmd_help },
+ { "zero-labels", .c_fn = cmd_zero_labels },
+ { "read-labels", .c_fn = cmd_read_labels },
+ { "write-labels", .c_fn = cmd_write_labels },
};
int main(int argc, const char **argv)
diff --git a/cxl/memdev.c b/cxl/memdev.c
new file mode 100644
index 0000000..5ee38e5
--- /dev/null
+++ b/cxl/memdev.c
@@ -0,0 +1,324 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
+#include <stdio.h>
+#include <errno.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <limits.h>
+#include <util/log.h>
+#include <util/filter.h>
+#include <cxl/libcxl.h>
+#include <util/parse-options.h>
+#include <ccan/minmax/minmax.h>
+#include <ccan/array_size/array_size.h>
+
+struct action_context {
+ FILE *f_out;
+ FILE *f_in;
+};
+
+static struct parameters {
+ const char *outfile;
+ const char *infile;
+ unsigned len;
+ unsigned offset;
+ bool verbose;
+} param;
+
+#define fail(fmt, ...) \
+do { \
+ fprintf(stderr, "cxl-%s:%s:%d: " fmt, \
+ VERSION, __func__, __LINE__, ##__VA_ARGS__); \
+} while (0)
+
+#define BASE_OPTIONS() \
+OPT_BOOLEAN('v',"verbose", ¶m.verbose, "turn on debug")
+
+#define READ_OPTIONS() \
+OPT_STRING('o', "output", ¶m.outfile, "output-file", \
+ "filename to write label area contents")
+
+#define WRITE_OPTIONS() \
+OPT_STRING('i', "input", ¶m.infile, "input-file", \
+ "filename to read label area data")
+
+#define LABEL_OPTIONS() \
+OPT_UINTEGER('s', "size", ¶m.len, "number of label bytes to operate"), \
+OPT_UINTEGER('O', "offset", ¶m.offset, \
+ "offset into the label area to start operation")
+
+static const struct option read_options[] = {
+ BASE_OPTIONS(),
+ LABEL_OPTIONS(),
+ READ_OPTIONS(),
+ OPT_END(),
+};
+
+static const struct option write_options[] = {
+ BASE_OPTIONS(),
+ LABEL_OPTIONS(),
+ WRITE_OPTIONS(),
+ OPT_END(),
+};
+
+static const struct option zero_options[] = {
+ BASE_OPTIONS(),
+ LABEL_OPTIONS(),
+ OPT_END(),
+};
+
+static int action_zero(struct cxl_memdev *memdev, struct action_context *actx)
+{
+ size_t size;
+ int rc;
+
+ if (param.len)
+ size = param.len;
+ else
+ size = cxl_memdev_get_label_size(memdev);
+
+ if (cxl_memdev_nvdimm_bridge_active(memdev)) {
+ fprintf(stderr,
+ "%s: has active nvdimm bridge, abort label write\n",
+ cxl_memdev_get_devname(memdev));
+ return -EBUSY;
+ }
+
+ rc = cxl_memdev_zero_label(memdev, size, param.offset);
+ if (rc < 0)
+ fprintf(stderr, "%s: label zeroing failed: %s\n",
+ cxl_memdev_get_devname(memdev), strerror(-rc));
+
+ return rc;
+}
+
+static int action_write(struct cxl_memdev *memdev, struct action_context *actx)
+{
+ size_t size = param.len, read_len;
+ unsigned char *buf;
+ int rc;
+
+ if (cxl_memdev_nvdimm_bridge_active(memdev)) {
+ fprintf(stderr,
+ "%s: has active nvdimm bridge, abort label write\n",
+ cxl_memdev_get_devname(memdev));
+ return -EBUSY;
+ }
+
+ if (!size) {
+ size_t label_size = cxl_memdev_get_label_size(memdev);
+
+ fseek(actx->f_in, 0L, SEEK_END);
+ size = ftell(actx->f_in);
+ fseek(actx->f_in, 0L, SEEK_SET);
+
+ if (size > label_size) {
+ fprintf(stderr,
+ "File size (%zu) greater than label area size (%zu), aborting\n",
+ size, label_size);
+ return -EINVAL;
+ }
+ }
+
+ buf = calloc(1, size);
+ if (!buf)
+ return -ENOMEM;
+
+ read_len = fread(buf, 1, size, actx->f_in);
+ if (read_len != size) {
+ rc = -ENXIO;
+ goto out;
+ }
+
+ rc = cxl_memdev_write_label(memdev, buf, size, param.offset);
+ if (rc < 0)
+ fprintf(stderr, "%s: label write failed: %s\n",
+ cxl_memdev_get_devname(memdev), strerror(-rc));
+
+out:
+ free(buf);
+ return rc;
+}
+
+static int action_read(struct cxl_memdev *memdev, struct action_context *actx)
+{
+ size_t size, write_len;
+ char *buf;
+ int rc;
+
+ if (param.len)
+ size = param.len;
+ else
+ size = cxl_memdev_get_label_size(memdev);
+
+ buf = calloc(1, size);
+ if (!buf)
+ return -ENOMEM;
+
+ rc = cxl_memdev_read_label(memdev, buf, size, param.offset);
+ if (rc < 0) {
+ fprintf(stderr, "%s: label read failed: %s\n",
+ cxl_memdev_get_devname(memdev), strerror(-rc));
+ goto out;
+ }
+
+ write_len = fwrite(buf, 1, size, actx->f_out);
+ if (write_len != size) {
+ rc = -ENXIO;
+ goto out;
+ }
+ fflush(actx->f_out);
+
+out:
+ free(buf);
+ return rc;
+}
+
+static int memdev_action(int argc, const char **argv, struct cxl_ctx *ctx,
+ int (*action)(struct cxl_memdev *memdev, struct action_context *actx),
+ const struct option *options, const char *usage)
+{
+ struct cxl_memdev *memdev, *single = NULL;
+ struct action_context actx = { 0 };
+ int i, rc = 0, count = 0, err = 0;
+ const char * const u[] = {
+ usage,
+ NULL
+ };
+ unsigned long id;
+
+ argc = parse_options(argc, argv, options, u, 0);
+
+ if (argc == 0)
+ usage_with_options(u, options);
+ for (i = 0; i < argc; i++) {
+ if (strcmp(argv[i], "all") == 0) {
+ argv[0] = "all";
+ argc = 1;
+ break;
+ }
+
+ if (sscanf(argv[i], "mem%lu", &id) != 1) {
+ fprintf(stderr, "'%s' is not a valid memdev name\n",
+ argv[i]);
+ err++;
+ }
+ }
+
+ if (err == argc) {
+ usage_with_options(u, options);
+ return -EINVAL;
+ }
+
+ if (!param.outfile)
+ actx.f_out = stdout;
+ else {
+ actx.f_out = fopen(param.outfile, "w+");
+ if (!actx.f_out) {
+ fprintf(stderr, "failed to open: %s: (%s)\n",
+ param.outfile, strerror(errno));
+ rc = -errno;
+ goto out;
+ }
+ }
+
+ if (!param.infile) {
+ actx.f_in = stdin;
+ } else {
+ actx.f_in = fopen(param.infile, "r");
+ if (!actx.f_in) {
+ fprintf(stderr, "failed to open: %s: (%s)\n",
+ param.infile, strerror(errno));
+ rc = -errno;
+ goto out_close_fout;
+ }
+ }
+
+ if (param.verbose)
+ cxl_set_log_priority(ctx, LOG_DEBUG);
+
+ rc = 0;
+ err = 0;
+ count = 0;
+
+ for (i = 0; i < argc; i++) {
+ if (sscanf(argv[i], "mem%lu", &id) != 1
+ && strcmp(argv[i], "all") != 0)
+ continue;
+
+ cxl_memdev_foreach (ctx, memdev) {
+ if (!util_cxl_memdev_filter(memdev, argv[i]))
+ continue;
+
+ if (action == action_write) {
+ single = memdev;
+ rc = 0;
+ } else
+ rc = action(memdev, &actx);
+
+ if (rc == 0)
+ count++;
+ else if (rc && !err)
+ err = rc;
+ }
+ }
+ rc = err;
+
+ if (action == action_write) {
+ if (count > 1) {
+ error("write-labels only supports writing a single memdev\n");
+ usage_with_options(u, options);
+ return -EINVAL;
+ } else if (single) {
+ rc = action(single, &actx);
+ if (rc)
+ count = 0;
+ }
+ }
+
+ if (actx.f_in != stdin)
+ fclose(actx.f_in);
+
+ out_close_fout:
+ if (actx.f_out != stdout)
+ fclose(actx.f_out);
+
+ out:
+ /*
+ * count if some actions succeeded, 0 if none were attempted,
+ * negative error code otherwise.
+ */
+ if (count > 0)
+ return count;
+ return rc;
+}
+
+int cmd_write_labels(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ int count = memdev_action(argc, argv, ctx, action_write, write_options,
+ "cxl write-labels <memdev> [-i <filename>]");
+
+ fprintf(stderr, "wrote %d mem%s\n", count >= 0 ? count : 0,
+ count > 1 ? "s" : "");
+ return count >= 0 ? 0 : EXIT_FAILURE;
+}
+
+int cmd_read_labels(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ int count = memdev_action(argc, argv, ctx, action_read, read_options,
+ "cxl read-labels <mem0> [<mem1>..<memN>] [-o <filename>]");
+
+ fprintf(stderr, "read %d mem%s\n", count >= 0 ? count : 0,
+ count > 1 ? "s" : "");
+ return count >= 0 ? 0 : EXIT_FAILURE;
+}
+
+int cmd_zero_labels(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+ int count = memdev_action(argc, argv, ctx, action_zero, zero_options,
+ "cxl zero-labels <mem0> [<mem1>..<memN>] [<options>]");
+
+ fprintf(stderr, "zeroed %d mem%s\n", count >= 0 ? count : 0,
+ count > 1 ? "s" : "");
+ return count >= 0 ? 0 : EXIT_FAILURE;
+}
diff --git a/Documentation/cxl/Makefile.am b/Documentation/cxl/Makefile.am
index db98dd7..efabaa3 100644
--- a/Documentation/cxl/Makefile.am
+++ b/Documentation/cxl/Makefile.am
@@ -19,7 +19,10 @@ endif
man1_MANS = \
cxl.1 \
- cxl-list.1
+ cxl-list.1 \
+ cxl-read-labels.1 \
+ cxl-write-labels.1 \
+ cxl-zero-labels.1
EXTRA_DIST = $(man1_MANS)
diff --git a/cxl/Makefile.am b/cxl/Makefile.am
index 98606b9..da9f91d 100644
--- a/cxl/Makefile.am
+++ b/cxl/Makefile.am
@@ -10,6 +10,7 @@ config.h: $(srcdir)/Makefile.am
cxl_SOURCES =\
cxl.c \
list.c \
+ memdev.c \
../util/json.c \
builtin.h
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* [ndctl PATCH v5 13/16] Documentation/cxl: add library API documentation
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
` (11 preceding siblings ...)
2021-11-11 20:44 ` [ndctl PATCH v5 12/16] cxl: add commands to read, write, and zero labels Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 14/16] ndctl: Add CXL packages to the RPM spec Vishal Verma
` (2 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Add library API documentation for libcxl(3) using the existing
asciidoc(tor) build system. Add a section 3 man page for 'libcxl' that
provides an overview of the library and its usage, and a man page for
the 'cxl_new()' API.
Cc: Ben Widawsky <ben.widawsky@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
Documentation/cxl/lib/cxl_new.txt | 43 +++++++++++++++++++++++
Documentation/cxl/lib/libcxl.txt | 56 +++++++++++++++++++++++++++++
configure.ac | 1 +
Makefile.am | 1 +
.gitignore | 3 ++
Documentation/cxl/lib/Makefile.am | 58 +++++++++++++++++++++++++++++++
6 files changed, 162 insertions(+)
create mode 100644 Documentation/cxl/lib/cxl_new.txt
create mode 100644 Documentation/cxl/lib/libcxl.txt
create mode 100644 Documentation/cxl/lib/Makefile.am
diff --git a/Documentation/cxl/lib/cxl_new.txt b/Documentation/cxl/lib/cxl_new.txt
new file mode 100644
index 0000000..147d4e0
--- /dev/null
+++ b/Documentation/cxl/lib/cxl_new.txt
@@ -0,0 +1,43 @@
+// SPDX-License-Identifier: LGPL-2.0
+
+cxl_new(3)
+==========
+
+NAME
+----
+cxl_new - Create a new library context object that acts as a handle for all
+library operations
+
+SYNOPSIS
+--------
+[verse]
+----
+#include <cxl/libcxl.h>
+
+int cxl_new(struct cxl_ctx **ctx);
+----
+
+DESCRIPTION
+-----------
+Instantiates a new library context, and stores an opaque pointer in ctx. The
+context is freed by linklibcxl:cxl_unref[3], i.e. cxl_new(3) implies an
+internal linklibcxl:cxl_ref[3].
+
+
+RETURN VALUE
+------------
+Returns 0 on success, and a negative errno on failure.
+Possible error codes are:
+
+ * -ENOMEM
+ * -ENXIO
+
+EXAMPLE
+-------
+See example usage in test/libcxl.c
+
+include::../../copyright.txt[]
+
+SEE ALSO
+--------
+linklibcxl:cxl_ref[3], linklibcxl:cxl_unref[3]
diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt
new file mode 100644
index 0000000..2539369
--- /dev/null
+++ b/Documentation/cxl/lib/libcxl.txt
@@ -0,0 +1,56 @@
+// SPDX-License-Identifier: LGPL-2.0
+
+libcxl(3)
+=========
+
+NAME
+----
+libcxl - A library to interact with CXL devices through sysfs(5)
+and ioctl(2) interfaces
+
+SYNOPSIS
+--------
+[verse]
+#include <cxl/libcxl.h>
+cc ... -lcxl
+
+DESCRIPTION
+-----------
+libcxl provides interfaces to interact with CXL devices in Linux, using sysfs
+interfaces for most kernel interactions, and the ioctl() interface for command
+submission.
+
+The starting point for all library interfaces is a 'cxl_ctx' object, returned
+by linklibcxl:cxl_new[3]. CXL 'Type 3' memory devices are children of the
+cxl_ctx object, and can be iterated through using an iterator API.
+
+Library level interfaces that are agnostic to any device, or a specific
+subclass of operations have the prefix 'cxl_'
+
+The object representing a CXL Type 3 device is 'cxl_memdev'. Library interfaces
+related to these devices have the prefix 'cxl_memdev_'. These interfaces are
+mostly associated with sysfs interactions (unless otherwise noted in their
+respective documentation pages). They are typically used to retrieve data
+published by the kernel, or to send data or trigger kernel operations for a
+given device.
+
+A 'cxl_cmd' is a reference counted object which is used to perform 'Mailbox'
+commands as described in the CXL Specification. A 'cxl_cmd' object is tied to a
+'cxl_memdev'. Associated library interfaces have the prefix 'cxl_cmd_'. Within
+this sub-class of interfaces, there are:
+
+ * 'cxl_cmd_new_*' interfaces that allocate a new cxl_cmd object for a given
+ command type.
+
+ * 'cxl_cmd_submit' which submits the command via ioctl()
+
+ * 'cxl_cmd_<name>_get_<field>' interfaces that get specific fields out of the
+ command response
+
+ * 'cxl_cmd_get_*' interfaces to get general command related information.
+
+include::../../copyright.txt[]
+
+SEE ALSO
+--------
+linklibcxl:cxl[1]
diff --git a/configure.ac b/configure.ac
index dadae0a..00497ae 100644
--- a/configure.ac
+++ b/configure.ac
@@ -231,6 +231,7 @@ AC_CONFIG_FILES([
Documentation/ndctl/Makefile
Documentation/daxctl/Makefile
Documentation/cxl/Makefile
+ Documentation/cxl/lib/Makefile
])
AC_OUTPUT
diff --git a/Makefile.am b/Makefile.am
index 4904ee7..e2f6bef 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -4,6 +4,7 @@ ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS}
SUBDIRS = . cxl/lib daxctl/lib ndctl/lib cxl ndctl daxctl
if ENABLE_DOCS
SUBDIRS += Documentation/ndctl Documentation/daxctl Documentation/cxl
+SUBDIRS += Documentation/cxl/lib
endif
SUBDIRS += test
diff --git a/.gitignore b/.gitignore
index 6a97b92..6468c7a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -14,12 +14,15 @@ Makefile.in
/libtool
/stamp-h1
*.1
+*.3
Documentation/daxctl/asciidoc.conf
Documentation/ndctl/asciidoc.conf
Documentation/cxl/asciidoc.conf
+Documentation/cxl/lib/asciidoc.conf
Documentation/daxctl/asciidoctor-extensions.rb
Documentation/ndctl/asciidoctor-extensions.rb
Documentation/cxl/asciidoctor-extensions.rb
+Documentation/cxl/lib/asciidoctor-extensions.rb
Documentation/ndctl/attrs.adoc
.dirstamp
daxctl/config.h
diff --git a/Documentation/cxl/lib/Makefile.am b/Documentation/cxl/lib/Makefile.am
new file mode 100644
index 0000000..41e3a5f
--- /dev/null
+++ b/Documentation/cxl/lib/Makefile.am
@@ -0,0 +1,58 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020-2021 Intel Corporation. All rights reserved.
+
+if USE_ASCIIDOCTOR
+
+do_subst = sed -e 's,@Utility@,Libcxl,g' -e's,@utility@,libcxl,g'
+CONFFILE = asciidoctor-extensions.rb
+asciidoctor-extensions.rb: ../../asciidoctor-extensions.rb.in
+ $(AM_V_GEN) $(do_subst) < $< > $@
+
+else
+
+do_subst = sed -e 's,UTILITY,libcxl,g'
+CONFFILE = asciidoc.conf
+asciidoc.conf: ../../asciidoc.conf.in
+ $(AM_V_GEN) $(do_subst) < $< > $@
+
+endif
+
+man3_MANS = \
+ libcxl.3 \
+ cxl_new.3
+
+EXTRA_DIST = $(man3_MANS)
+
+CLEANFILES = $(man3_MANS)
+
+XML_DEPS = \
+ ../../../version.m4 \
+ ../../copyright.txt \
+ Makefile \
+ $(CONFFILE)
+
+RM ?= rm -f
+
+if USE_ASCIIDOCTOR
+
+%.3: %.txt $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@+ $@ && \
+ $(ASCIIDOC) -b manpage -d manpage -acompat-mode \
+ -I. -rasciidoctor-extensions \
+ -amansource=libcxl -amanmanual="libcxl Manual" \
+ -andctl_version=$(VERSION) -o $@+ $< && \
+ mv $@+ $@
+
+else
+
+%.xml: %.txt $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@+ $@ && \
+ $(ASCIIDOC) -b docbook -d manpage -f asciidoc.conf \
+ --unsafe -alibcxl_version=$(VERSION) -o $@+ $< && \
+ mv $@+ $@
+
+%.3: %.xml $(XML_DEPS)
+ $(AM_V_GEN)$(RM) $@ && \
+ $(XMLTO) -o . -m ../../manpage-normal.xsl man $<
+
+endif
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* [ndctl PATCH v5 14/16] ndctl: Add CXL packages to the RPM spec
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
` (12 preceding siblings ...)
2021-11-11 20:44 ` [ndctl PATCH v5 13/16] Documentation/cxl: add library API documentation Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 15/16] cxl-cli: add bash completion Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 16/16] cxl: add health information to cxl-list Vishal Verma
15 siblings, 0 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Ira Weiny, Vishal Verma
From: Ira Weiny <ira.weiny@intel.com>
Add CXL related packages - the cxl-cli utility, the libcxl library, and
development headers to respective RPM packages in the main spec file.
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
Makefile.am | 4 ++++
ndctl.spec.in | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 53 insertions(+)
diff --git a/Makefile.am b/Makefile.am
index e2f6bef..fa2010a 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -23,17 +23,21 @@ CLEANFILES += $(noinst_SCRIPTS)
do_rhel_subst = sed -e 's,VERSION,$(VERSION),g' \
-e 's,DAX_DNAME,daxctl-devel,g' \
+ -e 's,CXL_DNAME,cxl-devel,g' \
-e 's,DNAME,ndctl-devel,g' \
-e '/^%defattr.*/d' \
-e 's,DAX_LNAME,daxctl-libs,g' \
+ -e 's,CXL_LNAME,cxl-libs,g' \
-e 's,LNAME,ndctl-libs,g'
do_sles_subst = sed -e 's,VERSION,$(VERSION),g' \
-e 's,DAX_DNAME,libdaxctl-devel,g' \
+ -e 's,CXL_DNAME,libcxl-devel,g' \
-e 's,DNAME,libndctl-devel,g' \
-e 's,%license,%doc,g' \
-e 's,\(^License:.*GPL\)v2,\1-2.0,g' \
-e "s,DAX_LNAME,libdaxctl$$(($(LIBDAXCTL_CURRENT) - $(LIBDAXCTL_AGE))),g" \
+ -e "s,CXL_LNAME,libcxl$$(($(LIBCXL_CURRENT) - $(LIBCXL_AGE))),g" \
-e "s,LNAME,libndctl$$(($(LIBNDCTL_CURRENT) - $(LIBNDCTL_AGE))),g"
rhel/ndctl.spec: ndctl.spec.in Makefile.am version.m4
diff --git a/ndctl.spec.in b/ndctl.spec.in
index 0563b2d..4b08c05 100644
--- a/ndctl.spec.in
+++ b/ndctl.spec.in
@@ -8,6 +8,7 @@ Source0: https://github.com/pmem/%{name}/archive/v%{version}.tar.gz#/%{name}-%{v
Requires: LNAME%{?_isa} = %{version}-%{release}
Requires: DAX_LNAME%{?_isa} = %{version}-%{release}
+Requires: CXL_LNAME%{?_isa} = %{version}-%{release}
BuildRequires: autoconf
%if 0%{?rhel} < 9
BuildRequires: asciidoc
@@ -54,6 +55,24 @@ the Linux kernel Device-DAX facility. This facility enables DAX mappings
of performance / feature differentiated memory without need of a
filesystem.
+%package -n cxl-cli
+Summary: Manage CXL devices
+License: GPLv2
+Requires: CXL_LNAME%{?_isa} = %{version}-%{release}
+
+%description -n cxl-cli
+The cxl utility provides enumeration and provisioning commands for
+the Linux kernel CXL devices.
+
+%package -n CXL_DNAME
+Summary: Development files for libcxl
+License: LGPLv2
+Requires: CXL_LNAME%{?_isa} = %{version}-%{release}
+
+%description -n CXL_DNAME
+This package contains libraries and header files for developing applications
+that use libcxl, a library for enumerating and communicating with CXL devices.
+
%package -n DAX_DNAME
Summary: Development files for libdaxctl
License: LGPLv2
@@ -84,6 +103,13 @@ Device DAX is a facility for establishing DAX mappings of performance /
feature-differentiated memory. DAX_LNAME provides an enumeration /
control API for these devices.
+%package -n CXL_LNAME
+Summary: Management library for CXL devices
+License: LGPLv2
+
+%description -n CXL_LNAME
+libcxl is a library for enumerating and communicating with CXL devices.
+
%prep
%setup -q ndctl-%{version}
@@ -105,6 +131,8 @@ make check
%ldconfig_scriptlets -n DAX_LNAME
+%ldconfig_scriptlets -n CXL_LNAME
+
%define bashcompdir %(pkg-config --variable=completionsdir bash-completion)
%files
@@ -126,6 +154,12 @@ make check
%{_mandir}/man1/daxctl*
%{_datadir}/daxctl/daxctl.conf
+%files -n cxl-cli
+%defattr(-,root,root)
+%license LICENSES/preferred/GPL-2.0 LICENSES/other/MIT LICENSES/other/CC0-1.0
+%{_bindir}/cxl
+%{_mandir}/man1/cxl*
+
%files -n LNAME
%defattr(-,root,root)
%doc README.md
@@ -138,6 +172,12 @@ make check
%license LICENSES/preferred/LGPL-2.1 LICENSES/other/MIT LICENSES/other/CC0-1.0
%{_libdir}/libdaxctl.so.*
+%files -n CXL_LNAME
+%defattr(-,root,root)
+%doc README.md
+%license LICENSES/preferred/LGPL-2.1 LICENSES/other/MIT LICENSES/other/CC0-1.0
+%{_libdir}/libcxl.so.*
+
%files -n DNAME
%defattr(-,root,root)
%license LICENSES/preferred/LGPL-2.1
@@ -152,6 +192,15 @@ make check
%{_libdir}/libdaxctl.so
%{_libdir}/pkgconfig/libdaxctl.pc
+%files -n CXL_DNAME
+%defattr(-,root,root)
+%license LICENSES/preferred/LGPL-2.1
+%{_includedir}/cxl/
+%{_libdir}/libcxl.so
+%{_libdir}/pkgconfig/libcxl.pc
+%{_mandir}/man3/cxl*
+%{_mandir}/man3/libcxl.3.gz
+
%changelog
* Fri May 27 2016 Dan Williams <dan.j.williams@intel.com> - 53-1
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* [ndctl PATCH v5 15/16] cxl-cli: add bash completion
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
` (13 preceding siblings ...)
2021-11-11 20:44 ` [ndctl PATCH v5 14/16] ndctl: Add CXL packages to the RPM spec Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 20:44 ` [ndctl PATCH v5 16/16] cxl: add health information to cxl-list Vishal Verma
15 siblings, 0 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Add bash completion for the cxl-cli commands implemented so far:
cxl-list
cxl-read-labels
cxl-write-labels
cxl-zero-labels
Acked-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
contrib/ndctl | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 109 insertions(+)
diff --git a/contrib/ndctl b/contrib/ndctl
index 680fe6a..cae4b1b 100755
--- a/contrib/ndctl
+++ b/contrib/ndctl
@@ -647,5 +647,114 @@ _daxctl()
__daxctl_main
}
+### cxl-cli ###
+
+__cxl_get_devs()
+{
+ local opts=("--memdevs" "$*")
+ cxl list "${opts[@]}" | grep -E "^\s*\"memdev\":" | cut -d'"' -f4
+}
+
+__cxlcomp()
+{
+ local i=0
+
+ COMPREPLY=( $( compgen -W "$1" -- "$2" ) )
+ for cword in "${COMPREPLY[@]}"; do
+ if [[ "$cword" == @(--memdev|--offset|--size|--input|--output) ]]; then
+ COMPREPLY[$i]="${cword}="
+ else
+ COMPREPLY[$i]="${cword} "
+ fi
+ ((i++))
+ done
+}
+
+__cxl_comp_options()
+{
+
+ local cur=$1
+ local opts
+
+ if [[ "$cur" == *=* ]]; then
+ local cur_subopt=${cur%%=*}
+ local cur_arg=${cur##*=}
+ case $cur_subopt in
+ --memdev)
+ opts="$(__cxl_get_devs -i)"
+ ;;
+ *)
+ return
+ ;;
+ esac
+ __cxlcomp "$opts" "$cur_arg"
+ fi
+}
+
+__cxl_comp_non_option_args()
+{
+ local subcmd=$1
+ local cur=$2
+ local opts
+
+ case $subcmd in
+ read-labels)
+ ;&
+ write-labels)
+ ;&
+ zero-labels)
+ opts="$(__cxl_get_devs -i) all"
+ ;;
+ *)
+ return
+ ;;
+ esac
+ __cxlcomp "$opts" "$cur"
+}
+
+__cxl_main()
+{
+ local cmd subcmd
+
+ cmd=${words[0]}
+ COMPREPLY=()
+
+ # Skip options backward and find the last cxl command
+ __nd_common_prev_skip_opts
+ subcmd=$prev_skip_opts
+ # List cxl subcommands or long options
+ if [ -z $subcmd ]; then
+ if [[ $cur == --* ]]; then
+ cmds="--version --help --list-cmds"
+ else
+ cmds=$($cmd --list-cmds)
+ fi
+ __cxlcomp "$cmds" "$cur"
+ else
+ # List long option names
+ if [[ $cur == --* ]]; then
+ opts=$($cmd $subcmd --list-opts)
+ __cxlcomp "$opts" "$cur"
+ __cxl_comp_options "$cur"
+ else
+ [ -z "$subcmd" ] && return
+ __cxl_comp_non_option_args "$subcmd" "$cur"
+ fi
+ fi
+}
+
+type cxl &>/dev/null &&
+_cxl()
+{
+ local cur words cword prev
+ if [ $preload_get_comp_words_by_ref = "true" ]; then
+ _get_comp_words_by_ref -n =: cur words cword prev
+ else
+ __nd_common_get_comp_words_by_ref -n =: cur words cword prev
+ fi
+ __cxl_main
+}
+
complete -o nospace -F _ndctl ndctl
complete -o nospace -F _daxctl daxctl
+complete -o nospace -F _cxl cxl
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* [ndctl PATCH v5 16/16] cxl: add health information to cxl-list
2021-11-11 20:44 [ndctl PATCH v5 00/16] Initial CXL support Vishal Verma
` (14 preceding siblings ...)
2021-11-11 20:44 ` [ndctl PATCH v5 15/16] cxl-cli: add bash completion Vishal Verma
@ 2021-11-11 20:44 ` Vishal Verma
2021-11-11 23:59 ` Dan Williams
2021-11-12 21:52 ` [ndctl PATCH v6 " Vishal Verma
15 siblings, 2 replies; 31+ messages in thread
From: Vishal Verma @ 2021-11-11 20:44 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Add JSON output for fields from the 'GET_HEALTH_INFO' mailbox command
to memory device listings.
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
Documentation/cxl/cxl-list.txt | 4 +
util/json.h | 1 +
cxl/list.c | 5 +
util/json.c | 179 +++++++++++++++++++++++++++++++++
4 files changed, 189 insertions(+)
diff --git a/Documentation/cxl/cxl-list.txt b/Documentation/cxl/cxl-list.txt
index bd377b3..dc86651 100644
--- a/Documentation/cxl/cxl-list.txt
+++ b/Documentation/cxl/cxl-list.txt
@@ -53,6 +53,10 @@ OPTIONS
--idle::
Include idle (not enabled / zero-sized) devices in the listing
+-H::
+--health::
+ Include health information in the memdev listing
+
include::human-option.txt[]
include::verbose-option.txt[]
diff --git a/util/json.h b/util/json.h
index 91918c8..ce575e6 100644
--- a/util/json.h
+++ b/util/json.h
@@ -19,6 +19,7 @@ enum util_json_flags {
UTIL_JSON_CONFIGURED = (1 << 7),
UTIL_JSON_FIRMWARE = (1 << 8),
UTIL_JSON_DAX_MAPPINGS = (1 << 9),
+ UTIL_JSON_HEALTH = (1 << 10),
};
struct json_object;
diff --git a/cxl/list.c b/cxl/list.c
index 3dea73f..2fa155a 100644
--- a/cxl/list.c
+++ b/cxl/list.c
@@ -16,6 +16,7 @@ static struct {
bool memdevs;
bool idle;
bool human;
+ bool health;
} list;
static unsigned long listopts_to_flags(void)
@@ -26,6 +27,8 @@ static unsigned long listopts_to_flags(void)
flags |= UTIL_JSON_IDLE;
if (list.human)
flags |= UTIL_JSON_HUMAN;
+ if (list.health)
+ flags |= UTIL_JSON_HEALTH;
return flags;
}
@@ -57,6 +60,8 @@ int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx)
OPT_BOOLEAN('i', "idle", &list.idle, "include idle devices"),
OPT_BOOLEAN('u', "human", &list.human,
"use human friendly number formats "),
+ OPT_BOOLEAN('H', "health", &list.health,
+ "include memory device health information "),
OPT_END(),
};
const char * const u[] = {
diff --git a/util/json.c b/util/json.c
index 3be3a92..f97cf07 100644
--- a/util/json.c
+++ b/util/json.c
@@ -1442,6 +1442,180 @@ struct json_object *util_badblock_rec_to_json(u64 block, u64 count,
return NULL;
}
+static struct json_object *util_cxl_memdev_health_to_json(
+ struct cxl_memdev *memdev, unsigned long flags)
+{
+ struct json_object *jhealth;
+ struct json_object *jobj;
+ struct cxl_cmd *cmd;
+ u32 field;
+ int rc;
+
+ jhealth = json_object_new_object();
+ if (!jhealth)
+ return NULL;
+ if (!memdev)
+ goto err_jobj;
+
+ cmd = cxl_cmd_new_get_health_info(memdev);
+ if (!cmd)
+ goto err_jobj;
+
+ rc = cxl_cmd_submit(cmd);
+ if (rc < 0)
+ goto err_cmd;
+ rc = cxl_cmd_get_mbox_status(cmd);
+ if (rc != 0)
+ goto err_cmd;
+
+ /* health_status fields */
+ rc = cxl_cmd_health_info_get_maintenance_needed(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "maintenance_needed", jobj);
+
+ rc = cxl_cmd_health_info_get_performance_degraded(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "performance_degraded", jobj);
+
+ rc = cxl_cmd_health_info_get_hw_replacement_needed(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "hw_replacement_needed", jobj);
+
+ /* media_status fields */
+ rc = cxl_cmd_health_info_get_media_normal(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_normal", jobj);
+
+ rc = cxl_cmd_health_info_get_media_not_ready(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_not_ready", jobj);
+
+ rc = cxl_cmd_health_info_get_media_persistence_lost(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_persistence_lost", jobj);
+
+ rc = cxl_cmd_health_info_get_media_data_lost(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_data_lost", jobj);
+
+ rc = cxl_cmd_health_info_get_media_powerloss_persistence_loss(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_powerloss_persistence_loss", jobj);
+
+ rc = cxl_cmd_health_info_get_media_shutdown_persistence_loss(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_shutdown_persistence_loss", jobj);
+
+ rc = cxl_cmd_health_info_get_media_persistence_loss_imminent(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_persistence_loss_imminent", jobj);
+
+ rc = cxl_cmd_health_info_get_media_powerloss_data_loss(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_powerloss_data_loss", jobj);
+
+ rc = cxl_cmd_health_info_get_media_shutdown_data_loss(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_shutdown_data_loss", jobj);
+
+ rc = cxl_cmd_health_info_get_media_data_loss_imminent(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_data_loss_imminent", jobj);
+
+ /* ext_status fields */
+ if (cxl_cmd_health_info_get_ext_life_used_normal(cmd))
+ jobj = json_object_new_string("normal");
+ else if (cxl_cmd_health_info_get_ext_life_used_warning(cmd))
+ jobj = json_object_new_string("warning");
+ else if (cxl_cmd_health_info_get_ext_life_used_critical(cmd))
+ jobj = json_object_new_string("critical");
+ else
+ jobj = json_object_new_string("unknown");
+ if (jobj)
+ json_object_object_add(jhealth, "ext_life_used", jobj);
+
+ if (cxl_cmd_health_info_get_ext_temperature_normal(cmd))
+ jobj = json_object_new_string("normal");
+ else if (cxl_cmd_health_info_get_ext_temperature_warning(cmd))
+ jobj = json_object_new_string("warning");
+ else if (cxl_cmd_health_info_get_ext_temperature_critical(cmd))
+ jobj = json_object_new_string("critical");
+ else
+ jobj = json_object_new_string("unknown");
+ if (jobj)
+ json_object_object_add(jhealth, "ext_temperature", jobj);
+
+ if (cxl_cmd_health_info_get_ext_corrected_volatile_normal(cmd))
+ jobj = json_object_new_string("normal");
+ else if (cxl_cmd_health_info_get_ext_corrected_volatile_warning(cmd))
+ jobj = json_object_new_string("warning");
+ else
+ jobj = json_object_new_string("unknown");
+ if (jobj)
+ json_object_object_add(jhealth, "ext_corrected_volatile", jobj);
+
+ if (cxl_cmd_health_info_get_ext_corrected_persistent_normal(cmd))
+ jobj = json_object_new_string("normal");
+ else if (cxl_cmd_health_info_get_ext_corrected_persistent_warning(cmd))
+ jobj = json_object_new_string("warning");
+ else
+ jobj = json_object_new_string("unknown");
+ if (jobj)
+ json_object_object_add(jhealth, "ext_corrected_persistent", jobj);
+
+ /* other fields */
+ field = cxl_cmd_health_info_get_life_used(cmd);
+ if (field != 0xff) {
+ jobj = json_object_new_int(field);
+ if (jobj)
+ json_object_object_add(jhealth, "life_used_percent", jobj);
+ }
+
+ field = cxl_cmd_health_info_get_temperature(cmd);
+ if (field != 0xffff) {
+ jobj = json_object_new_int(field);
+ if (jobj)
+ json_object_object_add(jhealth, "temperature", jobj);
+ }
+
+ field = cxl_cmd_health_info_get_dirty_shutdowns(cmd);
+ jobj = json_object_new_int64(field);
+ if (jobj)
+ json_object_object_add(jhealth, "dirty_shutdowns", jobj);
+
+ field = cxl_cmd_health_info_get_volatile_errors(cmd);
+ jobj = json_object_new_int64(field);
+ if (jobj)
+ json_object_object_add(jhealth, "volatile_errors", jobj);
+
+ field = cxl_cmd_health_info_get_pmem_errors(cmd);
+ jobj = json_object_new_int64(field);
+ if (jobj)
+ json_object_object_add(jhealth, "pmem_errors", jobj);
+
+ cxl_cmd_unref(cmd);
+ return jhealth;
+
+err_cmd:
+ cxl_cmd_unref(cmd);
+err_jobj:
+ json_object_put(jhealth);
+ return NULL;
+}
+
struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
unsigned long flags)
{
@@ -1464,5 +1638,10 @@ struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
if (jobj)
json_object_object_add(jdev, "ram_size", jobj);
+ if (flags & UTIL_JSON_HEALTH) {
+ jobj = util_cxl_memdev_health_to_json(memdev, flags);
+ if (jobj)
+ json_object_object_add(jdev, "health", jobj);
+ }
return jdev;
}
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* Re: [ndctl PATCH v5 16/16] cxl: add health information to cxl-list
2021-11-11 20:44 ` [ndctl PATCH v5 16/16] cxl: add health information to cxl-list Vishal Verma
@ 2021-11-11 23:59 ` Dan Williams
2021-11-12 21:52 ` [ndctl PATCH v6 " Vishal Verma
1 sibling, 0 replies; 31+ messages in thread
From: Dan Williams @ 2021-11-11 23:59 UTC (permalink / raw)
To: Vishal Verma; +Cc: linux-cxl, Ben Widawsky, Linux NVDIMM
On Thu, Nov 11, 2021 at 12:45 PM Vishal Verma <vishal.l.verma@intel.com> wrote:
>
> Add JSON output for fields from the 'GET_HEALTH_INFO' mailbox command
> to memory device listings.
>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
> ---
> Documentation/cxl/cxl-list.txt | 4 +
> util/json.h | 1 +
> cxl/list.c | 5 +
> util/json.c | 179 +++++++++++++++++++++++++++++++++
> 4 files changed, 189 insertions(+)
>
> diff --git a/Documentation/cxl/cxl-list.txt b/Documentation/cxl/cxl-list.txt
> index bd377b3..dc86651 100644
> --- a/Documentation/cxl/cxl-list.txt
> +++ b/Documentation/cxl/cxl-list.txt
> @@ -53,6 +53,10 @@ OPTIONS
> --idle::
> Include idle (not enabled / zero-sized) devices in the listing
>
> +-H::
> +--health::
> + Include health information in the memdev listing
Let's include a sample listing here in the man page, like the
following, so that people can get an idea ahead of time about the key
names and value types in the payload. Other than that you can add:
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
{
"memdev":"mem7",
"pmem_size":268435456,
"ram_size":268435456,
"health":{
"maintenance_needed":true,
"performance_degraded":true,
"hw_replacement_needed":true,
"media_normal":false,
"media_not_ready":false,
"media_persistence_lost":false,
"media_data_lost":true,
"media_powerloss_persistence_loss":false,
"media_shutdown_persistence_loss":false,
"media_persistence_loss_imminent":false,
"media_powerloss_data_loss":false,
"media_shutdown_data_loss":false,
"media_data_loss_imminent":false,
"ext_life_used":"normal",
"ext_temperature":"critical",
"ext_corrected_volatile":"warning",
"ext_corrected_persistent":"normal",
"life_used_percent":15,
"temperature":25,
"dirty_shutdowns":10,
"volatile_errors":20,
"pmem_errors":30
}
^ permalink raw reply [flat|nested] 31+ messages in thread* [ndctl PATCH v6 16/16] cxl: add health information to cxl-list
2021-11-11 20:44 ` [ndctl PATCH v5 16/16] cxl: add health information to cxl-list Vishal Verma
2021-11-11 23:59 ` Dan Williams
@ 2021-11-12 21:52 ` Vishal Verma
2021-11-12 21:55 ` Dan Williams
1 sibling, 1 reply; 31+ messages in thread
From: Vishal Verma @ 2021-11-12 21:52 UTC (permalink / raw)
To: linux-cxl; +Cc: Dan Williams, Ben Widawsky, nvdimm, Vishal Verma
Add JSON output for fields from the 'GET_HEALTH_INFO' mailbox command
to memory device listings.
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
Documentation/cxl/cxl-list.txt | 38 +++++++
util/json.h | 1 +
cxl/list.c | 5 +
util/json.c | 179 +++++++++++++++++++++++++++++++++
4 files changed, 223 insertions(+)
diff --git a/Documentation/cxl/cxl-list.txt b/Documentation/cxl/cxl-list.txt
index e761cfa..d305a37 100644
--- a/Documentation/cxl/cxl-list.txt
+++ b/Documentation/cxl/cxl-list.txt
@@ -53,6 +53,44 @@ OPTIONS
--idle::
Include idle (not enabled / zero-sized) devices in the listing
+-H::
+--health::
+ Include health information in the memdev listing. Example listing:
+----
+# cxl list -m mem0 -H
+[
+ {
+ "memdev":"mem0",
+ "pmem_size":268435456,
+ "ram_size":268435456,
+ "health":{
+ "maintenance_needed":true,
+ "performance_degraded":true,
+ "hw_replacement_needed":true,
+ "media_normal":false,
+ "media_not_ready":false,
+ "media_persistence_lost":false,
+ "media_data_lost":true,
+ "media_powerloss_persistence_loss":false,
+ "media_shutdown_persistence_loss":false,
+ "media_persistence_loss_imminent":false,
+ "media_powerloss_data_loss":false,
+ "media_shutdown_data_loss":false,
+ "media_data_loss_imminent":false,
+ "ext_life_used":"normal",
+ "ext_temperature":"critical",
+ "ext_corrected_volatile":"warning",
+ "ext_corrected_persistent":"normal",
+ "life_used_percent":15,
+ "temperature":25,
+ "dirty_shutdowns":10,
+ "volatile_errors":20,
+ "pmem_errors":30
+ }
+ }
+]
+----
+
include::human-option.txt[]
include::verbose-option.txt[]
diff --git a/util/json.h b/util/json.h
index 91918c8..ce575e6 100644
--- a/util/json.h
+++ b/util/json.h
@@ -19,6 +19,7 @@ enum util_json_flags {
UTIL_JSON_CONFIGURED = (1 << 7),
UTIL_JSON_FIRMWARE = (1 << 8),
UTIL_JSON_DAX_MAPPINGS = (1 << 9),
+ UTIL_JSON_HEALTH = (1 << 10),
};
struct json_object;
diff --git a/cxl/list.c b/cxl/list.c
index 7c1d2eb..931606a 100644
--- a/cxl/list.c
+++ b/cxl/list.c
@@ -16,6 +16,7 @@ static struct {
bool memdevs;
bool idle;
bool human;
+ bool health;
} list;
static unsigned long listopts_to_flags(void)
@@ -26,6 +27,8 @@ static unsigned long listopts_to_flags(void)
flags |= UTIL_JSON_IDLE;
if (list.human)
flags |= UTIL_JSON_HUMAN;
+ if (list.health)
+ flags |= UTIL_JSON_HEALTH;
return flags;
}
@@ -57,6 +60,8 @@ int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx)
OPT_BOOLEAN('i', "idle", &list.idle, "include idle devices"),
OPT_BOOLEAN('u', "human", &list.human,
"use human friendly number formats "),
+ OPT_BOOLEAN('H', "health", &list.health,
+ "include memory device health information "),
OPT_END(),
};
const char * const u[] = {
diff --git a/util/json.c b/util/json.c
index 3be3a92..f97cf07 100644
--- a/util/json.c
+++ b/util/json.c
@@ -1442,6 +1442,180 @@ struct json_object *util_badblock_rec_to_json(u64 block, u64 count,
return NULL;
}
+static struct json_object *util_cxl_memdev_health_to_json(
+ struct cxl_memdev *memdev, unsigned long flags)
+{
+ struct json_object *jhealth;
+ struct json_object *jobj;
+ struct cxl_cmd *cmd;
+ u32 field;
+ int rc;
+
+ jhealth = json_object_new_object();
+ if (!jhealth)
+ return NULL;
+ if (!memdev)
+ goto err_jobj;
+
+ cmd = cxl_cmd_new_get_health_info(memdev);
+ if (!cmd)
+ goto err_jobj;
+
+ rc = cxl_cmd_submit(cmd);
+ if (rc < 0)
+ goto err_cmd;
+ rc = cxl_cmd_get_mbox_status(cmd);
+ if (rc != 0)
+ goto err_cmd;
+
+ /* health_status fields */
+ rc = cxl_cmd_health_info_get_maintenance_needed(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "maintenance_needed", jobj);
+
+ rc = cxl_cmd_health_info_get_performance_degraded(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "performance_degraded", jobj);
+
+ rc = cxl_cmd_health_info_get_hw_replacement_needed(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "hw_replacement_needed", jobj);
+
+ /* media_status fields */
+ rc = cxl_cmd_health_info_get_media_normal(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_normal", jobj);
+
+ rc = cxl_cmd_health_info_get_media_not_ready(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_not_ready", jobj);
+
+ rc = cxl_cmd_health_info_get_media_persistence_lost(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_persistence_lost", jobj);
+
+ rc = cxl_cmd_health_info_get_media_data_lost(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_data_lost", jobj);
+
+ rc = cxl_cmd_health_info_get_media_powerloss_persistence_loss(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_powerloss_persistence_loss", jobj);
+
+ rc = cxl_cmd_health_info_get_media_shutdown_persistence_loss(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_shutdown_persistence_loss", jobj);
+
+ rc = cxl_cmd_health_info_get_media_persistence_loss_imminent(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_persistence_loss_imminent", jobj);
+
+ rc = cxl_cmd_health_info_get_media_powerloss_data_loss(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_powerloss_data_loss", jobj);
+
+ rc = cxl_cmd_health_info_get_media_shutdown_data_loss(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_shutdown_data_loss", jobj);
+
+ rc = cxl_cmd_health_info_get_media_data_loss_imminent(cmd);
+ jobj = json_object_new_boolean(rc);
+ if (jobj)
+ json_object_object_add(jhealth, "media_data_loss_imminent", jobj);
+
+ /* ext_status fields */
+ if (cxl_cmd_health_info_get_ext_life_used_normal(cmd))
+ jobj = json_object_new_string("normal");
+ else if (cxl_cmd_health_info_get_ext_life_used_warning(cmd))
+ jobj = json_object_new_string("warning");
+ else if (cxl_cmd_health_info_get_ext_life_used_critical(cmd))
+ jobj = json_object_new_string("critical");
+ else
+ jobj = json_object_new_string("unknown");
+ if (jobj)
+ json_object_object_add(jhealth, "ext_life_used", jobj);
+
+ if (cxl_cmd_health_info_get_ext_temperature_normal(cmd))
+ jobj = json_object_new_string("normal");
+ else if (cxl_cmd_health_info_get_ext_temperature_warning(cmd))
+ jobj = json_object_new_string("warning");
+ else if (cxl_cmd_health_info_get_ext_temperature_critical(cmd))
+ jobj = json_object_new_string("critical");
+ else
+ jobj = json_object_new_string("unknown");
+ if (jobj)
+ json_object_object_add(jhealth, "ext_temperature", jobj);
+
+ if (cxl_cmd_health_info_get_ext_corrected_volatile_normal(cmd))
+ jobj = json_object_new_string("normal");
+ else if (cxl_cmd_health_info_get_ext_corrected_volatile_warning(cmd))
+ jobj = json_object_new_string("warning");
+ else
+ jobj = json_object_new_string("unknown");
+ if (jobj)
+ json_object_object_add(jhealth, "ext_corrected_volatile", jobj);
+
+ if (cxl_cmd_health_info_get_ext_corrected_persistent_normal(cmd))
+ jobj = json_object_new_string("normal");
+ else if (cxl_cmd_health_info_get_ext_corrected_persistent_warning(cmd))
+ jobj = json_object_new_string("warning");
+ else
+ jobj = json_object_new_string("unknown");
+ if (jobj)
+ json_object_object_add(jhealth, "ext_corrected_persistent", jobj);
+
+ /* other fields */
+ field = cxl_cmd_health_info_get_life_used(cmd);
+ if (field != 0xff) {
+ jobj = json_object_new_int(field);
+ if (jobj)
+ json_object_object_add(jhealth, "life_used_percent", jobj);
+ }
+
+ field = cxl_cmd_health_info_get_temperature(cmd);
+ if (field != 0xffff) {
+ jobj = json_object_new_int(field);
+ if (jobj)
+ json_object_object_add(jhealth, "temperature", jobj);
+ }
+
+ field = cxl_cmd_health_info_get_dirty_shutdowns(cmd);
+ jobj = json_object_new_int64(field);
+ if (jobj)
+ json_object_object_add(jhealth, "dirty_shutdowns", jobj);
+
+ field = cxl_cmd_health_info_get_volatile_errors(cmd);
+ jobj = json_object_new_int64(field);
+ if (jobj)
+ json_object_object_add(jhealth, "volatile_errors", jobj);
+
+ field = cxl_cmd_health_info_get_pmem_errors(cmd);
+ jobj = json_object_new_int64(field);
+ if (jobj)
+ json_object_object_add(jhealth, "pmem_errors", jobj);
+
+ cxl_cmd_unref(cmd);
+ return jhealth;
+
+err_cmd:
+ cxl_cmd_unref(cmd);
+err_jobj:
+ json_object_put(jhealth);
+ return NULL;
+}
+
struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
unsigned long flags)
{
@@ -1464,5 +1638,10 @@ struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
if (jobj)
json_object_object_add(jdev, "ram_size", jobj);
+ if (flags & UTIL_JSON_HEALTH) {
+ jobj = util_cxl_memdev_health_to_json(memdev, flags);
+ if (jobj)
+ json_object_object_add(jdev, "health", jobj);
+ }
return jdev;
}
--
2.31.1
^ permalink raw reply related [flat|nested] 31+ messages in thread* Re: [ndctl PATCH v6 16/16] cxl: add health information to cxl-list
2021-11-12 21:52 ` [ndctl PATCH v6 " Vishal Verma
@ 2021-11-12 21:55 ` Dan Williams
0 siblings, 0 replies; 31+ messages in thread
From: Dan Williams @ 2021-11-12 21:55 UTC (permalink / raw)
To: Vishal Verma; +Cc: linux-cxl, Ben Widawsky, Linux NVDIMM
On Fri, Nov 12, 2021 at 1:52 PM Vishal Verma <vishal.l.verma@intel.com> wrote:
>
> Add JSON output for fields from the 'GET_HEALTH_INFO' mailbox command
> to memory device listings.
>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
Looks good,
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
^ permalink raw reply [flat|nested] 31+ messages in thread