linux-nvdimm.lists.01.org archive mirror
 help / color / mirror / Atom feed
* [ndctl RFC PATCH 0/5] Initial CXL support
@ 2021-01-12  0:33 Vishal Verma
  2021-01-12  0:33 ` [ndctl RFC PATCH 1/5] cxl: add a cxl utility and libcxl library Vishal Verma
                   ` (4 more replies)
  0 siblings, 5 replies; 6+ messages in thread
From: Vishal Verma @ 2021-01-12  0:33 UTC (permalink / raw)
  To: linux-cxl; +Cc: linux-nvdimm, Ben Widawsky

This is an RFC patchset to add a new utility and library to support
CXL devices. This comprehends the kernel's sysfs layout for CXL
devices, and implements a command submission harness for CXL mailbox
commands via ioctl()s definied by the cxl_mem driver.

These patches include:
- libcxl representation of cxl_mem devices
- A command submission harness through libcxl
- A 'cxl-list' command which displays information about a device

Things missing, or next steps are:
- A test/libcxl.c to exercise all library interfaces
- Testing 'vendor specific' commands exported by the QEMU
  implementation[1]
- API documentation

The latest kernel patches can be found at [2].
An ndctl branch with these patches is also available at [3]

[1]: https://lore.kernel.org/qemu-devel/20210105165323.783725-1-ben.widawsky@intel.com/
[2]: https://gitlab.com/bwidawsk/linux/-/commits/cxl-2.0v3
[3]: https://github.com/pmem/ndctl/tree/cxl-2.0v1

Vishal Verma (5):
  cxl: add a cxl utility and libcxl library
  cxl: add a local copy of the cxl_mem UAPI header
  libcxl: add support for command query and submission
  libcxl: add accessors for retrieving 'Identify' information
  cxl/list: augment cxl-list with more data from the identify command

 Documentation/cxl/cxl-list.txt       |  65 +++
 Documentation/cxl/cxl.txt            |  34 ++
 Documentation/cxl/human-option.txt   |   8 +
 Documentation/cxl/verbose-option.txt |   5 +
 configure.ac                         |   3 +
 Makefile.am                          |   4 +-
 Makefile.am.in                       |   5 +
 cxl/lib/private.h                    |  87 ++++
 cxl/lib/libcxl.c                     | 714 +++++++++++++++++++++++++++
 cxl/builtin.h                        |   8 +
 cxl/cxl_mem.h                        | 176 +++++++
 cxl/libcxl.h                         |  66 +++
 util/filter.h                        |   2 +
 util/json.h                          |   4 +
 util/main.h                          |   3 +
 cxl/cxl.c                            |  95 ++++
 cxl/list.c                           | 138 ++++++
 util/filter.c                        |  20 +
 util/json.c                          |  46 ++
 Documentation/cxl/Makefile.am        |  58 +++
 cxl/Makefile.am                      |  21 +
 cxl/lib/Makefile.am                  |  32 ++
 cxl/lib/libcxl.pc.in                 |  11 +
 cxl/lib/libcxl.sym                   |  41 ++
 24 files changed, 1644 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/cxl/cxl-list.txt
 create mode 100644 Documentation/cxl/cxl.txt
 create mode 100644 Documentation/cxl/human-option.txt
 create mode 100644 Documentation/cxl/verbose-option.txt
 create mode 100644 cxl/lib/private.h
 create mode 100644 cxl/lib/libcxl.c
 create mode 100644 cxl/builtin.h
 create mode 100644 cxl/cxl_mem.h
 create mode 100644 cxl/libcxl.h
 create mode 100644 cxl/cxl.c
 create mode 100644 cxl/list.c
 create mode 100644 Documentation/cxl/Makefile.am
 create mode 100644 cxl/Makefile.am
 create mode 100644 cxl/lib/Makefile.am
 create mode 100644 cxl/lib/libcxl.pc.in
 create mode 100644 cxl/lib/libcxl.sym

-- 
2.29.2
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org

^ permalink raw reply	[flat|nested] 6+ messages in thread
* [ndctl RFC PATCH 1/5] cxl: add a cxl utility and libcxl library
  2021-01-12  0:33 [ndctl RFC PATCH 0/5] Initial CXL support Vishal Verma
@ 2021-01-12  0:33 ` Vishal Verma
  2021-01-12  0:34 ` [ndctl RFC PATCH 2/5] cxl: add a local copy of the cxl_mem UAPI header Vishal Verma
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Vishal Verma @ 2021-01-12  0:33 UTC (permalink / raw)
  To: linux-cxl; +Cc: linux-nvdimm, Ben Widawsky

CXL - or Compute eXpress Link - is a new interconnect that extends PCIe
to support a wide range of devices, including cache coherent memory
expanders. As such, these devices can be new sources of 'persistent
memory', and the 'ndctl' umbrella of tools and libraries needs to be able
to interact with them.

Add a new utility and library for managing these CXL memory devices. This
is an initial bring-up for interacting with CXL devices, and only includes
adding the utility and library infrastructure, parsing device information
from sysfs for CXL devices, and providing a 'cxl-list' command to
display this information in JSON formatted output.

Cc: Ben Widawsky <ben.widawsky@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
 Documentation/cxl/cxl-list.txt       |  65 ++++++
 Documentation/cxl/cxl.txt            |  34 +++
 Documentation/cxl/human-option.txt   |   8 +
 Documentation/cxl/verbose-option.txt |   5 +
 configure.ac                         |   3 +
 Makefile.am                          |   4 +-
 Makefile.am.in                       |   4 +
 cxl/lib/private.h                    |  28 +++
 cxl/lib/libcxl.c                     | 318 +++++++++++++++++++++++++++
 cxl/builtin.h                        |   8 +
 cxl/libcxl.h                         |  52 +++++
 util/filter.h                        |   2 +
 util/json.h                          |   3 +
 util/main.h                          |   3 +
 cxl/cxl.c                            |  95 ++++++++
 cxl/list.c                           | 113 ++++++++++
 util/filter.c                        |  20 ++
 util/json.c                          |  26 +++
 Documentation/cxl/Makefile.am        |  58 +++++
 cxl/Makefile.am                      |  21 ++
 cxl/lib/Makefile.am                  |  32 +++
 cxl/lib/libcxl.pc.in                 |  11 +
 cxl/lib/libcxl.sym                   |  26 +++
 23 files changed, 937 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/cxl/cxl-list.txt
 create mode 100644 Documentation/cxl/cxl.txt
 create mode 100644 Documentation/cxl/human-option.txt
 create mode 100644 Documentation/cxl/verbose-option.txt
 create mode 100644 cxl/lib/private.h
 create mode 100644 cxl/lib/libcxl.c
 create mode 100644 cxl/builtin.h
 create mode 100644 cxl/libcxl.h
 create mode 100644 cxl/cxl.c
 create mode 100644 cxl/list.c
 create mode 100644 Documentation/cxl/Makefile.am
 create mode 100644 cxl/Makefile.am
 create mode 100644 cxl/lib/Makefile.am
 create mode 100644 cxl/lib/libcxl.pc.in
 create mode 100644 cxl/lib/libcxl.sym

diff --git a/Documentation/cxl/cxl-list.txt b/Documentation/cxl/cxl-list.txt
new file mode 100644
index 0000000..107b388
--- /dev/null
+++ b/Documentation/cxl/cxl-list.txt
@@ -0,0 +1,65 @@
+// SPDX-License-Identifier: GPL-2.0
+
+cxl-list(1)
+===========
+
+NAME
+----
+cxl-list - CXL capable host bridges, switches, devices, and their attributes
+in json.
+
+SYNOPSIS
+--------
+[verse]
+'cxl list' [<options>]
+
+Walk the CXL capable device hierarchy in the system and list all device
+instances along with some of their major attributes.
+
+Options can be specified to limit the output to specific devices.
+By default, 'cxl list' with no options is equivalent to:
+[verse]
+cxl list --devices
+
+EXAMPLE
+-------
+----
+# cxl list --devices
+{
+  "memdev":"mem0",
+  "pmem_size":268435456,
+  "ram_size":0,
+}
+----
+
+OPTIONS
+-------
+-d::
+--dev=::
+	Specify a cxl device name to filter the listing. For example:
+----
+# cxl list --dev=mem0
+{
+  "memdev":"mem0",
+  "pmem_size":268435456,
+  "ram_size":0,
+}
+----
+
+-D::
+--devices::
+	Include all CXL devices in the listing
+
+-i::
+--idle::
+	Include idle (not enabled / zero-sized) devices in the listing
+
+include::human-option.txt[]
+
+include::verbose-option.txt[]
+
+include::../copyright.txt[]
+
+SEE ALSO
+--------
+linkcxl:ndctl-list[1]
diff --git a/Documentation/cxl/cxl.txt b/Documentation/cxl/cxl.txt
new file mode 100644
index 0000000..e99e61b
--- /dev/null
+++ b/Documentation/cxl/cxl.txt
@@ -0,0 +1,34 @@
+// SPDX-License-Identifier: GPL-2.0
+
+cxl(1)
+======
+
+NAME
+----
+cxl - Provides enumeration and provisioning commands for CXL devices
+
+SYNOPSIS
+--------
+[verse]
+'cxl' [--version] [--help] COMMAND [ARGS]
+
+OPTIONS
+-------
+-v::
+--version::
+  Display the version of the 'cxl' utility.
+
+-h::
+--help::
+  Run the 'cxl help' command.
+
+DESCRIPTION
+-----------
+The cxl utility provides enumeration and provisioning commands for
+the CXL devices managed by the Linux kernel.
+
+include::../copyright.txt[]
+
+SEE ALSO
+--------
+linkcxl:ndctl[1]
diff --git a/Documentation/cxl/human-option.txt b/Documentation/cxl/human-option.txt
new file mode 100644
index 0000000..2f4de7a
--- /dev/null
+++ b/Documentation/cxl/human-option.txt
@@ -0,0 +1,8 @@
+// SPDX-License-Identifier: GPL-2.0
+
+-u::
+--human::
+	By default the command will output machine-friendly raw-integer
+	data. Instead, with this flag, numbers representing storage size
+	will be formatted as human readable strings with units, other
+	fields are converted to hexadecimal strings.
diff --git a/Documentation/cxl/verbose-option.txt b/Documentation/cxl/verbose-option.txt
new file mode 100644
index 0000000..cb62c8e
--- /dev/null
+++ b/Documentation/cxl/verbose-option.txt
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+-v::
+--verbose::
+	Emit more debug messages
diff --git a/configure.ac b/configure.ac
index 5ec8d2f..7f5e6f0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -222,12 +222,15 @@ AC_CONFIG_HEADERS(config.h)
 AC_CONFIG_FILES([
         Makefile
         daxctl/lib/Makefile
+        cxl/lib/Makefile
         ndctl/lib/Makefile
         ndctl/Makefile
         daxctl/Makefile
+        cxl/Makefile
         test/Makefile
         Documentation/ndctl/Makefile
         Documentation/daxctl/Makefile
+        Documentation/cxl/Makefile
 ])
 
 AC_OUTPUT
diff --git a/Makefile.am b/Makefile.am
index 60a1998..ec4b9ed 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,9 +1,9 @@
 include Makefile.am.in
 
 ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS}
-SUBDIRS = . daxctl/lib ndctl/lib ndctl daxctl
+SUBDIRS = . cxl/lib daxctl/lib ndctl/lib cxl ndctl daxctl
 if ENABLE_DOCS
-SUBDIRS += Documentation/ndctl Documentation/daxctl
+SUBDIRS += Documentation/ndctl Documentation/daxctl Documentation/cxl
 endif
 SUBDIRS += test
 
diff --git a/Makefile.am.in b/Makefile.am.in
index bdceda9..6931d8c 100644
--- a/Makefile.am.in
+++ b/Makefile.am.in
@@ -42,3 +42,7 @@ LIBNDCTL_AGE=19
 LIBDAXCTL_CURRENT=6
 LIBDAXCTL_REVISION=0
 LIBDAXCTL_AGE=5
+
+LIBCXL_CURRENT=0
+LIBCXL_REVISION=0
+LIBCXL_AGE=0
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
new file mode 100644
index 0000000..f787512
--- /dev/null
+++ b/cxl/lib/private.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+/* Copyright (C) 2020-2021, Intel Corporation. All rights reserved. */
+#ifndef _LIBCXL_PRIVATE_H_
+#define _LIBCXL_PRIVATE_H_
+
+#include <libkmod.h>
+
+#define CXL_EXPORT __attribute__ ((visibility("default")))
+
+struct cxl_memdev {
+	int id, major, minor;
+	void *dev_buf;
+	size_t buf_len;
+	char *dev_path;
+	struct cxl_ctx *ctx;
+	struct list_node list;
+	unsigned long long pmem_size;
+	unsigned long long ram_size;
+	int payload_max;
+	struct kmod_module *module;
+};
+
+static inline int check_kmod(struct kmod_ctx *kmod_ctx)
+{
+	return kmod_ctx ? 0 : -ENXIO;
+}
+
+#endif /* _LIBCXL_PRIVATE_H_ */
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
new file mode 100644
index 0000000..c864208
--- /dev/null
+++ b/cxl/lib/libcxl.c
@@ -0,0 +1,318 @@
+// SPDX-License-Identifier: LGPL-2.1
+// Copyright (C) 2020-2021, Intel Corporation. All rights reserved.
+#include <stdio.h>
+#include <errno.h>
+#include <limits.h>
+#include <libgen.h>
+#include <stdlib.h>
+#include <dirent.h>
+#include <unistd.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/sysmacros.h>
+#include <uuid/uuid.h>
+#include <ccan/list/list.h>
+#include <ccan/array_size/array_size.h>
+
+#include <util/log.h>
+#include <util/sysfs.h>
+#include <cxl/libcxl.h>
+#include "private.h"
+
+/**
+ * struct cxl_ctx - library user context to find "nd" instances
+ *
+ * Instantiate with cxl_new(), which takes an initial reference.  Free
+ * the context by dropping the reference count to zero with
+ * cxl_unref(), or take additional references with cxl_ref()
+ * @timeout: default library timeout in milliseconds
+ */
+struct cxl_ctx {
+	/* log_ctx must be first member for cxl_set_log_fn compat */
+	struct log_ctx ctx;
+	int refcount;
+	void *userdata;
+	int memdevs_init;
+	struct list_head memdevs;
+	struct kmod_ctx *kmod_ctx;
+};
+
+static void free_memdev(struct cxl_memdev *memdev, struct list_head *head)
+{
+	if (head)
+		list_del_from(head, &memdev->list);
+	kmod_module_unref(memdev->module);
+	free(memdev->dev_buf);
+	free(memdev->dev_path);
+	free(memdev);
+}
+
+/**
+ * cxl_get_userdata - retrieve stored data pointer from library context
+ * @ctx: cxl library context
+ *
+ * This might be useful to access from callbacks like a custom logging
+ * function.
+ */
+CXL_EXPORT void *cxl_get_userdata(struct cxl_ctx *ctx)
+{
+	if (ctx == NULL)
+		return NULL;
+	return ctx->userdata;
+}
+
+/**
+ * cxl_set_userdata - store custom @userdata in the library context
+ * @ctx: cxl library context
+ * @userdata: data pointer
+ */
+CXL_EXPORT void cxl_set_userdata(struct cxl_ctx *ctx, void *userdata)
+{
+	if (ctx == NULL)
+		return;
+	ctx->userdata = userdata;
+}
+
+/**
+ * cxl_new - instantiate a new library context
+ * @ctx: context to establish
+ *
+ * Returns zero on success and stores an opaque pointer in ctx.  The
+ * context is freed by cxl_unref(), i.e. cxl_new() implies an
+ * internal cxl_ref().
+ */
+CXL_EXPORT int cxl_new(struct cxl_ctx **ctx)
+{
+	struct kmod_ctx *kmod_ctx;
+	struct cxl_ctx *c;
+	int rc = 0;
+
+	c = calloc(1, sizeof(struct cxl_ctx));
+	if (!c)
+		return -ENOMEM;
+
+	kmod_ctx = kmod_new(NULL, NULL);
+	if (check_kmod(kmod_ctx) != 0) {
+		rc = -ENXIO;
+		goto out;
+	}
+
+	c->refcount = 1;
+	log_init(&c->ctx, "libcxl", "CXL_LOG");
+	info(c, "ctx %p created\n", c);
+	dbg(c, "log_priority=%d\n", c->ctx.log_priority);
+	*ctx = c;
+	list_head_init(&c->memdevs);
+	c->kmod_ctx = kmod_ctx;
+
+	return 0;
+out:
+	free(c);
+	return rc;
+}
+
+/**
+ * cxl_ref - take an additional reference on the context
+ * @ctx: context established by cxl_new()
+ */
+CXL_EXPORT struct cxl_ctx *cxl_ref(struct cxl_ctx *ctx)
+{
+	if (ctx == NULL)
+		return NULL;
+	ctx->refcount++;
+	return ctx;
+}
+
+/**
+ * cxl_unref - drop a context reference count
+ * @ctx: context established by cxl_new()
+ *
+ * Drop a reference and if the resulting reference count is 0 destroy
+ * the context.
+ */
+CXL_EXPORT void cxl_unref(struct cxl_ctx *ctx)
+{
+	struct cxl_memdev *memdev, *_d;
+
+	if (ctx == NULL)
+		return;
+	ctx->refcount--;
+	if (ctx->refcount > 0)
+		return;
+
+	list_for_each_safe(&ctx->memdevs, memdev, _d, list)
+		free_memdev(memdev, &ctx->memdevs);
+
+	kmod_unref(ctx->kmod_ctx);
+	info(ctx, "context %p released\n", ctx);
+	free(ctx);
+}
+
+/**
+ * cxl_set_log_fn - override default log routine
+ * @ctx: cxl library context
+ * @log_fn: function to be called for logging messages
+ *
+ * The built-in logging writes to stderr. It can be overridden by a
+ * custom function, to plug log messages into the user's logging
+ * functionality.
+ */
+CXL_EXPORT void cxl_set_log_fn(struct cxl_ctx *ctx,
+		void (*cxl_log_fn)(struct cxl_ctx *ctx, int priority,
+			const char *file, int line, const char *fn,
+			const char *format, va_list args))
+{
+	ctx->ctx.log_fn = (log_fn) cxl_log_fn;
+	info(ctx, "custom logging function %p registered\n", cxl_log_fn);
+}
+
+/**
+ * cxl_get_log_priority - retrieve current library loglevel (syslog)
+ * @ctx: cxl library context
+ */
+CXL_EXPORT int cxl_get_log_priority(struct cxl_ctx *ctx)
+{
+	return ctx->ctx.log_priority;
+}
+
+/**
+ * cxl_set_log_priority - set log verbosity
+ * @priority: from syslog.h, LOG_ERR, LOG_INFO, LOG_DEBUG
+ *
+ * Note: LOG_DEBUG requires library be built with "configure --enable-debug"
+ */
+CXL_EXPORT void cxl_set_log_priority(struct cxl_ctx *ctx, int priority)
+{
+	ctx->ctx.log_priority = priority;
+}
+
+static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
+{
+	const char *devname = devpath_to_devname(cxlmem_base);
+	char *path = calloc(1, strlen(cxlmem_base) + 100);
+	struct cxl_ctx *ctx = parent;
+	struct cxl_memdev *memdev, *memdev_dup;
+	char buf[SYSFS_ATTR_SIZE];
+	struct stat st;
+
+	if (!path)
+		return NULL;
+	dbg(ctx, "%s: base: \'%s\'\n", __func__, cxlmem_base);
+
+	memdev = calloc(1, sizeof(*memdev));
+	if (!memdev)
+		goto err_dev;
+	memdev->id = id;
+	memdev->ctx = ctx;
+
+	sprintf(path, "/dev/cxl/%s", devname);
+	if (stat(path, &st) < 0)
+		goto err_read;
+	memdev->major = major(st.st_rdev);
+	memdev->minor = minor(st.st_rdev);
+
+	sprintf(path, "%s/pmem/size", cxlmem_base);
+	if (sysfs_read_attr(ctx, path, buf) < 0)
+		goto err_read;
+	memdev->pmem_size = strtoull(buf, NULL, 0);
+
+	sprintf(path, "%s/ram/size", cxlmem_base);
+	if (sysfs_read_attr(ctx, path, buf) < 0)
+		goto err_read;
+	memdev->ram_size = strtoull(buf, NULL, 0);
+
+	sprintf(path, "%s/payload_max", cxlmem_base);
+	if (sysfs_read_attr(ctx, path, buf) < 0)
+		goto err_read;
+	memdev->payload_max = strtoull(buf, NULL, 0);
+	if (memdev->payload_max < 0)
+		goto err_read;
+
+	memdev->dev_path = strdup(cxlmem_base);
+	if (!memdev->dev_path)
+		goto err_read;
+
+	memdev->dev_buf = calloc(1, strlen(cxlmem_base) + 50);
+	if (!memdev->dev_buf)
+		goto err_read;
+	memdev->buf_len = strlen(cxlmem_base) + 50;
+
+	cxl_memdev_foreach(ctx, memdev_dup)
+		if (memdev_dup->id == memdev->id) {
+			free_memdev(memdev, NULL);
+			free(path);
+			return memdev_dup;
+		}
+
+	list_add(&ctx->memdevs, &memdev->list);
+	free(path);
+	return memdev;
+
+ err_read:
+	free(memdev->dev_buf);
+	free(memdev->dev_path);
+	free(memdev);
+ err_dev:
+	free(path);
+	return NULL;
+}
+
+static void cxl_memdevs_init(struct cxl_ctx *ctx)
+{
+	if (ctx->memdevs_init)
+		return;
+
+	ctx->memdevs_init = 1;
+
+	sysfs_device_parse(ctx, "/sys/bus/cxl/devices", "mem", ctx,
+			   add_cxl_memdev);
+}
+
+CXL_EXPORT struct cxl_ctx *cxl_memdev_get_ctx(struct cxl_memdev *memdev)
+{
+	return memdev->ctx;
+}
+
+CXL_EXPORT struct cxl_memdev *cxl_memdev_get_first(struct cxl_ctx *ctx)
+{
+	cxl_memdevs_init(ctx);
+
+	return list_top(&ctx->memdevs, struct cxl_memdev, list);
+}
+
+CXL_EXPORT struct cxl_memdev *cxl_memdev_get_next(struct cxl_memdev *memdev)
+{
+	struct cxl_ctx *ctx = memdev->ctx;
+
+	return list_next(&ctx->memdevs, memdev, list);
+}
+
+CXL_EXPORT int cxl_memdev_get_id(struct cxl_memdev *memdev)
+{
+	return memdev->id;
+}
+
+CXL_EXPORT const char *cxl_memdev_get_devname(struct cxl_memdev *memdev)
+{
+	return devpath_to_devname(memdev->dev_path);
+}
+
+CXL_EXPORT int cxl_memdev_get_major(struct cxl_memdev *memdev)
+{
+	return memdev->major;
+}
+
+CXL_EXPORT int cxl_memdev_get_minor(struct cxl_memdev *memdev)
+{
+	return memdev->minor;
+}
+
+CXL_EXPORT unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev)
+{
+	return memdev->pmem_size;
+}
+
+CXL_EXPORT unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev)
+{
+	return memdev->ram_size;
+}
diff --git a/cxl/builtin.h b/cxl/builtin.h
new file mode 100644
index 0000000..3797f98
--- /dev/null
+++ b/cxl/builtin.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2020-2021 Intel Corporation. All rights reserved. */
+#ifndef _CXL_BUILTIN_H_
+#define _CXL_BUILTIN_H_
+
+struct cxl_ctx;
+int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx);
+#endif /* _CXL_BUILTIN_H_ */
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
new file mode 100644
index 0000000..8ee6873
--- /dev/null
+++ b/cxl/libcxl.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+/* Copyright (C) 2020-2021, Intel Corporation. All rights reserved. */
+#ifndef _LIBCXL_H_
+#define _LIBCXL_H_
+
+#include <stdarg.h>
+#include <unistd.h>
+
+#ifdef HAVE_UUID
+#include <uuid/uuid.h>
+#else
+typedef unsigned char uuid_t[16];
+#endif
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct cxl_ctx;
+struct cxl_ctx *cxl_ref(struct cxl_ctx *ctx);
+void cxl_unref(struct cxl_ctx *ctx);
+int cxl_new(struct cxl_ctx **ctx);
+void cxl_set_log_fn(struct cxl_ctx *ctx,
+		void (*log_fn)(struct cxl_ctx *ctx, int priority,
+			const char *file, int line, const char *fn,
+			const char *format, va_list args));
+int cxl_get_log_priority(struct cxl_ctx *ctx);
+void cxl_set_log_priority(struct cxl_ctx *ctx, int priority);
+void cxl_set_userdata(struct cxl_ctx *ctx, void *userdata);
+void *cxl_get_userdata(struct cxl_ctx *ctx);
+
+struct cxl_memdev;
+struct cxl_memdev *cxl_memdev_get_first(struct cxl_ctx *ctx);
+struct cxl_memdev *cxl_memdev_get_next(struct cxl_memdev *memdev);
+int cxl_memdev_get_id(struct cxl_memdev *memdev);
+const char *cxl_memdev_get_devname(struct cxl_memdev *memdev);
+int cxl_memdev_get_major(struct cxl_memdev *memdev);
+int cxl_memdev_get_minor(struct cxl_memdev *memdev);
+struct cxl_ctx *cxl_memdev_get_ctx(struct cxl_memdev *memdev);
+unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev);
+unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev);
+
+#define cxl_memdev_foreach(ctx, memdev) \
+        for (memdev = cxl_memdev_get_first(ctx); \
+             memdev != NULL; \
+             memdev = cxl_memdev_get_next(memdev))
+
+#ifdef __cplusplus
+} /* extern "C" */
+#endif
+
+#endif
diff --git a/util/filter.h b/util/filter.h
index 1e1a41c..9a80d65 100644
--- a/util/filter.h
+++ b/util/filter.h
@@ -29,6 +29,8 @@ struct daxctl_dev *util_daxctl_dev_filter(struct daxctl_dev *dev,
 		const char *ident);
 struct daxctl_region *util_daxctl_region_filter(struct daxctl_region *region,
 		const char *ident);
+struct cxl_memdev *util_cxl_memdev_filter(struct cxl_memdev *memdev,
+		const char *ident);
 
 enum ndctl_namespace_mode util_nsmode(const char *mode);
 const char *util_nsmode_name(enum ndctl_namespace_mode mode);
diff --git a/util/json.h b/util/json.h
index 0f09e36..91918c8 100644
--- a/util/json.h
+++ b/util/json.h
@@ -55,4 +55,7 @@ struct json_object *util_dimm_health_to_json(struct ndctl_dimm *dimm);
 struct json_object *util_dimm_firmware_to_json(struct ndctl_dimm *dimm,
 		unsigned long flags);
 struct json_object *util_region_capabilities_to_json(struct ndctl_region *region);
+struct cxl_memdev;
+struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
+		unsigned long flags);
 #endif /* __NDCTL_JSON_H__ */
diff --git a/util/main.h b/util/main.h
index c89a843..80b55c4 100644
--- a/util/main.h
+++ b/util/main.h
@@ -10,16 +10,19 @@
 enum program {
 	PROG_NDCTL,
 	PROG_DAXCTL,
+	PROG_CXL,
 };
 
 struct ndctl_ctx;
 struct daxctl_ctx;
+struct cxl_ctx;
 
 struct cmd_struct {
 	const char *cmd;
 	union {
 		int (*n_fn)(int, const char **, struct ndctl_ctx *ctx);
 		int (*d_fn)(int, const char **, struct daxctl_ctx *ctx);
+		int (*c_fn)(int, const char **, struct cxl_ctx *ctx);
 	};
 };
 
diff --git a/cxl/cxl.c b/cxl/cxl.c
new file mode 100644
index 0000000..ff43470
--- /dev/null
+++ b/cxl/cxl.c
@@ -0,0 +1,95 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2020-2021 Intel Corporation. All rights reserved.
+// Copyright (C) 2005 Andreas Ericsson. All rights reserved.
+
+/* originally copied from perf and git */
+
+#include <stdio.h>
+#include <errno.h>
+#include <string.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <cxl/libcxl.h>
+#include <util/parse-options.h>
+#include <ccan/array_size/array_size.h>
+
+#include <util/strbuf.h>
+#include <util/util.h>
+#include <util/main.h>
+#include <cxl/builtin.h>
+
+const char cxl_usage_string[] = "cxl [--version] [--help] COMMAND [ARGS]";
+const char cxl_more_info_string[] =
+	"See 'cxl help COMMAND' for more information on a specific command.\n"
+	" cxl --list-cmds to see all available commands";
+
+static int cmd_version(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+	printf("%s\n", VERSION);
+	return 0;
+}
+
+static int cmd_help(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+	const char * const builtin_help_subcommands[] = {
+		"list", NULL,
+	};
+	struct option builtin_help_options[] = {
+		OPT_END(),
+	};
+	const char *builtin_help_usage[] = {
+		"cxl help [command]",
+		NULL
+	};
+
+	argc = parse_options_subcommand(argc, argv, builtin_help_options,
+			builtin_help_subcommands, builtin_help_usage, 0);
+
+	if (!argv[0]) {
+		printf("\n usage: %s\n\n", cxl_usage_string);
+		printf("\n %s\n\n", cxl_more_info_string);
+		return 0;
+	}
+
+	return help_show_man_page(argv[0], "cxl", "CXL_MAN_VIEWER");
+}
+
+static struct cmd_struct commands[] = {
+	{ "version", .c_fn = cmd_version },
+	{ "list", .c_fn = cmd_list },
+	{ "help", .c_fn = cmd_help },
+};
+
+int main(int argc, const char **argv)
+{
+	struct cxl_ctx *ctx;
+	int rc;
+
+	/* Look for flags.. */
+	argv++;
+	argc--;
+	main_handle_options(&argv, &argc, cxl_usage_string, commands,
+			ARRAY_SIZE(commands));
+
+	if (argc > 0) {
+		if (!prefixcmp(argv[0], "--"))
+			argv[0] += 2;
+	} else {
+		/* The user didn't specify a command; give them help */
+		printf("\n usage: %s\n\n", cxl_usage_string);
+		printf("\n %s\n\n", cxl_more_info_string);
+		goto out;
+	}
+
+	rc = cxl_new(&ctx);
+	if (rc)
+		goto out;
+	main_handle_internal_command(argc, argv, ctx, commands,
+			ARRAY_SIZE(commands), PROG_CXL);
+	cxl_unref(ctx);
+	fprintf(stderr, "Unknown command: '%s'\n", argv[0]);
+out:
+	return 1;
+}
diff --git a/cxl/list.c b/cxl/list.c
new file mode 100644
index 0000000..3d44244
--- /dev/null
+++ b/cxl/list.c
@@ -0,0 +1,113 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2020-2021 Intel Corporation. All rights reserved.
+#include <stdio.h>
+#include <errno.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <limits.h>
+#include <util/json.h>
+#include <util/filter.h>
+#include <json-c/json.h>
+#include <cxl/libcxl.h>
+#include <util/parse-options.h>
+#include <ccan/array_size/array_size.h>
+
+static struct {
+	bool memdevs;
+	bool idle;
+	bool human;
+} list;
+
+static unsigned long listopts_to_flags(void)
+{
+	unsigned long flags = 0;
+
+	if (list.idle)
+		flags |= UTIL_JSON_IDLE;
+	if (list.human)
+		flags |= UTIL_JSON_HUMAN;
+	return flags;
+}
+
+static struct {
+	const char *memdev;
+} param;
+
+static int did_fail;
+
+#define fail(fmt, ...) \
+do { \
+	did_fail = 1; \
+	fprintf(stderr, "cxl-%s:%s:%d: " fmt, \
+			VERSION, __func__, __LINE__, ##__VA_ARGS__); \
+} while (0)
+
+static int num_list_flags(void)
+{
+	return list.memdevs;
+}
+
+int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx)
+{
+	const struct option options[] = {
+		OPT_STRING('d', "memdev", &param.memdev, "memory device name",
+			   "filter by CXL memory device name"),
+		OPT_BOOLEAN('D', "memdevs", &list.memdevs,
+			    "include CXL memory device info"),
+		OPT_BOOLEAN('i', "idle", &list.idle, "include idle devices"),
+		OPT_BOOLEAN('u', "human", &list.human,
+				"use human friendly number formats "),
+		OPT_END(),
+	};
+	const char * const u[] = {
+		"cxl list [<options>]",
+		NULL
+	};
+	struct json_object *jdevs = NULL;
+	unsigned long list_flags;
+	struct cxl_memdev *memdev;
+	int i;
+
+        argc = parse_options(argc, argv, options, u, 0);
+	for (i = 0; i < argc; i++)
+		error("unknown parameter \"%s\"\n", argv[i]);
+
+	if (argc)
+		usage_with_options(u, options);
+
+	if (num_list_flags() == 0)
+		list.memdevs = true;
+
+	list_flags = listopts_to_flags();
+
+	cxl_memdev_foreach(ctx, memdev) {
+		struct json_object *jdev = NULL;
+
+		if (!util_cxl_memdev_filter(memdev, param.memdev))
+			continue;
+
+		if (list.memdevs) {
+			if (!jdevs) {
+				jdevs = json_object_new_array();
+				if (!jdevs) {
+					fail("\n");
+					continue;
+				}
+			}
+
+			jdev = util_cxl_memdev_to_json(memdev, list_flags);
+			if (!jdev) {
+				fail("\n");
+				continue;
+			}
+			json_object_array_add(jdevs, jdev);
+		}
+	}
+
+	if (jdevs)
+		util_display_json_array(stdout, jdevs, list_flags);
+
+	if (did_fail)
+		return -ENOMEM;
+	return 0;
+}
diff --git a/util/filter.c b/util/filter.c
index 8b4aad3..d81dade 100644
--- a/util/filter.c
+++ b/util/filter.c
@@ -12,6 +12,7 @@
 #include <util/filter.h>
 #include <ndctl/libndctl.h>
 #include <daxctl/libdaxctl.h>
+#include <cxl/libcxl.h>
 
 struct ndctl_bus *util_bus_filter(struct ndctl_bus *bus, const char *__ident)
 {
@@ -339,6 +340,25 @@ struct daxctl_region *util_daxctl_region_filter(struct daxctl_region *region,
 	return NULL;
 }
 
+struct cxl_memdev *util_cxl_memdev_filter(struct cxl_memdev *memdev,
+					  const char *ident)
+{
+	int memdev_id;
+
+	if (!ident || strcmp(ident, "all") == 0)
+		return memdev;
+
+	if (strcmp(ident, cxl_memdev_get_devname(memdev)) == 0)
+		return memdev;
+
+	if ((sscanf(ident, "%d", &memdev_id) == 1
+			|| sscanf(ident, "mem%d", &memdev_id) == 1)
+			&& cxl_memdev_get_id(memdev) == memdev_id)
+		return memdev;
+
+	return NULL;
+}
+
 enum ndctl_namespace_mode util_nsmode(const char *mode)
 {
 	if (!mode)
diff --git a/util/json.c b/util/json.c
index ca0167b..a855571 100644
--- a/util/json.c
+++ b/util/json.c
@@ -9,6 +9,7 @@
 #include <json-c/printbuf.h>
 #include <ndctl/libndctl.h>
 #include <daxctl/libdaxctl.h>
+#include <cxl/libcxl.h>
 #include <ccan/array_size/array_size.h>
 #include <ccan/short_types/short_types.h>
 #include <ndctl.h>
@@ -1429,3 +1430,28 @@ struct json_object *util_badblock_rec_to_json(u64 block, u64 count,
 	json_object_put(jerr);
 	return NULL;
 }
+
+struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
+		unsigned long flags)
+{
+	const char *devname = cxl_memdev_get_devname(memdev);
+	struct json_object *jdev, *jobj;
+
+	jdev = json_object_new_object();
+	if (!devname || !jdev)
+		return NULL;
+
+	jobj = json_object_new_string(devname);
+	if (jobj)
+		json_object_object_add(jdev, "memdev", jobj);
+
+	jobj = util_json_object_size(cxl_memdev_get_pmem_size(memdev), flags);
+	if (jobj)
+		json_object_object_add(jdev, "pmem_size", jobj);
+
+	jobj = util_json_object_size(cxl_memdev_get_ram_size(memdev), flags);
+	if (jobj)
+		json_object_object_add(jdev, "ram_size", jobj);
+
+	return jdev;
+}
diff --git a/Documentation/cxl/Makefile.am b/Documentation/cxl/Makefile.am
new file mode 100644
index 0000000..2023f55
--- /dev/null
+++ b/Documentation/cxl/Makefile.am
@@ -0,0 +1,58 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2015-2020 Intel Corporation. All rights reserved.
+
+if USE_ASCIIDOCTOR
+
+do_subst = sed -e 's,@Utility@,Cxl,g' -e's,@utility@,cxl,g'
+CONFFILE = asciidoctor-extensions.rb
+asciidoctor-extensions.rb: ../asciidoctor-extensions.rb.in
+	$(AM_V_GEN) $(do_subst) < $< > $@
+
+else
+
+do_subst = sed -e 's,UTILITY,cxl,g'
+CONFFILE = asciidoc.conf
+asciidoc.conf: ../asciidoc.conf.in
+	$(AM_V_GEN) $(do_subst) < $< > $@
+
+endif
+
+man1_MANS = \
+	cxl.1 \
+	cxl-list.1
+
+EXTRA_DIST = $(man1_MANS)
+
+CLEANFILES = $(man1_MANS)
+
+XML_DEPS = \
+	../../version.m4 \
+	../copyright.txt \
+	Makefile \
+	$(CONFFILE)
+
+RM ?= rm -f
+
+if USE_ASCIIDOCTOR
+
+%.1: %.txt $(XML_DEPS)
+	$(AM_V_GEN)$(RM) $@+ $@ && \
+		$(ASCIIDOC) -b manpage -d manpage -acompat-mode \
+		-I. -rasciidoctor-extensions \
+		-amansource=cxl -amanmanual="cxl Manual" \
+		-andctl_version=$(VERSION) -o $@+ $< && \
+		mv $@+ $@
+
+else
+
+%.xml: %.txt $(XML_DEPS)
+	$(AM_V_GEN)$(RM) $@+ $@ && \
+		$(ASCIIDOC) -b docbook -d manpage -f asciidoc.conf \
+		--unsafe -acxl_version=$(VERSION) -o $@+ $< && \
+		mv $@+ $@
+
+%.1: %.xml $(XML_DEPS)
+	$(AM_V_GEN)$(RM) $@ && \
+		$(XMLTO) -o . -m ../manpage-normal.xsl man $<
+
+endif
diff --git a/cxl/Makefile.am b/cxl/Makefile.am
new file mode 100644
index 0000000..98606b9
--- /dev/null
+++ b/cxl/Makefile.am
@@ -0,0 +1,21 @@
+include $(top_srcdir)/Makefile.am.in
+
+bin_PROGRAMS = cxl
+
+DISTCLEANFILES = config.h
+BUILT_SOURCES = config.h
+config.h: $(srcdir)/Makefile.am
+	$(AM_V_GEN) echo "/* Autogenerated by cxl/Makefile.am */" >$@
+
+cxl_SOURCES =\
+		cxl.c \
+		list.c \
+		../util/json.c \
+		builtin.h
+
+cxl_LDADD =\
+	lib/libcxl.la \
+	../libutil.a \
+	$(UUID_LIBS) \
+	$(KMOD_LIBS) \
+	$(JSON_LIBS)
diff --git a/cxl/lib/Makefile.am b/cxl/lib/Makefile.am
new file mode 100644
index 0000000..01adf9f
--- /dev/null
+++ b/cxl/lib/Makefile.am
@@ -0,0 +1,32 @@
+include $(top_srcdir)/Makefile.am.in
+
+%.pc: %.pc.in Makefile
+	$(SED_PROCESS)
+
+pkginclude_HEADERS = ../libcxl.h
+lib_LTLIBRARIES = libcxl.la
+
+libcxl_la_SOURCES =\
+	../libcxl.h \
+	libcxl-private.h \
+	../../util/sysfs.c \
+	../../util/sysfs.h \
+	../../util/log.c \
+	../../util/log.h \
+	libcxl.c
+
+libcxl_la_LIBADD =\
+	$(UUID_LIBS) \
+	$(KMOD_LIBS)
+
+EXTRA_DIST += libcxl.sym
+
+libcxl_la_LDFLAGS = $(AM_LDFLAGS) \
+	-version-info $(LIBCXL_CURRENT):$(LIBCXL_REVISION):$(LIBCXL_AGE) \
+	-Wl,--version-script=$(top_srcdir)/cxl/lib/libcxl.sym
+libcxl_la_DEPENDENCIES = libcxl.sym
+
+pkgconfigdir = $(libdir)/pkgconfig
+pkgconfig_DATA = libcxl.pc
+EXTRA_DIST += libcxl.pc.in
+CLEANFILES += libcxl.pc
diff --git a/cxl/lib/libcxl.pc.in b/cxl/lib/libcxl.pc.in
new file mode 100644
index 0000000..949fcdc
--- /dev/null
+++ b/cxl/lib/libcxl.pc.in
@@ -0,0 +1,11 @@
+prefix=@prefix@
+exec_prefix=@exec_prefix@
+libdir=@libdir@
+includedir=@includedir@
+
+Name: libcxl
+Description: Manage CXL devices
+Version: @VERSION@
+Libs: -L${libdir} -lcxl
+Libs.private:
+Cflags: -I${includedir}
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
new file mode 100644
index 0000000..9bbbeb4
--- /dev/null
+++ b/cxl/lib/libcxl.sym
@@ -0,0 +1,26 @@
+LIBCXL_1 {
+global:
+	cxl_get_userdata;
+	cxl_set_userdata;
+	cxl_ref;
+	cxl_get_log_priority;
+	cxl_set_log_fn;
+	cxl_unref;
+	cxl_set_log_priority;
+	cxl_new;
+local:
+        *;
+};
+
+LIBCXL_2 {
+global:
+	cxl_memdev_get_first;
+	cxl_memdev_get_next;
+	cxl_memdev_get_id;
+	cxl_memdev_get_devname;
+	cxl_memdev_get_major;
+	cxl_memdev_get_minor;
+	cxl_memdev_get_ctx;
+	cxl_memdev_get_pmem_size;
+	cxl_memdev_get_ram_size;
+} LIBCXL_1;
-- 
2.29.2
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org

^ permalink raw reply related	[flat|nested] 6+ messages in thread
* [ndctl RFC PATCH 2/5] cxl: add a local copy of the cxl_mem UAPI header
  2021-01-12  0:33 [ndctl RFC PATCH 0/5] Initial CXL support Vishal Verma
  2021-01-12  0:33 ` [ndctl RFC PATCH 1/5] cxl: add a cxl utility and libcxl library Vishal Verma
@ 2021-01-12  0:34 ` Vishal Verma
  2021-01-12  0:34 ` [ndctl RFC PATCH 3/5] libcxl: add support for command query and submission Vishal Verma
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Vishal Verma @ 2021-01-12  0:34 UTC (permalink / raw)
  To: linux-cxl; +Cc: linux-nvdimm, Ben Widawsky

While CXL functionality is under development, it is useful to have a
local copy of the UAPI header for cxl_mem definitions. This allows
building cxl and libcxl on systems where the appropriate kernel headers
are not installed in the usual locations.

Cc: Ben Widawsky <ben.widawsky@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
 Makefile.am.in      |   1 +
 cxl/cxl_mem.h       | 176 ++++++++++++++++++++++++++++++++++++++++++++
 cxl/lib/Makefile.am |   2 +-
 3 files changed, 178 insertions(+), 1 deletion(-)
 create mode 100644 cxl/cxl_mem.h

diff --git a/Makefile.am.in b/Makefile.am.in
index 6931d8c..e476bce 100644
--- a/Makefile.am.in
+++ b/Makefile.am.in
@@ -11,6 +11,7 @@ AM_CPPFLAGS = \
 	-DNDCTL_MAN_PATH=\""$(mandir)"\" \
 	-I${top_srcdir}/ndctl/lib \
 	-I${top_srcdir}/ndctl \
+	-I${top_srcdir}/cxl \
 	-I${top_srcdir}/ \
 	$(KMOD_CFLAGS) \
 	$(UDEV_CFLAGS) \
diff --git a/cxl/cxl_mem.h b/cxl/cxl_mem.h
new file mode 100644
index 0000000..ce309e9
--- /dev/null
+++ b/cxl/cxl_mem.h
@@ -0,0 +1,176 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+/* Copyright (C) 2020-2021, Intel Corporation. All rights reserved. */
+/*
+ * CXL IOCTLs for Memory Devices
+ */
+
+#ifndef _UAPI_CXL_MEM_H_
+#define _UAPI_CXL_MEM_H_
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+#include <linux/types.h>
+#include <sys/user.h>
+#include <unistd.h>
+
+#define __user
+
+/**
+ * DOC: UAPI
+ *
+ * CXL memory devices expose UAPI to have a standard user interface.
+ * Userspace can refer to these structure definitions and UAPI formats
+ * to communicate to driver. The commands themselves are somewhat obfuscated
+ * with macro magic. They have the form CXL_MEM_COMMAND_ID_<name>.
+ *
+ * For example "CXL_MEM_COMMAND_ID_INVALID"
+ *
+ * Not all of all commands that the driver supports are always available for use
+ * by userspace. Userspace must check the results from the QUERY command in
+ * order to determine the live set of commands.
+ */
+
+#define CXL_MEM_QUERY_COMMANDS _IOR(0xCE, 1, struct cxl_mem_query_commands)
+#define CXL_MEM_SEND_COMMAND _IOWR(0xCE, 2, struct cxl_send_command)
+
+#undef CMDS
+#define CMDS                                                                   \
+	C(INVALID,	"Invalid Command"),                                    \
+	C(IDENTIFY,	"Identify Command"),                                   \
+	C(RAW,		"Raw device command"),                                 \
+	C(GET_SUPPORTED_LOGS,		"Get Supported Logs"),                 \
+	C(GET_LOG,	"Get Log"),                                            \
+	C(MAX,		"Last command")
+
+#undef C
+#define C(a, b) CXL_MEM_COMMAND_ID_##a
+
+enum { CMDS };
+
+/**
+ * struct cxl_command_info - Command information returned from a query.
+ * @id: ID number for the command.
+ * @flags: Flags that specify command behavior.
+ *
+ *  * %CXL_MEM_COMMAND_FLAG_KERNEL: This command is reserved for exclusive
+ *    kernel use.
+ *  * %CXL_MEM_COMMAND_FLAG_MUTEX: This command may require coordination with
+ *    the kernel in order to complete successfully.
+ *
+ * @size_in: Expected input size, or -1 if variable length.
+ * @size_out: Expected output size, or -1 if variable length.
+ *
+ * Represents a single command that is supported by both the driver and the
+ * hardware. The is returned as part of an array from the query ioctl. The
+ * following would be a command named "foobar" that takes a variable length
+ * input and returns 0 bytes of output.
+ *
+ *  - @id = 10
+ *  - @flags = CXL_MEM_COMMAND_FLAG_MUTEX
+ *  - @size_in = -1
+ *  - @size_out = 0
+ *
+ * See struct cxl_mem_query_commands.
+ */
+struct cxl_command_info {
+	__u32 id;
+
+	__u32 flags;
+#define CXL_MEM_COMMAND_FLAG_NONE 0
+#define CXL_MEM_COMMAND_FLAG_KERNEL BIT(0)
+#define CXL_MEM_COMMAND_FLAG_MUTEX BIT(1)
+#define CXL_MEM_COMMAND_FLAG_MASK GENMASK(31, 2)
+
+	__s32 size_in;
+	__s32 size_out;
+};
+
+/**
+ * struct cxl_mem_query_commands - Query supported commands.
+ * @n_commands: In/out parameter. When @n_commands is > 0, the driver will
+ *		return min(num_support_commands, n_commands). When @n_commands
+ *		is 0, driver will return the number of total supported commands.
+ * @rsvd: Reserved for future use.
+ * @commands: Output array of supported commands. This array must be allocated
+ *            by userspace to be at least min(num_support_commands, @n_commands)
+ *
+ * Allow userspace to query the available commands supported by both the driver,
+ * and the hardware. Commands that aren't supported by either the driver, or the
+ * hardware are not returned in the query.
+ *
+ * Examples:
+ *
+ *  - { .n_commands = 0 } // Get number of supported commands
+ *  - { .n_commands = 15, .commands = buf } // Return first 15 (or less)
+ *    supported commands
+ *
+ *  See struct cxl_command_info.
+ */
+struct cxl_mem_query_commands {
+	/*
+	 * Input: Number of commands to return (space allocated by user)
+	 * Output: Number of commands supported by the driver/hardware
+	 *
+	 * If n_commands is 0, kernel will only return number of commands and
+	 * not try to populate commands[], thus allowing userspace to know how
+	 * much space to allocate
+	 */
+	__u32 n_commands;
+	__u32 rsvd;
+
+	struct cxl_command_info __user commands[]; /* out: supported commands */
+};
+
+/**
+ * struct cxl_send_command - Send a command to a memory device.
+ * @id: The command to send to the memory device. This must be one of the
+ *	commands returned by the query command.
+ * @flags: Flags for the command (input).
+ * @raw: Special fields for raw commands
+ * @raw.opcode: Opcode passed to hardware when using the RAW command.
+ * @raw.rsvd: Must be zero.
+ * @rsvd: Must be zero.
+ * @retval: Return value from the memory device (output).
+ * @size_in: Size of the payload to provide to the device (input).
+ * @size_out: Size of the payload received from the device (input/output). This
+ *	      field is filled in by userspace to let the driver know how much
+ *	      space was allocated for output. It is populated by the driver to
+ *	      let userspace know how large the output payload actually was.
+ * @in_payload: Pointer to memory for payload input (little endian order).
+ * @out_payload: Pointer to memory for payload output (little endian order).
+ *
+ * Mechanism for userspace to send a command to the hardware for processing. The
+ * driver will do basic validation on the command sizes, but the payload input
+ * and output are not introspected. Userspace is required to allocate large
+ * enough buffers for max(size_in, size_out).
+ */
+struct cxl_send_command {
+	__u32 id;
+	__u32 flags;
+	union {
+		struct {
+			__u16 opcode;
+			__u16 rsvd;
+		} raw;
+		__u32 rsvd;
+	};
+	__u32 retval;
+
+	struct {
+		__s32 size_in;
+		__u64 in_payload;
+	};
+
+	struct {
+		__s32 size_out;
+		__u64 out_payload;
+	};
+};
+
+#if defined(__cplusplus)
+}
+#endif
+
+#endif
diff --git a/cxl/lib/Makefile.am b/cxl/lib/Makefile.am
index 01adf9f..dfd2784 100644
--- a/cxl/lib/Makefile.am
+++ b/cxl/lib/Makefile.am
@@ -3,7 +3,7 @@ include $(top_srcdir)/Makefile.am.in
 %.pc: %.pc.in Makefile
 	$(SED_PROCESS)
 
-pkginclude_HEADERS = ../libcxl.h
+pkginclude_HEADERS = ../libcxl.h ../cxl_mem.h
 lib_LTLIBRARIES = libcxl.la
 
 libcxl_la_SOURCES =\
-- 
2.29.2
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org

^ permalink raw reply related	[flat|nested] 6+ messages in thread
* [ndctl RFC PATCH 3/5] libcxl: add support for command query and submission
  2021-01-12  0:33 [ndctl RFC PATCH 0/5] Initial CXL support Vishal Verma
  2021-01-12  0:33 ` [ndctl RFC PATCH 1/5] cxl: add a cxl utility and libcxl library Vishal Verma
  2021-01-12  0:34 ` [ndctl RFC PATCH 2/5] cxl: add a local copy of the cxl_mem UAPI header Vishal Verma
@ 2021-01-12  0:34 ` Vishal Verma
  2021-01-12  0:34 ` [ndctl RFC PATCH 4/5] libcxl: add accessors for retrieving 'Identify' information Vishal Verma
  2021-01-12  0:34 ` [ndctl RFC PATCH 5/5] cxl/list: augment cxl-list with more data from the identify command Vishal Verma
  4 siblings, 0 replies; 6+ messages in thread
From: Vishal Verma @ 2021-01-12  0:34 UTC (permalink / raw)
  To: linux-cxl; +Cc: linux-nvdimm, Ben Widawsky

Add a set of APIs around 'cxl_cmd' for querying the kernel for supported
commands, allocating and validating command structures against the
supported set, and submitting the commands.

'Query Commands' and 'Send Command' are implemented as IOCTLs in the
kernel. 'Query Commands' returns information about each supported
command, such as flags governing its use, or input and output payload
sizes. This information is used to validate command support, as well as
set up input and output buffers for the actual command submission.

Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
 cxl/lib/private.h  |  40 ++++++
 cxl/lib/libcxl.c   | 352 +++++++++++++++++++++++++++++++++++++++++++++
 cxl/libcxl.h       |  11 ++
 cxl/lib/libcxl.sym |  12 ++
 4 files changed, 415 insertions(+)

diff --git a/cxl/lib/private.h b/cxl/lib/private.h
index f787512..c7ebfcb 100644
--- a/cxl/lib/private.h
+++ b/cxl/lib/private.h
@@ -4,6 +4,9 @@
 #define _LIBCXL_PRIVATE_H_
 
 #include <libkmod.h>
+#include <cxl_mem.h>
+#include <ccan/endian/endian.h>
+#include <ccan/short_types/short_types.h>
 
 #define CXL_EXPORT __attribute__ ((visibility("default")))
 
@@ -20,6 +23,43 @@ struct cxl_memdev {
 	struct kmod_module *module;
 };
 
+#undef C
+#define C(a, b) { b }
+static struct {
+	const char *name;
+} command_names[] = { CMDS };
+#undef C
+
+enum cxl_cmd_query_status {
+	CXL_CMD_QUERY_NOT_RUN = 0,
+	CXL_CMD_QUERY_OK,
+	CXL_CMD_QUERY_UNSUPPORTED,
+};
+
+/**
+ * struct cxl_cmd - CXL memdev command
+ * @memdev: the memory device to which the command is being sent
+ * @query_cmd: structure for the Linux 'Query commands' ioctl
+ * @send_cmd: structure for the Linux 'Send command' ioctl
+ * @input_payload: buffer for any input payload
+ * @output_payload: buffer for any output payload
+ * @refcount: reference for passing command buffer around
+ * @query_status: status from query_commands
+ * @query_status: status from query_commands
+ * @status: command return status from the device
+ */
+struct cxl_cmd {
+	struct cxl_memdev *memdev;
+	struct cxl_mem_query_commands *query_cmd;
+	struct cxl_send_command *send_cmd;
+	void *input_payload;
+	void *output_payload;
+	int refcount;
+	int query_status;
+	int query_idx;
+	int status;
+};
+
 static inline int check_kmod(struct kmod_ctx *kmod_ctx)
 {
 	return kmod_ctx ? 0 : -ENXIO;
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
index c864208..02bc316 100644
--- a/cxl/lib/libcxl.c
+++ b/cxl/lib/libcxl.c
@@ -9,13 +9,16 @@
 #include <unistd.h>
 #include <sys/stat.h>
 #include <sys/types.h>
+#include <sys/ioctl.h>
 #include <sys/sysmacros.h>
 #include <uuid/uuid.h>
 #include <ccan/list/list.h>
 #include <ccan/array_size/array_size.h>
 
+#include <cxl_mem.h>
 #include <util/log.h>
 #include <util/sysfs.h>
+#include <util/bitmap.h>
 #include <cxl/libcxl.h>
 #include "private.h"
 
@@ -316,3 +319,352 @@ CXL_EXPORT unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev)
 {
 	return memdev->ram_size;
 }
+
+CXL_EXPORT void cxl_cmd_unref(struct cxl_cmd *cmd)
+{
+	if (!cmd)
+		return;
+	if (--cmd->refcount == 0) {
+		free(cmd->query_cmd);
+		free(cmd->send_cmd);
+		free(cmd->input_payload);
+		free(cmd->output_payload);
+		free(cmd);
+	}
+}
+
+CXL_EXPORT void cxl_cmd_ref(struct cxl_cmd *cmd)
+{
+	cmd->refcount++;
+}
+
+static int cxl_cmd_alloc_query(struct cxl_cmd *cmd, int num_cmds)
+{
+	struct cxl_mem_query_commands *qcmd;
+	size_t size;
+
+	if (!cmd)
+		return -EINVAL;
+
+	if (cmd->query_cmd != NULL)
+		free(cmd->query_cmd);
+
+	size = sizeof(struct cxl_mem_query_commands) +
+			(num_cmds * sizeof(struct cxl_command_info));
+	cmd->query_cmd = calloc(1, size);
+	if (!cmd->query_cmd)
+		return -ENOMEM;
+
+	qcmd = cmd->query_cmd;
+	qcmd->n_commands = num_cmds;
+
+	return 0;
+}
+
+static struct cxl_cmd *cxl_cmd_new(struct cxl_memdev *memdev)
+{
+	struct cxl_cmd *cmd;
+	size_t size;
+
+	size = sizeof(*cmd);
+	cmd = calloc(1, size);
+	if (!cmd)
+		return NULL;
+
+	cxl_cmd_ref(cmd);
+	cmd->memdev = memdev;
+
+	return cmd;
+}
+
+static int __do_cmd(struct cxl_cmd *cmd, int ioctl_cmd, int fd)
+{
+	struct cxl_memdev *memdev = cmd->memdev;
+	struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+	const char *devname = cxl_memdev_get_devname(memdev);
+	char *cmd_name;
+	void *cmd_buf;
+	int rc;
+
+	switch (ioctl_cmd) {
+	case CXL_MEM_QUERY_COMMANDS:
+		cmd_buf = cmd->query_cmd;
+		cmd_name = "query";
+		break;
+	case CXL_MEM_SEND_COMMAND:
+		cmd_buf = cmd->send_cmd;
+		cmd_name = "send";
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	rc = ioctl(fd, ioctl_cmd, cmd_buf);
+	if (rc < 0)
+		rc = -errno;
+
+	info(ctx, "%s: cmd: %s, status: %s\n", devname,
+		cmd_name, (rc < 0) ? strerror(-rc) : "success");
+
+	return rc;
+}
+
+static int do_cmd(struct cxl_cmd *cmd, int ioctl_cmd)
+{
+	char path[20];
+	struct stat st;
+	unsigned int major, minor;
+	int rc = 0, fd, len = sizeof(path);
+	struct cxl_memdev *memdev = cmd->memdev;
+	struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+	const char *devname = cxl_memdev_get_devname(memdev);
+
+	major = cxl_memdev_get_major(memdev);
+	minor = cxl_memdev_get_minor(memdev);
+
+	if (snprintf(path, len, "/dev/cxl/%s", devname) >= len) {
+		rc = -EINVAL;
+		goto out;
+	}
+
+	fd = open(path, O_RDWR);
+	if (fd < 0) {
+		err(ctx, "failed to open %s: %s\n", path, strerror(errno));
+		rc = -errno;
+		goto out;
+	}
+
+	if (fstat(fd, &st) >= 0 && S_ISCHR(st.st_mode)
+			&& major(st.st_rdev) == major
+			&& minor(st.st_rdev) == minor) {
+		rc = __do_cmd(cmd, ioctl_cmd, fd);
+	} else {
+		err(ctx, "failed to validate %s as a CXL memdev node\n", path);
+		rc = -ENXIO;
+	}
+	close(fd);
+ out:
+	return rc;
+}
+
+static int alloc_do_query(struct cxl_cmd *cmd, int num_cmds)
+{
+	struct cxl_ctx *ctx = cxl_memdev_get_ctx(cmd->memdev);
+	int rc;
+
+	rc = cxl_cmd_alloc_query(cmd, num_cmds);
+	if (rc)
+		return rc;
+
+	rc = do_cmd(cmd, CXL_MEM_QUERY_COMMANDS);
+	if (rc < 0)
+		err(ctx, "%s: query commands failed: %s\n",
+			cxl_memdev_get_devname(cmd->memdev),
+			strerror(-rc));
+	return rc;
+}
+
+static int cxl_cmd_do_query(struct cxl_cmd *cmd)
+{
+	struct cxl_memdev *memdev = cmd->memdev;
+	struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+	const char *devname = cxl_memdev_get_devname(memdev);
+	int rc, n_commands;
+
+	switch (cmd->query_status) {
+	case CXL_CMD_QUERY_OK:
+		return 0;
+	case CXL_CMD_QUERY_UNSUPPORTED:
+		return -EOPNOTSUPP;
+	case CXL_CMD_QUERY_NOT_RUN:
+		break;
+	default:
+		err(ctx, "%s: Unknown query_status %d\n",
+			devname, cmd->query_status);
+		return -EINVAL;
+	}
+
+	rc = alloc_do_query(cmd, 0);
+	if (rc)
+		return rc;
+
+	n_commands = cmd->query_cmd->n_commands;
+	dbg(ctx, "%s: supports %d commands\n", devname, n_commands);
+
+	return alloc_do_query(cmd, n_commands);
+}
+
+static int cxl_cmd_validate(struct cxl_cmd *cmd, u32 cmd_id)
+{
+	struct cxl_memdev *memdev = cmd->memdev;
+	struct cxl_mem_query_commands *query = cmd->query_cmd;
+	const char *devname = cxl_memdev_get_devname(memdev);
+	struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+	u32 i;
+
+	for (i = 0; i < query->n_commands; i++) {
+		struct cxl_command_info *cinfo = &query->commands[i];
+		const char *cmd_name = command_names[cinfo->id].name;
+
+		if (cinfo->id != cmd_id)
+			continue;
+
+		dbg(ctx, "%s: %s: in: %d, out %d, flags: %#08x\n",
+			devname, cmd_name, cinfo->size_in,
+			cinfo->size_out, cinfo->flags);
+
+		if (cinfo->flags & CXL_MEM_COMMAND_FLAG_KERNEL) {
+			err(ctx, "%s: %s: reserved for kernel use\n",
+				devname, cmd_name);
+			return -EPERM;
+		}
+		cmd->query_idx = i;
+		cmd->query_status = CXL_CMD_QUERY_OK;
+		return 0;
+	}
+	cmd->query_status = CXL_CMD_QUERY_UNSUPPORTED;
+	return -EOPNOTSUPP;
+}
+
+CXL_EXPORT int cxl_cmd_attach_payloads(struct cxl_cmd *cmd,
+		void *in, int size_in,
+		void *out, int size_out)
+{
+	struct cxl_memdev *memdev = cmd->memdev;
+
+	if (size_in > memdev->payload_max || size_out > memdev->payload_max)
+		return -EINVAL;
+
+	if (size_in > 0) {
+		cmd->send_cmd->in_payload = (u64)in;
+		cmd->send_cmd->size_in = size_in;
+	}
+	if (size_out > 0) {
+		cmd->send_cmd->out_payload = (u64)out;
+		cmd->send_cmd->size_out = size_out;
+	}
+
+	return 0;
+}
+
+static int cxl_cmd_alloc_send(struct cxl_cmd *cmd, u32 cmd_id)
+{
+	struct cxl_mem_query_commands *query = cmd->query_cmd;
+	struct cxl_command_info *cinfo = &query->commands[cmd->query_idx];
+	size_t size;
+
+	if (!query)
+		return -EINVAL;
+
+	size = sizeof(struct cxl_send_command);
+	cmd->send_cmd = calloc(1, size);
+	if (!cmd->send_cmd)
+		return -ENOMEM;
+
+	if (cinfo->id != cmd_id)
+		return -EINVAL;
+
+	cmd->send_cmd->id = cmd_id;
+
+	if (cinfo->size_in > 0) {
+		cmd->input_payload = calloc(1, cinfo->size_in);
+		if (!cmd->input_payload)
+			return -ENOMEM;
+	}
+	if (cinfo->size_out > 0) {
+		cmd->output_payload = calloc(1, cinfo->size_out);
+		if (!cmd->output_payload)
+			return -ENOMEM;
+	}
+	cxl_cmd_attach_payloads(cmd, cmd->input_payload, cinfo->size_in,
+			cmd->output_payload, cinfo->size_out);
+
+	return 0;
+}
+
+static struct cxl_cmd *cxl_cmd_new_generic(struct cxl_memdev *memdev,
+		u32 cmd_id)
+{
+	const char *devname = cxl_memdev_get_devname(memdev);
+	struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+	struct cxl_cmd *cmd;
+	int rc;
+
+	cmd = cxl_cmd_new(memdev);
+	if (!cmd)
+		return NULL;
+
+	rc = cxl_cmd_do_query(cmd);
+	if (rc) {
+		err(ctx, "%s: query returned: %s\n", devname, strerror(-rc));
+		goto fail;
+	}
+
+	rc = cxl_cmd_validate(cmd, cmd_id);
+	if (rc) {
+		errno = -rc;
+		goto fail;
+	}
+
+	rc = cxl_cmd_alloc_send(cmd, cmd_id);
+	if (rc) {
+		errno = -rc;
+		goto fail;
+	}
+
+	return cmd;
+
+fail:
+	cxl_cmd_unref(cmd);
+	return NULL;
+}
+
+CXL_EXPORT struct cxl_cmd *cxl_cmd_new_identify(struct cxl_memdev *memdev)
+{
+	return cxl_cmd_new_generic(memdev, CXL_MEM_COMMAND_ID_IDENTIFY);
+}
+
+CXL_EXPORT struct cxl_cmd *cxl_cmd_new_raw(struct cxl_memdev *memdev)
+{
+	return cxl_cmd_new_generic(memdev, CXL_MEM_COMMAND_ID_RAW);
+}
+
+CXL_EXPORT struct cxl_cmd *cxl_cmd_new_get_supported_logs(
+		struct cxl_memdev *memdev)
+{
+	return cxl_cmd_new_generic(memdev,
+			CXL_MEM_COMMAND_ID_GET_SUPPORTED_LOGS);
+}
+
+CXL_EXPORT struct cxl_cmd *cxl_cmd_new_get_log(struct cxl_memdev *memdev)
+{
+	return cxl_cmd_new_generic(memdev, CXL_MEM_COMMAND_ID_GET_LOG);
+}
+
+CXL_EXPORT int cxl_cmd_submit(struct cxl_cmd *cmd)
+{
+	struct cxl_memdev *memdev = cmd->memdev;
+	const char *devname = cxl_memdev_get_devname(memdev);
+	struct cxl_ctx *ctx = cxl_memdev_get_ctx(memdev);
+	int rc;
+
+	switch (cmd->query_status) {
+	case CXL_CMD_QUERY_OK:
+		break;
+	case CXL_CMD_QUERY_UNSUPPORTED:
+		return -EOPNOTSUPP;
+	case CXL_CMD_QUERY_NOT_RUN:
+		return -EINVAL;
+	default:
+		err(ctx, "%s: Unknown query_status %d\n",
+			devname, cmd->query_status);
+		return -EINVAL;
+	}
+
+	rc = do_cmd(cmd, CXL_MEM_SEND_COMMAND);
+	if (rc < 0)
+		err(ctx, "%s: send command failed: %s\n",
+			devname, strerror(-rc));
+	cmd->status = cmd->send_cmd->retval;
+	return rc;
+}
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
index 8ee6873..83888bc 100644
--- a/cxl/libcxl.h
+++ b/cxl/libcxl.h
@@ -45,6 +45,17 @@ unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev);
              memdev != NULL; \
              memdev = cxl_memdev_get_next(memdev))
 
+struct cxl_cmd;
+struct cxl_cmd *cxl_cmd_new_identify(struct cxl_memdev *memdev);
+struct cxl_cmd *cxl_cmd_new_raw(struct cxl_memdev *memdev);
+struct cxl_cmd *cxl_cmd_new_get_supported_logs(struct cxl_memdev *memdev);
+struct cxl_cmd *cxl_cmd_new_get_log(struct cxl_memdev *memdev);
+int cxl_cmd_attach_payloads(struct cxl_cmd *cmd,
+		void *in, int size_in, void *out, int size_out);
+void cxl_cmd_ref(struct cxl_cmd *cmd);
+void cxl_cmd_unref(struct cxl_cmd *cmd);
+int cxl_cmd_submit(struct cxl_cmd *cmd);
+
 #ifdef __cplusplus
 } /* extern "C" */
 #endif
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
index 9bbbeb4..3583bab 100644
--- a/cxl/lib/libcxl.sym
+++ b/cxl/lib/libcxl.sym
@@ -24,3 +24,15 @@ global:
 	cxl_memdev_get_pmem_size;
 	cxl_memdev_get_ram_size;
 } LIBCXL_1;
+
+LIBCXL_3 {
+global:
+	cxl_cmd_new_identify;
+	cxl_cmd_new_raw;
+	cxl_cmd_new_get_supported_logs;
+	cxl_cmd_new_get_log;
+	cxl_cmd_attach_payloads;
+	cxl_cmd_ref;
+	cxl_cmd_unref;
+	cxl_cmd_submit;
+} LIBCXL_2;
-- 
2.29.2
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org

^ permalink raw reply related	[flat|nested] 6+ messages in thread
* [ndctl RFC PATCH 4/5] libcxl: add accessors for retrieving 'Identify' information
  2021-01-12  0:33 [ndctl RFC PATCH 0/5] Initial CXL support Vishal Verma
                   ` (2 preceding siblings ...)
  2021-01-12  0:34 ` [ndctl RFC PATCH 3/5] libcxl: add support for command query and submission Vishal Verma
@ 2021-01-12  0:34 ` Vishal Verma
  2021-01-12  0:34 ` [ndctl RFC PATCH 5/5] cxl/list: augment cxl-list with more data from the identify command Vishal Verma
  4 siblings, 0 replies; 6+ messages in thread
From: Vishal Verma @ 2021-01-12  0:34 UTC (permalink / raw)
  To: linux-cxl; +Cc: linux-nvdimm, Ben Widawsky

Add APIs to access an incomplete list of fields from the 'Identify'
mailbox command. The fields added are fw_revision, partition_align, and
lsa_size.

Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
 cxl/lib/private.h  | 19 +++++++++++++++++++
 cxl/lib/libcxl.c   | 44 ++++++++++++++++++++++++++++++++++++++++++++
 cxl/libcxl.h       |  3 +++
 cxl/lib/libcxl.sym |  3 +++
 4 files changed, 69 insertions(+)

diff --git a/cxl/lib/private.h b/cxl/lib/private.h
index c7ebfcb..19f3a37 100644
--- a/cxl/lib/private.h
+++ b/cxl/lib/private.h
@@ -60,6 +60,25 @@ struct cxl_cmd {
 	int status;
 };
 
+#define CXL_CMD_IDENTIFY_FW_REV_LENGTH 0x10
+
+struct cxl_cmd_identify {
+	char fw_revision[CXL_CMD_IDENTIFY_FW_REV_LENGTH];
+	le64 total_capacity;
+	le64 volatile_capacity;
+	le64 persistent_capacity;
+	le64 partition_align;
+	le16 info_event_log_size;
+	le16 warning_event_log_size;
+	le16 failure_event_log_size;
+	le16 fatal_event_log_size;
+	le32 lsa_size;
+	u8 poison_list_max_mer[3];
+	le16 inject_poison_limit;
+	u8 poison_caps;
+	u8 qos_telemetry_caps;
+} __packed;
+
 static inline int check_kmod(struct kmod_ctx *kmod_ctx)
 {
 	return kmod_ctx ? 0 : -ENXIO;
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
index 02bc316..d54da95 100644
--- a/cxl/lib/libcxl.c
+++ b/cxl/lib/libcxl.c
@@ -13,7 +13,10 @@
 #include <sys/sysmacros.h>
 #include <uuid/uuid.h>
 #include <ccan/list/list.h>
+#include <ccan/endian/endian.h>
+#include <ccan/minmax/minmax.h>
 #include <ccan/array_size/array_size.h>
+#include <ccan/short_types/short_types.h>
 
 #include <cxl_mem.h>
 #include <util/log.h>
@@ -624,6 +627,47 @@ CXL_EXPORT struct cxl_cmd *cxl_cmd_new_identify(struct cxl_memdev *memdev)
 	return cxl_cmd_new_generic(memdev, CXL_MEM_COMMAND_ID_IDENTIFY);
 }
 
+CXL_EXPORT int cxl_cmd_identify_get_fw_rev(struct cxl_cmd *cmd, char *fw_rev,
+		int fw_len)
+{
+	struct cxl_cmd_identify *id = (void *)cmd->send_cmd->out_payload;
+
+	if (cmd->send_cmd->id != CXL_MEM_COMMAND_ID_IDENTIFY)
+		return -EINVAL;
+	if (cmd->status < 0)
+		return cmd->status;
+
+	if (fw_len > 0)
+		memcpy(fw_rev, id->fw_revision,
+			min(fw_len, CXL_CMD_IDENTIFY_FW_REV_LENGTH));
+	return 0;
+}
+
+CXL_EXPORT unsigned long long cxl_cmd_identify_get_partition_align(
+		struct cxl_cmd *cmd)
+{
+	struct cxl_cmd_identify *id = (void *)cmd->send_cmd->out_payload;
+
+	if (cmd->send_cmd->id != CXL_MEM_COMMAND_ID_IDENTIFY)
+		return -EINVAL;
+	if (cmd->status < 0)
+		return cmd->status;
+
+	return le64_to_cpu(id->partition_align);
+}
+
+CXL_EXPORT unsigned int cxl_cmd_identify_get_lsa_size(struct cxl_cmd *cmd)
+{
+	struct cxl_cmd_identify *id = (void *)cmd->send_cmd->out_payload;
+
+	if (cmd->send_cmd->id != CXL_MEM_COMMAND_ID_IDENTIFY)
+		return -EINVAL;
+	if (cmd->status < 0)
+		return cmd->status;
+
+	return le32_to_cpu(id->lsa_size);
+}
+
 CXL_EXPORT struct cxl_cmd *cxl_cmd_new_raw(struct cxl_memdev *memdev)
 {
 	return cxl_cmd_new_generic(memdev, CXL_MEM_COMMAND_ID_RAW);
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
index 83888bc..caa2e76 100644
--- a/cxl/libcxl.h
+++ b/cxl/libcxl.h
@@ -55,6 +55,9 @@ int cxl_cmd_attach_payloads(struct cxl_cmd *cmd,
 void cxl_cmd_ref(struct cxl_cmd *cmd);
 void cxl_cmd_unref(struct cxl_cmd *cmd);
 int cxl_cmd_submit(struct cxl_cmd *cmd);
+int cxl_cmd_identify_get_fw_rev(struct cxl_cmd *cmd, char *fw_rev, int fw_len);
+unsigned long long cxl_cmd_identify_get_partition_align(struct cxl_cmd *cmd);
+unsigned int cxl_cmd_identify_get_lsa_size(struct cxl_cmd *cmd);
 
 #ifdef __cplusplus
 } /* extern "C" */
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
index 3583bab..41311d9 100644
--- a/cxl/lib/libcxl.sym
+++ b/cxl/lib/libcxl.sym
@@ -35,4 +35,7 @@ global:
 	cxl_cmd_ref;
 	cxl_cmd_unref;
 	cxl_cmd_submit;
+	cxl_cmd_identify_get_fw_rev;
+	cxl_cmd_identify_get_partition_align;
+	cxl_cmd_identify_get_lsa_size;
 } LIBCXL_2;
-- 
2.29.2
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org

^ permalink raw reply related	[flat|nested] 6+ messages in thread
* [ndctl RFC PATCH 5/5] cxl/list: augment cxl-list with more data from the identify command
  2021-01-12  0:33 [ndctl RFC PATCH 0/5] Initial CXL support Vishal Verma
                   ` (3 preceding siblings ...)
  2021-01-12  0:34 ` [ndctl RFC PATCH 4/5] libcxl: add accessors for retrieving 'Identify' information Vishal Verma
@ 2021-01-12  0:34 ` Vishal Verma
  4 siblings, 0 replies; 6+ messages in thread
From: Vishal Verma @ 2021-01-12  0:34 UTC (permalink / raw)
  To: linux-cxl; +Cc: linux-nvdimm, Ben Widawsky

Augment cxl-list with some more fields obtained from sending an
'Identify' command to the device. If/when these fields are added to the
sysfs representation of the memdev, the command submission detour can be
removed and replaced with data from sysfs.

Cc: Ben Widawsky <ben.widawsky@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
 util/json.h |  3 ++-
 cxl/list.c  | 29 +++++++++++++++++++++++++++--
 util/json.c | 22 +++++++++++++++++++++-
 3 files changed, 50 insertions(+), 4 deletions(-)

diff --git a/util/json.h b/util/json.h
index 91918c8..831f9bb 100644
--- a/util/json.h
+++ b/util/json.h
@@ -6,6 +6,7 @@
 #include <stdbool.h>
 #include <ndctl/libndctl.h>
 #include <daxctl/libdaxctl.h>
+#include <cxl/libcxl.h>
 #include <ccan/short_types/short_types.h>
 
 enum util_json_flags {
@@ -57,5 +58,5 @@ struct json_object *util_dimm_firmware_to_json(struct ndctl_dimm *dimm,
 struct json_object *util_region_capabilities_to_json(struct ndctl_region *region);
 struct cxl_memdev;
 struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
-		unsigned long flags);
+		struct cxl_cmd *id, unsigned long flags);
 #endif /* __NDCTL_JSON_H__ */
diff --git a/cxl/list.c b/cxl/list.c
index 3d44244..7afe2b4 100644
--- a/cxl/list.c
+++ b/cxl/list.c
@@ -5,6 +5,7 @@
 #include <stdlib.h>
 #include <unistd.h>
 #include <limits.h>
+#include <util/log.h>
 #include <util/json.h>
 #include <util/filter.h>
 #include <json-c/json.h>
@@ -16,6 +17,7 @@ static struct {
 	bool memdevs;
 	bool idle;
 	bool human;
+	bool verbose;
 } list;
 
 static unsigned long listopts_to_flags(void)
@@ -47,6 +49,19 @@ static int num_list_flags(void)
 	return list.memdevs;
 }
 
+struct cxl_cmd *memdev_identify(struct cxl_memdev *memdev)
+{
+	struct cxl_cmd *id;
+
+	id = cxl_cmd_new_identify(memdev);
+	if (!id)
+		return NULL;
+
+	if (cxl_cmd_submit(id) != 0)
+		return NULL;
+	return id;
+}
+
 int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx)
 {
 	const struct option options[] = {
@@ -56,7 +71,9 @@ int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx)
 			    "include CXL memory device info"),
 		OPT_BOOLEAN('i', "idle", &list.idle, "include idle devices"),
 		OPT_BOOLEAN('u', "human", &list.human,
-				"use human friendly number formats "),
+				"use human friendly number formats"),
+		OPT_BOOLEAN('v', "verbose", &list.verbose,
+				"enable verbose output"),
 		OPT_END(),
 	};
 	const char * const u[] = {
@@ -80,27 +97,35 @@ int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx)
 
 	list_flags = listopts_to_flags();
 
+	if (list.verbose)
+		cxl_set_log_priority(ctx, LOG_DEBUG);
+
 	cxl_memdev_foreach(ctx, memdev) {
 		struct json_object *jdev = NULL;
+		struct cxl_cmd *id;
 
 		if (!util_cxl_memdev_filter(memdev, param.memdev))
 			continue;
 
 		if (list.memdevs) {
+			id = memdev_identify(memdev);
 			if (!jdevs) {
 				jdevs = json_object_new_array();
 				if (!jdevs) {
 					fail("\n");
+					cxl_cmd_unref(id);
 					continue;
 				}
 			}
 
-			jdev = util_cxl_memdev_to_json(memdev, list_flags);
+			jdev = util_cxl_memdev_to_json(memdev, id, list_flags);
 			if (!jdev) {
 				fail("\n");
+				cxl_cmd_unref(id);
 				continue;
 			}
 			json_object_array_add(jdevs, jdev);
+			cxl_cmd_unref(id);
 		}
 	}
 
diff --git a/util/json.c b/util/json.c
index a855571..24d4477 100644
--- a/util/json.c
+++ b/util/json.c
@@ -1432,10 +1432,11 @@ struct json_object *util_badblock_rec_to_json(u64 block, u64 count,
 }
 
 struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
-		unsigned long flags)
+		struct cxl_cmd *id, unsigned long flags)
 {
 	const char *devname = cxl_memdev_get_devname(memdev);
 	struct json_object *jdev, *jobj;
+	char fw_rev[0x10];
 
 	jdev = json_object_new_object();
 	if (!devname || !jdev)
@@ -1453,5 +1454,24 @@ struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
 	if (jobj)
 		json_object_object_add(jdev, "ram_size", jobj);
 
+	if (!id)
+		return jdev;
+
+	if (cxl_cmd_identify_get_fw_rev(id, fw_rev, 0x10) == 0) {
+		jobj = json_object_new_string(fw_rev);
+		if (jobj)
+			json_object_object_add(jdev, "fw_revision", jobj);
+	}
+
+	jobj = util_json_object_size(cxl_cmd_identify_get_partition_align(id),
+			flags);
+	if (jobj)
+		json_object_object_add(jdev, "partition_align", jobj);
+
+	jobj = util_json_object_size(cxl_cmd_identify_get_lsa_size(id),
+			flags);
+	if (jobj)
+		json_object_object_add(jdev, "lsa_size", jobj);
+
 	return jdev;
 }
-- 
2.29.2
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org

^ permalink raw reply related	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-01-12  0:34 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-12  0:33 [ndctl RFC PATCH 0/5] Initial CXL support Vishal Verma
2021-01-12  0:33 ` [ndctl RFC PATCH 1/5] cxl: add a cxl utility and libcxl library Vishal Verma
2021-01-12  0:34 ` [ndctl RFC PATCH 2/5] cxl: add a local copy of the cxl_mem UAPI header Vishal Verma
2021-01-12  0:34 ` [ndctl RFC PATCH 3/5] libcxl: add support for command query and submission Vishal Verma
2021-01-12  0:34 ` [ndctl RFC PATCH 4/5] libcxl: add accessors for retrieving 'Identify' information Vishal Verma
2021-01-12  0:34 ` [ndctl RFC PATCH 5/5] cxl/list: augment cxl-list with more data from the identify command Vishal Verma

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).