[PATCH 0/6] bcache-tools: store meta data on NVDIMM

[PATCH 0/6] bcache-tools: store meta data on NVDIMM

[Coly Li]

This series is the first effort to support NVDIMM for bcache: store
journal meta data on nvdimm namespace.

With this series, a NVDIMM namespace can be formatted as a bcache
meta device with '-M' option. This meta device can be shared among
multiple cache sets.

Except for adding BCH_FEATURE_INCOMPAT_NVDIMM_META to incompatible
feature set, there is no on-disk layout change for supporting NVDIMM.

A new super block format struct bch_nvm_pages_sb is introduced for
the NVDIMM meta-data device, it might be changed time-to-time before
the EXPERIMENTAL removed from Linux kernel code.

This series is just enough to make things work, more changes will
follow up to make more improvement later.

Coly Li
---

Coly Li (6):
  bcache-tools: add initial data structures for nvm_pages
  bcache-tools: reduce parameters of write_sb()
  bcache-tools: add BCH_FEATURE_INCOMPAT_NVDIMM_META to incompatible
    feature set
  bcache-tools: move super block info display routines into show.c
  bcache-tools: write nvm namespace super block on nvdimm
  bcache-tools: support "bcache show -d" for nvdimm-meta device

 Makefile            |   2 +-
 bcache-super-show.c |  24 ----
 bcache.c            | 289 +--------------------------------------
 bcache.h            |   7 +-
 features.c          |   2 +
 lib.c               | 158 +++++++++++++++++++---
 lib.h               |  28 +++-
 make.c              | 244 +++++++++++++++++++++++++++++----
 nvm_pages.h         | 187 ++++++++++++++++++++++++++
 show.c              | 321 ++++++++++++++++++++++++++++++++++++++++++++
 show.h              |  10 ++
 11 files changed, 917 insertions(+), 355 deletions(-)
 create mode 100644 nvm_pages.h
 create mode 100644 show.c
 create mode 100644 show.h

-- 
2.26.2

[PATCH 1/6] bcache-tools: add initial data structures for nvm_pages

[Coly Li]

This patch initializes the prototype data structures for nvm pages
allocator,

- struct bch_nvm_pages_sb
  This is the super block allocated on each nvmdimm name space. A nvdimm
set may have multiple namespaces, bch_nvm_pages_sb->set_uuid is used to
mark which nvmdimm set this name space belongs to. Normally we will use
the bcache's cache set UUID to initialize this uuid, to connect this
nvdimm set to a specified bcache cache set.

- struct bch_owner_list_head
  This is a table for all heads of all owner lists. A owner list records
which page(s) allocated to which owner. After reboot from power failure,
the ownwer may find all its requested and allocated pages from the owner
list by a handler which is converted by a UUID.

- struct bch_nvm_pages_owner_head
  This is a head of an owner list. Each owner only has one owner list,
and a nvm page only belongs to an specific owner. uuid[] will be set to
owner's uuid, for bcache it is the bcache's cache set uuid. label is not
mandatory, it is a human-readable string for debug purpose. The pointer
*recs references to separated nvm page which hold the table of struct
bch_nvm_pgalloc_rec.

- struct bch_nvm_pgalloc_recs
  This structure occupies a whole page, owner_uuid should match the uuid
in struct bch_nvm_pages_owner_head. recs[] is the real table contains all
allocated records.

- struct bch_nvm_pgalloc_rec
  Each structure records a range of allocated nvm pages. pgoff is offset
in unit of page size of this allocated nvm page range. The adjoint page
ranges of same owner can be merged into a larger one, therefore pages_nr
is NOT always power of 2.

Signed-off-by: Coly Li <colyli@suse.de>
---
 nvm_pages.h | 187 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 187 insertions(+)
 create mode 100644 nvm_pages.h

diff --git a/nvm_pages.h b/nvm_pages.h
new file mode 100644
index 0000000..1ed18a8
--- /dev/null
+++ b/nvm_pages.h
@@ -0,0 +1,187 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _BCACHE_NVM_PAGES_H
+#define _BCACHE_NVM_PAGES_H
+
+/*
+ * - struct bch_nvm_pages_sb
+ *   This is the super block allocated on each nvmdimm name space. A nvdimm
+ * set may have multiple namespaces, bch_nvm_pages_sb->set_uuid is used to
+ * mark which nvmdimm set this name space belongs to. Normally we will use
+ * the bcache's cache set UUID to initialize this uuid, to connect this nvdimm
+ * set to a specified bcache cache set.
+ *
+ * - struct bch_owner_list_head
+ *   This is a table for all heads of all owner lists. A owner list records
+ * which page(s) allocated to which owner. After reboot from power failure,
+ * the ownwer may find all its requested and allocated pages from the owner
+ * list by a handler which is converted by a UUID.
+ *
+ * - struct bch_nvm_pages_owner_head
+ *   This is a head of an owner list. Each owner only has one owner list,
+ * and a nvm page only belongs to an specific owner. uuid[] will be set to
+ * owner's uuid, for bcache it is the bcache's cache set uuid. label is not
+ * mandatory, it is a human-readable string for debug purpose. The pointer
+ * recs references to separated nvm page which hold the table of struct
+ * nvm_pgalloc_rec.
+ *
+ *- struct bch_nvm_pgalloc_recs
+ *  This structure occupies a whole page, owner_uuid should match the uuid
+ * in struct bch_nvm_pages_owner_head. recs[] is the real table contains all
+ * allocated records.
+ *
+ * - struct bch_nvm_pgalloc_rec
+ *   Each structure records a range of allocated nvm pages. pgoff is offset
+ * in unit of page size of this allocated nvm page range. The adjoint page
+ * ranges of same owner can be merged into a larger one, therefore pages_nr
+ * is NOT always power of 2.
+ *
+ *
+ * Memory layout on nvdimm namespace 0
+ *
+ *    0 +----------------------------+
+ *      |                            |
+ *  4KB +----------------------------+
+ *      |   bch_nvme_pages_sb        |
+ *  8KB +----------------------------+ <--- bch_nvme_pages_sb.owner_list_head
+ *      |   bch_owner_list_head      |
+ *      |                            |
+ * 16KB +----------------------------+ <--- bch_owner_list_head.heads[0].recs[0]
+ *      |   bch_nvm_pgalloc_recs     |
+ *      | (nvm pages internal usage) |
+ * 24KB +----------------------------+
+ *      |                            |
+ *      |                            |
+ * 1MB  +----------------------------+
+ *      |    allocable nvm pages     |
+ *      |    for buddy allocator     |
+ * end  +----------------------------+
+ *
+ *
+ *
+ * Memory layout on nvdimm namespace N
+ * (doesn't have owner list)
+ *
+ *    0 +----------------------------+
+ *      |                            |
+ *  4KB +----------------------------+
+ *      |     bch_nvme_pages_sb      |
+ *  8KB +----------------------------+
+ *      |                            |
+ *      |                            |
+ *      |                            |
+ *      |                            |
+ *      |                            |
+ *      |                            |
+ * 1MB  +----------------------------+
+ *      |    allocable nvm pages     |
+ *      |    for buddy allocator     |
+ * end  +----------------------------+
+ *
+
+ */
+
+#include <limits.h>
+#include <linux/types.h>
+
+/* In sectors */
+#define BCH_NVM_PAGES_SB_OFFSET			4096
+#define BCH_NVM_PAGES_OFFSET			(16 << 20)
+
+#define BCH_NVM_PAGES_LABEL_SIZE		32
+#define BCH_NVM_PAGES_NAMESPACES_MAX		8
+
+#define BCH_NVM_PAGES_OWNER_LIST_HEAD_OFFSET	(8<<10)
+#define BCH_NVM_PAGES_SYS_RECS_HEAD_OFFSET	(16<<10)
+
+#define BCH_NVM_PAGES_SB_VERSION		0
+#define BCH_NVM_PAGES_SB_VERSION_MAX		0
+
+static const char bch_nvm_pages_magic[] = {
+	0x17, 0xbd, 0x53, 0x7f, 0x1b, 0x23, 0xd6, 0x83,
+	0x46, 0xa4, 0xf8, 0x28, 0x17, 0xda, 0xec, 0xa9 };
+static const char bch_nvm_pages_pgalloc_magic[] = {
+	0x39, 0x25, 0x3f, 0xf7, 0x27, 0x17, 0xd0, 0xb9,
+	0x10, 0xe6, 0xd2, 0xda, 0x38, 0x68, 0x26, 0xae };
+
+struct bch_nvm_pgalloc_rec {
+	__u32			pgoff;
+	__u32			nr;
+};
+
+struct bch_nvm_pgalloc_recs {
+union {
+	struct {
+		struct bch_nvm_pages_owner_head	*owner;
+		struct bch_nvm_pgalloc_recs	*next;
+		__u8				magic[16];
+		__u8				owner_uuid[16];
+		__u32				size;
+		__u32				used;
+		__u64				_pad[4];
+		struct bch_nvm_pgalloc_rec	recs[];
+	};
+	__u8	pad[8192];
+};
+};
+
+struct bch_nvm_pages_owner_head {
+	__u8				uuid[16];
+	char				label[BCH_NVM_PAGES_LABEL_SIZE];
+	/* Per-namespace own lists */
+	struct bch_nvm_pgalloc_recs	*recs[BCH_NVM_PAGES_NAMESPACES_MAX];
+};
+
+/* heads[0] is always for nvm_pages internal usage */
+struct bch_owner_list_head {
+union {
+	struct {
+		__u32				size;
+		__u32				used;
+		__u64				_pad[4];
+		struct bch_nvm_pages_owner_head	heads[];
+	};
+	__u8	pad[8192];
+};
+};
+
+
+/* The on-media bit order is local CPU order */
+struct bch_nvm_pages_sb {
+	__u64			csum;
+	__u64			ns_start;
+	__u64			sb_offset;
+	__u64			version;
+	__u8			magic[16];
+	__u8			uuid[16];
+	__u32			page_size;
+	__u32			total_namespaces_nr;
+	__u32			this_namespace_nr;
+	union {
+		__u8		set_uuid[16];
+		__u64		set_magic;
+	};
+
+	__u64			flags;
+	__u64			seq;
+
+	__u64			feature_compat;
+	__u64			feature_incompat;
+	__u64			feature_ro_compat;
+
+	/* For allocable nvm pages from buddy systems */
+	__u64			pages_offset;
+	__u64			pages_total;
+
+	__u64			pad[8];
+
+	/* Only on the first name space */
+	struct bch_owner_list_head	*owner_list_head;
+
+	/* Just for csum_set() */
+	__u32			keys;
+	__u64			d[0];
+};
+
+
+#endif
-- 
2.26.2

[PATCH 2/6] bcache-tools: reduce parameters of write_sb()

[Coly Li]

There are 12 parameters in write_sb(), and in future there will be more
paramerter added for new feature. In order to make the code more clear
for adding more parameter, this patch introduces struct sb_context to
hold most of these parameters, and send them into write_sb() via pointer
of struct sb_context.

Signed-off-by: Coly Li <colyli@suse.de>
---
 make.c | 53 +++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 37 insertions(+), 16 deletions(-)

diff --git a/make.c b/make.c
index a3f97f6..e8840eb 100644
--- a/make.c
+++ b/make.c
@@ -37,6 +37,19 @@
 #include "bitwise.h"
 #include "zoned.h"
 
+struct sb_context {
+	unsigned int	block_size;
+	unsigned int	bucket_size;
+	bool		writeback;
+	bool		discard;
+	bool		wipe_bcache;
+	unsigned int	cache_replacement_policy;
+	uint64_t	data_offset;
+	uuid_t		set_uuid;
+	char		*label;
+};
+
+
 #define max(x, y) ({				\
 	typeof(x) _max1 = (x);			\
 	typeof(y) _max2 = (y);			\
@@ -225,18 +238,21 @@ err:
 	return -1;
 }
 
-static void write_sb(char *dev, unsigned int block_size,
-			unsigned int bucket_size,
-			bool writeback, bool discard, bool wipe_bcache,
-			unsigned int cache_replacement_policy,
-			uint64_t data_offset,
-			uuid_t set_uuid, bool bdev, bool force, char *label)
+static void write_sb(char *dev, struct sb_context *sbc, bool bdev, bool force)
 {
 	int fd;
 	char uuid_str[40], set_uuid_str[40], zeroes[SB_START] = {0};
 	struct cache_sb_disk sb_disk;
 	struct cache_sb sb;
 	blkid_probe pr;
+	unsigned int block_size = sbc->block_size;
+	unsigned int bucket_size = sbc->bucket_size;
+	bool wipe_bcache = sbc->wipe_bcache;
+	bool writeback = sbc->writeback;
+	bool discard = sbc->discard;
+	char *label = sbc->label;
+	uint64_t data_offset = sbc->data_offset;
+	unsigned int cache_replacement_policy = sbc->cache_replacement_policy;
 
 	fd = open(dev, O_RDWR|O_EXCL);
 
@@ -338,7 +354,7 @@ static void write_sb(char *dev, unsigned int block_size,
 
 	memcpy(sb.magic, bcache_magic, 16);
 	uuid_generate(sb.uuid);
-	memcpy(sb.set_uuid, set_uuid, sizeof(sb.set_uuid));
+	memcpy(sb.set_uuid, sbc->set_uuid, sizeof(sb.set_uuid));
 
 	sb.block_size	= block_size;
 
@@ -510,6 +526,7 @@ int make_bcache(int argc, char **argv)
 	unsigned int cache_replacement_policy = 0;
 	uint64_t data_offset = BDEV_DATA_START_DEFAULT;
 	uuid_t set_uuid;
+	struct sb_context sbc;
 
 	uuid_generate(set_uuid);
 
@@ -626,20 +643,24 @@ int make_bcache(int argc, char **argv)
 					 get_blocksize(backing_devices[i]));
 	}
 
+	sbc.block_size = block_size;
+	sbc.bucket_size = bucket_size;
+	sbc.writeback = writeback;
+	sbc.discard = discard;
+	sbc.wipe_bcache = wipe_bcache;
+	sbc.cache_replacement_policy = cache_replacement_policy;
+	sbc.data_offset = data_offset;
+	memcpy(sbc.set_uuid, set_uuid, sizeof(sbc.set_uuid));
+	sbc.label = label;
+
 	for (i = 0; i < ncache_devices; i++)
-		write_sb(cache_devices[i], block_size, bucket_size,
-			 writeback, discard, wipe_bcache,
-			 cache_replacement_policy,
-			 data_offset, set_uuid, false, force, label);
+		write_sb(cache_devices[i], &sbc, false, force);
 
 	for (i = 0; i < nbacking_devices; i++) {
 		check_data_offset_for_zoned_device(backing_devices[i],
-						   &data_offset);
+						   &sbc.data_offset);
 
-		write_sb(backing_devices[i], block_size, bucket_size,
-			 writeback, discard, wipe_bcache,
-			 cache_replacement_policy,
-			 data_offset, set_uuid, true, force, label);
+		write_sb(backing_devices[i], &sbc, true, force);
 	}
 
 	return 0;
-- 
2.26.2

[PATCH 3/6] bcache-tools: add BCH_FEATURE_INCOMPAT_NVDIMM_META to incompatible feature set

[Coly Li]

A new incompatible feature bit BCH_FEATURE_INCOMPAT_NVDIMM_META is
added into incompatible feature set. When "-M" or "--mdev" is specified
to store cache device meta data on NVDIMM, this feature bit will be
set on cache device's sb.feature_incompat. Then kernel code will know
the journal and btree nodes are stored on nvdimm meta device.

Signed-off-by: Coly Li <colyli@suse.de>
---
 bcache.h   | 7 ++++++-
 features.c | 2 ++
 make.c     | 6 ++++++
 3 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/bcache.h b/bcache.h
index 46d9683..2ae25ee 100644
--- a/bcache.h
+++ b/bcache.h
@@ -203,7 +203,8 @@ uint64_t crc64(const void *data, size_t len);
 #define BCH_FEATURE_COMPAT_SUPP		0
 #define BCH_FEATURE_RO_COMPAT_SUPP	0
 #define BCH_FEATURE_INCOMPAT_SUPP	(BCH_FEATURE_INCOMPAT_OBSO_LARGE_BUCKET| \
-					 BCH_FEATURE_INCOMPAT_LOG_LARGE_BUCKET_SIZE)
+					 BCH_FEATURE_INCOMPAT_LOG_LARGE_BUCKET_SIZE| \
+					 BCH_FEATURE_INCOMPAT_NVDIMM_META)
 
 #define BCH_HAS_COMPAT_FEATURE(sb, mask) \
 		((sb)->feature_compat & (mask))
@@ -219,6 +220,9 @@ uint64_t crc64(const void *data, size_t len);
 #define BCH_FEATURE_INCOMPAT_OBSO_LARGE_BUCKET		0x0001
 /* real bucket size is (1 << bucket_size) */
 #define BCH_FEATURE_INCOMPAT_LOG_LARGE_BUCKET_SIZE	0x0002
+/* store bcache meta data on nvdimm */
+#define BCH_FEATURE_INCOMPAT_NVDIMM_META		0x0004
+
 
 #define BCH_FEATURE_COMPAT_FUNCS(name, flagname) \
 static inline int bch_has_feature_##name(struct cache_sb *sb) \
@@ -279,5 +283,6 @@ static inline void bch_clear_feature_##name(struct cache_sb *sb) \
 
 BCH_FEATURE_INCOMPAT_FUNCS(obso_large_bucket, OBSO_LARGE_BUCKET);
 BCH_FEATURE_INCOMPAT_FUNCS(large_bucket, LOG_LARGE_BUCKET_SIZE);
+BCH_FEATURE_INCOMPAT_FUNCS(nvdimm_meta, NVDIMM_META);
 
 #endif
diff --git a/features.c b/features.c
index f7f6224..7d9eff6 100644
--- a/features.c
+++ b/features.c
@@ -24,6 +24,8 @@ static struct feature feature_list[] = {
 		"obso_large_bucket"},
 	{BCH_FEATURE_INCOMPAT, BCH_FEATURE_INCOMPAT_LOG_LARGE_BUCKET_SIZE,
 		"large_bucket"},
+	{BCH_FEATURE_INCOMPAT, BCH_FEATURE_INCOMPAT_NVDIMM_META,
+		"nvdimm_meta"},
 	{0, 0, 0 },
 };
 
diff --git a/make.c b/make.c
index e8840eb..5c2c1dd 100644
--- a/make.c
+++ b/make.c
@@ -43,6 +43,7 @@ struct sb_context {
 	bool		writeback;
 	bool		discard;
 	bool		wipe_bcache;
+	bool		nvdimm_meta;
 	unsigned int	cache_replacement_policy;
 	uint64_t	data_offset;
 	uuid_t		set_uuid;
@@ -250,6 +251,7 @@ static void write_sb(char *dev, struct sb_context *sbc, bool bdev, bool force)
 	bool wipe_bcache = sbc->wipe_bcache;
 	bool writeback = sbc->writeback;
 	bool discard = sbc->discard;
+	bool nvdimm_meta = sbc->nvdimm_meta;
 	char *label = sbc->label;
 	uint64_t data_offset = sbc->data_offset;
 	unsigned int cache_replacement_policy = sbc->cache_replacement_policy;
@@ -398,6 +400,9 @@ static void write_sb(char *dev, struct sb_context *sbc, bool bdev, bool force)
 		       data_offset);
 		putchar('\n');
 	} else {
+		if (nvdimm_meta)
+			bch_set_feature_nvdimm_meta(&sb);
+
 		set_bucket_size(&sb, bucket_size);
 
 		sb.nbuckets		= getblocks(fd) / sb.bucket_size;
@@ -652,6 +657,7 @@ int make_bcache(int argc, char **argv)
 	sbc.data_offset = data_offset;
 	memcpy(sbc.set_uuid, set_uuid, sizeof(sbc.set_uuid));
 	sbc.label = label;
+	sbc.nvdimm_meta = (mdev == 1) ? true : false;
 
 	for (i = 0; i < ncache_devices; i++)
 		write_sb(cache_devices[i], &sbc, false, force);
-- 
2.26.2

[PATCH 4/6] bcache-tools: move super block info display routines into show.c

[Coly Li]

This patch moves the following routines from bcache.c into show.c,
- show_bdevs_detail()
- show_bdevs()
- detail_single()

As side effect, in order to fix the calling dependences, these routines
are moved into lib.c,
- accepted_char()
- print_encode()
- free_dev()

Now following changes to show related routines won't make bcache.c more
complicated.

Signed-off-by: Coly Li <colyli@suse.de>
---
 Makefile            |   2 +-
 bcache-super-show.c |  24 ----
 bcache.c            | 278 +-------------------------------------------
 lib.c               |  97 +++++++++++++---
 lib.h               |   3 +
 make.c              |   1 -
 show.c              | 277 +++++++++++++++++++++++++++++++++++++++++++
 show.h              |  10 ++
 8 files changed, 371 insertions(+), 321 deletions(-)
 create mode 100644 show.c
 create mode 100644 show.h

diff --git a/Makefile b/Makefile
index d914200..5496c35 100644
--- a/Makefile
+++ b/Makefile
@@ -39,4 +39,4 @@ bcache-register: bcache-register.o
 bcache: CFLAGS += `pkg-config --cflags blkid uuid`
 bcache: LDLIBS += `pkg-config --libs blkid uuid`
 bcache: CFLAGS += -std=gnu99
-bcache: crc64.o lib.o make.o zoned.o features.o
+bcache: crc64.o lib.o make.o zoned.o features.o show.o
diff --git a/bcache-super-show.c b/bcache-super-show.c
index cc36029..407f418 100644
--- a/bcache-super-show.c
+++ b/bcache-super-show.c
@@ -33,30 +33,6 @@ static void usage()
 	fprintf(stderr, "Usage: bcache-super-show [-f] <device>\n");
 }
 
-
-static bool accepted_char(char c)
-{
-	if ('0' <= c && c <= '9')
-		return true;
-	if ('A' <= c && c <= 'Z')
-		return true;
-	if ('a' <= c && c <= 'z')
-		return true;
-	if (strchr(".-_", c))
-		return true;
-	return false;
-}
-
-static void print_encode(char* in)
-{
-	for (char* pos = in; *pos; pos++)
-		if (accepted_char(*pos))
-			putchar(*pos);
-		else
-			printf("%%%x", *pos);
-}
-
-
 int main(int argc, char **argv)
 {
 	bool force_csum = false;
diff --git a/bcache.c b/bcache.c
index 234702b..5558914 100644
--- a/bcache.c
+++ b/bcache.c
@@ -19,34 +19,10 @@
 #include <assert.h>
 
 #include "features.h"
+#include "show.h"
 
 #define BCACHE_TOOLS_VERSION	"1.1"
 
-//utils function
-static bool accepted_char(char c)
-{
-	if ('0' <= c && c <= '9')
-		return true;
-	if ('A' <= c && c <= 'Z')
-		return true;
-	if ('a' <= c && c <= 'z')
-		return true;
-	if (strchr(".-_", c))
-		return true;
-	return false;
-}
-
-static void print_encode(char *in)
-{
-	char *pos;
-
-	for (pos = in; *pos; pos++)
-		if (accepted_char(*pos))
-			putchar(*pos);
-		else
-			printf("%%%x", *pos);
-}
-
 bool bad_uuid(char *uuid)
 {
 	const char *pattern =
@@ -173,258 +149,6 @@ int version_usagee(void)
 	return EXIT_FAILURE;
 }
 
-void free_dev(struct list_head *head)
-{
-	struct dev *dev, *n;
-
-	list_for_each_entry_safe(dev, n, head, dev_list) {
-		free(dev);
-	}
-}
-
-int show_bdevs_detail(void)
-{
-	struct list_head head;
-	struct dev *devs, *n;
-
-	INIT_LIST_HEAD(&head);
-	int ret;
-
-	ret = list_bdevs(&head);
-	if (ret != 0) {
-		fprintf(stderr, "Failed to list devices\n");
-		return ret;
-	}
-	printf("Name\t\tUuid\t\t\t\t\tCset_Uuid\t\t\t\tType\t\t\tState");
-	printf("\t\t\tBname\t\tAttachToDev\tAttachToCset\n");
-	list_for_each_entry_safe(devs, n, &head, dev_list) {
-		printf("%s\t%s\t%s\t%lu", devs->name, devs->uuid,
-		       devs->cset, devs->version);
-		switch (devs->version) {
-			// These are handled the same by the kernel
-		case BCACHE_SB_VERSION_CDEV:
-		case BCACHE_SB_VERSION_CDEV_WITH_UUID:
-		case BCACHE_SB_VERSION_CDEV_WITH_FEATURES:
-			printf(" (cache)");
-			break;
-			// The second adds data offset supporet
-		case BCACHE_SB_VERSION_BDEV:
-		case BCACHE_SB_VERSION_BDEV_WITH_OFFSET:
-		case BCACHE_SB_VERSION_BDEV_WITH_FEATURES:
-			printf(" (data)");
-			break;
-		default:
-			printf(" (unknown)");
-			break;
-		}
-		printf("\t\t%-16s", devs->state);
-		printf("\t%-16s", devs->bname);
-		char attachdev[30];
-
-		if (strlen(devs->attachuuid) == 36) {
-			cset_to_devname(&head, devs->cset, attachdev);
-		} else if (devs->version == BCACHE_SB_VERSION_CDEV
-			   || devs->version ==
-			   BCACHE_SB_VERSION_CDEV_WITH_UUID) {
-			strcpy(attachdev, BCACHE_NO_SUPPORT);
-		} else {
-			strcpy(attachdev, BCACHE_ATTACH_ALONE);
-		}
-		printf("%-16s", attachdev);
-		printf("%s", devs->attachuuid);
-		putchar('\n');
-	}
-	free_dev(&head);
-	return 0;
-}
-
-
-int show_bdevs(void)
-{
-	struct list_head head;
-	struct dev *devs, *n;
-
-	INIT_LIST_HEAD(&head);
-	int ret;
-
-	ret = list_bdevs(&head);
-	if (ret != 0) {
-		fprintf(stderr, "Failed to list devices\n");
-		return ret;
-	}
-
-	printf("Name\t\tType\t\tState\t\t\tBname\t\tAttachToDev\n");
-	list_for_each_entry_safe(devs, n, &head, dev_list) {
-		printf("%s\t%lu", devs->name, devs->version);
-		switch (devs->version) {
-			// These are handled the same by the kernel
-		case BCACHE_SB_VERSION_CDEV:
-		case BCACHE_SB_VERSION_CDEV_WITH_UUID:
-		case BCACHE_SB_VERSION_CDEV_WITH_FEATURES:
-			printf(" (cache)");
-			break;
-
-			// The second adds data offset supporet
-		case BCACHE_SB_VERSION_BDEV:
-		case BCACHE_SB_VERSION_BDEV_WITH_OFFSET:
-		case BCACHE_SB_VERSION_BDEV_WITH_FEATURES:
-			printf(" (data)");
-			break;
-
-		default:
-			printf(" (unknown)");
-			break;
-		}
-
-		printf("\t%-16s", devs->state);
-		printf("\t%-16s", devs->bname);
-
-		char attachdev[30];
-
-		if (strlen(devs->attachuuid) == 36) {
-			cset_to_devname(&head, devs->cset, attachdev);
-		} else if (devs->version == BCACHE_SB_VERSION_CDEV
-			   || devs->version ==
-			   BCACHE_SB_VERSION_CDEV_WITH_UUID) {
-			strcpy(attachdev, BCACHE_NO_SUPPORT);
-		} else {
-			strcpy(attachdev, BCACHE_ATTACH_ALONE);
-		}
-		printf("%s", attachdev);
-		putchar('\n');
-	}
-	free_dev(&head);
-	return 0;
-}
-
-int detail_single(char *devname)
-{
-	struct bdev bd;
-	struct cdev cd;
-	int type = 1;
-	int ret;
-
-	ret = detail_dev(devname, &bd, &cd, &type);
-	if (ret != 0) {
-		fprintf(stderr, "Failed to detail device\n");
-		return ret;
-	}
-	if (type == BCACHE_SB_VERSION_BDEV ||
-	    type == BCACHE_SB_VERSION_BDEV_WITH_OFFSET ||
-	    type == BCACHE_SB_VERSION_BDEV_WITH_FEATURES) {
-		printf("sb.magic\t\t%s\n", bd.base.magic);
-		printf("sb.first_sector\t\t%" PRIu64 "\n",
-		       bd.base.first_sector);
-		printf("sb.csum\t\t\t%" PRIX64 "\n", bd.base.csum);
-		printf("sb.version\t\t%" PRIu64, bd.base.version);
-		printf(" [backing device]\n");
-		putchar('\n');
-		printf("dev.label\t\t");
-		if (*bd.base.label)
-			print_encode(bd.base.label);
-		else
-			printf("(empty)");
-		putchar('\n');
-		printf("dev.uuid\t\t%s\n", bd.base.uuid);
-		printf("dev.sectors_per_block\t%u\n"
-		       "dev.sectors_per_bucket\t%u\n",
-		       bd.base.sectors_per_block,
-		       bd.base.sectors_per_bucket);
-		printf("dev.data.first_sector\t%u\n"
-		       "dev.data.cache_mode\t%d",
-		       bd.first_sector, bd.cache_mode);
-		switch (bd.cache_mode) {
-		case CACHE_MODE_WRITETHROUGH:
-			printf(" [writethrough]\n");
-			break;
-		case CACHE_MODE_WRITEBACK:
-			printf(" [writeback]\n");
-			break;
-		case CACHE_MODE_WRITEAROUND:
-			printf(" [writearound]\n");
-			break;
-		case CACHE_MODE_NONE:
-			printf(" [no caching]\n");
-			break;
-		default:
-			putchar('\n');
-		}
-		printf("dev.data.cache_state\t%u", bd.cache_state);
-		switch (bd.cache_state) {
-		case BDEV_STATE_NONE:
-			printf(" [detached]\n");
-			break;
-		case BDEV_STATE_CLEAN:
-			printf(" [clean]\n");
-			break;
-		case BDEV_STATE_DIRTY:
-			printf(" [dirty]\n");
-			break;
-		case BDEV_STATE_STALE:
-			printf(" [inconsistent]\n");
-			break;
-		default:
-			putchar('\n');
-		}
-
-		putchar('\n');
-		printf("cset.uuid\t\t%s\n", bd.base.cset);
-	} else if (type == BCACHE_SB_VERSION_CDEV ||
-		   type == BCACHE_SB_VERSION_CDEV_WITH_UUID ||
-		   type == BCACHE_SB_VERSION_CDEV_WITH_FEATURES) {
-		printf("sb.magic\t\t%s\n", cd.base.magic);
-		printf("sb.first_sector\t\t%" PRIu64 "\n",
-		       cd.base.first_sector);
-		printf("sb.csum\t\t\t%" PRIX64 "\n", cd.base.csum);
-		printf("sb.version\t\t%" PRIu64, cd.base.version);
-		printf(" [cache device]\n");
-		print_cache_set_supported_feature_sets(&cd.base.sb);
-		putchar('\n');
-		printf("dev.label\t\t");
-		if (*cd.base.label)
-			print_encode(cd.base.label);
-		else
-			printf("(empty)");
-		putchar('\n');
-		printf("dev.uuid\t\t%s\n", cd.base.uuid);
-		printf("dev.sectors_per_block\t%u\n"
-		       "dev.sectors_per_bucket\t%u\n",
-		       cd.base.sectors_per_block,
-		       cd.base.sectors_per_bucket);
-		printf("dev.cache.first_sector\t%u\n"
-		       "dev.cache.cache_sectors\t%ju\n"
-		       "dev.cache.total_sectors\t%ju\n"
-		       "dev.cache.ordered\t%s\n"
-		       "dev.cache.discard\t%s\n"
-		       "dev.cache.pos\t\t%u\n"
-		       "dev.cache.replacement\t%d",
-		       cd.first_sector,
-		       cd.cache_sectors,
-		       cd.total_sectors,
-		       cd.ordered ? "yes" : "no",
-		       cd.discard ? "yes" : "no", cd.pos, cd.replacement);
-		switch (cd.replacement) {
-		case CACHE_REPLACEMENT_LRU:
-			printf(" [lru]\n");
-			break;
-		case CACHE_REPLACEMENT_FIFO:
-			printf(" [fifo]\n");
-			break;
-		case CACHE_REPLACEMENT_RANDOM:
-			printf(" [random]\n");
-			break;
-		default:
-			putchar('\n');
-		}
-
-		putchar('\n');
-		printf("cset.uuid\t\t%s\n", cd.base.cset);
-	} else {
-		return 1;
-	}
-	return 0;
-}
-
 void replace_line(char **dest, const char *from, const char *to)
 {
 	assert(strlen(from) == strlen(to));
diff --git a/lib.c b/lib.c
index b8487db..05ce9b9 100644
--- a/lib.c
+++ b/lib.c
@@ -84,6 +84,39 @@ bool prefix_with(char *dst, char *prefix)
 	return true;
 }
 
+void free_dev(struct list_head *head)
+{
+	struct dev *dev, *n;
+
+	list_for_each_entry_safe(dev, n, head, dev_list) {
+		free(dev);
+	}
+}
+
+bool accepted_char(char c)
+{
+	if ('0' <= c && c <= '9')
+		return true;
+	if ('A' <= c && c <= 'Z')
+		return true;
+	if ('a' <= c && c <= 'z')
+		return true;
+	if (strchr(".-_", c))
+		return true;
+	return false;
+}
+
+void print_encode(char *in)
+{
+	char *pos;
+
+	for (pos = in; *pos; pos++)
+		if (accepted_char(*pos))
+			putchar(*pos);
+		else
+			printf("%%%x", *pos);
+}
+
 bool part_of_disk(char *devname, char *partname)
 {
 	char pattern[40];
@@ -418,28 +451,17 @@ int list_bdevs(struct list_head *head)
 	return 0;
 }
 
-int detail_dev(char *devname, struct bdev *bd, struct cdev *cd, int *type)
+int __detail_dev(char *devname, struct cache_sb_disk *sb_disk,
+		 struct bdev *bd, struct cdev *cd, int *type)
 {
-	struct cache_sb_disk sb_disk;
 	struct cache_sb sb;
 	uint64_t expected_csum;
-	int fd = open(devname, O_RDONLY);
-
-	if (fd < 0) {
-		fprintf(stderr, "Error: Can't open dev  %s\n", devname);
-		return 1;
-	}
-
-	if (pread(fd, &sb_disk, sizeof(sb_disk), SB_START) != sizeof(sb_disk)) {
-		fprintf(stderr, "Couldn't read\n");
-		goto Fail;
-	}
 
-	to_cache_sb(&sb, &sb_disk);
+	to_cache_sb(&sb, sb_disk);
 
 	if (memcmp(sb.magic, bcache_magic, 16)) {
 		fprintf(stderr,
-			"Bad magic,make sure this is an bcache device\n");
+			"Bad magic, make sure this is an bcache device\n");
 		goto Fail;
 	}
 
@@ -448,8 +470,8 @@ int detail_dev(char *devname, struct bdev *bd, struct cdev *cd, int *type)
 		goto Fail;
 	}
 
-	expected_csum = csum_set(&sb_disk);
-	if (le64_to_cpu(sb_disk.csum) != expected_csum) {
+	expected_csum = csum_set(sb_disk);
+	if (le64_to_cpu(sb_disk->csum) != expected_csum) {
 		fprintf(stderr, "Csum is not match with expected one\n");
 		goto Fail;
 	}
@@ -509,10 +531,49 @@ int detail_dev(char *devname, struct bdev *bd, struct cdev *cd, int *type)
 	}
 	return 0;
 Fail:
-	close(fd);
 	return 1;
 }
 
+int detail_dev(char *devname, struct bdev *bd, struct cdev *cd, int *type)
+{
+	char *buf = NULL;
+	struct cache_sb_disk *sb_disk = NULL;
+	int buf_size = 16<<10, ret = 1;
+	int fd;
+
+	buf = malloc(buf_size);
+	if (buf == NULL) {
+		fprintf(stderr, "Error: fail to allocate read buffer\n");
+		goto out;
+	}
+
+	fd = open(devname, O_RDONLY);
+	if (fd < 0) {
+		fprintf(stderr, "Error: Can't open dev  %s\n", devname);
+		goto out;
+	}
+
+	memset(buf, 0, buf_size);
+	if (pread(fd, buf, buf_size, 0) != buf_size) {
+		fprintf(stderr, "Couldn't read\n");
+		goto out_memory;
+	}
+
+	/* Try whether it is bcache super block */
+	sb_disk = (struct cache_sb_disk *)(buf + SB_START);
+	if (!memcmp(sb_disk->magic, bcache_magic, 16)) {
+		ret = __detail_dev(devname, sb_disk, bd, cd, type);
+		goto out_memory;
+	}
+
+	fprintf(stderr, "Error: Bad magic, not bcache device\n");
+
+out_memory:
+	free(buf);
+out:
+	return ret;
+}
+
 int register_dev(char *devname)
 {
 	int fd;
diff --git a/lib.h b/lib.h
index 0357a11..4c8df97 100644
--- a/lib.h
+++ b/lib.h
@@ -57,6 +57,9 @@ int cset_to_devname(struct list_head *head, char *cset, char *devname);
 struct cache_sb *to_cache_sb(struct cache_sb *sb, struct cache_sb_disk *sb_disk);
 struct cache_sb_disk *to_cache_sb_disk(struct cache_sb_disk *sb_disk,struct cache_sb *sb);
 void set_bucket_size(struct cache_sb *sb, unsigned int bucket_size);
+void free_dev(struct list_head *head);
+void print_encode(char *in);
+bool accepted_char(char c);
 
 #define DEVLEN sizeof(struct dev)
 
diff --git a/make.c b/make.c
index 5c2c1dd..92fe2a2 100644
--- a/make.c
+++ b/make.c
@@ -657,7 +657,6 @@ int make_bcache(int argc, char **argv)
 	sbc.data_offset = data_offset;
 	memcpy(sbc.set_uuid, set_uuid, sizeof(sbc.set_uuid));
 	sbc.label = label;
-	sbc.nvdimm_meta = (mdev == 1) ? true : false;
 
 	for (i = 0; i < ncache_devices; i++)
 		write_sb(cache_devices[i], &sbc, false, force);
diff --git a/show.c b/show.c
new file mode 100644
index 0000000..ff49862
--- /dev/null
+++ b/show.c
@@ -0,0 +1,277 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#define _FILE_OFFSET_BITS       64
+#define __USE_FILE_OFFSET64
+#define _XOPEN_SOURCE 600
+#define _DEFAULT_SOURCE
+
+#include <stdio.h>
+#include <inttypes.h>
+#include <string.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <stdbool.h>
+#include <unistd.h>
+#include <locale.h>
+#include <limits.h>
+#include <assert.h>
+#include <blkid/blkid.h>
+#include <ctype.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <linux/fs.h>
+#include <sys/ioctl.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+#include <uuid/uuid.h>
+
+#include "bcache.h"
+#include "lib.h"
+#include "bitwise.h"
+#include "zoned.h"
+#include "features.h"
+#include "list.h"
+
+int show_bdevs_detail(void)
+{
+	struct list_head head;
+	struct dev *devs, *n;
+
+	INIT_LIST_HEAD(&head);
+	int ret;
+
+	ret = list_bdevs(&head);
+	if (ret != 0) {
+		fprintf(stderr, "Failed to list devices\n");
+		return ret;
+	}
+	printf("Name\t\tUuid\t\t\t\t\tCset_Uuid\t\t\t\tType\t\t\tState");
+	printf("\t\t\tBname\t\tAttachToDev\tAttachToCset\n");
+	list_for_each_entry_safe(devs, n, &head, dev_list) {
+		printf("%s\t%s\t%s\t%lu", devs->name, devs->uuid,
+		       devs->cset, devs->version);
+		switch (devs->version) {
+			// These are handled the same by the kernel
+		case BCACHE_SB_VERSION_CDEV:
+		case BCACHE_SB_VERSION_CDEV_WITH_UUID:
+		case BCACHE_SB_VERSION_CDEV_WITH_FEATURES:
+			printf(" (cache)");
+			break;
+			// The second adds data offset supporet
+		case BCACHE_SB_VERSION_BDEV:
+		case BCACHE_SB_VERSION_BDEV_WITH_OFFSET:
+		case BCACHE_SB_VERSION_BDEV_WITH_FEATURES:
+			printf(" (data)");
+			break;
+		default:
+			printf(" (unknown)");
+			break;
+		}
+		printf("\t\t%-16s", devs->state);
+		printf("\t%-16s", devs->bname);
+		char attachdev[30];
+
+		if (strlen(devs->attachuuid) == 36) {
+			cset_to_devname(&head, devs->cset, attachdev);
+		} else if (devs->version == BCACHE_SB_VERSION_CDEV
+			   || devs->version ==
+			   BCACHE_SB_VERSION_CDEV_WITH_UUID) {
+			strcpy(attachdev, BCACHE_NO_SUPPORT);
+		} else {
+			strcpy(attachdev, BCACHE_ATTACH_ALONE);
+		}
+		printf("%-16s", attachdev);
+		printf("%s", devs->attachuuid);
+		putchar('\n');
+	}
+	free_dev(&head);
+	return 0;
+}
+
+
+int show_bdevs(void)
+{
+	struct list_head head;
+	struct dev *devs, *n;
+
+	INIT_LIST_HEAD(&head);
+	int ret;
+
+	ret = list_bdevs(&head);
+	if (ret != 0) {
+		fprintf(stderr, "Failed to list devices\n");
+		return ret;
+	}
+
+	printf("Name\t\tType\t\tState\t\t\tBname\t\tAttachToDev\n");
+	list_for_each_entry_safe(devs, n, &head, dev_list) {
+		printf("%s\t%lu", devs->name, devs->version);
+		switch (devs->version) {
+			// These are handled the same by the kernel
+		case BCACHE_SB_VERSION_CDEV:
+		case BCACHE_SB_VERSION_CDEV_WITH_UUID:
+		case BCACHE_SB_VERSION_CDEV_WITH_FEATURES:
+			printf(" (cache)");
+			break;
+
+			// The second adds data offset supporet
+		case BCACHE_SB_VERSION_BDEV:
+		case BCACHE_SB_VERSION_BDEV_WITH_OFFSET:
+		case BCACHE_SB_VERSION_BDEV_WITH_FEATURES:
+			printf(" (data)");
+			break;
+
+		default:
+			printf(" (unknown)");
+			break;
+		}
+
+		printf("\t%-16s", devs->state);
+		printf("\t%-16s", devs->bname);
+
+		char attachdev[30];
+
+		if (strlen(devs->attachuuid) == 36) {
+			cset_to_devname(&head, devs->cset, attachdev);
+		} else if (devs->version == BCACHE_SB_VERSION_CDEV
+			   || devs->version ==
+			   BCACHE_SB_VERSION_CDEV_WITH_UUID) {
+			strcpy(attachdev, BCACHE_NO_SUPPORT);
+		} else {
+			strcpy(attachdev, BCACHE_ATTACH_ALONE);
+		}
+		printf("%s", attachdev);
+		putchar('\n');
+	}
+	free_dev(&head);
+	return 0;
+}
+
+int detail_single(char *devname)
+{
+	struct bdev bd;
+	struct cdev cd;
+	int type = 1;
+	int ret;
+
+	ret = detail_dev(devname, &bd, &cd, &type);
+	if (ret != 0) {
+		fprintf(stderr, "Failed to detail device\n");
+		return ret;
+	}
+	if (type == BCACHE_SB_VERSION_BDEV ||
+	    type == BCACHE_SB_VERSION_BDEV_WITH_OFFSET ||
+	    type == BCACHE_SB_VERSION_BDEV_WITH_FEATURES) {
+		printf("sb.magic\t\t%s\n", bd.base.magic);
+		printf("sb.first_sector\t\t%" PRIu64 "\n",
+		       bd.base.first_sector);
+		printf("sb.csum\t\t\t%" PRIX64 "\n", bd.base.csum);
+		printf("sb.version\t\t%" PRIu64, bd.base.version);
+		printf(" [backing device]\n");
+		putchar('\n');
+		printf("dev.label\t\t");
+		if (*bd.base.label)
+			print_encode(bd.base.label);
+		else
+			printf("(empty)");
+		putchar('\n');
+		printf("dev.uuid\t\t%s\n", bd.base.uuid);
+		printf("dev.sectors_per_block\t%u\n"
+		       "dev.sectors_per_bucket\t%u\n",
+		       bd.base.sectors_per_block,
+		       bd.base.sectors_per_bucket);
+		printf("dev.data.first_sector\t%u\n"
+		       "dev.data.cache_mode\t%d",
+		       bd.first_sector, bd.cache_mode);
+		switch (bd.cache_mode) {
+		case CACHE_MODE_WRITETHROUGH:
+			printf(" [writethrough]\n");
+			break;
+		case CACHE_MODE_WRITEBACK:
+			printf(" [writeback]\n");
+			break;
+		case CACHE_MODE_WRITEAROUND:
+			printf(" [writearound]\n");
+			break;
+		case CACHE_MODE_NONE:
+			printf(" [no caching]\n");
+			break;
+		default:
+			putchar('\n');
+		}
+		printf("dev.data.cache_state\t%u", bd.cache_state);
+		switch (bd.cache_state) {
+		case BDEV_STATE_NONE:
+			printf(" [detached]\n");
+			break;
+		case BDEV_STATE_CLEAN:
+			printf(" [clean]\n");
+			break;
+		case BDEV_STATE_DIRTY:
+			printf(" [dirty]\n");
+			break;
+		case BDEV_STATE_STALE:
+			printf(" [inconsistent]\n");
+			break;
+		default:
+			putchar('\n');
+		}
+
+		putchar('\n');
+		printf("cset.uuid\t\t%s\n", bd.base.cset);
+	} else if (type == BCACHE_SB_VERSION_CDEV ||
+		   type == BCACHE_SB_VERSION_CDEV_WITH_UUID ||
+		   type == BCACHE_SB_VERSION_CDEV_WITH_FEATURES) {
+		printf("sb.magic\t\t%s\n", cd.base.magic);
+		printf("sb.first_sector\t\t%" PRIu64 "\n",
+		       cd.base.first_sector);
+		printf("sb.csum\t\t\t%" PRIX64 "\n", cd.base.csum);
+		printf("sb.version\t\t%" PRIu64, cd.base.version);
+		printf(" [cache device]\n");
+		print_cache_set_supported_feature_sets(&cd.base.sb);
+		putchar('\n');
+		printf("dev.label\t\t");
+		if (*cd.base.label)
+			print_encode(cd.base.label);
+		else
+			printf("(empty)");
+		putchar('\n');
+		printf("dev.uuid\t\t%s\n", cd.base.uuid);
+		printf("dev.sectors_per_block\t%u\n"
+		       "dev.sectors_per_bucket\t%u\n",
+		       cd.base.sectors_per_block,
+		       cd.base.sectors_per_bucket);
+		printf("dev.cache.first_sector\t%u\n"
+		       "dev.cache.cache_sectors\t%ju\n"
+		       "dev.cache.total_sectors\t%ju\n"
+		       "dev.cache.ordered\t%s\n"
+		       "dev.cache.discard\t%s\n"
+		       "dev.cache.pos\t\t%u\n"
+		       "dev.cache.replacement\t%d",
+		       cd.first_sector,
+		       cd.cache_sectors,
+		       cd.total_sectors,
+		       cd.ordered ? "yes" : "no",
+		       cd.discard ? "yes" : "no", cd.pos, cd.replacement);
+		switch (cd.replacement) {
+		case CACHE_REPLACEMENT_LRU:
+			printf(" [lru]\n");
+			break;
+		case CACHE_REPLACEMENT_FIFO:
+			printf(" [fifo]\n");
+			break;
+		case CACHE_REPLACEMENT_RANDOM:
+			printf(" [random]\n");
+			break;
+		default:
+			putchar('\n');
+		}
+
+		putchar('\n');
+		printf("cset.uuid\t\t%s\n", cd.base.cset);
+	} else {
+		return 1;
+	}
+	return 0;
+}
diff --git a/show.h b/show.h
new file mode 100644
index 0000000..5ab9933
--- /dev/null
+++ b/show.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _BCH_MAKE_H
+#define _BCH_MAKE_H
+
+int show_bdevs_detail(void);
+int show_bdevs(void);
+int detail_single(char *devname);
+
+#endif
-- 
2.26.2

[PATCH 5/6] bcache-tools: write nvm namespace super block on nvdimm

[Coly Li]

This patch adds following option to "bcache make",
        -M, --mdev              Format a cache nvmdimm-meta device

This option is used to specify a nvdimm device to store bcache mata
data. Once one or more nvdimm devices are specified as nvdimm-meta
device, routine write_nvm_namespace_sb() will write their super block
(defined by struct bch_nvm_pages_sb) into corresponding location on
the nvdimm-meta device.

Signed-off-by: Coly Li <colyli@suse.de>
---
 bcache.c |   1 +
 make.c   | 184 +++++++++++++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 174 insertions(+), 11 deletions(-)

diff --git a/bcache.c b/bcache.c
index 5558914..def1e93 100644
--- a/bcache.c
+++ b/bcache.c
@@ -19,6 +19,7 @@
 #include <assert.h>
 
 #include "features.h"
+#include "nvm_pages.h"
 #include "show.h"
 
 #define BCACHE_TOOLS_VERSION	"1.1"
diff --git a/make.c b/make.c
index 92fe2a2..79ecada 100644
--- a/make.c
+++ b/make.c
@@ -13,6 +13,7 @@
 #define _FILE_OFFSET_BITS	64
 #define __USE_FILE_OFFSET64
 #define _XOPEN_SOURCE 600
+#define _DEFAULT_SOURCE
 
 #include <blkid/blkid.h>
 #include <ctype.h>
@@ -29,6 +30,7 @@
 #include <sys/ioctl.h>
 #include <sys/types.h>
 #include <sys/stat.h>
+#include <sys/mman.h>
 #include <unistd.h>
 #include <uuid/uuid.h>
 
@@ -36,6 +38,7 @@
 #include "lib.h"
 #include "bitwise.h"
 #include "zoned.h"
+#include "nvm_pages.h"
 
 struct sb_context {
 	unsigned int	block_size;
@@ -175,6 +178,7 @@ void usage(void)
 	fprintf(stderr,
 		   "Usage: make-bcache [options] device\n"
 	       "	-C, --cache		Format a cache device\n"
+	       "	-M, --mdev		Format a cache nvmdimm-meta device\n"
 	       "	-B, --bdev		Format a backing device\n"
 	       "	-b, --bucket		bucket size\n"
 	       "	-w, --block		block size (hard sector size of SSD, often 2k)\n"
@@ -409,6 +413,8 @@ static void write_sb(char *dev, struct sb_context *sbc, bool bdev, bool force)
 		sb.nr_in_set		= 1;
 		/* 23 is (SB_SECTOR + SB_SIZE) - 1 sectors */
 		sb.first_bucket		= (23 / sb.bucket_size) + 1;
+		if (nvdimm_meta)
+			sb.first_bucket += SB_JOURNAL_BUCKETS;
 
 		if (sb.nbuckets < 1 << 7) {
 			fprintf(stderr, "Not enough buckets: %llu, need %u\n",
@@ -477,6 +483,139 @@ static void write_sb(char *dev, struct sb_context *sbc, bool bdev, bool force)
 	close(fd);
 }
 
+static void write_nvm_namespace_sb(char *dev,
+				   int this_namespace_nr, int total_namespace_nr,
+				   struct sb_context *sbc, bool force)
+{
+	int fd;
+	struct bch_nvm_pages_sb *nvm_sb = NULL;
+	struct bch_owner_list_head owner_list_head;
+	struct bch_nvm_pages_owner_head system_owner_head;
+	struct bch_nvm_pgalloc_recs system_pgalloc_recs;
+	char uuid_str[40], nvm_pages_set_uuid_str[40];
+	int page_size = getpagesize();
+	void *start_addr = NULL;
+
+	memset(&owner_list_head, 0, sizeof(struct bch_owner_list_head));
+	memset(&system_owner_head, 0, sizeof(struct bch_nvm_pages_owner_head));
+	memset(&system_pgalloc_recs, 0, sizeof(struct bch_nvm_pgalloc_recs));
+
+	fd = open(dev, O_RDWR|O_EXCL);
+	if (fd < 0) {
+		printf("open %s failed: %s\n", dev, strerror(errno));
+		exit(EXIT_FAILURE);
+	}
+
+	start_addr = mmap(NULL, BCH_NVM_PAGES_OFFSET, PROT_READ | PROT_WRITE,
+			  MAP_SHARED, fd, 0);
+	if (start_addr == MAP_FAILED) {
+		printf("mmap to %s filed: %s\n", dev, strerror(errno));
+		exit(EXIT_FAILURE);
+	}
+
+	nvm_sb = (struct bch_nvm_pages_sb *)
+		(start_addr + BCH_NVM_PAGES_SB_OFFSET);
+
+	if ((!memcmp(nvm_sb->magic, bch_nvm_pages_magic, 16)) &&
+	    (!force)) {
+		fprintf(stderr, "Already a nvdimm meta device on %s,", dev);
+		fprintf(stderr, " overwrite with --force\n");
+		exit(EXIT_FAILURE);
+	}
+
+	memset(start_addr, 0, BCH_NVM_PAGES_OFFSET);
+
+	/* Initialize super block */
+	nvm_sb->sb_offset		= BCH_NVM_PAGES_SB_OFFSET;
+	nvm_sb->version			= BCH_NVM_PAGES_SB_VERSION;
+	memcpy(nvm_sb->magic,		bch_nvm_pages_magic, 16);
+	uuid_generate(nvm_sb->uuid);
+	/* Right now there is only one namespace in the nvm_pages set */
+	uuid_generate(nvm_sb->set_uuid);
+	nvm_sb->page_size		= page_size;
+	nvm_sb->total_namespaces_nr	= total_namespace_nr;
+	nvm_sb->this_namespace_nr	= this_namespace_nr;
+	nvm_sb->flags			= 0;
+	nvm_sb->seq			= 0;
+	nvm_sb->feature_compat		= 0;
+	nvm_sb->feature_incompat	= 0;
+	nvm_sb->feature_ro_compat	= 0;
+	nvm_sb->pages_offset		= BCH_NVM_PAGES_OFFSET;
+	nvm_sb->pages_total		= getblocks(fd) * 512 / page_size;
+
+	if (this_namespace_nr == 0)
+		nvm_sb->owner_list_head	= (struct bch_owner_list_head *)
+					BCH_NVM_PAGES_OWNER_LIST_HEAD_OFFSET;
+	else
+		nvm_sb->owner_list_head	= NULL;
+
+	/* Set checksum, don't modify nvm_sb anymore */
+	nvm_sb->csum = csum_set(nvm_sb);
+
+	uuid_unparse(nvm_sb->uuid, uuid_str);
+	uuid_unparse(nvm_sb->set_uuid, nvm_pages_set_uuid_str);
+
+	printf("Name			%s\n", dev);
+	printf("Type			nvdimm-meta\n");
+	printf("UUID:			%s\n"
+	       "NVM Set UUID:		%s\n"
+	       "version:		%u\n"
+	       "seq:			%u\n"
+	       "total_namespaces_nr:	%u\n"
+	       "this_namespace_nr:	%u\n"
+	       "ns_start:		N/A\n"
+	       "page_size:		%u\n"
+	       "pages_offset:		%llu\n"
+	       "pages_total:		%llu\n",
+	       uuid_str, nvm_pages_set_uuid_str,
+	       (unsigned int) nvm_sb->version,
+	       (unsigned int) nvm_sb->seq,
+	       nvm_sb->total_namespaces_nr,
+	       nvm_sb->this_namespace_nr,
+	       nvm_sb->page_size,
+	       nvm_sb->pages_offset,
+	       nvm_sb->pages_total);
+
+	memcpy(start_addr + BCH_NVM_PAGES_SB_OFFSET, nvm_sb,
+	       sizeof(struct bch_nvm_pages_sb));
+
+	/* Initialize bch_owner_list_head */
+	owner_list_head.size = (sizeof(struct bch_owner_list_head) -
+				offsetof(struct bch_owner_list_head, heads)) /
+			       sizeof(struct bch_nvm_pages_owner_head);
+	memcpy(system_owner_head.uuid, nvm_sb->set_uuid, sizeof(uuid_t));
+	snprintf(system_owner_head.label, BCH_NVM_PAGES_LABEL_SIZE - 1,
+		 "nvm_pages_internal");
+	system_owner_head.recs[0] = (struct bch_nvm_pgalloc_recs *)
+				    BCH_NVM_PAGES_SYS_RECS_HEAD_OFFSET;
+	owner_list_head.heads[0] = system_owner_head;
+	owner_list_head.used = 1;
+	memcpy(start_addr + BCH_NVM_PAGES_OWNER_LIST_HEAD_OFFSET,
+		&owner_list_head, sizeof(struct bch_nvm_pages_owner_head));
+
+	/*
+	 * Initialize bch_nvm_pages_owner_head.heads[0].recs[0]
+	 * - the system internal owner list
+	 */
+	system_pgalloc_recs.owner = (struct bch_nvm_pages_owner_head *)
+			(BCH_NVM_PAGES_OWNER_LIST_HEAD_OFFSET +
+			 offsetof(struct bch_owner_list_head, heads));
+	system_pgalloc_recs.next = NULL;
+	memcpy(system_pgalloc_recs.magic, bch_nvm_pages_pgalloc_magic, 16);
+	memcpy(system_pgalloc_recs.owner_uuid, system_owner_head.uuid, sizeof(uuid_t));
+	system_pgalloc_recs.size = (sizeof(struct bch_nvm_pgalloc_recs) -
+				    offsetof(struct bch_nvm_pgalloc_recs, recs)) /
+				   sizeof(struct bch_nvm_pgalloc_rec);
+	system_pgalloc_recs.used = 0;
+	memcpy(start_addr + BCH_NVM_PAGES_SYS_RECS_HEAD_OFFSET,
+	       &system_pgalloc_recs, sizeof(struct bch_nvm_pgalloc_recs));
+
+	msync(start_addr, BCH_NVM_PAGES_OFFSET, MS_SYNC);
+	munmap(start_addr, BCH_NVM_PAGES_OFFSET);
+
+	close(fd);
+}
+
 static unsigned int get_blocksize(const char *path)
 {
 	struct stat statbuf;
@@ -521,9 +660,13 @@ static unsigned int get_blocksize(const char *path)
 
 int make_bcache(int argc, char **argv)
 {
-	int c, bdev = -1;
-	unsigned int i, ncache_devices = 0, nbacking_devices = 0;
+	int c;
+	unsigned int i;
+	int cdev = -1, bdev = -1, mdev = -1;
+	unsigned int ncache_devices = 0, ncache_nvm_devices = 0;
+	unsigned int nbacking_devices = 0;
 	char *cache_devices[argc];
+	char *cache_nvm_devices[argc];
 	char *backing_devices[argc];
 	char label[SB_LABEL_SIZE] = { 0 };
 	unsigned int block_size = 0, bucket_size = 1024;
@@ -538,6 +681,7 @@ int make_bcache(int argc, char **argv)
 	struct option opts[] = {
 		{ "cache",		0, NULL,	'C' },
 		{ "bdev",		0, NULL,	'B' },
+		{ "nvdimm-meta",	0, NULL,	'M'},
 		{ "bucket",		1, NULL,	'b' },
 		{ "block",		1, NULL,	'w' },
 		{ "writeback",		0, &writeback,	1 },
@@ -554,16 +698,19 @@ int make_bcache(int argc, char **argv)
 		{ NULL,			0, NULL,	0 },
 	};
 
-	while ((c = getopt_long(argc, argv,
-				"-hCBUo:w:b:l:",
-				opts, NULL)) != -1)
+	while ((c = getopt_long(argc, argv, "-hCBMUo:w:b:l:",
+				opts, NULL)) != -1) {
+
 		switch (c) {
 		case 'C':
-			bdev = 0;
+			cdev = 1;
 			break;
 		case 'B':
 			bdev = 1;
 			break;
+		case 'M':
+			mdev = 1;
+			break;
 		case 'b':
 			bucket_size =
 				hatoi_validate(optarg, "bucket size", UINT_MAX);
@@ -610,19 +757,28 @@ int make_bcache(int argc, char **argv)
 			usage();
 			break;
 		case 1:
-			if (bdev == -1) {
-				fprintf(stderr, "Please specify -C or -B\n");
+			if (cdev == -1 && bdev == -1 && mdev == -1) {
+				fprintf(stderr, "Please specify -C, -B or -M\n");
 				exit(EXIT_FAILURE);
 			}
 
-			if (bdev)
+			if (bdev > 0) {
 				backing_devices[nbacking_devices++] = optarg;
-			else
+				printf("backing_devices[%d]: %s\n", nbacking_devices - 1, optarg);
+				bdev = -1;
+			} else if (cdev > 0) {
 				cache_devices[ncache_devices++] = optarg;
+				printf("cache_devices[%d]: %s\n", ncache_devices - 1, optarg);
+				cdev = -1;
+			} else if (mdev > 0) {
+				cache_nvm_devices[ncache_nvm_devices++] = optarg;
+				mdev = -1;
+			}
 			break;
 		}
+	} /* while */
 
-	if (!ncache_devices && !nbacking_devices) {
+	if (!ncache_devices && !ncache_nvm_devices && !nbacking_devices) {
 		fprintf(stderr, "Please supply a device\n");
 		usage();
 	}
@@ -657,6 +813,7 @@ int make_bcache(int argc, char **argv)
 	sbc.data_offset = data_offset;
 	memcpy(sbc.set_uuid, set_uuid, sizeof(sbc.set_uuid));
 	sbc.label = label;
+	sbc.nvdimm_meta = (ncache_nvm_devices > 0) ? true : false;
 
 	for (i = 0; i < ncache_devices; i++)
 		write_sb(cache_devices[i], &sbc, false, force);
@@ -668,5 +825,10 @@ int make_bcache(int argc, char **argv)
 		write_sb(backing_devices[i], &sbc, true, force);
 	}
 
+	for (i = 0; i < ncache_nvm_devices; i++) {
+		write_nvm_namespace_sb(cache_nvm_devices[i], i,
+				       ncache_nvm_devices, &sbc,
+				       force);
+	}
 	return 0;
 }
-- 
2.26.2

[PATCH 6/6] bcache-tools: support "bcache show -d" for nvdimm-meta device

[Coly Li]

This patch adds support to display detail information of a nvdimm-meta
device by command "bcache show -d".

struct mdev is added into lib.h to represent nvdimm-meta device
information.

At this moment, commands "bcache show" and "bcache show -m" don't
support nvdimm-meta device yet.

Signed-off-by: Coly Li <colyli@suse.de>
---
 bcache.c |  10 ++---
 lib.c    |  63 ++++++++++++++++++++++++++-
 lib.h    |  25 ++++++++++-
 make.c   |   2 +-
 show.c   | 128 +++++++++++++++++++++++++++++++++++++------------------
 5 files changed, 178 insertions(+), 50 deletions(-)

diff --git a/bcache.c b/bcache.c
index def1e93..044d401 100644
--- a/bcache.c
+++ b/bcache.c
@@ -220,7 +220,7 @@ int attach_both(char *cdev, char *backdev)
 	int ret;
 	char buf[100];
 
-	ret = detail_dev(backdev, &bd, &cd, &type);
+	ret = detail_dev(backdev, &bd, &cd, NULL, &type);
 	if (ret != 0)
 		return ret;
 	if (type != BCACHE_SB_VERSION_BDEV
@@ -235,7 +235,7 @@ int attach_both(char *cdev, char *backdev)
 	}
 
 	if (strlen(cdev) != 36) {
-		ret = detail_dev(cdev, &bd, &cd, &type);
+		ret = detail_dev(cdev, &bd, &cd, NULL, &type);
 		if (type != BCACHE_SB_VERSION_CDEV
 		    && type != BCACHE_SB_VERSION_CDEV_WITH_UUID) {
 			fprintf(stderr, "%s is not an cache device\n", cdev);
@@ -349,7 +349,7 @@ int main(int argc, char **argv)
 		int type = 1;
 		int ret;
 
-		ret = detail_dev(devname, &bd, &cd, &type);
+		ret = detail_dev(devname, &bd, &cd, NULL, &type);
 		if (ret != 0)
 			return ret;
 		if (type == BCACHE_SB_VERSION_BDEV) {
@@ -394,7 +394,7 @@ int main(int argc, char **argv)
 		int type = 1;
 		int ret;
 
-		ret = detail_dev(devname, &bd, &cd, &type);
+		ret = detail_dev(devname, &bd, &cd, NULL, &type);
 		if (ret != 0) {
 			fprintf(stderr,
 			"This device doesn't exist or failed to receive info from this device\n");
@@ -420,7 +420,7 @@ int main(int argc, char **argv)
 		int type = 5;
 		int ret;
 
-		ret = detail_dev(devname, &bd, &cd, &type);
+		ret = detail_dev(devname, &bd, &cd, NULL, &type);
 		if (ret != 0) {
 			fprintf(stderr,
 		"This device doesn't exist or failed to receive info from this device\n");
diff --git a/lib.c b/lib.c
index 05ce9b9..6341c61 100644
--- a/lib.c
+++ b/lib.c
@@ -13,10 +13,14 @@
 #include <string.h>
 #include <malloc.h>
 #include <regex.h>
+#include <libgen.h>
 
 #include "bcache.h"
+#include "nvm_pages.h"
 #include "lib.h"
 #include "bitwise.h"
+
+
 /*
  * utils function
  */
@@ -534,10 +538,59 @@ Fail:
 	return 1;
 }
 
-int detail_dev(char *devname, struct bdev *bd, struct cdev *cd, int *type)
+int __detail_mdev(char *devname, struct bch_nvm_pages_sb *nvm_sb, struct mdev *md)
+{
+	uint64_t expected_csum;
+	int ret = 1;
+
+	if (memcmp(nvm_sb->magic, bch_nvm_pages_magic, 16)) {
+		fprintf(stderr,
+			"Bad magic, make sure this is an bcache nvdimm meta device\n");
+		goto out;
+	}
+
+	if (nvm_sb->sb_offset != BCH_NVM_PAGES_SB_OFFSET) {
+		fprintf(stderr, "Invalid superblock (bad sector)\n");
+		goto out;
+	}
+
+	expected_csum = csum_set(nvm_sb);
+	if (expected_csum != nvm_sb->csum) {
+		fprintf(stderr, "Csum is not match with expected one\n");
+		goto out;
+	}
+
+	memset(md, 0, sizeof(struct mdev));
+
+	md->magic		= "ok";
+	md->csum		= nvm_sb->csum;
+	md->ns_start		= nvm_sb->ns_start;
+	md->sb_offset		= nvm_sb->sb_offset;
+	md->version		= nvm_sb->version;
+	uuid_unparse(nvm_sb->uuid, md->uuid);
+	md->page_size		= nvm_sb->page_size;
+	md->total_namespaces_nr	= nvm_sb->total_namespaces_nr;
+	md->this_namespace_nr	= nvm_sb->this_namespace_nr;
+	uuid_unparse(nvm_sb->set_uuid, md->set_uuid);
+	md->seq			= nvm_sb->seq;
+	strncpy(md->bname,	basename(devname), 40);
+	md->feature_compat	= nvm_sb->feature_compat;
+	md->feature_ro_compat	= nvm_sb->feature_ro_compat;
+	md->feature_incompat	= nvm_sb->feature_incompat;
+	md->pages_offset	= nvm_sb->pages_offset;
+	md->pages_total		= nvm_sb->pages_total;
+
+	ret = 0;
+out:
+	return ret;
+}
+
+int detail_dev(char *devname, struct bdev *bd, struct cdev *cd,
+	       struct mdev *md, int *type)
 {
 	char *buf = NULL;
 	struct cache_sb_disk *sb_disk = NULL;
+	struct bch_nvm_pages_sb *nvm_sb = NULL;
 	int buf_size = 16<<10, ret = 1;
 	int fd;
 
@@ -566,6 +619,14 @@ int detail_dev(char *devname, struct bdev *bd, struct cdev *cd, int *type)
 		goto out_memory;
 	}
 
+	/* Try whether it is nvm_pages super block */
+	nvm_sb = (struct bch_nvm_pages_sb *)(buf + BCH_NVM_PAGES_SB_OFFSET);
+	if (!memcmp(nvm_sb->magic, bch_nvm_pages_magic, 16)) {
+		ret = __detail_mdev(devname, nvm_sb, md);
+		*type = -1;
+		goto out_memory;
+	}
+
 	fprintf(stderr, "Error: Bad magic, not bcache device\n");
 
 out_memory:
diff --git a/lib.h b/lib.h
index 4c8df97..152d9e8 100644
--- a/lib.h
+++ b/lib.h
@@ -43,9 +43,32 @@ struct cdev {
 	unsigned int	replacement;
 };
 
+struct mdev {
+	struct cache_sb		*sb;
+	uint64_t		csum;
+	uint64_t		ns_start;
+	uint64_t		sb_offset;
+	uint64_t		version;
+	char			*magic;
+	char			uuid[40];
+	uint32_t		page_size;
+	uint32_t		total_namespaces_nr;
+	uint32_t		this_namespace_nr;
+	union {
+		char		set_uuid[40];
+		uint64_t	set_magic;
+	};
+	uint64_t		seq;
+	char			bname[40];
+	uint64_t		feature_compat;
+	uint64_t		feature_ro_compat;
+	uint64_t		feature_incompat;
+	uint64_t		pages_offset;
+	uint64_t		pages_total;
+};
 
 int list_bdevs(struct list_head *head);
-int detail_dev(char *devname, struct bdev *bd, struct cdev *cd, int *type);
+int detail_dev(char *devname, struct bdev *bd, struct cdev *cd, struct mdev *md, int *type);
 int register_dev(char *devname);
 int stop_backdev(char *devname);
 int unregister_cset(char *cset);
diff --git a/make.c b/make.c
index 79ecada..447c946 100644
--- a/make.c
+++ b/make.c
@@ -269,7 +269,7 @@ static void write_sb(char *dev, struct sb_context *sbc, bool bdev, bool force)
 			int type = 1;
 			int ret;
 
-			ret = detail_dev(dev, &bd, &cd, &type);
+			ret = detail_dev(dev, &bd, &cd, NULL, &type);
 			if (ret != 0)
 				exit(EXIT_FAILURE);
 			if (type == BCACHE_SB_VERSION_BDEV) {
diff --git a/show.c b/show.c
index ff49862..6175f3f 100644
--- a/show.c
+++ b/show.c
@@ -148,43 +148,33 @@ int show_bdevs(void)
 	return 0;
 }
 
-int detail_single(char *devname)
+int detail_single_dev(char *devname, struct bdev *bd, struct cdev *cd, int type)
 {
-	struct bdev bd;
-	struct cdev cd;
-	int type = 1;
-	int ret;
-
-	ret = detail_dev(devname, &bd, &cd, &type);
-	if (ret != 0) {
-		fprintf(stderr, "Failed to detail device\n");
-		return ret;
-	}
 	if (type == BCACHE_SB_VERSION_BDEV ||
 	    type == BCACHE_SB_VERSION_BDEV_WITH_OFFSET ||
 	    type == BCACHE_SB_VERSION_BDEV_WITH_FEATURES) {
-		printf("sb.magic\t\t%s\n", bd.base.magic);
+		printf("sb.magic\t\t%s\n", bd->base.magic);
 		printf("sb.first_sector\t\t%" PRIu64 "\n",
-		       bd.base.first_sector);
-		printf("sb.csum\t\t\t%" PRIX64 "\n", bd.base.csum);
-		printf("sb.version\t\t%" PRIu64, bd.base.version);
+		       bd->base.first_sector);
+		printf("sb.csum\t\t\t%" PRIX64 "\n", bd->base.csum);
+		printf("sb.version\t\t%" PRIu64, bd->base.version);
 		printf(" [backing device]\n");
 		putchar('\n');
 		printf("dev.label\t\t");
-		if (*bd.base.label)
-			print_encode(bd.base.label);
+		if (*bd->base.label)
+			print_encode(bd->base.label);
 		else
 			printf("(empty)");
 		putchar('\n');
-		printf("dev.uuid\t\t%s\n", bd.base.uuid);
+		printf("dev.uuid\t\t%s\n", bd->base.uuid);
 		printf("dev.sectors_per_block\t%u\n"
 		       "dev.sectors_per_bucket\t%u\n",
-		       bd.base.sectors_per_block,
-		       bd.base.sectors_per_bucket);
+		       bd->base.sectors_per_block,
+		       bd->base.sectors_per_bucket);
 		printf("dev.data.first_sector\t%u\n"
 		       "dev.data.cache_mode\t%d",
-		       bd.first_sector, bd.cache_mode);
-		switch (bd.cache_mode) {
+		       bd->first_sector, bd->cache_mode);
+		switch (bd->cache_mode) {
 		case CACHE_MODE_WRITETHROUGH:
 			printf(" [writethrough]\n");
 			break;
@@ -200,8 +190,8 @@ int detail_single(char *devname)
 		default:
 			putchar('\n');
 		}
-		printf("dev.data.cache_state\t%u", bd.cache_state);
-		switch (bd.cache_state) {
+		printf("dev.data.cache_state\t%u", bd->cache_state);
+		switch (bd->cache_state) {
 		case BDEV_STATE_NONE:
 			printf(" [detached]\n");
 			break;
@@ -219,29 +209,29 @@ int detail_single(char *devname)
 		}
 
 		putchar('\n');
-		printf("cset.uuid\t\t%s\n", bd.base.cset);
+		printf("cset.uuid\t\t%s\n", bd->base.cset);
 	} else if (type == BCACHE_SB_VERSION_CDEV ||
 		   type == BCACHE_SB_VERSION_CDEV_WITH_UUID ||
 		   type == BCACHE_SB_VERSION_CDEV_WITH_FEATURES) {
-		printf("sb.magic\t\t%s\n", cd.base.magic);
+		printf("sb.magic\t\t%s\n", cd->base.magic);
 		printf("sb.first_sector\t\t%" PRIu64 "\n",
-		       cd.base.first_sector);
-		printf("sb.csum\t\t\t%" PRIX64 "\n", cd.base.csum);
-		printf("sb.version\t\t%" PRIu64, cd.base.version);
+		       cd->base.first_sector);
+		printf("sb.csum\t\t\t%" PRIX64 "\n", cd->base.csum);
+		printf("sb.version\t\t%" PRIu64, cd->base.version);
 		printf(" [cache device]\n");
-		print_cache_set_supported_feature_sets(&cd.base.sb);
+		print_cache_set_supported_feature_sets(&cd->base.sb);
 		putchar('\n');
 		printf("dev.label\t\t");
-		if (*cd.base.label)
-			print_encode(cd.base.label);
+		if (*cd->base.label)
+			print_encode(cd->base.label);
 		else
 			printf("(empty)");
 		putchar('\n');
-		printf("dev.uuid\t\t%s\n", cd.base.uuid);
+		printf("dev.uuid\t\t%s\n", cd->base.uuid);
 		printf("dev.sectors_per_block\t%u\n"
 		       "dev.sectors_per_bucket\t%u\n",
-		       cd.base.sectors_per_block,
-		       cd.base.sectors_per_bucket);
+		       cd->base.sectors_per_block,
+		       cd->base.sectors_per_bucket);
 		printf("dev.cache.first_sector\t%u\n"
 		       "dev.cache.cache_sectors\t%ju\n"
 		       "dev.cache.total_sectors\t%ju\n"
@@ -249,12 +239,12 @@ int detail_single(char *devname)
 		       "dev.cache.discard\t%s\n"
 		       "dev.cache.pos\t\t%u\n"
 		       "dev.cache.replacement\t%d",
-		       cd.first_sector,
-		       cd.cache_sectors,
-		       cd.total_sectors,
-		       cd.ordered ? "yes" : "no",
-		       cd.discard ? "yes" : "no", cd.pos, cd.replacement);
-		switch (cd.replacement) {
+		       cd->first_sector,
+		       cd->cache_sectors,
+		       cd->total_sectors,
+		       cd->ordered ? "yes" : "no",
+		       cd->discard ? "yes" : "no", cd->pos, cd->replacement);
+		switch (cd->replacement) {
 		case CACHE_REPLACEMENT_LRU:
 			printf(" [lru]\n");
 			break;
@@ -269,9 +259,63 @@ int detail_single(char *devname)
 		}
 
 		putchar('\n');
-		printf("cset.uuid\t\t%s\n", cd.base.cset);
+		printf("cset.uuid\t\t%s\n", cd->base.cset);
 	} else {
 		return 1;
 	}
 	return 0;
 }
+
+int detail_single_mdev(char *devname, struct mdev *md)
+{
+	printf(	"sb.magic\t\t%s\n"
+		"sb.csum\t\t\t%" PRIX64 "\n"
+		"sb.ns_start\t\t0x%" PRIX64 "\n"
+		"sb.sb_offset\t\t0x%" PRIX64 "\n"
+		"sb.version\t\t%" PRIu64 " [nvdimm-meta device]\n"
+		"sb.uuid\t\t\t%s\n"
+		"sb.page_size\t\t%u\n"
+		"sb.total_namespaces_nr\t%u\n"
+		"sb.this_namespace_nr\t%u\n"
+		"sb.set_uuid\t\t%s\n"
+		"sb.seq\t\t\t%" PRIX64 "\n"
+		"sb.pages_offset\t\t0x%" PRIX64 "\n"
+		"sb.pages_total\t\t%" PRIu64 "\n",
+		md->magic,
+		md->csum,
+		md->ns_start,
+		md->sb_offset,
+		md->version,
+		md->uuid,
+		md->page_size,
+		md->total_namespaces_nr,
+		md->this_namespace_nr,
+		md->set_uuid,
+		md->seq,
+		md->pages_offset,
+		md->pages_total);
+	putchar('\n');
+
+	return 0;
+}
+
+int detail_single(char *devname)
+{
+	struct bdev bd;
+	struct cdev cd;
+	struct mdev md;
+	int type = 0;
+	int ret = 0;
+
+	ret = detail_dev(devname, &bd, &cd, &md, &type);
+	if (ret != 0)
+		goto out;
+
+	if (type >= 0 && type <= BCACHE_SB_MAX_VERSION)
+		ret = detail_single_dev(devname, &bd, &cd, type);
+	else
+		ret = detail_single_mdev(devname, &md);
+
+out:
+	return ret;
+}
-- 
2.26.2