[PATCH v3 0/3] block: show crypto capabilities in sysfs

[PATCH v3 0/3] block: show crypto capabilities in sysfs

[Eric Biggers]

This series adds sysfs files that expose the inline encryption
capabilities of request queues.

Patches 1 and 2 are some related cleanups for existing blk-sysfs code.
Patch 3 is the real change; see there for more details.

This is based on top of my other patch series
"[PATCH v2 0/8] docs: consolidate sysfs-block into Documentation/ABI/"
(https://lore.kernel.org/r/20211208005640.102814-1-ebiggers@kernel.org).

Changed v2 => v3:
   - Moved the documentation into Documentation/ABI/stable/sysfs-block,
     and improved it a bit.
   - Write "/sys/block/" instead of "/sys/class/block/".
   - Added Reviewed-by tags.

Changed v1 => v2:
   - Use sysfs_emit() instead of sprintf().
   - Use __ATTR_RO().

Eric Biggers (3):
  block: simplify calling convention of elv_unregister_queue()
  block: don't delete queue kobject before its children
  blk-crypto: show crypto capabilities in sysfs

 Documentation/ABI/stable/sysfs-block |  49 ++++++++
 block/Makefile                       |   3 +-
 block/blk-crypto-internal.h          |  12 ++
 block/blk-crypto-sysfs.c             | 172 +++++++++++++++++++++++++++
 block/blk-crypto.c                   |   3 +
 block/blk-sysfs.c                    |  17 ++-
 block/elevator.c                     |   8 +-
 include/linux/blkdev.h               |   1 +
 8 files changed, 255 insertions(+), 10 deletions(-)
 create mode 100644 block/blk-crypto-sysfs.c

-- 
2.34.1

[PATCH v3 1/3] block: simplify calling convention of elv_unregister_queue()

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

Make elv_unregister_queue() a no-op if q->elevator is NULL or is not
registered.

This simplifies the existing callers, as well as the future caller in
the error path of blk_register_queue().

Also don't bother checking whether q is NULL, since it never is.

Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 block/blk-sysfs.c | 3 +--
 block/elevator.c  | 8 ++++----
 2 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
index 3e6357321225f..3152d244e9b36 100644
--- a/block/blk-sysfs.c
+++ b/block/blk-sysfs.c
@@ -958,8 +958,7 @@ void blk_unregister_queue(struct gendisk *disk)
 	blk_trace_remove_sysfs(disk_to_dev(disk));
 
 	mutex_lock(&q->sysfs_lock);
-	if (q->elevator)
-		elv_unregister_queue(q);
+	elv_unregister_queue(q);
 	disk_unregister_independent_access_ranges(disk);
 	mutex_unlock(&q->sysfs_lock);
 	mutex_unlock(&q->sysfs_dir_lock);
diff --git a/block/elevator.c b/block/elevator.c
index ec98aed39c4f5..b062c5bc10b9a 100644
--- a/block/elevator.c
+++ b/block/elevator.c
@@ -516,9 +516,11 @@ int elv_register_queue(struct request_queue *q, bool uevent)
 
 void elv_unregister_queue(struct request_queue *q)
 {
+	struct elevator_queue *e = q->elevator;
+
 	lockdep_assert_held(&q->sysfs_lock);
 
-	if (q) {
+	if (e && e->registered) {
 		struct elevator_queue *e = q->elevator;
 
 		kobject_uevent(&e->kobj, KOBJ_REMOVE);
@@ -593,9 +595,7 @@ int elevator_switch_mq(struct request_queue *q,
 	lockdep_assert_held(&q->sysfs_lock);
 
 	if (q->elevator) {
-		if (q->elevator->registered)
-			elv_unregister_queue(q);
-
+		elv_unregister_queue(q);
 		ioc_clear_queue(q);
 		blk_mq_sched_free_rqs(q);
 		elevator_exit(q);
-- 
2.34.1

Re: [PATCH v3 1/3] block: simplify calling convention of elv_unregister_queue()

[Bart Van Assche]

On 12/7/21 5:35 PM, Eric Biggers wrote:
> From: Eric Biggers <ebiggers@google.com>
> 
> Make elv_unregister_queue() a no-op if q->elevator is NULL or is not
> registered.
> 
> This simplifies the existing callers, as well as the future caller in
> the error path of blk_register_queue().
> 
> Also don't bother checking whether q is NULL, since it never is.

Reviewed-by: Bart Van Assche <bvanassche@acm.org>

[PATCH v3 2/3] block: don't delete queue kobject before its children

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

kobjects aren't supposed to be deleted before their child kobjects are
deleted.  Apparently this is usually benign; however, a WARN will be
triggered if one of the child kobjects has a named attribute group:

    sysfs group 'modes' not found for kobject 'crypto'
    WARNING: CPU: 0 PID: 1 at fs/sysfs/group.c:278 sysfs_remove_group+0x72/0x80
    ...
    Call Trace:
      sysfs_remove_groups+0x29/0x40 fs/sysfs/group.c:312
      __kobject_del+0x20/0x80 lib/kobject.c:611
      kobject_cleanup+0xa4/0x140 lib/kobject.c:696
      kobject_release lib/kobject.c:736 [inline]
      kref_put include/linux/kref.h:65 [inline]
      kobject_put+0x53/0x70 lib/kobject.c:753
      blk_crypto_sysfs_unregister+0x10/0x20 block/blk-crypto-sysfs.c:159
      blk_unregister_queue+0xb0/0x110 block/blk-sysfs.c:962
      del_gendisk+0x117/0x250 block/genhd.c:610

Fix this by moving the kobject_del() and the corresponding
kobject_uevent() to the correct place.

Fixes: 2c2086afc2b8 ("block: Protect less code with sysfs_lock in blk_{un,}register_queue()")
Reviewed-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 block/blk-sysfs.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
index 3152d244e9b36..c11242ef88558 100644
--- a/block/blk-sysfs.c
+++ b/block/blk-sysfs.c
@@ -952,15 +952,17 @@ void blk_unregister_queue(struct gendisk *disk)
 	 */
 	if (queue_is_mq(q))
 		blk_mq_unregister_dev(disk_to_dev(disk), q);
-
-	kobject_uevent(&q->kobj, KOBJ_REMOVE);
-	kobject_del(&q->kobj);
 	blk_trace_remove_sysfs(disk_to_dev(disk));
 
 	mutex_lock(&q->sysfs_lock);
 	elv_unregister_queue(q);
 	disk_unregister_independent_access_ranges(disk);
 	mutex_unlock(&q->sysfs_lock);
+
+	/* Now that all child objects were deleted, the queue can be deleted. */
+	kobject_uevent(&q->kobj, KOBJ_REMOVE);
+	kobject_del(&q->kobj);
+
 	mutex_unlock(&q->sysfs_dir_lock);
 
 	kobject_put(&disk_to_dev(disk)->kobj);
-- 
2.34.1

Re: [PATCH v3 2/3] block: don't delete queue kobject before its children

[Bart Van Assche]

On 12/7/21 5:35 PM, Eric Biggers wrote:
> +	/* Now that all child objects were deleted, the queue can be deleted. */

Shouldn't the present tense be used above (were -> are)? Anyway:

Reviewed-by: Bart Van Assche <bvanassche@acm.org>

Re: [PATCH v3 2/3] block: don't delete queue kobject before its children

[Eric Biggers]

On Thu, Dec 09, 2021 at 02:38:02PM -0800, Bart Van Assche wrote:
> On 12/7/21 5:35 PM, Eric Biggers wrote:
> > +	/* Now that all child objects were deleted, the queue can be deleted. */
> 
> Shouldn't the present tense be used above (were -> are)? Anyway:
> 
> Reviewed-by: Bart Van Assche <bvanassche@acm.org>

"deleted" is an action here, not a state.  I think it's fine as-is, but maybe
you would prefer the following?

	/* Now that we've deleted all child objects, we can delete the queue. */

- Eric

Re: [PATCH v3 2/3] block: don't delete queue kobject before its children

[Bart Van Assche]

On 12/9/21 3:17 PM, Eric Biggers wrote:
> On Thu, Dec 09, 2021 at 02:38:02PM -0800, Bart Van Assche wrote:
>> On 12/7/21 5:35 PM, Eric Biggers wrote:
>>> +	/* Now that all child objects were deleted, the queue can be deleted. */
>>
>> Shouldn't the present tense be used above (were -> are)? Anyway:
>>
>> Reviewed-by: Bart Van Assche <bvanassche@acm.org>
> 
> "deleted" is an action here, not a state.  I think it's fine as-is, but maybe
> you would prefer the following?
> 
> 	/* Now that we've deleted all child objects, we can delete the queue. */

Both alternatives work for me.

Thanks,

Bart.

[PATCH v3 3/3] blk-crypto: show crypto capabilities in sysfs

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

Add sysfs files that expose the inline encryption capabilities of
request queues:

	/sys/block/$disk/queue/crypto/max_dun_bits
	/sys/block/$disk/queue/crypto/modes/$mode
	/sys/block/$disk/queue/crypto/num_keyslots

Userspace can use these new files to decide what encryption settings to
use, or whether to use inline encryption at all.  This also brings the
crypto capabilities in line with the other queue properties, which are
already discoverable via the queue directory in sysfs.

Design notes:

  - Place the new files in a new subdirectory "crypto" to group them
    together and to avoid complicating the main "queue" directory.  This
    also makes it possible to replace "crypto" with a symlink later if
    we ever make the blk_crypto_profiles into real kobjects (see below).

  - It was necessary to define a new kobject that corresponds to the
    crypto subdirectory.  For now, this kobject just contains a pointer
    to the blk_crypto_profile.  Note that multiple queues (and hence
    multiple such kobjects) may refer to the same blk_crypto_profile.

    An alternative design would more closely match the current kernel
    data structures: the blk_crypto_profile could be a kobject itself,
    located directly under the host controller device's kobject, while
    /sys/block/$disk/queue/crypto would be a symlink to it.

    I decided not to do that for now because it would require a lot more
    changes, such as no longer embedding blk_crypto_profile in other
    structures, and also because I'm not sure we can rule out moving the
    crypto capabilities into 'struct queue_limits' in the future.  (Even
    if multiple queues share the same crypto engine, maybe the supported
    data unit sizes could differ due to other queue properties.)  It
    would also still be possible to switch to that design later without
    breaking userspace, by replacing the directory with a symlink.

  - Use "max_dun_bits" instead of "max_dun_bytes".  Currently, the
    kernel internally stores this value in bytes, but that's an
    implementation detail.  It probably makes more sense to talk about
    this value in bits, and choosing bits is more future-proof.

  - "modes" is a sub-subdirectory, since there may be multiple supported
    crypto modes, sysfs is supposed to have one value per file, and it
    makes sense to group all the mode files together.

  - Each mode had to be named.  The crypto API names like "xts(aes)" are
    not appropriate because they don't specify the key size.  Therefore,
    I assigned new names.  The exact names chosen are arbitrary, but
    they happen to match the names used in log messages in fs/crypto/.

  - The "num_keyslots" file is a bit different from the others in that
    it is only useful to know for performance reasons.  However, it's
    included as it can still be useful.  For example, a user might not
    want to use inline encryption if there aren't very many keyslots.

Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 Documentation/ABI/stable/sysfs-block |  49 ++++++++
 block/Makefile                       |   3 +-
 block/blk-crypto-internal.h          |  12 ++
 block/blk-crypto-sysfs.c             | 172 +++++++++++++++++++++++++++
 block/blk-crypto.c                   |   3 +
 block/blk-sysfs.c                    |   6 +
 include/linux/blkdev.h               |   1 +
 7 files changed, 245 insertions(+), 1 deletion(-)
 create mode 100644 block/blk-crypto-sysfs.c

diff --git a/Documentation/ABI/stable/sysfs-block b/Documentation/ABI/stable/sysfs-block
index e988742a54a4c..b97a8cf7c3ad0 100644
--- a/Documentation/ABI/stable/sysfs-block
+++ b/Documentation/ABI/stable/sysfs-block
@@ -155,6 +155,55 @@ Description:
 		last zone of the device which may be smaller.
 
 
+What:		/sys/block/<disk>/queue/crypto/
+Date:		December 2021
+Contact:	linux-block@vger.kernel.org
+Description:
+		The presence of this subdirectory of /sys/block/<disk>/queue/
+		indicates that the device supports inline encryption.  This
+		subdirectory contains files which describe the inline encryption
+		capabilities of the device.  For more information about inline
+		encryption, refer to Documentation/block/inline-encryption.rst.
+
+
+What:		/sys/block/<disk>/queue/crypto/max_dun_bits
+Date:		December 2021
+Contact:	linux-block@vger.kernel.org
+Description:
+		[RO] This file shows the maximum length, in bits, of data unit
+		numbers accepted by the device in inline encryption requests.
+
+
+What:		/sys/block/<disk>/queue/crypto/modes/<mode>
+Date:		December 2021
+Contact:	linux-block@vger.kernel.org
+Description:
+		[RO] For each crypto mode (i.e., encryption/decryption
+		algorithm) the device supports with inline encryption, a file
+		will exist at this location.  It will contain a hexadecimal
+		number that is a bitmask of the supported data unit sizes, in
+		bytes, for that crypto mode.
+
+		Currently, the crypto modes that may be supported are:
+
+		   * AES-256-XTS
+		   * AES-128-CBC-ESSIV
+		   * Adiantum
+
+		For example, if a device supports AES-256-XTS inline encryption
+		with data unit sizes of 512 and 4096 bytes, the file
+		/sys/block/<disk>/queue/crypto/modes/AES-256-XTS will exist and
+		will contain "0x1200".
+
+
+What:		/sys/block/<disk>/queue/crypto/num_keyslots
+Date:		December 2021
+Contact:	linux-block@vger.kernel.org
+Description:
+		[RO] This file shows the number of keyslots the device has for
+		use with inline encryption.
+
+
 What:		/sys/block/<disk>/queue/dax
 Date:		June 2016
 Contact:	linux-block@vger.kernel.org
diff --git a/block/Makefile b/block/Makefile
index f38eaa6129296..3950ecbc5c263 100644
--- a/block/Makefile
+++ b/block/Makefile
@@ -36,6 +36,7 @@ obj-$(CONFIG_BLK_DEBUG_FS)	+= blk-mq-debugfs.o
 obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o
 obj-$(CONFIG_BLK_SED_OPAL)	+= sed-opal.o
 obj-$(CONFIG_BLK_PM)		+= blk-pm.o
-obj-$(CONFIG_BLK_INLINE_ENCRYPTION)	+= blk-crypto.o blk-crypto-profile.o
+obj-$(CONFIG_BLK_INLINE_ENCRYPTION)	+= blk-crypto.o blk-crypto-profile.o \
+					   blk-crypto-sysfs.o
 obj-$(CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK)	+= blk-crypto-fallback.o
 obj-$(CONFIG_BLOCK_HOLDER_DEPRECATED)	+= holder.o
diff --git a/block/blk-crypto-internal.h b/block/blk-crypto-internal.h
index 2fb0d65a464ca..e6818ffaddbf8 100644
--- a/block/blk-crypto-internal.h
+++ b/block/blk-crypto-internal.h
@@ -11,6 +11,7 @@
 
 /* Represents a crypto mode supported by blk-crypto  */
 struct blk_crypto_mode {
+	const char *name; /* name of this mode, shown in sysfs */
 	const char *cipher_str; /* crypto API name (for fallback case) */
 	unsigned int keysize; /* key size in bytes */
 	unsigned int ivsize; /* iv size in bytes */
@@ -20,6 +21,10 @@ extern const struct blk_crypto_mode blk_crypto_modes[];
 
 #ifdef CONFIG_BLK_INLINE_ENCRYPTION
 
+int blk_crypto_sysfs_register(struct request_queue *q);
+
+void blk_crypto_sysfs_unregister(struct request_queue *q);
+
 void bio_crypt_dun_increment(u64 dun[BLK_CRYPTO_DUN_ARRAY_SIZE],
 			     unsigned int inc);
 
@@ -62,6 +67,13 @@ static inline bool blk_crypto_rq_is_encrypted(struct request *rq)
 
 #else /* CONFIG_BLK_INLINE_ENCRYPTION */
 
+static inline int blk_crypto_sysfs_register(struct request_queue *q)
+{
+	return 0;
+}
+
+static inline void blk_crypto_sysfs_unregister(struct request_queue *q) { }
+
 static inline bool bio_crypt_rq_ctx_compatible(struct request *rq,
 					       struct bio *bio)
 {
diff --git a/block/blk-crypto-sysfs.c b/block/blk-crypto-sysfs.c
new file mode 100644
index 0000000000000..fd93bd2f33b75
--- /dev/null
+++ b/block/blk-crypto-sysfs.c
@@ -0,0 +1,172 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2021 Google LLC
+ *
+ * sysfs support for blk-crypto.  This file contains the code which exports the
+ * crypto capabilities of devices via /sys/block/$disk/queue/crypto/.
+ */
+
+#include <linux/blk-crypto-profile.h>
+
+#include "blk-crypto-internal.h"
+
+struct blk_crypto_kobj {
+	struct kobject kobj;
+	struct blk_crypto_profile *profile;
+};
+
+struct blk_crypto_attr {
+	struct attribute attr;
+	ssize_t (*show)(struct blk_crypto_profile *profile,
+			struct blk_crypto_attr *attr, char *page);
+};
+
+static struct blk_crypto_profile *kobj_to_crypto_profile(struct kobject *kobj)
+{
+	return container_of(kobj, struct blk_crypto_kobj, kobj)->profile;
+}
+
+static struct blk_crypto_attr *attr_to_crypto_attr(struct attribute *attr)
+{
+	return container_of(attr, struct blk_crypto_attr, attr);
+}
+
+static ssize_t max_dun_bits_show(struct blk_crypto_profile *profile,
+				 struct blk_crypto_attr *attr, char *page)
+{
+	return sysfs_emit(page, "%u\n", 8 * profile->max_dun_bytes_supported);
+}
+
+static ssize_t num_keyslots_show(struct blk_crypto_profile *profile,
+				 struct blk_crypto_attr *attr, char *page)
+{
+	return sysfs_emit(page, "%u\n", profile->num_slots);
+}
+
+#define BLK_CRYPTO_RO_ATTR(_name) \
+	static struct blk_crypto_attr _name##_attr = __ATTR_RO(_name)
+
+BLK_CRYPTO_RO_ATTR(max_dun_bits);
+BLK_CRYPTO_RO_ATTR(num_keyslots);
+
+static struct attribute *blk_crypto_attrs[] = {
+	&max_dun_bits_attr.attr,
+	&num_keyslots_attr.attr,
+	NULL,
+};
+
+static const struct attribute_group blk_crypto_attr_group = {
+	.attrs = blk_crypto_attrs,
+};
+
+/*
+ * The encryption mode attributes.  To avoid hard-coding the list of encryption
+ * modes, these are initialized at boot time by blk_crypto_sysfs_init().
+ */
+static struct blk_crypto_attr __blk_crypto_mode_attrs[BLK_ENCRYPTION_MODE_MAX];
+static struct attribute *blk_crypto_mode_attrs[BLK_ENCRYPTION_MODE_MAX + 1];
+
+static umode_t blk_crypto_mode_is_visible(struct kobject *kobj,
+					  struct attribute *attr, int n)
+{
+	struct blk_crypto_profile *profile = kobj_to_crypto_profile(kobj);
+	struct blk_crypto_attr *a = attr_to_crypto_attr(attr);
+	int mode_num = a - __blk_crypto_mode_attrs;
+
+	if (profile->modes_supported[mode_num])
+		return 0444;
+	return 0;
+}
+
+static ssize_t blk_crypto_mode_show(struct blk_crypto_profile *profile,
+				    struct blk_crypto_attr *attr, char *page)
+{
+	int mode_num = attr - __blk_crypto_mode_attrs;
+
+	return sysfs_emit(page, "0x%x\n", profile->modes_supported[mode_num]);
+}
+
+static const struct attribute_group blk_crypto_modes_attr_group = {
+	.name = "modes",
+	.attrs = blk_crypto_mode_attrs,
+	.is_visible = blk_crypto_mode_is_visible,
+};
+
+static const struct attribute_group *blk_crypto_attr_groups[] = {
+	&blk_crypto_attr_group,
+	&blk_crypto_modes_attr_group,
+	NULL,
+};
+
+static ssize_t blk_crypto_attr_show(struct kobject *kobj,
+				    struct attribute *attr, char *page)
+{
+	struct blk_crypto_profile *profile = kobj_to_crypto_profile(kobj);
+	struct blk_crypto_attr *a = attr_to_crypto_attr(attr);
+
+	return a->show(profile, a, page);
+}
+
+static const struct sysfs_ops blk_crypto_attr_ops = {
+	.show = blk_crypto_attr_show,
+};
+
+static void blk_crypto_release(struct kobject *kobj)
+{
+	kfree(container_of(kobj, struct blk_crypto_kobj, kobj));
+}
+
+static struct kobj_type blk_crypto_ktype = {
+	.default_groups = blk_crypto_attr_groups,
+	.sysfs_ops	= &blk_crypto_attr_ops,
+	.release	= blk_crypto_release,
+};
+
+/*
+ * If the request_queue has a blk_crypto_profile, create the "crypto"
+ * subdirectory in sysfs (/sys/block/$disk/queue/crypto/).
+ */
+int blk_crypto_sysfs_register(struct request_queue *q)
+{
+	struct blk_crypto_kobj *obj;
+	int err;
+
+	if (!q->crypto_profile)
+		return 0;
+
+	obj = kzalloc(sizeof(*obj), GFP_KERNEL);
+	if (!obj)
+		return -ENOMEM;
+	obj->profile = q->crypto_profile;
+
+	err = kobject_init_and_add(&obj->kobj, &blk_crypto_ktype, &q->kobj,
+				   "crypto");
+	if (err) {
+		kobject_put(&obj->kobj);
+		return err;
+	}
+	q->crypto_kobject = &obj->kobj;
+	return 0;
+}
+
+void blk_crypto_sysfs_unregister(struct request_queue *q)
+{
+	kobject_put(q->crypto_kobject);
+}
+
+static int __init blk_crypto_sysfs_init(void)
+{
+	int i;
+
+	BUILD_BUG_ON(BLK_ENCRYPTION_MODE_INVALID != 0);
+	for (i = 1; i < BLK_ENCRYPTION_MODE_MAX; i++) {
+		struct blk_crypto_attr *attr = &__blk_crypto_mode_attrs[i];
+
+		attr->attr.name = blk_crypto_modes[i].name;
+		attr->attr.mode = 0444;
+		attr->show = blk_crypto_mode_show;
+		blk_crypto_mode_attrs[i - 1] = &attr->attr;
+	}
+	return 0;
+}
+subsys_initcall(blk_crypto_sysfs_init);
diff --git a/block/blk-crypto.c b/block/blk-crypto.c
index ec9efeeeca918..f8a36c723a987 100644
--- a/block/blk-crypto.c
+++ b/block/blk-crypto.c
@@ -19,16 +19,19 @@
 
 const struct blk_crypto_mode blk_crypto_modes[] = {
 	[BLK_ENCRYPTION_MODE_AES_256_XTS] = {
+		.name = "AES-256-XTS",
 		.cipher_str = "xts(aes)",
 		.keysize = 64,
 		.ivsize = 16,
 	},
 	[BLK_ENCRYPTION_MODE_AES_128_CBC_ESSIV] = {
+		.name = "AES-128-CBC-ESSIV",
 		.cipher_str = "essiv(cbc(aes),sha256)",
 		.keysize = 16,
 		.ivsize = 16,
 	},
 	[BLK_ENCRYPTION_MODE_ADIANTUM] = {
+		.name = "Adiantum",
 		.cipher_str = "adiantum(xchacha12,aes)",
 		.keysize = 32,
 		.ivsize = 32,
diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
index c11242ef88558..b5f8029fe155e 100644
--- a/block/blk-sysfs.c
+++ b/block/blk-sysfs.c
@@ -877,6 +877,10 @@ int blk_register_queue(struct gendisk *disk)
 			goto put_dev;
 	}
 
+	ret = blk_crypto_sysfs_register(q);
+	if (ret)
+		goto put_dev;
+
 	blk_queue_flag_set(QUEUE_FLAG_REGISTERED, q);
 	wbt_enable_default(q);
 	blk_throtl_register_queue(q);
@@ -908,6 +912,7 @@ int blk_register_queue(struct gendisk *disk)
 	return ret;
 
 put_dev:
+	elv_unregister_queue(q);
 	disk_unregister_independent_access_ranges(disk);
 	mutex_unlock(&q->sysfs_lock);
 	mutex_unlock(&q->sysfs_dir_lock);
@@ -952,6 +957,7 @@ void blk_unregister_queue(struct gendisk *disk)
 	 */
 	if (queue_is_mq(q))
 		blk_mq_unregister_dev(disk_to_dev(disk), q);
+	blk_crypto_sysfs_unregister(q);
 	blk_trace_remove_sysfs(disk_to_dev(disk));
 
 	mutex_lock(&q->sysfs_lock);
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index c80cfaefc0a8f..c9461fb185d1f 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -262,6 +262,7 @@ struct request_queue {
 
 #ifdef CONFIG_BLK_INLINE_ENCRYPTION
 	struct blk_crypto_profile *crypto_profile;
+	struct kobject *crypto_kobject;
 #endif
 
 	unsigned int		rq_timeout;
-- 
2.34.1

Re: [PATCH v3 3/3] blk-crypto: show crypto capabilities in sysfs

[Bart Van Assche]

On 12/7/21 5:35 PM, Eric Biggers wrote:
> +What:		/sys/block/<disk>/queue/crypto/modes/<mode>
> +Date:		December 2021
> +Contact:	linux-block@vger.kernel.org
> +Description:
> +		[RO] For each crypto mode (i.e., encryption/decryption
> +		algorithm) the device supports with inline encryption, a file
> +		will exist at this location.  It will contain a hexadecimal
> +		number that is a bitmask of the supported data unit sizes, in
> +		bytes, for that crypto mode.
> +
> +		Currently, the crypto modes that may be supported are:
> +
> +		   * AES-256-XTS
> +		   * AES-128-CBC-ESSIV
> +		   * Adiantum
> +
> +		For example, if a device supports AES-256-XTS inline encryption
> +		with data unit sizes of 512 and 4096 bytes, the file
> +		/sys/block/<disk>/queue/crypto/modes/AES-256-XTS will exist and
> +		will contain "0x1200".

So a bitmask is used to combine multiple values into a single number? Has it been
considered to report each value separately, e.g. 512\n4096\n instead of 0x1200\n?
I think the former approach is more friendly for shell scripts.

> +struct blk_crypto_attr {
> +	struct attribute attr;
> +	ssize_t (*show)(struct blk_crypto_profile *profile,
> +			struct blk_crypto_attr *attr, char *page);
> +};

It is not clear to me why this structure has been introduced instead of using the
existing kobj_attribute structure?

> +static int __init blk_crypto_sysfs_init(void)
> +{
> +	int i;
> +
> +	BUILD_BUG_ON(BLK_ENCRYPTION_MODE_INVALID != 0);
> +	for (i = 1; i < BLK_ENCRYPTION_MODE_MAX; i++) {
> +		struct blk_crypto_attr *attr = &__blk_crypto_mode_attrs[i];
> +
> +		attr->attr.name = blk_crypto_modes[i].name;
> +		attr->attr.mode = 0444;
> +		attr->show = blk_crypto_mode_show;
> +		blk_crypto_mode_attrs[i - 1] = &attr->attr;
> +	}
> +	return 0;
> +}
> +subsys_initcall(blk_crypto_sysfs_init);

Would it be possible to remove the use of subsys_initcall() if the crypto attributes and
blk_crypto_modes[] would be defined in the same file?

Thanks,

Bart.

Re: [PATCH v3 3/3] blk-crypto: show crypto capabilities in sysfs

[Eric Biggers]

On Thu, Dec 09, 2021 at 02:51:59PM -0800, Bart Van Assche wrote:
> On 12/7/21 5:35 PM, Eric Biggers wrote:
> > +What:		/sys/block/<disk>/queue/crypto/modes/<mode>
> > +Date:		December 2021
> > +Contact:	linux-block@vger.kernel.org
> > +Description:
> > +		[RO] For each crypto mode (i.e., encryption/decryption
> > +		algorithm) the device supports with inline encryption, a file
> > +		will exist at this location.  It will contain a hexadecimal
> > +		number that is a bitmask of the supported data unit sizes, in
> > +		bytes, for that crypto mode.
> > +
> > +		Currently, the crypto modes that may be supported are:
> > +
> > +		   * AES-256-XTS
> > +		   * AES-128-CBC-ESSIV
> > +		   * Adiantum
> > +
> > +		For example, if a device supports AES-256-XTS inline encryption
> > +		with data unit sizes of 512 and 4096 bytes, the file
> > +		/sys/block/<disk>/queue/crypto/modes/AES-256-XTS will exist and
> > +		will contain "0x1200".
> 
> So a bitmask is used to combine multiple values into a single number?

You could think of it that way, yes.

> Has it been considered to report each value separately, e.g. 512\n4096\n
> instead of 0x1200\n?  I think the former approach is more friendly for shell
> scripts.

I don't think that would be acceptable to the sysfs folks, as they only allow
one value per file.  I suppose a bitmask could be viewed as unacceptable too,
but it seemed to make sense here, given that the data unit sizes are always
powers of 2, and the hardware reports them as bitmasks.

Greg already reviewed this patch, but maybe he wasn't looking at this part.

Greg, any opinion on the best way to report a set of power-of-2 values via
sysfs?

> 
> > +struct blk_crypto_attr {
> > +	struct attribute attr;
> > +	ssize_t (*show)(struct blk_crypto_profile *profile,
> > +			struct blk_crypto_attr *attr, char *page);
> > +};
> 
> It is not clear to me why this structure has been introduced instead of using the
> existing kobj_attribute structure?

kobj_attribute isn't strongly-typed to the specific type of kobject.  It also
includes a store function pointer, which is not necessary here.

What I'm doing here is very common; take a look at some other code that
implements sysfs attributes.  Even block/blk-sysfs.c.

> > +static int __init blk_crypto_sysfs_init(void)
> > +{
> > +	int i;
> > +
> > +	BUILD_BUG_ON(BLK_ENCRYPTION_MODE_INVALID != 0);
> > +	for (i = 1; i < BLK_ENCRYPTION_MODE_MAX; i++) {
> > +		struct blk_crypto_attr *attr = &__blk_crypto_mode_attrs[i];
> > +
> > +		attr->attr.name = blk_crypto_modes[i].name;
> > +		attr->attr.mode = 0444;
> > +		attr->show = blk_crypto_mode_show;
> > +		blk_crypto_mode_attrs[i - 1] = &attr->attr;
> > +	}
> > +	return 0;
> > +}
> > +subsys_initcall(blk_crypto_sysfs_init);
> 
> Would it be possible to remove the use of subsys_initcall() if the crypto attributes and
> blk_crypto_modes[] would be defined in the same file?

I want to make it so that all crypto modes show up in sysfs without having to
write extra code every time a new crypto mode is added.  That's not possible if
the attributes are defined statically.  Defining them in the same array would
come close, since then it would be hard for people to forgot to update one place
and not the other.  But that would mix together the sysfs support and the core
blk-crypto support, which seems undesirable.

- Eric

Re: [PATCH v3 3/3] blk-crypto: show crypto capabilities in sysfs

[Bart Van Assche]

On 12/9/21 3:40 PM, Eric Biggers wrote:
> On Thu, Dec 09, 2021 at 02:51:59PM -0800, Bart Van Assche wrote:
>> Has it been considered to report each value separately, e.g. 512\n4096\n
>> instead of 0x1200\n?  I think the former approach is more friendly for shell
>> scripts.
> 
> I don't think that would be acceptable to the sysfs folks, as they only allow
> one value per file.  I suppose a bitmask could be viewed as unacceptable too,
> but it seemed to make sense here, given that the data unit sizes are always
> powers of 2, and the hardware reports them as bitmasks.

In case Greg wouldn't have the time to reply, I think the following quote from
Documentation/filesystems/sysfs.txt is relevant in this context: "Attributes
should be ASCII text files, preferably with only one value per file. It is
noted that it may not be efficient to contain only one value per file, so it is
socially acceptable to express an array of values of the same type."

Thanks,

Bart.

Re: [PATCH v3 3/3] blk-crypto: show crypto capabilities in sysfs

[Eric Biggers]

On Thu, Dec 09, 2021 at 04:02:07PM -0800, Bart Van Assche wrote:
> On 12/9/21 3:40 PM, Eric Biggers wrote:
> > On Thu, Dec 09, 2021 at 02:51:59PM -0800, Bart Van Assche wrote:
> > > Has it been considered to report each value separately, e.g. 512\n4096\n
> > > instead of 0x1200\n?  I think the former approach is more friendly for shell
> > > scripts.
> > 
> > I don't think that would be acceptable to the sysfs folks, as they only allow
> > one value per file.  I suppose a bitmask could be viewed as unacceptable too,
> > but it seemed to make sense here, given that the data unit sizes are always
> > powers of 2, and the hardware reports them as bitmasks.
> 
> In case Greg wouldn't have the time to reply, I think the following quote from
> Documentation/filesystems/sysfs.txt is relevant in this context: "Attributes
> should be ASCII text files, preferably with only one value per file. It is
> noted that it may not be efficient to contain only one value per file, so it is
> socially acceptable to express an array of values of the same type."
> 
> Thanks,

It should be, but I thought that Greg had complained about people doing that
before, and required strictly one value per file.  So we would need his opinion.

Note that a bitmask isn't hard to handle in a shell script:

	mask=$(</sys/block/sda/queue/crypto/modes/AES-256-XTS)
	if (( mask & 4096 )); then
		echo "4096-byte data units supported"
	fi

But I could see how someone could prefer something like

	if grep -q '\<4096\>' /sys/block/sda/queue/crypto/modes/AES-256-XTS; then
		echo "4096-byte data units supported"
	fi

- Eric

Re: [PATCH v3 3/3] blk-crypto: show crypto capabilities in sysfs

[Greg Kroah-Hartman]

On Thu, Dec 09, 2021 at 03:40:46PM -0800, Eric Biggers wrote:
> On Thu, Dec 09, 2021 at 02:51:59PM -0800, Bart Van Assche wrote:
> > On 12/7/21 5:35 PM, Eric Biggers wrote:
> > > +What:		/sys/block/<disk>/queue/crypto/modes/<mode>
> > > +Date:		December 2021
> > > +Contact:	linux-block@vger.kernel.org
> > > +Description:
> > > +		[RO] For each crypto mode (i.e., encryption/decryption
> > > +		algorithm) the device supports with inline encryption, a file
> > > +		will exist at this location.  It will contain a hexadecimal
> > > +		number that is a bitmask of the supported data unit sizes, in
> > > +		bytes, for that crypto mode.
> > > +
> > > +		Currently, the crypto modes that may be supported are:
> > > +
> > > +		   * AES-256-XTS
> > > +		   * AES-128-CBC-ESSIV
> > > +		   * Adiantum
> > > +
> > > +		For example, if a device supports AES-256-XTS inline encryption
> > > +		with data unit sizes of 512 and 4096 bytes, the file
> > > +		/sys/block/<disk>/queue/crypto/modes/AES-256-XTS will exist and
> > > +		will contain "0x1200".
> > 
> > So a bitmask is used to combine multiple values into a single number?
> 
> You could think of it that way, yes.
> 
> > Has it been considered to report each value separately, e.g. 512\n4096\n
> > instead of 0x1200\n?  I think the former approach is more friendly for shell
> > scripts.

No multi-line outputs in sysfs please.  If you have to do a "selection
from an array", that is not how to do it.

Look at the output of /sys/power/pm_test/ as one way you can do it if
you have to choose one option from a list:
$ cat /sys/power/pm_test
[none] core processors platform devices freezer

But I don't think that applies here, right?

> I don't think that would be acceptable to the sysfs folks, as they only allow
> one value per file.  I suppose a bitmask could be viewed as unacceptable too,
> but it seemed to make sense here, given that the data unit sizes are always
> powers of 2, and the hardware reports them as bitmasks.
> 
> Greg already reviewed this patch, but maybe he wasn't looking at this part.
> 
> Greg, any opinion on the best way to report a set of power-of-2 values via
> sysfs?

A single hex value makes sense to me.

thanks,

greg k-h

Re: [PATCH v3 3/3] blk-crypto: show crypto capabilities in sysfs

[Bart Van Assche]

On 12/9/21 10:42 PM, Greg Kroah-Hartman wrote:
> A single hex value makes sense to me.

Hi Greg,

I'm not enthusiast about this approach because:
(a) A single hex value can be confused with a number. Reporting a bitfield in
     hex format is not sufficient to prevent confusion with a number.
(b) No other block layer sysfs attribute follows this encoding scheme.
(c) This encoding enforces the restriction that data unit sizes are a power of
     two. Is there anything fundamental in encryption that restricts data unit
     sizes to a power of two? I don't know the answer myself.

Thanks,

Bart.

Re: [PATCH v3 3/3] blk-crypto: show crypto capabilities in sysfs

[Eric Biggers]

On Fri, Dec 10, 2021 at 09:29:41AM -0800, Bart Van Assche wrote:
> (c) This encoding enforces the restriction that data unit sizes are a power of
>     two. Is there anything fundamental in encryption that restricts data unit
>     sizes to a power of two? I don't know the answer myself.

Well, the data unit size has to evenly divide the size of the request (for
requests that have an encryption context which specifies that data unit size).
So if the data unit size was, say, 1536 bytes, then all requests would have to
be multiples of 1536 bytes.  That has a factor of 3 in it, so it would be
impossible to make any power-of-2 size request.  That sounds pretty impractical;
it's hard to see how and why we would ever support such a thing.

- Eric

Re: [PATCH v3 3/3] blk-crypto: show crypto capabilities in sysfs

[Greg Kroah-Hartman]

On Fri, Dec 10, 2021 at 09:29:41AM -0800, Bart Van Assche wrote:
> On 12/9/21 10:42 PM, Greg Kroah-Hartman wrote:
> > A single hex value makes sense to me.
> 
> Hi Greg,
> 
> I'm not enthusiast about this approach because:
> (a) A single hex value can be confused with a number. Reporting a bitfield in
>     hex format is not sufficient to prevent confusion with a number.

Each sysfs file has their own "units" or values, or whatever.  So a hex
number or bitfield or something else is fine.

Again, single value, no need to parse, is the key here.

> (b) No other block layer sysfs attribute follows this encoding scheme.

Then follow what they do.  Do they have multiple values in a single
file?  If so, they are broken and we should change that.

> (c) This encoding enforces the restriction that data unit sizes are a power of
>     two. Is there anything fundamental in encryption that restricts data unit
>     sizes to a power of two? I don't know the answer myself.

Again, you all can pick the rules you want for this file, if you want to
have bitfields, wonderful!  If you want to make it an enum, wonderful!

thanks,

greg k-h

Re: [PATCH v3 3/3] blk-crypto: show crypto capabilities in sysfs

[Bart Van Assche]

On 12/11/21 02:50, Greg Kroah-Hartman wrote:
> On Fri, Dec 10, 2021 at 09:29:41AM -0800, Bart Van Assche wrote:
>> (b) No other block layer sysfs attribute follows this encoding scheme.
> 
> Then follow what they do.  Do they have multiple values in a single
> file?  If so, they are broken and we should change that.

Hi Greg,

The only other block layer sysfs attribute I know of that reports 
multiple values is the queue/scheduler attribute. Here is an example of 
the output that can be produced by reading that attribute:

# cat /sys/block/sda/queue/scheduler
[mq-deadline] kyber bfq none

I don't think that changing the behavior of that attribute is an option 
because that would break existing user space software, e.g. the 
https://github.com/osandov/blktests/ test suite.

Thanks,

Bart.

Re: [PATCH v3 3/3] blk-crypto: show crypto capabilities in sysfs

[Chaitanya Kulkarni]

On 12/13/21 9:04 PM, Bart Van Assche wrote:
> External email: Use caution opening links or attachments
> 
> 
> On 12/11/21 02:50, Greg Kroah-Hartman wrote:
>> On Fri, Dec 10, 2021 at 09:29:41AM -0800, Bart Van Assche wrote:
>>> (b) No other block layer sysfs attribute follows this encoding scheme.
>>
>> Then follow what they do.  Do they have multiple values in a single
>> file?  If so, they are broken and we should change that.
> 
> Hi Greg,
> 
> The only other block layer sysfs attribute I know of that reports
> multiple values is the queue/scheduler attribute. Here is an example of
> the output that can be produced by reading that attribute:
> 
> # cat /sys/block/sda/queue/scheduler
> [mq-deadline] kyber bfq none
> 
> I don't think that changing the behavior of that attribute is an option
> because that would break existing user space software, e.g. the
> https://github.com/osandov/blktests/ test suite.
> 
> Thanks,
> 
> Bart.

If possible please avoid changing these values.

Re: [PATCH v3 3/3] blk-crypto: show crypto capabilities in sysfs

[Greg Kroah-Hartman]

On Mon, Dec 13, 2021 at 09:04:29PM -0800, Bart Van Assche wrote:
> On 12/11/21 02:50, Greg Kroah-Hartman wrote:
> > On Fri, Dec 10, 2021 at 09:29:41AM -0800, Bart Van Assche wrote:
> > > (b) No other block layer sysfs attribute follows this encoding scheme.
> > 
> > Then follow what they do.  Do they have multiple values in a single
> > file?  If so, they are broken and we should change that.
> 
> Hi Greg,
> 
> The only other block layer sysfs attribute I know of that reports multiple
> values is the queue/scheduler attribute. Here is an example of the output
> that can be produced by reading that attribute:
> 
> # cat /sys/block/sda/queue/scheduler
> [mq-deadline] kyber bfq none

That's fine, there is no problem there.

That output is the "correct" way to show a list of options and the one
that is currently selected in a sysfs file as I thought I said earlier
in this thread.

> I don't think that changing the behavior of that attribute is an option
> because that would break existing user space software, e.g. the
> https://github.com/osandov/blktests/ test suite.

Very true, do not change this file format.

thanks,

greg k-h