All of lore.kernel.org
 help / color / mirror / Atom feed
* [fsverity-utils PATCH 0/4] Add option to write Merkle tree to a file
@ 2021-06-03 19:58 Eric Biggers
  2021-06-03 19:58 ` [fsverity-utils PATCH 1/4] lib/compute_digest: add callbacks for getting the verity metadata Eric Biggers
                   ` (5 more replies)
  0 siblings, 6 replies; 11+ messages in thread
From: Eric Biggers @ 2021-06-03 19:58 UTC (permalink / raw)
  To: linux-fscrypt; +Cc: Victor Hsieh

Make 'fsverity digest' and 'fsverity sign' support writing the Merkle
tree and fs-verity descriptor to files, using new options
'--out-merkle-tree=FILE' and '--out-descriptor=FILE'.

Normally these new options aren't useful, but they can be needed in
cases where the fs-verity metadata needs to be consumed by something
other than one of the native Linux kernel implementations of fs-verity.

This is different from 'fsverity dump_metadata' in that
'fsverity dump_metadata' only works on a file with fs-verity enabled,
whereas these new options are for the userspace file digest computation.

Supporting this required adding some optional callbacks to
libfsverity_compute_digest().

Eric Biggers (4):
  lib/compute_digest: add callbacks for getting the verity metadata
  programs/test_compute_digest: test the metadata callbacks
  programs/utils: add full_pwrite() and preallocate_file()
  programs/fsverity: add --out-merkle-tree and --out-descriptor options

 include/libfsverity.h          |  46 +++++++++++-
 lib/compute_digest.c           | 130 +++++++++++++++++++++++++++-----
 programs/cmd_digest.c          |   7 +-
 programs/cmd_sign.c            |  17 +++--
 programs/fsverity.c            |  88 +++++++++++++++++++++-
 programs/fsverity.h            |   4 +-
 programs/test_compute_digest.c | 133 +++++++++++++++++++++++++++++++++
 programs/utils.c               |  59 +++++++++++++++
 programs/utils.h               |   3 +
 9 files changed, 458 insertions(+), 29 deletions(-)


base-commit: cf8fa5e5a7ac5b3b2dbfcc87e5dbd5f984c2d83a
-- 
2.31.1


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [fsverity-utils PATCH 1/4] lib/compute_digest: add callbacks for getting the verity metadata
  2021-06-03 19:58 [fsverity-utils PATCH 0/4] Add option to write Merkle tree to a file Eric Biggers
@ 2021-06-03 19:58 ` Eric Biggers
  2021-06-03 19:58 ` [fsverity-utils PATCH 2/4] programs/test_compute_digest: test the metadata callbacks Eric Biggers
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 11+ messages in thread
From: Eric Biggers @ 2021-06-03 19:58 UTC (permalink / raw)
  To: linux-fscrypt; +Cc: Victor Hsieh

From: Eric Biggers <ebiggers@google.com>

Allow callers of libfsverity_compute_digest() to provide callback
functions which get passed the Merkle tree and fs-verity descriptor
after they are calculated.

This will allow adding options to 'fsverity digest' and 'fsverity sign'
which cause this metadata to be dumped to files.  Normally this isn't
useful, but this can be needed in cases where the fs-verity metadata
needs to be consumed by something other than one of the native Linux
kernel implementations of fs-verity.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 include/libfsverity.h |  46 ++++++++++++++-
 lib/compute_digest.c  | 130 ++++++++++++++++++++++++++++++++++++------
 2 files changed, 156 insertions(+), 20 deletions(-)

diff --git a/include/libfsverity.h b/include/libfsverity.h
index 6c42e5e..c2c6c18 100644
--- a/include/libfsverity.h
+++ b/include/libfsverity.h
@@ -61,8 +61,18 @@ struct libfsverity_merkle_tree_params {
 	/** @reserved1: must be 0 */
 	uint64_t reserved1[8];
 
+	/**
+	 * @metadata_callbacks: if non-NULL, this gives a set of callback
+	 * functions to which libfsverity_compute_digest() will pass the Merkle
+	 * tree blocks and fs-verity descriptor after they are computed.
+	 * Normally this isn't useful, but this can be needed in rare cases
+	 * where the metadata needs to be consumed by something other than one
+	 * of the native Linux kernel implementations of fs-verity.
+	 */
+	const struct libfsverity_metadata_callbacks *metadata_callbacks;
+
 	/** @reserved2: must be 0 */
-	uintptr_t reserved2[8];
+	uintptr_t reserved2[7];
 };
 
 struct libfsverity_digest {
@@ -78,6 +88,37 @@ struct libfsverity_signature_params {
 	uintptr_t reserved2[8];		/* must be 0 */
 };
 
+struct libfsverity_metadata_callbacks {
+
+	/** @ctx: context passed to the below callbacks (opaque to library) */
+	void *ctx;
+
+	/**
+	 * @merkle_tree_size: if non-NULL, called with the total size of the
+	 * Merkle tree in bytes, prior to any call to @merkle_tree_block.  Must
+	 * return 0 on success, or a negative errno value on failure.
+	 */
+	int (*merkle_tree_size)(void *ctx, uint64_t size);
+
+	/**
+	 * @merkle_tree_block: if non-NULL, called with each block of the
+	 * Merkle tree after it is computed.  The offset is the offset in bytes
+	 * to the block within the Merkle tree, using the Merkle tree layout
+	 * used by FS_IOC_READ_VERITY_METADATA.  The offsets won't necessarily
+	 * be in increasing order.  Must return 0 on success, or a negative
+	 * errno value on failure.
+	 */
+	int (*merkle_tree_block)(void *ctx, const void *block, size_t size,
+				 uint64_t offset);
+
+	/**
+	 * @descriptor: if non-NULL, called with the fs-verity descriptor after
+	 * it is computed.  Must return 0 on success, or a negative errno value
+	 * on failure.
+	 */
+	int (*descriptor)(void *ctx, const void *descriptor, size_t size);
+};
+
 /*
  * libfsverity_read_fn_t - callback that incrementally provides a file's data
  * @fd: the user-provided "file descriptor" (opaque to library)
@@ -101,7 +142,8 @@ typedef int (*libfsverity_read_fn_t)(void *fd, void *buf, size_t count);
  *
  * Returns:
  * * 0 for success, -EINVAL for invalid input arguments, -ENOMEM if libfsverity
- *   failed to allocate memory, or an error returned by @read_fn.
+ *   failed to allocate memory, or an error returned by @read_fn or by one of
+ *   the @params->metadata_callbacks.
  * * digest_ret returns a pointer to the digest on success. The digest object
  *   is allocated by libfsverity and must be freed by the caller using free().
  */
diff --git a/lib/compute_digest.c b/lib/compute_digest.c
index a4f649c..c5b0100 100644
--- a/lib/compute_digest.c
+++ b/lib/compute_digest.c
@@ -24,9 +24,8 @@ struct block_buffer {
 
 /*
  * Hash a block, writing the result to the next level's pending block buffer.
- * Returns true if the next level's block became full, else false.
  */
-static bool hash_one_block(struct hash_ctx *hash, struct block_buffer *cur,
+static void hash_one_block(struct hash_ctx *hash, struct block_buffer *cur,
 			   u32 block_size, const u8 *salt, u32 salt_size)
 {
 	struct block_buffer *next = cur + 1;
@@ -41,8 +40,60 @@ static bool hash_one_block(struct hash_ctx *hash, struct block_buffer *cur,
 
 	next->filled += hash->alg->digest_size;
 	cur->filled = 0;
+}
+
+static bool block_is_full(const struct block_buffer *block, u32 block_size,
+			  struct hash_ctx *hash)
+{
+	/* Would the next hash put us over the limit? */
+	return block->filled + hash->alg->digest_size > block_size;
+}
+
+static int report_merkle_tree_size(const struct libfsverity_metadata_callbacks *cbs,
+				   u64 size)
+{
+	if (cbs && cbs->merkle_tree_size) {
+		int err = cbs->merkle_tree_size(cbs->ctx, size);
 
-	return next->filled + hash->alg->digest_size > block_size;
+		if (err) {
+			libfsverity_error_msg("error processing Merkle tree size");
+			return err;
+		}
+	}
+	return 0;
+}
+
+static int report_merkle_tree_block(const struct libfsverity_metadata_callbacks *cbs,
+				    const struct block_buffer *block,
+				    u32 block_size, u64 *level_offset)
+{
+
+	if (cbs && cbs->merkle_tree_block) {
+		int err = cbs->merkle_tree_block(cbs->ctx, block->data,
+						 block_size,
+						 *level_offset * block_size);
+
+		if (err) {
+			libfsverity_error_msg("error processing Merkle tree block");
+			return err;
+		}
+		(*level_offset)++;
+	}
+	return 0;
+}
+
+static int report_descriptor(const struct libfsverity_metadata_callbacks *cbs,
+			     const void *descriptor, size_t size)
+{
+	if (cbs && cbs->descriptor) {
+		int err = cbs->descriptor(cbs->ctx, descriptor, size);
+
+		if (err) {
+			libfsverity_error_msg("error processing fs-verity descriptor");
+			return err;
+		}
+	}
+	return 0;
 }
 
 /*
@@ -52,6 +103,7 @@ static bool hash_one_block(struct hash_ctx *hash, struct block_buffer *cur,
 static int compute_root_hash(void *fd, libfsverity_read_fn_t read_fn,
 			     u64 file_size, struct hash_ctx *hash,
 			     u32 block_size, const u8 *salt, u32 salt_size,
+			     const struct libfsverity_metadata_callbacks *metadata_cbs,
 			     u8 *root_hash)
 {
 	const u32 hashes_per_block = block_size / hash->alg->digest_size;
@@ -60,6 +112,7 @@ static int compute_root_hash(void *fd, libfsverity_read_fn_t read_fn,
 	u64 blocks;
 	int num_levels = 0;
 	int level;
+	u64 level_offset[FS_VERITY_MAX_LEVELS];
 	struct block_buffer _buffers[1 + FS_VERITY_MAX_LEVELS + 1] = {};
 	struct block_buffer *buffers = &_buffers[1];
 	u64 offset;
@@ -68,7 +121,7 @@ static int compute_root_hash(void *fd, libfsverity_read_fn_t read_fn,
 	/* Root hash of empty file is all 0's */
 	if (file_size == 0) {
 		memset(root_hash, 0, hash->alg->digest_size);
-		return 0;
+		return report_merkle_tree_size(metadata_cbs, 0);
 	}
 
 	if (salt_size != 0) {
@@ -78,15 +131,39 @@ static int compute_root_hash(void *fd, libfsverity_read_fn_t read_fn,
 		memcpy(padded_salt, salt, salt_size);
 	}
 
-	/* Compute number of levels */
-	for (blocks = DIV_ROUND_UP(file_size, block_size); blocks > 1;
-	     blocks = DIV_ROUND_UP(blocks, hashes_per_block)) {
+	/* Compute number of levels and the number of blocks in each level. */
+	blocks = DIV_ROUND_UP(file_size, block_size);
+	while (blocks > 1)  {
 		if (WARN_ON(num_levels >= FS_VERITY_MAX_LEVELS)) {
 			err = -EINVAL;
 			goto out;
 		}
-		num_levels++;
+		blocks = DIV_ROUND_UP(blocks, hashes_per_block);
+		/*
+		 * Temporarily use level_offset[] to store the number of blocks
+		 * in each level.  It will be overwritten later.
+		 */
+		level_offset[num_levels++] = blocks;
+	}
+
+	/*
+	 * Compute the starting block of each level, using the convention where
+	 * the root level is first, i.e. the convention used by
+	 * FS_IOC_READ_VERITY_METADATA.  At the same time, compute the total
+	 * size of the Merkle tree.  These values are only needed for the
+	 * metadata callbacks (if they were given), as the hash computation
+	 * itself doesn't prescribe an ordering of the levels and doesn't
+	 * prescribe any special meaning to the total size of the Merkle tree.
+	 */
+	offset = 0;
+	for (level = num_levels - 1; level >= 0; level--) {
+		blocks = level_offset[level];
+		level_offset[level] = offset;
+		offset += blocks;
 	}
+	err = report_merkle_tree_size(metadata_cbs, offset * block_size);
+	if (err)
+		goto out;
 
 	/*
 	 * Allocate the block buffers.  Buffer "-1" is for data blocks.
@@ -112,21 +189,33 @@ static int compute_root_hash(void *fd, libfsverity_read_fn_t read_fn,
 			goto out;
 		}
 
-		level = -1;
-		while (hash_one_block(hash, &buffers[level], block_size,
-				      padded_salt, padded_salt_size)) {
-			level++;
-			if (WARN_ON(level >= num_levels)) {
-				err = -EINVAL;
+		hash_one_block(hash, &buffers[-1], block_size,
+			       padded_salt, padded_salt_size);
+		for (level = 0; level < num_levels; level++) {
+			if (!block_is_full(&buffers[level], block_size, hash))
+				break;
+			hash_one_block(hash, &buffers[level], block_size,
+				       padded_salt, padded_salt_size);
+			err = report_merkle_tree_block(metadata_cbs,
+						       &buffers[level],
+						       block_size,
+						       &level_offset[level]);
+			if (err)
 				goto out;
-			}
 		}
 	}
 	/* Finish all nonempty pending tree blocks */
 	for (level = 0; level < num_levels; level++) {
-		if (buffers[level].filled != 0)
+		if (buffers[level].filled != 0) {
 			hash_one_block(hash, &buffers[level], block_size,
 				       padded_salt, padded_salt_size);
+			err = report_merkle_tree_block(metadata_cbs,
+						       &buffers[level],
+						       block_size,
+						       &level_offset[level]);
+			if (err)
+				goto out;
+		}
 	}
 
 	/* Root hash was filled by the last call to hash_one_block() */
@@ -217,8 +306,13 @@ libfsverity_compute_digest(void *fd, libfsverity_read_fn_t read_fn,
 	}
 
 	err = compute_root_hash(fd, read_fn, params->file_size, hash,
-				block_size, params->salt,
-				params->salt_size, desc.root_hash);
+				block_size, params->salt, params->salt_size,
+				params->metadata_callbacks, desc.root_hash);
+	if (err)
+		goto out;
+
+	err = report_descriptor(params->metadata_callbacks,
+				&desc, sizeof(desc));
 	if (err)
 		goto out;
 
-- 
2.31.1


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [fsverity-utils PATCH 2/4] programs/test_compute_digest: test the metadata callbacks
  2021-06-03 19:58 [fsverity-utils PATCH 0/4] Add option to write Merkle tree to a file Eric Biggers
  2021-06-03 19:58 ` [fsverity-utils PATCH 1/4] lib/compute_digest: add callbacks for getting the verity metadata Eric Biggers
@ 2021-06-03 19:58 ` Eric Biggers
  2021-06-03 19:58 ` [fsverity-utils PATCH 3/4] programs/utils: add full_pwrite() and preallocate_file() Eric Biggers
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 11+ messages in thread
From: Eric Biggers @ 2021-06-03 19:58 UTC (permalink / raw)
  To: linux-fscrypt; +Cc: Victor Hsieh

From: Eric Biggers <ebiggers@google.com>

Test that the libfsverity_metadata_callbacks support seems to be working
correctly.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 programs/test_compute_digest.c | 133 +++++++++++++++++++++++++++++++++
 1 file changed, 133 insertions(+)

diff --git a/programs/test_compute_digest.c b/programs/test_compute_digest.c
index e7f2645..67266fa 100644
--- a/programs/test_compute_digest.c
+++ b/programs/test_compute_digest.c
@@ -13,6 +13,7 @@
 
 #include <ctype.h>
 #include <inttypes.h>
+#include <openssl/sha.h>
 
 struct mem_file {
 	u8 *data;
@@ -37,6 +38,13 @@ static int error_read_fn(void *fd __attribute__((unused)),
 	return -EIO;
 }
 
+static int zeroes_read_fn(void *fd __attribute__((unused)),
+			  void *buf, size_t count)
+{
+	memset(buf, 0, count);
+	return 0;
+}
+
 static const struct test_case {
 	u32 hash_algorithm;
 	u32 block_size;
@@ -249,6 +257,130 @@ static void test_invalid_params(void)
 	ASSERT(d == NULL);
 }
 
+static struct {
+	u64 merkle_tree_size;
+	u64 merkle_tree_block;
+	u64 descriptor;
+} metadata_callback_counts;
+
+static int handle_merkle_tree_size(void *ctx, u64 size)
+{
+	metadata_callback_counts.merkle_tree_size++;
+
+	/* Test that the ctx argument is passed through correctly. */
+	ASSERT(ctx == (void *)1);
+
+	/* Test that the expected Merkle tree size is reported. */
+	ASSERT(size == 5 * 1024);
+	return 0;
+}
+
+static int handle_merkle_tree_block(void *ctx, const void *block, size_t size,
+				    u64 offset)
+{
+	u8 digest[SHA256_DIGEST_LENGTH];
+	u64 count = metadata_callback_counts.merkle_tree_block++;
+	const char *expected_digest;
+
+	/* Test that ->merkle_tree_size() was called first. */
+	ASSERT(metadata_callback_counts.merkle_tree_size == 1);
+
+	/* Test that the ctx argument is passed through correctly. */
+	ASSERT(ctx == (void *)1);
+
+	/*
+	 * Test that this Merkle tree block has the expected size, offset, and
+	 * contents.  The 4 blocks at "level 0" should be reported first, in
+	 * order; then the 1 block at "level 1" should be reported last (but the
+	 * level 1 block should have the smallest offset).
+	 */
+	ASSERT(size == 1024);
+	SHA256(block, size, digest);
+	if (count == 4) {
+		/* 1 block at level 1 */
+		ASSERT(offset == 0);
+		expected_digest = "\x68\xc5\x38\xe1\x19\x58\xd6\x5d"
+				  "\x68\xb6\xfe\x8e\x9f\xb8\xcc\xab"
+				  "\xec\xfd\x92\x8b\x01\xd0\x63\x44"
+				  "\xe2\x23\xed\x41\xdd\xc4\x54\x4a";
+	} else {
+		/* 4 blocks at level 0 */
+		ASSERT(offset == 1024 + (count * 1024));
+		if (count < 3) {
+			expected_digest = "\xf7\x89\xba\xab\x53\x85\x9f\xaf"
+					  "\x36\xd6\xd7\x5d\x10\x42\x06\x42"
+					  "\x94\x20\x2d\x6e\x13\xe7\x71\x6f"
+					  "\x39\x4f\xba\x43\x4c\xcc\x49\x86";
+		} else {
+			expected_digest = "\x00\xfe\xd0\x3c\x5d\x6e\xab\x21"
+					  "\x31\x43\xf3\xd9\x6a\x5c\xa3\x1c"
+					  "\x2b\x89\xf5\x68\x4e\x6c\x8e\x07"
+					  "\x87\x3e\x5e\x97\x65\x17\xb4\x8f";
+		}
+	}
+	ASSERT(!memcmp(digest, expected_digest, SHA256_DIGEST_LENGTH));
+	return 0;
+}
+
+static const u8 expected_file_digest[SHA256_DIGEST_LENGTH] =
+	"\x09\xcb\xba\xee\xd2\xa0\x4c\x2d\xa2\x42\xc1\x0e\x15\x68\xd9\x6f"
+	"\x35\x8a\x16\xaa\x1e\xbe\x8c\xf0\x28\x61\x20\xc1\x3c\x93\x66\xd1";
+
+static int handle_descriptor(void *ctx, const void *descriptor, size_t size)
+{
+	u8 digest[SHA256_DIGEST_LENGTH];
+
+	metadata_callback_counts.descriptor++;
+	/* Test that the ctx argument is passed through correctly. */
+	ASSERT(ctx == (void *)1);
+
+	/* Test that the fs-verity descriptor is reported correctly. */
+	ASSERT(size == 256);
+	SHA256(descriptor, size, digest);
+	ASSERT(!memcmp(digest, expected_file_digest, SHA256_DIGEST_LENGTH));
+	return 0;
+}
+
+static const struct libfsverity_metadata_callbacks metadata_callbacks = {
+	.ctx = (void *)1, /* arbitrary value for testing purposes */
+	.merkle_tree_size = handle_merkle_tree_size,
+	.merkle_tree_block = handle_merkle_tree_block,
+	.descriptor = handle_descriptor,
+};
+
+/* Test that the libfsverity_metadata_callbacks work correctly. */
+static void test_metadata_callbacks(void)
+{
+	/*
+	 * For a useful test, we want a file whose Merkle tree will have at
+	 * least 2 levels (this one will have exactly 2).  The contents of the
+	 * file aren't too important.
+	 */
+	struct libfsverity_merkle_tree_params params = {
+		.version = 1,
+		.hash_algorithm = FS_VERITY_HASH_ALG_SHA256,
+		.block_size = 1024,
+		.file_size = 100000,
+		.metadata_callbacks = &metadata_callbacks,
+	};
+	struct libfsverity_digest *d;
+
+	ASSERT(libfsverity_compute_digest(NULL, zeroes_read_fn,
+					  &params, &d) == 0);
+
+	/* Test that the callbacks were called the correct number of times. */
+	ASSERT(metadata_callback_counts.merkle_tree_size == 1);
+	ASSERT(metadata_callback_counts.merkle_tree_block == 5);
+	ASSERT(metadata_callback_counts.descriptor == 1);
+
+	/* Test that the computed file digest is as expected. */
+	ASSERT(d->digest_algorithm == FS_VERITY_HASH_ALG_SHA256);
+	ASSERT(d->digest_size == SHA256_DIGEST_LENGTH);
+	ASSERT(!memcmp(d->digest, expected_file_digest, SHA256_DIGEST_LENGTH));
+
+	free(d);
+}
+
 int main(int argc, char *argv[])
 {
 	const bool update = (argc == 2 && !strcmp(argv[1], "--update"));
@@ -305,6 +437,7 @@ int main(int argc, char *argv[])
 	}
 
 	test_invalid_params();
+	test_metadata_callbacks();
 	printf("test_compute_digest passed\n");
 	return 0;
 }
-- 
2.31.1


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [fsverity-utils PATCH 3/4] programs/utils: add full_pwrite() and preallocate_file()
  2021-06-03 19:58 [fsverity-utils PATCH 0/4] Add option to write Merkle tree to a file Eric Biggers
  2021-06-03 19:58 ` [fsverity-utils PATCH 1/4] lib/compute_digest: add callbacks for getting the verity metadata Eric Biggers
  2021-06-03 19:58 ` [fsverity-utils PATCH 2/4] programs/test_compute_digest: test the metadata callbacks Eric Biggers
@ 2021-06-03 19:58 ` Eric Biggers
  2021-06-04  0:33   ` Victor Hsieh
  2021-06-03 19:58 ` [fsverity-utils PATCH 4/4] programs/fsverity: add --out-merkle-tree and --out-descriptor options Eric Biggers
                   ` (2 subsequent siblings)
  5 siblings, 1 reply; 11+ messages in thread
From: Eric Biggers @ 2021-06-03 19:58 UTC (permalink / raw)
  To: linux-fscrypt; +Cc: Victor Hsieh

From: Eric Biggers <ebiggers@google.com>

These helper functions will be used by the implementation of the
--out-merkle-tree option for 'fsverity digest' and 'fsverity sign'.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 programs/utils.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++++
 programs/utils.h |  3 +++
 2 files changed, 62 insertions(+)

diff --git a/programs/utils.c b/programs/utils.c
index ce19b57..116eb95 100644
--- a/programs/utils.c
+++ b/programs/utils.c
@@ -13,10 +13,14 @@
 
 #include <errno.h>
 #include <fcntl.h>
+#include <inttypes.h>
 #include <limits.h>
 #include <stdarg.h>
 #include <sys/stat.h>
 #include <unistd.h>
+#ifdef _WIN32
+#  include <windows.h>
+#endif
 
 /* ========== Memory allocation ========== */
 
@@ -126,6 +130,26 @@ bool get_file_size(struct filedes *file, u64 *size_ret)
 	return true;
 }
 
+bool preallocate_file(struct filedes *file, u64 size)
+{
+	int res;
+
+	if (size == 0)
+		return true;
+#ifdef _WIN32
+	/* Not exactly the same as posix_fallocate(), but good enough... */
+	res = _chsize_s(file->fd, size);
+#else
+	res = posix_fallocate(file->fd, 0, size);
+#endif
+	if (res != 0) {
+		error_msg_errno("preallocating %" PRIu64 "-byte file '%s'",
+				size, file->name);
+		return false;
+	}
+	return true;
+}
+
 bool full_read(struct filedes *file, void *buf, size_t count)
 {
 	while (count) {
@@ -160,6 +184,41 @@ bool full_write(struct filedes *file, const void *buf, size_t count)
 	return true;
 }
 
+static int raw_pwrite(int fd, const void *buf, int count, u64 offset)
+{
+#ifdef _WIN32
+	HANDLE h = (HANDLE)_get_osfhandle(fd);
+	OVERLAPPED pos = { .Offset = offset, .OffsetHigh = offset >> 32 };
+	DWORD written = 0;
+
+	/* Not exactly the same as pwrite(), but good enough... */
+	if (!WriteFile(h, buf, count, &written, &pos)) {
+		errno = EIO;
+		return -1;
+	}
+	return written;
+#else
+	return pwrite(fd, buf, count, offset);
+#endif
+}
+
+bool full_pwrite(struct filedes *file, const void *buf, size_t count,
+		 u64 offset)
+{
+	while (count) {
+		int n = raw_pwrite(file->fd, buf, min(count, INT_MAX), offset);
+
+		if (n < 0) {
+			error_msg_errno("writing to '%s'", file->name);
+			return false;
+		}
+		buf += n;
+		count -= n;
+		offset += n;
+	}
+	return true;
+}
+
 bool filedes_close(struct filedes *file)
 {
 	int res;
diff --git a/programs/utils.h b/programs/utils.h
index ab5005f..9a5c97a 100644
--- a/programs/utils.h
+++ b/programs/utils.h
@@ -40,8 +40,11 @@ struct filedes {
 
 bool open_file(struct filedes *file, const char *filename, int flags, int mode);
 bool get_file_size(struct filedes *file, u64 *size_ret);
+bool preallocate_file(struct filedes *file, u64 size);
 bool full_read(struct filedes *file, void *buf, size_t count);
 bool full_write(struct filedes *file, const void *buf, size_t count);
+bool full_pwrite(struct filedes *file, const void *buf, size_t count,
+		 u64 offset);
 bool filedes_close(struct filedes *file);
 int read_callback(void *file, void *buf, size_t count);
 
-- 
2.31.1


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [fsverity-utils PATCH 4/4] programs/fsverity: add --out-merkle-tree and --out-descriptor options
  2021-06-03 19:58 [fsverity-utils PATCH 0/4] Add option to write Merkle tree to a file Eric Biggers
                   ` (2 preceding siblings ...)
  2021-06-03 19:58 ` [fsverity-utils PATCH 3/4] programs/utils: add full_pwrite() and preallocate_file() Eric Biggers
@ 2021-06-03 19:58 ` Eric Biggers
  2021-06-04 15:25 ` [fsverity-utils PATCH 0/4] Add option to write Merkle tree to a file Victor Hsieh
  2021-06-09  6:48 ` Eric Biggers
  5 siblings, 0 replies; 11+ messages in thread
From: Eric Biggers @ 2021-06-03 19:58 UTC (permalink / raw)
  To: linux-fscrypt; +Cc: Victor Hsieh

From: Eric Biggers <ebiggers@google.com>

Make 'fsverity digest' and 'fsverity sign' support writing the Merkle
tree and fs-verity descriptor to files, using new options
'--out-merkle-tree=FILE' and '--out-descriptor=FILE'.

Normally these new options aren't useful, but they can be needed in
cases where the fs-verity metadata needs to be consumed by something
other than one of the native Linux kernel implementations of fs-verity.

This is different from 'fsverity dump_metadata' in that
'fsverity dump_metadata' only works on a file with fs-verity enabled,
whereas these new options are for the userspace file digest computation.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 programs/cmd_digest.c |  7 +++-
 programs/cmd_sign.c   | 17 ++++++---
 programs/fsverity.c   | 88 ++++++++++++++++++++++++++++++++++++++++++-
 programs/fsverity.h   |  4 +-
 4 files changed, 107 insertions(+), 9 deletions(-)

diff --git a/programs/cmd_digest.c b/programs/cmd_digest.c
index 1a3c769..fd9f4de 100644
--- a/programs/cmd_digest.c
+++ b/programs/cmd_digest.c
@@ -18,6 +18,8 @@ static const struct option longopts[] = {
 	{"hash-alg",		required_argument, NULL, OPT_HASH_ALG},
 	{"block-size",		required_argument, NULL, OPT_BLOCK_SIZE},
 	{"salt",		required_argument, NULL, OPT_SALT},
+	{"out-merkle-tree",     required_argument, NULL, OPT_OUT_MERKLE_TREE},
+	{"out-descriptor",      required_argument, NULL, OPT_OUT_DESCRIPTOR},
 	{"compact",		no_argument,	   NULL, OPT_COMPACT},
 	{"for-builtin-sig",	no_argument,	   NULL, OPT_FOR_BUILTIN_SIG},
 	{NULL, 0, NULL, 0}
@@ -40,6 +42,8 @@ int fsverity_cmd_digest(const struct fsverity_command *cmd,
 		case OPT_HASH_ALG:
 		case OPT_BLOCK_SIZE:
 		case OPT_SALT:
+		case OPT_OUT_MERKLE_TREE:
+		case OPT_OUT_DESCRIPTOR:
 			if (!parse_tree_param(c, optarg, &tree_params))
 				goto out_usage;
 			break;
@@ -114,7 +118,8 @@ int fsverity_cmd_digest(const struct fsverity_command *cmd,
 	}
 	status = 0;
 out:
-	destroy_tree_params(&tree_params);
+	if (!destroy_tree_params(&tree_params) && status == 0)
+		status = 1;
 	return status;
 
 out_err:
diff --git a/programs/cmd_sign.c b/programs/cmd_sign.c
index 47ba6a2..81a4ddc 100644
--- a/programs/cmd_sign.c
+++ b/programs/cmd_sign.c
@@ -27,11 +27,13 @@ static bool write_signature(const char *filename, const u8 *sig, u32 sig_size)
 }
 
 static const struct option longopts[] = {
-	{"hash-alg",	required_argument, NULL, OPT_HASH_ALG},
-	{"block-size",	required_argument, NULL, OPT_BLOCK_SIZE},
-	{"salt",	required_argument, NULL, OPT_SALT},
-	{"key",		required_argument, NULL, OPT_KEY},
-	{"cert",	required_argument, NULL, OPT_CERT},
+	{"hash-alg",	    required_argument, NULL, OPT_HASH_ALG},
+	{"block-size",	    required_argument, NULL, OPT_BLOCK_SIZE},
+	{"salt",	    required_argument, NULL, OPT_SALT},
+	{"out-merkle-tree", required_argument, NULL, OPT_OUT_MERKLE_TREE},
+	{"out-descriptor",  required_argument, NULL, OPT_OUT_DESCRIPTOR},
+	{"key",		    required_argument, NULL, OPT_KEY},
+	{"cert",	    required_argument, NULL, OPT_CERT},
 	{NULL, 0, NULL, 0}
 };
 
@@ -54,6 +56,8 @@ int fsverity_cmd_sign(const struct fsverity_command *cmd,
 		case OPT_HASH_ALG:
 		case OPT_BLOCK_SIZE:
 		case OPT_SALT:
+		case OPT_OUT_MERKLE_TREE:
+		case OPT_OUT_DESCRIPTOR:
 			if (!parse_tree_param(c, optarg, &tree_params))
 				goto out_usage;
 			break;
@@ -117,7 +121,8 @@ int fsverity_cmd_sign(const struct fsverity_command *cmd,
 	status = 0;
 out:
 	filedes_close(&file);
-	destroy_tree_params(&tree_params);
+	if (!destroy_tree_params(&tree_params) && status == 0)
+		status = 1;
 	free(digest);
 	free(sig);
 	return status;
diff --git a/programs/fsverity.c b/programs/fsverity.c
index 1168430..f6aff3a 100644
--- a/programs/fsverity.c
+++ b/programs/fsverity.c
@@ -11,6 +11,7 @@
 
 #include "fsverity.h"
 
+#include <fcntl.h>
 #include <limits.h>
 
 static const struct fsverity_command {
@@ -27,6 +28,7 @@ static const struct fsverity_command {
 		.usage_str =
 "    fsverity digest FILE...\n"
 "               [--hash-alg=HASH_ALG] [--block-size=BLOCK_SIZE] [--salt=SALT]\n"
+"               [--out-merkle-tree=FILE] [--out-descriptor=FILE]\n"
 "               [--compact] [--for-builtin-sig]\n"
 #ifndef _WIN32
 	}, {
@@ -58,6 +60,7 @@ static const struct fsverity_command {
 		.usage_str =
 "    fsverity sign FILE OUT_SIGFILE --key=KEYFILE\n"
 "               [--hash-alg=HASH_ALG] [--block-size=BLOCK_SIZE] [--salt=SALT]\n"
+"               [--out-merkle-tree=FILE] [--out-descriptor=FILE]\n"
 "               [--cert=CERTFILE]\n"
 	}
 };
@@ -200,6 +203,74 @@ static bool parse_salt_option(const char *arg, u8 **salt_ptr,
 	return true;
 }
 
+struct metadata_callback_ctx {
+	struct filedes merkle_tree_file;
+	struct filedes descriptor_file;
+	struct libfsverity_metadata_callbacks callbacks;
+};
+
+static int handle_merkle_tree_size(void *_ctx, u64 size)
+{
+	struct metadata_callback_ctx *ctx = _ctx;
+
+	if (!preallocate_file(&ctx->merkle_tree_file, size))
+		return -EIO;
+	return 0;
+}
+
+static int handle_merkle_tree_block(void *_ctx, const void *block, size_t size,
+				    u64 offset)
+{
+	struct metadata_callback_ctx *ctx = _ctx;
+
+	if (!full_pwrite(&ctx->merkle_tree_file, block, size, offset))
+		return -EIO;
+	return 0;
+}
+
+static int handle_descriptor(void *_ctx, const void *descriptor, size_t size)
+{
+	struct metadata_callback_ctx *ctx = _ctx;
+
+	if (!full_write(&ctx->descriptor_file, descriptor, size))
+		return -EIO;
+	return 0;
+}
+
+static bool parse_out_metadata_option(int opt_char, const char *arg,
+				      const struct libfsverity_metadata_callbacks **cbs)
+{
+	struct metadata_callback_ctx *ctx;
+	struct filedes *file;
+	const char *opt_name;
+
+	if (*cbs) {
+		ctx = (*cbs)->ctx;
+	} else {
+		ctx = xzalloc(sizeof(*ctx));
+		ctx->merkle_tree_file.fd = -1;
+		ctx->descriptor_file.fd = -1;
+		ctx->callbacks.ctx = ctx;
+		*cbs = &ctx->callbacks;
+	}
+
+	if (opt_char == OPT_OUT_MERKLE_TREE) {
+		file = &ctx->merkle_tree_file;
+		opt_name = "--out-merkle-tree";
+		ctx->callbacks.merkle_tree_size = handle_merkle_tree_size;
+		ctx->callbacks.merkle_tree_block = handle_merkle_tree_block;
+	} else {
+		file = &ctx->descriptor_file;
+		opt_name = "--out-descriptor";
+		ctx->callbacks.descriptor = handle_descriptor;
+	}
+	if (file->fd >= 0) {
+		error_msg("%s can only be specified once", opt_name);
+		return false;
+	}
+	return open_file(file, arg, O_WRONLY|O_CREAT|O_TRUNC, 0644);
+}
+
 bool parse_tree_param(int opt_char, const char *arg,
 		      struct libfsverity_merkle_tree_params *params)
 {
@@ -211,15 +282,30 @@ bool parse_tree_param(int opt_char, const char *arg,
 	case OPT_SALT:
 		return parse_salt_option(arg, (u8 **)&params->salt,
 					 &params->salt_size);
+	case OPT_OUT_MERKLE_TREE:
+	case OPT_OUT_DESCRIPTOR:
+		return parse_out_metadata_option(opt_char, arg,
+						 &params->metadata_callbacks);
 	default:
 		ASSERT(0);
 	}
 }
 
-void destroy_tree_params(struct libfsverity_merkle_tree_params *params)
+bool destroy_tree_params(struct libfsverity_merkle_tree_params *params)
 {
+	bool ok = true;
+
 	free((u8 *)params->salt);
+	if (params->metadata_callbacks) {
+		struct metadata_callback_ctx *ctx =
+			params->metadata_callbacks->ctx;
+
+		ok &= filedes_close(&ctx->merkle_tree_file);
+		ok &= filedes_close(&ctx->descriptor_file);
+		free(ctx);
+	}
 	memset(params, 0, sizeof(*params));
+	return ok;
 }
 
 int main(int argc, char *argv[])
diff --git a/programs/fsverity.h b/programs/fsverity.h
index 9785013..fe24087 100644
--- a/programs/fsverity.h
+++ b/programs/fsverity.h
@@ -29,6 +29,8 @@ enum {
 	OPT_KEY,
 	OPT_LENGTH,
 	OPT_OFFSET,
+	OPT_OUT_DESCRIPTOR,
+	OPT_OUT_MERKLE_TREE,
 	OPT_SALT,
 	OPT_SIGNATURE,
 };
@@ -59,6 +61,6 @@ int fsverity_cmd_sign(const struct fsverity_command *cmd,
 void usage(const struct fsverity_command *cmd, FILE *fp);
 bool parse_tree_param(int opt_char, const char *arg,
 		      struct libfsverity_merkle_tree_params *params);
-void destroy_tree_params(struct libfsverity_merkle_tree_params *params);
+bool destroy_tree_params(struct libfsverity_merkle_tree_params *params);
 
 #endif /* PROGRAMS_FSVERITY_H */
-- 
2.31.1


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [fsverity-utils PATCH 3/4] programs/utils: add full_pwrite() and preallocate_file()
  2021-06-03 19:58 ` [fsverity-utils PATCH 3/4] programs/utils: add full_pwrite() and preallocate_file() Eric Biggers
@ 2021-06-04  0:33   ` Victor Hsieh
  2021-06-04  0:57     ` Eric Biggers
  0 siblings, 1 reply; 11+ messages in thread
From: Victor Hsieh @ 2021-06-04  0:33 UTC (permalink / raw)
  To: Eric Biggers; +Cc: linux-fscrypt

On Thu, Jun 3, 2021 at 1:00 PM Eric Biggers <ebiggers@kernel.org> wrote:
>
> From: Eric Biggers <ebiggers@google.com>
>
> These helper functions will be used by the implementation of the
> --out-merkle-tree option for 'fsverity digest' and 'fsverity sign'.
>
> Signed-off-by: Eric Biggers <ebiggers@google.com>
> ---
>  programs/utils.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++++
>  programs/utils.h |  3 +++
>  2 files changed, 62 insertions(+)
>
> diff --git a/programs/utils.c b/programs/utils.c
> index ce19b57..116eb95 100644
> --- a/programs/utils.c
> +++ b/programs/utils.c
> @@ -13,10 +13,14 @@
>
>  #include <errno.h>
>  #include <fcntl.h>
> +#include <inttypes.h>
>  #include <limits.h>
>  #include <stdarg.h>
>  #include <sys/stat.h>
>  #include <unistd.h>
> +#ifdef _WIN32
> +#  include <windows.h>
> +#endif
>
>  /* ========== Memory allocation ========== */
>
> @@ -126,6 +130,26 @@ bool get_file_size(struct filedes *file, u64 *size_ret)
>         return true;
>  }
>
> +bool preallocate_file(struct filedes *file, u64 size)
> +{
> +       int res;
> +
> +       if (size == 0)
> +               return true;
> +#ifdef _WIN32
> +       /* Not exactly the same as posix_fallocate(), but good enough... */
> +       res = _chsize_s(file->fd, size);
> +#else
> +       res = posix_fallocate(file->fd, 0, size);
> +#endif
> +       if (res != 0) {
> +               error_msg_errno("preallocating %" PRIu64 "-byte file '%s'",
> +                               size, file->name);
> +               return false;
> +       }
> +       return true;
> +}
> +
>  bool full_read(struct filedes *file, void *buf, size_t count)
>  {
>         while (count) {
> @@ -160,6 +184,41 @@ bool full_write(struct filedes *file, const void *buf, size_t count)
>         return true;
>  }
>
> +static int raw_pwrite(int fd, const void *buf, int count, u64 offset)
> +{
> +#ifdef _WIN32
> +       HANDLE h = (HANDLE)_get_osfhandle(fd);
> +       OVERLAPPED pos = { .Offset = offset, .OffsetHigh = offset >> 32 };
> +       DWORD written = 0;
> +
> +       /* Not exactly the same as pwrite(), but good enough... */
> +       if (!WriteFile(h, buf, count, &written, &pos)) {
> +               errno = EIO;
> +               return -1;
> +       }
> +       return written;
> +#else
> +       return pwrite(fd, buf, count, offset);
> +#endif
> +}
> +
> +bool full_pwrite(struct filedes *file, const void *buf, size_t count,
> +                u64 offset)
> +{
> +       while (count) {
> +               int n = raw_pwrite(file->fd, buf, min(count, INT_MAX), offset);
> +
> +               if (n < 0) {
> +                       error_msg_errno("writing to '%s'", file->name);
> +                       return false;
> +               }
> +               buf += n;
I think this pointer arithmetic is not portable?  Consider changing
the type of buf to "const char*".

> +               count -= n;
> +               offset += n;
> +       }
> +       return true;
> +}
> +
>  bool filedes_close(struct filedes *file)
>  {
>         int res;
> diff --git a/programs/utils.h b/programs/utils.h
> index ab5005f..9a5c97a 100644
> --- a/programs/utils.h
> +++ b/programs/utils.h
> @@ -40,8 +40,11 @@ struct filedes {
>
>  bool open_file(struct filedes *file, const char *filename, int flags, int mode);
>  bool get_file_size(struct filedes *file, u64 *size_ret);
> +bool preallocate_file(struct filedes *file, u64 size);
>  bool full_read(struct filedes *file, void *buf, size_t count);
>  bool full_write(struct filedes *file, const void *buf, size_t count);
> +bool full_pwrite(struct filedes *file, const void *buf, size_t count,
> +                u64 offset);
>  bool filedes_close(struct filedes *file);
>  int read_callback(void *file, void *buf, size_t count);
>
> --
> 2.31.1
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [fsverity-utils PATCH 3/4] programs/utils: add full_pwrite() and preallocate_file()
  2021-06-04  0:33   ` Victor Hsieh
@ 2021-06-04  0:57     ` Eric Biggers
  2021-06-04 15:24       ` Victor Hsieh
  0 siblings, 1 reply; 11+ messages in thread
From: Eric Biggers @ 2021-06-04  0:57 UTC (permalink / raw)
  To: Victor Hsieh; +Cc: linux-fscrypt

On Thu, Jun 03, 2021 at 05:33:18PM -0700, Victor Hsieh wrote:
> > +
> > +bool full_pwrite(struct filedes *file, const void *buf, size_t count,
> > +                u64 offset)
> > +{
> > +       while (count) {
> > +               int n = raw_pwrite(file->fd, buf, min(count, INT_MAX), offset);
> > +
> > +               if (n < 0) {
> > +                       error_msg_errno("writing to '%s'", file->name);
> > +                       return false;
> > +               }
> > +               buf += n;
> I think this pointer arithmetic is not portable?  Consider changing
> the type of buf to "const char*".
> 

fsverity-utils is already using void pointer arithmetic elsewhere, for example
in full_read() and full_write().

I am allowing the use of some gcc/clang extensions which are widely used,
including in the Linux kernel (which fsverity-utils is generally trying to
follow the coding style of), and are annoying to do without.  Void pointer
arithmetic is one of these.

If we really needed to support someone compiling fsverity-utils with e.g.
Visual Studio, we could add -Wpedantic to the compiler flags and get rid of all
the gcc/clang extensions.  But I don't see a reason to do that now.

- Eric

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [fsverity-utils PATCH 3/4] programs/utils: add full_pwrite() and preallocate_file()
  2021-06-04  0:57     ` Eric Biggers
@ 2021-06-04 15:24       ` Victor Hsieh
  2021-06-04 16:55         ` Eric Biggers
  0 siblings, 1 reply; 11+ messages in thread
From: Victor Hsieh @ 2021-06-04 15:24 UTC (permalink / raw)
  To: Eric Biggers; +Cc: linux-fscrypt

On Thu, Jun 3, 2021 at 5:58 PM Eric Biggers <ebiggers@kernel.org> wrote:
>
> On Thu, Jun 03, 2021 at 05:33:18PM -0700, Victor Hsieh wrote:
> > > +
> > > +bool full_pwrite(struct filedes *file, const void *buf, size_t count,
> > > +                u64 offset)
> > > +{
> > > +       while (count) {
> > > +               int n = raw_pwrite(file->fd, buf, min(count, INT_MAX), offset);
> > > +
> > > +               if (n < 0) {
> > > +                       error_msg_errno("writing to '%s'", file->name);
> > > +                       return false;
> > > +               }
> > > +               buf += n;
> > I think this pointer arithmetic is not portable?  Consider changing
> > the type of buf to "const char*".
> >
>
> fsverity-utils is already using void pointer arithmetic elsewhere, for example
> in full_read() and full_write().
>
> I am allowing the use of some gcc/clang extensions which are widely used,
> including in the Linux kernel (which fsverity-utils is generally trying to
> follow the coding style of), and are annoying to do without.  Void pointer
> arithmetic is one of these.
>
> If we really needed to support someone compiling fsverity-utils with e.g.
> Visual Studio, we could add -Wpedantic to the compiler flags and get rid of all
> the gcc/clang extensions.  But I don't see a reason to do that now.

Yeah, that's what I was thinking since the code has #ifdef _WIN32.
I'd think the
"host" side program should be more portable than the kernel itself.
But if this is
already used elsewhere, no objection to keeping assuming so.

Reviewed-by: Victor Hsieh <victorhsieh@google.com>

>
> - Eric

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [fsverity-utils PATCH 0/4] Add option to write Merkle tree to a file
  2021-06-03 19:58 [fsverity-utils PATCH 0/4] Add option to write Merkle tree to a file Eric Biggers
                   ` (3 preceding siblings ...)
  2021-06-03 19:58 ` [fsverity-utils PATCH 4/4] programs/fsverity: add --out-merkle-tree and --out-descriptor options Eric Biggers
@ 2021-06-04 15:25 ` Victor Hsieh
  2021-06-09  6:48 ` Eric Biggers
  5 siblings, 0 replies; 11+ messages in thread
From: Victor Hsieh @ 2021-06-04 15:25 UTC (permalink / raw)
  To: Eric Biggers; +Cc: linux-fscrypt

Reviewed-by: Victor Hsieh <victorhsieh@google.com>

Thanks Eric!

On Thu, Jun 3, 2021 at 1:00 PM Eric Biggers <ebiggers@kernel.org> wrote:
>
> Make 'fsverity digest' and 'fsverity sign' support writing the Merkle
> tree and fs-verity descriptor to files, using new options
> '--out-merkle-tree=FILE' and '--out-descriptor=FILE'.
>
> Normally these new options aren't useful, but they can be needed in
> cases where the fs-verity metadata needs to be consumed by something
> other than one of the native Linux kernel implementations of fs-verity.
>
> This is different from 'fsverity dump_metadata' in that
> 'fsverity dump_metadata' only works on a file with fs-verity enabled,
> whereas these new options are for the userspace file digest computation.
>
> Supporting this required adding some optional callbacks to
> libfsverity_compute_digest().
>
> Eric Biggers (4):
>   lib/compute_digest: add callbacks for getting the verity metadata
>   programs/test_compute_digest: test the metadata callbacks
>   programs/utils: add full_pwrite() and preallocate_file()
>   programs/fsverity: add --out-merkle-tree and --out-descriptor options
>
>  include/libfsverity.h          |  46 +++++++++++-
>  lib/compute_digest.c           | 130 +++++++++++++++++++++++++++-----
>  programs/cmd_digest.c          |   7 +-
>  programs/cmd_sign.c            |  17 +++--
>  programs/fsverity.c            |  88 +++++++++++++++++++++-
>  programs/fsverity.h            |   4 +-
>  programs/test_compute_digest.c | 133 +++++++++++++++++++++++++++++++++
>  programs/utils.c               |  59 +++++++++++++++
>  programs/utils.h               |   3 +
>  9 files changed, 458 insertions(+), 29 deletions(-)
>
>
> base-commit: cf8fa5e5a7ac5b3b2dbfcc87e5dbd5f984c2d83a
> --
> 2.31.1
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [fsverity-utils PATCH 3/4] programs/utils: add full_pwrite() and preallocate_file()
  2021-06-04 15:24       ` Victor Hsieh
@ 2021-06-04 16:55         ` Eric Biggers
  0 siblings, 0 replies; 11+ messages in thread
From: Eric Biggers @ 2021-06-04 16:55 UTC (permalink / raw)
  To: Victor Hsieh; +Cc: linux-fscrypt

On Fri, Jun 04, 2021 at 08:24:50AM -0700, Victor Hsieh wrote:
> On Thu, Jun 3, 2021 at 5:58 PM Eric Biggers <ebiggers@kernel.org> wrote:
> >
> > On Thu, Jun 03, 2021 at 05:33:18PM -0700, Victor Hsieh wrote:
> > > > +
> > > > +bool full_pwrite(struct filedes *file, const void *buf, size_t count,
> > > > +                u64 offset)
> > > > +{
> > > > +       while (count) {
> > > > +               int n = raw_pwrite(file->fd, buf, min(count, INT_MAX), offset);
> > > > +
> > > > +               if (n < 0) {
> > > > +                       error_msg_errno("writing to '%s'", file->name);
> > > > +                       return false;
> > > > +               }
> > > > +               buf += n;
> > > I think this pointer arithmetic is not portable?  Consider changing
> > > the type of buf to "const char*".
> > >
> >
> > fsverity-utils is already using void pointer arithmetic elsewhere, for example
> > in full_read() and full_write().
> >
> > I am allowing the use of some gcc/clang extensions which are widely used,
> > including in the Linux kernel (which fsverity-utils is generally trying to
> > follow the coding style of), and are annoying to do without.  Void pointer
> > arithmetic is one of these.
> >
> > If we really needed to support someone compiling fsverity-utils with e.g.
> > Visual Studio, we could add -Wpedantic to the compiler flags and get rid of all
> > the gcc/clang extensions.  But I don't see a reason to do that now.
> 
> Yeah, that's what I was thinking since the code has #ifdef _WIN32.
> I'd think the
> "host" side program should be more portable than the kernel itself.
> But if this is
> already used elsewhere, no objection to keeping assuming so.
> 
> Reviewed-by: Victor Hsieh <victorhsieh@google.com>
> 

Windows builds are supported with Mingw-w64, not with Visual Studio.  So that
isn't an issue.

- Eric

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [fsverity-utils PATCH 0/4] Add option to write Merkle tree to a file
  2021-06-03 19:58 [fsverity-utils PATCH 0/4] Add option to write Merkle tree to a file Eric Biggers
                   ` (4 preceding siblings ...)
  2021-06-04 15:25 ` [fsverity-utils PATCH 0/4] Add option to write Merkle tree to a file Victor Hsieh
@ 2021-06-09  6:48 ` Eric Biggers
  5 siblings, 0 replies; 11+ messages in thread
From: Eric Biggers @ 2021-06-09  6:48 UTC (permalink / raw)
  To: linux-fscrypt; +Cc: Victor Hsieh

On Thu, Jun 03, 2021 at 12:58:08PM -0700, Eric Biggers wrote:
> Make 'fsverity digest' and 'fsverity sign' support writing the Merkle
> tree and fs-verity descriptor to files, using new options
> '--out-merkle-tree=FILE' and '--out-descriptor=FILE'.
> 
> Normally these new options aren't useful, but they can be needed in
> cases where the fs-verity metadata needs to be consumed by something
> other than one of the native Linux kernel implementations of fs-verity.
> 
> This is different from 'fsverity dump_metadata' in that
> 'fsverity dump_metadata' only works on a file with fs-verity enabled,
> whereas these new options are for the userspace file digest computation.
> 
> Supporting this required adding some optional callbacks to
> libfsverity_compute_digest().
> 
> Eric Biggers (4):
>   lib/compute_digest: add callbacks for getting the verity metadata
>   programs/test_compute_digest: test the metadata callbacks
>   programs/utils: add full_pwrite() and preallocate_file()
>   programs/fsverity: add --out-merkle-tree and --out-descriptor options
> 
>  include/libfsverity.h          |  46 +++++++++++-
>  lib/compute_digest.c           | 130 +++++++++++++++++++++++++++-----
>  programs/cmd_digest.c          |   7 +-
>  programs/cmd_sign.c            |  17 +++--
>  programs/fsverity.c            |  88 +++++++++++++++++++++-
>  programs/fsverity.h            |   4 +-
>  programs/test_compute_digest.c | 133 +++++++++++++++++++++++++++++++++
>  programs/utils.c               |  59 +++++++++++++++
>  programs/utils.h               |   3 +
>  9 files changed, 458 insertions(+), 29 deletions(-)
> 
> 
> base-commit: cf8fa5e5a7ac5b3b2dbfcc87e5dbd5f984c2d83a

All applied.

- Eric

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2021-06-09  6:48 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-03 19:58 [fsverity-utils PATCH 0/4] Add option to write Merkle tree to a file Eric Biggers
2021-06-03 19:58 ` [fsverity-utils PATCH 1/4] lib/compute_digest: add callbacks for getting the verity metadata Eric Biggers
2021-06-03 19:58 ` [fsverity-utils PATCH 2/4] programs/test_compute_digest: test the metadata callbacks Eric Biggers
2021-06-03 19:58 ` [fsverity-utils PATCH 3/4] programs/utils: add full_pwrite() and preallocate_file() Eric Biggers
2021-06-04  0:33   ` Victor Hsieh
2021-06-04  0:57     ` Eric Biggers
2021-06-04 15:24       ` Victor Hsieh
2021-06-04 16:55         ` Eric Biggers
2021-06-03 19:58 ` [fsverity-utils PATCH 4/4] programs/fsverity: add --out-merkle-tree and --out-descriptor options Eric Biggers
2021-06-04 15:25 ` [fsverity-utils PATCH 0/4] Add option to write Merkle tree to a file Victor Hsieh
2021-06-09  6:48 ` Eric Biggers

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.