[PATCH 0/4] e2fsprogs: fix and document the stable_inodes feature

[PATCH 0/4] e2fsprogs: fix and document the stable_inodes feature

[Eric Biggers]

Fix tune2fs to not allow cases where IV_INO_LBLK_64-encrypted files
(which can exist if the stable_inodes feature is set) could be broken:

- Changing the filesystem's UUID
- Clearing the stable_inodes feature

Also document the stable_inodes feature in the appropriate places.

Eric Biggers (4):
  tune2fs: prevent changing UUID of fs with stable_inodes feature
  tune2fs: prevent stable_inodes feature from being cleared
  ext4.5: document the stable_inodes feature
  tune2fs.8: document the stable_inodes feature

 misc/ext4.5.in    | 16 ++++++++++++++++
 misc/tune2fs.8.in |  7 +++++++
 misc/tune2fs.c    | 10 ++++++++--
 3 files changed, 31 insertions(+), 2 deletions(-)

-- 
2.26.0.rc2.310.g2932bb562d-goog

[PATCH 1/4] tune2fs: prevent changing UUID of fs with stable_inodes feature

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

The stable_inodes feature is intended to indicate that it's safe to use
IV_INO_LBLK_64 encryption policies, where the encryption depends on the
inode numbers and thus filesystem shrinking is not allowed.  However
since inode numbers are not unique across filesystems, the encryption
also depends on the filesystem UUID, and I missed that there is a
supported way to change the filesystem UUID (tune2fs -U).

So, make 'tune2fs -U' report an error if stable_inodes is set.

We could add a separate stable_uuid feature flag, but it seems unlikely
it would be useful enough on its own to warrant another flag.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 misc/tune2fs.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/misc/tune2fs.c b/misc/tune2fs.c
index 314cc0d0..ca06c98b 100644
--- a/misc/tune2fs.c
+++ b/misc/tune2fs.c
@@ -3236,6 +3236,13 @@ _("Warning: The journal is dirty. You may wish to replay the journal like:\n\n"
 		char buf[SUPERBLOCK_SIZE] __attribute__ ((aligned(8)));
 		__u8 old_uuid[UUID_SIZE];
 
+		if (ext2fs_has_feature_stable_inodes(fs->super)) {
+			fputs(_("Cannot change the UUID of this filesystem "
+				"because it has the stable_inodes feature "
+				"flag.\n"), stderr);
+			exit(1);
+		}
+
 		if (!ext2fs_has_feature_csum_seed(fs->super) &&
 		    (ext2fs_has_feature_metadata_csum(fs->super) ||
 		     ext2fs_has_feature_ea_inode(fs->super))) {
-- 
2.26.0.rc2.310.g2932bb562d-goog

[PATCH 2/4] tune2fs: prevent stable_inodes feature from being cleared

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

Similar to encrypt and verity, once the stable_inodes feature has been
enabled there may be files anywhere on the filesystem that require this
feature.  Therefore, in general it's unsafe to allow clearing it.  Don't
allow tune2fs to do so.  Like encrypt and verity, it can still be
cleared with debugfs if someone really knows what they're doing.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 misc/tune2fs.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/misc/tune2fs.c b/misc/tune2fs.c
index ca06c98b..81f90cbf 100644
--- a/misc/tune2fs.c
+++ b/misc/tune2fs.c
@@ -181,8 +181,7 @@ static __u32 clear_ok_features[3] = {
 	EXT3_FEATURE_COMPAT_HAS_JOURNAL |
 		EXT2_FEATURE_COMPAT_RESIZE_INODE |
 		EXT2_FEATURE_COMPAT_DIR_INDEX |
-		EXT4_FEATURE_COMPAT_FAST_COMMIT |
-		EXT4_FEATURE_COMPAT_STABLE_INODES,
+		EXT4_FEATURE_COMPAT_FAST_COMMIT,
 	/* Incompat */
 	EXT2_FEATURE_INCOMPAT_FILETYPE |
 		EXT4_FEATURE_INCOMPAT_FLEX_BG |
-- 
2.26.0.rc2.310.g2932bb562d-goog

[PATCH 3/4] ext4.5: document the stable_inodes feature

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 misc/ext4.5.in | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/misc/ext4.5.in b/misc/ext4.5.in
index 1db61a5f..90bc4f88 100644
--- a/misc/ext4.5.in
+++ b/misc/ext4.5.in
@@ -299,6 +299,20 @@ feature is essentially a more extreme version of sparse_super and is
 designed to allow a much larger percentage of the disk to have
 contiguous blocks available for data files.
 .TP
+.B stable_inodes
+.br
+Marks the filesystem's inode numbers and UUID as stable.
+.BR resize2fs (8)
+will not allow shrinking a filesystem with this feature, nor
+will
+.BR tune2fs (8)
+allow changing its UUID.  This feature allows the use of specialized encryption
+settings that make use of the inode numbers and UUID.  Note that the
+.B encrypt
+feature still needs to be enabled separately.
+.B stable_inodes
+is a "compat" feature, so old kernels will allow it.
+.TP
 .B uninit_bg
 .br
 This ext4 file system feature indicates that the block group descriptors
@@ -788,6 +802,8 @@ ext4, 4.13
 ext4, 5.2
 .IP "\fBverity\fR" 2i
 ext4, 5.4
+.IP "\fBstable_inodes\fR" 2i
+ext4, 5.5
 .SH SEE ALSO
 .BR mke2fs (8),
 .BR mke2fs.conf (5),
-- 
2.26.0.rc2.310.g2932bb562d-goog

[PATCH 4/4] tune2fs.8: document the stable_inodes feature

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 misc/tune2fs.8.in | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/misc/tune2fs.8.in b/misc/tune2fs.8.in
index 3cf1f5ed..582d1da5 100644
--- a/misc/tune2fs.8.in
+++ b/misc/tune2fs.8.in
@@ -630,6 +630,13 @@ Limit the number of backup superblocks to save space on large filesystems.
 .B Tune2fs
 currently only supports setting this filesystem feature.
 .TP
+.B stable_inodes
+Prevent the filesystem from being shrunk or having its UUID changed, in order to
+allow the use of specialized encryption settings that make use of the inode
+numbers and UUID.
+.B Tune2fs
+currently only supports setting this filesystem feature.
+.TP
 .B uninit_bg
 Allow the kernel to initialize bitmaps and inode tables lazily, and to
 keep a high watermark for the unused inodes in a filesystem, to reduce
-- 
2.26.0.rc2.310.g2932bb562d-goog

Re: [PATCH 4/4] tune2fs.8: document the stable_inodes feature

[Andreas Dilger]

[-- Attachment #1: Type: text/plain, Size: 1355 bytes --]

On Apr 1, 2020, at 2:32 PM, Eric Biggers <ebiggers@kernel.org> wrote:
> 
> From: Eric Biggers <ebiggers@google.com>
> 
> Signed-off-by: Eric Biggers <ebiggers@google.com>

IMHO, it would be better if the updates to the man pages were in the same
patch as the patch to misc/tune2fs.c.

That said, it's better than *not* getting an update to the man page, so:

Reviewed-by: Andreas Dilger <adilger@dilger.ca>

> ---
> misc/tune2fs.8.in | 7 +++++++
> 1 file changed, 7 insertions(+)
> 
> diff --git a/misc/tune2fs.8.in b/misc/tune2fs.8.in
> index 3cf1f5ed..582d1da5 100644
> --- a/misc/tune2fs.8.in
> +++ b/misc/tune2fs.8.in
> @@ -630,6 +630,13 @@ Limit the number of backup superblocks to save space on large filesystems.
> .B Tune2fs
> currently only supports setting this filesystem feature.
> .TP
> +.B stable_inodes
> +Prevent the filesystem from being shrunk or having its UUID changed, in order to
> +allow the use of specialized encryption settings that make use of the inode
> +numbers and UUID.
> +.B Tune2fs
> +currently only supports setting this filesystem feature.
> +.TP
> .B uninit_bg
> Allow the kernel to initialize bitmaps and inode tables lazily, and to
> keep a high watermark for the unused inodes in a filesystem, to reduce
> --
> 2.26.0.rc2.310.g2932bb562d-goog
> 


Cheers, Andreas






[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 873 bytes --]

Re: [PATCH 1/4] tune2fs: prevent changing UUID of fs with stable_inodes feature

[Andreas Dilger]

[-- Attachment #1: Type: text/plain, Size: 1989 bytes --]

On Apr 1, 2020, at 2:32 PM, Eric Biggers <ebiggers@kernel.org> wrote:
> 
> From: Eric Biggers <ebiggers@google.com>
> 
> The stable_inodes feature is intended to indicate that it's safe to use
> IV_INO_LBLK_64 encryption policies, where the encryption depends on the
> inode numbers and thus filesystem shrinking is not allowed.  However
> since inode numbers are not unique across filesystems, the encryption
> also depends on the filesystem UUID, and I missed that there is a
> supported way to change the filesystem UUID (tune2fs -U).
> 
> So, make 'tune2fs -U' report an error if stable_inodes is set.
> 
> We could add a separate stable_uuid feature flag, but it seems unlikely
> it would be useful enough on its own to warrant another flag.

What about having tune2fs walk the inode table checking for any inodes that
have this flag, and only refusing to clear the flag if it finds any?  That
takes some time on very large filesystems, but since inode table reading is
linear it is reasonable on most filesystems.

Cheers, Andreas

> Signed-off-by: Eric Biggers <ebiggers@google.com>
> ---
> misc/tune2fs.c | 7 +++++++
> 1 file changed, 7 insertions(+)
> 
> diff --git a/misc/tune2fs.c b/misc/tune2fs.c
> index 314cc0d0..ca06c98b 100644
> --- a/misc/tune2fs.c
> +++ b/misc/tune2fs.c
> @@ -3236,6 +3236,13 @@ _("Warning: The journal is dirty. You may wish to replay the journal like:\n\n"
> 		char buf[SUPERBLOCK_SIZE] __attribute__ ((aligned(8)));
> 		__u8 old_uuid[UUID_SIZE];
> 
> +		if (ext2fs_has_feature_stable_inodes(fs->super)) {
> +			fputs(_("Cannot change the UUID of this filesystem "
> +				"because it has the stable_inodes feature "
> +				"flag.\n"), stderr);
> +			exit(1);
> +		}
> +
> 		if (!ext2fs_has_feature_csum_seed(fs->super) &&
> 		    (ext2fs_has_feature_metadata_csum(fs->super) ||
> 		     ext2fs_has_feature_ea_inode(fs->super))) {
> --
> 2.26.0.rc2.310.g2932bb562d-goog
> 


Cheers, Andreas






[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 873 bytes --]

Re: [PATCH 4/4] tune2fs.8: document the stable_inodes feature

[Eric Biggers]

On Wed, Apr 01, 2020 at 08:12:14PM -0600, Andreas Dilger wrote:
> On Apr 1, 2020, at 2:32 PM, Eric Biggers <ebiggers@kernel.org> wrote:
> > 
> > From: Eric Biggers <ebiggers@google.com>
> > 
> > Signed-off-by: Eric Biggers <ebiggers@google.com>
> 
> IMHO, it would be better if the updates to the man pages were in the same
> patch as the patch to misc/tune2fs.c.
> 
> That said, it's better than *not* getting an update to the man page, so:
> 
> Reviewed-by: Andreas Dilger <adilger@dilger.ca>
> 

Well, I should have included a man page update when adding '-O stable_inodes'
originally several months ago.  But now it's just being updated, and it makes
more sense to have separate patches for fixes and new documentation.

- Eric

Re: [PATCH 1/4] tune2fs: prevent changing UUID of fs with stable_inodes feature

[Eric Biggers]

On Wed, Apr 01, 2020 at 08:19:38PM -0600, Andreas Dilger wrote:
> On Apr 1, 2020, at 2:32 PM, Eric Biggers <ebiggers@kernel.org> wrote:
> > 
> > From: Eric Biggers <ebiggers@google.com>
> > 
> > The stable_inodes feature is intended to indicate that it's safe to use
> > IV_INO_LBLK_64 encryption policies, where the encryption depends on the
> > inode numbers and thus filesystem shrinking is not allowed.  However
> > since inode numbers are not unique across filesystems, the encryption
> > also depends on the filesystem UUID, and I missed that there is a
> > supported way to change the filesystem UUID (tune2fs -U).
> > 
> > So, make 'tune2fs -U' report an error if stable_inodes is set.
> > 
> > We could add a separate stable_uuid feature flag, but it seems unlikely
> > it would be useful enough on its own to warrant another flag.
> 
> What about having tune2fs walk the inode table checking for any inodes that
> have this flag, and only refusing to clear the flag if it finds any?  That
> takes some time on very large filesystems, but since inode table reading is
> linear it is reasonable on most filesystems.
> 

I assume you meant to make this comment on patch 2,
"tune2fs: prevent stable_inodes feature from being cleared"?

It's a good suggestion, but it also applies equally to the encrypt, verity,
extents, and ea_inode features.  Currently tune2fs can't clear any of these,
since any inode might be using them.

Note that it would actually be slightly harder to implement your suggestion for
stable_inodes than those four existing features, since clearing stable_inodes
would require reading xattrs rather than just the inode flags.

So if I have time, I can certainly look into allowing tune2fs to clear the
encrypt, verity, extents, stable_inodes, and ea_inode features, by doing an
inode table scan to verify that it's safe.  IMO it doesn't make sense to hold up
this patch on it, though.  This patch just makes stable_inodes work like other
ext4 features.

- Eric

Re: [PATCH 1/4] tune2fs: prevent changing UUID of fs with stable_inodes feature

[Andreas Dilger]

[-- Attachment #1: Type: text/plain, Size: 2576 bytes --]


> On Apr 6, 2020, at 11:32 PM, Eric Biggers <ebiggers@kernel.org> wrote:
> 
> On Wed, Apr 01, 2020 at 08:19:38PM -0600, Andreas Dilger wrote:
>> On Apr 1, 2020, at 2:32 PM, Eric Biggers <ebiggers@kernel.org> wrote:
>>> 
>>> From: Eric Biggers <ebiggers@google.com>
>>> 
>>> The stable_inodes feature is intended to indicate that it's safe to use
>>> IV_INO_LBLK_64 encryption policies, where the encryption depends on the
>>> inode numbers and thus filesystem shrinking is not allowed.  However
>>> since inode numbers are not unique across filesystems, the encryption
>>> also depends on the filesystem UUID, and I missed that there is a
>>> supported way to change the filesystem UUID (tune2fs -U).
>>> 
>>> So, make 'tune2fs -U' report an error if stable_inodes is set.
>>> 
>>> We could add a separate stable_uuid feature flag, but it seems unlikely
>>> it would be useful enough on its own to warrant another flag.
>> 
>> What about having tune2fs walk the inode table checking for any inodes that
>> have this flag, and only refusing to clear the flag if it finds any?  That
>> takes some time on very large filesystems, but since inode table reading is
>> linear it is reasonable on most filesystems.
> 
> I assume you meant to make this comment on patch 2,
> "tune2fs: prevent stable_inodes feature from being cleared"?
> 
> It's a good suggestion, but it also applies equally to the encrypt, verity,
> extents, and ea_inode features.  Currently tune2fs can't clear any of these,
> since any inode might be using them.
> 
> Note that it would actually be slightly harder to implement your suggestion for
> stable_inodes than those four existing features, since clearing stable_inodes
> would require reading xattrs rather than just the inode flags.
> 
> So if I have time, I can certainly look into allowing tune2fs to clear the
> encrypt, verity, extents, stable_inodes, and ea_inode features, by doing an
> inode table scan to verify that it's safe.  IMO it doesn't make sense to hold up
> this patch on it, though.  This patch just makes stable_inodes work like other
> ext4 features.

Sure, I'm OK with this patch, since it avoids accidental breakage.

One question though - for the data checksums it uses s_checksum_seed to generate
checksums, rather than directly using the UUID itself, so that it *is* possible
to change the filesystem UUID after metadata_csum is in use, without the need
to rewrite all of the checksums in the filesystem.  Could the same be done for
stable_inode?

Cheers, Andreas






[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 873 bytes --]

Re: [PATCH 1/4] tune2fs: prevent changing UUID of fs with stable_inodes feature

[Eric Biggers]

On Tue, Apr 07, 2020 at 10:18:55AM -0600, Andreas Dilger wrote:
> 
> > On Apr 6, 2020, at 11:32 PM, Eric Biggers <ebiggers@kernel.org> wrote:
> > 
> > On Wed, Apr 01, 2020 at 08:19:38PM -0600, Andreas Dilger wrote:
> >> On Apr 1, 2020, at 2:32 PM, Eric Biggers <ebiggers@kernel.org> wrote:
> >>> 
> >>> From: Eric Biggers <ebiggers@google.com>
> >>> 
> >>> The stable_inodes feature is intended to indicate that it's safe to use
> >>> IV_INO_LBLK_64 encryption policies, where the encryption depends on the
> >>> inode numbers and thus filesystem shrinking is not allowed.  However
> >>> since inode numbers are not unique across filesystems, the encryption
> >>> also depends on the filesystem UUID, and I missed that there is a
> >>> supported way to change the filesystem UUID (tune2fs -U).
> >>> 
> >>> So, make 'tune2fs -U' report an error if stable_inodes is set.
> >>> 
> >>> We could add a separate stable_uuid feature flag, but it seems unlikely
> >>> it would be useful enough on its own to warrant another flag.
> >> 
> >> What about having tune2fs walk the inode table checking for any inodes that
> >> have this flag, and only refusing to clear the flag if it finds any?  That
> >> takes some time on very large filesystems, but since inode table reading is
> >> linear it is reasonable on most filesystems.
> > 
> > I assume you meant to make this comment on patch 2,
> > "tune2fs: prevent stable_inodes feature from being cleared"?
> > 
> > It's a good suggestion, but it also applies equally to the encrypt, verity,
> > extents, and ea_inode features.  Currently tune2fs can't clear any of these,
> > since any inode might be using them.
> > 
> > Note that it would actually be slightly harder to implement your suggestion for
> > stable_inodes than those four existing features, since clearing stable_inodes
> > would require reading xattrs rather than just the inode flags.
> > 
> > So if I have time, I can certainly look into allowing tune2fs to clear the
> > encrypt, verity, extents, stable_inodes, and ea_inode features, by doing an
> > inode table scan to verify that it's safe.  IMO it doesn't make sense to hold up
> > this patch on it, though.  This patch just makes stable_inodes work like other
> > ext4 features.
> 
> Sure, I'm OK with this patch, since it avoids accidental breakage.
> 
> One question though - for the data checksums it uses s_checksum_seed to generate
> checksums, rather than directly using the UUID itself, so that it *is* possible
> to change the filesystem UUID after metadata_csum is in use, without the need
> to rewrite all of the checksums in the filesystem.  Could the same be done for
> stable_inode?
> 

We could have used s_encrypt_pw_salt, but from a cryptographic perspective I
feel a bit safer using the UUID.  ext4 metadata checksums are non-cryptographic
and for integrity-only, so it's not disastrous if multiple filesystems share the
same s_checksum_seed.  So EXT4_FEATURE_INCOMPAT_CSUM_SEED makes sense as a
usability improvement for people doing things with filesystem cloning.

The new inode-number based encryption is a bit different since it may (depending
on how userspace chooses keys) depend on the per-filesystem ID for cryptographic
purposes.  So it can be much more important that these IDs are really unique.

On this basis, the UUID seems like a better choice since people doing things
with filesystem cloning are more likely to remember to set up the UUIDs as
unique, vs. some "second UUID" that's more hidden and would be forgotten about.

Using s_encrypt_pw_salt would also have been a bit more complex, as we'd have
had to add fscrypt_operations to retrieve it rather than just using s_uuid --
remembering to generate it if unset (mke2fs doesn't set it).  We'd also have
wanted to rename it to something else like s_encrypt_uuid to avoid confusion as
it would no longer be just a password salt.

Anyway, we couldn't really change this now even if we wanted to, since
IV_INO_LBLK_64 encryption policies were already released in v5.5.

- Eric

Re: [PATCH 1/4] tune2fs: prevent changing UUID of fs with stable_inodes feature

[Andreas Dilger]

[-- Attachment #1: Type: text/plain, Size: 4214 bytes --]

On Apr 7, 2020, at 9:11 PM, Eric Biggers <ebiggers@kernel.org> wrote:
> 
> On Tue, Apr 07, 2020 at 10:18:55AM -0600, Andreas Dilger wrote:
>> 
>> One question though - for the data checksums it uses s_checksum_seed
>> to generate checksums, rather than directly using the UUID itself,
>> so that it *is* possible to change the filesystem UUID after
>> metadata_csum is in use, without the need to rewrite all of the
>> checksums in the filesystem.  Could the same be done for stable_inode?
> 
> We could have used s_encrypt_pw_salt, but from a cryptographic perspective I
> feel a bit safer using the UUID.  ext4 metadata checksums are non-cryptographic
> and for integrity-only, so it's not disastrous if multiple filesystems share the
> same s_checksum_seed.  So EXT4_FEATURE_INCOMPAT_CSUM_SEED makes sense as a
> usability improvement for people doing things with filesystem cloning.
> 
> The new inode-number based encryption is a bit different since it may (depending
> on how userspace chooses keys) depend on the per-filesystem ID for cryptographic
> purposes.  So it can be much more important that these IDs are really unique.
> 
> On this basis, the UUID seems like a better choice since people doing things
> with filesystem cloning are more likely to remember to set up the UUIDs as
> unique, vs. some "second UUID" that's more hidden and would be forgotten about.

Actually, I think the opposite is true here.  To avoid usability problems,
users *have* to change the UUID of a cloned/snapshot filesystem to avoid
problems with mount-by-UUID (e.g. either filesystem may be mounted randomly
on each boot, depending on the device enumeration order).  However, if they
try to change the UUID, that would immediately break all of the encrypted
files in the filesystem, so that means with the stable_inode feature either:
- a snapshot/clone of a filesystem may subtly break your system, or
- you can't keep a snapshot/clone of such a filesystem on the same node

> Using s_encrypt_pw_salt would also have been a bit more complex, as we'd have
> had to add fscrypt_operations to retrieve it rather than just using s_uuid --
> remembering to generate it if unset (mke2fs doesn't set it).  We'd also have
> wanted to rename it to something else like s_encrypt_uuid to avoid confusion as
> it would no longer be just a password salt.
> 
> Anyway, we couldn't really change this now even if we wanted to, since
> IV_INO_LBLK_64 encryption policies were already released in v5.5.

I'm not sure I buy these arguments...  We changed handling of metadata_csum
after the fact, by checking at mount if s_checksum_seed is initialized,
otherwise hashing s_uuid and storing if it is zero.  Storing s_checksum_seed
proactively in the kernel and e2fsck allows users to change s_uuid if they
have a new enough kernel without noticing that the checksums were originally
based on s_uuid rather than the hash of it in s_checksum_seed.

I'm not sure of the details of whether s_encrypt_pw_salt is used in the
IV_INO_LBLK_64 case or not (since it uses inode/block number as the salt?),
but I see that the code is already initializing s_encrypt_pw_salt in the
kernel if unset, so that is not hard to do.  It could just make a copy from
s_uuid rather than generating a new UUID for s_encrypt_pw_salt, or for new
filesystems it can generate a unique s_encrypt_pw_salt and only use that?

Storing a feature flag to indicate whether s_uuid or s_encrypt_pw_salt is
used for the IV_INO_LBLK_64 case seems pretty straight forward?  Maybe any
filesystems that are using IV_INO_LBLK_64 with s_uuid can't change the UUID,
but a few bits and lines of code could allow any new filesystem to do so?
If you consider that 5.5 has been out for a few months, there aren't going
to be a lot of users of that approach, vs. the next 10 years or more.

In the end, you are the guy who has to deal with issues here, so I leave it
to you.  I just think it is a problem waiting to happen, and preventing the
users from shooting themselves in the foot with tune2fs doesn't mean that
they won't have significant problems later that could easily be solved now.

Cheers, Andreas






[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 873 bytes --]

Re: [PATCH 1/4] tune2fs: prevent changing UUID of fs with stable_inodes feature

[Theodore Y. Ts'o]

On Fri, Apr 10, 2020 at 05:53:54AM -0600, Andreas Dilger wrote:
> 
> Actually, I think the opposite is true here.  To avoid usability problems,
> users *have* to change the UUID of a cloned/snapshot filesystem to avoid
> problems with mount-by-UUID (e.g. either filesystem may be mounted randomly
> on each boot, depending on the device enumeration order).  However, if they
> try to change the UUID, that would immediately break all of the encrypted
> files in the filesystem, so that means with the stable_inode feature either:
> - a snapshot/clone of a filesystem may subtly break your system, or
> - you can't keep a snapshot/clone of such a filesystem on the same node

I don't think there is any reason why we would use IV_INO_LBLK_64 mode
on anything other than tablet/mobile devices using the latest UFS or
eMMC standards which support in-line crypto engines (ICE).  I'm not
aware of any cloud VM's, private or public, which supports ICE.  And
even if they did, hopefully they would use something more sane than
the UFS/eMMC spec, which only supports 64 bits of IV per I/O request,
and only support a small number of keys that can be loaded into the
hardware.  (This is what you get when you are optimizing Bill of
Materials costs down to a tenth of a cent; a million devices here,
retail store profit margins there, and before you know it you're
talking real money...)

Furthermore, on an modern x86_64, you can do AES encryption at less
than a cycle per CPU clock cycle, and in cloud VM's, battery life is
not a concern, so there really isn't any reason to use or implement
ICE, except maybe as a testing vehicle for fscrypt (e.g., someone
wanting to implement UFS 2.1 in qemu to make it easier to test the
Linux kernel's ICE support).

       	           	       	       	      	      - Ted

Re: [PATCH 0/4] e2fsprogs: fix and document the stable_inodes feature

[Theodore Y. Ts'o]

On Wed, Apr 01, 2020 at 01:32:35PM -0700, Eric Biggers wrote:
> Fix tune2fs to not allow cases where IV_INO_LBLK_64-encrypted files
> (which can exist if the stable_inodes feature is set) could be broken:
> 
> - Changing the filesystem's UUID
> - Clearing the stable_inodes feature
> 
> Also document the stable_inodes feature in the appropriate places.
> 
> Eric Biggers (4):
>   tune2fs: prevent changing UUID of fs with stable_inodes feature
>   tune2fs: prevent stable_inodes feature from being cleared
>   ext4.5: document the stable_inodes feature
>   tune2fs.8: document the stable_inodes feature

Thanks, I've applied this patch series.

						- Ted

Re: [PATCH 1/4] tune2fs: prevent changing UUID of fs with stable_inodes feature

[Eric Biggers]

On Fri, Apr 10, 2020 at 05:53:54AM -0600, Andreas Dilger wrote:
> On Apr 7, 2020, at 9:11 PM, Eric Biggers <ebiggers@kernel.org> wrote:
> > 
> > On Tue, Apr 07, 2020 at 10:18:55AM -0600, Andreas Dilger wrote:
> >> 
> >> One question though - for the data checksums it uses s_checksum_seed
> >> to generate checksums, rather than directly using the UUID itself,
> >> so that it *is* possible to change the filesystem UUID after
> >> metadata_csum is in use, without the need to rewrite all of the
> >> checksums in the filesystem.  Could the same be done for stable_inode?
> > 
> > We could have used s_encrypt_pw_salt, but from a cryptographic perspective I
> > feel a bit safer using the UUID.  ext4 metadata checksums are non-cryptographic
> > and for integrity-only, so it's not disastrous if multiple filesystems share the
> > same s_checksum_seed.  So EXT4_FEATURE_INCOMPAT_CSUM_SEED makes sense as a
> > usability improvement for people doing things with filesystem cloning.
> > 
> > The new inode-number based encryption is a bit different since it may (depending
> > on how userspace chooses keys) depend on the per-filesystem ID for cryptographic
> > purposes.  So it can be much more important that these IDs are really unique.
> > 
> > On this basis, the UUID seems like a better choice since people doing things
> > with filesystem cloning are more likely to remember to set up the UUIDs as
> > unique, vs. some "second UUID" that's more hidden and would be forgotten about.
> 
> Actually, I think the opposite is true here.  To avoid usability problems,
> users *have* to change the UUID of a cloned/snapshot filesystem to avoid
> problems with mount-by-UUID (e.g. either filesystem may be mounted randomly
> on each boot, depending on the device enumeration order).  However, if they
> try to change the UUID, that would immediately break all of the encrypted
> files in the filesystem, so that means with the stable_inode feature either:
> - a snapshot/clone of a filesystem may subtly break your system, or
> - you can't keep a snapshot/clone of such a filesystem on the same node

My concern is about security, not usability.  

If the filesystem IDs used to derive keys for inode-number based encryption
aren't unique, then ciphertext may be repeated across files.  Users wouldn't
notice this since it would be a silent bug, but it would be a cryptographic
vulnerability.  Systems need to be designed in such a way that silent
cryptographic vulnerabilities can't occur, or at least are less likely.

Using the actual UUID rather than a hidden second field encourages people to
keep the IDs unique and is simpler.

The existence of s_encrypt_pw_salt does provide some precedent to use a hidden
second field, but not much since that's intended to be used with per-file keys.
With inode-number based encryption, filesystem ID reuse is a greater concern.

One could validly argue that this is "just a theoretical issue" at the moment,
due to the limited systems on which inode-number based encryption would actually
be used (as Ted and I have described).  But the usability concerns are likewise
theoretical at the moment, for the same reason.

> 
> > Using s_encrypt_pw_salt would also have been a bit more complex, as we'd have
> > had to add fscrypt_operations to retrieve it rather than just using s_uuid --
> > remembering to generate it if unset (mke2fs doesn't set it).  We'd also have
> > wanted to rename it to something else like s_encrypt_uuid to avoid confusion as
> > it would no longer be just a password salt.
> > 
> > Anyway, we couldn't really change this now even if we wanted to, since
> > IV_INO_LBLK_64 encryption policies were already released in v5.5.
> 
> I'm not sure I buy these arguments...  We changed handling of metadata_csum
> after the fact, by checking at mount if s_checksum_seed is initialized,
> otherwise hashing s_uuid and storing if it is zero.  Storing s_checksum_seed
> proactively in the kernel and e2fsck allows users to change s_uuid if they
> have a new enough kernel without noticing that the checksums were originally
> based on s_uuid rather than the hash of it in s_checksum_seed.
> 
> I'm not sure of the details of whether s_encrypt_pw_salt is used in the
> IV_INO_LBLK_64 case or not (since it uses inode/block number as the salt?),
> but I see that the code is already initializing s_encrypt_pw_salt in the
> kernel if unset, so that is not hard to do.  It could just make a copy from
> s_uuid rather than generating a new UUID for s_encrypt_pw_salt, or for new
> filesystems it can generate a unique s_encrypt_pw_salt and only use that?
> 
> Storing a feature flag to indicate whether s_uuid or s_encrypt_pw_salt is
> used for the IV_INO_LBLK_64 case seems pretty straight forward?  Maybe any
> filesystems that are using IV_INO_LBLK_64 with s_uuid can't change the UUID,
> but a few bits and lines of code could allow any new filesystem to do so?
> If you consider that 5.5 has been out for a few months, there aren't going
> to be a lot of users of that approach, vs. the next 10 years or more.
> 
> In the end, you are the guy who has to deal with issues here, so I leave it
> to you.  I just think it is a problem waiting to happen, and preventing the
> users from shooting themselves in the foot with tune2fs doesn't mean that
> they won't have significant problems later that could easily be solved now.
> 

Sure, it wouldn't be *that* hard to add support for using s_encrypt_pw_salt
using a separate filesystem feature flag.  And it could be done after the fact.
My points are just that it would add *some* extra complexity, we already
implemented another approach that is fine for the users who would actually use
it, and the approach we implemented is more cryptographically robust so
switching to the other way wouldn't necessarily be an improvement.

- Eric

Re: [PATCH 0/4] e2fsprogs: fix and document the stable_inodes feature

[Eric Biggers]

On Fri, Apr 10, 2020 at 11:24:06AM -0400, Theodore Y. Ts'o wrote:
> On Wed, Apr 01, 2020 at 01:32:35PM -0700, Eric Biggers wrote:
> > Fix tune2fs to not allow cases where IV_INO_LBLK_64-encrypted files
> > (which can exist if the stable_inodes feature is set) could be broken:
> > 
> > - Changing the filesystem's UUID
> > - Clearing the stable_inodes feature
> > 
> > Also document the stable_inodes feature in the appropriate places.
> > 
> > Eric Biggers (4):
> >   tune2fs: prevent changing UUID of fs with stable_inodes feature
> >   tune2fs: prevent stable_inodes feature from being cleared
> >   ext4.5: document the stable_inodes feature
> >   tune2fs.8: document the stable_inodes feature
> 
> Thanks, I've applied this patch series.
> 

Ted, I still don't see this in git.  Are you planning to push it out?

- Eric

Re: [PATCH 0/4] e2fsprogs: fix and document the stable_inodes feature

[Eric Biggers]

On Thu, May 07, 2020 at 11:18:47AM -0700, Eric Biggers wrote:
> On Fri, Apr 10, 2020 at 11:24:06AM -0400, Theodore Y. Ts'o wrote:
> > On Wed, Apr 01, 2020 at 01:32:35PM -0700, Eric Biggers wrote:
> > > Fix tune2fs to not allow cases where IV_INO_LBLK_64-encrypted files
> > > (which can exist if the stable_inodes feature is set) could be broken:
> > > 
> > > - Changing the filesystem's UUID
> > > - Clearing the stable_inodes feature
> > > 
> > > Also document the stable_inodes feature in the appropriate places.
> > > 
> > > Eric Biggers (4):
> > >   tune2fs: prevent changing UUID of fs with stable_inodes feature
> > >   tune2fs: prevent stable_inodes feature from being cleared
> > >   ext4.5: document the stable_inodes feature
> > >   tune2fs.8: document the stable_inodes feature
> > 
> > Thanks, I've applied this patch series.
> > 
> 
> Ted, I still don't see this in git.  Are you planning to push it out?
> 

Ping?

Re: [PATCH 0/4] e2fsprogs: fix and document the stable_inodes feature

[Eric Biggers]

On Mon, Jun 15, 2020 at 03:22:40PM -0700, Eric Biggers wrote:
> On Thu, May 07, 2020 at 11:18:47AM -0700, Eric Biggers wrote:
> > On Fri, Apr 10, 2020 at 11:24:06AM -0400, Theodore Y. Ts'o wrote:
> > > On Wed, Apr 01, 2020 at 01:32:35PM -0700, Eric Biggers wrote:
> > > > Fix tune2fs to not allow cases where IV_INO_LBLK_64-encrypted files
> > > > (which can exist if the stable_inodes feature is set) could be broken:
> > > > 
> > > > - Changing the filesystem's UUID
> > > > - Clearing the stable_inodes feature
> > > > 
> > > > Also document the stable_inodes feature in the appropriate places.
> > > > 
> > > > Eric Biggers (4):
> > > >   tune2fs: prevent changing UUID of fs with stable_inodes feature
> > > >   tune2fs: prevent stable_inodes feature from being cleared
> > > >   ext4.5: document the stable_inodes feature
> > > >   tune2fs.8: document the stable_inodes feature
> > > 
> > > Thanks, I've applied this patch series.
> > > 
> > 
> > Ted, I still don't see this in git.  Are you planning to push it out?

Ping?

Re: [PATCH 0/4] e2fsprogs: fix and document the stable_inodes feature

[Eric Biggers]

On Mon, Jul 27, 2020 at 09:45:55AM -0700, Eric Biggers wrote:
> On Mon, Jun 15, 2020 at 03:22:40PM -0700, Eric Biggers wrote:
> > On Thu, May 07, 2020 at 11:18:47AM -0700, Eric Biggers wrote:
> > > On Fri, Apr 10, 2020 at 11:24:06AM -0400, Theodore Y. Ts'o wrote:
> > > > On Wed, Apr 01, 2020 at 01:32:35PM -0700, Eric Biggers wrote:
> > > > > Fix tune2fs to not allow cases where IV_INO_LBLK_64-encrypted files
> > > > > (which can exist if the stable_inodes feature is set) could be broken:
> > > > > 
> > > > > - Changing the filesystem's UUID
> > > > > - Clearing the stable_inodes feature
> > > > > 
> > > > > Also document the stable_inodes feature in the appropriate places.
> > > > > 
> > > > > Eric Biggers (4):
> > > > >   tune2fs: prevent changing UUID of fs with stable_inodes feature
> > > > >   tune2fs: prevent stable_inodes feature from being cleared
> > > > >   ext4.5: document the stable_inodes feature
> > > > >   tune2fs.8: document the stable_inodes feature
> > > > 
> > > > Thanks, I've applied this patch series.
> > > > 
> > > 
> > > Ted, I still don't see this in git.  Are you planning to push it out?
> 
> Ping?

Ping.

Re: [PATCH 0/4] e2fsprogs: fix and document the stable_inodes feature

[Eric Biggers]

On Tue, Sep 01, 2020 at 09:19:45AM -0700, Eric Biggers wrote:
> On Mon, Jul 27, 2020 at 09:45:55AM -0700, Eric Biggers wrote:
> > On Mon, Jun 15, 2020 at 03:22:40PM -0700, Eric Biggers wrote:
> > > On Thu, May 07, 2020 at 11:18:47AM -0700, Eric Biggers wrote:
> > > > On Fri, Apr 10, 2020 at 11:24:06AM -0400, Theodore Y. Ts'o wrote:
> > > > > On Wed, Apr 01, 2020 at 01:32:35PM -0700, Eric Biggers wrote:
> > > > > > Fix tune2fs to not allow cases where IV_INO_LBLK_64-encrypted files
> > > > > > (which can exist if the stable_inodes feature is set) could be broken:
> > > > > > 
> > > > > > - Changing the filesystem's UUID
> > > > > > - Clearing the stable_inodes feature
> > > > > > 
> > > > > > Also document the stable_inodes feature in the appropriate places.
> > > > > > 
> > > > > > Eric Biggers (4):
> > > > > >   tune2fs: prevent changing UUID of fs with stable_inodes feature
> > > > > >   tune2fs: prevent stable_inodes feature from being cleared
> > > > > >   ext4.5: document the stable_inodes feature
> > > > > >   tune2fs.8: document the stable_inodes feature
> > > > > 
> > > > > Thanks, I've applied this patch series.
> > > > > 
> > > > 
> > > > Ted, I still don't see this in git.  Are you planning to push it out?
> > 
> > Ping?
> 
> Ping.

Ping.