[dm-crypt] misaligned ending sector, 4096-byte luks sector size can't be used

[dm-crypt] misaligned ending sector, 4096-byte luks sector size can't be used

[Chris Murphy]

Hi,

cryptsetup-2.4.3-1.fc36.x86_64

On 512-byte logical sector size, 4096-byte physical sector size
drives, cryptsetup luksFormat uses a 4096-byte LUKS sector size except
when there's an unaligned last sector in a partition, i.e. the
partition ends within a 4096-byte physical sector. Unfortunately this
is pretty common due to the backup GPT, which makes the "last usable"
sector, unaligned. gdisk, fdisk, and parted can all run into this
problem. I can't really tell if this was ever considered by UEFI/GPT
spec writers. The backups GPT is 33 LBAs in size. So it's always
4096-byte unaligned.

All the sectors in the partition are 4096-byte aligned except the last
one. Therefore it seems suboptimal to fall back to 512-byte LUKS
sector size, or for luksFormat --sector-size 4096 to fail. Is there a
way for cryptsetup to just map out the dangling 1-7 512-byte sectors
at the end? They are useless anyway in this case, but the partitioning
tools aren't in a position to know the use case. The last 1-7 sectors
are legitimately individually addressable so it's not incorrect for
the partitioning tool to include them in the last partition on the
device.


-- 
Chris Murphy
_______________________________________________
dm-crypt mailing list -- dm-crypt@saout.de
To unsubscribe send an email to dm-crypt-leave@saout.de

[dm-crypt] Re: misaligned ending sector, 4096-byte luks sector size can't be used

[Milan Broz]

On 24/01/2022 18:07, Chris Murphy wrote:
> Hi,
> 
> cryptsetup-2.4.3-1.fc36.x86_64
> 
> On 512-byte logical sector size, 4096-byte physical sector size
> drives, cryptsetup luksFormat uses a 4096-byte LUKS sector size except
> when there's an unaligned last sector in a partition, i.e. the
> partition ends within a 4096-byte physical sector. Unfortunately this
> is pretty common due to the backup GPT, which makes the "last usable"
> sector, unaligned. gdisk, fdisk, and parted can all run into this
> problem. I can't really tell if this was ever considered by UEFI/GPT
> spec writers. The backups GPT is 33 LBAs in size. So it's always
> 4096-byte unaligned.
> 
> All the sectors in the partition are 4096-byte aligned except the last
> one.

And IMO this is the bug in partition tools. We already discussed this with
util-linux (and fdisk) maintainer, I am not sure if they will fix that.

But anything else would be a horrible hack. If the device is not aligned
to requested sector length (usually 4k), then it cannot be used with that
sector size.

The whole idea of misaligned backup GPT at the end of device is broken.
We should not follow that with adding hacks when the easy solution
is just to align a partition properly.

Years ago there was a general agreement to align partition start to 1 MiB
offset, I think we should do the same for the partition length.

> Therefore it seems suboptimal to fall back to 512-byte LUKS
> sector size, or for luksFormat --sector-size 4096 to fail. Is there a
> way for cryptsetup to just map out the dangling 1-7 512-byte sectors
> at the end? They are useless anyway in this case, but the partitioning
> tools aren't in a position to know the use case. The last 1-7 sectors
> are legitimately individually addressable so it's not incorrect for
> the partitioning tool to include them in the last partition on the
> device.

The default is to not store device length in LUKS header - so it follows
underlying device resize.
If you set sector size to 4k, then the unaligned sectors are no longer
"legitimately" addressable (dm-crypt will set 4k as "physical" sector,
IOW as atomic unit of the device). I understand that some filesystems use
hacks to ignore this, but it is really not a system solution.

Milan
_______________________________________________
dm-crypt mailing list -- dm-crypt@saout.de
To unsubscribe send an email to dm-crypt-leave@saout.de

[dm-crypt] Re: misaligned ending sector, 4096-byte luks sector size can't be used

[Chris Murphy]

On Thu, Jan 27, 2022 at 3:32 PM Chris Murphy <lists@colorremedies.com> wrote:
>
> On Thu, Jan 27, 2022 at 3:29 PM Christoph Anton Mitterer
> <calestyo@scientia.org> wrote:
> >
> > On Thu, 2022-01-27 at 14:21 +0100, Milan Broz wrote:
> > >
> > > And it should be fixed in fdisk now:
> > > https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/commit/?id=921c7da55ec78350e4067b3fd6b7de6f299106ee
> > >
> > > Not sure if parted is maintained these days, but it should follow
> > > this approach...
> >
> >
> > Does anyone know whether gdisk was affected?
>
> It is affected, that's what kicked off this thread - I just forgot to
> include the URL.
> https://bugzilla.redhat.com/show_bug.cgi?id=2044107

Oops! Wrong bug!

https://bugzilla.redhat.com/show_bug.cgi?id=2044108


-- 
Chris Murphy
_______________________________________________
dm-crypt mailing list -- dm-crypt@saout.de
To unsubscribe send an email to dm-crypt-leave@saout.de

[dm-crypt] Re: misaligned ending sector, 4096-byte luks sector size can't be used

[Chris Murphy]

On Thu, Jan 27, 2022 at 3:29 PM Christoph Anton Mitterer
<calestyo@scientia.org> wrote:
>
> On Thu, 2022-01-27 at 14:21 +0100, Milan Broz wrote:
> >
> > And it should be fixed in fdisk now:
> > https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/commit/?id=921c7da55ec78350e4067b3fd6b7de6f299106ee
> >
> > Not sure if parted is maintained these days, but it should follow
> > this approach...
>
>
> Does anyone know whether gdisk was affected?

It is affected, that's what kicked off this thread - I just forgot to
include the URL.
https://bugzilla.redhat.com/show_bug.cgi?id=2044107



-- 
Chris Murphy
_______________________________________________
dm-crypt mailing list -- dm-crypt@saout.de
To unsubscribe send an email to dm-crypt-leave@saout.de

[dm-crypt] Re: misaligned ending sector, 4096-byte luks sector size can't be used

[Christoph Anton Mitterer]

On Thu, 2022-01-27 at 14:21 +0100, Milan Broz wrote:
> 
> And it should be fixed in fdisk now:
> https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/commit/?id=921c7da55ec78350e4067b3fd6b7de6f299106ee
> 
> Not sure if parted is maintained these days, but it should follow
> this approach...


Does anyone know whether gdisk was affected?


Cheers,
Chris.
_______________________________________________
dm-crypt mailing list -- dm-crypt@saout.de
To unsubscribe send an email to dm-crypt-leave@saout.de

[dm-crypt] Re: misaligned ending sector, 4096-byte luks sector size can't be used

[Milan Broz]

On 25/01/2022 13:20, Arno Wagner wrote:
> On Mon, Jan 24, 2022 at 23:10:34 CET, Milan Broz wrote:
>> On 24/01/2022 18:07, Chris Murphy wrote:
> [...]
>> The whole idea of misaligned backup GPT at the end of device is broken.
>> We should not follow that with adding hacks when the easy solution
>> is just to align a partition properly.
> 
> I fully agree to that. It is like having a virtual sector size and
> then selectively in some places only not sticking to it. Not smart
> and a gross KISS violation. And, of course, fundamental problems
> like this one should always be fixed were they are caused.

And it should be fixed in fdisk now:
https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/commit/?id=921c7da55ec78350e4067b3fd6b7de6f299106ee

Not sure if parted is maintained these days, but it should follow this approach...

Milan
_______________________________________________
dm-crypt mailing list -- dm-crypt@saout.de
To unsubscribe send an email to dm-crypt-leave@saout.de

[dm-crypt] Re: misaligned ending sector, 4096-byte luks sector size can't be used

[Arno Wagner]

On Mon, Jan 24, 2022 at 23:10:34 CET, Milan Broz wrote:
> On 24/01/2022 18:07, Chris Murphy wrote:
[...]
> The whole idea of misaligned backup GPT at the end of device is broken.
> We should not follow that with adding hacks when the easy solution
> is just to align a partition properly.

I fully agree to that. It is like having a virtual sector size and 
then selectively in some places only not sticking to it. Not smart 
and a gross KISS violation. And, of course, fundamental problems
like this one should always be fixed were they are caused. Heaping 
hacks and special treatment on top of it just makes the situation a 
lot worse. Better to live with the problem that making it worse.

> Years ago there was a general agreement to align partition start 
> to 1 MiB offset, I think we should do the same for the partition 
> length.

I fully agree to that. The only downside I see to that is that 
you create some spaces that some malware could store information in.
But you have that anyways in lots of places, so it is not really 
a downside. For the case of wiping a disk you should wipe the raw
device anyways, not just partitions.

> > Therefore it seems suboptimal to fall back to 512-byte LUKS
> > sector size, or for luksFormat --sector-size 4096 to fail. Is there a
> > way for cryptsetup to just map out the dangling 1-7 512-byte sectors
> > at the end? They are useless anyway in this case, but the partitioning
> > tools aren't in a position to know the use case. The last 1-7 sectors
> > are legitimately individually addressable so it's not incorrect for
> > the partitioning tool to include them in the last partition on the
> > device.
> 
> The default is to not store device length in LUKS header - so it follows
> underlying device resize.

And that is a really good design. Single source of truth and all that.

> If you set sector size to 4k, then the unaligned sectors are no longer
> "legitimately" addressable (dm-crypt will set 4k as "physical" sector,
> IOW as atomic unit of the device). I understand that some filesystems use
> hacks to ignore this, but it is really not a system solution.

And it is a solution that likely will create hard to understand and 
hard to debug problems at some point in the future. Not good.

Unfortunately, even the Linux OS community has its share of people 
that do not really understand why KISS is so fundamental for all 
good engineering and that do not really consider what happens if 
some things change in the future and hence write non-resilient
stuff or stuff with surprising properties without any real need. 
In established engineering disciplines, this is called "an accident
waiting to happen" and a lot of the engineering education focuses
on avoiding those. In CS/IT/SW-Eng almost all teaching is still 
just on how to make things, not on how to make them well. 
 
Regards,
Arno
-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier
_______________________________________________
dm-crypt mailing list -- dm-crypt@saout.de
To unsubscribe send an email to dm-crypt-leave@saout.de