[dm-crypt] blkid and aes-cbc-plain/aes-cbc-essiv:sha256

[dm-crypt] blkid and aes-cbc-plain/aes-cbc-essiv:sha256

[.. ink ..]

[-- Attachment #1: Type: text/plain, Size: 1028 bytes --]

when a plain volume is created with "aes-cbc-plain" and opened with
"aes-cbc-essiv:sha256", the blkid program correctly reads the file system
of the volume but mount command fails to mount it. Is this behavior
expected behavior?

I am the developer of zulucrypt, the frontend to cryptsetup and i support
both by first trying to open the volume with "aes-cbc-essiv:sha256" and
then with "aes-cbc-plain" if the former fail.This allows the tool to
seamlessly open "old" plain  volumes in legacy mode

The above works but the mount system call adds ugly looking logs to
/var/syslog when it fails to mount the volume the first time around i am
trying to see if i can get around this.

It would have been great if blkid  would fail to read the file system to
tell me the plain volume is opened in wrong mode but it reads it correctly
leaving it up to mount command to notice the wrong mode and then fail to
mount and add the ugly looking log entry.

can i use any cryptsetup API to distinguish volumes created with those two
cyphers?

[-- Attachment #2: Type: text/html, Size: 1122 bytes --]

Re: [dm-crypt] blkid and aes-cbc-plain/aes-cbc-essiv:sha256

[Milan Broz]

On 03/12/2012 10:30 PM, .. ink .. wrote:
> when a plain volume is created with "aes-cbc-plain" and opened with
> "aes-cbc-essiv:sha256", the blkid program correctly reads the file
> system of the volume but mount command fails to mount it. Is this
> behavior expected behavior?

You have the same cipher (AES) and key, and the same CBC mode.
The only difference is IV generator.

So in reality it means, that only first block (16 bytes) on
plaintext disk will differ. (See how CBC mode operates.)

Of course that mount will fail. But unfortunately, blkid
uses signature to detect filesystem, which is still clearly visible.

> I am the developer of zulucrypt, the frontend to cryptsetup and i
> support both by first trying to open the volume with
> "aes-cbc-essiv:sha256" and then with "aes-cbc-plain" if the former
> fail.This allows the tool to seamlessly open "old" plain  volumes in
> legacy mode

For plain mode, user must provide the cipher, mode and keysize.
Please do not invent autodetection - you just proved it will lead
to data corruption. This is one of the reasons why LUKS was invented,
where cipher and mode is stored in metadata.

> The above works but the mount system call adds ugly looking logs to
> /var/syslog when it fails to mount the volume the first time around i
> am trying to see if i can get around this.
>
> It would have been great if blkid  would fail to read the file system
> to tell me the plain volume is opened in wrong mode but it reads it
> correctly leaving it up to mount command to notice the wrong mode and
> then fail to mount and add the ugly looking log entry.

Blkid has nothing to do with that.

Imagine this situation: overwrite every first 16 bytes
of every sector on disk and it will still detect ext3/4 -
because signature is located elsewhere.

(If we fix ext4 detection - not sure if it even possible - the
problem reappears on different format.)

> can i use any cryptsetup API to distinguish volumes created with
> those two cyphers?

Not for plain mode. There is no way how to check it automatically.
You can find way for one specific case but not a generic rule.
(What if plaintext disk is just random data?)

My suggestion is to ignore "aes-cbc-plain" and just use current default
(and give user option to overwrite it manually).

Milan

Re: [dm-crypt] blkid and aes-cbc-plain/aes-cbc-essiv:sha256

[.. ink ..]

[-- Attachment #1: Type: text/plain, Size: 1556 bytes --]

Thanks for your quick response,


> For plain mode, user must provide the cipher, mode and keysize.
> Please do not invent autodetection - you just proved it will lead
> to data corruption. This is one of the reasons why LUKS was invented,
> where cipher and mode is stored in metadata.
>


>
>> My suggestion is to ignore "aes-cbc-plain" and just use current default
> (and give user option to overwrite it manually).
>
> Milan
>

My tool is meant to be simple, that means the user provides only a path to
encrypted volume and a passphrase .All other options are default and hard
coded and a user can not change them.If a user want to use non default
option then the tool is too simple for them as it will lead to unnecessary
complexity.

Below is what happen when a user provides a path to an encrypted volume and
a passphrase to open a volume

1. The volume is checked to see if its luks or not. If it is, it is then
opened as luks with luks default as they are 1.4.1.

2. If a volume is not luks, it is then assumed to be plain and the first
attempt is to open it with plain defaults as they are in 1.4.1. If that
fail, it is then opened as plain with default values are they were in 1.3.0

I do not see any danger with the current design of supporting both plain
modes as long as,as it currently appear to be, mount can distinguish the
two.  The advise to drop "legacy mode" will make sense if the mount failure
is not a guaranteed way to distinguish the two(.it seem to be working so
far with vfat, ext3/4 file systems). Is this what you are saying?

[-- Attachment #2: Type: text/html, Size: 2277 bytes --]

Re: [dm-crypt] blkid and aes-cbc-plain/aes-cbc-essiv:sha256

[Sven Eschenberg]

What Milan said is this:

It is impossible to distinguish between ESSIV and non ESSIV modes. The
only means of knowing which is right is when the read data seems to be
plausible.

But that's the very problem: what do you consider plausible?

Here's an example, plain and ESSIV look the same (at the relevant area on
disk) and thus a FS signature is found, the FS driver tries to mount the
FS and then detects, that the rest of the FS structures seem to be broken.
It fails. Or it might aswell try to recover the FS and by doing so corrupt
the data irreversibly.

This becomes even more problematic, if the Volume does not host a FS but
something else.

Thus the only sane approach is, to not do any 'educated' guessing, since
it might work in some (or even many) cases, but corrupts significant
amounts of data, when it fails.

(I chose educated guessing, since auto-detection implies that you can make
sure your detection result is valid)

Regards

-Sven



On Tue, March 13, 2012 02:03, .. ink .. wrote:
> Thanks for your quick response,
>
>
>> For plain mode, user must provide the cipher, mode and keysize.
>> Please do not invent autodetection - you just proved it will lead
>> to data corruption. This is one of the reasons why LUKS was invented,
>> where cipher and mode is stored in metadata.
>>
>
>
>>
>>> My suggestion is to ignore "aes-cbc-plain" and just use current default
>> (and give user option to overwrite it manually).
>>
>> Milan
>>
>
> My tool is meant to be simple, that means the user provides only a path to
> encrypted volume and a passphrase .All other options are default and hard
> coded and a user can not change them.If a user want to use non default
> option then the tool is too simple for them as it will lead to unnecessary
> complexity.
>
> Below is what happen when a user provides a path to an encrypted volume
> and
> a passphrase to open a volume
>
> 1. The volume is checked to see if its luks or not. If it is, it is then
> opened as luks with luks default as they are 1.4.1.
>
> 2. If a volume is not luks, it is then assumed to be plain and the first
> attempt is to open it with plain defaults as they are in 1.4.1. If that
> fail, it is then opened as plain with default values are they were in
> 1.3.0
>
> I do not see any danger with the current design of supporting both plain
> modes as long as,as it currently appear to be, mount can distinguish the
> two.  The advise to drop "legacy mode" will make sense if the mount
> failure
> is not a guaranteed way to distinguish the two(.it seem to be working so
> far with vfat, ext3/4 file systems). Is this what you are saying?
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>

Re: [dm-crypt] blkid and aes-cbc-plain/aes-cbc-essiv:sha256

[Arno Wagner]

In fact it is impossible to distinguish ciphers except by looking
whether the decrypted contents makes sense. On the other hand,
looking for structire in the decrypted data works well, but
to distinguish mode in addition to cipher, you have to look
at more than just one cipher block. Ideally look at the whole 
sector. Now, if you know what filesystem is supposed to be in 
there, you can try to recognize its first block. Or you can to
a (sample) entropy analysis for a the whole first block decrypted. 
That would identify the mode quite reliably if (and only if) the 
first block is ensured to have relatively low entropy in 
decrypted state.

That said, I agree that guessing is not the right approach.
What if, for example, there is an old first block 
lying around, but the new crypto-container is at an offset?
In that case not even the cipher or the key might be the same!

Arno



On Tue, Mar 13, 2012 at 03:46:40PM +0100, Sven Eschenberg wrote:
> What Milan said is this:
> 
> It is impossible to distinguish between ESSIV and non ESSIV modes. The
> only means of knowing which is right is when the read data seems to be
> plausible.
> 
> But that's the very problem: what do you consider plausible?
> 
> Here's an example, plain and ESSIV look the same (at the relevant area on
> disk) and thus a FS signature is found, the FS driver tries to mount the
> FS and then detects, that the rest of the FS structures seem to be broken.
> It fails. Or it might aswell try to recover the FS and by doing so corrupt
> the data irreversibly.
> 
> This becomes even more problematic, if the Volume does not host a FS but
> something else.
> 
> Thus the only sane approach is, to not do any 'educated' guessing, since
> it might work in some (or even many) cases, but corrupts significant
> amounts of data, when it fails.
> 
> (I chose educated guessing, since auto-detection implies that you can make
> sure your detection result is valid)
> 
> Regards
> 
> -Sven
> 
> 
> 
> On Tue, March 13, 2012 02:03, .. ink .. wrote:
> > Thanks for your quick response,
> >
> >
> >> For plain mode, user must provide the cipher, mode and keysize.
> >> Please do not invent autodetection - you just proved it will lead
> >> to data corruption. This is one of the reasons why LUKS was invented,
> >> where cipher and mode is stored in metadata.
> >>
> >
> >
> >>
> >>> My suggestion is to ignore "aes-cbc-plain" and just use current default
> >> (and give user option to overwrite it manually).
> >>
> >> Milan
> >>
> >
> > My tool is meant to be simple, that means the user provides only a path to
> > encrypted volume and a passphrase .All other options are default and hard
> > coded and a user can not change them.If a user want to use non default
> > option then the tool is too simple for them as it will lead to unnecessary
> > complexity.
> >
> > Below is what happen when a user provides a path to an encrypted volume
> > and
> > a passphrase to open a volume
> >
> > 1. The volume is checked to see if its luks or not. If it is, it is then
> > opened as luks with luks default as they are 1.4.1.
> >
> > 2. If a volume is not luks, it is then assumed to be plain and the first
> > attempt is to open it with plain defaults as they are in 1.4.1. If that
> > fail, it is then opened as plain with default values are they were in
> > 1.3.0
> >
> > I do not see any danger with the current design of supporting both plain
> > modes as long as,as it currently appear to be, mount can distinguish the
> > two.  The advise to drop "legacy mode" will make sense if the mount
> > failure
> > is not a guaranteed way to distinguish the two(.it seem to be working so
> > far with vfat, ext3/4 file systems). Is this what you are saying?
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> >
> 
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
One of the painful things about our time is that those who feel certainty 
are stupid, and those with any imagination and understanding are filled 
with doubt and indecision. -- Bertrand Russell 

Re: [dm-crypt] blkid and aes-cbc-plain/aes-cbc-essiv:sha256

[.. ink ..]

[-- Attachment #1: Type: text/plain, Size: 1417 bytes --]

> That said, I agree that guessing is not the right approach.
> What if, for example, there is an old first block
> lying around, but the new crypto-container is at an offset?
> In that case not even the cipher or the key might be the same!
>
> Arno
>
>
ok, that makes sense but this raises a fundamental problem with plain
volumes, a problem that must have had a solution since the type was in use
when luks wasnt around.My problem was a specific problem from a general
problem.

Lets say somebody uses cryptsetup tool directly to create a plain mapper
with options they think are appropriate for the volume, how will this
person know they have used the right options? say the right mode or  did
not mistype a character in the passphrase?

If it is known that the volume has a file system, then looking for file
system signature is the most logical thing to do(this is the approach i
took with available tools). If they cant find the signature then they can
assume one or more of the options is wrong. If they find it then they can
assume the options are correct. If this way is wrong, then what is the
right way?

what is the recommended way and what tools are to be used to check if a
created plain mapper is created with proper options?

How did distros back in the day when luks wasnt around and when they were
using plain volumes checked to make sure a user created the plain mapper
with a proper a passphrase?

[-- Attachment #2: Type: text/html, Size: 1643 bytes --]

Re: [dm-crypt] blkid and aes-cbc-plain/aes-cbc-essiv:sha256

[Javier Juan Martínez Cabezón]

I have not enough knowledge about cryptographic questions and maybe this
could be and idiot suggestion from an heretic in this kinds of
questions, but could not be something like bruteforcing tools works (but
reversed)?

They ciphered (one way hashes) what it thinks is the key and then
compare it with the hash present if its the same then this is the key.

Is not feasible to read the first X data (enough to take the superblock)
from the ciphered harddisk and then test the key and options taken from
the user and in the result search for example a superblock. With this
since you don't work with the true harddisk you can't corrupt data, and
would be propably faster.

So, somethink like this: -dd bs=whatever count=whatever
if=/dev/ciphereddisk of=testing
                         -cryptsetup whatever testing

			 -now look for structures:


if filesystem structures then validkey/options
	else error








On 12/03/12 23:36, Milan Broz wrote:
> On 03/12/2012 10:30 PM, .. ink .. wrote:
>> when a plain volume is created with "aes-cbc-plain" and opened with
>> "aes-cbc-essiv:sha256", the blkid program correctly reads the file
>> system of the volume but mount command fails to mount it. Is this
>> behavior expected behavior?
> 
> You have the same cipher (AES) and key, and the same CBC mode.
> The only difference is IV generator.
> 
> So in reality it means, that only first block (16 bytes) on
> plaintext disk will differ. (See how CBC mode operates.)
> 
> Of course that mount will fail. But unfortunately, blkid
> uses signature to detect filesystem, which is still clearly visible.
> 
>> I am the developer of zulucrypt, the frontend to cryptsetup and i
>> support both by first trying to open the volume with
>> "aes-cbc-essiv:sha256" and then with "aes-cbc-plain" if the former
>> fail.This allows the tool to seamlessly open "old" plain  volumes in
>> legacy mode
> 
> For plain mode, user must provide the cipher, mode and keysize.
> Please do not invent autodetection - you just proved it will lead
> to data corruption. This is one of the reasons why LUKS was invented,
> where cipher and mode is stored in metadata.
> 
>> The above works but the mount system call adds ugly looking logs to
>> /var/syslog when it fails to mount the volume the first time around i
>> am trying to see if i can get around this.
>>
>> It would have been great if blkid  would fail to read the file system
>> to tell me the plain volume is opened in wrong mode but it reads it
>> correctly leaving it up to mount command to notice the wrong mode and
>> then fail to mount and add the ugly looking log entry.
> 
> Blkid has nothing to do with that.
> 
> Imagine this situation: overwrite every first 16 bytes
> of every sector on disk and it will still detect ext3/4 -
> because signature is located elsewhere.
> 
> (If we fix ext4 detection - not sure if it even possible - the
> problem reappears on different format.)
> 
>> can i use any cryptsetup API to distinguish volumes created with
>> those two cyphers?
> 
> Not for plain mode. There is no way how to check it automatically.
> You can find way for one specific case but not a generic rule.
> (What if plaintext disk is just random data?)
> 
> My suggestion is to ignore "aes-cbc-plain" and just use current default
> (and give user option to overwrite it manually).
> 
> Milan
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

Re: [dm-crypt] blkid and aes-cbc-plain/aes-cbc-essiv:sha256

[Arno Wagner]

On Tue, Mar 13, 2012 at 12:14:47PM -0400, .. ink .. wrote:
> > That said, I agree that guessing is not the right approach.
> > What if, for example, there is an old first block
> > lying around, but the new crypto-container is at an offset?
> > In that case not even the cipher or the key might be the same!
> >
> > Arno
> >
> >
> ok, that makes sense but this raises a fundamental problem with plain
> volumes, a problem that must have had a solution since the type was in use
> when luks wasnt around.My problem was a specific problem from a general
> problem.


Simple: Put the right stuff into /etc/fstab and maybe /etc/crypttab
and/or mount manually. Not everything needs to be automated.

 
> Lets say somebody uses cryptsetup tool directly to create a plain mapper
> with options they think are appropriate for the volume, how will this
> person know they have used the right options? say the right mode or  did
> not mistype a character in the passphrase?


Simple again: They created the volume, and if they did it with
non-default values, then if mounting fails they have to make
sure passphrase _and_ all other parameters are correct. In 
practice, you use the defaults unless you have a good reason 
not to. If you use something else, you will make sure that 
you remember, e.g. by encapsulating it in a script or writing
it down. Only the passphrase must be kept secret.


> If it is known that the volume has a file system, then looking for file
> system signature is the most logical thing to do(this is the approach i
> took with available tools). If they cant find the signature then they can
> assume one or more of the options is wrong. If they find it then they can
> assume the options are correct. If this way is wrong, then what is the
> right way?


There is no right way. The user is the only one that knows what
is in there. 


> what is the recommended way and what tools are to be used to check if a
> created plain mapper is created with proper options?


Remembering how it was created. There is no other "correct" way.


> How did distros back in the day when luks wasnt around and when they were
> using plain volumes checked to make sure a user created the plain mapper
> with a proper a passphrase?


In those days, distros generally did not help the user with encrypted
volumes. The best available was that the user had to write their
own mapping scripts or maybe cpuld manually make an entry in something
like /etc/crypttab. There certainly was no automatization and there
usually is none today. Or if you look at a recent Ubuntu bug, there
are some attempts at automaization that do more harm than good.

On a meta-layer, what you want to do is take the competent 
system administrator that knows what he/she is doing and how
they configured their system out of the loop. That is a very 
Windows or Mac thing to do, but not a very Unix one. Unix gives
you the tools and lets you decide how you want to use them.
That approach is not very compatible with automating things in 
a default way.

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
One of the painful things about our time is that those who feel certainty 
are stupid, and those with any imagination and understanding are filled 
with doubt and indecision. -- Bertrand Russell 

Re: [dm-crypt] blkid and aes-cbc-plain/aes-cbc-essiv:sha256

[Arno Wagner]

On Tue, Mar 13, 2012 at 06:49:25PM +0100, Javier Juan Mart?nez Cabez?n wrote:
> 
> 
> 
> I have not enough knowledge about cryptographic questions and maybe this
> could be and idiot suggestion from an heretic in this kinds of
> questions, but could not be something like bruteforcing tools works (but
> reversed)?

Yes, as I wrote before. But you can never be sure.
 
> They ciphered (one way hashes) what it thinks is the key and then
> compare it with the hash present if its the same then this is the key.

Ah, no. You decrypt the data and look whether it looks like something
non-random. 

> Is not feasible to read the first X data (enough to take the superblock)
> from the ciphered harddisk and then test the key and options taken from
> the user and in the result search for example a superblock. With this
> since you don't work with the true harddisk you can't corrupt data, and
> would be propably faster.

You could try to mount. But there are still scenarios where you 
could do damage. That is the reason why, e.g. Knoppix mounts
all detected filesystems read-only, just to be sure. Only
the user can know what is right and can be mounted r/w. That
is why there is /etc/fstab.

Arno



> So, somethink like this: -dd bs=whatever count=whatever
> if=/dev/ciphereddisk of=testing
>                          -cryptsetup whatever testing
> 
> 			 -now look for structures:
> 
> 
> if filesystem structures then validkey/options
> 	else error
> 
> 
> 
> 
> 
> 
> 
> 
> On 12/03/12 23:36, Milan Broz wrote:
> > On 03/12/2012 10:30 PM, .. ink .. wrote:
> >> when a plain volume is created with "aes-cbc-plain" and opened with
> >> "aes-cbc-essiv:sha256", the blkid program correctly reads the file
> >> system of the volume but mount command fails to mount it. Is this
> >> behavior expected behavior?
> > 
> > You have the same cipher (AES) and key, and the same CBC mode.
> > The only difference is IV generator.
> > 
> > So in reality it means, that only first block (16 bytes) on
> > plaintext disk will differ. (See how CBC mode operates.)
> > 
> > Of course that mount will fail. But unfortunately, blkid
> > uses signature to detect filesystem, which is still clearly visible.
> > 
> >> I am the developer of zulucrypt, the frontend to cryptsetup and i
> >> support both by first trying to open the volume with
> >> "aes-cbc-essiv:sha256" and then with "aes-cbc-plain" if the former
> >> fail.This allows the tool to seamlessly open "old" plain  volumes in
> >> legacy mode
> > 
> > For plain mode, user must provide the cipher, mode and keysize.
> > Please do not invent autodetection - you just proved it will lead
> > to data corruption. This is one of the reasons why LUKS was invented,
> > where cipher and mode is stored in metadata.
> > 
> >> The above works but the mount system call adds ugly looking logs to
> >> /var/syslog when it fails to mount the volume the first time around i
> >> am trying to see if i can get around this.
> >>
> >> It would have been great if blkid  would fail to read the file system
> >> to tell me the plain volume is opened in wrong mode but it reads it
> >> correctly leaving it up to mount command to notice the wrong mode and
> >> then fail to mount and add the ugly looking log entry.
> > 
> > Blkid has nothing to do with that.
> > 
> > Imagine this situation: overwrite every first 16 bytes
> > of every sector on disk and it will still detect ext3/4 -
> > because signature is located elsewhere.
> > 
> > (If we fix ext4 detection - not sure if it even possible - the
> > problem reappears on different format.)
> > 
> >> can i use any cryptsetup API to distinguish volumes created with
> >> those two cyphers?
> > 
> > Not for plain mode. There is no way how to check it automatically.
> > You can find way for one specific case but not a generic rule.
> > (What if plaintext disk is just random data?)
> > 
> > My suggestion is to ignore "aes-cbc-plain" and just use current default
> > (and give user option to overwrite it manually).
> > 
> > Milan
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
One of the painful things about our time is that those who feel certainty 
are stupid, and those with any imagination and understanding are filled 
with doubt and indecision. -- Bertrand Russell