[dm-crypt] LUKS linux encrypted file system causing file corruption

[dm-crypt] LUKS linux encrypted file system causing file corruption

[Sumit Kumar]

[-- Attachment #1: Type: text/plain, Size: 527 bytes --]

<http://stackoverflow.com/questions/33993264/luks-linux-encrypted-file-system-causing-file-corruption#>
I am having a LUKS encrypted Linux file system (dmCrypt). When I copy a
huge file, around 5 GB of file, from un-encrypted linux file system to LUKS
encrypted linux file system then the target copied file MD5 changed.

I am not sure does it corrupting the file as the MD5 changed?

I am using CentOS 6.5 and working on VMs not on physical machine.

Any help will be greatly appreciated .

-- 
Thanks & Regards
*Sumit Kumar*

[-- Attachment #2: Type: text/html, Size: 926 bytes --]

Re: [dm-crypt] LUKS linux encrypted file system causing file corruption

[Ralf Ramsauer]

[-- Attachment #1: Type: text/plain, Size: 1557 bytes --]

Hi Sumit,

a change of the checksum implies that the source and destination file
differ. The reason why it differs is another question.
Does the checksum also differ if you copy it from unencrypted to
unencrypted?

First, please check your dmesg log if something suspicious occurs during
the copy process. A hard drive failure might also explain your problem.
As you're using VMs, check the dmesg of the hypervisor machine which
accesses the actual hardware.

Second, ensure that the source file is not changed during the copy
process and also not afterwards when comparing the checksums. Same
applies to the destination. You might want to use 'lsof' to check if
some process is accessing the file.

Just a guess - ...you mentioned VMs and a file of around 5GB... Sounds
like you're trying to copy a VM image, eh?
If so, make sure that your virtualization software does not touch the
file during copy.

Cheers
  Ralf

On 11/30/2015 10:19 AM, Sumit Kumar wrote:
> I am having a LUKS encrypted Linux file system (dmCrypt). When I copy
> a huge file, around 5 GB of file, from un-encrypted linux file system
> to LUKS encrypted linux file system then the target copied file MD5
> changed.
>
> I am not sure does it corrupting the file as the MD5 changed?
>
> I am using CentOS 6.5 and working on VMs not on physical machine.
>
> Any help will be greatly appreciated .
>
>
> -- 
> Thanks & Regards
> *Sumit Kumar*
>
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt


[-- Attachment #2: Type: text/html, Size: 2986 bytes --]

Re: [dm-crypt] LUKS linux encrypted file system causing file corruption

[Arno Wagner]

Hi Sumit,

There are several possibilities. After you have made sure
that noting is chanmging the source during the copying,
the remaning possibilitiy is data corruption.

The first thing is to get an idea how many bytes are 
different.

  cmp -b file1 file2 | wc 

does that. Please run this (may take long) and post the 
results.

One of the usual suspects in data corruption is RAM. 
You ma want to run a memory-check overnight, just to 
be sure. See FAQ Item 4.3 and 4.4:

  https://gitlab.com/cryptsetup/cryptsetup/wikis/FrequentlyAskedQuestions


Regards,
Arno



On Mon, Nov 30, 2015 at 10:19:34 CET, Sumit Kumar wrote:
> <http://stackoverflow.com/questions/33993264/luks-linux-encrypted-file-system-causing-file-corruption#>
> I am having a LUKS encrypted Linux file system (dmCrypt). When I copy a
> huge file, around 5 GB of file, from un-encrypted linux file system to LUKS
> encrypted linux file system then the target copied file MD5 changed.
> 
> I am not sure does it corrupting the file as the MD5 changed?
> 
> I am using CentOS 6.5 and working on VMs not on physical machine.
> 
> Any help will be greatly appreciated .
> 
> -- 
> Thanks & Regards
> *Sumit Kumar*

> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt


-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] LUKS linux encrypted file system causing file corruption

[Ritesh Raj Sarraf]

[-- Attachment #1: Type: text/plain, Size: 2302 bytes --]

Did you get a chance to verify what Ralf and Arno asked for ?

I'd too, be interested to know the findings. In the initial post, you
did not mention what file system you were on ?

Also, interesting is Ralf's point. Are you copying the file while it is
opened by another process ?


Ritesh


On Mon, 2015-11-30 at 13:47 +0100, Ralf Ramsauer wrote:
> Hi Sumit,
> 
> a change of the checksum implies that the source and destination file
> differ. The reason why it differs is another question.
> Does the checksum also differ if you copy it from unencrypted to
> unencrypted?
> 
> First, please check your dmesg log if something suspicious occurs
> during the copy process. A hard drive failure might also explain your
> problem. As you're using VMs, check the dmesg of the hypervisor
> machine which accesses the actual hardware.
> 
> Second, ensure that the source file is not changed during the copy
> process and also not afterwards when comparing the checksums. Same
> applies to the destination. You might want to use 'lsof' to check if
> some process is accessing the file.
> 
> Just a guess - ...you mentioned VMs and a file of around 5GB...
> Sounds like you're trying to copy a VM image, eh?
> If so, make sure that your virtualization software does not touch the
> file during copy.
> 
> Cheers
>   Ralf
> 
> On 11/30/2015 10:19 AM, Sumit Kumar wrote:
> > I am having a LUKS encrypted Linux file system (dmCrypt). When I
> > copy a huge file, around 5 GB of file, from un-encrypted linux file
> > system to LUKS encrypted linux file system then the target copied
> > file MD5 changed.
> > I am not sure does it corrupting the file as the MD5 changed?
> > I am using CentOS 6.5 and working on VMs not on physical machine. 
> > Any help will be greatly appreciated . 
> > 
> > -- 
> > Thanks & Regards
> > Sumit Kumar
> > 
> > 
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
-- 
Given the large number of mailing lists I follow, I request you to CC
me in replies for quicker response


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [dm-crypt] LUKS linux encrypted file system causing file corruption

[Arno Wagner]

I second that. I have some experience with handling large 
data-sets. Doing a verify after copying larger amounts of data 
is one of the better ways to identiify problems with RAM, 
busses and controllers.

Regards,
Arno


On Wed, Dec 02, 2015 at 09:04:10 CET, Ritesh Raj Sarraf wrote:
> Did you get a chance to verify what Ralf and Arno asked for ?
> 
> I'd too, be interested to know the findings. In the initial post, you
> did not mention what file system you were on ?
> 
> Also, interesting is Ralf's point. Are you copying the file while it is
> opened by another process ?
> 
> 
> Ritesh
> 
> 
> On Mon, 2015-11-30 at 13:47 +0100, Ralf Ramsauer wrote:
> > Hi Sumit,
> > 
> > a change of the checksum implies that the source and destination file
> > differ. The reason why it differs is another question.
> > Does the checksum also differ if you copy it from unencrypted to
> > unencrypted?
> > 
> > First, please check your dmesg log if something suspicious occurs
> > during the copy process. A hard drive failure might also explain your
> > problem. As you're using VMs, check the dmesg of the hypervisor
> > machine which accesses the actual hardware.
> > 
> > Second, ensure that the source file is not changed during the copy
> > process and also not afterwards when comparing the checksums. Same
> > applies to the destination. You might want to use 'lsof' to check if
> > some process is accessing the file.
> > 
> > Just a guess - ...you mentioned VMs and a file of around 5GB...
> > Sounds like you're trying to copy a VM image, eh?
> > If so, make sure that your virtualization software does not touch the
> > file during copy.
> > 
> > Cheers
> >   Ralf
> > 
> > On 11/30/2015 10:19 AM, Sumit Kumar wrote:
> > > I am having a LUKS encrypted Linux file system (dmCrypt). When I
> > > copy a huge file, around 5 GB of file, from un-encrypted linux file
> > > system to LUKS encrypted linux file system then the target copied
> > > file MD5 changed.
> > > I am not sure does it corrupting the file as the MD5 changed?
> > > I am using CentOS 6.5 and working on VMs not on physical machine. 
> > > Any help will be greatly appreciated . 
> > > 
> > > -- 
> > > Thanks & Regards
> > > Sumit Kumar
> > > 
> > > 
> > > _______________________________________________
> > > dm-crypt mailing list
> > > dm-crypt@saout.de
> > > http://www.saout.de/mailman/listinfo/dm-crypt
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> -- 
> Given the large number of mailing lists I follow, I request you to CC
> me in replies for quicker response
> 



> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt


-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier