mkfs.xfs, lvm, multi-terrabyte hardware array and luks

mkfs.xfs, lvm, multi-terrabyte hardware array and luks

[pgf111000]

When I try to format partitions above 2-3gb my opteron experiences heavy io
wait; the mkfs.xfs fails, and I receive the following....

 "mkfs.xfs: libxfs_device_zero write failed: Input/output error"

When I format partions below 2-3gb, there is no problem whatsoever.  I can
mkfs.xfs on a +2-3GB non-luks formated partition without a problem... any
thoughts?


-- 
View this message in context: http://www.nabble.com/mkfs.xfs%2C-lvm%2C-multi-terrabyte-hardware-array-and-luks-tf3262532.html#a9068053
Sent from the linux-xfs mailing list archive at Nabble.com.

Re: mkfs.xfs, lvm, multi-terrabyte hardware array and luks

[Eric Sandeen]

pgf111000 wrote:
> When I try to format partitions above 2-3gb my opteron experiences heavy io
> wait; the mkfs.xfs fails, and I receive the following....
> 
>  "mkfs.xfs: libxfs_device_zero write failed: Input/output error"
> 
> When I format partions below 2-3gb, there is no problem whatsoever.  I can
> mkfs.xfs on a +2-3GB non-luks formated partition without a problem... any
> thoughts?

Sounds like a LUKS problem, maybe it can't do those large offsets?  xfs
certainly can...

I bet you'll find that the 2GB size is the threshold... xfs is just
trying a write():

        if ((bytes = write(fd, z, bytes)) < 0) {
                fprintf(stderr, _("%s: %s write failed: %s\n"),
                        progname, __FUNCTION__, strerror(errno));

maybe try a simple dd write at the end of your large luks device, see
how that goes.

-Eric

Re: [PATCH] mkfs.xfs, lvm, multi-terrabyte hardware array and luks

[pgf111000]

Thank you for the quick response.  I have posted on a few luks forums to try
to delve into this issue a little deeper; if they are aware of a resolution
I'll make sure to post it.  The interesting thing is that when I mkfs.ext3
on luks partitions above 2-3gb all is fine; I wish xfs and luks would play
nice.....


Eric Sandeen-3 wrote:
> 
> pgf111000 wrote:
>> When I try to format partitions above 2-3gb my opteron experiences heavy
>> io
>> wait; the mkfs.xfs fails, and I receive the following....
>> 
>>  "mkfs.xfs: libxfs_device_zero write failed: Input/output error"
>> 
>> When I format partions below 2-3gb, there is no problem whatsoever.  I
>> can
>> mkfs.xfs on a +2-3GB non-luks formated partition without a problem... any
>> thoughts?
> 
> Sounds like a LUKS problem, maybe it can't do those large offsets?  xfs
> certainly can...
> 
> I bet you'll find that the 2GB size is the threshold... xfs is just
> trying a write():
> 
>         if ((bytes = write(fd, z, bytes)) < 0) {
>                 fprintf(stderr, _("%s: %s write failed: %s\n"),
>                         progname, __FUNCTION__, strerror(errno));
> 
> maybe try a simple dd write at the end of your large luks device, see
> how that goes.
> 
> -Eric
> 
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/mkfs.xfs%2C-lvm%2C-multi-terrabyte-hardware-array-and-luks-tf3262532.html#a9069238
Sent from the linux-xfs mailing list archive at Nabble.com.

Re: [PATCH] mkfs.xfs, lvm, multi-terrabyte hardware array and luks

[Eric Sandeen]

pgf111000 wrote:
> Thank you for the quick response.  I have posted on a few luks forums to try
> to delve into this issue a little deeper; if they are aware of a resolution
> I'll make sure to post it.  The interesting thing is that when I mkfs.ext3
> on luks partitions above 2-3gb all is fine; I wish xfs and luks would play
> nice.....

mkfs.xfs actually does writes out at the end of the device and verifies
them; I'm not sure ext3 is doing the same.  You may find yourself in
trouble on ext3 post-mkfs (or you may not...)  I seem to recall that xfs
has shaken out similar problems on other block devices for this reason.

I'd do some simple device-level read/write tests around 2GB just for
fun, see how it goes.

-Eric

Re: [PATCH] mkfs.xfs, lvm, multi-terrabyte hardware array and luks

[pgf111000]

Eric thanks for the help.  It seems that you are likely correct in regard to
the LUKS 2gb threshold.  I am in the process of urandoming (via dd) a large
[20gb] LUKS based ext3 partition to see if the writes fail while less than
20gb... I haven't gotten the results yet... because it takes forever.  If
this fails to fill the 20gb ext3 with random data then it would be fair to
conclude that LUKS has a problem with large partitions [and ext3 has a
problem verifying the underlying partition size], right?

In the meantime I have executed a few trial and error tests (by creating
various sized lv/LUKS partitions and then mkfs.xfs on them); the results
overwhelming suggest that there is a 2gb xfs limitation on LUKS partitions
(probably more politically correct to say it the other way around).  Besides
relying on LUKS forums are there any other resources that you know of
that/who would/could provide a quick solution?

Is there anyway that lvm2 could be contributing to this problem?  Are there
any mkfs.xfs commands that I must issue in this [lvm2/LUKS/Large hardware
raid6 array] environment?  Are there certain properties that must remain
congruent in regard to the raid array, the lvm2 environment, LUKS and/or
XFS?

Again, thanks for your time and help.



pgf111000 wrote:
> 
> Thank you for the quick response.  I have posted on a few luks forums to
> try to delve into this issue a little deeper; if they are aware of a
> resolution I'll make sure to post it.  The interesting thing is that when
> I mkfs.ext3 on luks partitions above 2-3gb all is fine; I wish xfs and
> luks would play nice.....
> 
> 
> Eric Sandeen-3 wrote:
>> 
>> pgf111000 wrote:
>>> When I try to format partitions above 2-3gb my opteron experiences heavy
>>> io
>>> wait; the mkfs.xfs fails, and I receive the following....
>>> 
>>>  "mkfs.xfs: libxfs_device_zero write failed: Input/output error"
>>> 
>>> When I format partions below 2-3gb, there is no problem whatsoever.  I
>>> can
>>> mkfs.xfs on a +2-3GB non-luks formated partition without a problem...
>>> any
>>> thoughts?
>> 
>> Sounds like a LUKS problem, maybe it can't do those large offsets?  xfs
>> certainly can...
>> 
>> I bet you'll find that the 2GB size is the threshold... xfs is just
>> trying a write():
>> 
>>         if ((bytes = write(fd, z, bytes)) < 0) {
>>                 fprintf(stderr, _("%s: %s write failed: %s\n"),
>>                         progname, __FUNCTION__, strerror(errno));
>> 
>> maybe try a simple dd write at the end of your large luks device, see
>> how that goes.
>> 
>> -Eric
>> 
>> 
>> 
>> 
> 
> 

-- 
View this message in context: http://www.nabble.com/mkfs.xfs%2C-lvm%2C-multi-terrabyte-hardware-array-and-luks-tf3262532.html#a9073464
Sent from the linux-xfs mailing list archive at Nabble.com.

Re: [PATCH] mkfs.xfs, lvm, multi-terrabyte hardware array and luks

[pgf111000]

XFS.. HERE I COME!!!!!!!  The LUKS partition was indeed the culprit.  The
LUKS partition was not accurately representing the size of the underlying
logical volume (when over 2gb) automatically; in order to handle partitions
over 2gb it is necessary to explicitly specify the desired size of the LUKS
partition (size<lv) to the LUKS partition itself (via the LUKS "resize"
command).  It seems that mkfs.xfs relies on an accurate LUKS size in order
to work on logical volumes greater than or equal to 2gb.  I updated this
info on the LUKS wiki and other forums...

So just to recap.... if you are getting the error:

"mkfs.xfs: libxfs_device_zero write failed: Input/output error"

make sure that you have resized the underlying LUKS partition to the desired
size.

Thanks for the help; much appreciated.





Eric Sandeen-3 wrote:
> 
> pgf111000 wrote:
>> Thank you for the quick response.  I have posted on a few luks forums to
>> try
>> to delve into this issue a little deeper; if they are aware of a
>> resolution
>> I'll make sure to post it.  The interesting thing is that when I
>> mkfs.ext3
>> on luks partitions above 2-3gb all is fine; I wish xfs and luks would
>> play
>> nice.....
> 
> mkfs.xfs actually does writes out at the end of the device and verifies
> them; I'm not sure ext3 is doing the same.  You may find yourself in
> trouble on ext3 post-mkfs (or you may not...)  I seem to recall that xfs
> has shaken out similar problems on other block devices for this reason.
> 
> I'd do some simple device-level read/write tests around 2GB just for
> fun, see how it goes.
> 
> -Eric
> 
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/mkfs.xfs%2C-lvm%2C-multi-terrabyte-hardware-array-and-luks-tf3262532.html#a9074554
Sent from the linux-xfs mailing list archive at Nabble.com.