* krbd + format=2 ?
@ 2013-06-03 9:24 Chris Dunlop
2013-06-07 16:54 ` Alex Elder
0 siblings, 1 reply; 10+ messages in thread
From: Chris Dunlop @ 2013-06-03 9:24 UTC (permalink / raw)
To: ceph-devel
G'day,
Sage's recent pull message to Linus said:
----
Please pull the following Ceph patches from
git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client.git for-linus
This is a big pull. Most of it is culmination of Alex's work to implement
RBD image layering, which is now complete (yay!).
----
Am I correct in thinking "RBD image layering... is now complete"
implies there should be full(?) support for format=2?
I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and it's
letting me map a format=2 image (created under bobtail), however reading
from the block device returns zeros rather than the data. The same image
correctly shows data (NTFS filesystem) when mounted into kvm using librbd.
----
# uname -r
3.10.0-rc4-00010-g0326739
# rbd ls -l
NAME SIZE PARENT FMT PROT LOCK
xxx 1536G 2
# rbd map rbd/xxx
# rbd showmapped
id pool image snap device
1 rbd xxx - /dev/rbd1
# dd if=/dev/rbd1 of=/tmp/xxx count=20480
20480+0 records in
20480+0 records out
10485760 bytes (10 MB) copied, 0.757754 s, 13.8 MB/s
# od -c /tmp/xxx | less
0000000 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
*
50000000
----
Cheers,
Chris
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: krbd + format=2 ?
2013-06-03 9:24 krbd + format=2 ? Chris Dunlop
@ 2013-06-07 16:54 ` Alex Elder
2013-06-08 2:48 ` Chris Dunlop
0 siblings, 1 reply; 10+ messages in thread
From: Alex Elder @ 2013-06-07 16:54 UTC (permalink / raw)
To: Chris Dunlop; +Cc: ceph-devel
On 06/03/2013 04:24 AM, Chris Dunlop wrote:
> G'day,
>
> Sage's recent pull message to Linus said:
>
> ----
> Please pull the following Ceph patches from
>
> git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client.git for-linus
>
> This is a big pull. Most of it is culmination of Alex's work to implement
> RBD image layering, which is now complete (yay!).
> ----
>
> Am I correct in thinking "RBD image layering... is now complete"
> implies there should be full(?) support for format=2?
>
> I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and it's
> letting me map a format=2 image (created under bobtail), however reading
> from the block device returns zeros rather than the data. The same image
> correctly shows data (NTFS filesystem) when mounted into kvm using librbd.
Have you tried using a format 2 image that you created using
the Linux rbd environment? It would be good to know whether
that works for you.
Can you also send me the result of running each of these commands:
foo=$(rados get --pool=rbd rbd_id.xxx - | strings)
rados ls --pool=rbd | grep rbd_data | grep "${foo}"
unset foo
Thanks.
-Alex
>
> ----
> # uname -r
> 3.10.0-rc4-00010-g0326739
> # rbd ls -l
> NAME SIZE PARENT FMT PROT LOCK
> xxx 1536G 2
> # rbd map rbd/xxx
> # rbd showmapped
> id pool image snap device
> 1 rbd xxx - /dev/rbd1
> # dd if=/dev/rbd1 of=/tmp/xxx count=20480
> 20480+0 records in
> 20480+0 records out
> 10485760 bytes (10 MB) copied, 0.757754 s, 13.8 MB/s
> # od -c /tmp/xxx | less
> 0000000 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
> *
> 50000000
> ----
>
>
> Cheers,
>
> Chris
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: krbd + format=2 ?
2013-06-07 16:54 ` Alex Elder
@ 2013-06-08 2:48 ` Chris Dunlop
2013-06-12 4:59 ` Chris Dunlop
0 siblings, 1 reply; 10+ messages in thread
From: Chris Dunlop @ 2013-06-08 2:48 UTC (permalink / raw)
To: Alex Elder; +Cc: ceph-devel
On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote:
> On 06/03/2013 04:24 AM, Chris Dunlop wrote:
>> I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and it's
>> letting me map a format=2 image (created under bobtail), however reading
>> from the block device returns zeros rather than the data. The same image
>> correctly shows data (NTFS filesystem) when mounted into kvm using librbd.
>
> Have you tried using a format 2 image that you created using
> the Linux rbd environment? It would be good to know whether
> that works for you.
Sorry, how to you mean "created using the Linux rbd environment"?
The one I was trying was created using:
rbd create --format 2 xxx --size nnnnn
...then populated using qemu/librbd.
> Can you also send me the result of running each of these commands:
>
> foo=$(rados get --pool=rbd rbd_id.xxx - | strings)
> rados ls --pool=rbd | grep rbd_data | grep "${foo}"
> unset foo
That results in a 3.2MB, 83973-line file, starting like:
rbd_data.e4d72ae8944a.0000000000027066
rbd_data.e4d72ae8944a.000000000001bc77
rbd_data.e4d72ae8944a.000000000002112e
rbd_data.e4d72ae8944a.000000000002550c
rbd_data.e4d72ae8944a.000000000001b6b3
rbd_data.e4d72ae8944a.000000000002875b
Complete file is at: https://www.dropbox.com/s/5bw6ba7ifatf1mu/rbd_data.out
(Note: it's a long w'end here on Oz, I won't be back at this till Tue.)
Cheers,
Chris
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: krbd + format=2 ?
2013-06-08 2:48 ` Chris Dunlop
@ 2013-06-12 4:59 ` Chris Dunlop
2013-06-13 3:56 ` Josh Durgin
0 siblings, 1 reply; 10+ messages in thread
From: Chris Dunlop @ 2013-06-12 4:59 UTC (permalink / raw)
To: Alex Elder; +Cc: ceph-devel
On Sat, Jun 08, 2013 at 12:48:52PM +1000, Chris Dunlop wrote:
> On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote:
>> On 06/03/2013 04:24 AM, Chris Dunlop wrote:
>>> I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and it's
>>> letting me map a format=2 image (created under bobtail), however reading
>>> from the block device returns zeros rather than the data. The same image
>>> correctly shows data (NTFS filesystem) when mounted into kvm using librbd.
>>
>> Have you tried using a format 2 image that you created using
>> the Linux rbd environment? It would be good to know whether
>> that works for you.
>
> Sorry, how to you mean "created using the Linux rbd environment"?
> The one I was trying was created using:
>
> rbd create --format 2 xxx --size nnnnn
>
> ...then populated using qemu/librbd.
Looks like the kernel rbd and librbd aren't compatible, as at
3.10.0-rc4+ceph-client/for-linus@3abef3b vs librbd1 0.56.6-1~bpo70+1.
I can create a format=2 rbd image, map it, format and populate it using the
mapped device. I can then see that data after unmounting unmapping, then
re-mapping and re-mounting. However if I then attach that image via librbd to a
qemu-1.5.0~rc3 vm using the invocation below, the vm sees the whole device as
zeros.
Likewise, if I create a new format=2 rbd image, attach it via librbd to a
qemu-1.5.0~rc3 vm, format it and populate it, that data is thereafter visible
in the vm (e.g. after remounting etc.). But if I map that image using the
kernel client, again I get a device full of zeros.
# dpkg -l \*ceph\* \*rbd\* \*rados\* | grep ^i
ii ceph 0.56.6-1~bpo70+1 amd64 distributed storage and file system
ii ceph-common 0.56.6-1~bpo70+1 amd64 common utilities to mount and interact with a ceph storage cluster
ii librados2 0.56.6-1~bpo70+1 amd64 RADOS distributed object store client library
ii librbd1 0.56.6-1~bpo70+1 amd64 RADOS block device client library
##
## Attach disk to vm
##
# virsh attach-device test <(cat <<END
<disk type='network' device='disk'>
<driver name='qemu' type='raw'/>
<auth username='vm'>
<secret type='ceph' uuid='xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'/>
</auth>
<source protocol='rbd' name='rbd/crd-test'/>
<target dev='vdc' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x7' function='0x0'/>
</disk>
END
)
Cheers,
Chris
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: krbd + format=2 ?
2013-06-12 4:59 ` Chris Dunlop
@ 2013-06-13 3:56 ` Josh Durgin
2013-06-13 6:35 ` Chris Dunlop
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Josh Durgin @ 2013-06-13 3:56 UTC (permalink / raw)
To: Chris Dunlop; +Cc: Alex Elder, ceph-devel
On 06/11/2013 09:59 PM, Chris Dunlop wrote:
> On Sat, Jun 08, 2013 at 12:48:52PM +1000, Chris Dunlop wrote:
>> On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote:
>>> On 06/03/2013 04:24 AM, Chris Dunlop wrote:
>>>> I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and it's
>>>> letting me map a format=2 image (created under bobtail), however reading
>>>> from the block device returns zeros rather than the data. The same image
>>>> correctly shows data (NTFS filesystem) when mounted into kvm using librbd.
>>>
>>> Have you tried using a format 2 image that you created using
>>> the Linux rbd environment? It would be good to know whether
>>> that works for you.
>>
>> Sorry, how to you mean "created using the Linux rbd environment"?
>> The one I was trying was created using:
>>
>> rbd create --format 2 xxx --size nnnnn
>>
>> ...then populated using qemu/librbd.
>
> Looks like the kernel rbd and librbd aren't compatible, as at
> 3.10.0-rc4+ceph-client/for-linus@3abef3b vs librbd1 0.56.6-1~bpo70+1.
Thanks for the detailed report Chris. The kernel client was using the
wrong object names for format 2 (zero-padding them with a different
length than librbd). I just posted a patch fixing this.
Josh
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: krbd + format=2 ?
2013-06-13 3:56 ` Josh Durgin
@ 2013-06-13 6:35 ` Chris Dunlop
2013-06-15 17:01 ` Alex Elder
2013-07-04 14:45 ` Laurent Barbe
2 siblings, 0 replies; 10+ messages in thread
From: Chris Dunlop @ 2013-06-13 6:35 UTC (permalink / raw)
To: Josh Durgin; +Cc: Alex Elder, ceph-devel
On Wed, Jun 12, 2013 at 08:56:50PM -0700, Josh Durgin wrote:
> On 06/11/2013 09:59 PM, Chris Dunlop wrote:
>> Looks like the kernel rbd and librbd aren't compatible, as at
>> 3.10.0-rc4+ceph-client/for-linus@3abef3b vs librbd1 0.56.6-1~bpo70+1.
>
> Thanks for the detailed report Chris. The kernel client was using the
> wrong object names for format 2 (zero-padding them with a different
> length than librbd). I just posted a patch fixing this.
Works for me!
(I had to modify the first of your posted patches to apply it to
current linux master (@26e04462c) which pulled in Sage's
for-linus branch: see below.)
Cheers,
Chris.
----------------------------------------------------------------------
commit 932c369af21c9449649238f1b91d1062ab45384e
Author: Josh Durgin <josh.durgin@inktank.com>
Date: Wed Jun 12 20:10:45 2013 -0700
rbd: fetch object order before using it
rbd_dev_v2_header_onetime() fetches striping information, and
checks whether the image can be read by compariing the stripe unit
to the object size. It determines the object size by shifting
the object order, which is 0 at this point since it has not been
read yet. Move the call to get the image size and object order
before rbd_dev_v2_header_onetime() so it is set before use.
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c
index 3063452..2fecab2 100644
--- a/drivers/block/rbd.c
+++ b/drivers/block/rbd.c
@@ -4239,6 +4239,10 @@ static int rbd_dev_v2_header_info(struct rbd_device *rbd_dev)
down_write(&rbd_dev->header_rwsem);
+ ret = rbd_dev_v2_image_size(rbd_dev);
+ if (ret)
+ goto out;
+
if (first_time) {
ret = rbd_dev_v2_header_onetime(rbd_dev);
if (ret)
@@ -4272,10 +4276,6 @@ static int rbd_dev_v2_header_info(struct rbd_device *rbd_dev)
"is EXPERIMENTAL!");
}
- ret = rbd_dev_v2_image_size(rbd_dev);
- if (ret)
- goto out;
-
if (rbd_dev->spec->snap_id == CEPH_NOSNAP)
if (rbd_dev->mapping.size != rbd_dev->header.image_size)
rbd_dev->mapping.size = rbd_dev->header.image_size;
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: krbd + format=2 ?
2013-06-13 3:56 ` Josh Durgin
2013-06-13 6:35 ` Chris Dunlop
@ 2013-06-15 17:01 ` Alex Elder
2013-07-04 14:45 ` Laurent Barbe
2 siblings, 0 replies; 10+ messages in thread
From: Alex Elder @ 2013-06-15 17:01 UTC (permalink / raw)
To: Josh Durgin; +Cc: Chris Dunlop, Alex Elder, ceph-devel
On 06/12/2013 10:56 PM, Josh Durgin wrote:
> On 06/11/2013 09:59 PM, Chris Dunlop wrote:
>> On Sat, Jun 08, 2013 at 12:48:52PM +1000, Chris Dunlop wrote:
>>> On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote:
>>>> On 06/03/2013 04:24 AM, Chris Dunlop wrote:
>>>>> I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and
>>>>> it's
>>>>> letting me map a format=2 image (created under bobtail), however
>>>>> reading
>>>>> from the block device returns zeros rather than the data. The same
>>>>> image
>>>>> correctly shows data (NTFS filesystem) when mounted into kvm using
>>>>> librbd.
>>>>
>>>> Have you tried using a format 2 image that you created using
>>>> the Linux rbd environment? It would be good to know whether
>>>> that works for you.
>>>
>>> Sorry, how to you mean "created using the Linux rbd environment"?
>>> The one I was trying was created using:
>>>
>>> rbd create --format 2 xxx --size nnnnn
>>>
>>> ...then populated using qemu/librbd.
>>
>> Looks like the kernel rbd and librbd aren't compatible, as at
>> 3.10.0-rc4+ceph-client/for-linus@3abef3b vs librbd1 0.56.6-1~bpo70+1.
>
> Thanks for the detailed report Chris. The kernel client was using the
> wrong object names for format 2 (zero-padding them with a different
> length than librbd). I just posted a patch fixing this.
I remember talking about this Josh. I had suggested we
use a longer one so the full range of values would be
represented by the default name, and as I recall you said
it had already been done on the user space side. I'm
sorry if no bug got opened to be sure that got resolved...
-Alex
> Josh
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: krbd + format=2 ?
2013-06-13 3:56 ` Josh Durgin
2013-06-13 6:35 ` Chris Dunlop
2013-06-15 17:01 ` Alex Elder
@ 2013-07-04 14:45 ` Laurent Barbe
2013-07-05 4:12 ` Sage Weil
2 siblings, 1 reply; 10+ messages in thread
From: Laurent Barbe @ 2013-07-04 14:45 UTC (permalink / raw)
To: Josh Durgin; +Cc: Chris Dunlop, Alex Elder, ceph-devel
Hello,
Since I upgrade kernel 3.10-rc6 to 3.10 final, it seems that the format
of block device has changed and I can't mount them anymore.
I'm using rbd format 2 / xfs.
Are you aware of this incompatibility?
SGI XFS with ACLs, security attributes, realtime, large block/inode ug
enabled
XFS (rbd1): bad magic number
ffff880037241000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .
ffff880037241010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .
ffff880037241020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .
ffff880037241030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .
XFS (rbd1): Internal error xfs_sb_read_verify at line 730 of file t.c.
Caller 0xffffffffa0522d95
CPU: 0 PID: 182 Comm: kworker/0:1H Not tainted 3.10.0-ccmbg1 #1
Hardware name: Dell Computer Corporation PowerEdge 860/0XM089, BIOS
Workqueue: xfslogd xfs_buf_iodone_work [xfs]
ffffffff81362b9c 0000000000000071 ffffffffa0524c61 ffffffffa0522d95
00000000000002da 0000000000000000 0000000000000016 ffff88007cbdde00
ffff88003704c800 ffff88007fc1a500 ffffffffa0566222 ffffffffa0522d95
Call Trace:
[<ffffffff81362b9c>] ? dump_stack+0xd/0x17
[<ffffffffa0524c61>] ? xfs_corruption_error+0x54/0x6f [xfs]
[<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
[<ffffffffa0566222>] ? xfs_sb_read_verify+0xa4/0xbf [xfs]
[<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
[<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
[<ffffffff81046588>] ? process_one_work+0x191/0x28f
[<ffffffff813650f4>] ? __schedule+0x516/0x51b
[<ffffffff81046a35>] ? worker_thread+0x121/0x1e7
[<ffffffff81046914>] ? rescuer_thread+0x269/0x269
[<ffffffff8104aedd>] ? kthread+0x81/0x89
[<ffffffff8104ae5c>] ? __kthread_parkme+0x5d/0x5d
[<ffffffff8136adec>] ? ret_from_fork+0x7c/0xb0
[<ffffffff8104ae5c>] ? __kthread_parkme+0x5d/0x5d
XFS (rbd1): Corruption detected. Unmount and run xfs_repair
XFS (rbd1): SB validate failed with error 22.
Thanks,
Laurent Barbe
Le 13/06/2013 05:56, Josh Durgin a écrit :
> On 06/11/2013 09:59 PM, Chris Dunlop wrote:
>> On Sat, Jun 08, 2013 at 12:48:52PM +1000, Chris Dunlop wrote:
>>> On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote:
>>>> On 06/03/2013 04:24 AM, Chris Dunlop wrote:
>>>>> I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and
>>>>> it's
>>>>> letting me map a format=2 image (created under bobtail), however
>>>>> reading
>>>>> from the block device returns zeros rather than the data. The same
>>>>> image
>>>>> correctly shows data (NTFS filesystem) when mounted into kvm using
>>>>> librbd.
>>>>
>>>> Have you tried using a format 2 image that you created using
>>>> the Linux rbd environment? It would be good to know whether
>>>> that works for you.
>>>
>>> Sorry, how to you mean "created using the Linux rbd environment"?
>>> The one I was trying was created using:
>>>
>>> rbd create --format 2 xxx --size nnnnn
>>>
>>> ...then populated using qemu/librbd.
>>
>> Looks like the kernel rbd and librbd aren't compatible, as at
>> 3.10.0-rc4+ceph-client/for-linus@3abef3b vs librbd1 0.56.6-1~bpo70+1.
>
> Thanks for the detailed report Chris. The kernel client was using the
> wrong object names for format 2 (zero-padding them with a different
> length than librbd). I just posted a patch fixing this.
>
> Josh
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: krbd + format=2 ?
2013-07-04 14:45 ` Laurent Barbe
@ 2013-07-05 4:12 ` Sage Weil
2013-07-05 7:39 ` Laurent Barbe
0 siblings, 1 reply; 10+ messages in thread
From: Sage Weil @ 2013-07-05 4:12 UTC (permalink / raw)
To: Laurent Barbe; +Cc: Josh Durgin, Chris Dunlop, Alex Elder, ceph-devel
On Thu, 4 Jul 2013, Laurent Barbe wrote:
> Hello,
>
> Since I upgrade kernel 3.10-rc6 to 3.10 final, it seems that the format of
> block device has changed and I can't mount them anymore.
> I'm using rbd format 2 / xfs.
>
> Are you aware of this incompatibility?
Hi Laurent,
There was a problem with earlier -rc's not interoperating with librbd
because of the object naming. To recover this image, boot into an -rc6
kernel, copy the block device out of rbd or into a new image (e.g., rbd
import /dev/rbd1 newrbd), and then use the new image with 3.10.
Sorry!
sage
>
>
> SGI XFS with ACLs, security attributes, realtime, large block/inode ug enabled
> XFS (rbd1): bad magic number
> ffff880037241000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .
> ffff880037241010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .
> ffff880037241020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .
> ffff880037241030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .
> XFS (rbd1): Internal error xfs_sb_read_verify at line 730 of file t.c. Caller
> 0xffffffffa0522d95
>
> CPU: 0 PID: 182 Comm: kworker/0:1H Not tainted 3.10.0-ccmbg1 #1
> Hardware name: Dell Computer Corporation PowerEdge 860/0XM089, BIOS
> Workqueue: xfslogd xfs_buf_iodone_work [xfs]
> ffffffff81362b9c 0000000000000071 ffffffffa0524c61 ffffffffa0522d95
> 00000000000002da 0000000000000000 0000000000000016 ffff88007cbdde00
> ffff88003704c800 ffff88007fc1a500 ffffffffa0566222 ffffffffa0522d95
> Call Trace:
> [<ffffffff81362b9c>] ? dump_stack+0xd/0x17
> [<ffffffffa0524c61>] ? xfs_corruption_error+0x54/0x6f [xfs]
> [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
> [<ffffffffa0566222>] ? xfs_sb_read_verify+0xa4/0xbf [xfs]
> [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
> [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
> [<ffffffff81046588>] ? process_one_work+0x191/0x28f
> [<ffffffff813650f4>] ? __schedule+0x516/0x51b
> [<ffffffff81046a35>] ? worker_thread+0x121/0x1e7
> [<ffffffff81046914>] ? rescuer_thread+0x269/0x269
> [<ffffffff8104aedd>] ? kthread+0x81/0x89
> [<ffffffff8104ae5c>] ? __kthread_parkme+0x5d/0x5d
> [<ffffffff8136adec>] ? ret_from_fork+0x7c/0xb0
> [<ffffffff8104ae5c>] ? __kthread_parkme+0x5d/0x5d
> XFS (rbd1): Corruption detected. Unmount and run xfs_repair
> XFS (rbd1): SB validate failed with error 22.
>
> Thanks,
>
> Laurent Barbe
>
>
> Le 13/06/2013 05:56, Josh Durgin a ?crit :
> > On 06/11/2013 09:59 PM, Chris Dunlop wrote:
> > > On Sat, Jun 08, 2013 at 12:48:52PM +1000, Chris Dunlop wrote:
> > > > On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote:
> > > > > On 06/03/2013 04:24 AM, Chris Dunlop wrote:
> > > > > > I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and
> > > > > > it's
> > > > > > letting me map a format=2 image (created under bobtail), however
> > > > > > reading
> > > > > > from the block device returns zeros rather than the data. The same
> > > > > > image
> > > > > > correctly shows data (NTFS filesystem) when mounted into kvm using
> > > > > > librbd.
> > > > >
> > > > > Have you tried using a format 2 image that you created using
> > > > > the Linux rbd environment? It would be good to know whether
> > > > > that works for you.
> > > >
> > > > Sorry, how to you mean "created using the Linux rbd environment"?
> > > > The one I was trying was created using:
> > > >
> > > > rbd create --format 2 xxx --size nnnnn
> > > >
> > > > ...then populated using qemu/librbd.
> > >
> > > Looks like the kernel rbd and librbd aren't compatible, as at
> > > 3.10.0-rc4+ceph-client/for-linus@3abef3b vs librbd1 0.56.6-1~bpo70+1.
> >
> > Thanks for the detailed report Chris. The kernel client was using the
> > wrong object names for format 2 (zero-padding them with a different
> > length than librbd). I just posted a patch fixing this.
> >
> > Josh
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: krbd + format=2 ?
2013-07-05 4:12 ` Sage Weil
@ 2013-07-05 7:39 ` Laurent Barbe
0 siblings, 0 replies; 10+ messages in thread
From: Laurent Barbe @ 2013-07-05 7:39 UTC (permalink / raw)
To: Sage Weil; +Cc: Josh Durgin, Chris Dunlop, Alex Elder, ceph-devel
Hello Sage,
Thank you for your reply,
I will do like this.
Laurent
Le 05/07/2013 06:12, Sage Weil a écrit :
> On Thu, 4 Jul 2013, Laurent Barbe wrote:
>> Hello,
>>
>> Since I upgrade kernel 3.10-rc6 to 3.10 final, it seems that the format of
>> block device has changed and I can't mount them anymore.
>> I'm using rbd format 2 / xfs.
>>
>> Are you aware of this incompatibility?
>
> Hi Laurent,
>
> There was a problem with earlier -rc's not interoperating with librbd
> because of the object naming. To recover this image, boot into an -rc6
> kernel, copy the block device out of rbd or into a new image (e.g., rbd
> import /dev/rbd1 newrbd), and then use the new image with 3.10.
>
> Sorry!
> sage
>
>
>
>>
>>
>> SGI XFS with ACLs, security attributes, realtime, large block/inode ug enabled
>> XFS (rbd1): bad magic number
>> ffff880037241000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .
>> ffff880037241010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .
>> ffff880037241020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .
>> ffff880037241030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .
>> XFS (rbd1): Internal error xfs_sb_read_verify at line 730 of file t.c. Caller
>> 0xffffffffa0522d95
>>
>> CPU: 0 PID: 182 Comm: kworker/0:1H Not tainted 3.10.0-ccmbg1 #1
>> Hardware name: Dell Computer Corporation PowerEdge 860/0XM089, BIOS
>> Workqueue: xfslogd xfs_buf_iodone_work [xfs]
>> ffffffff81362b9c 0000000000000071 ffffffffa0524c61 ffffffffa0522d95
>> 00000000000002da 0000000000000000 0000000000000016 ffff88007cbdde00
>> ffff88003704c800 ffff88007fc1a500 ffffffffa0566222 ffffffffa0522d95
>> Call Trace:
>> [<ffffffff81362b9c>] ? dump_stack+0xd/0x17
>> [<ffffffffa0524c61>] ? xfs_corruption_error+0x54/0x6f [xfs]
>> [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
>> [<ffffffffa0566222>] ? xfs_sb_read_verify+0xa4/0xbf [xfs]
>> [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
>> [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
>> [<ffffffff81046588>] ? process_one_work+0x191/0x28f
>> [<ffffffff813650f4>] ? __schedule+0x516/0x51b
>> [<ffffffff81046a35>] ? worker_thread+0x121/0x1e7
>> [<ffffffff81046914>] ? rescuer_thread+0x269/0x269
>> [<ffffffff8104aedd>] ? kthread+0x81/0x89
>> [<ffffffff8104ae5c>] ? __kthread_parkme+0x5d/0x5d
>> [<ffffffff8136adec>] ? ret_from_fork+0x7c/0xb0
>> [<ffffffff8104ae5c>] ? __kthread_parkme+0x5d/0x5d
>> XFS (rbd1): Corruption detected. Unmount and run xfs_repair
>> XFS (rbd1): SB validate failed with error 22.
>>
>> Thanks,
>>
>> Laurent Barbe
>>
>>
>> Le 13/06/2013 05:56, Josh Durgin a ?crit :
>>> On 06/11/2013 09:59 PM, Chris Dunlop wrote:
>>>> On Sat, Jun 08, 2013 at 12:48:52PM +1000, Chris Dunlop wrote:
>>>>> On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote:
>>>>>> On 06/03/2013 04:24 AM, Chris Dunlop wrote:
>>>>>>> I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and
>>>>>>> it's
>>>>>>> letting me map a format=2 image (created under bobtail), however
>>>>>>> reading
>>>>>>> from the block device returns zeros rather than the data. The same
>>>>>>> image
>>>>>>> correctly shows data (NTFS filesystem) when mounted into kvm using
>>>>>>> librbd.
>>>>>>
>>>>>> Have you tried using a format 2 image that you created using
>>>>>> the Linux rbd environment? It would be good to know whether
>>>>>> that works for you.
>>>>>
>>>>> Sorry, how to you mean "created using the Linux rbd environment"?
>>>>> The one I was trying was created using:
>>>>>
>>>>> rbd create --format 2 xxx --size nnnnn
>>>>>
>>>>> ...then populated using qemu/librbd.
>>>>
>>>> Looks like the kernel rbd and librbd aren't compatible, as at
>>>> 3.10.0-rc4+ceph-client/for-linus@3abef3b vs librbd1 0.56.6-1~bpo70+1.
>>>
>>> Thanks for the detailed report Chris. The kernel client was using the
>>> wrong object names for format 2 (zero-padding them with a different
>>> length than librbd). I just posted a patch fixing this.
>>>
>>> Josh
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2013-07-05 7:39 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-06-03 9:24 krbd + format=2 ? Chris Dunlop
2013-06-07 16:54 ` Alex Elder
2013-06-08 2:48 ` Chris Dunlop
2013-06-12 4:59 ` Chris Dunlop
2013-06-13 3:56 ` Josh Durgin
2013-06-13 6:35 ` Chris Dunlop
2013-06-15 17:01 ` Alex Elder
2013-07-04 14:45 ` Laurent Barbe
2013-07-05 4:12 ` Sage Weil
2013-07-05 7:39 ` Laurent Barbe
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.