Ceph doesn't update the block device size while a rbd image is mounted

Ceph doesn't update the block device size while a rbd image is mounted

[Sébastien Han]

Hi Cephers!

I'm working with rbd mapping. I figured out that the block device size
of the rbd device is not update while the device is mounted. Here my
tests:

1. Pick up a device and check his size

# rbd ls
size

# rbd info test
rbd image 'test':
size 10000 MB in 2500 objects
order 22 (4096 KB objects)
block_name_prefix: rb.0.6
parent:  (pool -1)

2. Map the device

# rbd map --secret /etc/ceph/secret test
# rbd showmapped
id pool image snap device
1 rbd test - /dev/rbd1

3. Put a fs on it and check the block device size

# mkfs.ext4 /dev/rdb1
...
...

# fdisk -l /dev/rbd1

Disk /dev/rbd1: 10.5 GB, 10485760000 bytes

4. Mount it

# mount /dev/rbd1 /mnt
# df -h
/dev/rbd1                   9.8G  277M  9.0G   3% /mnt


5. Change the image size

# rbd resize --size 11000 test
Resizing image: 100% complete...done.

# rbd info test
rbd image 'test':
size 11000 MB in 2750 objects
order 22 (4096 KB objects)
block_name_prefix: rb.0.6
parent:  (pool -1)

At this point of time, if you perform the fdisk -l /dev/rbd1, the
block device size will remain the same.

6. Unmount the device:

# umount /media

# fdisk -l /dev/rbd1
Disk /dev/rbd1: 11.5 GB, 11534336000 bytes

Unmounting the directory did update the block device size.

Of course you can do something really fast like:

# umount /media && mount /dev/rbd1 /media

That will work, it's a valid solution as long as there is no opened
file. I won't use this trick in production...

I also tried to "mount -o remount" and it didn't work.

7. Resize the fs (this can be performed while the fs is mounted):

# e2fsck -f /dev/rbd1
e2fsck 1.42 (29-Nov-2011)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks

# resize2fs /dev/rbd1
resize2fs 1.42 (29-Nov-2011)
Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks.
The filesystem on /dev/rbd1 is now 2816000 blocks long.


Did I miss something?
Is this feature coming?

Thank you in advance :)

Re: Ceph doesn't update the block device size while a rbd image is mounted

[Wido den Hollander]

Hi,

On 19-07-12 16:55, Sébastien Han wrote:
> Hi Cephers!
>
> I'm working with rbd mapping. I figured out that the block device size
> of the rbd device is not update while the device is mounted. Here my
> tests:
>

iirc this is not something RBD specific, but since the device is in use 
it can't be re-read by the kernel.

So when you unmount it the kernel can re-read the header and resize the 
device.

Wido

> 1. Pick up a device and check his size
>
> # rbd ls
> size
>
> # rbd info test
> rbd image 'test':
> size 10000 MB in 2500 objects
> order 22 (4096 KB objects)
> block_name_prefix: rb.0.6
> parent:  (pool -1)
>
> 2. Map the device
>
> # rbd map --secret /etc/ceph/secret test
> # rbd showmapped
> id pool image snap device
> 1 rbd test - /dev/rbd1
>
> 3. Put a fs on it and check the block device size
>
> # mkfs.ext4 /dev/rdb1
> ...
> ...
>
> # fdisk -l /dev/rbd1
>
> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes
>
> 4. Mount it
>
> # mount /dev/rbd1 /mnt
> # df -h
> /dev/rbd1                   9.8G  277M  9.0G   3% /mnt
>
>
> 5. Change the image size
>
> # rbd resize --size 11000 test
> Resizing image: 100% complete...done.
>
> # rbd info test
> rbd image 'test':
> size 11000 MB in 2750 objects
> order 22 (4096 KB objects)
> block_name_prefix: rb.0.6
> parent:  (pool -1)
>
> At this point of time, if you perform the fdisk -l /dev/rbd1, the
> block device size will remain the same.
>
> 6. Unmount the device:
>
> # umount /media
>
> # fdisk -l /dev/rbd1
> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes
>
> Unmounting the directory did update the block device size.
>
> Of course you can do something really fast like:
>
> # umount /media && mount /dev/rbd1 /media
>
> That will work, it's a valid solution as long as there is no opened
> file. I won't use this trick in production...
>
> I also tried to "mount -o remount" and it didn't work.
>
> 7. Resize the fs (this can be performed while the fs is mounted):
>
> # e2fsck -f /dev/rbd1
> e2fsck 1.42 (29-Nov-2011)
> Pass 1: Checking inodes, blocks, and sizes
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks
>
> # resize2fs /dev/rbd1
> resize2fs 1.42 (29-Nov-2011)
> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks.
> The filesystem on /dev/rbd1 is now 2816000 blocks long.
>
>
> Did I miss something?
> Is this feature coming?
>
> Thank you in advance :)
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Ceph doesn't update the block device size while a rbd image is mounted

[Sébastien Han]

Ok I got your point seems logic, but why is this possible with LVM for example?

You can easily do this with LVM without un-mounting the device.

Cheers.


On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl> wrote:
> Hi,
>
>
> On 19-07-12 16:55, Sébastien Han wrote:
>>
>> Hi Cephers!
>>
>> I'm working with rbd mapping. I figured out that the block device size
>> of the rbd device is not update while the device is mounted. Here my
>> tests:
>>
>
> iirc this is not something RBD specific, but since the device is in use it
> can't be re-read by the kernel.
>
> So when you unmount it the kernel can re-read the header and resize the
> device.
>
> Wido
>
>> 1. Pick up a device and check his size
>>
>> # rbd ls
>> size
>>
>> # rbd info test
>> rbd image 'test':
>> size 10000 MB in 2500 objects
>> order 22 (4096 KB objects)
>> block_name_prefix: rb.0.6
>> parent:  (pool -1)
>>
>> 2. Map the device
>>
>> # rbd map --secret /etc/ceph/secret test
>> # rbd showmapped
>> id pool image snap device
>> 1 rbd test - /dev/rbd1
>>
>> 3. Put a fs on it and check the block device size
>>
>> # mkfs.ext4 /dev/rdb1
>> ...
>> ...
>>
>> # fdisk -l /dev/rbd1
>>
>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes
>>
>> 4. Mount it
>>
>> # mount /dev/rbd1 /mnt
>> # df -h
>> /dev/rbd1                   9.8G  277M  9.0G   3% /mnt
>>
>>
>> 5. Change the image size
>>
>> # rbd resize --size 11000 test
>> Resizing image: 100% complete...done.
>>
>> # rbd info test
>> rbd image 'test':
>> size 11000 MB in 2750 objects
>> order 22 (4096 KB objects)
>> block_name_prefix: rb.0.6
>> parent:  (pool -1)
>>
>> At this point of time, if you perform the fdisk -l /dev/rbd1, the
>> block device size will remain the same.
>>
>> 6. Unmount the device:
>>
>> # umount /media
>>
>> # fdisk -l /dev/rbd1
>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes
>>
>> Unmounting the directory did update the block device size.
>>
>> Of course you can do something really fast like:
>>
>> # umount /media && mount /dev/rbd1 /media
>>
>> That will work, it's a valid solution as long as there is no opened
>> file. I won't use this trick in production...
>>
>> I also tried to "mount -o remount" and it didn't work.
>>
>> 7. Resize the fs (this can be performed while the fs is mounted):
>>
>> # e2fsck -f /dev/rbd1
>> e2fsck 1.42 (29-Nov-2011)
>> Pass 1: Checking inodes, blocks, and sizes
>> Pass 2: Checking directory structure
>> Pass 3: Checking directory connectivity
>> Pass 4: Checking reference counts
>> Pass 5: Checking group summary information
>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks
>>
>> # resize2fs /dev/rbd1
>> resize2fs 1.42 (29-Nov-2011)
>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks.
>> The filesystem on /dev/rbd1 is now 2816000 blocks long.
>>
>>
>> Did I miss something?
>> Is this feature coming?
>>
>> Thank you in advance :)
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Ceph doesn't update the block device size while a rbd image is mounted

[Tommi Virtanen]

On Thu, Jul 19, 2012 at 8:26 AM, Sébastien Han <han.sebastien@gmail.com> wrote:
> Ok I got your point seems logic, but why is this possible with LVM for example?
>
> You can easily do this with LVM without un-mounting the device.

Do your LVM volumes have partition tables inside them? That might be
the difference.

Of course, you can put your filesystem straight on the RBD; that would
be a good experiment to run.
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Ceph doesn't update the block device size while a rbd image is mounted

[Tommi Virtanen]

On Thu, Jul 19, 2012 at 8:38 AM, Tommi Virtanen <tv@inktank.com> wrote:
> Do your LVM volumes have partition tables inside them? That might be
> the difference.
>
> Of course, you can put your filesystem straight on the RBD; that would
> be a good experiment to run.

Oops, I see you did put your fs straight on the RBD already; seeing
"fdisk" tripped my brain.

Re: Ceph doesn't update the block device size while a rbd image is mounted

[Wido den Hollander]

On 19-07-12 17:26, Sébastien Han wrote:
> Ok I got your point seems logic, but why is this possible with LVM for example?
>
> You can easily do this with LVM without un-mounting the device.
>


LVM runs through the device mapper and are not regular block devices.

If you resize the disk underneath LVM you won't see an increased VG or 
PV size unless you change the availability of the VG to unavailable and 
back to available again.

I'm not a 100% sure what the exact root cause is, but the kernel won't 
read the new size of a block device as long as it is in use.

Wido


> Cheers.
>
>
> On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl> wrote:
>> Hi,
>>
>>
>> On 19-07-12 16:55, Sébastien Han wrote:
>>>
>>> Hi Cephers!
>>>
>>> I'm working with rbd mapping. I figured out that the block device size
>>> of the rbd device is not update while the device is mounted. Here my
>>> tests:
>>>
>>
>> iirc this is not something RBD specific, but since the device is in use it
>> can't be re-read by the kernel.
>>
>> So when you unmount it the kernel can re-read the header and resize the
>> device.
>>
>> Wido
>>
>>> 1. Pick up a device and check his size
>>>
>>> # rbd ls
>>> size
>>>
>>> # rbd info test
>>> rbd image 'test':
>>> size 10000 MB in 2500 objects
>>> order 22 (4096 KB objects)
>>> block_name_prefix: rb.0.6
>>> parent:  (pool -1)
>>>
>>> 2. Map the device
>>>
>>> # rbd map --secret /etc/ceph/secret test
>>> # rbd showmapped
>>> id pool image snap device
>>> 1 rbd test - /dev/rbd1
>>>
>>> 3. Put a fs on it and check the block device size
>>>
>>> # mkfs.ext4 /dev/rdb1
>>> ...
>>> ...
>>>
>>> # fdisk -l /dev/rbd1
>>>
>>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes
>>>
>>> 4. Mount it
>>>
>>> # mount /dev/rbd1 /mnt
>>> # df -h
>>> /dev/rbd1                   9.8G  277M  9.0G   3% /mnt
>>>
>>>
>>> 5. Change the image size
>>>
>>> # rbd resize --size 11000 test
>>> Resizing image: 100% complete...done.
>>>
>>> # rbd info test
>>> rbd image 'test':
>>> size 11000 MB in 2750 objects
>>> order 22 (4096 KB objects)
>>> block_name_prefix: rb.0.6
>>> parent:  (pool -1)
>>>
>>> At this point of time, if you perform the fdisk -l /dev/rbd1, the
>>> block device size will remain the same.
>>>
>>> 6. Unmount the device:
>>>
>>> # umount /media
>>>
>>> # fdisk -l /dev/rbd1
>>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes
>>>
>>> Unmounting the directory did update the block device size.
>>>
>>> Of course you can do something really fast like:
>>>
>>> # umount /media && mount /dev/rbd1 /media
>>>
>>> That will work, it's a valid solution as long as there is no opened
>>> file. I won't use this trick in production...
>>>
>>> I also tried to "mount -o remount" and it didn't work.
>>>
>>> 7. Resize the fs (this can be performed while the fs is mounted):
>>>
>>> # e2fsck -f /dev/rbd1
>>> e2fsck 1.42 (29-Nov-2011)
>>> Pass 1: Checking inodes, blocks, and sizes
>>> Pass 2: Checking directory structure
>>> Pass 3: Checking directory connectivity
>>> Pass 4: Checking reference counts
>>> Pass 5: Checking group summary information
>>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks
>>>
>>> # resize2fs /dev/rbd1
>>> resize2fs 1.42 (29-Nov-2011)
>>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks.
>>> The filesystem on /dev/rbd1 is now 2816000 blocks long.
>>>
>>>
>>> Did I miss something?
>>> Is this feature coming?
>>>
>>> Thank you in advance :)
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Ceph doesn't update the block device size while a rbd image is mounted

[Sébastien Han]

Hum ok, I see. Thanks!

But if you have any clue to force the kernel to re-read without
unmont/mounting :)

On Thu, Jul 19, 2012 at 5:47 PM, Wido den Hollander <wido@widodh.nl> wrote:
>
>
> On 19-07-12 17:26, Sébastien Han wrote:
>>
>> Ok I got your point seems logic, but why is this possible with LVM for
>> example?
>>
>> You can easily do this with LVM without un-mounting the device.
>>
>
>
> LVM runs through the device mapper and are not regular block devices.
>
> If you resize the disk underneath LVM you won't see an increased VG or PV
> size unless you change the availability of the VG to unavailable and back to
> available again.
>
> I'm not a 100% sure what the exact root cause is, but the kernel won't read
> the new size of a block device as long as it is in use.
>
> Wido
>
>
>
>> Cheers.
>>
>>
>> On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl>
>> wrote:
>>>
>>> Hi,
>>>
>>>
>>> On 19-07-12 16:55, Sébastien Han wrote:
>>>>
>>>>
>>>> Hi Cephers!
>>>>
>>>> I'm working with rbd mapping. I figured out that the block device size
>>>> of the rbd device is not update while the device is mounted. Here my
>>>> tests:
>>>>
>>>
>>> iirc this is not something RBD specific, but since the device is in use
>>> it
>>> can't be re-read by the kernel.
>>>
>>> So when you unmount it the kernel can re-read the header and resize the
>>> device.
>>>
>>> Wido
>>>
>>>> 1. Pick up a device and check his size
>>>>
>>>> # rbd ls
>>>> size
>>>>
>>>> # rbd info test
>>>> rbd image 'test':
>>>> size 10000 MB in 2500 objects
>>>> order 22 (4096 KB objects)
>>>> block_name_prefix: rb.0.6
>>>> parent:  (pool -1)
>>>>
>>>> 2. Map the device
>>>>
>>>> # rbd map --secret /etc/ceph/secret test
>>>> # rbd showmapped
>>>> id pool image snap device
>>>> 1 rbd test - /dev/rbd1
>>>>
>>>> 3. Put a fs on it and check the block device size
>>>>
>>>> # mkfs.ext4 /dev/rdb1
>>>> ...
>>>> ...
>>>>
>>>> # fdisk -l /dev/rbd1
>>>>
>>>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes
>>>>
>>>> 4. Mount it
>>>>
>>>> # mount /dev/rbd1 /mnt
>>>> # df -h
>>>> /dev/rbd1                   9.8G  277M  9.0G   3% /mnt
>>>>
>>>>
>>>> 5. Change the image size
>>>>
>>>> # rbd resize --size 11000 test
>>>> Resizing image: 100% complete...done.
>>>>
>>>> # rbd info test
>>>> rbd image 'test':
>>>> size 11000 MB in 2750 objects
>>>> order 22 (4096 KB objects)
>>>> block_name_prefix: rb.0.6
>>>> parent:  (pool -1)
>>>>
>>>> At this point of time, if you perform the fdisk -l /dev/rbd1, the
>>>> block device size will remain the same.
>>>>
>>>> 6. Unmount the device:
>>>>
>>>> # umount /media
>>>>
>>>> # fdisk -l /dev/rbd1
>>>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes
>>>>
>>>> Unmounting the directory did update the block device size.
>>>>
>>>> Of course you can do something really fast like:
>>>>
>>>> # umount /media && mount /dev/rbd1 /media
>>>>
>>>> That will work, it's a valid solution as long as there is no opened
>>>> file. I won't use this trick in production...
>>>>
>>>> I also tried to "mount -o remount" and it didn't work.
>>>>
>>>> 7. Resize the fs (this can be performed while the fs is mounted):
>>>>
>>>> # e2fsck -f /dev/rbd1
>>>> e2fsck 1.42 (29-Nov-2011)
>>>> Pass 1: Checking inodes, blocks, and sizes
>>>> Pass 2: Checking directory structure
>>>> Pass 3: Checking directory connectivity
>>>> Pass 4: Checking reference counts
>>>> Pass 5: Checking group summary information
>>>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks
>>>>
>>>> # resize2fs /dev/rbd1
>>>> resize2fs 1.42 (29-Nov-2011)
>>>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks.
>>>> The filesystem on /dev/rbd1 is now 2816000 blocks long.
>>>>
>>>>
>>>> Did I miss something?
>>>> Is this feature coming?
>>>>
>>>> Thank you in advance :)
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Ceph doesn't update the block device size while a rbd image is mounted

[Calvin Morrow]

I've had a little more luck using cfdisk than vanilla fdisk when it
comes to detecting changes.  You might try running partprobe and then
cfdisk and seeing if you get anything different.

Calvin

On Thu, Jul 19, 2012 at 9:50 AM, Sébastien Han <han.sebastien@gmail.com> wrote:
> Hum ok, I see. Thanks!
>
> But if you have any clue to force the kernel to re-read without
> unmont/mounting :)
>
> On Thu, Jul 19, 2012 at 5:47 PM, Wido den Hollander <wido@widodh.nl> wrote:
>>
>>
>> On 19-07-12 17:26, Sébastien Han wrote:
>>>
>>> Ok I got your point seems logic, but why is this possible with LVM for
>>> example?
>>>
>>> You can easily do this with LVM without un-mounting the device.
>>>
>>
>>
>> LVM runs through the device mapper and are not regular block devices.
>>
>> If you resize the disk underneath LVM you won't see an increased VG or PV
>> size unless you change the availability of the VG to unavailable and back to
>> available again.
>>
>> I'm not a 100% sure what the exact root cause is, but the kernel won't read
>> the new size of a block device as long as it is in use.
>>
>> Wido
>>
>>
>>
>>> Cheers.
>>>
>>>
>>> On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl>
>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>>
>>>> On 19-07-12 16:55, Sébastien Han wrote:
>>>>>
>>>>>
>>>>> Hi Cephers!
>>>>>
>>>>> I'm working with rbd mapping. I figured out that the block device size
>>>>> of the rbd device is not update while the device is mounted. Here my
>>>>> tests:
>>>>>
>>>>
>>>> iirc this is not something RBD specific, but since the device is in use
>>>> it
>>>> can't be re-read by the kernel.
>>>>
>>>> So when you unmount it the kernel can re-read the header and resize the
>>>> device.
>>>>
>>>> Wido
>>>>
>>>>> 1. Pick up a device and check his size
>>>>>
>>>>> # rbd ls
>>>>> size
>>>>>
>>>>> # rbd info test
>>>>> rbd image 'test':
>>>>> size 10000 MB in 2500 objects
>>>>> order 22 (4096 KB objects)
>>>>> block_name_prefix: rb.0.6
>>>>> parent:  (pool -1)
>>>>>
>>>>> 2. Map the device
>>>>>
>>>>> # rbd map --secret /etc/ceph/secret test
>>>>> # rbd showmapped
>>>>> id pool image snap device
>>>>> 1 rbd test - /dev/rbd1
>>>>>
>>>>> 3. Put a fs on it and check the block device size
>>>>>
>>>>> # mkfs.ext4 /dev/rdb1
>>>>> ...
>>>>> ...
>>>>>
>>>>> # fdisk -l /dev/rbd1
>>>>>
>>>>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes
>>>>>
>>>>> 4. Mount it
>>>>>
>>>>> # mount /dev/rbd1 /mnt
>>>>> # df -h
>>>>> /dev/rbd1                   9.8G  277M  9.0G   3% /mnt
>>>>>
>>>>>
>>>>> 5. Change the image size
>>>>>
>>>>> # rbd resize --size 11000 test
>>>>> Resizing image: 100% complete...done.
>>>>>
>>>>> # rbd info test
>>>>> rbd image 'test':
>>>>> size 11000 MB in 2750 objects
>>>>> order 22 (4096 KB objects)
>>>>> block_name_prefix: rb.0.6
>>>>> parent:  (pool -1)
>>>>>
>>>>> At this point of time, if you perform the fdisk -l /dev/rbd1, the
>>>>> block device size will remain the same.
>>>>>
>>>>> 6. Unmount the device:
>>>>>
>>>>> # umount /media
>>>>>
>>>>> # fdisk -l /dev/rbd1
>>>>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes
>>>>>
>>>>> Unmounting the directory did update the block device size.
>>>>>
>>>>> Of course you can do something really fast like:
>>>>>
>>>>> # umount /media && mount /dev/rbd1 /media
>>>>>
>>>>> That will work, it's a valid solution as long as there is no opened
>>>>> file. I won't use this trick in production...
>>>>>
>>>>> I also tried to "mount -o remount" and it didn't work.
>>>>>
>>>>> 7. Resize the fs (this can be performed while the fs is mounted):
>>>>>
>>>>> # e2fsck -f /dev/rbd1
>>>>> e2fsck 1.42 (29-Nov-2011)
>>>>> Pass 1: Checking inodes, blocks, and sizes
>>>>> Pass 2: Checking directory structure
>>>>> Pass 3: Checking directory connectivity
>>>>> Pass 4: Checking reference counts
>>>>> Pass 5: Checking group summary information
>>>>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks
>>>>>
>>>>> # resize2fs /dev/rbd1
>>>>> resize2fs 1.42 (29-Nov-2011)
>>>>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks.
>>>>> The filesystem on /dev/rbd1 is now 2816000 blocks long.
>>>>>
>>>>>
>>>>> Did I miss something?
>>>>> Is this feature coming?
>>>>>
>>>>> Thank you in advance :)
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Ceph doesn't update the block device size while a rbd image is mounted

[Sébastien Han]

With LVM, you can re-scan the scsi bus to extend a physical drive and
then run a pvextend.

@Calvin: I tried your solution

# partprobe /dev/rbd1

Unfortunatly nothing changed.

Did you make it working?

Cheers!


On Thu, Jul 19, 2012 at 5:50 PM, Sébastien Han <han.sebastien@gmail.com> wrote:
> Hum ok, I see. Thanks!
>
> But if you have any clue to force the kernel to re-read without
> unmont/mounting :)
>
> On Thu, Jul 19, 2012 at 5:47 PM, Wido den Hollander <wido@widodh.nl> wrote:
>>
>>
>> On 19-07-12 17:26, Sébastien Han wrote:
>>>
>>> Ok I got your point seems logic, but why is this possible with LVM for
>>> example?
>>>
>>> You can easily do this with LVM without un-mounting the device.
>>>
>>
>>
>> LVM runs through the device mapper and are not regular block devices.
>>
>> If you resize the disk underneath LVM you won't see an increased VG or PV
>> size unless you change the availability of the VG to unavailable and back to
>> available again.
>>
>> I'm not a 100% sure what the exact root cause is, but the kernel won't read
>> the new size of a block device as long as it is in use.
>>
>> Wido
>>
>>
>>
>>> Cheers.
>>>
>>>
>>> On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl>
>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>>
>>>> On 19-07-12 16:55, Sébastien Han wrote:
>>>>>
>>>>>
>>>>> Hi Cephers!
>>>>>
>>>>> I'm working with rbd mapping. I figured out that the block device size
>>>>> of the rbd device is not update while the device is mounted. Here my
>>>>> tests:
>>>>>
>>>>
>>>> iirc this is not something RBD specific, but since the device is in use
>>>> it
>>>> can't be re-read by the kernel.
>>>>
>>>> So when you unmount it the kernel can re-read the header and resize the
>>>> device.
>>>>
>>>> Wido
>>>>
>>>>> 1. Pick up a device and check his size
>>>>>
>>>>> # rbd ls
>>>>> size
>>>>>
>>>>> # rbd info test
>>>>> rbd image 'test':
>>>>> size 10000 MB in 2500 objects
>>>>> order 22 (4096 KB objects)
>>>>> block_name_prefix: rb.0.6
>>>>> parent:  (pool -1)
>>>>>
>>>>> 2. Map the device
>>>>>
>>>>> # rbd map --secret /etc/ceph/secret test
>>>>> # rbd showmapped
>>>>> id pool image snap device
>>>>> 1 rbd test - /dev/rbd1
>>>>>
>>>>> 3. Put a fs on it and check the block device size
>>>>>
>>>>> # mkfs.ext4 /dev/rdb1
>>>>> ...
>>>>> ...
>>>>>
>>>>> # fdisk -l /dev/rbd1
>>>>>
>>>>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes
>>>>>
>>>>> 4. Mount it
>>>>>
>>>>> # mount /dev/rbd1 /mnt
>>>>> # df -h
>>>>> /dev/rbd1                   9.8G  277M  9.0G   3% /mnt
>>>>>
>>>>>
>>>>> 5. Change the image size
>>>>>
>>>>> # rbd resize --size 11000 test
>>>>> Resizing image: 100% complete...done.
>>>>>
>>>>> # rbd info test
>>>>> rbd image 'test':
>>>>> size 11000 MB in 2750 objects
>>>>> order 22 (4096 KB objects)
>>>>> block_name_prefix: rb.0.6
>>>>> parent:  (pool -1)
>>>>>
>>>>> At this point of time, if you perform the fdisk -l /dev/rbd1, the
>>>>> block device size will remain the same.
>>>>>
>>>>> 6. Unmount the device:
>>>>>
>>>>> # umount /media
>>>>>
>>>>> # fdisk -l /dev/rbd1
>>>>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes
>>>>>
>>>>> Unmounting the directory did update the block device size.
>>>>>
>>>>> Of course you can do something really fast like:
>>>>>
>>>>> # umount /media && mount /dev/rbd1 /media
>>>>>
>>>>> That will work, it's a valid solution as long as there is no opened
>>>>> file. I won't use this trick in production...
>>>>>
>>>>> I also tried to "mount -o remount" and it didn't work.
>>>>>
>>>>> 7. Resize the fs (this can be performed while the fs is mounted):
>>>>>
>>>>> # e2fsck -f /dev/rbd1
>>>>> e2fsck 1.42 (29-Nov-2011)
>>>>> Pass 1: Checking inodes, blocks, and sizes
>>>>> Pass 2: Checking directory structure
>>>>> Pass 3: Checking directory connectivity
>>>>> Pass 4: Checking reference counts
>>>>> Pass 5: Checking group summary information
>>>>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks
>>>>>
>>>>> # resize2fs /dev/rbd1
>>>>> resize2fs 1.42 (29-Nov-2011)
>>>>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks.
>>>>> The filesystem on /dev/rbd1 is now 2816000 blocks long.
>>>>>
>>>>>
>>>>> Did I miss something?
>>>>> Is this feature coming?
>>>>>
>>>>> Thank you in advance :)
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>
>>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Ceph doesn't update the block device size while a rbd image is mounted

[Calvin Morrow]

I haven't tried resizing an rbd yet, but I was changing partitions on
a non-ceph two-node cluster with shared storage yesterday while
certain partitions were in use (partitions 1,2,5 were mounted,
deleting partition ids 6+, adding new ones) and fdisk wasn't
re-reading disk changes.  Partprobe followed by cfdisk seemed to
update the kernel with the on-disk changes.

I realize its not exactly what you are doing, but I figured it might
be close enough to be worth a shot.  cfdisk seems to make a system
call to refresh disk information that vanilla fdisk doesn't.

Calvin

On Thu, Jul 19, 2012 at 1:44 PM, Sébastien Han <han.sebastien@gmail.com> wrote:
> With LVM, you can re-scan the scsi bus to extend a physical drive and
> then run a pvextend.
>
> @Calvin: I tried your solution
>
> # partprobe /dev/rbd1
>
> Unfortunatly nothing changed.
>
> Did you make it working?
>
> Cheers!
>
>
> On Thu, Jul 19, 2012 at 5:50 PM, Sébastien Han <han.sebastien@gmail.com> wrote:
>> Hum ok, I see. Thanks!
>>
>> But if you have any clue to force the kernel to re-read without
>> unmont/mounting :)
>>
>> On Thu, Jul 19, 2012 at 5:47 PM, Wido den Hollander <wido@widodh.nl> wrote:
>>>
>>>
>>> On 19-07-12 17:26, Sébastien Han wrote:
>>>>
>>>> Ok I got your point seems logic, but why is this possible with LVM for
>>>> example?
>>>>
>>>> You can easily do this with LVM without un-mounting the device.
>>>>
>>>
>>>
>>> LVM runs through the device mapper and are not regular block devices.
>>>
>>> If you resize the disk underneath LVM you won't see an increased VG or PV
>>> size unless you change the availability of the VG to unavailable and back to
>>> available again.
>>>
>>> I'm not a 100% sure what the exact root cause is, but the kernel won't read
>>> the new size of a block device as long as it is in use.
>>>
>>> Wido
>>>
>>>
>>>
>>>> Cheers.
>>>>
>>>>
>>>> On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl>
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>>
>>>>> On 19-07-12 16:55, Sébastien Han wrote:
>>>>>>
>>>>>>
>>>>>> Hi Cephers!
>>>>>>
>>>>>> I'm working with rbd mapping. I figured out that the block device size
>>>>>> of the rbd device is not update while the device is mounted. Here my
>>>>>> tests:
>>>>>>
>>>>>
>>>>> iirc this is not something RBD specific, but since the device is in use
>>>>> it
>>>>> can't be re-read by the kernel.
>>>>>
>>>>> So when you unmount it the kernel can re-read the header and resize the
>>>>> device.
>>>>>
>>>>> Wido
>>>>>
>>>>>> 1. Pick up a device and check his size
>>>>>>
>>>>>> # rbd ls
>>>>>> size
>>>>>>
>>>>>> # rbd info test
>>>>>> rbd image 'test':
>>>>>> size 10000 MB in 2500 objects
>>>>>> order 22 (4096 KB objects)
>>>>>> block_name_prefix: rb.0.6
>>>>>> parent:  (pool -1)
>>>>>>
>>>>>> 2. Map the device
>>>>>>
>>>>>> # rbd map --secret /etc/ceph/secret test
>>>>>> # rbd showmapped
>>>>>> id pool image snap device
>>>>>> 1 rbd test - /dev/rbd1
>>>>>>
>>>>>> 3. Put a fs on it and check the block device size
>>>>>>
>>>>>> # mkfs.ext4 /dev/rdb1
>>>>>> ...
>>>>>> ...
>>>>>>
>>>>>> # fdisk -l /dev/rbd1
>>>>>>
>>>>>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes
>>>>>>
>>>>>> 4. Mount it
>>>>>>
>>>>>> # mount /dev/rbd1 /mnt
>>>>>> # df -h
>>>>>> /dev/rbd1                   9.8G  277M  9.0G   3% /mnt
>>>>>>
>>>>>>
>>>>>> 5. Change the image size
>>>>>>
>>>>>> # rbd resize --size 11000 test
>>>>>> Resizing image: 100% complete...done.
>>>>>>
>>>>>> # rbd info test
>>>>>> rbd image 'test':
>>>>>> size 11000 MB in 2750 objects
>>>>>> order 22 (4096 KB objects)
>>>>>> block_name_prefix: rb.0.6
>>>>>> parent:  (pool -1)
>>>>>>
>>>>>> At this point of time, if you perform the fdisk -l /dev/rbd1, the
>>>>>> block device size will remain the same.
>>>>>>
>>>>>> 6. Unmount the device:
>>>>>>
>>>>>> # umount /media
>>>>>>
>>>>>> # fdisk -l /dev/rbd1
>>>>>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes
>>>>>>
>>>>>> Unmounting the directory did update the block device size.
>>>>>>
>>>>>> Of course you can do something really fast like:
>>>>>>
>>>>>> # umount /media && mount /dev/rbd1 /media
>>>>>>
>>>>>> That will work, it's a valid solution as long as there is no opened
>>>>>> file. I won't use this trick in production...
>>>>>>
>>>>>> I also tried to "mount -o remount" and it didn't work.
>>>>>>
>>>>>> 7. Resize the fs (this can be performed while the fs is mounted):
>>>>>>
>>>>>> # e2fsck -f /dev/rbd1
>>>>>> e2fsck 1.42 (29-Nov-2011)
>>>>>> Pass 1: Checking inodes, blocks, and sizes
>>>>>> Pass 2: Checking directory structure
>>>>>> Pass 3: Checking directory connectivity
>>>>>> Pass 4: Checking reference counts
>>>>>> Pass 5: Checking group summary information
>>>>>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks
>>>>>>
>>>>>> # resize2fs /dev/rbd1
>>>>>> resize2fs 1.42 (29-Nov-2011)
>>>>>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks.
>>>>>> The filesystem on /dev/rbd1 is now 2816000 blocks long.
>>>>>>
>>>>>>
>>>>>> Did I miss something?
>>>>>> Is this feature coming?
>>>>>>
>>>>>> Thank you in advance :)
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>>
>>>>>
>>>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Ceph doesn't update the block device size while a rbd image is mounted

[Andreas Kurz]

[-- Attachment #1: Type: text/plain, Size: 5151 bytes --]

On 07/19/2012 09:44 PM, Sébastien Han wrote:
> With LVM, you can re-scan the scsi bus to extend a physical drive and
> then run a pvextend.
> 
> @Calvin: I tried your solution
> 
> # partprobe /dev/rbd1

Did you try blockdev?

# blockdev --rereadpt /dev/rbd1

Regards,
Andreas

> 
> Unfortunatly nothing changed.
> 
> Did you make it working?
> 
> Cheers!
> 
> 
> On Thu, Jul 19, 2012 at 5:50 PM, Sébastien Han <han.sebastien@gmail.com> wrote:
>> Hum ok, I see. Thanks!
>>
>> But if you have any clue to force the kernel to re-read without
>> unmont/mounting :)
>>
>> On Thu, Jul 19, 2012 at 5:47 PM, Wido den Hollander <wido@widodh.nl> wrote:
>>>
>>>
>>> On 19-07-12 17:26, Sébastien Han wrote:
>>>>
>>>> Ok I got your point seems logic, but why is this possible with LVM for
>>>> example?
>>>>
>>>> You can easily do this with LVM without un-mounting the device.
>>>>
>>>
>>>
>>> LVM runs through the device mapper and are not regular block devices.
>>>
>>> If you resize the disk underneath LVM you won't see an increased VG or PV
>>> size unless you change the availability of the VG to unavailable and back to
>>> available again.
>>>
>>> I'm not a 100% sure what the exact root cause is, but the kernel won't read
>>> the new size of a block device as long as it is in use.
>>>
>>> Wido
>>>
>>>
>>>
>>>> Cheers.
>>>>
>>>>
>>>> On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl>
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>>
>>>>> On 19-07-12 16:55, Sébastien Han wrote:
>>>>>>
>>>>>>
>>>>>> Hi Cephers!
>>>>>>
>>>>>> I'm working with rbd mapping. I figured out that the block device size
>>>>>> of the rbd device is not update while the device is mounted. Here my
>>>>>> tests:
>>>>>>
>>>>>
>>>>> iirc this is not something RBD specific, but since the device is in use
>>>>> it
>>>>> can't be re-read by the kernel.
>>>>>
>>>>> So when you unmount it the kernel can re-read the header and resize the
>>>>> device.
>>>>>
>>>>> Wido
>>>>>
>>>>>> 1. Pick up a device and check his size
>>>>>>
>>>>>> # rbd ls
>>>>>> size
>>>>>>
>>>>>> # rbd info test
>>>>>> rbd image 'test':
>>>>>> size 10000 MB in 2500 objects
>>>>>> order 22 (4096 KB objects)
>>>>>> block_name_prefix: rb.0.6
>>>>>> parent:  (pool -1)
>>>>>>
>>>>>> 2. Map the device
>>>>>>
>>>>>> # rbd map --secret /etc/ceph/secret test
>>>>>> # rbd showmapped
>>>>>> id pool image snap device
>>>>>> 1 rbd test - /dev/rbd1
>>>>>>
>>>>>> 3. Put a fs on it and check the block device size
>>>>>>
>>>>>> # mkfs.ext4 /dev/rdb1
>>>>>> ...
>>>>>> ...
>>>>>>
>>>>>> # fdisk -l /dev/rbd1
>>>>>>
>>>>>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes
>>>>>>
>>>>>> 4. Mount it
>>>>>>
>>>>>> # mount /dev/rbd1 /mnt
>>>>>> # df -h
>>>>>> /dev/rbd1                   9.8G  277M  9.0G   3% /mnt
>>>>>>
>>>>>>
>>>>>> 5. Change the image size
>>>>>>
>>>>>> # rbd resize --size 11000 test
>>>>>> Resizing image: 100% complete...done.
>>>>>>
>>>>>> # rbd info test
>>>>>> rbd image 'test':
>>>>>> size 11000 MB in 2750 objects
>>>>>> order 22 (4096 KB objects)
>>>>>> block_name_prefix: rb.0.6
>>>>>> parent:  (pool -1)
>>>>>>
>>>>>> At this point of time, if you perform the fdisk -l /dev/rbd1, the
>>>>>> block device size will remain the same.
>>>>>>
>>>>>> 6. Unmount the device:
>>>>>>
>>>>>> # umount /media
>>>>>>
>>>>>> # fdisk -l /dev/rbd1
>>>>>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes
>>>>>>
>>>>>> Unmounting the directory did update the block device size.
>>>>>>
>>>>>> Of course you can do something really fast like:
>>>>>>
>>>>>> # umount /media && mount /dev/rbd1 /media
>>>>>>
>>>>>> That will work, it's a valid solution as long as there is no opened
>>>>>> file. I won't use this trick in production...
>>>>>>
>>>>>> I also tried to "mount -o remount" and it didn't work.
>>>>>>
>>>>>> 7. Resize the fs (this can be performed while the fs is mounted):
>>>>>>
>>>>>> # e2fsck -f /dev/rbd1
>>>>>> e2fsck 1.42 (29-Nov-2011)
>>>>>> Pass 1: Checking inodes, blocks, and sizes
>>>>>> Pass 2: Checking directory structure
>>>>>> Pass 3: Checking directory connectivity
>>>>>> Pass 4: Checking reference counts
>>>>>> Pass 5: Checking group summary information
>>>>>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks
>>>>>>
>>>>>> # resize2fs /dev/rbd1
>>>>>> resize2fs 1.42 (29-Nov-2011)
>>>>>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks.
>>>>>> The filesystem on /dev/rbd1 is now 2816000 blocks long.
>>>>>>
>>>>>>
>>>>>> Did I miss something?
>>>>>> Is this feature coming?
>>>>>>
>>>>>> Thank you in advance :)
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>>
>>>>>
>>>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 222 bytes --]

Re: Ceph doesn't update the block device size while a rbd image is mounted

[Sébastien Han]

Hi Andreas,

Sorry the long response, I also tried blockdev but without success :/

Cheers!


On Thu, Jul 19, 2012 at 10:35 PM, Andreas Kurz <andreas@hastexo.com> wrote:
>
> On 07/19/2012 09:44 PM, Sébastien Han wrote:
> > With LVM, you can re-scan the scsi bus to extend a physical drive and
> > then run a pvextend.
> >
> > @Calvin: I tried your solution
> >
> > # partprobe /dev/rbd1
>
> Did you try blockdev?
>
> # blockdev --rereadpt /dev/rbd1
>
> Regards,
> Andreas
>
> >
> > Unfortunatly nothing changed.
> >
> > Did you make it working?
> >
> > Cheers!
> >
> >
> > On Thu, Jul 19, 2012 at 5:50 PM, Sébastien Han <han.sebastien@gmail.com> wrote:
> >> Hum ok, I see. Thanks!
> >>
> >> But if you have any clue to force the kernel to re-read without
> >> unmont/mounting :)
> >>
> >> On Thu, Jul 19, 2012 at 5:47 PM, Wido den Hollander <wido@widodh.nl> wrote:
> >>>
> >>>
> >>> On 19-07-12 17:26, Sébastien Han wrote:
> >>>>
> >>>> Ok I got your point seems logic, but why is this possible with LVM for
> >>>> example?
> >>>>
> >>>> You can easily do this with LVM without un-mounting the device.
> >>>>
> >>>
> >>>
> >>> LVM runs through the device mapper and are not regular block devices.
> >>>
> >>> If you resize the disk underneath LVM you won't see an increased VG or PV
> >>> size unless you change the availability of the VG to unavailable and back to
> >>> available again.
> >>>
> >>> I'm not a 100% sure what the exact root cause is, but the kernel won't read
> >>> the new size of a block device as long as it is in use.
> >>>
> >>> Wido
> >>>
> >>>
> >>>
> >>>> Cheers.
> >>>>
> >>>>
> >>>> On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl>
> >>>> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>>
> >>>>> On 19-07-12 16:55, Sébastien Han wrote:
> >>>>>>
> >>>>>>
> >>>>>> Hi Cephers!
> >>>>>>
> >>>>>> I'm working with rbd mapping. I figured out that the block device size
> >>>>>> of the rbd device is not update while the device is mounted. Here my
> >>>>>> tests:
> >>>>>>
> >>>>>
> >>>>> iirc this is not something RBD specific, but since the device is in use
> >>>>> it
> >>>>> can't be re-read by the kernel.
> >>>>>
> >>>>> So when you unmount it the kernel can re-read the header and resize the
> >>>>> device.
> >>>>>
> >>>>> Wido
> >>>>>
> >>>>>> 1. Pick up a device and check his size
> >>>>>>
> >>>>>> # rbd ls
> >>>>>> size
> >>>>>>
> >>>>>> # rbd info test
> >>>>>> rbd image 'test':
> >>>>>> size 10000 MB in 2500 objects
> >>>>>> order 22 (4096 KB objects)
> >>>>>> block_name_prefix: rb.0.6
> >>>>>> parent:  (pool -1)
> >>>>>>
> >>>>>> 2. Map the device
> >>>>>>
> >>>>>> # rbd map --secret /etc/ceph/secret test
> >>>>>> # rbd showmapped
> >>>>>> id pool image snap device
> >>>>>> 1 rbd test - /dev/rbd1
> >>>>>>
> >>>>>> 3. Put a fs on it and check the block device size
> >>>>>>
> >>>>>> # mkfs.ext4 /dev/rdb1
> >>>>>> ...
> >>>>>> ...
> >>>>>>
> >>>>>> # fdisk -l /dev/rbd1
> >>>>>>
> >>>>>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes
> >>>>>>
> >>>>>> 4. Mount it
> >>>>>>
> >>>>>> # mount /dev/rbd1 /mnt
> >>>>>> # df -h
> >>>>>> /dev/rbd1                   9.8G  277M  9.0G   3% /mnt
> >>>>>>
> >>>>>>
> >>>>>> 5. Change the image size
> >>>>>>
> >>>>>> # rbd resize --size 11000 test
> >>>>>> Resizing image: 100% complete...done.
> >>>>>>
> >>>>>> # rbd info test
> >>>>>> rbd image 'test':
> >>>>>> size 11000 MB in 2750 objects
> >>>>>> order 22 (4096 KB objects)
> >>>>>> block_name_prefix: rb.0.6
> >>>>>> parent:  (pool -1)
> >>>>>>
> >>>>>> At this point of time, if you perform the fdisk -l /dev/rbd1, the
> >>>>>> block device size will remain the same.
> >>>>>>
> >>>>>> 6. Unmount the device:
> >>>>>>
> >>>>>> # umount /media
> >>>>>>
> >>>>>> # fdisk -l /dev/rbd1
> >>>>>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes
> >>>>>>
> >>>>>> Unmounting the directory did update the block device size.
> >>>>>>
> >>>>>> Of course you can do something really fast like:
> >>>>>>
> >>>>>> # umount /media && mount /dev/rbd1 /media
> >>>>>>
> >>>>>> That will work, it's a valid solution as long as there is no opened
> >>>>>> file. I won't use this trick in production...
> >>>>>>
> >>>>>> I also tried to "mount -o remount" and it didn't work.
> >>>>>>
> >>>>>> 7. Resize the fs (this can be performed while the fs is mounted):
> >>>>>>
> >>>>>> # e2fsck -f /dev/rbd1
> >>>>>> e2fsck 1.42 (29-Nov-2011)
> >>>>>> Pass 1: Checking inodes, blocks, and sizes
> >>>>>> Pass 2: Checking directory structure
> >>>>>> Pass 3: Checking directory connectivity
> >>>>>> Pass 4: Checking reference counts
> >>>>>> Pass 5: Checking group summary information
> >>>>>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks
> >>>>>>
> >>>>>> # resize2fs /dev/rbd1
> >>>>>> resize2fs 1.42 (29-Nov-2011)
> >>>>>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks.
> >>>>>> The filesystem on /dev/rbd1 is now 2816000 blocks long.
> >>>>>>
> >>>>>>
> >>>>>> Did I miss something?
> >>>>>> Is this feature coming?
> >>>>>>
> >>>>>> Thank you in advance :)
> >>>>>> --
> >>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> >>>>>> the body of a message to majordomo@vger.kernel.org
> >>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>>>>>
> >>>>>
> >>>
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html