sparse file support

sparse file support

[Brian Chrisman]

I've performed some simple testing of sparse files on ceph.  Summary,
ceph reports full file size to 'du' and 'ls', where it should return
actual storage usage to 'du'.  Creating a sparse file on ceph does not
actually use the full file size in storage (which is correct
behavior).  Is this a known issue, or should I submit a bug?

dd if=/dev/urandom of=random.data bs=1024 count=1024
dd if=random.data of=sparse_file.data
dd if=random.data of=sparse_file.data bs=1024k seek=1024
ls -l sparse_file.data
du -sh sparse_file.data

A reference output: ext3
[root@test11 tmp]# ls -l sparse_file.data
-rw-r-r- 1 root root 1074790400 Jan 19 18:43 sparse_file.data
[root@test11 tmp]#
[root@test11 tmp]# du -sh sparse_file.data
2.1M sparse_file.data
[root@test11 tmp]#

Output from: ceph
[root@test11 fs0]# ls -l sparse_file.data
-rw-r-r- 1 root root 1074790400 Jan 19 18:45 sparse_file.data
[root@test11 fs0]# du -sh sparse_file.data
1.1Gsparse_file.data
[root@test11 fs0]#

However, df shows that 1GB of space is not actually used in the filesytem:

[root@test11 fs0]# df -h
Filesystem            Size  Used Avail Use% Mounted on
...
10.200.98.111:/        11T   30M   11T   1% /fs0
[root@test11 fs0]#

-Brian

Re: sparse file support

[Sage Weil]

On Thu, 20 Jan 2011, Brian Chrisman wrote:
> I've performed some simple testing of sparse files on ceph.  Summary,
> ceph reports full file size to 'du' and 'ls', where it should return
> actual storage usage to 'du'.  Creating a sparse file on ceph does not
> actually use the full file size in storage (which is correct
> behavior).  Is this a known issue, or should I submit a bug?

This is a design choice.  The MDS doesn't know how many blocks are 
allocated because, with the exception of reporting st_blocks to du, it 
doesn't need to.  This makes metadata smaller and the protocols simpler.

sage


> 
> dd if=/dev/urandom of=random.data bs=1024 count=1024
> dd if=random.data of=sparse_file.data
> dd if=random.data of=sparse_file.data bs=1024k seek=1024
> ls -l sparse_file.data
> du -sh sparse_file.data
> 
> A reference output: ext3
> [root@test11 tmp]# ls -l sparse_file.data
> -rw-r-r- 1 root root 1074790400 Jan 19 18:43 sparse_file.data
> [root@test11 tmp]#
> [root@test11 tmp]# du -sh sparse_file.data
> 2.1M sparse_file.data
> [root@test11 tmp]#
> 
> Output from: ceph
> [root@test11 fs0]# ls -l sparse_file.data
> -rw-r-r- 1 root root 1074790400 Jan 19 18:45 sparse_file.data
> [root@test11 fs0]# du -sh sparse_file.data
> 1.1Gsparse_file.data
> [root@test11 fs0]#
> 
> However, df shows that 1GB of space is not actually used in the filesytem:
> 
> [root@test11 fs0]# df -h
> Filesystem            Size  Used Avail Use% Mounted on
> ...
> 10.200.98.111:/        11T   30M   11T   1% /fs0
> [root@test11 fs0]#
> 
> -Brian
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: sparse file support

[Brian Chrisman]

On Thu, Jan 20, 2011 at 1:28 PM, Sage Weil <sage@newdream.net> wrote:
> On Thu, 20 Jan 2011, Brian Chrisman wrote:
>> I've performed some simple testing of sparse files on ceph.  Summary,
>> ceph reports full file size to 'du' and 'ls', where it should return
>> actual storage usage to 'du'.  Creating a sparse file on ceph does not
>> actually use the full file size in storage (which is correct
>> behavior).  Is this a known issue, or should I submit a bug?
>
> This is a design choice.  The MDS doesn't know how many blocks are
> allocated because, with the exception of reporting st_blocks to du, it
> doesn't need to.  This makes metadata smaller and the protocols simpler.
>
> sage

Ahh ok, understood.
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

how to do when data size big than osd disk.

[Longguang Yue]

2 out of 3 is down,so the live osd is full, data lose.
How to solve the case? 

Re: how to do when data size big than osd disk.

[Gregory Farnum]

On Sun, Jan 23, 2011 at 8:28 PM, Longguang Yue
<longguang_yue@tcloudcomputing.com> wrote:
> 2 out of 3 is down,so the live osd is full, data lose.
> How to solve the case?
Well, there's no way to magically make Ceph store more data than the
OSD can hold. However, if you can bring the other OSDs back up you
haven't lost any data.
It's possible that in this scenario the cluster won't recover properly
due to the surviving OSD running out of space, but let's leave dealing
with that until we know it's going to be a problem.