Two questions

Two questions

[Sławomir Skowron]

Hello. I have some questions.

1. Is there any chance to change default 4MB object size to for
example 1MB or less ??

2. I have create cluster of two mons, and 32 osd (1TB each) on two
machines. At this radosgw with apache2 for test. When i putting data
from s3 client to rados, everything is ok, but when one of osd is
going down all s3 clients going to be freeze for some seconds, before
ceph sign this osd as down, and then everything is starting working
again wit degraded osd. How can i tune this marking as a down for ms
or 0 :) and not for seconds freeze ??

My setup is debian 6.0, kernel 2.6.37.6  x86_64 and ceph version
0.31-10 from stable repo.

Best regards

Slawomir Skowron

Re: Two questions

[Wido den Hollander]

Hi,

On Wed, 2011-07-27 at 07:58 +0200, Sławomir Skowron wrote:
> Hello. I have some questions.
> 
> 1. Is there any chance to change default 4MB object size to for
> example 1MB or less ??

If you are using the filesystem, you can change the stripe-size per
directory with the "cephfs" tool.

> 
> 2. I have create cluster of two mons, and 32 osd (1TB each) on two
> machines. At this radosgw with apache2 for test. When i putting data
> from s3 client to rados, everything is ok, but when one of osd is
> going down all s3 clients going to be freeze for some seconds, before
> ceph sign this osd as down, and then everything is starting working
> again wit degraded osd. How can i tune this marking as a down for ms
> or 0 :) and not for seconds freeze ??

Normally a OSD should be down within about 10 seconds and shouldn't
affect all I/O operations.

By default it takes 300 seconds before a OSD gets marked as "out", this
is handled by "mon osd down out interval"

If you want to change the "down" reporting behaviour you could do this
with "osd min down reporters" and "osd min down reports". I'm not sure
you really want to do this, as the default values should be ok.

Wido

> 
> My setup is debian 6.0, kernel 2.6.37.6  x86_64 and ceph version
> 0.31-10 from stable repo.
> 
> Best regards
> 
> Slawomir Skowron
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Two questions

[Sławomir Skowron]

Thanks.

2011/7/27 Wido den Hollander <wido@widodh.nl>:
> Hi,
>
> On Wed, 2011-07-27 at 07:58 +0200, Sławomir Skowron wrote:
>> Hello. I have some questions.
>>
>> 1. Is there any chance to change default 4MB object size to for
>> example 1MB or less ??
>
> If you are using the filesystem, you can change the stripe-size per
> directory with the "cephfs" tool.

Unfortunately i only use rados layer in this case, and i even don't
have any mds :) is there any chance to change this in rados layer ??

>>
>> 2. I have create cluster of two mons, and 32 osd (1TB each) on two
>> machines. At this radosgw with apache2 for test. When i putting data
>> from s3 client to rados, everything is ok, but when one of osd is
>> going down all s3 clients going to be freeze for some seconds, before
>> ceph sign this osd as down, and then everything is starting working
>> again wit degraded osd. How can i tune this marking as a down for ms
>> or 0 :) and not for seconds freeze ??
>
> Normally a OSD should be down within about 10 seconds and shouldn't
> affect all I/O operations.
>
> By default it takes 300 seconds before a OSD gets marked as "out", this
> is handled by "mon osd down out interval"
>
> If you want to change the "down" reporting behaviour you could do this
> with "osd min down reporters" and "osd min down reports". I'm not sure
> you really want to do this, as the default values should be ok.

Ok. I will check this.

> Wido
>
>>
>> My setup is debian 6.0, kernel 2.6.37.6  x86_64 and ceph version
>> 0.31-10 from stable repo.
>>
>> Best regards
>>
>> Slawomir Skowron
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>

-- 
-----
Pozdrawiam

Sławek "sZiBis" Skowron
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Two questions

[Wido den Hollander]

Hi,

On Wed, 2011-07-27 at 12:19 +0200, Sławomir Skowron wrote:
> Thanks.
> 
> 2011/7/27 Wido den Hollander <wido@widodh.nl>:
> > Hi,
> >
> > On Wed, 2011-07-27 at 07:58 +0200, Sławomir Skowron wrote:
> >> Hello. I have some questions.
> >>
> >> 1. Is there any chance to change default 4MB object size to for
> >> example 1MB or less ??
> >
> > If you are using the filesystem, you can change the stripe-size per
> > directory with the "cephfs" tool.
> 
> Unfortunately i only use rados layer in this case, and i even don't
> have any mds :) is there any chance to change this in rados layer ??

The RADOS gateway does NOT stripe over multiple objects. If you upload a
1GB object through the RADOS gateway it will be a 1GB object.

Only the RBD (Rados Block Device) and the Ceph filesystem do striping
over RADOS objects.

RADOS itself doesn't stripe and the RADOS gateway is a almost 1:1
mapping of pools and objects.

Wido

> 
> >>
> >> 2. I have create cluster of two mons, and 32 osd (1TB each) on two
> >> machines. At this radosgw with apache2 for test. When i putting data
> >> from s3 client to rados, everything is ok, but when one of osd is
> >> going down all s3 clients going to be freeze for some seconds, before
> >> ceph sign this osd as down, and then everything is starting working
> >> again wit degraded osd. How can i tune this marking as a down for ms
> >> or 0 :) and not for seconds freeze ??
> >
> > Normally a OSD should be down within about 10 seconds and shouldn't
> > affect all I/O operations.
> >
> > By default it takes 300 seconds before a OSD gets marked as "out", this
> > is handled by "mon osd down out interval"
> >
> > If you want to change the "down" reporting behaviour you could do this
> > with "osd min down reporters" and "osd min down reports". I'm not sure
> > you really want to do this, as the default values should be ok.
> 
> Ok. I will check this.
> 
> > Wido
> >
> >>
> >> My setup is debian 6.0, kernel 2.6.37.6  x86_64 and ceph version
> >> 0.31-10 from stable repo.
> >>
> >> Best regards
> >>
> >> Slawomir Skowron
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> >
> >
> 


--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Two questions

[Sławomir Skowron]

Ok, I will show example:

rados df
pool name                 KB      objects       clones     degraded
  unfound           rd        rd KB           wr        wr KB
.log                  558212            5            0            0
       0            0            0      2844888      2844888
.pool                      1            1            0            0
       0            0            0            8            8
.rgw                       0            6            0            0
       0            0            0            1            0
.users                     1            1            0            0
       0            0            0            1            1
.users.email               1            1            0            0
       0            0            0            1            1
.users.uid                 2            2            0            0
       0            1            0            2            2
data                       0            0            0            0
       0            0            0            0            0
metadata                   0            0            0            0
       0            0            0            0            0
rbd                        0            0            0            0
       0            0            0            0            0
sstest              32244226      2841055            0       653353
       0            0            0     17066724     32370391
  total used       324792996      2841071
  total avail    31083452176
  total space    33043244460

It means I have almost 3mln of objects in sstest.

pg_pool 7 'sstest' pg_pool(rep pg_size 3 crush_ruleset 0 object_hash
rjenkins pg_num 8 pgp_num 8 lpg_num 0 lpgp_num 0 last_change 21 owner
0)

3 copies in this pool.

sstest used 32.244.226 KB + log 558.212 KB = 32.802.438 KB

Total used is 324.792.996 KB and it's almost 10x more.

2011-07-27 12:57:35.541556    pg v54158: 6986 pgs: 8 active, 6978
active+clean; 32104 MB data, 310 GB used, 29642 GB / 31512 GB avail;

I'am putting files beetwen 4-50KB on RADOS via s3 clilent, and radosgw.


Can you explain that to me on this example from real life ??


2011/7/27 Wido den Hollander <wido@widodh.nl>:
> Hi,
>
> On Wed, 2011-07-27 at 12:19 +0200, Sławomir Skowron wrote:
>> Thanks.
>>
>> 2011/7/27 Wido den Hollander <wido@widodh.nl>:
>> > Hi,
>> >
>> > On Wed, 2011-07-27 at 07:58 +0200, Sławomir Skowron wrote:
>> >> Hello. I have some questions.
>> >>
>> >> 1. Is there any chance to change default 4MB object size to for
>> >> example 1MB or less ??
>> >
>> > If you are using the filesystem, you can change the stripe-size per
>> > directory with the "cephfs" tool.
>>
>> Unfortunately i only use rados layer in this case, and i even don't
>> have any mds :) is there any chance to change this in rados layer ??
>
> The RADOS gateway does NOT stripe over multiple objects. If you upload a
> 1GB object through the RADOS gateway it will be a 1GB object.
>
> Only the RBD (Rados Block Device) and the Ceph filesystem do striping
> over RADOS objects.
>
> RADOS itself doesn't stripe and the RADOS gateway is a almost 1:1
> mapping of pools and objects.
>
> Wido
>
>>
>> >>
>> >> 2. I have create cluster of two mons, and 32 osd (1TB each) on two
>> >> machines. At this radosgw with apache2 for test. When i putting data
>> >> from s3 client to rados, everything is ok, but when one of osd is
>> >> going down all s3 clients going to be freeze for some seconds, before
>> >> ceph sign this osd as down, and then everything is starting working
>> >> again wit degraded osd. How can i tune this marking as a down for ms
>> >> or 0 :) and not for seconds freeze ??
>> >
>> > Normally a OSD should be down within about 10 seconds and shouldn't
>> > affect all I/O operations.
>> >
>> > By default it takes 300 seconds before a OSD gets marked as "out", this
>> > is handled by "mon osd down out interval"
>> >
>> > If you want to change the "down" reporting behaviour you could do this
>> > with "osd min down reporters" and "osd min down reports". I'm not sure
>> > you really want to do this, as the default values should be ok.
>>
>> Ok. I will check this.
>>
>> > Wido
>> >
>> >>
>> >> My setup is debian 6.0, kernel 2.6.37.6  x86_64 and ceph version
>> >> 0.31-10 from stable repo.
>> >>
>> >> Best regards
>> >>
>> >> Slawomir Skowron
>> >> --
>> >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> >> the body of a message to majordomo@vger.kernel.org
>> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> >
>> >
>> >
>>
>
>
>



-- 
-----
Pozdrawiam

Sławek "sZiBis" Skowron
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Two questions

[Sage Weil]

On Wed, 27 Jul 2011, S?awomir Skowron wrote:
> Hello. I have some questions.
> 
> 1. Is there any chance to change default 4MB object size to for
> example 1MB or less ??

Yeah.  You can use the cephfs to set the default layout on the root 
directory, which will apply to any new files.  See 'man cephfs'.

> 2. I have create cluster of two mons, and 32 osd (1TB each) on two
> machines. At this radosgw with apache2 for test. When i putting data
> from s3 client to rados, everything is ok, but when one of osd is
> going down all s3 clients going to be freeze for some seconds, before
> ceph sign this osd as down, and then everything is starting working
> again wit degraded osd. How can i tune this marking as a down for ms
> or 0 :) and not for seconds freeze ??

The 'osd heartbeat grace' defualt is 20.  Stick something like

	osd heartbeat grace = 5

in the [osd] section of your ceph.conf.

sage

Re: Two questions

[Gregory Farnum]

2011/7/27 Sławomir Skowron <szibis@gmail.com>:
> Ok, I will show example:
>
> rados df
> pool name                 KB      objects       clones     degraded
>  unfound           rd        rd KB           wr        wr KB
> .log                  558212            5            0            0
>       0            0            0      2844888      2844888
> .pool                      1            1            0            0
>       0            0            0            8            8
> .rgw                       0            6            0            0
>       0            0            0            1            0
> .users                     1            1            0            0
>       0            0            0            1            1
> .users.email               1            1            0            0
>       0            0            0            1            1
> .users.uid                 2            2            0            0
>       0            1            0            2            2
> data                       0            0            0            0
>       0            0            0            0            0
> metadata                   0            0            0            0
>       0            0            0            0            0
> rbd                        0            0            0            0
>       0            0            0            0            0
> sstest              32244226      2841055            0       653353
>       0            0            0     17066724     32370391
>  total used       324792996      2841071
>  total avail    31083452176
>  total space    33043244460
>
> It means I have almost 3mln of objects in sstest.
>
> pg_pool 7 'sstest' pg_pool(rep pg_size 3 crush_ruleset 0 object_hash
> rjenkins pg_num 8 pgp_num 8 lpg_num 0 lpgp_num 0 last_change 21 owner
> 0)
>
> 3 copies in this pool.
>
> sstest used 32.244.226 KB + log 558.212 KB = 32.802.438 KB
>
> Total used is 324.792.996 KB and it's almost 10x more.
>
> 2011-07-27 12:57:35.541556    pg v54158: 6986 pgs: 8 active, 6978
> active+clean; 32104 MB data, 310 GB used, 29642 GB / 31512 GB avail;
>
> I'am putting files beetwen 4-50KB on RADOS via s3 clilent, and radosgw.
>
>
> Can you explain that to me on this example from real life ??

Hmm, what underlying filesystem are you using? Do you have any logging
enabled, and what disk is it logging to? Are all your OSDs running
under the same OS, or are they in virtual machines?
If I remember correctly, that "total used" count is generated by
looking at df or something for the drives in question -- if there's
other data on the same drive as the OSD, it'll get (admittedly
incorrectly) counted as part of the "total used" by RADOS even if
RADOS can't touch it.
-Greg

Re: Two questions

[Sławomir Skowron]

Dnia 27 lip 2011 o godz. 18:15 Gregory Farnum
<gregory.farnum@dreamhost.com> napisał(a):

> 2011/7/27 Sławomir Skowron <szibis@gmail.com>:
>> Ok, I will show example:
>>
>> rados df
>> pool name                 KB      objects       clones     degraded
>>  unfound           rd        rd KB           wr        wr KB
>> .log                  558212            5            0            0
>>       0            0            0      2844888      2844888
>> .pool                      1            1            0            0
>>       0            0            0            8            8
>> .rgw                       0            6            0            0
>>       0            0            0            1            0
>> .users                     1            1            0            0
>>       0            0            0            1            1
>> .users.email               1            1            0            0
>>       0            0            0            1            1
>> .users.uid                 2            2            0            0
>>       0            1            0            2            2
>> data                       0            0            0            0
>>       0            0            0            0            0
>> metadata                   0            0            0            0
>>       0            0            0            0            0
>> rbd                        0            0            0            0
>>       0            0            0            0            0
>> sstest              32244226      2841055            0       653353
>>       0            0            0     17066724     32370391
>>  total used       324792996      2841071
>>  total avail    31083452176
>>  total space    33043244460
>>
>> It means I have almost 3mln of objects in sstest.
>>
>> pg_pool 7 'sstest' pg_pool(rep pg_size 3 crush_ruleset 0 object_hash
>> rjenkins pg_num 8 pgp_num 8 lpg_num 0 lpgp_num 0 last_change 21 owner
>> 0)
>>
>> 3 copies in this pool.
>>
>> sstest used 32.244.226 KB + log 558.212 KB = 32.802.438 KB
>>
>> Total used is 324.792.996 KB and it's almost 10x more.
>>
>> 2011-07-27 12:57:35.541556    pg v54158: 6986 pgs: 8 active, 6978
>> active+clean; 32104 MB data, 310 GB used, 29642 GB / 31512 GB avail;
>>
>> I'am putting files beetwen 4-50KB on RADOS via s3 clilent, and radosgw.
>>
>>
>> Can you explain that to me on this example from real life ??
>
> Hmm, what underlying filesystem are you using? Do you have any logging
> enabled, and what disk is it logging to? Are all your OSDs running
> under the same OS, or are they in virtual machines?

I use ext4. I have loging enabled in osd and mon for most everything in 20.
Every osd is running on same version of debian 6 booted from network.
In my testing case i use two machines and they are using the same
configuration and system.
They are not a VM's.

> If I remember correctly, that "total used" count is generated by
> looking at df or something for the drives in question -- if there's
> other data on the same drive as the OSD, it'll get (admittedly
> incorrectly) counted as part of the "total used" by RADOS even if
> RADOS can't touch it.
> -Greg

Every machine setup is same and looks like this:

2 x 300GB SAS -> hardware RAID1 -> root filesystem and ceph logs in
/var/log/(osd,mon)
12 x 2TB SATA -> hardware RAID5 + 1 Spare -> 16 x 1TB (raid usable
space is bigger) ceph osd mounted as ext4 in /data/osd.(osd id)

For test only i run journal in the same lun as the osd, because two
SAS drives is not enough, but in real scenario system will run on one
SAS drive for OS, journals on one SSD, and rest on SATA drives, or
system and journals on RAID 1 SSD drives.

If i count journals on two machines it 32 x 512MB, and its only 16GB
more in rados df in calculation. Where is more ??

With regards
Slawomir Skowron
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Two questions

[Sławomir Skowron]

Dnia 27 lip 2011 o godz. 17:52 Sage Weil <sage@newdream.net> napisał(a):

> On Wed, 27 Jul 2011, S?awomir Skowron wrote:
>> Hello. I have some questions.
>>
>> 1. Is there any chance to change default 4MB object size to for
>> example 1MB or less ??
>
> Yeah.  You can use the cephfs to set the default layout on the root
> directory, which will apply to any new files.  See 'man cephfs'.

Ok. Thanks.

>> 2. I have create cluster of two mons, and 32 osd (1TB each) on two
>> machines. At this radosgw with apache2 for test. When i putting data
>> from s3 client to rados, everything is ok, but when one of osd is
>> going down all s3 clients going to be freeze for some seconds, before
>> ceph sign this osd as down, and then everything is starting working
>> again wit degraded osd. How can i tune this marking as a down for ms
>> or 0 :) and not for seconds freeze ??
>
> The 'osd heartbeat grace' defualt is 20.  Stick something like
>
>    osd heartbeat grace = 5
>
> in the [osd] section of your ceph.conf

Thanks, i will check this in work.

> sage
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Two questions

[Sławomir Skowron]

2011/7/28 Sławomir Skowron <szibis@gmail.com>:
> Dnia 27 lip 2011 o godz. 17:52 Sage Weil <sage@newdream.net> napisał(a):
>
>> On Wed, 27 Jul 2011, S?awomir Skowron wrote:
>>> Hello. I have some questions.
>>>
>>> 1. Is there any chance to change default 4MB object size to for
>>> example 1MB or less ??
>>
>> Yeah.  You can use the cephfs to set the default layout on the root
>> directory, which will apply to any new files.  See 'man cephfs'.
>
> Ok. Thanks.
>
>>> 2. I have create cluster of two mons, and 32 osd (1TB each) on two
>>> machines. At this radosgw with apache2 for test. When i putting data
>>> from s3 client to rados, everything is ok, but when one of osd is
>>> going down all s3 clients going to be freeze for some seconds, before
>>> ceph sign this osd as down, and then everything is starting working
>>> again wit degraded osd. How can i tune this marking as a down for ms
>>> or 0 :) and not for seconds freeze ??
>>
>> The 'osd heartbeat grace' defualt is 20.  Stick something like
>>
>>    osd heartbeat grace = 5
>>
>> in the [osd] section of your ceph.conf
>
> Thanks, i will check this in work.

It works perfect thanks a lot

>
>> sage
>>
>



-- 
-----
Pozdrawiam

Sławek "sZiBis" Skowron
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Two questions

[Sławomir Skowron]

2011/7/28 Sławomir Skowron <szibis@gmail.com>:
> Dnia 27 lip 2011 o godz. 18:15 Gregory Farnum
> <gregory.farnum@dreamhost.com> napisał(a):
>
>> 2011/7/27 Sławomir Skowron <szibis@gmail.com>:
>>> Ok, I will show example:
>>>
>>> rados df
>>> pool name                 KB      objects       clones     degraded
>>>  unfound           rd        rd KB           wr        wr KB
>>> .log                  558212            5            0            0
>>>       0            0            0      2844888      2844888
>>> .pool                      1            1            0            0
>>>       0            0            0            8            8
>>> .rgw                       0            6            0            0
>>>       0            0            0            1            0
>>> .users                     1            1            0            0
>>>       0            0            0            1            1
>>> .users.email               1            1            0            0
>>>       0            0            0            1            1
>>> .users.uid                 2            2            0            0
>>>       0            1            0            2            2
>>> data                       0            0            0            0
>>>       0            0            0            0            0
>>> metadata                   0            0            0            0
>>>       0            0            0            0            0
>>> rbd                        0            0            0            0
>>>       0            0            0            0            0
>>> sstest              32244226      2841055            0       653353
>>>       0            0            0     17066724     32370391
>>>  total used       324792996      2841071
>>>  total avail    31083452176
>>>  total space    33043244460
>>>
>>> It means I have almost 3mln of objects in sstest.
>>>
>>> pg_pool 7 'sstest' pg_pool(rep pg_size 3 crush_ruleset 0 object_hash
>>> rjenkins pg_num 8 pgp_num 8 lpg_num 0 lpgp_num 0 last_change 21 owner
>>> 0)
>>>
>>> 3 copies in this pool.
>>>
>>> sstest used 32.244.226 KB + log 558.212 KB = 32.802.438 KB
>>>
>>> Total used is 324.792.996 KB and it's almost 10x more.
>>>
>>> 2011-07-27 12:57:35.541556    pg v54158: 6986 pgs: 8 active, 6978
>>> active+clean; 32104 MB data, 310 GB used, 29642 GB / 31512 GB avail;
>>>
>>> I'am putting files beetwen 4-50KB on RADOS via s3 clilent, and radosgw.
>>>
>>>
>>> Can you explain that to me on this example from real life ??
>>
>> Hmm, what underlying filesystem are you using? Do you have any logging
>> enabled, and what disk is it logging to? Are all your OSDs running
>> under the same OS, or are they in virtual machines?
>
> I use ext4. I have loging enabled in osd and mon for most everything in 20.
> Every osd is running on same version of debian 6 booted from network.
> In my testing case i use two machines and they are using the same
> configuration and system.
> They are not a VM's.
>
>> If I remember correctly, that "total used" count is generated by
>> looking at df or something for the drives in question -- if there's
>> other data on the same drive as the OSD, it'll get (admittedly
>> incorrectly) counted as part of the "total used" by RADOS even if
>> RADOS can't touch it.

When you write this i check something and i think it is what you suggest.

Because of my test before i mount ext4 filesystems in /data/osd.(osd
id), but /data was a symlink to /var/data/ and i think total used
space was higher by a size of var, and there are logs, lots of logs
:). Ceph produce many logs in such verbosity. Tell me if im wrong.

Now its look like this, and it's looks better :)

2011-07-28 11:44:08.227278    pg v110939: 6986 pgs: 8 active, 6978
active+clean; 42441 MB data, 223 GB used, 29457 GB / 31240 GB avail

rados df
pool name                 KB      objects       clones     degraded
  unfound           rd        rd KB           wr        wr KB
.log                  694273            6            0            0
       0            0            0      3539909      3539909
.pool                      1            1            0            0
       0            0            0            8            8
.rgw                       0            6            0            0
       0            0            0            1            0
.users                     1            1            0            0
       0            0            0            1            1
.users.email               1            1            0            0
       0            0            0            1            1
.users.uid                 2            2            0            0
       0            1            0            2            2
data                       0            0            0            0
       0            0            0            0            0
metadata                   0            0            0            0
       0            0            0            0            0
rbd                        0            0            0            0
       0            0            0            0            0
sstest              42766318      3483690            0            0
       0            0            0     20922736     42892546
  total used       234415408      3483707
  total avail    30888365376
  total space    32757780072

>> -Greg
>
> Every machine setup is same and looks like this:
>
> 2 x 300GB SAS -> hardware RAID1 -> root filesystem and ceph logs in
> /var/log/(osd,mon)
> 12 x 2TB SATA -> hardware RAID5 + 1 Spare -> 16 x 1TB (raid usable
> space is bigger) ceph osd mounted as ext4 in /data/osd.(osd id)
>
> For test only i run journal in the same lun as the osd, because two
> SAS drives is not enough, but in real scenario system will run on one
> SAS drive for OS, journals on one SSD, and rest on SATA drives, or
> system and journals on RAID 1 SSD drives.
>
> If i count journals on two machines it 32 x 512MB, and its only 16GB
> more in rados df in calculation. Where is more ??
>
> With regards
> Slawomir Skowron
>



-- 
-----
Pozdrawiam

Sławek "sZiBis" Skowron
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Two questions

[Gregory Farnum]

2011/7/28 Sławomir Skowron <szibis@gmail.com>:
> Because of my test before i mount ext4 filesystems in /data/osd.(osd
> id), but /data was a symlink to /var/data/ and i think total used
> space was higher by a size of var, and there are logs, lots of logs
> :). Ceph produce many logs in such verbosity. Tell me if im wrong.
>
> Now its look like this, and it's looks better :)
>
> 2011-07-28 11:44:08.227278    pg v110939: 6986 pgs: 8 active, 6978
> active+clean; 42441 MB data, 223 GB used, 29457 GB / 31240 GB avail
>
> rados df
> pool name                 KB      objects       clones     degraded
>  unfound           rd        rd KB           wr        wr KB
> .log                  694273            6            0            0
>       0            0            0      3539909      3539909
> .pool                      1            1            0            0
>       0            0            0            8            8
> .rgw                       0            6            0            0
>       0            0            0            1            0
> .users                     1            1            0            0
>       0            0            0            1            1
> .users.email               1            1            0            0
>       0            0            0            1            1
> .users.uid                 2            2            0            0
>       0            1            0            2            2
> data                       0            0            0            0
>       0            0            0            0            0
> metadata                   0            0            0            0
>       0            0            0            0            0
> rbd                        0            0            0            0
>       0            0            0            0            0
> sstest              42766318      3483690            0            0
>       0            0            0     20922736     42892546
>  total used       234415408      3483707
>  total avail    30888365376
>  total space    32757780072

Okay, so now you've got (42766318+694273)*3KB=124GB of data, and 223GB
used. I guess your OSD journals are 512MB each, so that's another
16GB, which still leaves more unexplained space usage than I would
expect.
But it's probably just some peculiarity of how your system is set up;
you could check and see how the numbers change when you add new
objects to the system to make sure it's just a base case rather than
something to worry about. :)
-Greg

Re: Two questions

[Sławomir Skowron]

Yes i have made a test, and now everything is ok. Thanks for help.

iSS

Dnia 28 lip 2011 o godz. 18:36 Gregory Farnum
<gregory.farnum@dreamhost.com> napisał(a):

> 2011/7/28 Sławomir Skowron <szibis@gmail.com>:
>> Because of my test before i mount ext4 filesystems in /data/osd.(osd
>> id), but /data was a symlink to /var/data/ and i think total used
>> space was higher by a size of var, and there are logs, lots of logs
>> :). Ceph produce many logs in such verbosity. Tell me if im wrong.
>>
>> Now its look like this, and it's looks better :)
>>
>> 2011-07-28 11:44:08.227278    pg v110939: 6986 pgs: 8 active, 6978
>> active+clean; 42441 MB data, 223 GB used, 29457 GB / 31240 GB avail
>>
>> rados df
>> pool name                 KB      objects       clones     degraded
>>  unfound           rd        rd KB           wr        wr KB
>> .log                  694273            6            0            0
>>       0            0            0      3539909      3539909
>> .pool                      1            1            0            0
>>       0            0            0            8            8
>> .rgw                       0            6            0            0
>>       0            0            0            1            0
>> .users                     1            1            0            0
>>       0            0            0            1            1
>> .users.email               1            1            0            0
>>       0            0            0            1            1
>> .users.uid                 2            2            0            0
>>       0            1            0            2            2
>> data                       0            0            0            0
>>       0            0            0            0            0
>> metadata                   0            0            0            0
>>       0            0            0            0            0
>> rbd                        0            0            0            0
>>       0            0            0            0            0
>> sstest              42766318      3483690            0            0
>>       0            0            0     20922736     42892546
>>  total used       234415408      3483707
>>  total avail    30888365376
>>  total space    32757780072
>
> Okay, so now you've got (42766318+694273)*3KB=124GB of data, and 223GB
> used. I guess your OSD journals are 512MB each, so that's another
> 16GB, which still leaves more unexplained space usage than I would
> expect.
> But it's probably just some peculiarity of how your system is set up;
> you could check and see how the numbers change when you add new
> objects to the system to make sure it's just a base case rather than
> something to worry about. :)
> -Greg
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Two questions

[Greg KH]

On Wed, Oct 28, 2015 at 04:14:19PM +0000, Anuz Pratap Singh Tomar wrote:
> 
> 
> On Wed, Oct 28, 2015 at 4:08 PM, Albino B Neto <bino@bino.us> wrote:
> 
>     2015-10-28 13:57 GMT-02:00 Anuz Pratap Singh Tomar <
>     chambilkethakur at gmail.com>:
>     > 1. Is this book still relevant?
>     > Linux Kernel in Nutshell by GKH
>     > http://www.kroah.com/lkn/
> 
>     Yes. You read here also [0].
> 
>     0 - https://www.kernel.org/doc/Documentation/
> 
> 
> Thanks, I am aware of documentation. My question was wrt the changing nature of
> kernel and book has not been changed for quite few years now.?

It's a bit out of date, yes, but the core ideas should still be mostly
the same.

>     > 2. Which is the official mailing list discussion on staging tree/driver?
> 
>     Mailing[1] of Linux.
> 
>     1 - http://vger.kernel.org/vger-lists.html
> 
> 
> I have searched for a bit before asking the question. Which particular one is
> for staging tree/driver?
> As quick Ctrl+f with staging yields no results.

The MAINTAINERS file will contain that information, and you can, and
should, use the scripts/get_maintainer.pl script on a patch or a file to
find out where to send it to, including the proper mailing list.

good luck!

greg k-h

Two questions

[michi1 at michaelblizek.twilightparadox.com]

Hi!
On 15:57 Wed 28 Oct     , Anuz Pratap Singh Tomar wrote:
...
> 2. Which is the official mailing list discussion on staging tree/driver?

You can find relevant lists+contacts in the MAINTAINERS file which is located
in the root directory of the kernel source.

	-Michi
-- 
programing a layer 3+4 network protocol for mesh networks
see http://michaelblizek.twilightparadox.com

Two questions

[Anuz Pratap Singh Tomar]

On Wed, Oct 28, 2015 at 4:08 PM, Albino B Neto <bino@bino.us> wrote:

> 2015-10-28 13:57 GMT-02:00 Anuz Pratap Singh Tomar <
> chambilkethakur at gmail.com>:
> > 1. Is this book still relevant?
> > Linux Kernel in Nutshell by GKH
> > http://www.kroah.com/lkn/
>
> Yes. You read here also [0].
>
> 0 - https://www.kernel.org/doc/Documentation/


Thanks, I am aware of documentation. My question was wrt the changing
nature of kernel and book has not been changed for quite few years now.

>
>
> > 2. Which is the official mailing list discussion on staging tree/driver?
>
> Mailing[1] of Linux.
>
> 1 - http://vger.kernel.org/vger-lists.html


I have searched for a bit before asking the question. Which particular one
is for staging tree/driver?
As quick Ctrl+f with staging yields no results.


>
>
>
>        Albino
>



-- 
Thank you
Warm Regards
Anuz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20151028/c4ad3fd9/attachment.html 

Two questions

[Albino B Neto]

2015-10-28 13:57 GMT-02:00 Anuz Pratap Singh Tomar <chambilkethakur@gmail.com>:
> 1. Is this book still relevant?
> Linux Kernel in Nutshell by GKH
> http://www.kroah.com/lkn/

Yes. You read here also [0].

0 - https://www.kernel.org/doc/Documentation/

> 2. Which is the official mailing list discussion on staging tree/driver?

Mailing[1] of Linux.

1 - http://vger.kernel.org/vger-lists.html


       Albino

Two questions

[Anuz Pratap Singh Tomar]

1. Is this book still relevant?
Linux Kernel in Nutshell by GKH
http://www.kroah.com/lkn/

2. Which is the official mailing list discussion on staging tree/driver?

-- 
Thank you
Warm Regards
Anuz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20151028/436b5282/attachment.html 

Re: Two questions

[Glynn Clements]

MARG wrote:

> Following my previous question ("Subject: cutting"), how can i:
> 
> 1) Detect if the file filname being stripped is missing in the
> filesystem, and accordingly, abort the script with an error;

	if [ ! -f "$file" ] ; then
		echo "file '$file' not found" >&2
		exit 1
	fi

> 2) How can i change directories inside a bash script ("cd" doesn't works).

A process can only change its own working directory; you cannot change
the working directory of another process. E.g. a script cannot change
the working directory of its parent (the process from which the script
was run).

-- 
Glynn Clements <glynn@gclements.plus.com>

Re: Two questions

[Adrian C.]

What do you mean `cd` is not working?

#This is scr.sh
#!/bin/bash
for i in `find /root -name "$1"` ; do echo "it's 
here $i" ; do_whatever_to $i ; done
cd /root ; pwd ; ls -l tcp*.tar.gz
cd /home ; pwd ; ls -l ps.tar
cd ~

it will give you this:

root@ludmila:~# . scr php*.tar.gz
it's here /root/phpshell-latest.tar.gz
/root
-rw-r--r--  1 root root  655088 2005-04-07 03:47 tcpdump-3.9.1-096.tar.gz
-rw-r--r--  1 root root 3773308 2004-11-05 01:09 tcptrace-6.6.7.tar.gz
/home
-rw-r--r--  1 root root 163840 2005-04-13 22:44 ps.tar



--Adrian.

At 02:19 AM 12/11/2005, MARG wrote:
>Hi,
>
>Following my previous question ("Subject: cutting"), how can i:
>
>1) Detect if the file filname being stripped is missing in the
>filesystem, and accordingly, abort the script with an error;
>
>2) How can i change directories inside a bash script ("cd" doesn't works).
>
>Any help would be apreciated.
>
>Warm Regards,
>Mário Gamito
>-
>To unsubscribe from this list: send the line "unsubscribe linux-admin" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html


-
To unsubscribe from this list: send the line "unsubscribe linux-admin" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Two questions

[MARG]

Hi,

Following my previous question ("Subject: cutting"), how can i:

1) Detect if the file filname being stripped is missing in the
filesystem, and accordingly, abort the script with an error;

2) How can i change directories inside a bash script ("cd" doesn't works).

Any help would be apreciated.

Warm Regards,
Mário Gamito
-
To unsubscribe from this list: send the line "unsubscribe linux-admin" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

two questions

[kartikey bhatt]

I have two questions.

(1) in bsd when a socket is created, the process
creating the socket is passed as a structure
to so_create call. in linux how can we have
this mechanism by which when a socket is created
we can access the task_t of the process creating the
socket?

(2) given the sk_buff* how to lookup the associated
socket?

can anyone please help me out.

--Bhatt Kartikey Mahendra

_________________________________________________________________
Looking for a soulmate? http://www.shaadi.com/ptnr.php?ptnr=hmltag Log onto 
shaadi.com

Re: two questions

[kartikey bhatt]

Matthias Urlichs wrote
>>Those questions imply that you want to do "something".
>>Please tell us what that "something" is.

i am porting netnice patch of freebsd to linux and thats "something".
please help.

--kartikey

_________________________________________________________________
Drive away out of trouble. http://go.msnserver.com/IN/52042.asp Get a loan 
against your existing car.

Re: two questions

[Matthias Urlichs]

Hi, kartikey bhatt wrote:

> (1) in bsd when a socket is created, the process
> creating the socket is passed as a structure
> to so_create call. in linux how can we have
> this mechanism by which when a socket is created
> we can access the task_t of the process creating the
> socket?
> 
> (2) given the sk_buff* how to lookup the associated
> socket?
> 
Those questions imply that you want to do "something".
Please tell us what that "something" is.

-- 
Matthias Urlichs

two questions

[kartikey bhatt]

i have two questions if anyone can help me out?

(1) in bsd when a socket is created, the process
creating the socket is passed as a structure
to so_create call. in linux how can we have
this mechanism by which when a socket is created
we can access the task_t of the process creating the
socket?

(2) given the sk_buff* how to lookup the associated
socket?

please help me out.

--kartikey

_________________________________________________________________
Get FREE unlimited Citibank ATM transactions! 
http://go.msnserver.com/IN/52040.asp Discover real-value banking here.

Re: Two Questions

[Arun Sen]

Thanks ... several of them ... I am looking all the old patches and new ones
...


----- Original Message ----- 
From: "Greg KH" <greg@kroah.com>
To: "Arun Sen" <arunsen@verizon.net>
Cc: <linux-kernel@vger.kernel.org>
Sent: Saturday, June 19, 2004 11:42 PM
Subject: Re: Two Questions


> On Sat, Jun 19, 2004 at 06:47:47PM -0500, Arun Sen wrote:
> > Hi
> > I am doing some research that deals with patches to the kernel.
>
> What kind of research?  For what end result?
>
> > Here are my questions:
> >
> > [1]  I have obtained the patches for different versions of the kernel
from a
> > mirror website.
>
> What web site?  What location?
>
> > However, they do not tell me who are the authors of these
> > patches and what are their addresses/emails.  How can I get this
> > information?
>
> Which patches do you have questions about?
>
> > [2]  What is the actual process of approval of a patch that goes into
the
> > kernel?
>
> Did you read the Documentation/SubmittingPatches file which documents
> the process?
>
> thanks,
>
> greg k-h

Re: Two Questions

[Jesper Juhl]

On Sat, 19 Jun 2004, Arun Sen wrote:

> mirror website.  However, they do not tell me who are the authors of these
> patches and what are their addresses/emails.  How can I get this
> information?

Looking in the Changelogs for the different kernel versions is one place
to get some info.

--
Jesper Juhl <juhl-lkml@dif.dk>

Re: Two Questions

[Greg KH]

On Sat, Jun 19, 2004 at 06:47:47PM -0500, Arun Sen wrote:
> Hi
> I am doing some research that deals with patches to the kernel.

What kind of research?  For what end result?

> Here are my questions:
> 
> [1]  I have obtained the patches for different versions of the kernel from a
> mirror website.

What web site?  What location?

> However, they do not tell me who are the authors of these
> patches and what are their addresses/emails.  How can I get this
> information?

Which patches do you have questions about?

> [2]  What is the actual process of approval of a patch that goes into the
> kernel?

Did you read the Documentation/SubmittingPatches file which documents
the process?

thanks,

greg k-h

Two Questions

[Arun Sen]

Hi
I am doing some research that deals with patches to the kernel.  Here are my
questions:

[1]  I have obtained the patches for different versions of the kernel from a
mirror website.  However, they do not tell me who are the authors of these
patches and what are their addresses/emails.  How can I get this
information?

[2]  What is the actual process of approval of a patch that goes into the
kernel?

Thanks.

Arun

Re: two questions

[Glynn Clements]

pacho baratta wrote:

> 1° my cat -v is like this:
> ^@V^@e^@r^@i^@f^@y^@ ^@c^@o^@m^@p^@l^@e^@t^@e^@d^@
> ^@o^@n^@ ^@9^@/^@5^@/^@2^@0^@0^@3^@ ^@a^@t^@ ^@1^@:^@5^@3^@
> ^@A^@M^@.^@^M^@
> 
> i do i delete al this "^@" ?
> 
> 2° i have a file in this format:
> Little-endian UTF-16 Unicode English character data, with CRLF, CR line
> terminators
> 
> how can i convert to simple ascii??

	iconv -f UTF-16 -t UTF-8 oldfile > newfile
or:
	iconv -f UTF-16 -t ASCII oldfile > newfile

If the source file only contains characters which can be represented
as ASCII (i.e. codes 0-127), there will be no difference between
converting to UTF-8 and converting to ASCII. OTOH, if the source file
contains any non-ASCII characters, converting to UTF-8 will succeed
while converting to ASCII will fail at the first non-ASCII character.

-- 
Glynn Clements <glynn.clements@virgin.net>
-
To unsubscribe from this list: send the line "unsubscribe linux-admin" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

two questions

[pacho baratta]

morning all.

i have 2 questions:

1° my cat -v is like this:
^@V^@e^@r^@i^@f^@y^@ ^@c^@o^@m^@p^@l^@e^@t^@e^@d^@
^@o^@n^@ ^@9^@/^@5^@/^@2^@0^@0^@3^@ ^@a^@t^@ ^@1^@:^@5^@3^@
^@A^@M^@.^@^M^@

i do i delete al this "^@" ?

2° i have a file in this format:
Little-endian UTF-16 Unicode English character data, with CRLF, CR line
terminators

how can i convert to simple ascii??

tnx, pacho


-
To unsubscribe from this list: send the line "unsubscribe linux-admin" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: two questions

[Jim Reimer]

On Saturday 10 August 2002 00:51 am, Adam Luchjenbroers wrote:
> 1) How do I manually fsck my root filesystem?
>
> 2) Where can I find a program to convert windows .ICO files into a more
> open file format (ie. PNG).
>

Regarding second question:  netpbm from http://sourceforge.net

netpbm is a group of graphics-manipulation programs (you may
already have it on your system) that will convert from just
about any graphics file format (including Windows icon) to
any other format.  It's a two-step process, first convert
from the source format to netpbm's file format, then convert
that to whatever you want.  Like this:

winicontoppm winiconfile.ico > temp
ppmtogif temp > winiconfile.gif
(or)
pnmtopng temp > winiconfile.png

-jdr-


-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: two questions

[szonyi calin]

 --- Adam Luchjenbroers <adam@luchjenbroers.com> a
écrit : > 1) How do I manually fsck my root
filesystem?
> 

su -
init 1
<mount every *nix filesystem you have on your 
hard drive read-only>
mount -o ro,remount /
fsck /dev/hdax 
x - your root partition 

> 2) Where can I find a program to convert windows
> .ICO files into a more open 
> file format (ie. PNG).
> 

try acdsee on windoze
There was an old program called pv.exe (pve.exe)
for dos which did that.

Calin

___________________________________________________________
Do You Yahoo!? -- Une adresse @yahoo.fr gratuite et en français !
Yahoo! Mail : http://fr.mail.yahoo.com
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: two questions

[Haines Brown]

Adam,

Back to you again. As I mentioned, I had wiped out my boot diskette,
and needed to figure out how to make another. That I have done. 

You need to use the mkdisk disk command, with the argument consisting
of your kernel number (what returns from uname -a). You will be
prompted to insert the diskette. The image created on it will use most
of its free space, and so you may want to create a second diskette
with various commands such as fsck.

HOWEVER, I have not tested this (except on my wife's Windows machine,
where it seems to boot the kernel, but then passes the boot back to
her machine's BIOS for some reason, with a result of a loop rather
than ending in a command prompt. Also, I've never (unwisely) tried to
run commands from a second floppy, but will definitely try it
now. Hope all this helps, although only a guess at this point.

Haines 

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

two questions

[Adam Luchjenbroers]

1) How do I manually fsck my root filesystem?

2) Where can I find a program to convert windows .ICO files into a more open 
file format (ie. PNG).


-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs