CEPHFS mount error !!!

CEPHFS mount error !!!

[femi anjorin]

Linux 2.6.32-279.19.1.el6.x86_64 x86_64 CentOS 6.3

Pls can somebody help ... This command not working on CentOS 6.3 !!!!

mount -t ceph 172.16.0.25:6789:/ /mnt/mycephfs -o
name=admin,secret=AQCqnw9RmBPpOBAAeJjkIgKGYvyRGlekTpUPog==

 FATAL: Module ceph not found.
 mount.ceph: modprobe failed, exit status 1
 mount error: ceph filesystem not supported by the system


Regards,
Femi





On Tue, Feb 5, 2013 at 1:49 PM, femi anjorin <femi.anjorin@gmail.com> wrote:
>
> Hi ...
>
> Thanks. i set --debug ms = 0. The result is HEALTH_OK ...but i get an
> error when trying to setup an client access to the cluster CEPHFS!!!
> ----------------------------------------------------------------------------------------
> I tried setting up another server which should act as client..
> - i install ceph on it.
> - got the configuration file from the cluster servers to the new
> server ...  /etc/ceph/ceph.conf
> -i did ...  mkdir /mnt/mycephfs
> -i copied the key from ceph.keyring and used it in the command below
> -i try to run this command: mount -t ceph 172.16.0.25:6789:/
> /mnt/mycephfs -o
> name=admin,secret=AQCqnw9RmBPpOBAAeJjkIgKGYvyRGlekTpUPog==
> Here is the result i got:::
>
> [root@testclient]# mount -t ceph 172.16.0.25:6789:/ /mnt/mycephfs -o
> name=admin,secret=AQCqnw9RmBPpOBAAeJjkIgKGYvyRGlekTpUPog==
> FATAL: Module ceph not found.
> mount.ceph: modprobe failed, exit status 1
> mount error: ceph filesystem not supported by the system
>
> Regards,
> Femi.
>
> On Mon, Feb 4, 2013 at 3:27 PM, Joao Eduardo Luis <joao.luis@inktank.com> wrote:
> > This wasn't obvious due to all the debug being outputted, but here's why
> > 'ceph health' wasn't replying with HEALTH_OK:
> >
> >
> > On 02/04/2013 12:21 PM, femi anjorin wrote:
> >>
> >> 2013-02-04 12:56:15.818985 7f149bfff700  1 HEALTH_WARN 4987 pgs
> >> peering; 4987 pgs stuck inactive; 5109 pgs stuck unclean
> >
> >
> > Furthermore, on your other email in which you ran 'ceph health detail', this
> > appear to have gone away, as it is replying with HEALTH_OK again.
> >
> > You might want to set '--debug-ms 0' when you run 'ceph', or set it in your
> > ceph.conf, leaving it at a higher level only for daemons (i.e., under [mds],
> > [mon], [osd]...).  The resulting output will be clearer and more easily
> > understandable.
> >
> >   -Joao
> >

Re: CEPHFS mount error !!!

[Dan Mick]

Yes; as Martin said last night, you don't have the ceph module.
Did you build your own kernel?

See
http://ceph.com/docs/master/install/os-recommendations/#linux-kernel

On 02/05/2013 09:37 PM, femi anjorin wrote:
> Linux 2.6.32-279.19.1.el6.x86_64 x86_64 CentOS 6.3
>
> Pls can somebody help ... This command not working on CentOS 6.3 !!!!
>
> mount -t ceph 172.16.0.25:6789:/ /mnt/mycephfs -o
> name=admin,secret=AQCqnw9RmBPpOBAAeJjkIgKGYvyRGlekTpUPog==
>
>   FATAL: Module ceph not found.
>   mount.ceph: modprobe failed, exit status 1
>   mount error: ceph filesystem not supported by the system
>
>
> Regards,
> Femi
>
>
>
>
>
> On Tue, Feb 5, 2013 at 1:49 PM, femi anjorin <femi.anjorin@gmail.com> wrote:
>>
>> Hi ...
>>
>> Thanks. i set --debug ms = 0. The result is HEALTH_OK ...but i get an
>> error when trying to setup an client access to the cluster CEPHFS!!!
>> ----------------------------------------------------------------------------------------
>> I tried setting up another server which should act as client..
>> - i install ceph on it.
>> - got the configuration file from the cluster servers to the new
>> server ...  /etc/ceph/ceph.conf
>> -i did ...  mkdir /mnt/mycephfs
>> -i copied the key from ceph.keyring and used it in the command below
>> -i try to run this command: mount -t ceph 172.16.0.25:6789:/
>> /mnt/mycephfs -o
>> name=admin,secret=AQCqnw9RmBPpOBAAeJjkIgKGYvyRGlekTpUPog==
>> Here is the result i got:::
>>
>> [root@testclient]# mount -t ceph 172.16.0.25:6789:/ /mnt/mycephfs -o
>> name=admin,secret=AQCqnw9RmBPpOBAAeJjkIgKGYvyRGlekTpUPog==
>> FATAL: Module ceph not found.
>> mount.ceph: modprobe failed, exit status 1
>> mount error: ceph filesystem not supported by the system
>>
>> Regards,
>> Femi.
>>
>> On Mon, Feb 4, 2013 at 3:27 PM, Joao Eduardo Luis <joao.luis@inktank.com> wrote:
>>> This wasn't obvious due to all the debug being outputted, but here's why
>>> 'ceph health' wasn't replying with HEALTH_OK:
>>>
>>>
>>> On 02/04/2013 12:21 PM, femi anjorin wrote:
>>>>
>>>> 2013-02-04 12:56:15.818985 7f149bfff700  1 HEALTH_WARN 4987 pgs
>>>> peering; 4987 pgs stuck inactive; 5109 pgs stuck unclean
>>>
>>>
>>> Furthermore, on your other email in which you ran 'ceph health detail', this
>>> appear to have gone away, as it is replying with HEALTH_OK again.
>>>
>>> You might want to set '--debug-ms 0' when you run 'ceph', or set it in your
>>> ceph.conf, leaving it at a higher level only for daemons (i.e., under [mds],
>>> [mon], [osd]...).  The resulting output will be clearer and more easily
>>> understandable.
>>>
>>>    -Joao
>>>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: CEPHFS mount error !!!

[femi anjorin]

No I didnt build my kernel.

CentOS was working fine on the system and yum install was used to
install ceph. i followed the instruction on this link
http://ceph.com/docs/master/install/rpm/. and it installed fine but i
dont why the cephfs isnt working.

The storage cluster is working fine with all the daemon running
(osd,mon and mds)  and 'ceph health' is responding 'health ok'... and
the storage cluster are also centos 6.3 machines.

But the client machine which i want to use to access the storage
'using CEPHFS' is the one giving this error... this client machine
also have the ceph.conf file in /etc/ceph/ceph.conf. and ceph is
installed on it. if i do ceph -s it respond with health ok ...meaning
it syncs with the cluster.but when i do mount -t ceph
172.16.0.25:6789:/ /mnt/mycephfs -o
name=admin,secret=AQCqnw9RmBPpOBAAeJjkIgKGYvyRGlekTpUPog==

i get:::

FATAL: Module ceph not found.
mount.ceph: modprobe failed, exit status 1
mount error: ceph filesystem not supported by the system

what do u recommend???

Regards

Femi




On Wed, Feb 6, 2013 at 7:55 AM, Dan Mick <dan.mick@inktank.com> wrote:
> Yes; as Martin said last night, you don't have the ceph module.
> Did you build your own kernel?
>
> See
> http://ceph.com/docs/master/install/os-recommendations/#linux-kernel
>
>
> On 02/05/2013 09:37 PM, femi anjorin wrote:
>>
>> Linux 2.6.32-279.19.1.el6.x86_64 x86_64 CentOS 6.3
>>
>> Pls can somebody help ... This command not working on CentOS 6.3 !!!!
>>
>> mount -t ceph 172.16.0.25:6789:/ /mnt/mycephfs -o
>> name=admin,secret=AQCqnw9RmBPpOBAAeJjkIgKGYvyRGlekTpUPog==
>>
>>   FATAL: Module ceph not found.
>>   mount.ceph: modprobe failed, exit status 1
>>   mount error: ceph filesystem not supported by the system
>>
>>
>> Regards,
>> Femi
>>
>>
>>
>>
>>
>> On Tue, Feb 5, 2013 at 1:49 PM, femi anjorin <femi.anjorin@gmail.com>
>> wrote:
>>>
>>>
>>> Hi ...
>>>
>>> Thanks. i set --debug ms = 0. The result is HEALTH_OK ...but i get an
>>> error when trying to setup an client access to the cluster CEPHFS!!!
>>>
>>> ----------------------------------------------------------------------------------------
>>> I tried setting up another server which should act as client..
>>> - i install ceph on it.
>>> - got the configuration file from the cluster servers to the new
>>> server ...  /etc/ceph/ceph.conf
>>> -i did ...  mkdir /mnt/mycephfs
>>> -i copied the key from ceph.keyring and used it in the command below
>>> -i try to run this command: mount -t ceph 172.16.0.25:6789:/
>>> /mnt/mycephfs -o
>>> name=admin,secret=AQCqnw9RmBPpOBAAeJjkIgKGYvyRGlekTpUPog==
>>> Here is the result i got:::
>>>
>>> [root@testclient]# mount -t ceph 172.16.0.25:6789:/ /mnt/mycephfs -o
>>> name=admin,secret=AQCqnw9RmBPpOBAAeJjkIgKGYvyRGlekTpUPog==
>>> FATAL: Module ceph not found.
>>> mount.ceph: modprobe failed, exit status 1
>>> mount error: ceph filesystem not supported by the system
>>>
>>> Regards,
>>> Femi.
>>>
>>> On Mon, Feb 4, 2013 at 3:27 PM, Joao Eduardo Luis <joao.luis@inktank.com>
>>> wrote:
>>>>
>>>> This wasn't obvious due to all the debug being outputted, but here's why
>>>> 'ceph health' wasn't replying with HEALTH_OK:
>>>>
>>>>
>>>> On 02/04/2013 12:21 PM, femi anjorin wrote:
>>>>>
>>>>>
>>>>> 2013-02-04 12:56:15.818985 7f149bfff700  1 HEALTH_WARN 4987 pgs
>>>>> peering; 4987 pgs stuck inactive; 5109 pgs stuck unclean
>>>>
>>>>
>>>>
>>>> Furthermore, on your other email in which you ran 'ceph health detail',
>>>> this
>>>> appear to have gone away, as it is replying with HEALTH_OK again.
>>>>
>>>> You might want to set '--debug-ms 0' when you run 'ceph', or set it in
>>>> your
>>>> ceph.conf, leaving it at a higher level only for daemons (i.e., under
>>>> [mds],
>>>> [mon], [osd]...).  The resulting output will be clearer and more easily
>>>> understandable.
>>>>
>>>>    -Joao
>>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>

Re: CEPHFS mount error !!!

[Jens Kristian Søgaard]

Hi,

> No I didnt build my kernel.

This is the problem. You need to do that on CentOS 6.3.

> But the client machine which i want to use to access the storage
> 'using CEPHFS' is the one giving this error... this client machine

To mount cephfs like that you need to have kernel support. As the Linux 
kernel on CentOS 6.3 is version 2.6.32 and Ceph support wasn't added 
until 2.6.34, you need to compile your own kernel.

-- 
Jens Kristian Søgaard, Mermaid Consulting ApS,
jens@mermaidconsulting.dk,
http://www.mermaidconsulting.com/
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CEPHFS mount error !!!

[Dennis Jacobfeuerborn]

On 02/06/2013 08:24 AM, Jens Kristian Søgaard wrote:
> Hi,
> 
>> No I didnt build my kernel.
> 
> This is the problem. You need to do that on CentOS 6.3.
> 
>> But the client machine which i want to use to access the storage
>> 'using CEPHFS' is the one giving this error... this client machine
> 
> To mount cephfs like that you need to have kernel support. As the Linux
> kernel on CentOS 6.3 is version 2.6.32 and Ceph support wasn't added until
> 2.6.34, you need to compile your own kernel.
> 

The better alternative is probably to install a kernel from
http://elrepo.org/tiki/kernel-lt . "lt" stand for "long term" and should be
fairly stable and "ml" is "mainline" which is even more current but because
of that not quite as stable (currently 3.7.6).

Regards,
  Dennis
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CEPHFS mount error !!!

[Dimitri Maziuk]

On 2/6/2013 5:54 AM, Dennis Jacobfeuerborn wrote:
...
>> To mount cephfs like that you need to have kernel support. As the Linux
>> kernel on CentOS 6.3 is version 2.6.32 and Ceph support wasn't added until
>> 2.6.34, you need to compile your own kernel.
>>
>
> The better alternative is probably to install a kernel from
> http://elrepo.org/tiki/kernel-lt . "lt" stand for "long term" and should be
> fairly stable and "ml" is "mainline" which is even more current but because
> of that not quite as stable (currently 3.7.6).

I had problems booting ml on some/most (dep. on the version) our 
machines, plus it's a pain to track: there's a new one every day.

I do have lt running without problems and mounting cephfs, however, I 
haven't gotten around to the actual ceph testing on it yet so I can't 
say anything about ceph client's performance/stability on it. ("lt" is 
3.0, as I understand it doesn't have the latest and greatest ceph module(?))

Dimitri

Re: CEPHFS mount error !!!

[femi anjorin]

Hi ... I am now testing cephfs on an ubuntu client before going back to centos.

A quick question about this command:
mount -t ceph 192.168.0.1:6789:/ /mnt/mycephfs -o
name=admin,secretfile=/etc/ceph/admin.secret ?
mount -t ceph 192.168.0.1:6789:/ /mnt/mycephfs -o
name=ceph,secretfile=/etc/ceph/ceph.keyring ?

Does it need admin.secret  or can i use ceph.keyring to mount with
cephfs file system?

Just a note: At the initial setup of the cluster i used:
mkcephfs -c /etc/ceph/ceph.conf --mkfs -a -k  /etc/ceph/ceph.keyring.

So i only have ceph.keyring NOT admin.secret ....


if i do:

#mount -t ceph 172.16.0.25:6789:/ /mnt/mycephfs
mount error 22 = Invalid argument
#dmesg
[ 7182.027665] libceph: client0 fsid 31c4d83e-cb0a-4fed-ab6f-581e8d5356f5
[ 7182.027715] libceph: no secret set (for auth_x protocol)
[ 7182.027719] libceph: error -22 on auth protocol 2 init


#mount -t ceph 172.16.0.25:6789:/ /mnt/mycephfs -o
name=ceph,secretfile=/etc/ceph/ceph.keyring
secret is not valid base64: Invalid argument.
adding ceph secret key to kernel failed: Invalid argument.
failed to parse ceph_options
#dmesg
[ 7639.377337] libceph: client0 fsid 31c4d83e-cb0a-4fed-ab6f-581e8d5356f5
[ 7639.378047] libceph: auth method 'x' error -1


Regards,
Femi







On Wed, Feb 6, 2013 at 4:29 PM, Dimitri Maziuk <dmaziuk@bmrb.wisc.edu> wrote:
> On 2/6/2013 5:54 AM, Dennis Jacobfeuerborn wrote:
> ...
>
>>> To mount cephfs like that you need to have kernel support. As the Linux
>>> kernel on CentOS 6.3 is version 2.6.32 and Ceph support wasn't added
>>> until
>>> 2.6.34, you need to compile your own kernel.
>>>
>>
>> The better alternative is probably to install a kernel from
>> http://elrepo.org/tiki/kernel-lt . "lt" stand for "long term" and should
>> be
>> fairly stable and "ml" is "mainline" which is even more current but
>> because
>> of that not quite as stable (currently 3.7.6).
>
>
> I had problems booting ml on some/most (dep. on the version) our machines,
> plus it's a pain to track: there's a new one every day.
>
> I do have lt running without problems and mounting cephfs, however, I
> haven't gotten around to the actual ceph testing on it yet so I can't say
> anything about ceph client's performance/stability on it. ("lt" is 3.0, as
> I
> understand it doesn't have the latest and greatest ceph module(?))
>
> Dimitri
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CEPHFS mount error !!!

[Dan Mick]

On 02/07/2013 03:05 AM, femi anjorin wrote:
> Hi ... I am now testing cephfs on an ubuntu client before going back to centos.
>
> A quick question about this command:
> mount -t ceph 192.168.0.1:6789:/ /mnt/mycephfs -o
> name=admin,secretfile=/etc/ceph/admin.secret ?
> mount -t ceph 192.168.0.1:6789:/ /mnt/mycephfs -o
> name=ceph,secretfile=/etc/ceph/ceph.keyring ?
>
> Does it need admin.secret  or can i use ceph.keyring to mount with
> cephfs file system?

It needs the unadorned base64 key; it doesn't parse keyring files.
You can either supply the key on the commandline or in the file
(which is a good idea to keep the key string out of your command line
and command history).

> Just a note: At the initial setup of the cluster i used:
> mkcephfs -c /etc/ceph/ceph.conf --mkfs -a -k  /etc/ceph/ceph.keyring.
>
> So i only have ceph.keyring NOT admin.secret ....

Right.  admin.secret is a file containing only the key string.  You
can create that file with your own editing, or you can use
ceph-authtool --name client.admin <keyringfile> --print-key

> if i do:
>
> #mount -t ceph 172.16.0.25:6789:/ /mnt/mycephfs
> mount error 22 = Invalid argument
> #dmesg
> [ 7182.027665] libceph: client0 fsid 31c4d83e-cb0a-4fed-ab6f-581e8d5356f5
> [ 7182.027715] libceph: no secret set (for auth_x protocol)
> [ 7182.027719] libceph: error -22 on auth protocol 2 init
>
>
> #mount -t ceph 172.16.0.25:6789:/ /mnt/mycephfs -o
> name=ceph,secretfile=/etc/ceph/ceph.keyring
> secret is not valid base64: Invalid argument.
> adding ceph secret key to kernel failed: Invalid argument.
> failed to parse ceph_options
> #dmesg
> [ 7639.377337] libceph: client0 fsid 31c4d83e-cb0a-4fed-ab6f-581e8d5356f5
> [ 7639.378047] libceph: auth method 'x' error -1
>
>
> Regards,
> Femi
>
>
>
>
>
>
>
> On Wed, Feb 6, 2013 at 4:29 PM, Dimitri Maziuk <dmaziuk@bmrb.wisc.edu> wrote:
>> On 2/6/2013 5:54 AM, Dennis Jacobfeuerborn wrote:
>> ...
>>
>>>> To mount cephfs like that you need to have kernel support. As the Linux
>>>> kernel on CentOS 6.3 is version 2.6.32 and Ceph support wasn't added
>>>> until
>>>> 2.6.34, you need to compile your own kernel.
>>>>
>>>
>>> The better alternative is probably to install a kernel from
>>> http://elrepo.org/tiki/kernel-lt . "lt" stand for "long term" and should
>>> be
>>> fairly stable and "ml" is "mainline" which is even more current but
>>> because
>>> of that not quite as stable (currently 3.7.6).
>>
>>
>> I had problems booting ml on some/most (dep. on the version) our machines,
>> plus it's a pain to track: there's a new one every day.
>>
>> I do have lt running without problems and mounting cephfs, however, I
>> haven't gotten around to the actual ceph testing on it yet so I can't say
>> anything about ceph client's performance/stability on it. ("lt" is 3.0, as
>> I
>> understand it doesn't have the latest and greatest ceph module(?))
>>
>> Dimitri
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>