[Cluster-devel] when do I need to start cpglockd

[Cluster-devel] when do I need to start cpglockd

[Dietmar Maurer]

I just tried to upgrade my packages form STABEL31 to STABLE32, and noticed that there is a new daemon 'cpglockd'

That daemon is always started with the default init script. Is that really necessary?

I just wonder why rgmanager needs 2 different locking mechanism - dlm and cpglockd?

The manual page says:

     -C     Explicitly  disable or enable CPG-based locking.  The default is
              to enable this when RRP is turned on (which requires  a  cluster
              outage).  This option MUST be the same on all hosts in the clus\u2010
              ter and must only be enabled or disabled with all  instances  of
              rgmanager turned off.


But why do I want to enable CPG-based locking, or why not?

- Dietmar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/cluster-devel/attachments/20120613/23ec4a1f/attachment.htm>

[Cluster-devel] when do I need to start cpglockd

[Ryan McCabe]

On Wed, Jun 13, 2012 at 03:26:22PM +0000, Dietmar Maurer wrote:
> I just tried to upgrade my packages form STABEL31 to STABLE32, and noticed that there is a new daemon 'cpglockd'
> 
> That daemon is always started with the default init script. Is that really necessary?
> 
> I just wonder why rgmanager needs 2 different locking mechanism - dlm and cpglockd?
[snip]
> But why do I want to enable CPG-based locking, or why not?

Cpglockd was created to work around problems (mainly very slow failover) when
corosync's RRP mode is enabled and dlm locking is used. It will be
started from the cman init script only when RRP mode is detected. In
this case, dlm locks won't be used at all by rgmanager.


Ryan 

[Cluster-devel] when do I need to start cpglockd

[Dietmar Maurer]

> -----Original Message-----
> From: Ryan McCabe [mailto:rmccabe at redhat.com]
> Sent: Donnerstag, 14. Juni 2012 14:21
> To: Dietmar Maurer
> Cc: cluster-devel at redhat.com
> Subject: Re: [Cluster-devel] when do I need to start cpglockd
> 
> On Wed, Jun 13, 2012 at 03:26:22PM +0000, Dietmar Maurer wrote:
> > I just tried to upgrade my packages form STABEL31 to STABLE32, and
> noticed that there is a new daemon 'cpglockd'
> >
> > That daemon is always started with the default init script. Is that really
> necessary?
> >
> > I just wonder why rgmanager needs 2 different locking mechanism - dlm
> and cpglockd?
> [snip]
> > But why do I want to enable CPG-based locking, or why not?
> 
> Cpglockd was created to work around problems (mainly very slow failover)
> when corosync's RRP mode is enabled and dlm locking is used. It will be
> started from the cman init script only when RRP mode is detected. In this
> case, dlm locks won't be used at all by rgmanager.

I can't see that in the current cman init script. Instead, the rgmanager 
init script depends on the cpglockd unconditionally:

# Required-Start:       cman cpglockd

So that is a bug?

- Dietmar

[Cluster-devel] when do I need to start cpglockd

[Ryan McCabe]

On Thu, Jun 14, 2012 at 03:41:39PM +0000, Dietmar Maurer wrote:
> I can't see that in the current cman init script. Instead, the rgmanager 
> init script depends on the cpglockd unconditionally:
> 
> # Required-Start:       cman cpglockd
> 
> So that is a bug?

Hi,

Yes, that's a bug. cpglockd will be started from the rgmanager init
script when RRP mode is enabled.


Ryan

[Cluster-devel] when do I need to start cpglockd

[Fabio M. Di Nitto]

On 06/14/2012 06:06 PM, Ryan McCabe wrote:
> On Thu, Jun 14, 2012 at 03:41:39PM +0000, Dietmar Maurer wrote:
>> I can't see that in the current cman init script. Instead, the rgmanager 
>> init script depends on the cpglockd unconditionally:
>>
>> # Required-Start:       cman cpglockd
>>
>> So that is a bug?
> 
> Hi,
> 
> Yes, that's a bug. cpglockd will be started from the rgmanager init
> script when RRP mode is enabled.
> 
> 
> Ryan
> 

Actually no, it's not a bug.

cpglockd has it's own init script too.

The Required-Start: tells sysvinint that if cpglockd is enabled, it has
to be started before rgmanager.

rgmanager snippet to start cpglockd is there only for backward
compatibility mode that avoids breaking upgrades from non RRP
environments to RRP. This was done so that users didn't need to enable
cpglockd via chkconfig (being a new daemon and all is not known yet).

A "perfect" install would see the user doing:

chkconfig cpglockd on
chkconfig rgmanager on

only for RRP installations. But then again, docs are fresh, cpglockd is
new.. might as well help the users not to shoot their foot with an RRP
gun ;)

Fabio

[Cluster-devel] when do I need to start cpglockd

[Dietmar Maurer]

> > Yes, that's a bug. cpglockd will be started from the rgmanager init
> > script when RRP mode is enabled.
> >
> >
> > Ryan
> >
> 
> Actually no, it's not a bug.
> 
> cpglockd has it's own init script too.

Yes, and that script 'unconditionally' (always) starts cpglockd
 
> The Required-Start: tells sysvinint that if cpglockd is enabled, it has to be
> started before rgmanager.

That tells sysvinint to always start that script before rgmanager.

So we end up with cpglockd always running, although it is not required at all.

What do I miss?

- Dietmar

[Cluster-devel] when do I need to start cpglockd

[Fabio M. Di Nitto]

On 6/19/2012 6:23 AM, Dietmar Maurer wrote:
>>> Yes, that's a bug. cpglockd will be started from the rgmanager init
>>> script when RRP mode is enabled.
>>>
>>>
>>> Ryan
>>>
>>
>> Actually no, it's not a bug.
>>
>> cpglockd has it's own init script too.
> 
> Yes, and that script 'unconditionally' (always) starts cpglockd

Nothing wrong with that. If you ask a daemon to start it will start :)

On top of that, cpglockd is harmless if there is no RRP mode active, or
forcefully disabled.

>  
>> The Required-Start: tells sysvinint that if cpglockd is enabled, it has to be
>> started before rgmanager.
> 
> That tells sysvinint to always start that script before rgmanager.
> 
> So we end up with cpglockd always running, although it is not required at all.
> 
> What do I miss?

It tells sysvinit to start cpglockd before rgmanager IF cpglockd is
enabled via chkconfig, otherwise it is not started. That value is used
only to calculate the symlink S* K** values for rc.d/

Fabio

[Cluster-devel] when do I need to start cpglockd

[Dietmar Maurer]

> > Yes, and that script 'unconditionally' (always) starts cpglockd
> 
> Nothing wrong with that. If you ask a daemon to start it will start :)

For me this is wrong. I have to maintain a debian package, and I do not want to start
unnecessary daemons. So I simply remove that dependency.

Anyways, no problem (I just want to understand what's going on)

Many thanks for your help,

- Dietmar

[Cluster-devel] when do I need to start cpglockd

[Fabio M. Di Nitto]

On 6/19/2012 8:54 AM, Dietmar Maurer wrote:
>>> Yes, and that script 'unconditionally' (always) starts cpglockd
>>
>> Nothing wrong with that. If you ask a daemon to start it will start :)
> 
> For me this is wrong. I have to maintain a debian package, and I do not want to start
> unnecessary daemons. So I simply remove that dependency.
> 

If Debian handling of daemons has changed, then the change is debian
specific, it doesn?t make it a bug for all distributions.

Last I checked if I run:

apt-get install bind9 -> bind9 will start automatically. Or for that
matter also apache2 or....

The init scripts we deliver are "as generic as possible", it doesn?t
mean that they fit everything everywhere.

And then again, expressing an order is correct. If "Required-Start"
behavior in Debian is different than in other distro (I can speak for
Fedora/RHEL here), then clearly there needs to be some distro specific
"tuning".

Fabio

[Cluster-devel] when do I need to start cpglockd

[Dietmar Maurer]

> And then again, expressing an order is correct. If "Required-Start"
> behavior in Debian is different than in other distro (I can speak for
> Fedora/RHEL here), then clearly there needs to be some distro specific
> "tuning".

You simply start a daemon which is not necessary.  And I guess you do that on 
all distros if there is a Required-Start start dependency.

- Dietmar

[Cluster-devel] when do I need to start cpglockd

[Fabio M. Di Nitto]

On 6/19/2012 9:24 AM, Dietmar Maurer wrote:
>> And then again, expressing an order is correct. If "Required-Start"
>> behavior in Debian is different than in other distro (I can speak for
>> Fedora/RHEL here), then clearly there needs to be some distro specific
>> "tuning".
> 
> You simply start a daemon which is not necessary.


>  And I guess you do that on 
> all distros if there is a Required-Start start dependency.

Fresh install on Fedora:

root at fedora16-node2 ~]# chkconfig --list |grep cpg
cpglockd        0:off   1:off   2:off   3:off   4:off   5:off   6:off

[root at fedora16-node2 ~]# chkconfig rgmanager on
[root at fedora16-node2 ~]# chkconfig --list |grep rg
rgmanager       0:off   1:off   2:on    3:on    4:on    5:on    6:off

[root at fedora16-node2 ~]# chkconfig --list |grep cpg
cpglockd        0:off   1:off   2:off   3:off   4:off   5:off   6:off

[reboot]

[root at fedora16-node2 ~]# ps ax|grep cpglockd
 3741 pts/1    S+     0:00 grep --color=auto cpglockd
[root at fedora16-node2 ~]#

[root at fedora16-node2 ~]# clustat
[SNIP]
 service:vip1                   fedora16-node2                 started

As you can see, rgmanager is on, cpglockd off.

At boot rgmanager starts fine, without cpglockd running.

I think the problem here is the interpretation of the LSB specifications
between different distributions. I am not going to argue which one is
right or wrong but the key issue is here:

"An init.d shell script may declare using the "Required-Start: " header
that it shall not be run until certain boot facilities are provided.
This information is used by the installation tool or the boot-time
boot-script execution facility to assure that init scripts are run in
the correct order."

In the fedora world that means that if cpglockd is enabled (via
chkconfig), the Required-Start: make sure that cpglockd is started
before rgmanager, always.

It is possible that other distributions might interpret that as:
"cpglockd must be started even if disabled" when rgmanager
Required-Start: cpglockd and rgmanager is enabled.

So based on the platform I use for testing/development, the daemon does
not start unless it is necessary :)

Fabio

[Cluster-devel] when do I need to start cpglockd

[Dietmar Maurer]

> As you can see, rgmanager is on, cpglockd off.

I see.
 
> At boot rgmanager starts fine, without cpglockd running.
> I think the problem here is the interpretation of the LSB specifications
> between different distributions. I am not going to argue which one is right or
> wrong but the key issue is here:
> 
> "An init.d shell script may declare using the "Required-Start: " header that it
> shall not be run until certain boot facilities are provided.
> This information is used by the installation tool or the boot-time boot-script
> execution facility to assure that init scripts are run in the correct order."
> 
> In the fedora world that means that if cpglockd is enabled (via chkconfig), the
> Required-Start: make sure that cpglockd is started before rgmanager, always.
> 
> It is possible that other distributions might interpret that as:
> "cpglockd must be started even if disabled" when rgmanager
> Required-Start: cpglockd and rgmanager is enabled.
> 
> So based on the platform I use for testing/development, the daemon does
> not start unless it is necessary :)

OK, I was not aware of that.

Many thanks for that detailed reply!

- Dietmar

[Cluster-devel] when do I need to start cpglockd

[Fabio M. Di Nitto]

On 6/19/2012 10:12 AM, Dietmar Maurer wrote:

>> At boot rgmanager starts fine, without cpglockd running.
>> I think the problem here is the interpretation of the LSB specifications
>> between different distributions. I am not going to argue which one is right or
>> wrong but the key issue is here:
>>
>> "An init.d shell script may declare using the "Required-Start: " header that it
>> shall not be run until certain boot facilities are provided.
>> This information is used by the installation tool or the boot-time boot-script
>> execution facility to assure that init scripts are run in the correct order."
>>
>> In the fedora world that means that if cpglockd is enabled (via chkconfig), the
>> Required-Start: make sure that cpglockd is started before rgmanager, always.
>>
>> It is possible that other distributions might interpret that as:
>> "cpglockd must be started even if disabled" when rgmanager
>> Required-Start: cpglockd and rgmanager is enabled.
>>
>> So based on the platform I use for testing/development, the daemon does
>> not start unless it is necessary :)
> 
> OK, I was not aware of that.
> 
> Many thanks for that detailed reply!

So let?s instead try to figure out the correct fix.

Let?s put one minute aside the possibility that some distributions might
use the second interpretation of LSB header and focus only on the
ordering instead.

Dropping Required-Start: might look like an easy fix in the Debian
world, but that could cripple the startup order as cpglockd could
theoretically land after rgmanager (i don?t think it?s possible, but
let?s not take a chance).

I think the correct fix should be:

move the conditional start start_cpglockd function/check from
rgmanager.init to cpglockd.init.

move the "cpglockd is up and running" test from rgmanager.init to
cpglockd.init (that?s a bug as-is now).

cpglockd.init should return 0 (success) if it does not need to run and
would allow rgmanager to start given Debian current interpretation of
LSB header.

rgmanager.init can simply fire cpglockd.init without any check, as those
would be done properly by cpglockd.init.

I think this should solve the issue for Debian and keep current behavior
in Fedora.

Fabio

[Cluster-devel] when do I need to start cpglockd

[Dietmar Maurer]

> rgmanager.init can simply fire cpglockd.init without any check, as those
> would be done properly by cpglockd.init.
> 
> I think this should solve the issue for Debian and keep current behavior in
> Fedora.

Yes, that would work.

- Dietmar