* subpackages for mgr modules @ 2017-05-19 6:19 Tim Serong 2017-05-19 12:14 ` Sage Weil 2017-05-19 18:25 ` Ken Dreyer 0 siblings, 2 replies; 10+ messages in thread From: Tim Serong @ 2017-05-19 6:19 UTC (permalink / raw) To: ceph-devel Hi All, At some point, we're going to want to have (some of) mgr's python modules shipped not as part of the ceph-mgr package, but as subpackages that can be optionally installed (ceph-mgr-foo, ceph-mgr-bar, whatever). That's easy to do. What I'm wondering about is having installation of an mgr module subpackage automatically enable the loading of that module. Ordinarily, to enable an mgr module that isn't enabled by default at compile time, you'd set "mgr modules = foo bar" in the [mgr] section of ceph.conf. But that will get irritating. We can't have mgr just load everything that exists (someone might want to disable some modules, without necessarily uninstalling them). A couple of approaches I thought of are: 1) Add a command line argument (or arguments) to ceph-mgr to list modules to enable (or disable). This could in turn be set by some variable(s) in /etc/sysconfig/ceph. Subpackages then tweak those variables at install time. 2) Do something with symlinks; set up a directory structure something like this: /usr/lib64/ceph/mgr/enabled/ /usr/lib64/ceph/mgr/modules/ /usr/lib64/ceph/mgr/modules/fsstatus /usr/lib64/ceph/mgr/modules/rest /usr/lib64/ceph/mgr/modules/... Then, any module you want to enable, create a symlink to it from the "enabled" directory. Subpackages could do this at install time. mgr iterates through that directory and loads anything present. (Personally I rather prefer approach 2 to approach 1). I'm sure there's other possibilities. Thoughts, anyone? Thanks, Tim -- Tim Serong Senior Clustering Engineer SUSE tserong@suse.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: subpackages for mgr modules 2017-05-19 6:19 subpackages for mgr modules Tim Serong @ 2017-05-19 12:14 ` Sage Weil 2017-05-19 18:25 ` Ken Dreyer 1 sibling, 0 replies; 10+ messages in thread From: Sage Weil @ 2017-05-19 12:14 UTC (permalink / raw) To: Tim Serong; +Cc: ceph-devel On Fri, 19 May 2017, Tim Serong wrote: > Hi All, > > At some point, we're going to want to have (some of) mgr's python > modules shipped not as part of the ceph-mgr package, but as subpackages > that can be optionally installed (ceph-mgr-foo, ceph-mgr-bar, whatever). > That's easy to do. What I'm wondering about is having installation of > an mgr module subpackage automatically enable the loading of that module. > > Ordinarily, to enable an mgr module that isn't enabled by default at > compile time, you'd set "mgr modules = foo bar" in the [mgr] section of > ceph.conf. But that will get irritating. > > We can't have mgr just load everything that exists (someone might want > to disable some modules, without necessarily uninstalling them). > > A couple of approaches I thought of are: > > 1) Add a command line argument (or arguments) to ceph-mgr to list > modules to enable (or disable). This could in turn be set by some > variable(s) in /etc/sysconfig/ceph. Subpackages then tweak those > variables at install time. > > 2) Do something with symlinks; set up a directory structure something > like this: > > /usr/lib64/ceph/mgr/enabled/ > /usr/lib64/ceph/mgr/modules/ > /usr/lib64/ceph/mgr/modules/fsstatus > /usr/lib64/ceph/mgr/modules/rest > /usr/lib64/ceph/mgr/modules/... > > Then, any module you want to enable, create a symlink to it from the > "enabled" directory. Subpackages could do this at install time. mgr > iterates through that directory and loads anything present. > > (Personally I rather prefer approach 2 to approach 1). > > I'm sure there's other possibilities. Thoughts, anyone? I like #2, too! This is similar to the sites-enabled thing apache does. We can make the config option blank (or something like *?) to indicate using the directory, otherwise the specified list? sage ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: subpackages for mgr modules 2017-05-19 6:19 subpackages for mgr modules Tim Serong 2017-05-19 12:14 ` Sage Weil @ 2017-05-19 18:25 ` Ken Dreyer 2017-05-20 0:24 ` kefu chai 2017-05-21 22:31 ` John Spray 1 sibling, 2 replies; 10+ messages in thread From: Ken Dreyer @ 2017-05-19 18:25 UTC (permalink / raw) To: Tim Serong; +Cc: ceph-devel On Fri, May 19, 2017 at 12:19 AM, Tim Serong <tserong@suse.com> wrote: > We can't have mgr just load everything that exists (someone might want > to disable some modules, without necessarily uninstalling them). I'm curious why not? Ceph is already hard to configure, and "install to enable / uninstall to disable" would make it simpler to use. - Ken ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: subpackages for mgr modules 2017-05-19 18:25 ` Ken Dreyer @ 2017-05-20 0:24 ` kefu chai 2017-05-21 22:31 ` John Spray 1 sibling, 0 replies; 10+ messages in thread From: kefu chai @ 2017-05-20 0:24 UTC (permalink / raw) To: Ken Dreyer; +Cc: Tim Serong, ceph-devel On Sat, May 20, 2017 at 2:25 AM, Ken Dreyer <kdreyer@redhat.com> wrote: > On Fri, May 19, 2017 at 12:19 AM, Tim Serong <tserong@suse.com> wrote: >> We can't have mgr just load everything that exists (someone might want >> to disable some modules, without necessarily uninstalling them). it does not. user can always enable/disable a plugin using "mgr_modules", right? > > I'm curious why not? > > Ceph is already hard to configure, and "install to enable / uninstall > to disable" would make it simpler to use. i agree, if he/she wants to disable it, he/she could just uninstall the plugin. actually, i am planning to split the dashboard into a separated package. so for example, mgr does not need to depend on cherrypy. > > - Ken > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Regards Kefu Chai ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: subpackages for mgr modules 2017-05-19 18:25 ` Ken Dreyer 2017-05-20 0:24 ` kefu chai @ 2017-05-21 22:31 ` John Spray 2017-05-22 8:22 ` Tim Serong 2017-05-22 15:45 ` Ken Dreyer 1 sibling, 2 replies; 10+ messages in thread From: John Spray @ 2017-05-21 22:31 UTC (permalink / raw) To: Ken Dreyer; +Cc: Tim Serong, ceph-devel On Fri, May 19, 2017 at 7:25 PM, Ken Dreyer <kdreyer@redhat.com> wrote: > On Fri, May 19, 2017 at 12:19 AM, Tim Serong <tserong@suse.com> wrote: >> We can't have mgr just load everything that exists (someone might want >> to disable some modules, without necessarily uninstalling them). > > I'm curious why not? Perhaps "can't" is too strong a word, but the way it only loads explicitly enabled modules is intentional. The thinking was... 1. It's not a given that every module will be packaged independently. That's not necessarily going to be useful in all cases -- if/when we reach a point where some modules are so ubiquitous that we want them everywhere, then having lots of separate packages may not be desirable. I'm thinking back to things like the gstreamer-plugins-good/bad/ugly packaging, and the bajillion apache sub-packages -- as a user I usually find this stuff really frustrating because I have a big root partition and my life is simpler if I just pre-emptively have all the modules (but that doesn't mean I want them all running). 2. Which packages are installed on a system is usually something controlled by an external tool, which tends to be different on different sites/in diffferent products. Consequently, the exact steps required to turn off a module are not going to be consistent across sites. It is not a given that a support/admin person will know how to turn off an individual package, unless they're also the person who configured whatever orchestration tool was installing packages. 3. The "install to enable" convention is more welcome on some distros than others. For example, debian (I think) historically did/does this for services, RPM distros historically didn't. > > Ceph is already hard to configure, and "install to enable / uninstall > to disable" would make it simpler to use. My preferred solution would be mon commands for toggling modules "ceph mgr module [enable|disable] <module>", and a record in the MgrMap that says which are enabled. There would be a bit of extra data in the mgr beacon message for the daemons to tell the MgrMonitor which modules are available. The MgrMonitor could also raise a health warning if the mgr daemons have inconsistent plugins. The major advantage of doing that is that all the mgr daemons see the same setting. That way, if someone enables a module on one of their mgr nodes, they don't get a nasty shock when a failover happens and the module is gone on the node they failed over to. It would also let them do it without having to go and directly log into each of their mgr nodes to make a change (as they currently have to do with config files, or would also have to do with package install/uninstall or symlinks). John > > - Ken > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: subpackages for mgr modules 2017-05-21 22:31 ` John Spray @ 2017-05-22 8:22 ` Tim Serong 2017-05-22 8:51 ` Willem Jan Withagen 2017-05-22 15:45 ` Ken Dreyer 1 sibling, 1 reply; 10+ messages in thread From: Tim Serong @ 2017-05-22 8:22 UTC (permalink / raw) To: John Spray, Ken Dreyer; +Cc: ceph-devel On 05/22/2017 08:31 AM, John Spray wrote: > On Fri, May 19, 2017 at 7:25 PM, Ken Dreyer <kdreyer@redhat.com> wrote: >> On Fri, May 19, 2017 at 12:19 AM, Tim Serong <tserong@suse.com> wrote: >>> We can't have mgr just load everything that exists (someone might want >>> to disable some modules, without necessarily uninstalling them). >> >> I'm curious why not? > > Perhaps "can't" is too strong a word, but the way it only loads > explicitly enabled modules is intentional. > > The thinking was... > 1. It's not a given that every module will be packaged independently. > That's not necessarily going to be useful in all cases -- if/when we > reach a point where some modules are so ubiquitous that we want them > everywhere, then having lots of separate packages may not be > desirable. I'm thinking back to things like the > gstreamer-plugins-good/bad/ugly packaging, and the bajillion apache > sub-packages -- as a user I usually find this stuff really frustrating > because I have a big root partition and my life is simpler if I just > pre-emptively have all the modules (but that doesn't mean I want them > all running). > 2. Which packages are installed on a system is usually something > controlled by an external tool, which tends to be different on > different sites/in diffferent products. Consequently, the exact steps > required to turn off a module are not going to be consistent across > sites. It is not a given that a support/admin person will know how to > turn off an individual package, unless they're also the person who > configured whatever orchestration tool was installing packages. What John said. I might not mind having installation of a package trigger default enablement of something (depends on the service), but I really don't like the idea of being required to uninstall something in order to disable it. In addition to the above, even if I *am* the person immediately responsible for installing packages on a system and I know what they all are, I can't necessarily guarantee that package repos are always available, or that if available their content won't change. This causes problems if I uninstall something to disable it, then later want to re-enable it. Here's a fun possible edge case: install a module, then later decide to disable it temporarily by uninstalling it. Later still, I want to re-enable it, but meanwhile updates to ceph have been released in my distro's update repo. I can no longer install the module which matches the version of ceph I have installed, and so need to upgrade my base ceph packages too in order to install the module I want to enable. > 3. The "install to enable" convention is more welcome on some distros > than others. For example, debian (I think) historically did/does this > for services, RPM distros historically didn't. Some services will be enabled by default if installed on openSUSE and SLE now. But not very many :) See https://build.opensuse.org/package/view_file/openSUSE:Factory/systemd-presets-branding-openSUSE/default-openSUSE.preset?expand=1 >> >> Ceph is already hard to configure, and "install to enable / uninstall >> to disable" would make it simpler to use. > > My preferred solution would be mon commands for toggling modules "ceph > mgr module [enable|disable] <module>", and a record in the MgrMap that > says which are enabled. There would be a bit of extra data in the mgr > beacon message for the daemons to tell the MgrMonitor which modules > are available. The MgrMonitor could also raise a health warning if > the mgr daemons have inconsistent plugins. > > The major advantage of doing that is that all the mgr daemons see the > same setting. That way, if someone enables a module on one of their > mgr nodes, they don't get a nasty shock when a failover happens and > the module is gone on the node they failed over to. > > It would also let them do it without having to go and directly log > into each of their mgr nodes to make a change (as they currently have > to do with config files, or would also have to do with package > install/uninstall or symlinks). Good idea, that's much better than my symlink thing. Regards, Tim -- Tim Serong Senior Clustering Engineer SUSE tserong@suse.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: subpackages for mgr modules 2017-05-22 8:22 ` Tim Serong @ 2017-05-22 8:51 ` Willem Jan Withagen 0 siblings, 0 replies; 10+ messages in thread From: Willem Jan Withagen @ 2017-05-22 8:51 UTC (permalink / raw) To: Tim Serong, John Spray, Ken Dreyer; +Cc: ceph-devel On 22-05-17 10:22, Tim Serong wrote: > On 05/22/2017 08:31 AM, John Spray wrote: >> On Fri, May 19, 2017 at 7:25 PM, Ken Dreyer <kdreyer@redhat.com> wrote: >>> On Fri, May 19, 2017 at 12:19 AM, Tim Serong <tserong@suse.com> wrote: >>>> We can't have mgr just load everything that exists (someone might want >>>> to disable some modules, without necessarily uninstalling them). >>> >>> I'm curious why not? >> >> Perhaps "can't" is too strong a word, but the way it only loads >> explicitly enabled modules is intentional. >> >> The thinking was... >> 1. It's not a given that every module will be packaged independently. >> That's not necessarily going to be useful in all cases -- if/when we >> reach a point where some modules are so ubiquitous that we want them >> everywhere, then having lots of separate packages may not be >> desirable. I'm thinking back to things like the >> gstreamer-plugins-good/bad/ugly packaging, and the bajillion apache >> sub-packages -- as a user I usually find this stuff really frustrating >> because I have a big root partition and my life is simpler if I just >> pre-emptively have all the modules (but that doesn't mean I want them >> all running). >> 2. Which packages are installed on a system is usually something >> controlled by an external tool, which tends to be different on >> different sites/in diffferent products. Consequently, the exact steps >> required to turn off a module are not going to be consistent across >> sites. It is not a given that a support/admin person will know how to >> turn off an individual package, unless they're also the person who >> configured whatever orchestration tool was installing packages. > > What John said. I might not mind having installation of a package > trigger default enablement of something (depends on the service), but I > really don't like the idea of being required to uninstall something in > order to disable it. In addition to the above, even if I *am* the > person immediately responsible for installing packages on a system and I > know what they all are, I can't necessarily guarantee that package repos > are always available, or that if available their content won't change. > This causes problems if I uninstall something to disable it, then later > want to re-enable it. > > Here's a fun possible edge case: install a module, then later decide to > disable it temporarily by uninstalling it. Later still, I want to > re-enable it, but meanwhile updates to ceph have been released in my > distro's update repo. I can no longer install the module which matches > the version of ceph I have installed, and so need to upgrade my base > ceph packages too in order to install the module I want to enable. > >> 3. The "install to enable" convention is more welcome on some distros >> than others. For example, debian (I think) historically did/does this >> for services, RPM distros historically didn't. > > Some services will be enabled by default if installed on openSUSE and > SLE now. But not very many :) See > https://build.opensuse.org/package/view_file/openSUSE:Factory/systemd-presets-branding-openSUSE/default-openSUSE.preset?expand=1 > >>> >>> Ceph is already hard to configure, and "install to enable / uninstall >>> to disable" would make it simpler to use. >> >> My preferred solution would be mon commands for toggling modules "ceph >> mgr module [enable|disable] <module>", and a record in the MgrMap that >> says which are enabled. There would be a bit of extra data in the mgr >> beacon message for the daemons to tell the MgrMonitor which modules >> are available. The MgrMonitor could also raise a health warning if >> the mgr daemons have inconsistent plugins. >> >> The major advantage of doing that is that all the mgr daemons see the >> same setting. That way, if someone enables a module on one of their >> mgr nodes, they don't get a nasty shock when a failover happens and >> the module is gone on the node they failed over to. >> >> It would also let them do it without having to go and directly log >> into each of their mgr nodes to make a change (as they currently have >> to do with config files, or would also have to do with package >> install/uninstall or symlinks). Just 2 FreeBSD cts. It is the modus there that installation is just a "get the software step", nothing is automagically enabled. As not to surprise the sysadmin with servers running that he/she has not configured to their liking. Another valid point is versioning. FreeBSD has quarterly package building releases. Once you cross that line for a new release. It could be that you need to upgrade quite some tools to actually have a new and compatible set of applications. So unstall to disable will very likely get you into trouble. Next to the fact that I have been caught in the module-jungle that John references to more that once. Especially Apache/php/php-modules and security issue were/are a notorious pain. --WjW ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: subpackages for mgr modules 2017-05-21 22:31 ` John Spray 2017-05-22 8:22 ` Tim Serong @ 2017-05-22 15:45 ` Ken Dreyer 2017-05-24 20:15 ` Alfredo Deza 1 sibling, 1 reply; 10+ messages in thread From: Ken Dreyer @ 2017-05-22 15:45 UTC (permalink / raw) To: John Spray; +Cc: Tim Serong, ceph-devel On Sun, May 21, 2017 at 4:31 PM, John Spray <jspray@redhat.com> wrote: > My preferred solution would be mon commands for toggling modules "ceph > mgr module [enable|disable] <module>", and a record in the MgrMap that > says which are enabled. Sure. It's just another command users have to do to get to a preferred state. The classic response on these things has been "ceph setup is hard, let's wrap it in ceph-ansible" (or whatever higher-level tool will do all these tricky steps). Then ceph-ansible becomes more and more complex, and I wonder whether the complexity could be addressed elsewhere by simply scaling down the flexibility. - Ken ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: subpackages for mgr modules 2017-05-22 15:45 ` Ken Dreyer @ 2017-05-24 20:15 ` Alfredo Deza 2017-05-24 20:53 ` John Spray 0 siblings, 1 reply; 10+ messages in thread From: Alfredo Deza @ 2017-05-24 20:15 UTC (permalink / raw) To: Ken Dreyer; +Cc: John Spray, Tim Serong, ceph-devel On Mon, May 22, 2017 at 11:45 AM, Ken Dreyer <kdreyer@redhat.com> wrote: > On Sun, May 21, 2017 at 4:31 PM, John Spray <jspray@redhat.com> wrote: >> My preferred solution would be mon commands for toggling modules "ceph >> mgr module [enable|disable] <module>", and a record in the MgrMap that >> says which are enabled. > > Sure. It's just another command users have to do to get to a preferred state. > > The classic response on these things has been "ceph setup is hard, > let's wrap it in ceph-ansible" (or whatever higher-level tool will do > all these tricky steps). Then ceph-ansible becomes more and more > complex, and I wonder whether the complexity could be addressed > elsewhere by simply scaling down the flexibility. I can agree here with how these helpful commands tend to add complexity on the deployment systems. Maybe a good compromise would be the allow both? That is: installation would enable a module, but that could also be disabled (and re-enabled) by the CLI. That way, we aren't forcing users to uninstall always and keeps other systems like ceph-ansible lean. > > - Ken > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: subpackages for mgr modules 2017-05-24 20:15 ` Alfredo Deza @ 2017-05-24 20:53 ` John Spray 0 siblings, 0 replies; 10+ messages in thread From: John Spray @ 2017-05-24 20:53 UTC (permalink / raw) To: Alfredo Deza; +Cc: Ken Dreyer, Tim Serong, ceph-devel On Wed, May 24, 2017 at 9:15 PM, Alfredo Deza <adeza@redhat.com> wrote: > On Mon, May 22, 2017 at 11:45 AM, Ken Dreyer <kdreyer@redhat.com> wrote: >> On Sun, May 21, 2017 at 4:31 PM, John Spray <jspray@redhat.com> wrote: >>> My preferred solution would be mon commands for toggling modules "ceph >>> mgr module [enable|disable] <module>", and a record in the MgrMap that >>> says which are enabled. >> >> Sure. It's just another command users have to do to get to a preferred state. >> >> The classic response on these things has been "ceph setup is hard, >> let's wrap it in ceph-ansible" (or whatever higher-level tool will do >> all these tricky steps). Then ceph-ansible becomes more and more >> complex, and I wonder whether the complexity could be addressed >> elsewhere by simply scaling down the flexibility. > > I can agree here with how these helpful commands tend to add > complexity on the deployment systems. > > Maybe a good compromise would be the allow both? That is: installation > would enable a module, but that > could also be disabled (and re-enabled) by the CLI. That way, we > aren't forcing users to uninstall always and keeps > other systems like ceph-ansible lean. Oops, I wrote a reply to Ken but then failed to click send on it... it was wordy so I'll be briefer now: We do need a way to define what the default set of enabled modules should be, so it probably makes sense to do that via a config setting (perhaps just the existing config setting), and then let installation tools give their preference via that interface too, killing two birds with one stone. I still think none of this should be triggered by packaging. If the user has never used the CLI to enable/disable things, then updates to the config setting would continue to be authoritative. If a user has explicitly disabled a module using the CLI, then that would take precedence over what was in the config setting. That would mean that if an ansible-like tool wanted to definitely enabled a module (even if the user had used the CLI to disable it) then it would need to understand the CLI too. However, if a user was configuring everything using ansible, then ansible would never need to touch the CLI. John > >> >> - Ken >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2017-05-24 20:54 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-05-19 6:19 subpackages for mgr modules Tim Serong 2017-05-19 12:14 ` Sage Weil 2017-05-19 18:25 ` Ken Dreyer 2017-05-20 0:24 ` kefu chai 2017-05-21 22:31 ` John Spray 2017-05-22 8:22 ` Tim Serong 2017-05-22 8:51 ` Willem Jan Withagen 2017-05-22 15:45 ` Ken Dreyer 2017-05-24 20:15 ` Alfredo Deza 2017-05-24 20:53 ` John Spray
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.