From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dan van der Ster Subject: Re: removing cluster name support Date: Thu, 8 Jun 2017 21:55:15 +0200 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0185772262==" Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ceph-users-bounces-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org Sender: "ceph-users" To: Sage Weil Cc: ceph-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, ceph-users List-Id: ceph-devel.vger.kernel.org --===============0185772262== Content-Type: multipart/alternative; boundary="94eb2c18d98c4b975b0551783bd5" --94eb2c18d98c4b975b0551783bd5 Content-Type: text/plain; charset="UTF-8" Hi Sage, We need named clusters on the client side. RBD or CephFS clients, or monitoring/admin machines all need to be able to access several clusters. Internally, each cluster is indeed called "ceph", but the clients use distinct names to differentiate their configs/keyrings. Cheers, Dan On Jun 8, 2017 9:37 PM, "Sage Weil" wrote: At CDM yesterday we talked about removing the ability to name your ceph clusters. There are a number of hurtles that make it difficult to fully get rid of this functionality, not the least of which is that some (many?) deployed clusters make use of it. We decided that the most we can do at this point is remove support for it in ceph-deploy and ceph-ansible so that no new clusters or deployed nodes use it. The first PR in this effort: https://github.com/ceph/ceph-deploy/pull/441 Background: The cluster name concept was added to allow multiple clusters to have daemons coexist on the same host. At the type it was a hypothetical requirement for a user that never actually made use of it, and the support is kludgey: - default cluster name is 'ceph' - default config is /etc/ceph/$cluster.conf, so that the normal 'ceph.conf' still works - daemon data paths include the cluster name, /var/lib/ceph/osd/$cluster-$id which is weird (but mostly people are used to it?) - any cli command you want to touch a non-ceph cluster name needs -C $name or --cluster $name passed to it. Also, as of jewel, - systemd only supports a single cluster per host, as defined by $CLUSTER in /etc/{sysconfig,default}/ceph which you'll notice removes support for the original "requirement". Also note that you can get the same effect by specifying the config path explicitly (-c /etc/ceph/foo.conf) along with the various options that substitute $cluster in (e.g., osd_data=/var/lib/ceph/osd/$cluster-$id). Crap preventing us from removing this entirely: - existing daemon directories for existing clusters - various scripts parse the cluster name out of paths Converting an existing cluster "foo" back to "ceph": - rename /etc/ceph/foo.conf -> ceph.conf - rename /var/lib/ceph/*/foo-* -> /var/lib/ceph/*/ceph-* - remove the CLUSTER=foo line in /etc/{default,sysconfig}/ceph - reboot Questions: - Does anybody on the list use a non-default cluster name? - If so, do you have a reason not to switch back to 'ceph'? Thanks! sage _______________________________________________ ceph-users mailing list ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com --94eb2c18d98c4b975b0551783bd5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Sage,

We need named clusters on the client side. RBD or CephFS clients, or moni= toring/admin machines all need to be able to access several clusters.
=

Internally, each cluster is i= ndeed called "ceph", but the clients use distinct names to differ= entiate their configs/keyrings.

Cheers, Dan


On Jun 8, 2017 9:37 PM, "Sage We= il" <sweil@re= dhat.com> wrote:
At CDM yesterday we talked about removing the ability to= name your ceph
clusters.=C2=A0 There are a number of hurtles that make it difficult to ful= ly
get rid of this functionality, not the least of which is that some
(many?) deployed clusters make use of it.=C2=A0 We decided that the most we= can
do at this point is remove support for it in ceph-deploy and ceph-ansible so that no new clusters or deployed nodes use it.

The first PR in this effort:

=C2=A0 =C2=A0 =C2=A0 =C2=A0 https://github.com/ceph/ceph= -deploy/pull/441

Background:

The cluster name concept was added to allow multiple clusters to have
daemons coexist on the same host.=C2=A0 At the type it was a hypothetical requirement for a user that never actually made use of it, and the
support is kludgey:

=C2=A0- default cluster name is 'ceph'
=C2=A0- default config is /etc/ceph/$cluster.conf, so that the normal
'ceph.conf' still works
=C2=A0- daemon data paths include the cluster name,
=C2=A0 =C2=A0 =C2=A0/var/lib/ceph/osd/$cluster-$id
=C2=A0 =C2=A0which is weird (but mostly people are used to it?)
=C2=A0- any cli command you want to touch a non-ceph cluster name
needs -C $name or --cluster $name passed to it.

Also, as of jewel,

=C2=A0- systemd only supports a single cluster per host, as defined by $CLU= STER
in /etc/{sysconfig,default}/ceph

which you'll notice removes support for the original "requirement&= quot;.

Also note that you can get the same effect by specifying the config path explicitly (-c /etc/ceph/foo.conf) along with the various options that
substitute $cluster in (e.g., osd_data=3D/var/lib/ceph/osd/$cluster-$i= d).


Crap preventing us from removing this entirely:

=C2=A0- existing daemon directories for existing clusters
=C2=A0- various scripts parse the cluster name out of paths


Converting an existing cluster "foo" back to "ceph":
=C2=A0- rename /etc/ceph/foo.conf -> ceph.conf
=C2=A0- rename /var/lib/ceph/*/foo-* -> /var/lib/ceph/*/ceph-*
=C2=A0- remove the CLUSTER=3Dfoo line in /etc/{default,sysconfig}/ceph
=C2=A0- reboot


Questions:

=C2=A0- Does anybody on the list use a non-default cluster name?
=C2=A0- If so, do you have a reason not to switch back to 'ceph'?
Thanks!
sage
_______________________________________________
ceph-users mailing list
ceph-users@l= ists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-u= sers-ceph.com

--94eb2c18d98c4b975b0551783bd5-- --===============0185772262== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ ceph-users mailing list ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com --===============0185772262==--