From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sage Weil Subject: Re: [ceph-users] killing ceph-disk [was Re: ceph-volume: migration and disk partition support] Date: Mon, 16 Oct 2017 22:34:55 +0000 (UTC) Message-ID: References: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Return-path: Received: from cobra.newdream.net ([66.33.205.236]:44937 "EHLO tragedy.dreamhost.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753352AbdJPWe6 (ORCPT ); Mon, 16 Oct 2017 18:34:58 -0400 In-Reply-To: Sender: ceph-devel-owner@vger.kernel.org List-ID: To: Anthony Verevkin Cc: ceph-devel , ceph-users@lists.ceph.com, Alfredo Deza On Mon, 16 Oct 2017, Anthony Verevkin wrote: > > > From: "Sage Weil" > > To: "Alfredo Deza" > > Cc: "ceph-devel" , ceph-users@lists.ceph.com > > Sent: Monday, October 9, 2017 11:09:29 AM > > Subject: [ceph-users] killing ceph-disk [was Re: ceph-volume: migration and disk partition support] > > > > To put this in context, the goal here is to kill ceph-disk in mimic. > > > > > > Perhaps the "out" here is to support a "dir" option where the user > > can > > manually provision and mount an OSD on /var/lib/ceph/osd/*, with > > 'journal' > > or 'block' symlinks, and ceph-volume will do the last bits that > > initialize > > the filestore or bluestore OSD from there. Then if someone has a > > scenario > > that isn't captured by LVM (or whatever else we support) they can > > always > > do it manually? > > > > In fact, now that bluestore only requires a few small files and symlinks > to remain in /var/lib/ceph/osd/* without the extra requirements for > xattrs support and xfs, why not simply leave those folders on OS root > filesystem and only point symlinks to bluestore block and db devices? > That would simplify the osd deployment so much - and the symlinks can > then point to /dev/disk/by-uuid or by-path or lvm path or whatever. The > only downside for this approach that I see is that disks themselves > would no longer be transferable between the hosts as those few files > that describe the OSD are no longer on the disk itself. :) this is exactly what we're doing, actually: https://github.com/ceph/ceph/pull/18256 We plan to backport this to luminous, hopefully in time for the next point release. dm-crypt is still slightly annoying to set up, but it will still be much easier. sage