ceph-disk vs keyvaluestore

ceph-disk vs keyvaluestore

[Sage Weil]

Hi Haomai,

Not sure if you saw

	http://tracker.ceph.com/issues/9580

which came from an issue Mark had getting ceph-disk to work with the 
keyvaluestore backend.  I think the answer is to make a command 
like

	ceph-osd -i 123 --check-needs-journal

to query whether the configured backend wants a journal backend.  That 
would let ceph-disk skip the work.  Would need to be careful to 
distinguish between errors and a true/false answer.

Any interest in wiring that up?
sage

Re: ceph-disk vs keyvaluestore

[Haomai Wang]

No problem, I would like do it.

On Tue, Sep 30, 2014 at 7:48 AM, Sage Weil <sweil@redhat.com> wrote:
> Hi Haomai,
>
> Not sure if you saw
>
>         http://tracker.ceph.com/issues/9580
>
> which came from an issue Mark had getting ceph-disk to work with the
> keyvaluestore backend.  I think the answer is to make a command
> like
>
>         ceph-osd -i 123 --check-needs-journal
>
> to query whether the configured backend wants a journal backend.  That
> would let ceph-disk skip the work.  Would need to be careful to
> distinguish between errors and a true/false answer.
>
> Any interest in wiring that up?
> sage
>



-- 
Best Regards,

Wheat

Re: ceph-disk vs keyvaluestore

[Haomai Wang]

Hi sage,

What do you think use existing ObjectStore::peek_journal_fsid
interface to detect whether journal needed.

KeyValueStore and MemStore could set passing argument "fsid" to zero
to indicate no journal.

On Tue, Sep 30, 2014 at 10:23 AM, Haomai Wang <haomaiwang@gmail.com> wrote:
> No problem, I would like do it.
>
> On Tue, Sep 30, 2014 at 7:48 AM, Sage Weil <sweil@redhat.com> wrote:
>> Hi Haomai,
>>
>> Not sure if you saw
>>
>>         http://tracker.ceph.com/issues/9580
>>
>> which came from an issue Mark had getting ceph-disk to work with the
>> keyvaluestore backend.  I think the answer is to make a command
>> like
>>
>>         ceph-osd -i 123 --check-needs-journal
>>
>> to query whether the configured backend wants a journal backend.  That
>> would let ceph-disk skip the work.  Would need to be careful to
>> distinguish between errors and a true/false answer.
>>
>> Any interest in wiring that up?
>> sage
>>
>
>
>
> --
> Best Regards,
>
> Wheat



-- 
Best Regards,

Wheat

Re: ceph-disk vs keyvaluestore

[Sage Weil]

On Tue, 30 Sep 2014, Haomai Wang wrote:
> Hi sage,
> 
> What do you think use existing ObjectStore::peek_journal_fsid
> interface to detect whether journal needed.
> 
> KeyValueStore and MemStore could set passing argument "fsid" to zero
> to indicate no journal.

I'm not sure it's a good fit... it's called on an existing store and tells 
you something about it.  I would probably expect a zeroed fsid to mean 
that the specified journal is empty or does not exist, not that it doesn't 
need to exist.

What we need, on the other hand, is something that just tells us whether 
we should create a journal...

sage

Re: ceph-disk vs keyvaluestore

[Mark Kirkwood]

On 30/09/14 17:05, Sage Weil wrote:
> On Tue, 30 Sep 2014, Haomai Wang wrote:
>> Hi sage,
>>
>> What do you think use existing ObjectStore::peek_journal_fsid
>> interface to detect whether journal needed.
>>
>> KeyValueStore and MemStore could set passing argument "fsid" to zero
>> to indicate no journal.
>
> I'm not sure it's a good fit... it's called on an existing store and tells
> you something about it.  I would probably expect a zeroed fsid to mean
> that the specified journal is empty or does not exist, not that it doesn't
> need to exist.
>
> What we need, on the other hand, is something that just tells us whether
> we should create a journal...
>
>

And perhaps the "type" (probably not the right word) of journal:

- filestore: device or filesystem
- rocksdb: filesystem (wal directory)
- leveldb: none (I think)

FWIW, it looks like the log files for leveldb *could* benefit from being 
placed somewhere else (e.g on faster storage). Is it worthwhile 
suggesting to the leveldb devs?

Cheers

Mark

Re: ceph-disk vs keyvaluestore

[Sage Weil]

On Thu, 2 Oct 2014, Mark Kirkwood wrote:
> On 30/09/14 17:05, Sage Weil wrote:
> > On Tue, 30 Sep 2014, Haomai Wang wrote:
> > > Hi sage,
> > > 
> > > What do you think use existing ObjectStore::peek_journal_fsid
> > > interface to detect whether journal needed.
> > > 
> > > KeyValueStore and MemStore could set passing argument "fsid" to zero
> > > to indicate no journal.
> > 
> > I'm not sure it's a good fit... it's called on an existing store and tells
> > you something about it.  I would probably expect a zeroed fsid to mean
> > that the specified journal is empty or does not exist, not that it doesn't
> > need to exist.
> > 
> > What we need, on the other hand, is something that just tells us whether
> > we should create a journal...
> > 
> > 
> 
> And perhaps the "type" (probably not the right word) of journal:
> 
> - filestore: device or filesystem
> - rocksdb: filesystem (wal directory)
> - leveldb: none (I think)

Hmm, good point.  Haomai added this as yes/no, maybe 'none' and 'device' 
is a bit more flexible in case we add 'dir' in the future?

> FWIW, it looks like the log files for leveldb *could* benefit from being
> placed somewhere else (e.g on faster storage). Is it worthwhile suggesting to
> the leveldb devs?

Worth a shot, but don't hold your breath :)

sage

Re: ceph-disk vs keyvaluestore

[Haomai Wang]

On Thu, Oct 2, 2014 at 7:53 AM, Sage Weil <sweil@redhat.com> wrote:
> On Thu, 2 Oct 2014, Mark Kirkwood wrote:
>> On 30/09/14 17:05, Sage Weil wrote:
>> > On Tue, 30 Sep 2014, Haomai Wang wrote:
>> > > Hi sage,
>> > >
>> > > What do you think use existing ObjectStore::peek_journal_fsid
>> > > interface to detect whether journal needed.
>> > >
>> > > KeyValueStore and MemStore could set passing argument "fsid" to zero
>> > > to indicate no journal.
>> >
>> > I'm not sure it's a good fit... it's called on an existing store and tells
>> > you something about it.  I would probably expect a zeroed fsid to mean
>> > that the specified journal is empty or does not exist, not that it doesn't
>> > need to exist.
>> >
>> > What we need, on the other hand, is something that just tells us whether
>> > we should create a journal...
>> >
>> >
>>
>> And perhaps the "type" (probably not the right word) of journal:
>>
>> - filestore: device or filesystem
>> - rocksdb: filesystem (wal directory)
>> - leveldb: none (I think)
>
> Hmm, good point.  Haomai added this as yes/no, maybe 'none' and 'device'
> is a bit more flexible in case we add 'dir' in the future?

Looks good to me. Any other points?

>
>> FWIW, it looks like the log files for leveldb *could* benefit from being
>> placed somewhere else (e.g on faster storage). Is it worthwhile suggesting to
>> the leveldb devs?
>
> Worth a shot, but don't hold your breath :)
>
> sage



-- 
Best Regards,

Wheat