librados3

librados3

[kefu chai]

in luminous, we introduced a change not backward compatible with
pre-luminous librados. see see http://tracker.ceph.com/issues/21573.
and i cannot see an obvious solution without bumping up the so version
of librados to 3.

if we have the luxury to do it,  we will be able to give librados an
overhaul. for a todo list, see http://pad.ceph.com/p/librados3.

what do you think?

-- 
Regards
Kefu Chai

Re: librados3

[John Spray]

On Thu, Oct 19, 2017 at 10:25 AM, kefu chai <tchaikov@gmail.com> wrote:
> in luminous, we introduced a change not backward compatible with
> pre-luminous librados. see see http://tracker.ceph.com/issues/21573.
> and i cannot see an obvious solution without bumping up the so version
> of librados to 3.
>
> if we have the luxury to do it,  we will be able to give librados an
> overhaul. for a todo list, see http://pad.ceph.com/p/librados3.
>
> what do you think?

I've added a couple of items to the pad for deprecated bits to remove
when we go up a version.

I guess we would do the version bump for Mimic and for Luminous we
will just have to apologetically tell people they need to recompile?

Or we could have a so.3 version (ABI change only) in Luminous that
just bumps the version to force a recompile, and a so.4 version in
Mimic that actually makes the API changes?

John

>
> --
> Regards
> Kefu Chai
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: librados3

[Jason Dillaman]

As long as we plan to continue to ship librados2 side-by-side with
librados3 for a while, I'm fine with starting down that path for
mimic. There are a lot of applications linked against librados2
floating in the wild.

For luminous, we could perhaps just drop the "_mempool" member
variable from bufferlist and chance the semantics where the caller
needs to add at least a single buffer::raw before the mempool can be
assigned. That preserves the ABI since we aren't changing the size of
bufferlist and preserves the mempool functionality (with a little
extra work).  That tweak would only be needed for the first
buffer::ptr appended to the bufferlist since subsequent appends can
just copy the mempool value from the first buffer::ptr.

On Thu, Oct 19, 2017 at 6:43 AM, John Spray <jspray@redhat.com> wrote:
> On Thu, Oct 19, 2017 at 10:25 AM, kefu chai <tchaikov@gmail.com> wrote:
>> in luminous, we introduced a change not backward compatible with
>> pre-luminous librados. see see http://tracker.ceph.com/issues/21573.
>> and i cannot see an obvious solution without bumping up the so version
>> of librados to 3.
>>
>> if we have the luxury to do it,  we will be able to give librados an
>> overhaul. for a todo list, see http://pad.ceph.com/p/librados3.
>>
>> what do you think?
>
> I've added a couple of items to the pad for deprecated bits to remove
> when we go up a version.
>
> I guess we would do the version bump for Mimic and for Luminous we
> will just have to apologetically tell people they need to recompile?
>
> Or we could have a so.3 version (ABI change only) in Luminous that
> just bumps the version to force a recompile, and a so.4 version in
> Mimic that actually makes the API changes?
>
> John
>
>>
>> --
>> Regards
>> Kefu Chai
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Jason

Re: librados3

[Sage Weil]

On Thu, 19 Oct 2017, John Spray wrote:
> On Thu, Oct 19, 2017 at 10:25 AM, kefu chai <tchaikov@gmail.com> wrote:
> > in luminous, we introduced a change not backward compatible with
> > pre-luminous librados. see see http://tracker.ceph.com/issues/21573.
> > and i cannot see an obvious solution without bumping up the so version
> > of librados to 3.
> >
> > if we have the luxury to do it,  we will be able to give librados an
> > overhaul. for a todo list, see http://pad.ceph.com/p/librados3.
> >
> > what do you think?
> 
> I've added a couple of items to the pad for deprecated bits to remove
> when we go up a version.
> 
> I guess we would do the version bump for Mimic and for Luminous we
> will just have to apologetically tell people they need to recompile?
> 
> Or we could have a so.3 version (ABI change only) in Luminous that
> just bumps the version to force a recompile, and a so.4 version in
> Mimic that actually makes the API changes?

Yeah, it sounds like this is what we actually want in this case?

sage

Re: librados3

[Jason Dillaman]

I vote against bumping to so.3 for Luminous since it would prevent
distros from upgrading to Luminous w/o re-linking apps like QEMU (e.g.
qemu-kvm rpm in RHEL has a requirement on "
librados.so.2()(64bit)". Therefore, we would have to still ship
librados2 (with the broken ABI) so it doesn't seem like we gain
anything.

On Thu, Oct 19, 2017 at 8:00 AM, Sage Weil <sage@newdream.net> wrote:
> On Thu, 19 Oct 2017, John Spray wrote:
>> On Thu, Oct 19, 2017 at 10:25 AM, kefu chai <tchaikov@gmail.com> wrote:
>> > in luminous, we introduced a change not backward compatible with
>> > pre-luminous librados. see see http://tracker.ceph.com/issues/21573.
>> > and i cannot see an obvious solution without bumping up the so version
>> > of librados to 3.
>> >
>> > if we have the luxury to do it,  we will be able to give librados an
>> > overhaul. for a todo list, see http://pad.ceph.com/p/librados3.
>> >
>> > what do you think?
>>
>> I've added a couple of items to the pad for deprecated bits to remove
>> when we go up a version.
>>
>> I guess we would do the version bump for Mimic and for Luminous we
>> will just have to apologetically tell people they need to recompile?
>>
>> Or we could have a so.3 version (ABI change only) in Luminous that
>> just bumps the version to force a recompile, and a so.4 version in
>> Mimic that actually makes the API changes?
>
> Yeah, it sounds like this is what we actually want in this case?
>
> sage
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Jason

Re: librados3

[Sage Weil]

On Thu, 19 Oct 2017, Jason Dillaman wrote:
> As long as we plan to continue to ship librados2 side-by-side with
> librados3 for a while, I'm fine with starting down that path for
> mimic. There are a lot of applications linked against librados2
> floating in the wild.
> 
> For luminous, we could perhaps just drop the "_mempool" member
> variable from bufferlist and chance the semantics where the caller
> needs to add at least a single buffer::raw before the mempool can be
> assigned. That preserves the ABI since we aren't changing the size of
> bufferlist and preserves the mempool functionality (with a little
> extra work).  That tweak would only be needed for the first
> buffer::ptr appended to the bufferlist since subsequent appends can
> just copy the mempool value from the first buffer::ptr.

I think that will still be sufficient for bluestore (which relies on 
mempool accounting to make its cache trimming work).  I can give it a try 
today!

sage

> On Thu, Oct 19, 2017 at 6:43 AM, John Spray <jspray@redhat.com> wrote:
> > On Thu, Oct 19, 2017 at 10:25 AM, kefu chai <tchaikov@gmail.com> wrote:
> >> in luminous, we introduced a change not backward compatible with
> >> pre-luminous librados. see see http://tracker.ceph.com/issues/21573.
> >> and i cannot see an obvious solution without bumping up the so version
> >> of librados to 3.
> >>
> >> if we have the luxury to do it,  we will be able to give librados an
> >> overhaul. for a todo list, see http://pad.ceph.com/p/librados3.
> >>
> >> what do you think?
> >
> > I've added a couple of items to the pad for deprecated bits to remove
> > when we go up a version.
> >
> > I guess we would do the version bump for Mimic and for Luminous we
> > will just have to apologetically tell people they need to recompile?
> >
> > Or we could have a so.3 version (ABI change only) in Luminous that
> > just bumps the version to force a recompile, and a so.4 version in
> > Mimic that actually makes the API changes?
> >
> > John
> >
> >>
> >> --
> >> Regards
> >> Kefu Chai
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> 
> -- 
> Jason
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: librados3

[Alan Somers]

Would it be possible to use ELF symbol versioning to bump the versions
of all affected functions without bumping the .so number?  That would
provide an upgrade path for distros.  The downside is that the new
librados would need to keep compat shims for old clients.

On Thu, Oct 19, 2017 at 6:13 AM, Sage Weil <sweil@redhat.com> wrote:
> On Thu, 19 Oct 2017, Jason Dillaman wrote:
>> As long as we plan to continue to ship librados2 side-by-side with
>> librados3 for a while, I'm fine with starting down that path for
>> mimic. There are a lot of applications linked against librados2
>> floating in the wild.
>>
>> For luminous, we could perhaps just drop the "_mempool" member
>> variable from bufferlist and chance the semantics where the caller
>> needs to add at least a single buffer::raw before the mempool can be
>> assigned. That preserves the ABI since we aren't changing the size of
>> bufferlist and preserves the mempool functionality (with a little
>> extra work).  That tweak would only be needed for the first
>> buffer::ptr appended to the bufferlist since subsequent appends can
>> just copy the mempool value from the first buffer::ptr.
>
> I think that will still be sufficient for bluestore (which relies on
> mempool accounting to make its cache trimming work).  I can give it a try
> today!
>
> sage
>
>> On Thu, Oct 19, 2017 at 6:43 AM, John Spray <jspray@redhat.com> wrote:
>> > On Thu, Oct 19, 2017 at 10:25 AM, kefu chai <tchaikov@gmail.com> wrote:
>> >> in luminous, we introduced a change not backward compatible with
>> >> pre-luminous librados. see see http://tracker.ceph.com/issues/21573.
>> >> and i cannot see an obvious solution without bumping up the so version
>> >> of librados to 3.
>> >>
>> >> if we have the luxury to do it,  we will be able to give librados an
>> >> overhaul. for a todo list, see http://pad.ceph.com/p/librados3.
>> >>
>> >> what do you think?
>> >
>> > I've added a couple of items to the pad for deprecated bits to remove
>> > when we go up a version.
>> >
>> > I guess we would do the version bump for Mimic and for Luminous we
>> > will just have to apologetically tell people they need to recompile?
>> >
>> > Or we could have a so.3 version (ABI change only) in Luminous that
>> > just bumps the version to force a recompile, and a so.4 version in
>> > Mimic that actually makes the API changes?
>> >
>> > John
>> >
>> >>
>> >> --
>> >> Regards
>> >> Kefu Chai
>> >> --
>> >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> >> the body of a message to majordomo@vger.kernel.org
>> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>
>>
>> --
>> Jason
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: librados3

[Brad Hubbard]

On Fri, Oct 20, 2017 at 1:26 AM, Alan Somers <asomers@freebsd.org> wrote:
> Would it be possible to use ELF symbol versioning to bump the versions
> of all affected functions without bumping the .so number?  That would
> provide an upgrade path for distros.  The downside is that the new
> librados would need to keep compat shims for old clients.

I agree. The change in name has caused problems in the past with
upgrades and I'm not sure it's necessary if we implemented versioning
akin to https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html ?

The problem, as Alan has mentioned, is maintenance. Kefu, how much of
a headache would this sort of version maintenance actually be?

>
> On Thu, Oct 19, 2017 at 6:13 AM, Sage Weil <sweil@redhat.com> wrote:
>> On Thu, 19 Oct 2017, Jason Dillaman wrote:
>>> As long as we plan to continue to ship librados2 side-by-side with
>>> librados3 for a while, I'm fine with starting down that path for
>>> mimic. There are a lot of applications linked against librados2
>>> floating in the wild.
>>>
>>> For luminous, we could perhaps just drop the "_mempool" member
>>> variable from bufferlist and chance the semantics where the caller
>>> needs to add at least a single buffer::raw before the mempool can be
>>> assigned. That preserves the ABI since we aren't changing the size of
>>> bufferlist and preserves the mempool functionality (with a little
>>> extra work).  That tweak would only be needed for the first
>>> buffer::ptr appended to the bufferlist since subsequent appends can
>>> just copy the mempool value from the first buffer::ptr.
>>
>> I think that will still be sufficient for bluestore (which relies on
>> mempool accounting to make its cache trimming work).  I can give it a try
>> today!
>>
>> sage
>>
>>> On Thu, Oct 19, 2017 at 6:43 AM, John Spray <jspray@redhat.com> wrote:
>>> > On Thu, Oct 19, 2017 at 10:25 AM, kefu chai <tchaikov@gmail.com> wrote:
>>> >> in luminous, we introduced a change not backward compatible with
>>> >> pre-luminous librados. see see http://tracker.ceph.com/issues/21573.
>>> >> and i cannot see an obvious solution without bumping up the so version
>>> >> of librados to 3.
>>> >>
>>> >> if we have the luxury to do it,  we will be able to give librados an
>>> >> overhaul. for a todo list, see http://pad.ceph.com/p/librados3.
>>> >>
>>> >> what do you think?
>>> >
>>> > I've added a couple of items to the pad for deprecated bits to remove
>>> > when we go up a version.
>>> >
>>> > I guess we would do the version bump for Mimic and for Luminous we
>>> > will just have to apologetically tell people they need to recompile?
>>> >
>>> > Or we could have a so.3 version (ABI change only) in Luminous that
>>> > just bumps the version to force a recompile, and a so.4 version in
>>> > Mimic that actually makes the API changes?
>>> >
>>> > John
>>> >
>>> >>
>>> >> --
>>> >> Regards
>>> >> Kefu Chai
>>> >> --
>>> >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>> >> the body of a message to majordomo@vger.kernel.org
>>> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>>
>>>
>>> --
>>> Jason
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Cheers,
Brad

Re: librados3

[kefu chai]

thanks for your inputs, guys!

On Fri, Oct 20, 2017 at 8:15 AM, Brad Hubbard <bhubbard@redhat.com> wrote:
> On Fri, Oct 20, 2017 at 1:26 AM, Alan Somers <asomers@freebsd.org> wrote:
>> Would it be possible to use ELF symbol versioning to bump the versions
>> of all affected functions without bumping the .so number?  That would
>> provide an upgrade path for distros.  The downside is that the new
>> librados would need to keep compat shims for old clients.

we don't release header files and library with versioned symbols at
least in pre-mimic, so i don't think this approach would help the
librados client to link against the expected old functions. see the
output of

$ objdump -x bin/rados | grep -A42 References


anyway, thanks to Jason and Sage, we now have a fix for the
ceph::buffer ABI incompatibility and it's backported to luminous. see
https://github.com/ceph/ceph/pull/18408. probably we won't have
librados3 in this release.

>
> I agree. The change in name has caused problems in the past with
> upgrades and I'm not sure it's necessary if we implemented versioning
> akin to https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html ?
>
> The problem, as Alan has mentioned, is maintenance. Kefu, how much of
> a headache would this sort of version maintenance actually be?
>
>>
>> On Thu, Oct 19, 2017 at 6:13 AM, Sage Weil <sweil@redhat.com> wrote:
>>> On Thu, 19 Oct 2017, Jason Dillaman wrote:
>>>> As long as we plan to continue to ship librados2 side-by-side with
>>>> librados3 for a while, I'm fine with starting down that path for
>>>> mimic. There are a lot of applications linked against librados2
>>>> floating in the wild.
>>>>
>>>> For luminous, we could perhaps just drop the "_mempool" member
>>>> variable from bufferlist and chance the semantics where the caller
>>>> needs to add at least a single buffer::raw before the mempool can be
>>>> assigned. That preserves the ABI since we aren't changing the size of
>>>> bufferlist and preserves the mempool functionality (with a little
>>>> extra work).  That tweak would only be needed for the first
>>>> buffer::ptr appended to the bufferlist since subsequent appends can
>>>> just copy the mempool value from the first buffer::ptr.
>>>
>>> I think that will still be sufficient for bluestore (which relies on
>>> mempool accounting to make its cache trimming work).  I can give it a try
>>> today!
>>>
>>> sage
>>>
>>>> On Thu, Oct 19, 2017 at 6:43 AM, John Spray <jspray@redhat.com> wrote:
>>>> > On Thu, Oct 19, 2017 at 10:25 AM, kefu chai <tchaikov@gmail.com> wrote:
>>>> >> in luminous, we introduced a change not backward compatible with
>>>> >> pre-luminous librados. see see http://tracker.ceph.com/issues/21573.
>>>> >> and i cannot see an obvious solution without bumping up the so version
>>>> >> of librados to 3.
>>>> >>
>>>> >> if we have the luxury to do it,  we will be able to give librados an
>>>> >> overhaul. for a todo list, see http://pad.ceph.com/p/librados3.
>>>> >>
>>>> >> what do you think?
>>>> >
>>>> > I've added a couple of items to the pad for deprecated bits to remove
>>>> > when we go up a version.
>>>> >
>>>> > I guess we would do the version bump for Mimic and for Luminous we
>>>> > will just have to apologetically tell people they need to recompile?
>>>> >
>>>> > Or we could have a so.3 version (ABI change only) in Luminous that
>>>> > just bumps the version to force a recompile, and a so.4 version in
>>>> > Mimic that actually makes the API changes?
>>>> >
>>>> > John
>>>> >
>>>> >>
>>>> >> --
-- 
Regards
Kefu Chai

Re: librados3

[Josh Durgin]

On 10/25/2017 01:35 AM, kefu chai wrote:
> thanks for your inputs, guys!
> 
> On Fri, Oct 20, 2017 at 8:15 AM, Brad Hubbard <bhubbard@redhat.com> wrote:
>> On Fri, Oct 20, 2017 at 1:26 AM, Alan Somers <asomers@freebsd.org> wrote:
>>> Would it be possible to use ELF symbol versioning to bump the versions
>>> of all affected functions without bumping the .so number?  That would
>>> provide an upgrade path for distros.  The downside is that the new
>>> librados would need to keep compat shims for old clients.
> 
> we don't release header files and library with versioned symbols at
> least in pre-mimic, so i don't think this approach would help the
> librados client to link against the expected old functions. see the
> output of
> 
> $ objdump -x bin/rados | grep -A42 References
> 
> 
> anyway, thanks to Jason and Sage, we now have a fix for the
> ceph::buffer ABI incompatibility and it's backported to luminous. see
> https://github.com/ceph/ceph/pull/18408. probably we won't have
> librados3 in this release.

The most common culprit for ABI issues is bufferlist since we expose the
internal ceph type directly. This has caused problems with almost every
release lately.

At this point it it seems worth finally separating the internal and
external types there, without doing a full new version of librados.

It may be possible to do this without introducing a new type, by 
freezing the existing bufferlist as the external type for librados and 
librbd, and making a conversion if necessary. We could add a true pimpl 
wrapper as a new type to avoid the overhead of conversion if it is a 
problem.

Josh

Re: librados3

[Jesse Williamson]

On Wed, 25 Oct 2017, Josh Durgin wrote:

> At this point it it seems worth finally separating the internal and
> external types there, without doing a full new version of librados.

+1

Even though it looks like we won't move forward on too many extravagant 
things (as per the etherpad, "http://pad.ceph.com/p/librados3"), I do 
still have one wishlist item I should throw in here:  If we could enable a 
CephContext can live on the stack while we're in here, that would be 
awesome. Does anyone know why it's dtor has been made private?

> It may be possible to do this without introducing a new type, by freezing the 
> existing bufferlist as the external type for librados and librbd, and making 
> a conversion if necessary. We could add a true pimpl wrapper as a new type to 
> avoid the overhead of conversion if it is a problem.

Assuming the doesn't preclude adding other future types, seems like a 
reasonable way to address the situation. The PIMPL will still have some 
overhead from the indirection, but certainly less than forcing 
conversion/copy.

-Jesse

Re: librados3

[Josh Durgin]

On 10/25/2017 02:51 PM, Jesse Williamson wrote:
> On Wed, 25 Oct 2017, Josh Durgin wrote:
> 
>> At this point it it seems worth finally separating the internal and
>> external types there, without doing a full new version of librados.
> 
> +1
> 
> Even though it looks like we won't move forward on too many extravagant 
> things (as per the etherpad, "http://pad.ceph.com/p/librados3"), I do 
> still have one wishlist item I should throw in here:  If we could enable 
> a CephContext can live on the stack while we're in here, that would be 
> awesome. Does anyone know why it's dtor has been made private?

It's not meant to be a publicly exposed type, it's just there as a
convenience for internal ceph stuff using librados that needs to pass
configuration that's already in a CephContext.

Re: librados3

[Jesse Williamson]

On Wed, 25 Oct 2017, Josh Durgin wrote:

> It's not meant to be a publicly exposed type, it's just there as a
> convenience for internal ceph stuff using librados that needs to pass
> configuration that's already in a CephContext.

Ok, so no particular objections to making it something that can be put on 
the stack? It's been making my life difficult in trying to decouple some 
parts of librados (in particular, exactly configuration, etc.). :-)

Thanks,

-Jesse

Re: librados3

[kefu chai]

On Thu, Oct 26, 2017 at 3:06 AM, Josh Durgin <jdurgin@redhat.com> wrote:
>
>
> The most common culprit for ABI issues is bufferlist since we expose the
> internal ceph type directly. This has caused problems with almost every
> release lately.
>
> At this point it it seems worth finally separating the internal and
> external types there, without doing a full new version of librados.

i don't think we can do this without doing a new version of librados.
the legacy librados clients are still expecting the internal types at
some degree, that's how we noticed the problem
(http://tracker.ceph.com/issues/21573), where the librados client was
compiled with an old librados header, and inlined the old constructor
in its binary. and some APIs force us to do so, for example,
"ObjectReadOperation::getxattrs(std::map<std::string, bufferlist>
*pattrs, int *prval)" requires a complete type definition of
ceph::bufferlist. in other words, the buffer.h exposing the innards of
ceph::buffer types was shipped with librados2, and the librados client
is using them, we can hardly fix this w/o bumping the so version.

>
> It may be possible to do this without introducing a new type, by freezing
> the existing bufferlist as the external type for librados and librbd, and
> making a conversion if necessary. We could add a true pimpl wrapper as a new
> type to avoid the overhead of conversion if it is a problem.

yeah, probably by introducing a bufferlist2, which is a pimpl wrapper,
and start to deprecate APIs like getxattrs() with the ones using
bufferlist2. and convert bufferlist to the actual bufferlist in the
legacy APIs. feasible, but a pains-taking plan..

i still like librados3. =)


-- 
Regards
Kefu Chai

Re: librados3

[Jason Dillaman]

On Mon, Oct 29, 2018 at 7:48 AM Wido den Hollander <wido-fspyXLx8qC4@public.gmane.org> wrote:
> On 10/29/18 12:42 PM, kefu chai wrote:
> > + ceph-user for more inputs in hope to get more inputs from librados
> > and librbd 's C++ interfaces.
> >
> > On Wed, Oct 24, 2018 at 1:34 AM Jason Dillaman <jdillama-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> >>
> >> On Tue, Oct 23, 2018 at 11:38 AM kefu chai <tchaikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >>>
> >>> we plan to introduce some non-backward-compatible changes[0] in
> >>> librados in the coming nautilus release. to be specific, these changes
> >>> are not API/ABI compatible with existing librados2. so i think it's
> >>> time to bump up the soversion of librados from 2 to 3.  as bumping up
> >>> soversion is a major change in the life cycle of a library, i think we
> >>> will expect following changes:
> >>>
> >>> 0. piggyback more non-backward-compatible changes listed in
> >>> https://pad.ceph.com/p/librados3 in this change, so we can have less
> >>> soversion bump up.
> >>> 1. in nautilus, librados3 and librados3-{dev,devel} will be packaged
> >>> instead, librados2* will be maintained in LTS releases.
> >>
> >> Just note that this is going to have a potential ripple effect on
> >> non-Ceph packages that use librbd1 if they are also (incorrectly)
> >> specifying a dependency on librados2 [1].
> >>
> >>> 2. we will have separated C++ and C API librados after this change. so
> >>> librados3 will only provide the C API of librados, the C++ API will be
> >>> offered by libradospp, (the name may vary if you suggest a better
> >>> one). and they will be versioned and packaged separately, and will not
> >>> depend on each other. a PR is posted to do this, see [1]
> >>
> >> Do we have a good idea and/or list about who uses the librados C++
> >> API? I seem to vaguely recall perhaps one or so external users. If
> >> librgw/RGW and librbd eventually switch over to something similar to
> >> the API being worked on by Adam [2] and if there are no external users
> >> of the C++ API, should we take the time now to kill all the legacy C++
> >> API methods prior to the release of Nautilus?
> >>
>
> I know various users who use librados through phprados (PHP) and the
> rados-java (Java) bindings.
>
> Those bindings will need to be updated as well.
>
> Qemu and libvirt both mainly use librbd, but they use a very small part
> of librados to initiate the connection and set configuration values.

These should all be using the C API which isn't changing. We would
like to take inventory of who uses the librados/librbd C++ APIs
outside of the core Ceph project.

> >>> 3. in order to enable user to install librados2 and librados3 at the
> >>> same time, libceph-common.so will be versioned since nautilus.
> >>> libceph-common.so is an internal shared library used by ceph
> >>> libraries, tools and daemons. librados depends on libceph-common.
> >>> 4. some executables/libraries' dependencies will be updated
> >>> accordingly . for instance, librbd will depend on libradospp.
> >>>
> >>> if this model works fine, i guess we probably could expand it to librbd.
> >>
> >> It would be interesting to get a feel for who (if anyone) actually
> >> uses the librbd C++ interface before heading down that path. AFAIK,
> >> it's probably really only the rbd CLI tool -- and that's basically for
> >> the convenience of C++-style containers. If it's not being used by
> >> anyone else, perhaps our time could be better spent deprecating the
> >> library for external use to focus our development and maintanence
> >> efforts on the C (and wrapped-C) interface.
> >>
>
> Again, don't forget Qemu/KVM and libvirt who both are very heavy users
> of librbd
>
> Wido
>
> >>> any concerns?
> >>>
> >>> BTW, we discussed this topic last year the same time, see
> >>> https://www.spinics.net/lists/ceph-devel/msg38830.html =)
> >>>
> >>> ---
> >>> [0] for instance, https://github.com/ceph/ceph/pull/24498
> >>> [1] https://github.com/ceph/ceph/pull/24616
> >>> [2] a trello card for librados3:
> >>> https://trello.com/c/pmRkawYV/165-librados3-api-update-cleanup
> >>>
> >>> --
> >>> Regards
> >>> Kefu Chai
> >>
> >> [1] https://src.fedoraproject.org/rpms/qemu/blob/f29/f/qemu.spec#_192
> >> [2] https://github.com/ceph/ceph/pull/16715
> >>
> >> --
> >> Jason
> >
> >
> >
> _______________________________________________
> ceph-users mailing list
> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



-- 
Jason

Re: librados3

[Wido den Hollander]

On 10/29/18 12:42 PM, kefu chai wrote:
> + ceph-user for more inputs in hope to get more inputs from librados
> and librbd 's C++ interfaces.
> 
> On Wed, Oct 24, 2018 at 1:34 AM Jason Dillaman <jdillama-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>>
>> On Tue, Oct 23, 2018 at 11:38 AM kefu chai <tchaikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>
>>> we plan to introduce some non-backward-compatible changes[0] in
>>> librados in the coming nautilus release. to be specific, these changes
>>> are not API/ABI compatible with existing librados2. so i think it's
>>> time to bump up the soversion of librados from 2 to 3.  as bumping up
>>> soversion is a major change in the life cycle of a library, i think we
>>> will expect following changes:
>>>
>>> 0. piggyback more non-backward-compatible changes listed in
>>> https://pad.ceph.com/p/librados3 in this change, so we can have less
>>> soversion bump up.
>>> 1. in nautilus, librados3 and librados3-{dev,devel} will be packaged
>>> instead, librados2* will be maintained in LTS releases.
>>
>> Just note that this is going to have a potential ripple effect on
>> non-Ceph packages that use librbd1 if they are also (incorrectly)
>> specifying a dependency on librados2 [1].
>>
>>> 2. we will have separated C++ and C API librados after this change. so
>>> librados3 will only provide the C API of librados, the C++ API will be
>>> offered by libradospp, (the name may vary if you suggest a better
>>> one). and they will be versioned and packaged separately, and will not
>>> depend on each other. a PR is posted to do this, see [1]
>>
>> Do we have a good idea and/or list about who uses the librados C++
>> API? I seem to vaguely recall perhaps one or so external users. If
>> librgw/RGW and librbd eventually switch over to something similar to
>> the API being worked on by Adam [2] and if there are no external users
>> of the C++ API, should we take the time now to kill all the legacy C++
>> API methods prior to the release of Nautilus?
>>

I know various users who use librados through phprados (PHP) and the
rados-java (Java) bindings.

Those bindings will need to be updated as well.

Qemu and libvirt both mainly use librbd, but they use a very small part
of librados to initiate the connection and set configuration values.

>>> 3. in order to enable user to install librados2 and librados3 at the
>>> same time, libceph-common.so will be versioned since nautilus.
>>> libceph-common.so is an internal shared library used by ceph
>>> libraries, tools and daemons. librados depends on libceph-common.
>>> 4. some executables/libraries' dependencies will be updated
>>> accordingly . for instance, librbd will depend on libradospp.
>>>
>>> if this model works fine, i guess we probably could expand it to librbd.
>>
>> It would be interesting to get a feel for who (if anyone) actually
>> uses the librbd C++ interface before heading down that path. AFAIK,
>> it's probably really only the rbd CLI tool -- and that's basically for
>> the convenience of C++-style containers. If it's not being used by
>> anyone else, perhaps our time could be better spent deprecating the
>> library for external use to focus our development and maintanence
>> efforts on the C (and wrapped-C) interface.
>>

Again, don't forget Qemu/KVM and libvirt who both are very heavy users
of librbd

Wido

>>> any concerns?
>>>
>>> BTW, we discussed this topic last year the same time, see
>>> https://www.spinics.net/lists/ceph-devel/msg38830.html =)
>>>
>>> ---
>>> [0] for instance, https://github.com/ceph/ceph/pull/24498
>>> [1] https://github.com/ceph/ceph/pull/24616
>>> [2] a trello card for librados3:
>>> https://trello.com/c/pmRkawYV/165-librados3-api-update-cleanup
>>>
>>> --
>>> Regards
>>> Kefu Chai
>>
>> [1] https://src.fedoraproject.org/rpms/qemu/blob/f29/f/qemu.spec#_192
>> [2] https://github.com/ceph/ceph/pull/16715
>>
>> --
>> Jason
> 
> 
> 

Re: librados3

[kefu chai]

+ ceph-user for more inputs in hope to get more inputs from librados
and librbd 's C++ interfaces.

On Wed, Oct 24, 2018 at 1:34 AM Jason Dillaman <jdillama-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>
> On Tue, Oct 23, 2018 at 11:38 AM kefu chai <tchaikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >
> > we plan to introduce some non-backward-compatible changes[0] in
> > librados in the coming nautilus release. to be specific, these changes
> > are not API/ABI compatible with existing librados2. so i think it's
> > time to bump up the soversion of librados from 2 to 3.  as bumping up
> > soversion is a major change in the life cycle of a library, i think we
> > will expect following changes:
> >
> > 0. piggyback more non-backward-compatible changes listed in
> > https://pad.ceph.com/p/librados3 in this change, so we can have less
> > soversion bump up.
> > 1. in nautilus, librados3 and librados3-{dev,devel} will be packaged
> > instead, librados2* will be maintained in LTS releases.
>
> Just note that this is going to have a potential ripple effect on
> non-Ceph packages that use librbd1 if they are also (incorrectly)
> specifying a dependency on librados2 [1].
>
> > 2. we will have separated C++ and C API librados after this change. so
> > librados3 will only provide the C API of librados, the C++ API will be
> > offered by libradospp, (the name may vary if you suggest a better
> > one). and they will be versioned and packaged separately, and will not
> > depend on each other. a PR is posted to do this, see [1]
>
> Do we have a good idea and/or list about who uses the librados C++
> API? I seem to vaguely recall perhaps one or so external users. If
> librgw/RGW and librbd eventually switch over to something similar to
> the API being worked on by Adam [2] and if there are no external users
> of the C++ API, should we take the time now to kill all the legacy C++
> API methods prior to the release of Nautilus?
>
> > 3. in order to enable user to install librados2 and librados3 at the
> > same time, libceph-common.so will be versioned since nautilus.
> > libceph-common.so is an internal shared library used by ceph
> > libraries, tools and daemons. librados depends on libceph-common.
> > 4. some executables/libraries' dependencies will be updated
> > accordingly . for instance, librbd will depend on libradospp.
> >
> > if this model works fine, i guess we probably could expand it to librbd.
>
> It would be interesting to get a feel for who (if anyone) actually
> uses the librbd C++ interface before heading down that path. AFAIK,
> it's probably really only the rbd CLI tool -- and that's basically for
> the convenience of C++-style containers. If it's not being used by
> anyone else, perhaps our time could be better spent deprecating the
> library for external use to focus our development and maintanence
> efforts on the C (and wrapped-C) interface.
>
> > any concerns?
> >
> > BTW, we discussed this topic last year the same time, see
> > https://www.spinics.net/lists/ceph-devel/msg38830.html =)
> >
> > ---
> > [0] for instance, https://github.com/ceph/ceph/pull/24498
> > [1] https://github.com/ceph/ceph/pull/24616
> > [2] a trello card for librados3:
> > https://trello.com/c/pmRkawYV/165-librados3-api-update-cleanup
> >
> > --
> > Regards
> > Kefu Chai
>
> [1] https://src.fedoraproject.org/rpms/qemu/blob/f29/f/qemu.spec#_192
> [2] https://github.com/ceph/ceph/pull/16715
>
> --
> Jason



-- 
Regards
Kefu Chai