client-facing feature bits and pending kernel changes

client-facing feature bits and pending kernel changes

[Sage Weil]

Hi Zheng,

We have one feature already merged to ceph.git, CRUSH_TUNABLES5, that's a 
pretty trivial change to crush/mapper.c that we can add to the kernel.

But there are two more coming:

OSDOPREPLY_ENCODING makes a small change to the OSDOpReply encoding for 
performance reasons.  Here is the PR

	https://github.com/ceph/ceph/pull/7386

The other is the file layout changes, which are much more invovled:

	https://github.com/ceph/ceph/pull/7098

Ideally we'd use the same feature bit for all of these, but it means they 
all need to upstream into Linux together.  Do you see a problem with that?
Assuming we get it all ready to go before the next merge window..

The tricky bit is we really need to know that we'll be able to do it 
together before we cut jewel too and commit to the overlapping feature 
bits.

sage

Re: client-facing feature bits and pending kernel changes

[Ilya Dryomov]

On Fri, Jan 29, 2016 at 5:09 PM, Sage Weil <sweil@redhat.com> wrote:
> Hi Zheng,
>
> We have one feature already merged to ceph.git, CRUSH_TUNABLES5, that's a
> pretty trivial change to crush/mapper.c that we can add to the kernel.
>
> But there are two more coming:
>
> OSDOPREPLY_ENCODING makes a small change to the OSDOpReply encoding for
> performance reasons.  Here is the PR
>
>         https://github.com/ceph/ceph/pull/7386
>
> The other is the file layout changes, which are much more invovled:
>
>         https://github.com/ceph/ceph/pull/7098
>
> Ideally we'd use the same feature bit for all of these, but it means they
> all need to upstream into Linux together.  Do you see a problem with that?
> Assuming we get it all ready to go before the next merge window..
>
> The tricky bit is we really need to know that we'll be able to do it
> together before we cut jewel too and commit to the overlapping feature
> bits.

In the worst case, we could advertise support for the new bit for rbd
and not for cephfs - I don't think we ever get/put ceph file layout
from/to the wire in the rbd case.

As for the two easy ones (TUNABLES5 in particular), I think we should
try and get them in ASAP into 4.5 and then finalize the feature bit
arrangement closer to the end of the cycle, if necessary.  Otherwise
we'll have jewel and no kernel supporting its default tunables.

Thanks,

                Ilya

Re: client-facing feature bits and pending kernel changes

[Ilya Dryomov]

On Fri, Jan 29, 2016 at 6:07 PM, Ilya Dryomov <idryomov@gmail.com> wrote:
> On Fri, Jan 29, 2016 at 5:09 PM, Sage Weil <sweil@redhat.com> wrote:
>> Hi Zheng,
>>
>> We have one feature already merged to ceph.git, CRUSH_TUNABLES5, that's a
>> pretty trivial change to crush/mapper.c that we can add to the kernel.
>>
>> But there are two more coming:
>>
>> OSDOPREPLY_ENCODING makes a small change to the OSDOpReply encoding for
>> performance reasons.  Here is the PR
>>
>>         https://github.com/ceph/ceph/pull/7386
>>
>> The other is the file layout changes, which are much more invovled:
>>
>>         https://github.com/ceph/ceph/pull/7098
>>
>> Ideally we'd use the same feature bit for all of these, but it means they
>> all need to upstream into Linux together.  Do you see a problem with that?
>> Assuming we get it all ready to go before the next merge window..
>>
>> The tricky bit is we really need to know that we'll be able to do it
>> together before we cut jewel too and commit to the overlapping feature
>> bits.
>
> In the worst case, we could advertise support for the new bit for rbd
> and not for cephfs - I don't think we ever get/put ceph file layout
> from/to the wire in the rbd case.
>
> As for the two easy ones (TUNABLES5 in particular), I think we should
> try and get them in ASAP into 4.5 and then finalize the feature bit
> arrangement closer to the end of the cycle, if necessary.  Otherwise
> we'll have jewel and no kernel supporting its default tunables.

To that end, I've pulled crush changes into the kernel client, aiming
for -rc3.

Thanks,

                Ilya

Re: client-facing feature bits and pending kernel changes

[Yan, Zheng]

On Sat, Jan 30, 2016 at 12:09 AM, Sage Weil <sweil@redhat.com> wrote:
> Hi Zheng,
>
> We have one feature already merged to ceph.git, CRUSH_TUNABLES5, that's a
> pretty trivial change to crush/mapper.c that we can add to the kernel.
>
> But there are two more coming:
>
> OSDOPREPLY_ENCODING makes a small change to the OSDOpReply encoding for
> performance reasons.  Here is the PR
>
>         https://github.com/ceph/ceph/pull/7386
>
> The other is the file layout changes, which are much more invovled:
>
>         https://github.com/ceph/ceph/pull/7098

Can we avoid changing how file layout is encoded in MClientCaps and
MClientReply? Instead, we encode the namespace information at the end
of MClientCaps/MClientReply message. (like how we added inline data
related fields to these message).

Regards
Yan, Zheng

>
> Ideally we'd use the same feature bit for all of these, but it means they
> all need to upstream into Linux together.  Do you see a problem with that?
> Assuming we get it all ready to go before the next merge window..
>
> The tricky bit is we really need to know that we'll be able to do it
> together before we cut jewel too and commit to the overlapping feature
> bits.
>
> sage
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: client-facing feature bits and pending kernel changes

[Sage Weil]

On Mon, 1 Feb 2016, Yan, Zheng wrote:
> On Sat, Jan 30, 2016 at 12:09 AM, Sage Weil <sweil@redhat.com> wrote:
> > Hi Zheng,
> >
> > We have one feature already merged to ceph.git, CRUSH_TUNABLES5, that's a
> > pretty trivial change to crush/mapper.c that we can add to the kernel.
> >
> > But there are two more coming:
> >
> > OSDOPREPLY_ENCODING makes a small change to the OSDOpReply encoding for
> > performance reasons.  Here is the PR
> >
> >         https://github.com/ceph/ceph/pull/7386
> >
> > The other is the file layout changes, which are much more invovled:
> >
> >         https://github.com/ceph/ceph/pull/7098
> 
> Can we avoid changing how file layout is encoded in MClientCaps and
> MClientReply? Instead, we encode the namespace information at the end
> of MClientCaps/MClientReply message. (like how we added inline data
> related fields to these message).

Hmm.. yeah, we do that, at least for MClientCaps.  It just means that if 
we ever add more fields to that struct we need to maintain the message 
encoding in parallel by hand, but that shouldn't be too bad.

The MClientReply one is a bit trickier because there can be N instances of 
InodeStat (or whatever) in there... on that one I'd lean toward switching 
the encoding.  What do you think?

sage

Re: client-facing feature bits and pending kernel changes

[Yan, Zheng]

On Mon, Feb 1, 2016 at 5:30 PM, Yan, Zheng <ukernel@gmail.com> wrote:
> On Sat, Jan 30, 2016 at 12:09 AM, Sage Weil <sweil@redhat.com> wrote:
>> Hi Zheng,
>>
>> We have one feature already merged to ceph.git, CRUSH_TUNABLES5, that's a
>> pretty trivial change to crush/mapper.c that we can add to the kernel.
>>
>> But there are two more coming:
>>
>> OSDOPREPLY_ENCODING makes a small change to the OSDOpReply encoding for
>> performance reasons.  Here is the PR
>>
>>         https://github.com/ceph/ceph/pull/7386
>>
>> The other is the file layout changes, which are much more invovled:
>>
>>         https://github.com/ceph/ceph/pull/7098
>
> Can we avoid changing how file layout is encoded in MClientCaps and
> MClientReply? Instead, we encode the namespace information at the end
> of MClientCaps/MClientReply message. (like how we added inline data
> related fields to these message).

see

https://github.com/ukernel/ceph/commit/930c4982e9fda9e894dda9d5c91d17a7bed1b13a
https://github.com/ceph/ceph-client/commit/4cc460f0d57ca735bf484e63fcde917827c1775b

patch for kernel is small. should be OK for 4.5-rc


Regards
Yan, Zheng

>
> Regards
> Yan, Zheng
>
>>
>> Ideally we'd use the same feature bit for all of these, but it means they
>> all need to upstream into Linux together.  Do you see a problem with that?
>> Assuming we get it all ready to go before the next merge window..
>>
>> The tricky bit is we really need to know that we'll be able to do it
>> together before we cut jewel too and commit to the overlapping feature
>> bits.
>>
>> sage
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: client-facing feature bits and pending kernel changes

[Yan, Zheng]

On Mon, Feb 1, 2016 at 9:08 PM, Sage Weil <sweil@redhat.com> wrote:
> On Mon, 1 Feb 2016, Yan, Zheng wrote:
>> On Sat, Jan 30, 2016 at 12:09 AM, Sage Weil <sweil@redhat.com> wrote:
>> > Hi Zheng,
>> >
>> > We have one feature already merged to ceph.git, CRUSH_TUNABLES5, that's a
>> > pretty trivial change to crush/mapper.c that we can add to the kernel.
>> >
>> > But there are two more coming:
>> >
>> > OSDOPREPLY_ENCODING makes a small change to the OSDOpReply encoding for
>> > performance reasons.  Here is the PR
>> >
>> >         https://github.com/ceph/ceph/pull/7386
>> >
>> > The other is the file layout changes, which are much more invovled:
>> >
>> >         https://github.com/ceph/ceph/pull/7098
>>
>> Can we avoid changing how file layout is encoded in MClientCaps and
>> MClientReply? Instead, we encode the namespace information at the end
>> of MClientCaps/MClientReply message. (like how we added inline data
>> related fields to these message).
>
> Hmm.. yeah, we do that, at least for MClientCaps.  It just means that if
> we ever add more fields to that struct we need to maintain the message
> encoding in parallel by hand, but that shouldn't be too bad.
>
> The MClientReply one is a bit trickier because there can be N instances of
> InodeStat (or whatever) in there... on that one I'd lean toward switching
> the encoding.  What do you think?
>

We can use conection's features to decide how many memory each
InodeStat uses. It's how inline data was added to InodeStat

Regards
Yan, Zheng

Re: client-facing feature bits and pending kernel changes

[Sage Weil]

On Mon, 1 Feb 2016, Yan, Zheng wrote:
> On Mon, Feb 1, 2016 at 5:30 PM, Yan, Zheng <ukernel@gmail.com> wrote:
> > On Sat, Jan 30, 2016 at 12:09 AM, Sage Weil <sweil@redhat.com> wrote:
> >> Hi Zheng,
> >>
> >> We have one feature already merged to ceph.git, CRUSH_TUNABLES5, that's a
> >> pretty trivial change to crush/mapper.c that we can add to the kernel.
> >>
> >> But there are two more coming:
> >>
> >> OSDOPREPLY_ENCODING makes a small change to the OSDOpReply encoding for
> >> performance reasons.  Here is the PR
> >>
> >>         https://github.com/ceph/ceph/pull/7386
> >>
> >> The other is the file layout changes, which are much more invovled:
> >>
> >>         https://github.com/ceph/ceph/pull/7098
> >
> > Can we avoid changing how file layout is encoded in MClientCaps and
> > MClientReply? Instead, we encode the namespace information at the end
> > of MClientCaps/MClientReply message. (like how we added inline data
> > related fields to these message).
> 
> see
> 
> https://github.com/ukernel/ceph/commit/930c4982e9fda9e894dda9d5c91d17a7bed1b13a
> https://github.com/ceph/ceph-client/commit/4cc460f0d57ca735bf484e63fcde917827c1775b
> 
> patch for kernel is small. should be OK for 4.5-rc

Cool, sounds good to me, then!  I'll squash the ceph.git patch down into 
the original commit(s) in wip-file-layout.

sage