CephFS Samba VFS RHEL packages

CephFS Samba VFS RHEL packages

[Blair Bethwaite]

Hi all,

We've started a CephFS Samba PoC on RHEL but just noticed the Samba
Ceph VFS doesn't seem to be included with Samba on RHEL, or we're not
looking in the right place. Trying to avoid needing to build Samba
from source if possible. Any pointers appreciated.

-- 
Cheers,
~Blairo

Re: CephFS Samba VFS RHEL packages

[John Spray]

On Wed, Jul 20, 2016 at 6:45 AM, Blair Bethwaite
<blair.bethwaite@gmail.com> wrote:
> Hi all,
>
> We've started a CephFS Samba PoC on RHEL but just noticed the Samba
> Ceph VFS doesn't seem to be included with Samba on RHEL, or we're not
> looking in the right place. Trying to avoid needing to build Samba
> from source if possible. Any pointers appreciated.

RHEL packagers tend to be conservative about building in feature flags
that aren't official parts of their product.

Unfortunately we do our upstream Samba testing on Ubuntu, so don't
have any RPMs from the gitbuilders either; you're probably going to
have to build your own.

CCing Ira who was recently evaluating the state of Samba+CephFS and
might have some tips on setting this up on RHEL.

Cheers,
John

> --
> Cheers,
> ~Blairo
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CephFS Samba VFS RHEL packages

[Ira Cooper]

On Wed, Jul 20, 2016 at 10:59:36AM +0100, John Spray wrote:
> On Wed, Jul 20, 2016 at 6:45 AM, Blair Bethwaite
> <blair.bethwaite@gmail.com> wrote:
> > Hi all,
> >
> > We've started a CephFS Samba PoC on RHEL but just noticed the Samba
> > Ceph VFS doesn't seem to be included with Samba on RHEL, or we're not
> > looking in the right place. Trying to avoid needing to build Samba
> > from source if possible. Any pointers appreciated.
> 
> RHEL packagers tend to be conservative about building in feature flags
> that aren't official parts of their product.
> 
> Unfortunately we do our upstream Samba testing on Ubuntu, so don't
> have any RPMs from the gitbuilders either; you're probably going to
> have to build your own.
> 
> CCing Ira who was recently evaluating the state of Samba+CephFS and
> might have some tips on setting this up on RHEL.

RHEL: I don't expect that the packages will ever be part of the
RHEL Samba build.  The ones for Gluster aren't, as precident.  But
I don't speak for Red Hat.   :)

I also don't work on RHEL.  I work on Fedora.  So all of this is
written from the point of view of Fedora 23.  I'm working on
straightening out my Fedora 24 development setup.

As far as state of the world?

CTDB: It can be made to work, but I don't recommend it at this
time.  The settings needed aren't friendly to CephFS, and I
would not recommend this in production today.

ACLs: Aren't implemented in vfs_ceph, because I still have to
submit the refactored vfs_ceph code to the Samba team list,
but I ran into testing issues on the CephFS side, regarding
ACL setting, if the Gluster side works, I may just see if we
can push these anyways.  The refactoring unifies the ACL
handling between vfs_ceph and vfs_gluster.

Working absent those two issues: Seems ok, for modern Ceph
bits.  most recent Jewel should be "ok", as I understand
it.  I did my work on pre-release Jewel bits.

I recommend looking at my presentation that I gave at SambaXP.
http://sambaxp.org/ day 3.  It may be out of date in a few
spots, but it should be close.

Cheers,

-Ira / ira@(samba.org|wakeful.net)

Re: CephFS Samba VFS RHEL packages

[Ken Dreyer]

The Samba packages in Fedora 22+ do enable the Samba VFS:
https://bugzilla.redhat.com/1174412

From what Ira said downthread, this is pretty experimental, so you
could run your tests on a Fedora system and see how it goes :)

- Ken

On Tue, Jul 19, 2016 at 11:45 PM, Blair Bethwaite
<blair.bethwaite@gmail.com> wrote:
> Hi all,
>
> We've started a CephFS Samba PoC on RHEL but just noticed the Samba
> Ceph VFS doesn't seem to be included with Samba on RHEL, or we're not
> looking in the right place. Trying to avoid needing to build Samba
> from source if possible. Any pointers appreciated.
>
> --
> Cheers,
> ~Blairo
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CephFS Samba VFS RHEL packages

[Blair Bethwaite]

Ken, Ira, John -

Thanks a lot for the replies. Our initial setup is simply running
samba atop a cephfs kernel mount, and initial cursory checks seem to
show the basics are working as expected (even clustered with ctdb -
what are your concerns here Ira?). Though we've yet to try any of our
planned test scenarios/datasets.

From the conversation here I'm thinking we'd be better off using
Xenial for our CephFS+Samba test nodes at the moment... though I only
see packages for Precise and Trusty on gitbuilder.ceph.com at the
moment. We were planning to compare samba on kernel mount versus ceph
vfs.

Can someone clarify what state the cephfs kernel client in RHEL 7.x
will be in when RHCS 2.0 is released, i.e., will that be back-ported
or are RHEL users expected to use FUSE? (I'm happy to go ask support
directly but I suspect this is useful information for others too).

Cheers,

On 21 July 2016 at 02:32, Ken Dreyer <kdreyer@redhat.com> wrote:
> The Samba packages in Fedora 22+ do enable the Samba VFS:
> https://bugzilla.redhat.com/1174412
>
> From what Ira said downthread, this is pretty experimental, so you
> could run your tests on a Fedora system and see how it goes :)
>
> - Ken
>
> On Tue, Jul 19, 2016 at 11:45 PM, Blair Bethwaite
> <blair.bethwaite@gmail.com> wrote:
>> Hi all,
>>
>> We've started a CephFS Samba PoC on RHEL but just noticed the Samba
>> Ceph VFS doesn't seem to be included with Samba on RHEL, or we're not
>> looking in the right place. Trying to avoid needing to build Samba
>> from source if possible. Any pointers appreciated.
>>
>> --
>> Cheers,
>> ~Blairo
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Cheers,
~Blairo

Re: CephFS Samba VFS RHEL packages

[Ira Cooper]

On Fri, Jul 22, 2016 at 08:29:39AM +1100, Blair Bethwaite wrote:
> Ken, Ira, John -
> 
> Thanks a lot for the replies. Our initial setup is simply running
> samba atop a cephfs kernel mount, and initial cursory checks seem to
> show the basics are working as expected (even clustered with ctdb -
> what are your concerns here Ira?). Though we've yet to try any of our
> planned test scenarios/datasets.
> 
> From the conversation here I'm thinking we'd be better off using
> Xenial for our CephFS+Samba test nodes at the moment... though I only
> see packages for Precise and Trusty on gitbuilder.ceph.com at the
> moment. We were planning to compare samba on kernel mount versus ceph
> vfs.
> 
> Can someone clarify what state the cephfs kernel client in RHEL 7.x
> will be in when RHCS 2.0 is released, i.e., will that be back-ported
> or are RHEL users expected to use FUSE? (I'm happy to go ask support
> directly but I suspect this is useful information for others too).

Please note: icooper@redhat.com is not on this list, so I can't
reply from there ;).

I recommend you test your setup THROUGHLY.  CTDB is really in
place to handle failures, so test node failures.  I recommend
hard power-offs for this.  The real world is rarely as kind
as a "nice" poweroff, and it might trick you into thinking
more is working than is.

If you get it right, all the nodes should end up banned if I
remember right.  It's been a bit since I tested with stock
settings.

For my setup I use:

FUSE for the mount so CTDB works.
vfs_ceph for the Samba datapath.

That's what I'd recommend based on talking to people.

No idea on RHCS/Kernel.

Cheers,

-Ira

Re: CephFS Samba VFS RHEL packages

[Yan, Zheng]

On Fri, Jul 22, 2016 at 9:11 AM, Ira Cooper <ira-mA6sH8tAw6XR7s880joybQ@public.gmane.org> wrote:
> On Fri, Jul 22, 2016 at 08:29:39AM +1100, Blair Bethwaite wrote:
>> Ken, Ira, John -
>>
>> Thanks a lot for the replies. Our initial setup is simply running
>> samba atop a cephfs kernel mount, and initial cursory checks seem to
>> show the basics are working as expected (even clustered with ctdb -
>> what are your concerns here Ira?). Though we've yet to try any of our
>> planned test scenarios/datasets.
>>
>> From the conversation here I'm thinking we'd be better off using
>> Xenial for our CephFS+Samba test nodes at the moment... though I only
>> see packages for Precise and Trusty on gitbuilder.ceph.com at the
>> moment. We were planning to compare samba on kernel mount versus ceph
>> vfs.
>>
>> Can someone clarify what state the cephfs kernel client in RHEL 7.x
>> will be in when RHCS 2.0 is released, i.e., will that be back-ported
>> or are RHEL users expected to use FUSE? (I'm happy to go ask support
>> directly but I suspect this is useful information for others too).

We actively back-port fixes to RHEL 7.x kernel.  When RHCS2.0 release,
the RHEL kernel should contain fixes up to 3.7 upstream kernel.

Regards
Yan, Zheng

>
> Please note: icooper-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org is not on this list, so I can't
> reply from there ;).
>
> I recommend you test your setup THROUGHLY.  CTDB is really in
> place to handle failures, so test node failures.  I recommend
> hard power-offs for this.  The real world is rarely as kind
> as a "nice" poweroff, and it might trick you into thinking
> more is working than is.
>
> If you get it right, all the nodes should end up banned if I
> remember right.  It's been a bit since I tested with stock
> settings.
>
> For my setup I use:
>
> FUSE for the mount so CTDB works.
> vfs_ceph for the Samba datapath.
>
> That's what I'd recommend based on talking to people.
>
> No idea on RHCS/Kernel.
>
> Cheers,
>
> -Ira
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CephFS Samba VFS RHEL packages

[Blair Bethwaite]

Thanks Zheng,

On 22 July 2016 at 12:12, Yan, Zheng <ukernel@gmail.com> wrote:
> We actively back-port fixes to RHEL 7.x kernel.  When RHCS2.0 release,
> the RHEL kernel should contain fixes up to 3.7 upstream kernel.

You meant 4.7 right?

-- 
Cheers,
~Blairo

Re: CephFS Samba VFS RHEL packages

[Yan, Zheng]

On Fri, Jul 22, 2016 at 11:15 AM, Blair Bethwaite
<blair.bethwaite@gmail.com> wrote:
> Thanks Zheng,
>
> On 22 July 2016 at 12:12, Yan, Zheng <ukernel@gmail.com> wrote:
>> We actively back-port fixes to RHEL 7.x kernel.  When RHCS2.0 release,
>> the RHEL kernel should contain fixes up to 3.7 upstream kernel.
>
> You meant 4.7 right?

I mean 4.7. sorry

>
> --
> Cheers,
> ~Blairo

Re: CephFS Samba VFS RHEL packages

[Brett Niver]

[-- Attachment #1.1: Type: text/plain, Size: 914 bytes --]

So Blair,

As far as you know right now, you don't need anything from the CephFS team,
correct?
Thanks,
Brett


On Fri, Jul 22, 2016 at 2:18 AM, Yan, Zheng <ukernel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> On Fri, Jul 22, 2016 at 11:15 AM, Blair Bethwaite
> <blair.bethwaite-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > Thanks Zheng,
> >
> > On 22 July 2016 at 12:12, Yan, Zheng <ukernel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >> We actively back-port fixes to RHEL 7.x kernel.  When RHCS2.0 release,
> >> the RHEL kernel should contain fixes up to 3.7 upstream kernel.
> >
> > You meant 4.7 right?
>
> I mean 4.7. sorry
>
> >
> > --
> > Cheers,
> > ~Blairo
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

[-- Attachment #1.2: Type: text/html, Size: 1732 bytes --]

[-- Attachment #2: Type: text/plain, Size: 178 bytes --]

_______________________________________________
ceph-users mailing list
ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: CephFS Samba VFS RHEL packages

[Blair Bethwaite]

Hi Brett,

Don't think so thanks, but we'll shout if we find any breakage/weirdness.

Cheers,

On 23 July 2016 at 05:42, Brett Niver <bniver-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> So Blair,
>
> As far as you know right now, you don't need anything from the CephFS team,
> correct?
> Thanks,
> Brett
>
>
> On Fri, Jul 22, 2016 at 2:18 AM, Yan, Zheng <ukernel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>
>> On Fri, Jul 22, 2016 at 11:15 AM, Blair Bethwaite
>> <blair.bethwaite-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> > Thanks Zheng,
>> >
>> > On 22 July 2016 at 12:12, Yan, Zheng <ukernel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> >> We actively back-port fixes to RHEL 7.x kernel.  When RHCS2.0 release,
>> >> the RHEL kernel should contain fixes up to 3.7 upstream kernel.
>> >
>> > You meant 4.7 right?
>>
>> I mean 4.7. sorry
>>
>> >
>> > --
>> > Cheers,
>> > ~Blairo
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>



-- 
Cheers,
~Blairo