* Questions about ceph/cephfs @ 2021-11-29 13:36 Christian Brauner 2021-11-29 13:57 ` Jeff Layton 0 siblings, 1 reply; 3+ messages in thread From: Christian Brauner @ 2021-11-29 13:36 UTC (permalink / raw) To: Jeff Layton; +Cc: Christian Brauner, ceph-devel, Ilya Dryomov Hey Jeff, Hey everyone, I have a question about the relationship between the cephfs kernel client/filesystem and the code in [1]. I'm working on patches to let cephfs support mapped mounts. I've got everything in place and I'm able to run the full mapped mounts/vfs testsuite against a ceph cluster. So things like: mount -t ceph 1.2.3.4:6789:/ /mnt -o name=admin,secret=wootwoot mount-idmapped --map-mount b:1000:0:1 /mnt /opt (Where in /opt all inodes owned by id 1000 are owned by id 0 and callers with id 0 write files to disk as id 1000.) My next step was to read to the ceph code in github.com/ceph/ceph to make sure that I don't miss any corner cases that wouldn't work correctly with mapped mounts after the relevant kernel changes. So I read through [1] and I could use some guidance if possible. The code in there to me reads like it is a full implementation of cephfs in userspace and I'm wondering how this related to the cephfs kernel client/filesystem. The client code in [1] seems to implement full-blown userspace path lookup and permission checking that seems to reimplement the vfs layer in userspace. A lot of the helpers such as ceph_ll_mknod() in addition expose a UserPerms struct that seems to be settable by the caller which is used during lookup. How does this permission/path lookup code relate to the cephfs kernel client/filesystem? Do they interact with each other in any way? Thanks for taking the time to answer! Christian [1]: https://github.com/ceph/ceph/tree/master/src/client/* ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Questions about ceph/cephfs 2021-11-29 13:36 Questions about ceph/cephfs Christian Brauner @ 2021-11-29 13:57 ` Jeff Layton 2021-11-29 14:03 ` Christian Brauner 0 siblings, 1 reply; 3+ messages in thread From: Jeff Layton @ 2021-11-29 13:57 UTC (permalink / raw) To: Christian Brauner; +Cc: Christian Brauner, ceph-devel, Ilya Dryomov On Mon, 2021-11-29 at 14:36 +0100, Christian Brauner wrote: > Hey Jeff, > Hey everyone, > > I have a question about the relationship between the cephfs kernel > client/filesystem and the code in [1]. > > I'm working on patches to let cephfs support mapped mounts. I've got > everything in place and I'm able to run the full mapped mounts/vfs > testsuite against a ceph cluster. So things like: > > mount -t ceph 1.2.3.4:6789:/ /mnt -o name=admin,secret=wootwoot > mount-idmapped --map-mount b:1000:0:1 /mnt /opt > Sounds like a cool feature. > (Where in /opt all inodes owned by id 1000 are owned by id 0 and callers > with id 0 write files to disk as id 1000.) > > My next step was to read to the ceph code in github.com/ceph/ceph to > make sure that I don't miss any corner cases that wouldn't work > correctly with mapped mounts after the relevant kernel changes. > > So I read through [1] and I could use some guidance if possible. The > code in there to me reads like it is a full implementation of cephfs in > userspace and I'm wondering how this related to the cephfs kernel > client/filesystem. > > The client code in [1] seems to implement full-blown userspace path > lookup and permission checking that seems to reimplement the vfs layer > in userspace. > A lot of the helpers such as ceph_ll_mknod() in addition expose a > UserPerms struct that seems to be settable by the caller which is used > during lookup. > > How does this permission/path lookup code relate to the cephfs kernel > client/filesystem? Do they interact with each other in any way? > > Thanks for taking the time to answer! > Christian > > [1]: https://github.com/ceph/ceph/tree/master/src/client/* There are really two "official" cephfs clients: the userland client which is (mostly) represented by src/client in the userland ceph tree, and the kernel cephfs client (aka the kclient), which is (mostly) fs/ceph and net/ceph in the Linux kernel tree. The userland code usually gets compiled into libcephfs.so and is the backend engine for ceph-fuse, the nfs-ganesha FSAL driver, and the samba vfs backend driver, as well as a bunch of testcases in the ceph tree. libcephfs communicates directly with the cluster without interacting with any of the kernel ceph code, so ceph-fuse, ganesha, etc. do use the userland code for lookups and permission checks. Technically, the kclient is the reimplentation. Most of the ceph and libceph code in the kernel has been copied/reimplemented from the ceph userland code. There is no real dependency between the kclient and libcephfs. Maintaining the kclient is a continual game of catchup, unfortunately, though in some cases we do leading edge client development there instead and play catchup in the userland client. Cheers, -- Jeff Layton <jlayton@kernel.org> ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Questions about ceph/cephfs 2021-11-29 13:57 ` Jeff Layton @ 2021-11-29 14:03 ` Christian Brauner 0 siblings, 0 replies; 3+ messages in thread From: Christian Brauner @ 2021-11-29 14:03 UTC (permalink / raw) To: Jeff Layton; +Cc: Christian Brauner, ceph-devel, Ilya Dryomov On Mon, Nov 29, 2021 at 08:57:37AM -0500, Jeff Layton wrote: > On Mon, 2021-11-29 at 14:36 +0100, Christian Brauner wrote: > > Hey Jeff, > > Hey everyone, > > > > I have a question about the relationship between the cephfs kernel > > client/filesystem and the code in [1]. > > > > I'm working on patches to let cephfs support mapped mounts. I've got > > everything in place and I'm able to run the full mapped mounts/vfs > > testsuite against a ceph cluster. So things like: > > > > mount -t ceph 1.2.3.4:6789:/ /mnt -o name=admin,secret=wootwoot > > mount-idmapped --map-mount b:1000:0:1 /mnt /opt > > > > Sounds like a cool feature. > > > (Where in /opt all inodes owned by id 1000 are owned by id 0 and callers > > with id 0 write files to disk as id 1000.) > > > > My next step was to read to the ceph code in github.com/ceph/ceph to > > make sure that I don't miss any corner cases that wouldn't work > > correctly with mapped mounts after the relevant kernel changes. > > > > So I read through [1] and I could use some guidance if possible. The > > code in there to me reads like it is a full implementation of cephfs in > > userspace and I'm wondering how this related to the cephfs kernel > > client/filesystem. > > > > The client code in [1] seems to implement full-blown userspace path > > lookup and permission checking that seems to reimplement the vfs layer > > in userspace. > > A lot of the helpers such as ceph_ll_mknod() in addition expose a > > UserPerms struct that seems to be settable by the caller which is used > > during lookup. > > > > How does this permission/path lookup code relate to the cephfs kernel > > client/filesystem? Do they interact with each other in any way? > > > > Thanks for taking the time to answer! > > Christian > > > > [1]: https://github.com/ceph/ceph/tree/master/src/client/* > > There are really two "official" cephfs clients: the userland client > which is (mostly) represented by src/client in the userland ceph tree, > and the kernel cephfs client (aka the kclient), which is (mostly) > fs/ceph and net/ceph in the Linux kernel tree. > > The userland code usually gets compiled into libcephfs.so and is the > backend engine for ceph-fuse, the nfs-ganesha FSAL driver, and the samba > vfs backend driver, as well as a bunch of testcases in the ceph tree. > > libcephfs communicates directly with the cluster without interacting > with any of the kernel ceph code, so ceph-fuse, ganesha, etc. do use the > userland code for lookups and permission checks. > > Technically, the kclient is the reimplentation. Most of the ceph and > libceph code in the kernel has been copied/reimplemented from the ceph > userland code. There is no real dependency between the kclient and > libcephfs. > > Maintaining the kclient is a continual game of catchup, unfortunately, > though in some cases we do leading edge client development there instead > and play catchup in the userland client. Thank you for answering this so quickly this makes my life a lot easier! This is good news as this means that the kernel changes to cephfs should be fairly simple and self-contained. I'll continue polishing them and will send them out once ready. Thanks again! Christian ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-11-29 15:50 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-11-29 13:36 Questions about ceph/cephfs Christian Brauner 2021-11-29 13:57 ` Jeff Layton 2021-11-29 14:03 ` Christian Brauner
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.