From: xiubli@redhat.com
To: idryomov@gmail.com, ceph-devel@vger.kernel.org
Cc: jlayton@kernel.org, lhenriques@suse.de, vshankar@redhat.com,
mchangir@redhat.com, Xiubo Li <xiubli@redhat.com>
Subject: [PATCH v16 00/68] ceph+fscrypt: full support
Date: Mon, 27 Feb 2023 11:27:05 +0800 [thread overview]
Message-ID: <20230227032813.337906-1-xiubli@redhat.com> (raw)
From: Xiubo Li <xiubli@redhat.com>
This patch series is based on Jeff Layton's previous great work and effort
on this.
Since v15 we have added the ceph qa teuthology test cases for this [1][2],
which will test both the file name and contents encryption features and at
the same time they will also test the IO benchmarks.
To support the fscrypt we also have some other work in ceph [3][4][5][6][7][8]:
[1] https://github.com/ceph/ceph/pull/48628
[2] https://github.com/ceph/ceph/pull/49934
[3] https://github.com/ceph/ceph/pull/43588
[4] https://github.com/ceph/ceph/pull/37297
[5] https://github.com/ceph/ceph/pull/45192
[6] https://github.com/ceph/ceph/pull/45312
[7] https://github.com/ceph/ceph/pull/40828
[8] https://github.com/ceph/ceph/pull/45224
[9] https://github.com/ceph/ceph/pull/45073
The [8] and [9] are still undering testing and will soon be merged after
that. All the others had been merged long time ago.
The main changes since v15:
- rebased onto v6.2
- some minor cleanups and revises reported by:
kernel test robot <lkp@intel.com>
Milind Changire <mchangir@redhat.com>
Ilya Dryomov <idryomov@gmail.com>
- Luis' fixes:
ceph: mark directory as non-complete complete after loading key
ceph: fix memory leak in mount error path when using test_dummy_encryption
ceph: allow encrypting a directory while not having Ax caps
NOTE: the second one has been folded into a previous commit.
- fixed a sequence bug to make sure the osd req->r_callback()s are
called before removing the reqs from the osdc, which will cause
a warning in fscrypt_destroy_keyring() and potential use-after-free
crash bugs in other places.
Jeff Layton (47):
libceph: add spinlock around osd->o_requests
libceph: define struct ceph_sparse_extent and add some helpers
libceph: add sparse read support to msgr2 crc state machine
libceph: add sparse read support to OSD client
libceph: support sparse reads on msgr2 secure codepath
libceph: add sparse read support to msgr1
ceph: add new mount option to enable sparse reads
ceph: preallocate inode for ops that may create one
ceph: make ceph_msdc_build_path use ref-walk
libceph: add new iov_iter-based ceph_msg_data_type and
ceph_osd_data_type
ceph: use osd_req_op_extent_osd_iter for netfs reads
ceph: fscrypt_auth handling for ceph
ceph: ensure that we accept a new context from MDS for new inodes
ceph: add support for fscrypt_auth/fscrypt_file to cap messages
ceph: implement -o test_dummy_encryption mount option
ceph: decode alternate_name in lease info
ceph: add fscrypt ioctls
ceph: add encrypted fname handling to ceph_mdsc_build_path
ceph: send altname in MClientRequest
ceph: encode encrypted name in dentry release
ceph: properly set DCACHE_NOKEY_NAME flag in lookup
ceph: set DCACHE_NOKEY_NAME in atomic open
ceph: make d_revalidate call fscrypt revalidator for encrypted
dentries
ceph: add helpers for converting names for userland presentation
ceph: add fscrypt support to ceph_fill_trace
ceph: create symlinks with encrypted and base64-encoded targets
ceph: make ceph_get_name decrypt filenames
ceph: add a new ceph.fscrypt.auth vxattr
ceph: add some fscrypt guardrails
libceph: add CEPH_OSD_OP_ASSERT_VER support
ceph: size handling for encrypted inodes in cap updates
ceph: fscrypt_file field handling in MClientRequest messages
ceph: handle fscrypt fields in cap messages from MDS
ceph: update WARN_ON message to pr_warn
ceph: add infrastructure for file encryption and decryption
libceph: allow ceph_osdc_new_request to accept a multi-op read
ceph: disable fallocate for encrypted inodes
ceph: disable copy offload on encrypted inodes
ceph: don't use special DIO path for encrypted inodes
ceph: align data in pages in ceph_sync_write
ceph: add read/modify/write to ceph_sync_write
ceph: plumb in decryption during sync reads
ceph: add fscrypt decryption support to ceph_netfs_issue_op
ceph: set i_blkbits to crypto block size for encrypted inodes
ceph: add encryption support to writepage
ceph: fscrypt support for writepages
ceph: report STATX_ATTR_ENCRYPTED on encrypted inodes
Luís Henriques (9):
ceph: add base64 endcoding routines for encrypted names
ceph: allow encrypting a directory while not having Ax caps
ceph: mark directory as non-complete after loading key
ceph: don't allow changing layout on encrypted files/directories
ceph: invalidate pages when doing direct/sync writes
ceph: add support for encrypted snapshot names
ceph: add support for handling encrypted snapshot names
ceph: update documentation regarding snapshot naming limitations
ceph: prevent snapshots to be created in encrypted locked directories
Xiubo Li (12):
ceph: make the ioctl cmd more readable in debug log
ceph: fix base64 encoded name's length check in ceph_fname_to_usr()
ceph: pass the request to parse_reply_info_readdir()
ceph: add ceph_encode_encrypted_dname() helper
ceph: add support to readdir for encrypted filenames
ceph: get file size from fscrypt_file when present in inode traces
ceph: add __ceph_get_caps helper support
ceph: add __ceph_sync_read helper support
ceph: add object version support for sync read
ceph: add truncate size handling support for fscrypt
libceph: defer removing the req from osdc just after req->r_callback
ceph: drop the messages from MDS when unmounting
Documentation/filesystems/ceph.rst | 10 +
fs/ceph/Makefile | 1 +
fs/ceph/acl.c | 4 +-
fs/ceph/addr.c | 182 ++++++--
fs/ceph/caps.c | 222 ++++++++--
fs/ceph/crypto.c | 669 +++++++++++++++++++++++++++++
fs/ceph/crypto.h | 270 ++++++++++++
fs/ceph/dir.c | 187 ++++++--
fs/ceph/export.c | 44 +-
fs/ceph/file.c | 595 +++++++++++++++++++++----
fs/ceph/inode.c | 580 +++++++++++++++++++++++--
fs/ceph/ioctl.c | 126 +++++-
fs/ceph/mds_client.c | 477 +++++++++++++++++---
fs/ceph/mds_client.h | 29 +-
fs/ceph/quota.c | 4 +
fs/ceph/snap.c | 6 +
fs/ceph/super.c | 169 +++++++-
fs/ceph/super.h | 44 +-
fs/ceph/xattr.c | 29 ++
include/linux/ceph/ceph_fs.h | 21 +-
include/linux/ceph/messenger.h | 40 ++
include/linux/ceph/osd_client.h | 93 +++-
include/linux/ceph/rados.h | 4 +
net/ceph/messenger.c | 79 ++++
net/ceph/messenger_v1.c | 98 ++++-
net/ceph/messenger_v2.c | 286 +++++++++++-
net/ceph/osd_client.c | 369 +++++++++++++++-
27 files changed, 4243 insertions(+), 395 deletions(-)
create mode 100644 fs/ceph/crypto.c
create mode 100644 fs/ceph/crypto.h
--
2.31.1
next reply other threads:[~2023-02-27 3:29 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-27 3:27 xiubli [this message]
2023-02-27 3:27 ` [PATCH v16 01/68] libceph: add spinlock around osd->o_requests xiubli
2023-02-27 3:27 ` [PATCH v16 02/68] libceph: define struct ceph_sparse_extent and add some helpers xiubli
2023-02-27 3:27 ` [PATCH v16 03/68] libceph: add sparse read support to msgr2 crc state machine xiubli
2023-02-27 3:27 ` [PATCH v16 04/68] libceph: add sparse read support to OSD client xiubli
2023-02-27 3:27 ` [PATCH v16 05/68] libceph: support sparse reads on msgr2 secure codepath xiubli
2023-02-27 3:27 ` [PATCH v16 06/68] libceph: add sparse read support to msgr1 xiubli
2023-02-27 3:27 ` [PATCH v16 07/68] ceph: add new mount option to enable sparse reads xiubli
2023-02-27 3:27 ` [PATCH v16 08/68] ceph: preallocate inode for ops that may create one xiubli
2023-02-27 3:27 ` [PATCH v16 09/68] ceph: make ceph_msdc_build_path use ref-walk xiubli
2023-02-27 3:27 ` [PATCH v16 10/68] libceph: add new iov_iter-based ceph_msg_data_type and ceph_osd_data_type xiubli
2023-02-27 3:27 ` [PATCH v16 11/68] ceph: use osd_req_op_extent_osd_iter for netfs reads xiubli
2023-02-27 3:27 ` [PATCH v16 12/68] ceph: fscrypt_auth handling for ceph xiubli
2023-02-27 3:27 ` [PATCH v16 13/68] ceph: ensure that we accept a new context from MDS for new inodes xiubli
2023-02-27 3:27 ` [PATCH v16 14/68] ceph: add support for fscrypt_auth/fscrypt_file to cap messages xiubli
2023-02-27 3:27 ` [PATCH v16 15/68] ceph: implement -o test_dummy_encryption mount option xiubli
2023-02-27 3:27 ` [PATCH v16 16/68] ceph: decode alternate_name in lease info xiubli
2023-02-27 3:27 ` [PATCH v16 17/68] ceph: add fscrypt ioctls xiubli
2023-02-27 3:27 ` [PATCH v16 18/68] ceph: make the ioctl cmd more readable in debug log xiubli
2023-02-27 3:27 ` [PATCH v16 19/68] ceph: add base64 endcoding routines for encrypted names xiubli
2023-02-27 3:27 ` [PATCH v16 20/68] ceph: add encrypted fname handling to ceph_mdsc_build_path xiubli
2023-02-27 3:27 ` [PATCH v16 21/68] ceph: send altname in MClientRequest xiubli
2023-02-27 3:27 ` [PATCH v16 22/68] ceph: encode encrypted name in dentry release xiubli
2023-02-27 3:27 ` [PATCH v16 23/68] ceph: properly set DCACHE_NOKEY_NAME flag in lookup xiubli
2023-02-27 3:27 ` [PATCH v16 24/68] ceph: set DCACHE_NOKEY_NAME in atomic open xiubli
2023-02-27 3:27 ` [PATCH v16 25/68] ceph: make d_revalidate call fscrypt revalidator for encrypted dentries xiubli
2023-03-07 18:53 ` Luís Henriques
2023-03-08 1:50 ` Xiubo Li
2023-03-08 9:29 ` Luís Henriques
2023-03-08 10:42 ` Xiubo Li
2023-03-08 17:14 ` Luís Henriques
2023-03-08 17:54 ` Jeff Layton
2023-03-08 18:30 ` Luís Henriques
2023-03-08 19:32 ` Jeff Layton
2023-03-09 9:52 ` Luís Henriques
2023-03-09 7:06 ` Xiubo Li
2023-03-09 9:55 ` Luís Henriques
2023-03-09 11:41 ` Xiubo Li
2023-02-27 3:27 ` [PATCH v16 26/68] ceph: add helpers for converting names for userland presentation xiubli
2023-02-27 3:27 ` [PATCH v16 27/68] ceph: fix base64 encoded name's length check in ceph_fname_to_usr() xiubli
2023-02-27 3:27 ` [PATCH v16 28/68] ceph: add fscrypt support to ceph_fill_trace xiubli
2023-02-27 3:27 ` [PATCH v16 29/68] ceph: pass the request to parse_reply_info_readdir() xiubli
2023-02-27 3:27 ` [PATCH v16 30/68] ceph: add ceph_encode_encrypted_dname() helper xiubli
2023-02-27 3:27 ` [PATCH v16 31/68] ceph: add support to readdir for encrypted filenames xiubli
2023-02-27 3:27 ` [PATCH v16 32/68] ceph: create symlinks with encrypted and base64-encoded targets xiubli
2023-02-27 3:27 ` [PATCH v16 33/68] ceph: make ceph_get_name decrypt filenames xiubli
2023-02-27 3:27 ` [PATCH v16 34/68] ceph: add a new ceph.fscrypt.auth vxattr xiubli
2023-02-27 3:27 ` [PATCH v16 35/68] ceph: add some fscrypt guardrails xiubli
2023-02-27 3:27 ` [PATCH v16 36/68] ceph: allow encrypting a directory while not having Ax caps xiubli
2023-02-27 3:27 ` [PATCH v16 37/68] ceph: mark directory as non-complete after loading key xiubli
2023-02-27 3:27 ` [PATCH v16 38/68] ceph: don't allow changing layout on encrypted files/directories xiubli
2023-02-27 3:27 ` [PATCH v16 39/68] libceph: add CEPH_OSD_OP_ASSERT_VER support xiubli
2023-02-27 3:27 ` [PATCH v16 40/68] ceph: size handling for encrypted inodes in cap updates xiubli
2023-02-27 3:27 ` [PATCH v16 41/68] ceph: fscrypt_file field handling in MClientRequest messages xiubli
2023-02-27 3:27 ` [PATCH v16 42/68] ceph: get file size from fscrypt_file when present in inode traces xiubli
2023-02-27 3:27 ` [PATCH v16 43/68] ceph: handle fscrypt fields in cap messages from MDS xiubli
2023-02-27 3:27 ` [PATCH v16 44/68] ceph: update WARN_ON message to pr_warn xiubli
2023-02-27 3:27 ` [PATCH v16 45/68] ceph: add __ceph_get_caps helper support xiubli
2023-02-27 3:27 ` [PATCH v16 46/68] ceph: add __ceph_sync_read " xiubli
2023-02-27 3:27 ` [PATCH v16 47/68] ceph: add object version support for sync read xiubli
2023-02-27 3:27 ` [PATCH v16 48/68] ceph: add infrastructure for file encryption and decryption xiubli
2023-02-27 3:27 ` [PATCH v16 49/68] ceph: add truncate size handling support for fscrypt xiubli
2023-02-27 3:27 ` [PATCH v16 50/68] libceph: allow ceph_osdc_new_request to accept a multi-op read xiubli
2023-02-27 3:27 ` [PATCH v16 51/68] ceph: disable fallocate for encrypted inodes xiubli
2023-02-27 3:27 ` [PATCH v16 52/68] ceph: disable copy offload on " xiubli
2023-02-27 3:27 ` [PATCH v16 53/68] ceph: don't use special DIO path for " xiubli
2023-02-27 3:27 ` [PATCH v16 54/68] ceph: align data in pages in ceph_sync_write xiubli
2023-02-27 3:28 ` [PATCH v16 55/68] ceph: add read/modify/write to ceph_sync_write xiubli
2023-02-27 3:28 ` [PATCH v16 56/68] ceph: plumb in decryption during sync reads xiubli
2023-02-27 3:28 ` [PATCH v16 57/68] ceph: add fscrypt decryption support to ceph_netfs_issue_op xiubli
2023-02-27 3:28 ` [PATCH v16 58/68] ceph: set i_blkbits to crypto block size for encrypted inodes xiubli
2023-02-27 3:28 ` [PATCH v16 59/68] ceph: add encryption support to writepage xiubli
2023-02-27 3:28 ` [PATCH v16 60/68] ceph: fscrypt support for writepages xiubli
2023-02-27 3:28 ` [PATCH v16 61/68] ceph: invalidate pages when doing direct/sync writes xiubli
2023-02-27 3:28 ` [PATCH v16 62/68] ceph: add support for encrypted snapshot names xiubli
2023-02-27 3:28 ` [PATCH v16 63/68] ceph: add support for handling " xiubli
2023-02-27 3:28 ` [PATCH v16 64/68] ceph: update documentation regarding snapshot naming limitations xiubli
2023-02-27 3:28 ` [PATCH v16 65/68] ceph: prevent snapshots to be created in encrypted locked directories xiubli
2023-02-27 3:28 ` [PATCH v16 66/68] ceph: report STATX_ATTR_ENCRYPTED on encrypted inodes xiubli
2023-02-27 3:28 ` [PATCH v16 67/68] libceph: defer removing the req from osdc just after req->r_callback xiubli
2023-02-27 3:28 ` [PATCH v16 68/68] ceph: drop the messages from MDS when unmounting xiubli
2023-02-27 9:27 ` [PATCH v16 00/68] ceph+fscrypt: full support Luís Henriques
2023-02-27 9:58 ` Xiubo Li
2023-02-27 10:30 ` Luís Henriques
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230227032813.337906-1-xiubli@redhat.com \
--to=xiubli@redhat.com \
--cc=ceph-devel@vger.kernel.org \
--cc=idryomov@gmail.com \
--cc=jlayton@kernel.org \
--cc=lhenriques@suse.de \
--cc=mchangir@redhat.com \
--cc=vshankar@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).