From: Luis Henriques <lhenriques@suse.de>
To: Jeff Layton <jlayton@kernel.org>
Cc: ceph-devel@vger.kernel.org, idryomov@gmail.com,
xiubli@redhat.com, "Jozef Kováč" <kovac@firma.zoznam.sk>
Subject: Re: [PATCH v2] ceph: request Fw caps before updating the mtime in ceph_write_iter
Date: Thu, 12 Aug 2021 10:23:37 +0100 [thread overview]
Message-ID: <87h7fvqana.fsf@suse.de> (raw)
In-Reply-To: <20210811173738.29574-1-jlayton@kernel.org> (Jeff Layton's message of "Wed, 11 Aug 2021 13:37:38 -0400")
Jeff Layton <jlayton@kernel.org> writes:
> The current code will update the mtime and then try to get caps to
> handle the write. If we end up having to request caps from the MDS, then
> the mtime in the cap grant will clobber the updated mtime and it'll be
> lost.
>
> This is most noticable when two clients are alternately writing to the
> same file. Fw caps are continually being granted and revoked, and the
> mtime ends up stuck because the updated mtimes are always being
> overwritten with the old one.
>
> Fix this by changing the order of operations in ceph_write_iter. Get the
> caps much earlier, and only update the times afterward. Also, make sure
> we check the NEARFULL conditions before making any changes to the inode.
>
> URL: https://tracker.ceph.com/issues/46574
> Reported-by: Jozef Kováč <kovac@firma.zoznam.sk>
> Signed-off-by: Jeff Layton <jlayton@kernel.org>
> ---
> fs/ceph/file.c | 35 ++++++++++++++++++-----------------
> 1 file changed, 18 insertions(+), 17 deletions(-)
>
> v2: fix error handling -- make sure we release i_rwsem on error exit
>
> diff --git a/fs/ceph/file.c b/fs/ceph/file.c
> index d1755ac1d964..da856bd5eaa5 100644
> --- a/fs/ceph/file.c
> +++ b/fs/ceph/file.c
> @@ -1722,22 +1722,6 @@ static ssize_t ceph_write_iter(struct kiocb *iocb, struct iov_iter *from)
> goto out;
> }
>
> - err = file_remove_privs(file);
> - if (err)
> - goto out;
> -
> - err = file_update_time(file);
> - if (err)
> - goto out;
> -
> - inode_inc_iversion_raw(inode);
> -
> - if (ci->i_inline_version != CEPH_INLINE_NONE) {
> - err = ceph_uninline_data(file, NULL);
> - if (err < 0)
> - goto out;
> - }
> -
> down_read(&osdc->lock);
> map_flags = osdc->osdmap->flags;
> pool_flags = ceph_pg_pool_flags(osdc->osdmap, ci->i_layout.pool_id);
> @@ -1748,6 +1732,12 @@ static ssize_t ceph_write_iter(struct kiocb *iocb, struct iov_iter *from)
> goto out;
> }
>
> + if (ci->i_inline_version != CEPH_INLINE_NONE) {
> + err = ceph_uninline_data(file, NULL);
> + if (err < 0)
> + goto out;
> + }
> +
> dout("aio_write %p %llx.%llx %llu~%zd getting caps. i_size %llu\n",
> inode, ceph_vinop(inode), pos, count, i_size_read(inode));
> if (fi->fmode & CEPH_FILE_MODE_LAZY)
> @@ -1759,6 +1749,16 @@ static ssize_t ceph_write_iter(struct kiocb *iocb, struct iov_iter *from)
> if (err < 0)
> goto out;
>
> + err = file_remove_privs(file);
> + if (err)
> + goto out_caps;
> +
> + err = file_update_time(file);
> + if (err)
> + goto out_caps;
> +
> + inode_inc_iversion_raw(inode);
> +
> dout("aio_write %p %llx.%llx %llu~%zd got cap refs on %s\n",
> inode, ceph_vinop(inode), pos, count, ceph_cap_string(got));
>
> @@ -1822,7 +1822,6 @@ static ssize_t ceph_write_iter(struct kiocb *iocb, struct iov_iter *from)
> if (ceph_quota_is_max_bytes_approaching(inode, iocb->ki_pos))
> ceph_check_caps(ci, 0, NULL);
> }
> -
> dout("aio_write %p %llx.%llx %llu~%u dropping cap refs on %s\n",
> inode, ceph_vinop(inode), pos, (unsigned)count,
> ceph_cap_string(got));
> @@ -1842,6 +1841,8 @@ static ssize_t ceph_write_iter(struct kiocb *iocb, struct iov_iter *from)
> }
>
> goto out_unlocked;
> +out_caps:
> + ceph_put_cap_refs(ci, got);
> out:
> if (direct_lock)
> ceph_end_io_direct(inode);
> --
>
> 2.31.1
>
LGTM too!
Reviewed-by: Luis Henriques <lhenriques@suse.de>
Cheers,
--
Luis
prev parent reply other threads:[~2021-08-12 9:23 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-11 17:37 [PATCH v2] ceph: request Fw caps before updating the mtime in ceph_write_iter Jeff Layton
2021-08-12 2:51 ` Xiubo Li
2021-08-12 9:23 ` Luis Henriques [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87h7fvqana.fsf@suse.de \
--to=lhenriques@suse.de \
--cc=ceph-devel@vger.kernel.org \
--cc=idryomov@gmail.com \
--cc=jlayton@kernel.org \
--cc=kovac@firma.zoznam.sk \
--cc=xiubli@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).