From: Amir Goldstein <amir73il@gmail.com>
To: Miklos Szeredi <miklos@szeredi.hu>
Cc: Konstantin Khlebnikov <koct9i@gmail.com>,
Raphael Hertzog <hertzog@debian.org>,
Miklos Szeredi <mszeredi@redhat.com>,
"linux-unionfs@vger.kernel.org" <linux-unionfs@vger.kernel.org>,
Guillem Jover <guillem@debian.org>,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 3/3] ovl: redirect on rename-dir
Date: Sun, 13 Nov 2016 11:11:37 +0200 [thread overview]
Message-ID: <CAOQ4uxgmjXjpECMXcF3YVqocLh7siVtiSh-NiPqNExNG=wKp7Q@mail.gmail.com> (raw)
In-Reply-To: <CAOQ4uxgpsmO9bMU1Bu0aVLot+0zVpQ-OHytH017o2F8_1Hb0cQ@mail.gmail.com>
On Fri, Nov 11, 2016 at 2:42 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> On Fri, Nov 11, 2016 at 12:06 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>> On Fri, Nov 11, 2016 at 10:46 AM, Konstantin Khlebnikov
>> <koct9i@gmail.com> wrote:
>>> On Fri, Nov 11, 2016 at 1:56 AM, Amir Goldstein <amir73il@gmail.com> wrote:
>>>> On Mon, Nov 7, 2016 at 3:38 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>>>>> On Mon, Nov 7, 2016 at 12:08 PM, Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>>>>>> On Mon, Nov 7, 2016 at 1:04 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>>>>>>> On Mon, Nov 7, 2016 at 10:58 AM, Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>>>>>>>
>>>>>>>> I've stumbled on somehow related problem - concurrent copy-ups are
>>>>>>>> strictly serialized by rename locks.
>>>>>>>> Obviously, file copying could be done in parallel: locks are required
>>>>>>>> only for final rename.
>>>>>>>> Because of that overlay slower that aufs for some workloads.
>>>>>>>
>>>>>>> Easy to fix: for each copy up create a separate subdir of "work".
>>>>>>> Then the contention is only for the time of creating the subdir, which
>>>>>>> is very short.
>>>>>>
>>>>>> Yeah, but lock_rename() also takes per-sb s_vfs_rename_mutex (kludge by Al Viro)
>>>>>> I think proper synchronization for concurrent copy-up (for example
>>>>>> round flag on ovl_entry) and locking rename only for rename could be
>>>>>> better.
>>>>>
>>>>> Removing s_vfs_rename_mutex from copy-up path is something I have been
>>>>> pondering about.
>>>>> Assuming that I understand Al's comment above vfs_rename() correctly,
>>>>> the sole purpose of per-sb serialization is to prevent loop creations.
>>>>> However, how can one create a loop by moving a non-directory?
>>>>> So it looks like at least for the non-dir copy up case, a much finer grained
>>>>> lock is in order.
>>>>>
>>>>
>>>>
>>>> I posted patches to relax the s_vfs_rename_mutex for copy-up and
>>>> whiteout in some use cases.
>>>>
>>>> Konstantin,
>>>>
>>>> It would be useful to know if those patches help with your use case.
>>>>
>>>
>>> Well.. I think relaxing only s_vfs_rename_mutex wouldn't help much here.
>>> Copying is still serialized by i_mutex on workdir?
>>> Data copying should be done without rename locks at all.
>>
>> We do need something to prevent multiple copy-ups starting up in
>> parallel on the same file, though.
>>
>
> I guess an inode_lock on the copy-up victim should suffice?
Just to follow up on this hijacked thread.
I posted patches to lock the overlay inode of copied up file
and relaxed the lock_rename during data copy up.
Amir.
next prev parent reply other threads:[~2016-11-13 9:11 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-10-25 7:34 [PATCH 0/3] overlayfs: allow moving directory trees Miklos Szeredi
2016-10-25 7:34 ` [PATCH 1/3] ovl: check fs features Miklos Szeredi
2016-10-25 11:24 ` Amir Goldstein
2016-11-05 20:40 ` Amir Goldstein
2016-10-25 7:34 ` [PATCH 2/3] vfs: export vfs_path_lookup() Miklos Szeredi
2016-10-25 7:34 ` [PATCH 3/3] ovl: redirect on rename-dir Miklos Szeredi
2016-10-25 11:57 ` Raphael Hertzog
2016-10-26 11:12 ` Miklos Szeredi
2016-10-28 12:56 ` Raphael Hertzog
2016-10-28 12:59 ` Miklos Szeredi
2016-11-06 19:14 ` Konstantin Khlebnikov
2016-11-07 8:07 ` Miklos Szeredi
2016-11-07 9:58 ` Konstantin Khlebnikov
2016-11-07 10:04 ` Miklos Szeredi
2016-11-07 10:08 ` Konstantin Khlebnikov
2016-11-07 13:38 ` Amir Goldstein
2016-11-10 22:56 ` Amir Goldstein
2016-11-11 9:46 ` Konstantin Khlebnikov
2016-11-11 10:06 ` Miklos Szeredi
2016-11-11 12:42 ` Amir Goldstein
2016-11-13 9:11 ` Amir Goldstein [this message]
2016-11-07 11:03 ` Raphael Hertzog
2016-11-07 11:31 ` Konstantin Khlebnikov
2016-11-07 13:42 ` Raphael Hertzog
2016-11-10 22:39 ` Miklos Szeredi
2016-11-11 9:41 ` Konstantin Khlebnikov
2016-11-13 10:00 ` Amir Goldstein
2016-11-14 16:25 ` Amir Goldstein
2016-11-16 22:00 ` Miklos Szeredi
2016-11-18 15:37 ` Amir Goldstein
2016-11-20 11:39 ` Amir Goldstein
2016-11-21 9:54 ` Miklos Szeredi
2016-11-21 10:13 ` Amir Goldstein
2016-11-21 10:16 ` Miklos Szeredi
2016-11-22 13:42 ` Amir Goldstein
2016-10-25 12:49 ` Amir Goldstein
2016-10-26 11:26 ` Miklos Szeredi
2016-10-26 12:11 ` Amir Goldstein
2016-10-26 12:51 ` Miklos Szeredi
2016-10-26 19:56 ` Amir Goldstein
2016-10-28 16:15 ` Al Viro
2016-11-03 15:50 ` Miklos Szeredi
2016-11-04 9:29 ` Amir Goldstein
2016-11-04 13:48 ` Miklos Szeredi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAOQ4uxgmjXjpECMXcF3YVqocLh7siVtiSh-NiPqNExNG=wKp7Q@mail.gmail.com' \
--to=amir73il@gmail.com \
--cc=guillem@debian.org \
--cc=hertzog@debian.org \
--cc=koct9i@gmail.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-unionfs@vger.kernel.org \
--cc=miklos@szeredi.hu \
--cc=mszeredi@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).