a question about opening file

a question about opening file

[Chengguang Xu]

Hello,
 
I have a question about opening file of underlying filesystem in overlayfs,

why we use overlayfs' path(vfsmount/dentry) struct for underlying fs' file  

in ovl_open_realfile()?  Is it by design?


Thanks,
Chengguang

Re: a question about opening file

[Amir Goldstein]

On Thu, Nov 5, 2020 at 6:39 AM Chengguang Xu <cgxu519@mykernel.net> wrote:
>
> Hello,
>
> I have a question about opening file of underlying filesystem in overlayfs,
>
> why we use overlayfs' path(vfsmount/dentry) struct for underlying fs' file
>
> in ovl_open_realfile()?  Is it by design?

Sure. open_with_fake_path() is only used by overlayfs.

IIRC, one of the reasons was to display the user expected path in
/proc/<pid>/maps.
There may have been other reasons.

Thanks,
Amir.

>
>
> Thanks,
> Chengguang

Re: a question about opening file

[Chengguang Xu]

 ---- 在 星期四, 2020-11-05 16:07:26 Amir Goldstein <amir73il@gmail.com> 撰写 ----
 > On Thu, Nov 5, 2020 at 6:39 AM Chengguang Xu <cgxu519@mykernel.net> wrote:
 > >
 > > Hello,
 > >
 > > I have a question about opening file of underlying filesystem in overlayfs,
 > >
 > > why we use overlayfs' path(vfsmount/dentry) struct for underlying fs' file
 > >
 > > in ovl_open_realfile()?  Is it by design?
 > 
 > Sure. open_with_fake_path() is only used by overlayfs.
 > 
 > IIRC, one of the reasons was to display the user expected path in
 > /proc/<pid>/maps.
 > There may have been other reasons.
 > 
 
So if we do the mmap with overlayfs'  own page cache, then we don't have to
use pseudo path for the reason above, right?

Actually, the background is I'm trying to implement overlayfs' page cache for
fixing mmap rorw issue. The reason why asking this is I need to open a writeback
file which is used for syncing dirty data from overlayfs' own page cache to upper inode. 
However, if I use the pseudo path just like current opening behavior, the writeback
file will hold a reference of vfsmount of overlayfs and it will cause umount fail with -EBUSY.
So I want to open a writeback file with correct underlying path struct but not sure if
there is any unexpected side effect. Any suggestion?

Thanks,
Chengguang

Re: a question about opening file

[Miklos Szeredi]

On Thu, Nov 5, 2020 at 10:38 AM Chengguang Xu <cgxu519@mykernel.net> wrote:
>
>  ---- 在 星期四, 2020-11-05 16:07:26 Amir Goldstein <amir73il@gmail.com> 撰写 ----
>  > On Thu, Nov 5, 2020 at 6:39 AM Chengguang Xu <cgxu519@mykernel.net> wrote:
>  > >
>  > > Hello,
>  > >
>  > > I have a question about opening file of underlying filesystem in overlayfs,
>  > >
>  > > why we use overlayfs' path(vfsmount/dentry) struct for underlying fs' file
>  > >
>  > > in ovl_open_realfile()?  Is it by design?
>  >
>  > Sure. open_with_fake_path() is only used by overlayfs.
>  >
>  > IIRC, one of the reasons was to display the user expected path in
>  > /proc/<pid>/maps.
>  > There may have been other reasons.
>  >
>
> So if we do the mmap with overlayfs'  own page cache, then we don't have to
> use pseudo path for the reason above, right?
>
> Actually, the background is I'm trying to implement overlayfs' page cache for
> fixing mmap rorw issue. The reason why asking this is I need to open a writeback
> file which is used for syncing dirty data from overlayfs' own page cache to upper inode.
> However, if I use the pseudo path just like current opening behavior, the writeback
> file will hold a reference of vfsmount of overlayfs and it will cause umount fail with -EBUSY.
> So I want to open a writeback file with correct underlying path struct but not sure if
> there is any unexpected side effect. Any suggestion?

Should be no issue with plain dentry_open() for that purpose.  In fact
it would be really good to get rid of all that d_real*() mess
completely, but that seems some ways off.

Did you find the prototype we did with Amir a couple of years back?  I
can only find bits and pieces in my mailbox...

Thanks,
Miklos

Re: a question about opening file

[Chengguang Xu]

 ---- 在 星期四, 2020-11-05 17:57:15 Miklos Szeredi <miklos@szeredi.hu> 撰写 ----
 > On Thu, Nov 5, 2020 at 10:38 AM Chengguang Xu <cgxu519@mykernel.net> wrote:
 > >
 > >  ---- 在 星期四, 2020-11-05 16:07:26 Amir Goldstein <amir73il@gmail.com> 撰写 ----
 > >  > On Thu, Nov 5, 2020 at 6:39 AM Chengguang Xu <cgxu519@mykernel.net> wrote:
 > >  > >
 > >  > > Hello,
 > >  > >
 > >  > > I have a question about opening file of underlying filesystem in overlayfs,
 > >  > >
 > >  > > why we use overlayfs' path(vfsmount/dentry) struct for underlying fs' file
 > >  > >
 > >  > > in ovl_open_realfile()?  Is it by design?
 > >  >
 > >  > Sure. open_with_fake_path() is only used by overlayfs.
 > >  >
 > >  > IIRC, one of the reasons was to display the user expected path in
 > >  > /proc/<pid>/maps.
 > >  > There may have been other reasons.
 > >  >
 > >
 > > So if we do the mmap with overlayfs'  own page cache, then we don't have to
 > > use pseudo path for the reason above, right?
 > >
 > > Actually, the background is I'm trying to implement overlayfs' page cache for
 > > fixing mmap rorw issue. The reason why asking this is I need to open a writeback
 > > file which is used for syncing dirty data from overlayfs' own page cache to upper inode.
 > > However, if I use the pseudo path just like current opening behavior, the writeback
 > > file will hold a reference of vfsmount of overlayfs and it will cause umount fail with -EBUSY.
 > > So I want to open a writeback file with correct underlying path struct but not sure if
 > > there is any unexpected side effect. Any suggestion?
 > 
 > Should be no issue with plain dentry_open() for that purpose.  In fact
 > it would be really good to get rid of all that d_real*() mess
 > completely, but that seems some ways off.
 > 
 > Did you find the prototype we did with Amir a couple of years back?  I
 > can only find bits and pieces in my mailbox...
 > 

I searched in overlayfs mail list but unfortunately didn't  get useful info.
Seems Amir has a git tree for aops prototype but I'm not sure if that is the
prototype you mentioned.

https://github.com/amir73il/linux/commits/ovl-aops-wip

Hi Amir,

Do you know the prototype that Miklos mentioned above? Is that the
code in your ovl-aops-wip git tree?


Thanks,
Chengguang

Re: a question about opening file

[Amir Goldstein]

On Thu, Nov 5, 2020 at 1:39 PM Chengguang Xu <cgxu519@mykernel.net> wrote:
>
>  ---- 在 星期四, 2020-11-05 17:57:15 Miklos Szeredi <miklos@szeredi.hu> 撰写 ----
>  > On Thu, Nov 5, 2020 at 10:38 AM Chengguang Xu <cgxu519@mykernel.net> wrote:
>  > >
>  > >  ---- 在 星期四, 2020-11-05 16:07:26 Amir Goldstein <amir73il@gmail.com> 撰写 ----
>  > >  > On Thu, Nov 5, 2020 at 6:39 AM Chengguang Xu <cgxu519@mykernel.net> wrote:
>  > >  > >
>  > >  > > Hello,
>  > >  > >
>  > >  > > I have a question about opening file of underlying filesystem in overlayfs,
>  > >  > >
>  > >  > > why we use overlayfs' path(vfsmount/dentry) struct for underlying fs' file
>  > >  > >
>  > >  > > in ovl_open_realfile()?  Is it by design?
>  > >  >
>  > >  > Sure. open_with_fake_path() is only used by overlayfs.
>  > >  >
>  > >  > IIRC, one of the reasons was to display the user expected path in
>  > >  > /proc/<pid>/maps.
>  > >  > There may have been other reasons.
>  > >  >
>  > >
>  > > So if we do the mmap with overlayfs'  own page cache, then we don't have to
>  > > use pseudo path for the reason above, right?
>  > >
>  > > Actually, the background is I'm trying to implement overlayfs' page cache for
>  > > fixing mmap rorw issue. The reason why asking this is I need to open a writeback
>  > > file which is used for syncing dirty data from overlayfs' own page cache to upper inode.
>  > > However, if I use the pseudo path just like current opening behavior, the writeback
>  > > file will hold a reference of vfsmount of overlayfs and it will cause umount fail with -EBUSY.
>  > > So I want to open a writeback file with correct underlying path struct but not sure if
>  > > there is any unexpected side effect. Any suggestion?
>  >
>  > Should be no issue with plain dentry_open() for that purpose.  In fact
>  > it would be really good to get rid of all that d_real*() mess
>  > completely, but that seems some ways off.
>  >
>  > Did you find the prototype we did with Amir a couple of years back?  I
>  > can only find bits and pieces in my mailbox...
>  >
>
> I searched in overlayfs mail list but unfortunately didn't  get useful info.
> Seems Amir has a git tree for aops prototype but I'm not sure if that is the
> prototype you mentioned.
>
> https://github.com/amir73il/linux/commits/ovl-aops-wip
>
> Hi Amir,
>
> Do you know the prototype that Miklos mentioned above? Is that the
> code in your ovl-aops-wip git tree?
>

Yes it is.
The discussion over that branch and remaining TODO is on this thread [1].

CCing Ritesh who also expressed intention to start beating this
branch into shape.

If you go over the code/emails and still feel lost, I think it is best to have
a video call.

Thanks,
Amir.

[1] https://lore.kernel.org/linux-unionfs/CAJfpegsyA4SjmtAEpkMoKsvgmW0CiEwWEAbU7v3yJztLKmC0Eg@mail.gmail.com/

Re: a question about opening file

[Chengguang Xu]

 ---- 在 星期四, 2020-11-05 22:05:26 Amir Goldstein <amir73il@gmail.com> 撰写 ----
 > On Thu, Nov 5, 2020 at 1:39 PM Chengguang Xu <cgxu519@mykernel.net> wrote:
 > >
 > >  ---- 在 星期四, 2020-11-05 17:57:15 Miklos Szeredi <miklos@szeredi.hu> 撰写 ----
 > >  > On Thu, Nov 5, 2020 at 10:38 AM Chengguang Xu <cgxu519@mykernel.net> wrote:
 > >  > >
 > >  > >  ---- 在 星期四, 2020-11-05 16:07:26 Amir Goldstein <amir73il@gmail.com> 撰写 ----
 > >  > >  > On Thu, Nov 5, 2020 at 6:39 AM Chengguang Xu <cgxu519@mykernel.net> wrote:
 > >  > >  > >
 > >  > >  > > Hello,
 > >  > >  > >
 > >  > >  > > I have a question about opening file of underlying filesystem in overlayfs,
 > >  > >  > >
 > >  > >  > > why we use overlayfs' path(vfsmount/dentry) struct for underlying fs' file
 > >  > >  > >
 > >  > >  > > in ovl_open_realfile()?  Is it by design?
 > >  > >  >
 > >  > >  > Sure. open_with_fake_path() is only used by overlayfs.
 > >  > >  >
 > >  > >  > IIRC, one of the reasons was to display the user expected path in
 > >  > >  > /proc/<pid>/maps.
 > >  > >  > There may have been other reasons.
 > >  > >  >
 > >  > >
 > >  > > So if we do the mmap with overlayfs'  own page cache, then we don't have to
 > >  > > use pseudo path for the reason above, right?
 > >  > >
 > >  > > Actually, the background is I'm trying to implement overlayfs' page cache for
 > >  > > fixing mmap rorw issue. The reason why asking this is I need to open a writeback
 > >  > > file which is used for syncing dirty data from overlayfs' own page cache to upper inode.
 > >  > > However, if I use the pseudo path just like current opening behavior, the writeback
 > >  > > file will hold a reference of vfsmount of overlayfs and it will cause umount fail with -EBUSY.
 > >  > > So I want to open a writeback file with correct underlying path struct but not sure if
 > >  > > there is any unexpected side effect. Any suggestion?
 > >  >
 > >  > Should be no issue with plain dentry_open() for that purpose.  In fact
 > >  > it would be really good to get rid of all that d_real*() mess
 > >  > completely, but that seems some ways off.
 > >  >
 > >  > Did you find the prototype we did with Amir a couple of years back?  I
 > >  > can only find bits and pieces in my mailbox...
 > >  >
 > >
 > > I searched in overlayfs mail list but unfortunately didn't  get useful info.
 > > Seems Amir has a git tree for aops prototype but I'm not sure if that is the
 > > prototype you mentioned.
 > >
 > > https://github.com/amir73il/linux/commits/ovl-aops-wip
 > >
 > > Hi Amir,
 > >
 > > Do you know the prototype that Miklos mentioned above? Is that the
 > > code in your ovl-aops-wip git tree?
 > >
 > 
 > Yes it is.
 > The discussion over that branch and remaining TODO is on this thread [1].
 > 
 > CCing Ritesh who also expressed intention to start beating this
 > branch into shape.
 > 
 > If you go over the code/emails and still feel lost, I think it is best to have
 > a video call.
 > 
 > Thanks,
 > Amir.
 > 
 > [1] https://lore.kernel.org/linux-unionfs/CAJfpegsyA4SjmtAEpkMoKsvgmW0CiEwWEAbU7v3yJztLKmC0Eg@mail.gmail.com/
 > 

Hi Amir,

Thank you very much, it's very helpful, let me check the mail thread and the content in your branch.

Thanks,
Chengguang