[QUESTION] problem about origin xattr

[QUESTION] problem about origin xattr

[yangerkun]

Hi,

While thinking about "origin" xattr, i find a problem which retrieval 
methods show as below:

$mkdir lower lower1 lower2 upper work merge
$touch lower1/a
$mount -t overlay -olowerdir=lower:lower1,upperdir=upper,workdir=work 
merge merge
$echo abc > merge/a
$ls -li merge/a
Then, we can get the ino of file a equals to the ino of lower1/a.
$umount merge
$mount -t overlay -olowerdir=lower:lower2,upperdir=upper,workdir=work 
merge merge
$ls -li merge/a

The ino for file a is same as file lower1/a, while there is no file a in 
lower and lower2. Because ovl_get_origin will get the wrong origin.

Thanks,
yangerkun

Re: [QUESTION] problem about origin xattr

[Amir Goldstein]

On Wed, Jan 31, 2018 at 5:20 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> On Wed, Jan 31, 2018 at 03:57:12PM +0200, Amir Goldstein wrote:
>
> ORIGIN behavior is little unintuitive though. Despite the fact that file
> is not searchable through lower, it is visible through decoding of file
> handle and it is atleast non-intuitive.
>

Maybe not intuitive at first glance, but try again:

The only thing we *need* from underlying fs is to provide us with a unique
and persistent inode number we can use for the overlay object.

Even if the inode number we get from underlying fs is not in any of the
layers, it is still a  viable inode number we can use in overlay coupled
with overlay unique st_dev, to create a system wide unique st_dev;st_ino
tuple.

So I don't see any reason to fix this.
However, if one would want to fix this, that will require at least:
1. storing a 'connectable' non-dir file handle in ORIGIN xattr
2. but still use the non-connectable file handle for index
3. in ovl_acceptable(), check if non-dir is !IS_ROOT and validate
    it with is_subdir() same as done for directories

Cheers,
Amir.

>
>
>> Hi yangerkun,
>>
>> Replying from phone so sorry for excluding list.
>> Please add list back if replying.
>>
>> This behavior is expected and sort of documented in ovl_acceptable()
>>
>> The only property required from st_ino
>> Is that it is unique together with st_dev of overlay.
>>
>> Nothing in that example breaks this requirement.
>>
>> The only semi problem is that when the old origin will be unlinked st_ino
>> of overlay inode will change.
>> That is an acceptable behavior for offline changes in overlay layers.
>>
>> Cheers,
>> Amir.
>> On Jan 31, 2018 12:36 PM, "yangerkun" <yangerkun@huawei.com> wrote:
>>
>> Hi,
>>
>> While thinking about "origin" xattr, i find a problem which retrieval
>> methods show as below:
>>
>> $mkdir lower lower1 lower2 upper work merge
>> $touch lower1/a
>> $mount -t overlay -olowerdir=lower:lower1,upperdir=upper,workdir=work merge
>> merge
>> $echo abc > merge/a
>> $ls -li merge/a
>> Then, we can get the ino of file a equals to the ino of lower1/a.
>> $umount merge
>> $mount -t overlay -olowerdir=lower:lower2,upperdir=upper,workdir=work merge
>> merge
>> $ls -li merge/a
>>
>> The ino for file a is same as file lower1/a, while there is no file a in
>> lower and lower2. Because ovl_get_origin will get the wrong origin.
>>
>> Thanks,
>> yangerkun

Re: [QUESTION] problem about origin xattr

[Vivek Goyal]

On Wed, Jan 31, 2018 at 05:38:45PM +0200, Amir Goldstein wrote:
> On Wed, Jan 31, 2018 at 5:20 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> > On Wed, Jan 31, 2018 at 03:57:12PM +0200, Amir Goldstein wrote:
> >
> > ORIGIN behavior is little unintuitive though. Despite the fact that file
> > is not searchable through lower, it is visible through decoding of file
> > handle and it is atleast non-intuitive.
> >
> 
> Maybe not intuitive at first glance, but try again:
> 
> The only thing we *need* from underlying fs is to provide us with a unique
> and persistent inode number we can use for the overlay object.
> 
> Even if the inode number we get from underlying fs is not in any of the
> layers, it is still a  viable inode number we can use in overlay coupled
> with overlay unique st_dev, to create a system wide unique st_dev;st_ino
> tuple.

As long as we use only inode number, it probably is still fine.

But I look at ORIGIN as a generic infrastructure which other features can
make use of it. For example, metacopy is using it to copy up file later.
And there it will be non-intuitive that a file is not in any of the
lower, still ORIGIN was decoded and file was copied up. It can come
as a surprise to user. Atleast I was surprised when I ran into this
while testing the feature.

Vivek
> 
> So I don't see any reason to fix this.
> However, if one would want to fix this, that will require at least:
> 1. storing a 'connectable' non-dir file handle in ORIGIN xattr
> 2. but still use the non-connectable file handle for index
> 3. in ovl_acceptable(), check if non-dir is !IS_ROOT and validate
>     it with is_subdir() same as done for directories
> 
> Cheers,
> Amir.
> 
> >
> >
> >> Hi yangerkun,
> >>
> >> Replying from phone so sorry for excluding list.
> >> Please add list back if replying.
> >>
> >> This behavior is expected and sort of documented in ovl_acceptable()
> >>
> >> The only property required from st_ino
> >> Is that it is unique together with st_dev of overlay.
> >>
> >> Nothing in that example breaks this requirement.
> >>
> >> The only semi problem is that when the old origin will be unlinked st_ino
> >> of overlay inode will change.
> >> That is an acceptable behavior for offline changes in overlay layers.
> >>
> >> Cheers,
> >> Amir.
> >> On Jan 31, 2018 12:36 PM, "yangerkun" <yangerkun@huawei.com> wrote:
> >>
> >> Hi,
> >>
> >> While thinking about "origin" xattr, i find a problem which retrieval
> >> methods show as below:
> >>
> >> $mkdir lower lower1 lower2 upper work merge
> >> $touch lower1/a
> >> $mount -t overlay -olowerdir=lower:lower1,upperdir=upper,workdir=work merge
> >> merge
> >> $echo abc > merge/a
> >> $ls -li merge/a
> >> Then, we can get the ino of file a equals to the ino of lower1/a.
> >> $umount merge
> >> $mount -t overlay -olowerdir=lower:lower2,upperdir=upper,workdir=work merge
> >> merge
> >> $ls -li merge/a
> >>
> >> The ino for file a is same as file lower1/a, while there is no file a in
> >> lower and lower2. Because ovl_get_origin will get the wrong origin.
> >>
> >> Thanks,
> >> yangerkun

Re: [QUESTION] problem about origin xattr

[Amir Goldstein]

On Wed, Jan 31, 2018 at 5:46 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> On Wed, Jan 31, 2018 at 05:38:45PM +0200, Amir Goldstein wrote:
>> On Wed, Jan 31, 2018 at 5:20 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>> > On Wed, Jan 31, 2018 at 03:57:12PM +0200, Amir Goldstein wrote:
>> >
>> > ORIGIN behavior is little unintuitive though. Despite the fact that file
>> > is not searchable through lower, it is visible through decoding of file
>> > handle and it is atleast non-intuitive.
>> >
>>
>> Maybe not intuitive at first glance, but try again:
>>
>> The only thing we *need* from underlying fs is to provide us with a unique
>> and persistent inode number we can use for the overlay object.
>>
>> Even if the inode number we get from underlying fs is not in any of the
>> layers, it is still a  viable inode number we can use in overlay coupled
>> with overlay unique st_dev, to create a system wide unique st_dev;st_ino
>> tuple.
>
> As long as we use only inode number, it probably is still fine.
>
> But I look at ORIGIN as a generic infrastructure which other features can
> make use of it. For example, metacopy is using it to copy up file later.
> And there it will be non-intuitive that a file is not in any of the
> lower, still ORIGIN was decoded and file was copied up. It can come
> as a surprise to user. Atleast I was surprised when I ran into this
> while testing the feature.
>

In context of metacopy it is indeed strange and you may need to address
it somehow. In the end I see it as a security concern that can be addressed
the same way as redirect_dir=nofollow.
nfsd has a similar security concern for non-dir file handles. an nfs clients can
get hold of a file that is no longer under the exported share or that is under a
directory with reduces permissions. This is the reason for the 'subtree_check'
nfs export configuration which enforces 'connectable' non-dir file handles
and does ancestry check by nfsd_acceptable().

So down the road, you may want to implement metacopy=subtree_check
(or something shorter) to store connectable ORIGIN xattr and verify
subtree when checking origin chain.

Cheers,
Amir.


>>
>> So I don't see any reason to fix this.
>> However, if one would want to fix this, that will require at least:
>> 1. storing a 'connectable' non-dir file handle in ORIGIN xattr
>> 2. but still use the non-connectable file handle for index
>> 3. in ovl_acceptable(), check if non-dir is !IS_ROOT and validate
>>     it with is_subdir() same as done for directories
>>
>> Cheers,
>> Amir.
>>
>> >
>> >
>> >> Hi yangerkun,
>> >>
>> >> Replying from phone so sorry for excluding list.
>> >> Please add list back if replying.
>> >>
>> >> This behavior is expected and sort of documented in ovl_acceptable()
>> >>
>> >> The only property required from st_ino
>> >> Is that it is unique together with st_dev of overlay.
>> >>
>> >> Nothing in that example breaks this requirement.
>> >>
>> >> The only semi problem is that when the old origin will be unlinked st_ino
>> >> of overlay inode will change.
>> >> That is an acceptable behavior for offline changes in overlay layers.
>> >>
>> >> Cheers,
>> >> Amir.
>> >> On Jan 31, 2018 12:36 PM, "yangerkun" <yangerkun@huawei.com> wrote:
>> >>
>> >> Hi,
>> >>
>> >> While thinking about "origin" xattr, i find a problem which retrieval
>> >> methods show as below:
>> >>
>> >> $mkdir lower lower1 lower2 upper work merge
>> >> $touch lower1/a
>> >> $mount -t overlay -olowerdir=lower:lower1,upperdir=upper,workdir=work merge
>> >> merge
>> >> $echo abc > merge/a
>> >> $ls -li merge/a
>> >> Then, we can get the ino of file a equals to the ino of lower1/a.
>> >> $umount merge
>> >> $mount -t overlay -olowerdir=lower:lower2,upperdir=upper,workdir=work merge
>> >> merge
>> >> $ls -li merge/a
>> >>
>> >> The ino for file a is same as file lower1/a, while there is no file a in
>> >> lower and lower2. Because ovl_get_origin will get the wrong origin.
>> >>
>> >> Thanks,
>> >> yangerkun

Re: [QUESTION] problem about origin xattr

[Miklos Szeredi]

On Wed, Jan 31, 2018 at 4:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> On Wed, Jan 31, 2018 at 5:46 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>> On Wed, Jan 31, 2018 at 05:38:45PM +0200, Amir Goldstein wrote:
>>> On Wed, Jan 31, 2018 at 5:20 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>>> > On Wed, Jan 31, 2018 at 03:57:12PM +0200, Amir Goldstein wrote:
>>> >
>>> > ORIGIN behavior is little unintuitive though. Despite the fact that file
>>> > is not searchable through lower, it is visible through decoding of file
>>> > handle and it is atleast non-intuitive.
>>> >
>>>
>>> Maybe not intuitive at first glance, but try again:
>>>
>>> The only thing we *need* from underlying fs is to provide us with a unique
>>> and persistent inode number we can use for the overlay object.
>>>
>>> Even if the inode number we get from underlying fs is not in any of the
>>> layers, it is still a  viable inode number we can use in overlay coupled
>>> with overlay unique st_dev, to create a system wide unique st_dev;st_ino
>>> tuple.
>>
>> As long as we use only inode number, it probably is still fine.
>>
>> But I look at ORIGIN as a generic infrastructure which other features can
>> make use of it. For example, metacopy is using it to copy up file later.
>> And there it will be non-intuitive that a file is not in any of the
>> lower, still ORIGIN was decoded and file was copied up. It can come
>> as a surprise to user. Atleast I was surprised when I ran into this
>> while testing the feature.

How about using REDIRECT for metacopy origin?   Keeping ORIGIN only
for inode, also meaning ORIGIN is only ever used on upper layer, never
on middle layers.

Thanks,
Miklos

Re: [QUESTION] problem about origin xattr

[Vivek Goyal]

On Wed, Jan 31, 2018 at 05:10:28PM +0100, Miklos Szeredi wrote:
> On Wed, Jan 31, 2018 at 4:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> > On Wed, Jan 31, 2018 at 5:46 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> >> On Wed, Jan 31, 2018 at 05:38:45PM +0200, Amir Goldstein wrote:
> >>> On Wed, Jan 31, 2018 at 5:20 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> >>> > On Wed, Jan 31, 2018 at 03:57:12PM +0200, Amir Goldstein wrote:
> >>> >
> >>> > ORIGIN behavior is little unintuitive though. Despite the fact that file
> >>> > is not searchable through lower, it is visible through decoding of file
> >>> > handle and it is atleast non-intuitive.
> >>> >
> >>>
> >>> Maybe not intuitive at first glance, but try again:
> >>>
> >>> The only thing we *need* from underlying fs is to provide us with a unique
> >>> and persistent inode number we can use for the overlay object.
> >>>
> >>> Even if the inode number we get from underlying fs is not in any of the
> >>> layers, it is still a  viable inode number we can use in overlay coupled
> >>> with overlay unique st_dev, to create a system wide unique st_dev;st_ino
> >>> tuple.
> >>
> >> As long as we use only inode number, it probably is still fine.
> >>
> >> But I look at ORIGIN as a generic infrastructure which other features can
> >> make use of it. For example, metacopy is using it to copy up file later.
> >> And there it will be non-intuitive that a file is not in any of the
> >> lower, still ORIGIN was decoded and file was copied up. It can come
> >> as a surprise to user. Atleast I was surprised when I ran into this
> >> while testing the feature.
> 
> How about using REDIRECT for metacopy origin?   Keeping ORIGIN only
> for inode, also meaning ORIGIN is only ever used on upper layer, never
> on middle layers.

Hi Miklos,

Trying to understand it better. So proposal seems to be that when a file
is copied up metacopy only, we store both REDIRECT and ORIGIN in upper
inode. When traversing metacopy inode chain, use ORIGIN info on upper
inode and REDIRECT info on lower/midlayer metacopy inode.

I am assuming that this is to handle the use case of tar of upper layer
and untaring it as lower layer.

One of the concerns Amir had raised with usage of REDIRECT was that it
will be significantly slower as comapred to decoding ORIGIN. So by using
ORIGIN on upper, we are trying to mitigate it up to some extent? We will
still pay the cost of decoding REDIECT in midlayer.

Am I understanding it right.

Vivek

Re: [QUESTION] problem about origin xattr

[Miklos Szeredi]

On Wed, Jan 31, 2018 at 5:55 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> On Wed, Jan 31, 2018 at 05:10:28PM +0100, Miklos Szeredi wrote:
>> On Wed, Jan 31, 2018 at 4:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>> > On Wed, Jan 31, 2018 at 5:46 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>> >> On Wed, Jan 31, 2018 at 05:38:45PM +0200, Amir Goldstein wrote:
>> >>> On Wed, Jan 31, 2018 at 5:20 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>> >>> > On Wed, Jan 31, 2018 at 03:57:12PM +0200, Amir Goldstein wrote:
>> >>> >
>> >>> > ORIGIN behavior is little unintuitive though. Despite the fact that file
>> >>> > is not searchable through lower, it is visible through decoding of file
>> >>> > handle and it is atleast non-intuitive.
>> >>> >
>> >>>
>> >>> Maybe not intuitive at first glance, but try again:
>> >>>
>> >>> The only thing we *need* from underlying fs is to provide us with a unique
>> >>> and persistent inode number we can use for the overlay object.
>> >>>
>> >>> Even if the inode number we get from underlying fs is not in any of the
>> >>> layers, it is still a  viable inode number we can use in overlay coupled
>> >>> with overlay unique st_dev, to create a system wide unique st_dev;st_ino
>> >>> tuple.
>> >>
>> >> As long as we use only inode number, it probably is still fine.
>> >>
>> >> But I look at ORIGIN as a generic infrastructure which other features can
>> >> make use of it. For example, metacopy is using it to copy up file later.
>> >> And there it will be non-intuitive that a file is not in any of the
>> >> lower, still ORIGIN was decoded and file was copied up. It can come
>> >> as a surprise to user. Atleast I was surprised when I ran into this
>> >> while testing the feature.
>>
>> How about using REDIRECT for metacopy origin?   Keeping ORIGIN only
>> for inode, also meaning ORIGIN is only ever used on upper layer, never
>> on middle layers.
>
> Hi Miklos,
>
> Trying to understand it better. So proposal seems to be that when a file
> is copied up metacopy only, we store both REDIRECT and ORIGIN in upper
> inode. When traversing metacopy inode chain, use ORIGIN info on upper
> inode and REDIRECT info on lower/midlayer metacopy inode.
>
> I am assuming that this is to handle the use case of tar of upper layer
> and untaring it as lower layer.
>
> One of the concerns Amir had raised with usage of REDIRECT was that it
> will be significantly slower as comapred to decoding ORIGIN. So by using
> ORIGIN on upper, we are trying to mitigate it up to some extent? We will
> still pay the cost of decoding REDIECT in midlayer.
>
> Am I understanding it right.

Like directories, we'd only need to set REDIRECT on rename.

So when file has METACOPY, but not REDIRECT, we just fall through to
next layer below one we are currently operating on.  If we find
METACOPY there, we just continue looking until we find a file
containing the data.

When we rename or hardlink a file with METACOPY, we add REDIRECT.

If file has METACOPY and REDIRECT, we follow REDIRECT to find a file
on the next level and keep iterating until we have the one with the
data.

ORIGIN would not be used in this case.  We might be able to use ORIGIN
for some kind of verification, like we do for directories.   Amir has
a better idea, I think.

Another way to think about it is: METACOPY is the opposite of OPAQUE.
For directories the default is "metacopy" and contents are merged.
For files the default is "opaque" and content is not merged.  METACOPY
turns that around and enables "merging" of data from a lower layer.
I could even imagine real merging of data, but it's unlikely to be
worth the effort, clone is much better for that; METACOPY is just a
very restricted (and so much simpler) way of merging data.

Thanks,
Miklos

>
> Vivek

Re: [QUESTION] problem about origin xattr

[Vivek Goyal]

On Wed, Jan 31, 2018 at 07:08:18PM +0100, Miklos Szeredi wrote:
> On Wed, Jan 31, 2018 at 5:55 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> > On Wed, Jan 31, 2018 at 05:10:28PM +0100, Miklos Szeredi wrote:
> >> On Wed, Jan 31, 2018 at 4:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> >> > On Wed, Jan 31, 2018 at 5:46 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> >> >> On Wed, Jan 31, 2018 at 05:38:45PM +0200, Amir Goldstein wrote:
> >> >>> On Wed, Jan 31, 2018 at 5:20 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> >> >>> > On Wed, Jan 31, 2018 at 03:57:12PM +0200, Amir Goldstein wrote:
> >> >>> >
> >> >>> > ORIGIN behavior is little unintuitive though. Despite the fact that file
> >> >>> > is not searchable through lower, it is visible through decoding of file
> >> >>> > handle and it is atleast non-intuitive.
> >> >>> >
> >> >>>
> >> >>> Maybe not intuitive at first glance, but try again:
> >> >>>
> >> >>> The only thing we *need* from underlying fs is to provide us with a unique
> >> >>> and persistent inode number we can use for the overlay object.
> >> >>>
> >> >>> Even if the inode number we get from underlying fs is not in any of the
> >> >>> layers, it is still a  viable inode number we can use in overlay coupled
> >> >>> with overlay unique st_dev, to create a system wide unique st_dev;st_ino
> >> >>> tuple.
> >> >>
> >> >> As long as we use only inode number, it probably is still fine.
> >> >>
> >> >> But I look at ORIGIN as a generic infrastructure which other features can
> >> >> make use of it. For example, metacopy is using it to copy up file later.
> >> >> And there it will be non-intuitive that a file is not in any of the
> >> >> lower, still ORIGIN was decoded and file was copied up. It can come
> >> >> as a surprise to user. Atleast I was surprised when I ran into this
> >> >> while testing the feature.
> >>
> >> How about using REDIRECT for metacopy origin?   Keeping ORIGIN only
> >> for inode, also meaning ORIGIN is only ever used on upper layer, never
> >> on middle layers.
> >
> > Hi Miklos,
> >
> > Trying to understand it better. So proposal seems to be that when a file
> > is copied up metacopy only, we store both REDIRECT and ORIGIN in upper
> > inode. When traversing metacopy inode chain, use ORIGIN info on upper
> > inode and REDIRECT info on lower/midlayer metacopy inode.
> >
> > I am assuming that this is to handle the use case of tar of upper layer
> > and untaring it as lower layer.
> >
> > One of the concerns Amir had raised with usage of REDIRECT was that it
> > will be significantly slower as comapred to decoding ORIGIN. So by using
> > ORIGIN on upper, we are trying to mitigate it up to some extent? We will
> > still pay the cost of decoding REDIECT in midlayer.
> >
> > Am I understanding it right.
> 
> Like directories, we'd only need to set REDIRECT on rename.
> 
> So when file has METACOPY, but not REDIRECT, we just fall through to
> next layer below one we are currently operating on.  If we find
> METACOPY there, we just continue looking until we find a file
> containing the data.
> 
> When we rename or hardlink a file with METACOPY, we add REDIRECT.
> 
> If file has METACOPY and REDIRECT, we follow REDIRECT to find a file
> on the next level and keep iterating until we have the one with the
> data.
> 
> ORIGIN would not be used in this case.  We might be able to use ORIGIN
> for some kind of verification, like we do for directories.   Amir has
> a better idea, I think.
> 
> Another way to think about it is: METACOPY is the opposite of OPAQUE.
> For directories the default is "metacopy" and contents are merged.
> For files the default is "opaque" and content is not merged.  METACOPY
> turns that around and enables "merging" of data from a lower layer.
> I could even imagine real merging of data, but it's unlikely to be
> worth the effort, clone is much better for that; METACOPY is just a
> very restricted (and so much simpler) way of merging data.

Ok, thanks. I am beginning to understand it better now.

First implementaion issue which comes to my mind is that stack[0] location
conflict. Right now this is taken up by dentry which was obtained by following
ORIGIN from upper and acts as copy up origin.

May be I should continue to use ORIGIN for upper dentry and when stack[0] is
filled and if its metacopy, then continue to find data dentry using either
REDIRECT or using same name and store in stack[1].

Vivek

Re: [QUESTION] problem about origin xattr

[Amir Goldstein]

On Wed, Jan 31, 2018 at 9:05 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> On Wed, Jan 31, 2018 at 07:08:18PM +0100, Miklos Szeredi wrote:
>> On Wed, Jan 31, 2018 at 5:55 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>> > On Wed, Jan 31, 2018 at 05:10:28PM +0100, Miklos Szeredi wrote:
>> >> On Wed, Jan 31, 2018 at 4:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>> >> > On Wed, Jan 31, 2018 at 5:46 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>> >> >> On Wed, Jan 31, 2018 at 05:38:45PM +0200, Amir Goldstein wrote:
>> >> >>> On Wed, Jan 31, 2018 at 5:20 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>> >> >>> > On Wed, Jan 31, 2018 at 03:57:12PM +0200, Amir Goldstein wrote:
>> >> >>> >
>> >> >>> > ORIGIN behavior is little unintuitive though. Despite the fact that file
>> >> >>> > is not searchable through lower, it is visible through decoding of file
>> >> >>> > handle and it is atleast non-intuitive.
>> >> >>> >
>> >> >>>
>> >> >>> Maybe not intuitive at first glance, but try again:
>> >> >>>
>> >> >>> The only thing we *need* from underlying fs is to provide us with a unique
>> >> >>> and persistent inode number we can use for the overlay object.
>> >> >>>
>> >> >>> Even if the inode number we get from underlying fs is not in any of the
>> >> >>> layers, it is still a  viable inode number we can use in overlay coupled
>> >> >>> with overlay unique st_dev, to create a system wide unique st_dev;st_ino
>> >> >>> tuple.
>> >> >>
>> >> >> As long as we use only inode number, it probably is still fine.
>> >> >>
>> >> >> But I look at ORIGIN as a generic infrastructure which other features can
>> >> >> make use of it. For example, metacopy is using it to copy up file later.
>> >> >> And there it will be non-intuitive that a file is not in any of the
>> >> >> lower, still ORIGIN was decoded and file was copied up. It can come
>> >> >> as a surprise to user. Atleast I was surprised when I ran into this
>> >> >> while testing the feature.
>> >>
>> >> How about using REDIRECT for metacopy origin?   Keeping ORIGIN only
>> >> for inode, also meaning ORIGIN is only ever used on upper layer, never
>> >> on middle layers.
>> >
>> > Hi Miklos,
>> >
>> > Trying to understand it better. So proposal seems to be that when a file
>> > is copied up metacopy only, we store both REDIRECT and ORIGIN in upper
>> > inode. When traversing metacopy inode chain, use ORIGIN info on upper
>> > inode and REDIRECT info on lower/midlayer metacopy inode.
>> >
>> > I am assuming that this is to handle the use case of tar of upper layer
>> > and untaring it as lower layer.
>> >
>> > One of the concerns Amir had raised with usage of REDIRECT was that it
>> > will be significantly slower as comapred to decoding ORIGIN. So by using
>> > ORIGIN on upper, we are trying to mitigate it up to some extent? We will
>> > still pay the cost of decoding REDIECT in midlayer.
>> >
>> > Am I understanding it right.
>>
>> Like directories, we'd only need to set REDIRECT on rename.
>>
>> So when file has METACOPY, but not REDIRECT, we just fall through to
>> next layer below one we are currently operating on.  If we find
>> METACOPY there, we just continue looking until we find a file
>> containing the data.
>>
>> When we rename or hardlink a file with METACOPY, we add REDIRECT.
>>
>> If file has METACOPY and REDIRECT, we follow REDIRECT to find a file
>> on the next level and keep iterating until we have the one with the
>> data.
>>
>> ORIGIN would not be used in this case.  We might be able to use ORIGIN
>> for some kind of verification, like we do for directories.   Amir has
>> a better idea, I think.
>>
>> Another way to think about it is: METACOPY is the opposite of OPAQUE.
>> For directories the default is "metacopy" and contents are merged.
>> For files the default is "opaque" and content is not merged.  METACOPY
>> turns that around and enables "merging" of data from a lower layer.
>> I could even imagine real merging of data, but it's unlikely to be
>> worth the effort, clone is much better for that; METACOPY is just a
>> very restricted (and so much simpler) way of merging data.
>
> Ok, thanks. I am beginning to understand it better now.
>
> First implementaion issue which comes to my mind is that stack[0] location
> conflict. Right now this is taken up by dentry which was obtained by following
> ORIGIN from upper and acts as copy up origin.
>
> May be I should continue to use ORIGIN for upper dentry and when stack[0] is
> filled and if its metacopy, then continue to find data dentry using either
> REDIRECT or using same name and store in stack[1].
>

Question: don't you think it would be beneficial to get metacopy working and
tested only from upper and without taking security considerations into the mix
for first version?
Do you know there is a real use case for middle layer metacopy and chaining
and all that Jazz?
When you first presented metacopy it sounded like you have a very solid use
case (chown -R). Does your specific use case extend to middle layers?
Is metacopy valueable enough without middle layers following?
Heck, AFAIK, container runtime doesn't even know how to deal with redirect
yet when committing an upper layer to an image. right?

Just wondering...

Amir.

Re: [QUESTION] problem about origin xattr

[Vivek Goyal]

On Wed, Jan 31, 2018 at 09:59:07PM +0200, Amir Goldstein wrote:

[..]
> >> >> >> As long as we use only inode number, it probably is still fine.
> >> >> >>
> >> >> >> But I look at ORIGIN as a generic infrastructure which other features can
> >> >> >> make use of it. For example, metacopy is using it to copy up file later.
> >> >> >> And there it will be non-intuitive that a file is not in any of the
> >> >> >> lower, still ORIGIN was decoded and file was copied up. It can come
> >> >> >> as a surprise to user. Atleast I was surprised when I ran into this
> >> >> >> while testing the feature.
> >> >>
> >> >> How about using REDIRECT for metacopy origin?   Keeping ORIGIN only
> >> >> for inode, also meaning ORIGIN is only ever used on upper layer, never
> >> >> on middle layers.
> >> >
> >> > Hi Miklos,
> >> >
> >> > Trying to understand it better. So proposal seems to be that when a file
> >> > is copied up metacopy only, we store both REDIRECT and ORIGIN in upper
> >> > inode. When traversing metacopy inode chain, use ORIGIN info on upper
> >> > inode and REDIRECT info on lower/midlayer metacopy inode.
> >> >
> >> > I am assuming that this is to handle the use case of tar of upper layer
> >> > and untaring it as lower layer.
> >> >
> >> > One of the concerns Amir had raised with usage of REDIRECT was that it
> >> > will be significantly slower as comapred to decoding ORIGIN. So by using
> >> > ORIGIN on upper, we are trying to mitigate it up to some extent? We will
> >> > still pay the cost of decoding REDIECT in midlayer.
> >> >
> >> > Am I understanding it right.
> >>
> >> Like directories, we'd only need to set REDIRECT on rename.
> >>
> >> So when file has METACOPY, but not REDIRECT, we just fall through to
> >> next layer below one we are currently operating on.  If we find
> >> METACOPY there, we just continue looking until we find a file
> >> containing the data.
> >>
> >> When we rename or hardlink a file with METACOPY, we add REDIRECT.
> >>
> >> If file has METACOPY and REDIRECT, we follow REDIRECT to find a file
> >> on the next level and keep iterating until we have the one with the
> >> data.
> >>
> >> ORIGIN would not be used in this case.  We might be able to use ORIGIN
> >> for some kind of verification, like we do for directories.   Amir has
> >> a better idea, I think.
> >>
> >> Another way to think about it is: METACOPY is the opposite of OPAQUE.
> >> For directories the default is "metacopy" and contents are merged.
> >> For files the default is "opaque" and content is not merged.  METACOPY
> >> turns that around and enables "merging" of data from a lower layer.
> >> I could even imagine real merging of data, but it's unlikely to be
> >> worth the effort, clone is much better for that; METACOPY is just a
> >> very restricted (and so much simpler) way of merging data.
> >
> > Ok, thanks. I am beginning to understand it better now.
> >
> > First implementaion issue which comes to my mind is that stack[0] location
> > conflict. Right now this is taken up by dentry which was obtained by following
> > ORIGIN from upper and acts as copy up origin.
> >
> > May be I should continue to use ORIGIN for upper dentry and when stack[0] is
> > filled and if its metacopy, then continue to find data dentry using either
> > REDIRECT or using same name and store in stack[1].
> >
> 
> Question: don't you think it would be beneficial to get metacopy working and
> tested only from upper and without taking security considerations into the mix
> for first version?

metacopy is working even now. I am posting new patches because there are
suggestions after posting patches and I try to take care of these.

> Do you know there is a real use case for middle layer metacopy and chaining
> and all that Jazz?

You asked for support of mid layer support in V9. So I did it.

https://www.spinics.net/lists/linux-unionfs/msg03712.html

> When you first presented metacopy it sounded like you have a very solid use
> case (chown -R). Does your specific use case extend to middle layers?

I thought about it later and I think docker will probably need mid layer
support. Reason being, that they probably will do chown and use that
chowned directory as lower layer for container so that they can later
do the diff w.r.t chowned copy and figure out what changes container
did. If we do chown on upper and let container use it as upper, then it
will appear that whole image has been changed by container.

So I feel mid layer support is important for proper integration of
this feature.

> Is metacopy valueable enough without middle layers following?
> Heck, AFAIK, container runtime doesn't even know how to deal with redirect
> yet when committing an upper layer to an image. right?

You probably are right. And they probably will fall back to native diff
interface when metacopy feature is on. But even in that case, they will
need to figure out what exactly container has changed w.r.t chowned
copy and that means chowned copy has to be the lower layer and that
means metacopy in mid layer support will be needed.

If we can teach them to store REDIRECT xattr, their commit operation will
become faster.

> 
> Just wondering...

I am just trying to figure out a point where you and miklos are happy
with the design and patches. Mid layer support seems to be important.

I get a feeling that miklos is still not entirely convinced about the
usage of ORIGIN to get to follow metacopy chain and he still somehow
wants to see making use of REDIRECT when need be. 

ORIGIN vs REDIRECT seems to be the only major sticking point w.r.t
these patches at this point of time. As long as you and miklos agree
on that semantics, things will be fine.

Vivek
> 
> Amir.

Re: [QUESTION] problem about origin xattr

[Miklos Szeredi]

On Wed, Jan 31, 2018 at 9:34 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> On Wed, Jan 31, 2018 at 09:59:07PM +0200, Amir Goldstein wrote:
>
> [..]
>> >> >> >> As long as we use only inode number, it probably is still fine.
>> >> >> >>
>> >> >> >> But I look at ORIGIN as a generic infrastructure which other features can
>> >> >> >> make use of it. For example, metacopy is using it to copy up file later.
>> >> >> >> And there it will be non-intuitive that a file is not in any of the
>> >> >> >> lower, still ORIGIN was decoded and file was copied up. It can come
>> >> >> >> as a surprise to user. Atleast I was surprised when I ran into this
>> >> >> >> while testing the feature.
>> >> >>
>> >> >> How about using REDIRECT for metacopy origin?   Keeping ORIGIN only
>> >> >> for inode, also meaning ORIGIN is only ever used on upper layer, never
>> >> >> on middle layers.
>> >> >
>> >> > Hi Miklos,
>> >> >
>> >> > Trying to understand it better. So proposal seems to be that when a file
>> >> > is copied up metacopy only, we store both REDIRECT and ORIGIN in upper
>> >> > inode. When traversing metacopy inode chain, use ORIGIN info on upper
>> >> > inode and REDIRECT info on lower/midlayer metacopy inode.
>> >> >
>> >> > I am assuming that this is to handle the use case of tar of upper layer
>> >> > and untaring it as lower layer.
>> >> >
>> >> > One of the concerns Amir had raised with usage of REDIRECT was that it
>> >> > will be significantly slower as comapred to decoding ORIGIN. So by using
>> >> > ORIGIN on upper, we are trying to mitigate it up to some extent? We will
>> >> > still pay the cost of decoding REDIECT in midlayer.
>> >> >
>> >> > Am I understanding it right.
>> >>
>> >> Like directories, we'd only need to set REDIRECT on rename.
>> >>
>> >> So when file has METACOPY, but not REDIRECT, we just fall through to
>> >> next layer below one we are currently operating on.  If we find
>> >> METACOPY there, we just continue looking until we find a file
>> >> containing the data.
>> >>
>> >> When we rename or hardlink a file with METACOPY, we add REDIRECT.
>> >>
>> >> If file has METACOPY and REDIRECT, we follow REDIRECT to find a file
>> >> on the next level and keep iterating until we have the one with the
>> >> data.
>> >>
>> >> ORIGIN would not be used in this case.  We might be able to use ORIGIN
>> >> for some kind of verification, like we do for directories.   Amir has
>> >> a better idea, I think.
>> >>
>> >> Another way to think about it is: METACOPY is the opposite of OPAQUE.
>> >> For directories the default is "metacopy" and contents are merged.
>> >> For files the default is "opaque" and content is not merged.  METACOPY
>> >> turns that around and enables "merging" of data from a lower layer.
>> >> I could even imagine real merging of data, but it's unlikely to be
>> >> worth the effort, clone is much better for that; METACOPY is just a
>> >> very restricted (and so much simpler) way of merging data.
>> >
>> > Ok, thanks. I am beginning to understand it better now.
>> >
>> > First implementaion issue which comes to my mind is that stack[0] location
>> > conflict. Right now this is taken up by dentry which was obtained by following
>> > ORIGIN from upper and acts as copy up origin.
>> >
>> > May be I should continue to use ORIGIN for upper dentry and when stack[0] is
>> > filled and if its metacopy, then continue to find data dentry using either
>> > REDIRECT or using same name and store in stack[1].
>> >
>>
>> Question: don't you think it would be beneficial to get metacopy working and
>> tested only from upper and without taking security considerations into the mix
>> for first version?
>
> metacopy is working even now. I am posting new patches because there are
> suggestions after posting patches and I try to take care of these.
>
>> Do you know there is a real use case for middle layer metacopy and chaining
>> and all that Jazz?
>
> You asked for support of mid layer support in V9. So I did it.
>
> https://www.spinics.net/lists/linux-unionfs/msg03712.html
>
>> When you first presented metacopy it sounded like you have a very solid use
>> case (chown -R). Does your specific use case extend to middle layers?
>
> I thought about it later and I think docker will probably need mid layer
> support. Reason being, that they probably will do chown and use that
> chowned directory as lower layer for container so that they can later
> do the diff w.r.t chowned copy and figure out what changes container
> did. If we do chown on upper and let container use it as upper, then it
> will appear that whole image has been changed by container.
>
> So I feel mid layer support is important for proper integration of
> this feature.
>
>> Is metacopy valueable enough without middle layers following?
>> Heck, AFAIK, container runtime doesn't even know how to deal with redirect
>> yet when committing an upper layer to an image. right?
>
> You probably are right. And they probably will fall back to native diff
> interface when metacopy feature is on. But even in that case, they will
> need to figure out what exactly container has changed w.r.t chowned
> copy and that means chowned copy has to be the lower layer and that
> means metacopy in mid layer support will be needed.
>
> If we can teach them to store REDIRECT xattr, their commit operation will
> become faster.
>
>>
>> Just wondering...
>
> I am just trying to figure out a point where you and miklos are happy
> with the design and patches. Mid layer support seems to be important.
>
> I get a feeling that miklos is still not entirely convinced about the
> usage of ORIGIN to get to follow metacopy chain and he still somehow
> wants to see making use of REDIRECT when need be.
>
> ORIGIN vs REDIRECT seems to be the only major sticking point w.r.t
> these patches at this point of time. As long as you and miklos agree
> on that semantics, things will be fine.

I think there are many problems with using ORIGIN for data.

I also think it should not be difficult to generalize the REDIRECT
code from directory to regular file.  It should just be adding more
conditions to create and handle redirects, no?  The actual code is
already there, because we do it for directories.

So what's the issue with lowerstack[0]?  Can't we just use the same
object for both purposes (i.e. the one found by going down the stack,
just like for directories)?

Thanks,
Miklos

Re: [QUESTION] problem about origin xattr

[Vivek Goyal]

On Wed, Jan 31, 2018 at 09:48:43PM +0100, Miklos Szeredi wrote:
> On Wed, Jan 31, 2018 at 9:34 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> > On Wed, Jan 31, 2018 at 09:59:07PM +0200, Amir Goldstein wrote:
> >
> > [..]
> >> >> >> >> As long as we use only inode number, it probably is still fine.
> >> >> >> >>
> >> >> >> >> But I look at ORIGIN as a generic infrastructure which other features can
> >> >> >> >> make use of it. For example, metacopy is using it to copy up file later.
> >> >> >> >> And there it will be non-intuitive that a file is not in any of the
> >> >> >> >> lower, still ORIGIN was decoded and file was copied up. It can come
> >> >> >> >> as a surprise to user. Atleast I was surprised when I ran into this
> >> >> >> >> while testing the feature.
> >> >> >>
> >> >> >> How about using REDIRECT for metacopy origin?   Keeping ORIGIN only
> >> >> >> for inode, also meaning ORIGIN is only ever used on upper layer, never
> >> >> >> on middle layers.
> >> >> >
> >> >> > Hi Miklos,
> >> >> >
> >> >> > Trying to understand it better. So proposal seems to be that when a file
> >> >> > is copied up metacopy only, we store both REDIRECT and ORIGIN in upper
> >> >> > inode. When traversing metacopy inode chain, use ORIGIN info on upper
> >> >> > inode and REDIRECT info on lower/midlayer metacopy inode.
> >> >> >
> >> >> > I am assuming that this is to handle the use case of tar of upper layer
> >> >> > and untaring it as lower layer.
> >> >> >
> >> >> > One of the concerns Amir had raised with usage of REDIRECT was that it
> >> >> > will be significantly slower as comapred to decoding ORIGIN. So by using
> >> >> > ORIGIN on upper, we are trying to mitigate it up to some extent? We will
> >> >> > still pay the cost of decoding REDIECT in midlayer.
> >> >> >
> >> >> > Am I understanding it right.
> >> >>
> >> >> Like directories, we'd only need to set REDIRECT on rename.
> >> >>
> >> >> So when file has METACOPY, but not REDIRECT, we just fall through to
> >> >> next layer below one we are currently operating on.  If we find
> >> >> METACOPY there, we just continue looking until we find a file
> >> >> containing the data.
> >> >>
> >> >> When we rename or hardlink a file with METACOPY, we add REDIRECT.
> >> >>
> >> >> If file has METACOPY and REDIRECT, we follow REDIRECT to find a file
> >> >> on the next level and keep iterating until we have the one with the
> >> >> data.
> >> >>
> >> >> ORIGIN would not be used in this case.  We might be able to use ORIGIN
> >> >> for some kind of verification, like we do for directories.   Amir has
> >> >> a better idea, I think.
> >> >>
> >> >> Another way to think about it is: METACOPY is the opposite of OPAQUE.
> >> >> For directories the default is "metacopy" and contents are merged.
> >> >> For files the default is "opaque" and content is not merged.  METACOPY
> >> >> turns that around and enables "merging" of data from a lower layer.
> >> >> I could even imagine real merging of data, but it's unlikely to be
> >> >> worth the effort, clone is much better for that; METACOPY is just a
> >> >> very restricted (and so much simpler) way of merging data.
> >> >
> >> > Ok, thanks. I am beginning to understand it better now.
> >> >
> >> > First implementaion issue which comes to my mind is that stack[0] location
> >> > conflict. Right now this is taken up by dentry which was obtained by following
> >> > ORIGIN from upper and acts as copy up origin.
> >> >
> >> > May be I should continue to use ORIGIN for upper dentry and when stack[0] is
> >> > filled and if its metacopy, then continue to find data dentry using either
> >> > REDIRECT or using same name and store in stack[1].
> >> >
> >>
> >> Question: don't you think it would be beneficial to get metacopy working and
> >> tested only from upper and without taking security considerations into the mix
> >> for first version?
> >
> > metacopy is working even now. I am posting new patches because there are
> > suggestions after posting patches and I try to take care of these.
> >
> >> Do you know there is a real use case for middle layer metacopy and chaining
> >> and all that Jazz?
> >
> > You asked for support of mid layer support in V9. So I did it.
> >
> > https://www.spinics.net/lists/linux-unionfs/msg03712.html
> >
> >> When you first presented metacopy it sounded like you have a very solid use
> >> case (chown -R). Does your specific use case extend to middle layers?
> >
> > I thought about it later and I think docker will probably need mid layer
> > support. Reason being, that they probably will do chown and use that
> > chowned directory as lower layer for container so that they can later
> > do the diff w.r.t chowned copy and figure out what changes container
> > did. If we do chown on upper and let container use it as upper, then it
> > will appear that whole image has been changed by container.
> >
> > So I feel mid layer support is important for proper integration of
> > this feature.
> >
> >> Is metacopy valueable enough without middle layers following?
> >> Heck, AFAIK, container runtime doesn't even know how to deal with redirect
> >> yet when committing an upper layer to an image. right?
> >
> > You probably are right. And they probably will fall back to native diff
> > interface when metacopy feature is on. But even in that case, they will
> > need to figure out what exactly container has changed w.r.t chowned
> > copy and that means chowned copy has to be the lower layer and that
> > means metacopy in mid layer support will be needed.
> >
> > If we can teach them to store REDIRECT xattr, their commit operation will
> > become faster.
> >
> >>
> >> Just wondering...
> >
> > I am just trying to figure out a point where you and miklos are happy
> > with the design and patches. Mid layer support seems to be important.
> >
> > I get a feeling that miklos is still not entirely convinced about the
> > usage of ORIGIN to get to follow metacopy chain and he still somehow
> > wants to see making use of REDIRECT when need be.
> >
> > ORIGIN vs REDIRECT seems to be the only major sticking point w.r.t
> > these patches at this point of time. As long as you and miklos agree
> > on that semantics, things will be fine.
> 
> I think there are many problems with using ORIGIN for data.
> 
> I also think it should not be difficult to generalize the REDIRECT
> code from directory to regular file.  It should just be adding more
> conditions to create and handle redirects, no?  The actual code is
> already there, because we do it for directories.

I guess so. We already are doing it for directories so we should be
able to extend it for regular files too. I don't know enough to be
able to say what affect this will have on performance.

> 
> So what's the issue with lowerstack[0]?  Can't we just use the same
> object for both purposes (i.e. the one found by going down the stack,
> just like for directories)?

I think we should be able to. But then it seems to make ORIGIN redundant.
Because currently we are using ORIGIN to retrieve lowerstack[0]. And if
we change that, that means I will have to rip out ORIGIN logic altogether.
Its a relatively bigger change. So wanted to figure out is that what
we are looking for.

Vivek

Re: [QUESTION] problem about origin xattr

[Miklos Szeredi]

On Wed, Jan 31, 2018 at 9:58 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> On Wed, Jan 31, 2018 at 09:48:43PM +0100, Miklos Szeredi wrote:
>> On Wed, Jan 31, 2018 at 9:34 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>> > On Wed, Jan 31, 2018 at 09:59:07PM +0200, Amir Goldstein wrote:
>> >
>> > [..]
>> >> >> >> >> As long as we use only inode number, it probably is still fine.
>> >> >> >> >>
>> >> >> >> >> But I look at ORIGIN as a generic infrastructure which other features can
>> >> >> >> >> make use of it. For example, metacopy is using it to copy up file later.
>> >> >> >> >> And there it will be non-intuitive that a file is not in any of the
>> >> >> >> >> lower, still ORIGIN was decoded and file was copied up. It can come
>> >> >> >> >> as a surprise to user. Atleast I was surprised when I ran into this
>> >> >> >> >> while testing the feature.
>> >> >> >>
>> >> >> >> How about using REDIRECT for metacopy origin?   Keeping ORIGIN only
>> >> >> >> for inode, also meaning ORIGIN is only ever used on upper layer, never
>> >> >> >> on middle layers.
>> >> >> >
>> >> >> > Hi Miklos,
>> >> >> >
>> >> >> > Trying to understand it better. So proposal seems to be that when a file
>> >> >> > is copied up metacopy only, we store both REDIRECT and ORIGIN in upper
>> >> >> > inode. When traversing metacopy inode chain, use ORIGIN info on upper
>> >> >> > inode and REDIRECT info on lower/midlayer metacopy inode.
>> >> >> >
>> >> >> > I am assuming that this is to handle the use case of tar of upper layer
>> >> >> > and untaring it as lower layer.
>> >> >> >
>> >> >> > One of the concerns Amir had raised with usage of REDIRECT was that it
>> >> >> > will be significantly slower as comapred to decoding ORIGIN. So by using
>> >> >> > ORIGIN on upper, we are trying to mitigate it up to some extent? We will
>> >> >> > still pay the cost of decoding REDIECT in midlayer.
>> >> >> >
>> >> >> > Am I understanding it right.
>> >> >>
>> >> >> Like directories, we'd only need to set REDIRECT on rename.
>> >> >>
>> >> >> So when file has METACOPY, but not REDIRECT, we just fall through to
>> >> >> next layer below one we are currently operating on.  If we find
>> >> >> METACOPY there, we just continue looking until we find a file
>> >> >> containing the data.
>> >> >>
>> >> >> When we rename or hardlink a file with METACOPY, we add REDIRECT.
>> >> >>
>> >> >> If file has METACOPY and REDIRECT, we follow REDIRECT to find a file
>> >> >> on the next level and keep iterating until we have the one with the
>> >> >> data.
>> >> >>
>> >> >> ORIGIN would not be used in this case.  We might be able to use ORIGIN
>> >> >> for some kind of verification, like we do for directories.   Amir has
>> >> >> a better idea, I think.
>> >> >>
>> >> >> Another way to think about it is: METACOPY is the opposite of OPAQUE.
>> >> >> For directories the default is "metacopy" and contents are merged.
>> >> >> For files the default is "opaque" and content is not merged.  METACOPY
>> >> >> turns that around and enables "merging" of data from a lower layer.
>> >> >> I could even imagine real merging of data, but it's unlikely to be
>> >> >> worth the effort, clone is much better for that; METACOPY is just a
>> >> >> very restricted (and so much simpler) way of merging data.
>> >> >
>> >> > Ok, thanks. I am beginning to understand it better now.
>> >> >
>> >> > First implementaion issue which comes to my mind is that stack[0] location
>> >> > conflict. Right now this is taken up by dentry which was obtained by following
>> >> > ORIGIN from upper and acts as copy up origin.
>> >> >
>> >> > May be I should continue to use ORIGIN for upper dentry and when stack[0] is
>> >> > filled and if its metacopy, then continue to find data dentry using either
>> >> > REDIRECT or using same name and store in stack[1].
>> >> >
>> >>
>> >> Question: don't you think it would be beneficial to get metacopy working and
>> >> tested only from upper and without taking security considerations into the mix
>> >> for first version?
>> >
>> > metacopy is working even now. I am posting new patches because there are
>> > suggestions after posting patches and I try to take care of these.
>> >
>> >> Do you know there is a real use case for middle layer metacopy and chaining
>> >> and all that Jazz?
>> >
>> > You asked for support of mid layer support in V9. So I did it.
>> >
>> > https://www.spinics.net/lists/linux-unionfs/msg03712.html
>> >
>> >> When you first presented metacopy it sounded like you have a very solid use
>> >> case (chown -R). Does your specific use case extend to middle layers?
>> >
>> > I thought about it later and I think docker will probably need mid layer
>> > support. Reason being, that they probably will do chown and use that
>> > chowned directory as lower layer for container so that they can later
>> > do the diff w.r.t chowned copy and figure out what changes container
>> > did. If we do chown on upper and let container use it as upper, then it
>> > will appear that whole image has been changed by container.
>> >
>> > So I feel mid layer support is important for proper integration of
>> > this feature.
>> >
>> >> Is metacopy valueable enough without middle layers following?
>> >> Heck, AFAIK, container runtime doesn't even know how to deal with redirect
>> >> yet when committing an upper layer to an image. right?
>> >
>> > You probably are right. And they probably will fall back to native diff
>> > interface when metacopy feature is on. But even in that case, they will
>> > need to figure out what exactly container has changed w.r.t chowned
>> > copy and that means chowned copy has to be the lower layer and that
>> > means metacopy in mid layer support will be needed.
>> >
>> > If we can teach them to store REDIRECT xattr, their commit operation will
>> > become faster.
>> >
>> >>
>> >> Just wondering...
>> >
>> > I am just trying to figure out a point where you and miklos are happy
>> > with the design and patches. Mid layer support seems to be important.
>> >
>> > I get a feeling that miklos is still not entirely convinced about the
>> > usage of ORIGIN to get to follow metacopy chain and he still somehow
>> > wants to see making use of REDIRECT when need be.
>> >
>> > ORIGIN vs REDIRECT seems to be the only major sticking point w.r.t
>> > these patches at this point of time. As long as you and miklos agree
>> > on that semantics, things will be fine.
>>
>> I think there are many problems with using ORIGIN for data.
>>
>> I also think it should not be difficult to generalize the REDIRECT
>> code from directory to regular file.  It should just be adding more
>> conditions to create and handle redirects, no?  The actual code is
>> already there, because we do it for directories.
>
> I guess so. We already are doing it for directories so we should be
> able to extend it for regular files too. I don't know enough to be
> able to say what affect this will have on performance.
>
>>
>> So what's the issue with lowerstack[0]?  Can't we just use the same
>> object for both purposes (i.e. the one found by going down the stack,
>> just like for directories)?
>
> I think we should be able to. But then it seems to make ORIGIN redundant.
> Because currently we are using ORIGIN to retrieve lowerstack[0]. And if
> we change that, that means I will have to rip out ORIGIN logic altogether.
> Its a relatively bigger change. So wanted to figure out is that what
> we are looking for.

Don't rip out ORIGIN logic, just disable it when we find METACOPY.

So logic should be:

 - check METACOPY xattr, if exists continue to lower layers just like
non-opaque directory
 - otherwise use ORIGIN xattr, just like we used to

Thanks,
Miklos

>
> Vivek

Re: [QUESTION] problem about origin xattr

[Vivek Goyal]

On Wed, Jan 31, 2018 at 10:06:22PM +0100, Miklos Szeredi wrote:
> On Wed, Jan 31, 2018 at 9:58 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> > On Wed, Jan 31, 2018 at 09:48:43PM +0100, Miklos Szeredi wrote:
> >> On Wed, Jan 31, 2018 at 9:34 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> >> > On Wed, Jan 31, 2018 at 09:59:07PM +0200, Amir Goldstein wrote:
> >> >
> >> > [..]
> >> >> >> >> >> As long as we use only inode number, it probably is still fine.
> >> >> >> >> >>
> >> >> >> >> >> But I look at ORIGIN as a generic infrastructure which other features can
> >> >> >> >> >> make use of it. For example, metacopy is using it to copy up file later.
> >> >> >> >> >> And there it will be non-intuitive that a file is not in any of the
> >> >> >> >> >> lower, still ORIGIN was decoded and file was copied up. It can come
> >> >> >> >> >> as a surprise to user. Atleast I was surprised when I ran into this
> >> >> >> >> >> while testing the feature.
> >> >> >> >>
> >> >> >> >> How about using REDIRECT for metacopy origin?   Keeping ORIGIN only
> >> >> >> >> for inode, also meaning ORIGIN is only ever used on upper layer, never
> >> >> >> >> on middle layers.
> >> >> >> >
> >> >> >> > Hi Miklos,
> >> >> >> >
> >> >> >> > Trying to understand it better. So proposal seems to be that when a file
> >> >> >> > is copied up metacopy only, we store both REDIRECT and ORIGIN in upper
> >> >> >> > inode. When traversing metacopy inode chain, use ORIGIN info on upper
> >> >> >> > inode and REDIRECT info on lower/midlayer metacopy inode.
> >> >> >> >
> >> >> >> > I am assuming that this is to handle the use case of tar of upper layer
> >> >> >> > and untaring it as lower layer.
> >> >> >> >
> >> >> >> > One of the concerns Amir had raised with usage of REDIRECT was that it
> >> >> >> > will be significantly slower as comapred to decoding ORIGIN. So by using
> >> >> >> > ORIGIN on upper, we are trying to mitigate it up to some extent? We will
> >> >> >> > still pay the cost of decoding REDIECT in midlayer.
> >> >> >> >
> >> >> >> > Am I understanding it right.
> >> >> >>
> >> >> >> Like directories, we'd only need to set REDIRECT on rename.
> >> >> >>
> >> >> >> So when file has METACOPY, but not REDIRECT, we just fall through to
> >> >> >> next layer below one we are currently operating on.  If we find
> >> >> >> METACOPY there, we just continue looking until we find a file
> >> >> >> containing the data.
> >> >> >>
> >> >> >> When we rename or hardlink a file with METACOPY, we add REDIRECT.
> >> >> >>
> >> >> >> If file has METACOPY and REDIRECT, we follow REDIRECT to find a file
> >> >> >> on the next level and keep iterating until we have the one with the
> >> >> >> data.
> >> >> >>
> >> >> >> ORIGIN would not be used in this case.  We might be able to use ORIGIN
> >> >> >> for some kind of verification, like we do for directories.   Amir has
> >> >> >> a better idea, I think.
> >> >> >>
> >> >> >> Another way to think about it is: METACOPY is the opposite of OPAQUE.
> >> >> >> For directories the default is "metacopy" and contents are merged.
> >> >> >> For files the default is "opaque" and content is not merged.  METACOPY
> >> >> >> turns that around and enables "merging" of data from a lower layer.
> >> >> >> I could even imagine real merging of data, but it's unlikely to be
> >> >> >> worth the effort, clone is much better for that; METACOPY is just a
> >> >> >> very restricted (and so much simpler) way of merging data.
> >> >> >
> >> >> > Ok, thanks. I am beginning to understand it better now.
> >> >> >
> >> >> > First implementaion issue which comes to my mind is that stack[0] location
> >> >> > conflict. Right now this is taken up by dentry which was obtained by following
> >> >> > ORIGIN from upper and acts as copy up origin.
> >> >> >
> >> >> > May be I should continue to use ORIGIN for upper dentry and when stack[0] is
> >> >> > filled and if its metacopy, then continue to find data dentry using either
> >> >> > REDIRECT or using same name and store in stack[1].
> >> >> >
> >> >>
> >> >> Question: don't you think it would be beneficial to get metacopy working and
> >> >> tested only from upper and without taking security considerations into the mix
> >> >> for first version?
> >> >
> >> > metacopy is working even now. I am posting new patches because there are
> >> > suggestions after posting patches and I try to take care of these.
> >> >
> >> >> Do you know there is a real use case for middle layer metacopy and chaining
> >> >> and all that Jazz?
> >> >
> >> > You asked for support of mid layer support in V9. So I did it.
> >> >
> >> > https://www.spinics.net/lists/linux-unionfs/msg03712.html
> >> >
> >> >> When you first presented metacopy it sounded like you have a very solid use
> >> >> case (chown -R). Does your specific use case extend to middle layers?
> >> >
> >> > I thought about it later and I think docker will probably need mid layer
> >> > support. Reason being, that they probably will do chown and use that
> >> > chowned directory as lower layer for container so that they can later
> >> > do the diff w.r.t chowned copy and figure out what changes container
> >> > did. If we do chown on upper and let container use it as upper, then it
> >> > will appear that whole image has been changed by container.
> >> >
> >> > So I feel mid layer support is important for proper integration of
> >> > this feature.
> >> >
> >> >> Is metacopy valueable enough without middle layers following?
> >> >> Heck, AFAIK, container runtime doesn't even know how to deal with redirect
> >> >> yet when committing an upper layer to an image. right?
> >> >
> >> > You probably are right. And they probably will fall back to native diff
> >> > interface when metacopy feature is on. But even in that case, they will
> >> > need to figure out what exactly container has changed w.r.t chowned
> >> > copy and that means chowned copy has to be the lower layer and that
> >> > means metacopy in mid layer support will be needed.
> >> >
> >> > If we can teach them to store REDIRECT xattr, their commit operation will
> >> > become faster.
> >> >
> >> >>
> >> >> Just wondering...
> >> >
> >> > I am just trying to figure out a point where you and miklos are happy
> >> > with the design and patches. Mid layer support seems to be important.
> >> >
> >> > I get a feeling that miklos is still not entirely convinced about the
> >> > usage of ORIGIN to get to follow metacopy chain and he still somehow
> >> > wants to see making use of REDIRECT when need be.
> >> >
> >> > ORIGIN vs REDIRECT seems to be the only major sticking point w.r.t
> >> > these patches at this point of time. As long as you and miklos agree
> >> > on that semantics, things will be fine.
> >>
> >> I think there are many problems with using ORIGIN for data.
> >>
> >> I also think it should not be difficult to generalize the REDIRECT
> >> code from directory to regular file.  It should just be adding more
> >> conditions to create and handle redirects, no?  The actual code is
> >> already there, because we do it for directories.
> >
> > I guess so. We already are doing it for directories so we should be
> > able to extend it for regular files too. I don't know enough to be
> > able to say what affect this will have on performance.
> >
> >>
> >> So what's the issue with lowerstack[0]?  Can't we just use the same
> >> object for both purposes (i.e. the one found by going down the stack,
> >> just like for directories)?
> >
> > I think we should be able to. But then it seems to make ORIGIN redundant.
> > Because currently we are using ORIGIN to retrieve lowerstack[0]. And if
> > we change that, that means I will have to rip out ORIGIN logic altogether.
> > Its a relatively bigger change. So wanted to figure out is that what
> > we are looking for.
> 
> Don't rip out ORIGIN logic, just disable it when we find METACOPY.
> 
> So logic should be:
> 
>  - check METACOPY xattr, if exists continue to lower layers just like
> non-opaque directory
>  - otherwise use ORIGIN xattr, just like we used to
> 
> Thanks,
> Miklos

Ok, I will look into it.

In the mean time I am posting my V11 patches with current design. Just to
keep track of last working patches, just in case we decide to go back to
old design.

Vivek

Re: [QUESTION] problem about origin xattr

[Amir Goldstein]

On Wed, Jan 31, 2018 at 11:12 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> On Wed, Jan 31, 2018 at 10:06:22PM +0100, Miklos Szeredi wrote:
>> On Wed, Jan 31, 2018 at 9:58 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>> > On Wed, Jan 31, 2018 at 09:48:43PM +0100, Miklos Szeredi wrote:
>> >> On Wed, Jan 31, 2018 at 9:34 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>> >> > On Wed, Jan 31, 2018 at 09:59:07PM +0200, Amir Goldstein wrote:
>> >> >
>> >> > [..]
>> >> >> >> >> >> As long as we use only inode number, it probably is still fine.
>> >> >> >> >> >>
>> >> >> >> >> >> But I look at ORIGIN as a generic infrastructure which other features can
>> >> >> >> >> >> make use of it. For example, metacopy is using it to copy up file later.
>> >> >> >> >> >> And there it will be non-intuitive that a file is not in any of the
>> >> >> >> >> >> lower, still ORIGIN was decoded and file was copied up. It can come
>> >> >> >> >> >> as a surprise to user. Atleast I was surprised when I ran into this
>> >> >> >> >> >> while testing the feature.
>> >> >> >> >>
>> >> >> >> >> How about using REDIRECT for metacopy origin?   Keeping ORIGIN only
>> >> >> >> >> for inode, also meaning ORIGIN is only ever used on upper layer, never
>> >> >> >> >> on middle layers.
>> >> >> >> >
>> >> >> >> > Hi Miklos,
>> >> >> >> >
>> >> >> >> > Trying to understand it better. So proposal seems to be that when a file
>> >> >> >> > is copied up metacopy only, we store both REDIRECT and ORIGIN in upper
>> >> >> >> > inode. When traversing metacopy inode chain, use ORIGIN info on upper
>> >> >> >> > inode and REDIRECT info on lower/midlayer metacopy inode.
>> >> >> >> >
>> >> >> >> > I am assuming that this is to handle the use case of tar of upper layer
>> >> >> >> > and untaring it as lower layer.
>> >> >> >> >
>> >> >> >> > One of the concerns Amir had raised with usage of REDIRECT was that it
>> >> >> >> > will be significantly slower as comapred to decoding ORIGIN. So by using
>> >> >> >> > ORIGIN on upper, we are trying to mitigate it up to some extent? We will
>> >> >> >> > still pay the cost of decoding REDIECT in midlayer.
>> >> >> >> >
>> >> >> >> > Am I understanding it right.
>> >> >> >>
>> >> >> >> Like directories, we'd only need to set REDIRECT on rename.
>> >> >> >>
>> >> >> >> So when file has METACOPY, but not REDIRECT, we just fall through to
>> >> >> >> next layer below one we are currently operating on.  If we find
>> >> >> >> METACOPY there, we just continue looking until we find a file
>> >> >> >> containing the data.
>> >> >> >>
>> >> >> >> When we rename or hardlink a file with METACOPY, we add REDIRECT.
>> >> >> >>
>> >> >> >> If file has METACOPY and REDIRECT, we follow REDIRECT to find a file
>> >> >> >> on the next level and keep iterating until we have the one with the
>> >> >> >> data.
>> >> >> >>
>> >> >> >> ORIGIN would not be used in this case.  We might be able to use ORIGIN
>> >> >> >> for some kind of verification, like we do for directories.   Amir has
>> >> >> >> a better idea, I think.
>> >> >> >>
>> >> >> >> Another way to think about it is: METACOPY is the opposite of OPAQUE.
>> >> >> >> For directories the default is "metacopy" and contents are merged.
>> >> >> >> For files the default is "opaque" and content is not merged.  METACOPY
>> >> >> >> turns that around and enables "merging" of data from a lower layer.
>> >> >> >> I could even imagine real merging of data, but it's unlikely to be
>> >> >> >> worth the effort, clone is much better for that; METACOPY is just a
>> >> >> >> very restricted (and so much simpler) way of merging data.
>> >> >> >
>> >> >> > Ok, thanks. I am beginning to understand it better now.
>> >> >> >
>> >> >> > First implementaion issue which comes to my mind is that stack[0] location
>> >> >> > conflict. Right now this is taken up by dentry which was obtained by following
>> >> >> > ORIGIN from upper and acts as copy up origin.
>> >> >> >
>> >> >> > May be I should continue to use ORIGIN for upper dentry and when stack[0] is
>> >> >> > filled and if its metacopy, then continue to find data dentry using either
>> >> >> > REDIRECT or using same name and store in stack[1].
>> >> >> >
>> >> >>
>> >> >> Question: don't you think it would be beneficial to get metacopy working and
>> >> >> tested only from upper and without taking security considerations into the mix
>> >> >> for first version?
>> >> >
>> >> > metacopy is working even now. I am posting new patches because there are
>> >> > suggestions after posting patches and I try to take care of these.
>> >> >
>> >> >> Do you know there is a real use case for middle layer metacopy and chaining
>> >> >> and all that Jazz?
>> >> >
>> >> > You asked for support of mid layer support in V9. So I did it.
>> >> >
>> >> > https://www.spinics.net/lists/linux-unionfs/msg03712.html
>> >> >
>> >> >> When you first presented metacopy it sounded like you have a very solid use
>> >> >> case (chown -R). Does your specific use case extend to middle layers?
>> >> >
>> >> > I thought about it later and I think docker will probably need mid layer
>> >> > support. Reason being, that they probably will do chown and use that
>> >> > chowned directory as lower layer for container so that they can later
>> >> > do the diff w.r.t chowned copy and figure out what changes container
>> >> > did. If we do chown on upper and let container use it as upper, then it
>> >> > will appear that whole image has been changed by container.
>> >> >
>> >> > So I feel mid layer support is important for proper integration of
>> >> > this feature.
>> >> >
>> >> >> Is metacopy valueable enough without middle layers following?
>> >> >> Heck, AFAIK, container runtime doesn't even know how to deal with redirect
>> >> >> yet when committing an upper layer to an image. right?
>> >> >
>> >> > You probably are right. And they probably will fall back to native diff
>> >> > interface when metacopy feature is on. But even in that case, they will
>> >> > need to figure out what exactly container has changed w.r.t chowned
>> >> > copy and that means chowned copy has to be the lower layer and that
>> >> > means metacopy in mid layer support will be needed.
>> >> >
>> >> > If we can teach them to store REDIRECT xattr, their commit operation will
>> >> > become faster.
>> >> >
>> >> >>
>> >> >> Just wondering...
>> >> >
>> >> > I am just trying to figure out a point where you and miklos are happy
>> >> > with the design and patches. Mid layer support seems to be important.
>> >> >
>> >> > I get a feeling that miklos is still not entirely convinced about the
>> >> > usage of ORIGIN to get to follow metacopy chain and he still somehow
>> >> > wants to see making use of REDIRECT when need be.
>> >> >
>> >> > ORIGIN vs REDIRECT seems to be the only major sticking point w.r.t
>> >> > these patches at this point of time. As long as you and miklos agree
>> >> > on that semantics, things will be fine.
>> >>
>> >> I think there are many problems with using ORIGIN for data.
>> >>
>> >> I also think it should not be difficult to generalize the REDIRECT
>> >> code from directory to regular file.  It should just be adding more
>> >> conditions to create and handle redirects, no?  The actual code is
>> >> already there, because we do it for directories.
>> >
>> > I guess so. We already are doing it for directories so we should be
>> > able to extend it for regular files too. I don't know enough to be
>> > able to say what affect this will have on performance.
>> >
>> >>
>> >> So what's the issue with lowerstack[0]?  Can't we just use the same
>> >> object for both purposes (i.e. the one found by going down the stack,
>> >> just like for directories)?
>> >
>> > I think we should be able to. But then it seems to make ORIGIN redundant.
>> > Because currently we are using ORIGIN to retrieve lowerstack[0]. And if
>> > we change that, that means I will have to rip out ORIGIN logic altogether.
>> > Its a relatively bigger change. So wanted to figure out is that what
>> > we are looking for.
>>
>> Don't rip out ORIGIN logic, just disable it when we find METACOPY.
>>
>> So logic should be:
>>
>>  - check METACOPY xattr, if exists continue to lower layers just like
>> non-opaque directory
>>  - otherwise use ORIGIN xattr, just like we used to

Careful there, when following metacopy by path, you also need to apply
ovl_verify_lower() logic for indexed files, i.e. all files with nfs_export
and lower hardlinks with index=on. same as I did for merge dir lookup
with nfs_export.

Cheers,
Amir.

Re: [QUESTION] problem about origin xattr

[yangerkun]

On 2/1/2018 3:05 AM, Vivek Goyal wrote:
> On Wed, Jan 31, 2018 at 07:08:18PM +0100, Miklos Szeredi wrote:
>> On Wed, Jan 31, 2018 at 5:55 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>>> On Wed, Jan 31, 2018 at 05:10:28PM +0100, Miklos Szeredi wrote:
>>>> On Wed, Jan 31, 2018 at 4:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>>>>> On Wed, Jan 31, 2018 at 5:46 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>>>>>> On Wed, Jan 31, 2018 at 05:38:45PM +0200, Amir Goldstein wrote:
>>>>>>> On Wed, Jan 31, 2018 at 5:20 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>>>>>>>> On Wed, Jan 31, 2018 at 03:57:12PM +0200, Amir Goldstein wrote:
>>>>>>>>
>>>>>>>> ORIGIN behavior is little unintuitive though. Despite the fact that file
>>>>>>>> is not searchable through lower, it is visible through decoding of file
>>>>>>>> handle and it is atleast non-intuitive.
>>>>>>>>
>>>>>>>
>>>>>>> Maybe not intuitive at first glance, but try again:
>>>>>>>
>>>>>>> The only thing we *need* from underlying fs is to provide us with a unique
>>>>>>> and persistent inode number we can use for the overlay object.
>>>>>>>
>>>>>>> Even if the inode number we get from underlying fs is not in any of the
>>>>>>> layers, it is still a  viable inode number we can use in overlay coupled
>>>>>>> with overlay unique st_dev, to create a system wide unique st_dev;st_ino
>>>>>>> tuple.
>>>>>>
>>>>>> As long as we use only inode number, it probably is still fine.
>>>>>>
>>>>>> But I look at ORIGIN as a generic infrastructure which other features can
>>>>>> make use of it. For example, metacopy is using it to copy up file later.
>>>>>> And there it will be non-intuitive that a file is not in any of the
>>>>>> lower, still ORIGIN was decoded and file was copied up. It can come
>>>>>> as a surprise to user. Atleast I was surprised when I ran into this
>>>>>> while testing the feature.
>>>>
>>>> How about using REDIRECT for metacopy origin?   Keeping ORIGIN only
>>>> for inode, also meaning ORIGIN is only ever used on upper layer, never
>>>> on middle layers.
>>>
>>> Hi Miklos,
>>>
>>> Trying to understand it better. So proposal seems to be that when a file
>>> is copied up metacopy only, we store both REDIRECT and ORIGIN in upper
>>> inode. When traversing metacopy inode chain, use ORIGIN info on upper
>>> inode and REDIRECT info on lower/midlayer metacopy inode.
>>>
>>> I am assuming that this is to handle the use case of tar of upper layer
>>> and untaring it as lower layer.
>>>
>>> One of the concerns Amir had raised with usage of REDIRECT was that it
>>> will be significantly slower as comapred to decoding ORIGIN. So by using
>>> ORIGIN on upper, we are trying to mitigate it up to some extent? We will
>>> still pay the cost of decoding REDIECT in midlayer.
>>>
>>> Am I understanding it right.
>>
>> Like directories, we'd only need to set REDIRECT on rename.
>>
>> So when file has METACOPY, but not REDIRECT, we just fall through to
>> next layer below one we are currently operating on.  If we find
>> METACOPY there, we just continue looking until we find a file
>> containing the data.
>>
>> When we rename or hardlink a file with METACOPY, we add REDIRECT.
>>
>> If file has METACOPY and REDIRECT, we follow REDIRECT to find a file
>> on the next level and keep iterating until we have the one with the
>> data.
>>
>> ORIGIN would not be used in this case.  We might be able to use ORIGIN
>> for some kind of verification, like we do for directories.   Amir has
>> a better idea, I think.
>>
>> Another way to think about it is: METACOPY is the opposite of OPAQUE.
>> For directories the default is "metacopy" and contents are merged.
>> For files the default is "opaque" and content is not merged.  METACOPY
>> turns that around and enables "merging" of data from a lower layer.
>> I could even imagine real merging of data, but it's unlikely to be
>> worth the effort, clone is much better for that; METACOPY is just a
>> very restricted (and so much simpler) way of merging data.
> 
> Ok, thanks. I am beginning to understand it better now.

Does there any problem using REDIRECT to confirm the file with METACOPY? 
If there is a rename of the REDIRECT file in lower while overlay 
offline, METACOPY will get no file?

I prefer suggestion of Amir, use subtree_check like nfs to make sure the 
origin dentry existing in subdir of the lower layer.

Thanks,
yangerkun

Re: [QUESTION] problem about origin xattr

[Amir Goldstein]

On Thu, Feb 1, 2018 at 5:57 AM, yangerkun <yangerkun@huawei.com> wrote:
>
>
[...]
>>>
>>>
>>> Like directories, we'd only need to set REDIRECT on rename.
>>>
>>> So when file has METACOPY, but not REDIRECT, we just fall through to
>>> next layer below one we are currently operating on.  If we find
>>> METACOPY there, we just continue looking until we find a file
>>> containing the data.
>>>
>>> When we rename or hardlink a file with METACOPY, we add REDIRECT.
>>>
>>> If file has METACOPY and REDIRECT, we follow REDIRECT to find a file
>>> on the next level and keep iterating until we have the one with the
>>> data.
>>>
>>> ORIGIN would not be used in this case.  We might be able to use ORIGIN
>>> for some kind of verification, like we do for directories.   Amir has
>>> a better idea, I think.
>>>
>>> Another way to think about it is: METACOPY is the opposite of OPAQUE.
>>> For directories the default is "metacopy" and contents are merged.
>>> For files the default is "opaque" and content is not merged.  METACOPY
>>> turns that around and enables "merging" of data from a lower layer.
>>> I could even imagine real merging of data, but it's unlikely to be
>>> worth the effort, clone is much better for that; METACOPY is just a
>>> very restricted (and so much simpler) way of merging data.
>>
>>
>> Ok, thanks. I am beginning to understand it better now.
>
>
> Does there any problem using REDIRECT to confirm the file with METACOPY? If
> there is a rename of the REDIRECT file in lower while overlay offline,
> METACOPY will get no file?
>
> I prefer suggestion of Amir, use subtree_check like nfs to make sure the
> origin dentry existing in subdir of the lower layer.
>

This preference depends on the use case.
For your use case that is sensitive to offline lower renames, as well as my
snapshot use case, the chosen behavior could be to follow ORIGIN and fix
REDIRECT to match it, as I implemented here:
https://github.com/amir73il/linux/commits/ovl-redirect-origin

For the default use case where lower content is not expected to change,
but layers could be copied/migrated, relying on file handles is a problem,
so the default would be to follow REDIRECT and verify/fix ORIGIN to match it.

When Vivek implements METACOPY using the same code path used
for directories, the work I did for redirect_dir=origin could be applied
also to METACOPY, given that ORIGIN is changed to use connectable
file handles in that case.

Cheers,
Amir.

Re: [QUESTION] problem about origin xattr

[Vivek Goyal]

On Thu, Feb 01, 2018 at 01:26:58AM +0200, Amir Goldstein wrote:

[..]
> >> >> > ORIGIN vs REDIRECT seems to be the only major sticking point w.r.t
> >> >> > these patches at this point of time. As long as you and miklos agree
> >> >> > on that semantics, things will be fine.
> >> >>
> >> >> I think there are many problems with using ORIGIN for data.
> >> >>
> >> >> I also think it should not be difficult to generalize the REDIRECT
> >> >> code from directory to regular file.  It should just be adding more
> >> >> conditions to create and handle redirects, no?  The actual code is
> >> >> already there, because we do it for directories.
> >> >
> >> > I guess so. We already are doing it for directories so we should be
> >> > able to extend it for regular files too. I don't know enough to be
> >> > able to say what affect this will have on performance.
> >> >
> >> >>
> >> >> So what's the issue with lowerstack[0]?  Can't we just use the same
> >> >> object for both purposes (i.e. the one found by going down the stack,
> >> >> just like for directories)?
> >> >
> >> > I think we should be able to. But then it seems to make ORIGIN redundant.
> >> > Because currently we are using ORIGIN to retrieve lowerstack[0]. And if
> >> > we change that, that means I will have to rip out ORIGIN logic altogether.
> >> > Its a relatively bigger change. So wanted to figure out is that what
> >> > we are looking for.
> >>
> >> Don't rip out ORIGIN logic, just disable it when we find METACOPY.
> >>
> >> So logic should be:
> >>
> >>  - check METACOPY xattr, if exists continue to lower layers just like
> >> non-opaque directory
> >>  - otherwise use ORIGIN xattr, just like we used to
> 
> Careful there, when following metacopy by path, you also need to apply
> ovl_verify_lower() logic for indexed files, i.e. all files with nfs_export
> and lower hardlinks with index=on. same as I did for merge dir lookup
> with nfs_export.

Hmm..., so for metacopy files, if ORIGIN fh does not match the lower found
in lowerstack[0] what do we do. Return -ESTALE?

Vivek

Re: [QUESTION] problem about origin xattr

[Amir Goldstein]

On Thu, Feb 1, 2018 at 5:25 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> On Thu, Feb 01, 2018 at 01:26:58AM +0200, Amir Goldstein wrote:
>
> [..]
>> >> >> > ORIGIN vs REDIRECT seems to be the only major sticking point w.r.t
>> >> >> > these patches at this point of time. As long as you and miklos agree
>> >> >> > on that semantics, things will be fine.
>> >> >>
>> >> >> I think there are many problems with using ORIGIN for data.
>> >> >>
>> >> >> I also think it should not be difficult to generalize the REDIRECT
>> >> >> code from directory to regular file.  It should just be adding more
>> >> >> conditions to create and handle redirects, no?  The actual code is
>> >> >> already there, because we do it for directories.
>> >> >
>> >> > I guess so. We already are doing it for directories so we should be
>> >> > able to extend it for regular files too. I don't know enough to be
>> >> > able to say what affect this will have on performance.
>> >> >
>> >> >>
>> >> >> So what's the issue with lowerstack[0]?  Can't we just use the same
>> >> >> object for both purposes (i.e. the one found by going down the stack,
>> >> >> just like for directories)?
>> >> >
>> >> > I think we should be able to. But then it seems to make ORIGIN redundant.
>> >> > Because currently we are using ORIGIN to retrieve lowerstack[0]. And if
>> >> > we change that, that means I will have to rip out ORIGIN logic altogether.
>> >> > Its a relatively bigger change. So wanted to figure out is that what
>> >> > we are looking for.
>> >>
>> >> Don't rip out ORIGIN logic, just disable it when we find METACOPY.
>> >>
>> >> So logic should be:
>> >>
>> >>  - check METACOPY xattr, if exists continue to lower layers just like
>> >> non-opaque directory
>> >>  - otherwise use ORIGIN xattr, just like we used to
>>
>> Careful there, when following metacopy by path, you also need to apply
>> ovl_verify_lower() logic for indexed files, i.e. all files with nfs_export
>> and lower hardlinks with index=on. same as I did for merge dir lookup
>> with nfs_export.
>
> Hmm..., so for metacopy files, if ORIGIN fh does not match the lower found
> in lowerstack[0] what do we do. Return -ESTALE?
>

Whether you follow an unverified lower or return -ESTALE may
depend on configuration. it makes sense to follow unverified lower at least
for the use case of copied layers, but if you do follow unverified lower you
cannot use the index, so something like this:

----

diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index de3e6da1d5a5..fe2a9d980129 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -900,13 +900,14 @@ struct dentry *ovl_lookup(struct inode *dir,
struct dentry *dentry,
                 */
                if (upperdentry && !ctr && ovl_verify_lower(dentry->d_sb)) {
                        err = ovl_verify_origin(upperdentry, this, false);
-                       if (err) {
+                       if (!err) {
+                               /* Bless lower as verified origin */
+                               origin = this;
+                       } else if (d.is_dir) {
                                dput(this);
                                break;
                        }

-                       /* Bless lower dir as verified origin */
-                       origin = this;
                }

                stack[ctr].dentry = this;
@@ -942,11 +943,11 @@ struct dentry *ovl_lookup(struct inode *dir,
struct dentry *dentry,

        /*
         * Lookup index by lower inode and verify it matches upper inode.
-        * We only trust dir index if we verified that lower dir matches
-        * origin, otherwise dir index entries may be inconsistent and we
-        * ignore them. Always lookup index of non-dir and non-upper.
+        * We only trust dir and metacopy index if we verified that lower
+        * matches origin, otherwise index entries may be inconsistent and we
+        * ignore them. Always lookup index of non-upper non-dir.
         */
-       if (ctr && (!upperdentry || !d.is_dir))
+       if (ctr && (!upperdentry && !d.is_dir))
                origin = stack[0].dentry;

        if (origin && ovl_indexdir(dentry->d_sb) &&