[PATCH] ovl: document lower modification caveats

[PATCH] ovl: document lower modification caveats

[Kevin Locke]

Some overlayfs optional features are incompatible with offline changes
to the lower tree[1][2][3] and may result in -EXDEV[4], -EIO[5], or
other errors.  Such modification is not supported and the error behavior
is intentionally not specified.

Update the "Changes to underlying filesystems" section to note this
restriction.  Move the paragraph describing the offline behavior below
the online behavior so it is adjacent to the following 3 paragraphs
describing the NFS export offline modification behavior.

[1]: https://lore.kernel.org/linux-unionfs/20200708142353.GA103536@redhat.com/
[2]: https://lore.kernel.org/linux-unionfs/CAOQ4uxi23Zsmfb4rCed1n=On0NNA5KZD74jjjeyz+et32sk-gg@mail.gmail.com/
[3]: https://lore.kernel.org/linux-unionfs/20200817135651.GA637139@redhat.com/
[4]: https://lore.kernel.org/linux-unionfs/20200709153616.GE150543@redhat.com/
[5]: https://lore.kernel.org/linux-unionfs/20200812135529.GA122370@kevinolos/

Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
---

This patch is based on the overlayfs-devel branch of
https://github.com/amir73il/linux.git  If there is a more suitable base,
let me know and I'll rebase the next version.

It appears that Vivek's patch[4] has not been applied and serves a
similar purpose to this patch, although it applies to a different
section of the docs.  I'd be happy to collaborate on a combined patch,
if there's any interest/need.

This patch does not mention nfs_export being incompatible with offline
changes to lower, since the following 3 paragraphs specify the behavior
(IIUC).  If the behavior of offline modification with nfs_export is
undefined, should these paragraphs be removed?

Thanks for considering,
Kevin

 Documentation/filesystems/overlayfs.rst | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst
index fcda5d6ba9ac..9b39374afdbd 100644
--- a/Documentation/filesystems/overlayfs.rst
+++ b/Documentation/filesystems/overlayfs.rst
@@ -467,14 +467,18 @@ summarized in the `Inode properties`_ table above.
 Changes to underlying filesystems
 ---------------------------------
 
-Offline changes, when the overlay is not mounted, are allowed to either
-the upper or the lower trees.
-
 Changes to the underlying filesystems while part of a mounted overlay
 filesystem are not allowed.  If the underlying filesystem is changed,
 the behavior of the overlay is undefined, though it will not result in
 a crash or deadlock.
 
+Offline changes, when the overlay is not mounted, are allowed to the
+upper tree.  Offline changes to the lower tree are only allowed if the
+"metadata only copy up", "inode index", and "redirect_dir" features
+have not been used.  If the lower tree is modified and any of these
+features has been used, the behavior of the overlay is undefined,
+though it will not result in a crash or deadlock.
+
 When the overlay NFS export feature is enabled, overlay filesystems
 behavior on offline changes of the underlying lower layer is different
 than the behavior when NFS export is disabled.
-- 
2.28.0

[PATCH v2] ovl: document lower modification caveats

[Kevin Locke]

Some overlayfs optional features are incompatible with offline changes
to the lower tree[1][2][3] and may result in -EXDEV[4], -EIO[5], or
other errors.  Such modification is not supported and the error behavior
is intentionally not specified.

Update the "Changes to underlying filesystems" section to note this
restriction.  Move the paragraph describing the offline behavior below
the online behavior so it is adjacent to the following 3 paragraphs
describing the NFS export offline modification behavior.

[1]: https://lore.kernel.org/linux-unionfs/20200708142353.GA103536@redhat.com/
[2]: https://lore.kernel.org/linux-unionfs/CAOQ4uxi23Zsmfb4rCed1n=On0NNA5KZD74jjjeyz+et32sk-gg@mail.gmail.com/
[3]: https://lore.kernel.org/linux-unionfs/20200817135651.GA637139@redhat.com/
[4]: https://lore.kernel.org/linux-unionfs/20200709153616.GE150543@redhat.com/
[5]: https://lore.kernel.org/linux-unionfs/20200812135529.GA122370@kevinolos/

Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
---

Changes since v1:
- Actually send to the maintainer this time.
  (Sorry Miklos Szeredi, not sure what I was thinking on v1!)
- Rebase onto overlayfs-next of
  git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git

As mentioned with v1, I'm still open to collaborating on a combined
patch if there is still interest in including Vivek's changes to the
"Sharing and copying layers" section.

This patch does not mention nfs_export being incompatible with offline
changes to lower, since the following 3 paragraphs specify the behavior
(IIUC).  If the behavior of offline modification with nfs_export is
undefined, should these paragraphs be removed?

Thanks for considering (again),
Kevin

 Documentation/filesystems/overlayfs.rst | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst
index fcda5d6ba9ac..9b39374afdbd 100644
--- a/Documentation/filesystems/overlayfs.rst
+++ b/Documentation/filesystems/overlayfs.rst
@@ -467,14 +467,18 @@ summarized in the `Inode properties`_ table above.
 Changes to underlying filesystems
 ---------------------------------
 
-Offline changes, when the overlay is not mounted, are allowed to either
-the upper or the lower trees.
-
 Changes to the underlying filesystems while part of a mounted overlay
 filesystem are not allowed.  If the underlying filesystem is changed,
 the behavior of the overlay is undefined, though it will not result in
 a crash or deadlock.
 
+Offline changes, when the overlay is not mounted, are allowed to the
+upper tree.  Offline changes to the lower tree are only allowed if the
+"metadata only copy up", "inode index", and "redirect_dir" features
+have not been used.  If the lower tree is modified and any of these
+features has been used, the behavior of the overlay is undefined,
+though it will not result in a crash or deadlock.
+
 When the overlay NFS export feature is enabled, overlay filesystems
 behavior on offline changes of the underlying lower layer is different
 than the behavior when NFS export is disabled.
-- 
2.28.0

Re: [PATCH v2] ovl: document lower modification caveats

[Kevin Locke]

On Sat, 2020-08-22 at 20:22 -0600, Kevin Locke wrote:
> Some overlayfs optional features are incompatible with offline changes
> to the lower tree[1][2][3] and may result in -EXDEV[4], -EIO[5], or
> other errors.  Such modification is not supported and the error behavior
> is intentionally not specified.
> 
> Update the "Changes to underlying filesystems" section to note this
> restriction.  Move the paragraph describing the offline behavior below
> the online behavior so it is adjacent to the following 3 paragraphs
> describing the NFS export offline modification behavior.
> 
> [1]: https://lore.kernel.org/linux-unionfs/20200708142353.GA103536@redhat.com/
> [2]: https://lore.kernel.org/linux-unionfs/CAOQ4uxi23Zsmfb4rCed1n=On0NNA5KZD74jjjeyz+et32sk-gg@mail.gmail.com/
> [3]: https://lore.kernel.org/linux-unionfs/20200817135651.GA637139@redhat.com/
> [4]: https://lore.kernel.org/linux-unionfs/20200709153616.GE150543@redhat.com/
> [5]: https://lore.kernel.org/linux-unionfs/20200812135529.GA122370@kevinolos/
> 
> Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
> ---
> 
> Changes since v1:
> - Actually send to the maintainer this time.
>   (Sorry Miklos Szeredi, not sure what I was thinking on v1!)
> - Rebase onto overlayfs-next of
>   git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git
> 
> As mentioned with v1, I'm still open to collaborating on a combined
> patch if there is still interest in including Vivek's changes to the
> "Sharing and copying layers" section.
> 
> This patch does not mention nfs_export being incompatible with offline
> changes to lower, since the following 3 paragraphs specify the behavior
> (IIUC).  If the behavior of offline modification with nfs_export is
> undefined, should these paragraphs be removed?
> 
> Thanks for considering (again),
> Kevin
> 
>  Documentation/filesystems/overlayfs.rst | 10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)
> 
> diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst
> index fcda5d6ba9ac..9b39374afdbd 100644
> --- a/Documentation/filesystems/overlayfs.rst
> +++ b/Documentation/filesystems/overlayfs.rst
> @@ -467,14 +467,18 @@ summarized in the `Inode properties`_ table above.
>  Changes to underlying filesystems
>  ---------------------------------
>  
> -Offline changes, when the overlay is not mounted, are allowed to either
> -the upper or the lower trees.
> -
>  Changes to the underlying filesystems while part of a mounted overlay
>  filesystem are not allowed.  If the underlying filesystem is changed,
>  the behavior of the overlay is undefined, though it will not result in
>  a crash or deadlock.
>  
> +Offline changes, when the overlay is not mounted, are allowed to the
> +upper tree.  Offline changes to the lower tree are only allowed if the
> +"metadata only copy up", "inode index", and "redirect_dir" features
> +have not been used.  If the lower tree is modified and any of these
> +features has been used, the behavior of the overlay is undefined,
> +though it will not result in a crash or deadlock.
> +
>  When the overlay NFS export feature is enabled, overlay filesystems
>  behavior on offline changes of the underlying lower layer is different
>  than the behavior when NFS export is disabled.

Ping.

I think users would benefit from documenting these restrictions, using
the above patch Vivek's or another.  I'm still open to working out any
issues over how best to do that.

Thanks,
Kevin

Re: [PATCH v2] ovl: document lower modification caveats

[Miklos Szeredi]

On Sun, Aug 23, 2020 at 4:23 AM Kevin Locke <kevin@kevinlocke.name> wrote:
>
> Some overlayfs optional features are incompatible with offline changes
> to the lower tree[1][2][3] and may result in -EXDEV[4], -EIO[5], or
> other errors.  Such modification is not supported and the error behavior
> is intentionally not specified.
>
> Update the "Changes to underlying filesystems" section to note this
> restriction.  Move the paragraph describing the offline behavior below
> the online behavior so it is adjacent to the following 3 paragraphs
> describing the NFS export offline modification behavior.
>
> [1]: https://lore.kernel.org/linux-unionfs/20200708142353.GA103536@redhat.com/
> [2]: https://lore.kernel.org/linux-unionfs/CAOQ4uxi23Zsmfb4rCed1n=On0NNA5KZD74jjjeyz+et32sk-gg@mail.gmail.com/
> [3]: https://lore.kernel.org/linux-unionfs/20200817135651.GA637139@redhat.com/
> [4]: https://lore.kernel.org/linux-unionfs/20200709153616.GE150543@redhat.com/
> [5]: https://lore.kernel.org/linux-unionfs/20200812135529.GA122370@kevinolos/
>
> Signed-off-by: Kevin Locke <kevin@kevinlocke.name>

Thanks, applied.

Miklos