[PATCH] sstate.bbclass: adds warnings for the exceptions of os.utime in siginfo

[PATCH] sstate.bbclass: adds warnings for the exceptions of os.utime in siginfo

[Jose Quaresma]

When we can't update the access and modified times of sstate siginfo
it is useful to infor the user about that.

Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
---
 meta/classes/sstate.bbclass | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 92a73114bb..2deecd8777 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -720,11 +720,15 @@ def sstate_package(ss, d):
         try:
             os.utime(siginfo, None)
         except PermissionError:
+            bb.warn("Cannot sets the access and modified times of sstate siginfo %s, no access rights" % siginfo)
             pass
         except OSError as e:
             # Handle read-only file systems gracefully
             import errno
-            if e.errno != errno.EROFS:
+            if e.errno == errno.EROFS:
+                bb.warn("Cannot sets the access and modified times of sstate siginfo %s, read-only file system" % siginfo)
+                pass
+            else:
                 raise e
 
     return
@@ -1165,11 +1169,15 @@ python sstate_eventhandler() {
             try:
                 os.utime(siginfo, None)
             except PermissionError:
+                bb.warn("Cannot sets the access and modified times of sstate siginfo %s, no access rights" % siginfo)
                 pass
             except OSError as e:
                 # Handle read-only file systems gracefully
                 import errno
-                if e.errno != errno.EROFS:
+                if e.errno == errno.EROFS:
+                    bb.warn("Cannot sets the access and modified times of sstate siginfo %s, read-only file system" % siginfo)
+                    pass
+                else:
                     raise e
 
 }
-- 
2.33.0

Re: [OE-core] [PATCH] sstate.bbclass: adds warnings for the exceptions of os.utime in siginfo

[Richard Purdie]

On Sun, 2021-10-03 at 22:38 +0100, Jose Quaresma wrote:
> When we can't update the access and modified times of sstate siginfo
> it is useful to infor the user about that.
> 
> Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
> ---
>  meta/classes/sstate.bbclass | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> index 92a73114bb..2deecd8777 100644
> --- a/meta/classes/sstate.bbclass
> +++ b/meta/classes/sstate.bbclass
> @@ -720,11 +720,15 @@ def sstate_package(ss, d):
>          try:
>              os.utime(siginfo, None)
>          except PermissionError:
> +            bb.warn("Cannot sets the access and modified times of sstate siginfo %s, no access rights" % siginfo)
>              pass
>          except OSError as e:
>              # Handle read-only file systems gracefully
>              import errno
> -            if e.errno != errno.EROFS:
> +            if e.errno == errno.EROFS:
> +                bb.warn("Cannot sets the access and modified times of sstate siginfo %s, read-only file system" % siginfo)
> +                pass
> +            else:
>                  raise e
>  
>      return
> @@ -1165,11 +1169,15 @@ python sstate_eventhandler() {
>              try:
>                  os.utime(siginfo, None)
>              except PermissionError:
> +                bb.warn("Cannot sets the access and modified times of sstate siginfo %s, no access rights" % siginfo)
>                  pass
>              except OSError as e:
>                  # Handle read-only file systems gracefully
>                  import errno
> -                if e.errno != errno.EROFS:
> +                if e.errno == errno.EROFS:
> +                    bb.warn("Cannot sets the access and modified times of sstate siginfo %s, read-only file system" % siginfo)
> +                    pass
> +                else:
>                      raise e
>  
> 

I'm not sure we can do this as there are some cases where this is unavoidable by
people's state setups and we wouldn't want to show warnings in those cases.

(e.g. think of a shared sstate dir between multiple users where the objects
remain owned by the user that created them).


Cheers,

Richard

Re: [OE-core] [PATCH] sstate.bbclass: adds warnings for the exceptions of os.utime in siginfo

[Jose Quaresma]

[-- Attachment #1: Type: text/plain, Size: 2999 bytes --]

Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia
segunda, 4/10/2021 à(s) 15:11:

> On Sun, 2021-10-03 at 22:38 +0100, Jose Quaresma wrote:
> > When we can't update the access and modified times of sstate siginfo
> > it is useful to infor the user about that.
> >
> > Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
> > ---
> >  meta/classes/sstate.bbclass | 12 ++++++++++--
> >  1 file changed, 10 insertions(+), 2 deletions(-)
> >
> > diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> > index 92a73114bb..2deecd8777 100644
> > --- a/meta/classes/sstate.bbclass
> > +++ b/meta/classes/sstate.bbclass
> > @@ -720,11 +720,15 @@ def sstate_package(ss, d):
> >          try:
> >              os.utime(siginfo, None)
> >          except PermissionError:
> > +            bb.warn("Cannot sets the access and modified times of
> sstate siginfo %s, no access rights" % siginfo)
> >              pass
> >          except OSError as e:
> >              # Handle read-only file systems gracefully
> >              import errno
> > -            if e.errno != errno.EROFS:
> > +            if e.errno == errno.EROFS:
> > +                bb.warn("Cannot sets the access and modified times of
> sstate siginfo %s, read-only file system" % siginfo)
> > +                pass
> > +            else:
> >                  raise e
> >
> >      return
> > @@ -1165,11 +1169,15 @@ python sstate_eventhandler() {
> >              try:
> >                  os.utime(siginfo, None)
> >              except PermissionError:
> > +                bb.warn("Cannot sets the access and modified times of
> sstate siginfo %s, no access rights" % siginfo)
> >                  pass
> >              except OSError as e:
> >                  # Handle read-only file systems gracefully
> >                  import errno
> > -                if e.errno != errno.EROFS:
> > +                if e.errno == errno.EROFS:
> > +                    bb.warn("Cannot sets the access and modified times
> of sstate siginfo %s, read-only file system" % siginfo)
> > +                    pass
> > +                else:
> >                      raise e
> >
> >
>
> I'm not sure we can do this as there are some cases where this is
> unavoidable by
> people's state setups and we wouldn't want to show warnings in those cases.
>
> (e.g. think of a shared sstate dir between multiple users where the objects
> remain owned by the user that created them).
>
>
> Cheers,
>
> Richard
>
>
>
Hi Richard,

I understand the exception for the use cases but I think it would be useful
to show this to the users. Will it be more appropriate perhaps to log this
as debug messages?

I have a final question that I don't understand clearly.
Can the omission of these timestamps updates on siginfo invalidate the use
of the sstate-cache for that task,
bitbake-dumpsig can complain about that?

-- 
Best regards,

José Quaresma

[-- Attachment #2: Type: text/html, Size: 4091 bytes --]

Re: [OE-core] [PATCH] sstate.bbclass: adds warnings for the exceptions of os.utime in siginfo

[Richard Purdie]

On Mon, 2021-10-04 at 18:59 +0100, Jose Quaresma wrote:
> I understand the exception for the use cases but I think it would be useful
> to show this to the users. Will it be more appropriate perhaps to log this as
> debug messages?

The trouble is in the situations that is guarding against, the user cannot do
anything about it so warnings definitely aren't appropriate and I'm not sure
debug messages would be welcome either. Those would be less invasive though.

> I have a final question that I don't understand clearly.
> Can the omission of these timestamps updates on siginfo invalidate the use of
> the sstate-cache for that task,
> bitbake-dumpsig can complain about that?

I think there is a misunderstanding here. The timestamps I've been talking about
are part of reproducible builds and the SOURCE_DATE_EPOCH variable and code.
These aim to make the output identical. If the output is identical, hash
equivalence is more successful and the more successful hash equivalence is, the
more sstate reuse you get.

The timestamps in this part of the sstate code are simply there to handle sstate
artefact aging (e.g. delete things which haven't been used in X weeks). The
timestamp change is therefore "nice" but if it doesn't happen, it isn't a
problem. Hope that helps clarify.

Cheers,

Richard

Re: [OE-core] [PATCH] sstate.bbclass: adds warnings for the exceptions of os.utime in siginfo

[Jose Quaresma]

[-- Attachment #1: Type: text/plain, Size: 1940 bytes --]

Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia
segunda, 4/10/2021 à(s) 21:03:

> On Mon, 2021-10-04 at 18:59 +0100, Jose Quaresma wrote:
> > I understand the exception for the use cases but I think it would be
> useful
> > to show this to the users. Will it be more appropriate perhaps to log
> this as
> > debug messages?
>
> The trouble is in the situations that is guarding against, the user cannot
> do
> anything about it so warnings definitely aren't appropriate and I'm not
> sure
> debug messages would be welcome either. Those would be less invasive
> though.
>

After your explanation, it is not so useful.


> > I have a final question that I don't understand clearly.
> > Can the omission of these timestamps updates on siginfo invalidate the
> use of
> > the sstate-cache for that task,
> > bitbake-dumpsig can complain about that?
>
> I think there is a misunderstanding here. The timestamps I've been talking
> about
> are part of reproducible builds and the SOURCE_DATE_EPOCH variable and
> code.
> These aim to make the output identical. If the output is identical, hash
> equivalence is more successful and the more successful hash equivalence
> is, the
> more sstate reuse you get.


> The timestamps in this part of the sstate code are simply there to handle
> sstate
> artefact aging (e.g. delete things which haven't been used in X weeks). The
> timestamp change is therefore "nice" but if it doesn't happen, it isn't a
> problem. Hope that helps clarify.
>

Yes, it is true I have misunderstood that part of the reproducible builds
and SOURCE_DATE_EPOCH.
It's hard for me to figure out all the pieces involved in hash equivalence
and sstate handling.
One more detail that I understand better.
Your opinions and explanations always help a lot.

Thank you very much.


> Cheers,
>
> Richard
>
>
>

-- 
Best regards,

José Quaresma

[-- Attachment #2: Type: text/html, Size: 3030 bytes --]