Re: failed open: No data available

[Michael Labriola <michael.d.labriola@gmail.com>]

On Tue, Dec 15, 2020 at 11:31 AM Amir Goldstein <amir73il@gmail.com> wrote:
>
> On Tue, Dec 15, 2020 at 5:33 PM Michael D Labriola
> <michael.d.labriola@gmail.com> wrote:
> >
> > On Mon, Dec 14, 2020 at 6:06 PM Michael D Labriola <michael.d.labriola@gmail.com> wrote:
> > >
> > > I'm sporatically getting "no data available" as a reason to fail to
> > > open files on an overlay mount.  Most obvious is during ln of backup
> > > file during apt install.  Only seems to happen on copy_up from lower
>
> How do you know that? Do you have some more tracing info?

I haven't done any tracing, perhaps I overstated.  The problem I'm
seeing only happens when overlayfs goes to create a copy of a lower
layer file in the upper layer.  When the problem occurs, it's always
on a file that exists on lower but not upper, and is about to be
modified.

>
> > > layer.  Lower layer is squashfs (I've seen it happen with both the
> > > default zlib and also zstd compression), upper is EXT4.
> > >
> > > I've only bumped into this problem recently with 5.9+ kernels.  I'm
> > > gonna go see if I can reproduce in some older kernels I still have
> > > installed.
> >
> > Rebooting into 5.4 made the problem go away and I can apt upgrade
> > w/out any problems.  Rebooting an affected virtual machine into 5.8
> > also fixed the problem, so it looks to be something introduced in 5.9.
>
> There are no overlayfs changes v5.8..v5.9 nor squashfs changes.
> Are you sure that your reproducer is reliable enough for the bisection?
> If it is, please try to bisect the offending commit because I have no idea
> where it may be.

I'm having a hard time reproducing the problem.  It's only happening
frequently enough for me to be pretty sure it's a bug.  I've been
using an overlay of squashfs/EXT4 on my development laptop for over a
year, using the squashfs image to fork off disposable virtual machines
for testing.  It's worked flawlessly up until I started testing w/
5.9... but I couldn't correlate my problems to anything specific until
just recently.

More than once now, my host system or a virtual machine has randomly
failed to process an apt update.  Either a backup hardlink creation
fails or some other processing command fails, always with an error
message of "No data available", which makes no sense to me.  Booting
back into my 5.4 or 5.8 kernel and performing the upgrade, then back
into my 5.9 kernel alleviates the problem until it happens again on
some other package.

I have also seen "No data available" pop up in seemingly random
places.  For example, yesterday postfix refused to send mail, and when
I went to restart the service I got this:

postfix/bounce[24836]: fatal: open lock file pid/unix.bounce: cannot
open file: No data available

Today, in 5.9.14, I did an apt upgrade which didn't fail creating
backup files, but instead failed doing a postinstall task like this:

Setting up sudo (1.8.21p2-3ubuntu1.3) ...
chown: changing ownership of '/etc/sudoers': No data available
dpkg: error processing package sudo (--configure):
 installed sudo package post-installation script subprocess returned
error exit status 1

Rebooting the vm resulted in the same problem.  Booting into 5.8.18,
apt upgrade succeeded.  Then I rebooted back into 5.9.

>
> >
> > I suppose I should try 5.10 and see if this problem has already been
> > fixed.
> >
>
> Wouldn't hurt.

Trying that shortly.  Also trying to figure out how to force the
problem to happen...  I'll never get to the bottom of it at this rate.
I was really hoping somebody on the list would recognize the
problem...  :-/  Just my luck.

> Thanks,
> Amir.

-- 
Michael D Labriola
21 Rip Van Winkle Cir
Warwick, RI 02886
401-316-9844 (cell)