Linux-CIFS Archive on lore.kernel.org
 help / color / Atom feed
From: Florian Weimer <fw@deneb.enyo.de>
To: Rich Felker <dalias@libc.org>
Cc: linux-fsdevel@vger.kernel.org, musl@lists.openwall.com,
	linux-kernel@vger.kernel.org, linux-nfs@vger.kernel.org,
	linux-cifs@vger.kernel.org
Subject: Re: [musl] getdents64 lost direntries with SMB/NFS and buffer size < unknown threshold
Date: Wed, 20 Nov 2019 20:57:32 +0100
Message-ID: <8736eiqq1f.fsf@mid.deneb.enyo.de> (raw)
In-Reply-To: <20191120001522.GA25139@brightrain.aerifal.cx> (Rich Felker's message of "Tue, 19 Nov 2019 19:15:22 -0500")

* Rich Felker:

> An issue was reported today on the Alpine Linux tracker at
> https://gitlab.alpinelinux.org/alpine/aports/issues/10960 regarding
> readdir results from SMB/NFS shares with musl libc.
>
> After a good deal of analysis, we determined the root cause to be that
> the second and subsequent calls to getdents64 are dropping/skipping
> direntries (that have not yet been deleted) when some entries were
> deleted following the previous call. The issue appears to happen only
> when the buffer size passed to getdents64 is below some threshold
> greater than 2k (the size musl uses) but less than 32k (the size glibc
> uses, with which we were unable to reproduce the issue).

From the Gitlab issue:

  while ((dp = readdir(dir)) != NULL) {
      unlink(dp->d_name);
      ++file_cnt;
  }

I'm not sure that this is valid code to delete the contents of a
directory.  It's true that POSIX says this:

| If a file is removed from or added to the directory after the most
| recent call to opendir() or rewinddir(), whether a subsequent call
| to readdir() returns an entry for that file is unspecified.

But many file systems simply provide not the necessary on-disk data
structures which are need to ensure stable iteration in the face of
modification of the directory.  There are hacks, of course, such as
compacting the on-disk directory only on file creation, which solves
the file removal case.

For deleting an entire directory, that is not really a problem because
you can stick another loop around this while loop which re-reads the
directory after rewinddir.  Eventually, it will become empty.

  reply index

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-20  0:15 Rich Felker
2019-11-20 19:57 ` Florian Weimer [this message]
2019-11-20 20:59   ` [musl] " Rich Felker
2019-11-21 17:54     ` Theodore Y. Ts'o
2019-12-25 19:38       ` Florian Weimer
2019-12-26  3:56         ` Theodore Y. Ts'o

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=8736eiqq1f.fsf@mid.deneb.enyo.de \
    --to=fw@deneb.enyo.de \
    --cc=dalias@libc.org \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=musl@lists.openwall.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-CIFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-cifs/0 linux-cifs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-cifs linux-cifs/ https://lore.kernel.org/linux-cifs \
		linux-cifs@vger.kernel.org
	public-inbox-index linux-cifs

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-cifs


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git