Fwd: Bug#974900: dash removes trailing slash from script arguments

Fwd: Bug#974900: dash removes trailing slash from script arguments

[Andrej Shadura]

Hi,

this is another bug report I have received.

----- Original message -----
From: Jeff King <peff@peff.net>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: Bug#974900: dash removes trailing slash from script arguments
Date: Monday, 16 November 2020 10:45

Package: dash
Version: 0.5.11+git20200708+dd9ef66-2
Severity: normal
Tags: upstream

With the latest version of dash, I get this behavior:

  $ touch here
  $ dash -c 'printf "%s\n" "$@"' -- here/ not-here/
  here
  not-here/

The trailing slash is stripped from the argument "here/", when the file
"here" exists in the current directory (but not from "not-here/", which
does not exist).

This is rather surprising to scripts which may not even intend for their
arguments to be files (I noticed because it breaks Git's test suite,
which expects "some-script foo/" to preserve the trailing slash, which
is meaningful in its internal path matching). And certainly it differs
from the behavior of 0.5.10.2-7, which prints "here/".

Bisection points to upstream 7638476 (shell: Enable fnmatch/glob by
default, 2020-05-28). And indeed, building locally with "./configure
--disable-fnmatch" makes the problem go away. But since that commit was
only flipping the defaults, presumably the problem was already there.
Bisecting with "--enable-fnmatch --enable-glob" shows that it comes from
6900ff6 (expand: Fix glibc glob(3) support, 2018-03-26).

-- System Information:
Debian Release: bullseye/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'testing'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.9.0-2-amd64 (SMP w/16 CPU threads)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages dash depends on:
ii  debconf [debconf-2.0]  1.5.74
ii  debianutils            4.11.2
ii  dpkg                   1.20.5
ii  libc6                  2.31-4

dash recommends no packages.

dash suggests no packages.

-- debconf information:
* dash/sh: true

Re: Fwd: Bug#974900: dash removes trailing slash from script arguments

[Herbert Xu]

Andrej Shadura <andrew@shadura.me> wrote:
> 
> this is another bug report I have received.

This is a bug in glob(3).  Please dup and reassign to libc6.  There
is a patch in the queue to disable glob by default again.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Fwd: Bug#974900: dash removes trailing slash from script arguments

[Aurelien Jarno]

On 2020-11-17 14:32, Herbert Xu wrote:
> Andrej Shadura <andrew@shadura.me> wrote:
> > 
> > this is another bug report I have received.
> 
> This is a bug in glob(3).  Please dup and reassign to libc6.  There
> is a patch in the queue to disable glob by default again.
> 

Can you please describe more precisely what is the problem with glob(3)?

Thanks,
Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: Bug#976865: Fwd: Bug#974900: dash removes trailing slash from script arguments

[Herbert Xu]

Aurelien Jarno <aurelien@aurel32.net> wrote:
>
> Can you please describe more precisely what is the problem with glob(3)?

It's stripping trailing slashes from the pattern, even when the
name in question is a regular file.

https://patchwork.kernel.org/project/dash/patch/20201116025222.GA28742@gondor.apana.org.au/

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Bug#976865: Fwd: Bug#974900: dash removes trailing slash from script arguments

[Aurelien Jarno]

On 2020-12-10 13:56, Herbert Xu wrote:
> Aurelien Jarno <aurelien@aurel32.net> wrote:
> >
> > Can you please describe more precisely what is the problem with glob(3)?
> 
> It's stripping trailing slashes from the pattern, even when the
> name in question is a regular file.
> 
> https://patchwork.kernel.org/project/dash/patch/20201116025222.GA28742@gondor.apana.org.au/

That's the dash symptoms. glob(3) takes a pattern and just returns the
paths matching the pattern, as they are named on the filesystem. That
said, the option GLOB_MARK can return a trailing slash for all matched
path that are a directory.

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: Bug#976865: Fwd: Bug#974900: dash removes trailing slash from script arguments

[Herbert Xu]

On Thu, Dec 10, 2020 at 08:58:37AM +0100, Aurelien Jarno wrote:
>
> That's the dash symptoms. glob(3) takes a pattern and just returns the
> paths matching the pattern, as they are named on the filesystem. That
> said, the option GLOB_MARK can return a trailing slash for all matched
> path that are a directory.

Yes but it's really a bug in glob(3).  It should really return
a no-match for the case in question, rather than matching and then
returning a filename without the slash.

IOW the pattern "foo\/" should not match a regular file foo.

Note that the problem doesn't occur for "foo/".

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Fwd: Bug#974900: dash removes trailing slash from script arguments

[Herbert Xu]

On Wed, Dec 09, 2020 at 01:27:17PM +0100, Aurelien Jarno wrote:
>
> Can you please describe more precisely what is the problem with glob(3)?

Create a regular file called "foo", then call glob(3) with the
pattern "foo\/".  This returns a single match with the string
"foo".  This should return no match.

If you change the pattern to "foo/", then it also matches but
returns with the string "foo/" as expected.

The only flag we pass to glob(3) is GLOB_NOMAGIC.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Bug#976865: Fwd: Bug#974900: dash removes trailing slash from script arguments

[Jeff King]

On Thu, Dec 10, 2020 at 10:35:08PM +1100, Herbert Xu wrote:

> Yes but it's really a bug in glob(3).  It should really return
> a no-match for the case in question, rather than matching and then
> returning a filename without the slash.
> 
> IOW the pattern "foo\/" should not match a regular file foo.
> 
> Note that the problem doesn't occur for "foo/".

It seems like it happens for "foo/", too. If I compile:

-- >8 --
#include <glob.h>
#include <stdio.h>

int main(int argc, const char **argv)
{
	while (*++argv) {
		glob_t r;
		if (glob(*argv, 0, NULL, &r))
			perror(*argv);
		else {
			size_t i;
			for (i = 0; i < r.gl_pathc; i++)
				printf("%s -> %s\n", *argv, r.gl_pathv[i]);
			globfree(&r);
		}
	}
	return 0;
}
-- >8 --

I get:

  $ rm -f foo bar
  $ touch foo
  $ ./a.out foo foo/ 'foo\/' bar bar/ 'bar\/'
  foo -> foo
  foo/ -> foo
  foo\/ -> foo
  bar: No such file or directory
  bar/: No such file or directory
  bar\/: No such file or directory

-Peff

Re: Bug#976865: Fwd: Bug#974900: dash removes trailing slash from script arguments

[Herbert Xu]

On Thu, Dec 10, 2020 at 10:20:29AM -0500, Jeff King wrote:
>
> It seems like it happens for "foo/", too. If I compile:

I think the key is that dash uses GLOB_NOMAGIC.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Bug#976865: Fwd: Bug#974900: dash removes trailing slash from script arguments

[Jeff King]

On Fri, Dec 11, 2020 at 01:03:11PM +1100, Herbert Xu wrote:

> On Thu, Dec 10, 2020 at 10:20:29AM -0500, Jeff King wrote:
> >
> > It seems like it happens for "foo/", too. If I compile:
> 
> I think the key is that dash uses GLOB_NOMAGIC.

I wondered that, too, but adding GLOB_NOMAGIC to the call doesn't seem
to change the results. This is all with libc6 2.31-5, by the way.

-Peff

Re: Bug#976865: Fwd: Bug#974900: dash removes trailing slash from script arguments

[Florian Weimer]

* Herbert Xu:

> On Thu, Dec 10, 2020 at 08:58:37AM +0100, Aurelien Jarno wrote:
>>
>> That's the dash symptoms. glob(3) takes a pattern and just returns the
>> paths matching the pattern, as they are named on the filesystem. That
>> said, the option GLOB_MARK can return a trailing slash for all matched
>> path that are a directory.
>
> Yes but it's really a bug in glob(3).  It should really return
> a no-match for the case in question, rather than matching and then
> returning a filename without the slash.
>
> IOW the pattern "foo\/" should not match a regular file foo.

I believe this has been reported upstream here:

  <https://sourceware.org/bugzilla/show_bug.cgi?id=25659>

(But I have not reveiwed this particular bug thread here, sorry.)