* dash 0.5.12 parameter expansion using classes not working
@ 2023-02-08 5:14 Brian Inglis
2023-02-08 8:43 ` Harald van Dijk
0 siblings, 1 reply; 3+ messages in thread
From: Brian Inglis @ 2023-02-08 5:14 UTC (permalink / raw)
To: Herbert Xu; +Cc: Brian Inglis
[-- Attachment #1: Type: text/plain, Size: 1119 bytes --]
Hi folks,
Under dash 0.5.12 parameter expansion using regexp classes [[:space:]] to trim
no longer works, but releases up to 0.5.11.5 still work correctly with the same
environment and default config options.
In the attached dash script and logs, one of our users has an abstruse string
they use to clean up build command lines, but I trimmed that down to the
locale-dependent [[:space:]] regexp, then to the ASCII whitespace characters
generated by echo, and only the last still works with 0.5.12, all work under
0.5.11.5.
Nothing obvious jumps out from the commitdiffs between then and now, so I am
hoping some possible cause occurs to you.
Build environment was Cygwin+newlib autoreconf with autoconf 2.71, autogen
5.18.16, automake 1.16.5, gcc 11.3, binutils 2.29, coreutils 9.0, dash 0.5.11.5
as sh.
--
Take care. Thanks, Brian Inglis Calgary, Alberta, Canada
La perfection est atteinte Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut
-- Antoine de Saint-Exupéry
[-- Attachment #2: par-ext-t.sh --]
[-- Type: text/plain, Size: 802 bytes --]
#!/bin/dash -vx
trim() {
# remove leading whitespace characters
__trim="${*#${*%%[![:space:]]*}}"
# remove trailing whitespace characters
printf "%s" "${__trim%${__trim##*[![:space:]]}}"
}
trim " alpha" | od -t x1a
trim "beta " | od -t x1a
trim "\rgamma " | od -t x1a
trim2() {
# remove leading whitespace characters
__trim="${*##[[:space:]]}"
# remove trailing whitespace characters
printf "%s" "${__trim%%[[:space:]]}"
}
trim2 " alpha" | od -t x1a
trim2 "beta " | od -t x1a
trim2 "\rgamma " | od -t x1a
trim3() {
_space="$(echo -n '[\b\t\n\v\f\r ]')"
# remove leading whitespace characters
__trim="${*##$_space}"
# remove trailing whitespace characters
printf "%s" "${__trim%%$_space}"
}
trim3 " alpha" | od -t x1a
trim3 "beta " | od -t x1a
trim3 "\rgamma " | od -t x1a
[-- Attachment #3: test11.log --]
[-- Type: text/plain, Size: 2026 bytes --]
#!/bin/dash -vx
trim() {
# remove leading whitespace characters
__trim="${*#${*%%[![:space:]]*}}"
# remove trailing whitespace characters
printf "%s" "${__trim%${__trim##*[![:space:]]}}"
}
trim " alpha" | od -t x1a
+ trim alpha
+ __trim=alpha
+ printf %s alpha
+ od -t x1a
0000000 61 6c 70 68 61
a l p h a
0000005
trim "beta " | od -t x1a
+ trim beta
+ __trim=beta
+ printf %s beta
+ od -t x1a
0000000 62 65 74 61
b e t a
0000004
trim "\rgamma " | od -t x1a
+ trim \rgamma
+ __trim=gamma
+ printf %s gamma
+ od -t x1a
0000000 67 61 6d 6d 61
g a m m a
0000005
trim2() {
# remove leading whitespace characters
__trim="${*##[[:space:]]}"
# remove trailing whitespace characters
printf "%s" "${__trim%%[[:space:]]}"
}
trim2 " alpha" | od -t x1a
+ trim2 alpha
+ __trim=alpha
+ printf %s alpha
+ od -t x1a
0000000 61 6c 70 68 61
a l p h a
0000005
trim2 "beta " | od -t x1a
+ trim2 beta
+ __trim=beta
+ printf %s beta
+ od -t x1a
0000000 62 65 74 61
b e t a
0000004
trim2 "\rgamma " | od -t x1a
+ trim2 \rgamma
+ __trim=gamma
+ printf %s gamma
+ od -t x1a
0000000 67 61 6d 6d 61
g a m m a
0000005
trim3() {
_space="$(echo -n '[\b\t\n\v\f\r ]')"
# remove leading whitespace characters
__trim="${*##$_space}"
# remove trailing whitespace characters
printf "%s" "${__trim%%$_space}"
}
trim3 " alpha" | od -t x1a
+ trim3 alpha
+ od -t x1a
+ echo -n [\b\t\n\v\f\r ]
+ _space=[\b
\v\f\r ]
+ __trim=alpha
+ printf %s alpha
0000000 61 6c 70 68 61
a l p h a
0000005
trim3 "beta " | od -t x1a
+ trim3 beta
+ od -t x1a
+ echo -n [\b\t\n\v\f\r ]
+ _space=[\b
\v\f\r ]
+ __trim=beta
+ printf %s beta
0000000 62 65 74 61
b e t a
0000004
trim3 "\rgamma " | od -t x1a
+ trim3 \rgamma
+ + od -t x1a
echo -n [\b\t\n\v\f\r ]
+ _space=[\b
\v\f\r ]
+ __trim=gamma
+ printf %s gamma
0000000 67 61 6d 6d 61
g a m m a
0000005
[-- Attachment #4: test12.log --]
[-- Type: text/plain, Size: 1874 bytes --]
#!/bin/dash -vx
trim() {
# remove leading whitespace characters
__trim="${*#${*%%[![:space:]]*}}"
# remove trailing whitespace characters
printf "%s" "${__trim%${__trim##*[![:space:]]}}"
}
trim " alpha" | od -t x1a
+ trim alpha
+ __trim=
+ printf %s
+ od -t x1a
0000000
trim "beta " | od -t x1a
+ trim beta
+ __trim=
+ printf %s
+ od -t x1a
0000000
trim "\rgamma " | od -t x1a
+ trim \rgamma
+ __trim=
+ printf %s
+ od -t x1a
0000000
trim2() {
# remove leading whitespace characters
__trim="${*##[[:space:]]}"
# remove trailing whitespace characters
printf "%s" "${__trim%%[[:space:]]}"
}
trim2 " alpha" | od -t x1a
+ trim2 alpha
+ __trim= alpha
+ printf %s alpha
+ od -t x1a
0000000 20 61 6c 70 68 61
sp a l p h a
0000006
trim2 "beta " | od -t x1a
+ trim2 beta
+ __trim=beta
+ printf %s beta
+ od -t x1a
0000000 62 65 74 61 20
b e t a sp
0000005
trim2 "\rgamma " | od -t x1a
+ trim2 \rgamma
+ __trim=\rgamma
+ printf %s \rgamma
+ od -t x1a
0000000 0d 67 61 6d 6d 61 09
cr g a m m a ht
0000007
trim3() {
_space="$(echo -n '[\b\t\n\v\f\r ]')"
# remove leading whitespace characters
__trim="${*##$_space}"
# remove trailing whitespace characters
printf "%s" "${__trim%%$_space}"
}
trim3 " alpha" | od -t x1a
+ trim3 alpha
+ od -t x1a
+ echo -n [\b\t\n\v\f\r ]
+ _space=[\b
\v\f\r ]
+ __trim=alpha
+ printf %s alpha
0000000 61 6c 70 68 61
a l p h a
0000005
trim3 "beta " | od -t x1a
+ trim3 beta
+ od -t x1a
+ echo -n [\b\t\n\v\f\r ]
+ _space=[\b
\v\f\r ]
+ __trim=beta
+ printf %s beta
0000000 62 65 74 61
b e t a
0000004
trim3 "\rgamma " | od -t x1a
+ trim3 \rgamma
+ od -t x1a
+ echo -n [\b\t\n\v\f\r ]
+ _space=[\b
\v\f\r ]
+ __trim=gamma
+ printf %s gamma
0000000 67 61 6d 6d 61
g a m m a
0000005
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: dash 0.5.12 parameter expansion using classes not working
2023-02-08 5:14 dash 0.5.12 parameter expansion using classes not working Brian Inglis
@ 2023-02-08 8:43 ` Harald van Dijk
2023-02-08 13:56 ` Brian Inglis
0 siblings, 1 reply; 3+ messages in thread
From: Harald van Dijk @ 2023-02-08 8:43 UTC (permalink / raw)
To: Brian.Inglis; +Cc: DASH shell mailing list
On 08/02/2023 05:14, Brian Inglis wrote:
> Hi folks,
>
> Under dash 0.5.12 parameter expansion using regexp classes [[:space:]]
> to trim no longer works, but releases up to 0.5.11.5 still work
> correctly with the same environment and default config options.
>
> In the attached dash script and logs, one of our users has an abstruse
> string they use to clean up build command lines, but I trimmed that down
> to the locale-dependent [[:space:]] regexp, then to the ASCII whitespace
> characters generated by echo, and only the last still works with 0.5.12,
> all work under 0.5.11.5.
>
> Nothing obvious jumps out from the commitdiffs between then and now, so
> I am hoping some possible cause occurs to you.
>
> Build environment was Cygwin+newlib autoreconf with autoconf 2.71,
> autogen 5.18.16, automake 1.16.5, gcc 11.3, binutils 2.29, coreutils
> 9.0, dash 0.5.11.5 as sh.
Hi,
dash 0.5.12 includes commit 7638476c
<https://git.kernel.org/pub/scm/utils/dash/dash.git/commit/?id=7638476c18f2a8451b05d49f4e767ee1f98cfc17>
which enables the use of libc's fnmatch() by default. Looking at newlib
sources,
<https://github.com/eblot/newlib/blob/master/newlib/libc/posix/fnmatch.c>,
it appears that newlib's fnmatch() does not support character classes
and should not be used for dash. Would you be able to check that
explicitly passing --disable-fnmatch to the configure options gets it
working properly again?
Cheers,
Harald van Dijk
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: dash 0.5.12 parameter expansion using classes not working
2023-02-08 8:43 ` Harald van Dijk
@ 2023-02-08 13:56 ` Brian Inglis
0 siblings, 0 replies; 3+ messages in thread
From: Brian Inglis @ 2023-02-08 13:56 UTC (permalink / raw)
To: Harald van Dijk; +Cc: DASH shell mailing list
On 2023-02-08 01:43, Harald van Dijk wrote:
> On 08/02/2023 05:14, Brian Inglis wrote:
>> Under dash 0.5.12 parameter expansion using regexp classes [[:space:]] to trim
>> no longer works, but releases up to 0.5.11.5 still work correctly with the
>> same environment and default config options.
>>
>> In the attached dash script and logs, one of our users has an abstruse string
>> they use to clean up build command lines, but I trimmed that down to the
>> locale-dependent [[:space:]] regexp, then to the ASCII whitespace characters
>> generated by echo, and only the last still works with 0.5.12, all work under
>> 0.5.11.5.
>>
>> Nothing obvious jumps out from the commitdiffs between then and now, so I am
>> hoping some possible cause occurs to you.
>>
>> Build environment was Cygwin+newlib autoreconf with autoconf 2.71, autogen
>> 5.18.16, automake 1.16.5, gcc 11.3, binutils 2.29, coreutils 9.0, dash
>> 0.5.11.5 as sh.
> dash 0.5.12 includes commit 7638476c
> <https://git.kernel.org/pub/scm/utils/dash/dash.git/commit/?id=7638476c18f2a8451b05d49f4e767ee1f98cfc17>
> which enables the use of libc's fnmatch() by default. Looking at newlib
> sources,
> <https://github.com/eblot/newlib/blob/master/newlib/libc/posix/fnmatch.c>,
> it appears that newlib's fnmatch() does not support character classes and
> should not be used for dash. Would you be able to check that explicitly
> passing --disable-fnmatch to the configure options gets it working properly
> again?
Thanks very much Harald,
That indeed was the cause and solution - test release on its way to mirrors.
Cygwin/newlib is tricky in that it may extend newlib with hosted features, some
implemented with C++, so there is sometimes another source to check for on the
same servers hosting automake, binutils, gdb, glibc, valgrind, etc.:
https://sourceware.org/git/?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/libc/fnmatch.c
https://sourceware.org/git/?p=newlib-cygwin.git;a=blob;f=newlib/libc/posix/fnmatch.c
https://sourceware.org/git/?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/glob.cc
https://sourceware.org/git/?p=newlib-cygwin.git;a=blob;f=newlib/libc/posix/glob.c
Also tricky is that newlib is mainly BSD or similarly licensed for use by and in
commercial embedded tool chains and RTOSes whereas Cygwin is mainly LGPL and
packages GPL licensed.
[I noticed that dash src/compile sets up Cygwin to use cygpath -m to produce
Windows file names, and patched that out in our build.
If anyone reports anything under Cygwin being non-POSIX conforming, or behaving
differently to any most Unix platforms, please have them report that to the
cygwin@cygwin.com public mailing list (no subscription required) or possibly
discuss on the Libera.Chat IRC channel #cygwin, where it may be addressed.
This does not apply to Cygwin hosted Mingw64 or Msys2 cross-build toolchains.
Few Cygwin packages now have any support for using Windows features or file names.
That handling is hidden way down in the emulation shared library where possible.
All sources and tests should build and run identically to Linux, *BSDs, Solaris.
Features like Windows text files and file path names now mostly cause grief,
unless they happen to end up being passed down verbatim to the correct part of
the emulation library where they are taken care of.]
--
Take care. Thanks, Brian Inglis Calgary, Alberta, Canada
La perfection est atteinte Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut
-- Antoine de Saint-Exupéry
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2023-02-08 13:56 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-08 5:14 dash 0.5.12 parameter expansion using classes not working Brian Inglis
2023-02-08 8:43 ` Harald van Dijk
2023-02-08 13:56 ` Brian Inglis
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.