echo "\\1"?

echo "\\1"?

[Bosco]

According the man pages, for echo command, "\\" should print '\'
character, and \0digits should print the byte in octal base.
But the command

echo "\\1"

outputs the byte 0x01 in hexadecimal (or 001 in octal).
Is this a bad behavior or is intentional?

Bosco.

Re: echo "\\1"?

[Eric Blake]

[-- Attachment #1.1: Type: text/plain, Size: 1439 bytes --]

On 07/27/2017 07:23 AM, Bosco wrote:
> According the man pages,

Which man pages?  Echo is one of those programs that varies widely, and
you are MUCH better off using printf(1) instead of echo(1) if you are
trying to get newline suppression, trying to print something that might
begin with -, or trying to print something that might contain \.

> for echo command, "\\" should print '\'
> character, and \0digits should print the byte in octal base.
> But the command
> 
> echo "\\1"

This is the same as
echo '\1'

which is NOT defined by POSIX as being a valid escape sequence that echo
must recognize.

(Did you mean to test
echo '\\1'
instead?)

Here's the POSIX list of required escape sequences:

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html

> 
> outputs the byte 0x01 in hexadecimal (or 001 in octal).
> Is this a bad behavior or is intentional?

Arguably, since it is not required by POSIX, we don't have to do it. But
I also can't argue that POSIX forbids us to support \1 as an extension
(it says nothing about whether implementations can have additional
escape sequences).  So I'll argue that it is intentional as a dash
extension.  But if you can make dash smaller by getting rid of the
extension, that might be an acceptable patch.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: echo "\\1"?

[Bosco]

On 27 July 2017 at 12:54, Eric Blake <eblake@redhat.com> wrote:
> Which man pages?  Echo is one of those programs that varies widely, and
> you are MUCH better off using printf(1) instead of echo(1) if you are
> trying to get newline suppression, trying to print something that might
> begin with -, or trying to print something that might contain \.

Sorry, maybe I did't explain it correctly, I mean the man pages of the
dash source:
https://git.kernel.org/pub/scm/utils/dash/dash.git/tree/src/dash.1#n1202

And because of this, I got an error compiling zziplib, you may see
https://github.com/gdraheim/zziplib/blob/v0.13.67/configure#L17542

>
>> for echo command, "\\" should print '\'
>> character, and \0digits should print the byte in octal base.
>> But the command
>>
>> echo "\\1"
>
> This is the same as
> echo '\1'
>
> which is NOT defined by POSIX as being a valid escape sequence that echo
> must recognize.
>
> (Did you mean to test
> echo '\\1'
> instead?)
>

I refer the following commands

echo "\\1"

or

echo \\1



> Here's the POSIX list of required escape sequences:
>
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html
>

There, it says the same as the man page of dash.

I understand that \\ is converted to '\' character.

> Arguably, since it is not required by POSIX, we don't have to do it. But
> I also can't argue that POSIX forbids us to support \1 as an extension
> (it says nothing about whether implementations can have additional
> escape sequences).  So I'll argue that it is intentional as a dash
> extension.  But if you can make dash smaller by getting rid of the
> extension, that might be an acceptable patch.

In that case, I think, the man page of dash should be modified with
that extension.

Thank you.
Bosco.

Re: echo "\\1"?

[Eric Blake]

[-- Attachment #1.1: Type: text/plain, Size: 2502 bytes --]

On 07/27/2017 08:13 AM, Bosco wrote:
> On 27 July 2017 at 12:54, Eric Blake <eblake@redhat.com> wrote:
>> Which man pages?  Echo is one of those programs that varies widely, and
>> you are MUCH better off using printf(1) instead of echo(1) if you are
>> trying to get newline suppression, trying to print something that might
>> begin with -, or trying to print something that might contain \.
> 
> Sorry, maybe I did't explain it correctly, I mean the man pages of the
> dash source:
> https://git.kernel.org/pub/scm/utils/dash/dash.git/tree/src/dash.1#n1202
> 
> And because of this, I got an error compiling zziplib, you may see
> https://github.com/gdraheim/zziplib/blob/v0.13.67/configure#L17542

Eww - storing generated files in git - that forces everyone that checks
out your project to use the EXACT same version of autotools to avoid
changing the generated files unintentionally.

Looking at those lines:

>   if test -f $ac_prefix_conf_INP ; then
>     echo "s/^#undef  *\\([ABCDEFGHIJKLMNOPQRSTUVWXYZ_]\\)/#undef $ac_prefix_conf_UPP""_\\1/" > conftest.prefix

ac_prefix_conf_INP is not defined anywhere in autoconf 2.69 sources (and
you really shouldn't use the ac_ prefix if you are writing code that is
not part of autoconf proper).  I couldn't find mention of it at
https://github.com/gdraheim/zziplib/blob/v0.13.67/configure.ac, but it
may be in one of your other included files.  Can you pinpoint which part
of your configure.ac results in that part of the generated configure
file?  In all likelihood, you are using a buggy macro that is using
autoconf primitives incorrectly, and thus resulting in non-portable
code.  But without seeing the true source, I can't help you debug your
problem.

>> Arguably, since it is not required by POSIX, we don't have to do it. But
>> I also can't argue that POSIX forbids us to support \1 as an extension
>> (it says nothing about whether implementations can have additional
>> escape sequences).  So I'll argue that it is intentional as a dash
>> extension.  But if you can make dash smaller by getting rid of the
>> extension, that might be an acceptable patch.
> 
> In that case, I think, the man page of dash should be modified with
> that extension.

Indeed, or the fact that it is NOT documented means that it is an
unintentional bug for providing the extension.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: echo "\\1"?

[Bosco]

I don't have any problem, I only show you one case where I had to fix
an script in order to work a compilation in dash.

I'm using dash because of simplicity, unfortunately I don't understand
how things work.

I think the problem is of dash, I only want to report that dash fails
at parse scape sequence.
For instance, the command

  echo \\\\

prints

  \

I think it should print
\\

I don't have to decided how dash should understand things because it
isn't my project. I only advise of a bad behavior, of course from my
point of view.

Bosco.

Re: echo "\\1"?

[Eric Blake]

[-- Attachment #1.1: Type: text/plain, Size: 1358 bytes --]

On 07/27/2017 09:44 AM, Bosco wrote:
> I don't have any problem, I only show you one case where I had to fix
> an script in order to work a compilation in dash.
> 
> I'm using dash because of simplicity, unfortunately I don't understand
> how things work.
> 
> I think the problem is of dash, I only want to report that dash fails
> at parse scape sequence.
> For instance, the command
> 
>   echo \\\\
> 
> prints
> 
>   \

Correct, per POSIX.  Incorrect, for portable code (since not all echo
implementations obey POSIX; many require to type

echo -e \\\\

to get single output, but -e is not POSIX).

> 
> I think it should print
> \\

Wrong per POSIX, but it matches what you get on bash if you don't turn
on 'shopt -s xpg_echo' and 'set -o posix' simultaneously (yes, bash has
two separate knobs that BOTH have to be on to get POSIX behavior).

> 
> I don't have to decided how dash should understand things because it
> isn't my project. I only advise of a bad behavior, of course from my
> point of view.

The bad behavior is not in dash, but in your non-portable use of echo
with a \ in the argument.  Use printf instead, if you want your script
to work in more than one shell.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: echo "\\1"?

[Bosco]

That script of zziplib isn't mine, I only had to compile it once
because it was necessary for compile other program (TeX Live).

I'm not talking about POSIX, and I don't mind what it said. I'm
talking about the man page of dash, that said:

>  echo [-n] args...
>            Print the arguments on the standard output, separated by spaces.
>            Unless the -n  option is present, a newline is output following the
>            arguments.
>

According to

>            If any of the following sequences of characters is encountered
>            during output, the sequence is not output.  Instead, the specified
>            action is performed:

when \\ is reached is replaced by \.

>
>            \b      A backspace character is output.
>
>            \c      Subsequent output is suppressed.  This is normally used at
>                    the end of the last argument to suppress the trailing
>                    newline that e ec ch ho o would otherwise output.
>
>            \f      Output a form feed.
>
>            \n      Output a newline character.
>
>            \r      Output a carriage return.
>
>            \t      Output a (horizontal) tab character.
>
>            \v      Output a vertical tab.
>
>            \0digits
>                    Output the character whose value is given by zero to three
>                    octal digits.  If there are zero digits, a nul character
>                    is output.
>
>            \\      Output a backslash.
>
>            All other backslash sequences elicit undefined behaviour.
>

Then, in the command
  echo \\\\
because \\ is reached first, then it will be replaced by '\'
character, immediately after that another \\ is reached, then it will
be replaced by another '\' character. It turns out the ouput '\\'.

Bosco.

Re: echo "\\1"?

[Eric Blake]

[-- Attachment #1.1: Type: text/plain, Size: 1315 bytes --]

On 07/27/2017 10:10 AM, Bosco wrote:
> That script of zziplib isn't mine, I only had to compile it once
> because it was necessary for compile other program (TeX Live).
> 
> I'm not talking about POSIX, and I don't mind what it said. I'm
> talking about the man page of dash, that said:
> 

> when \\ is reached is replaced by \.

When \\ is reached AS THE ARGUMENT to echo.

> Then, in the command
>   echo \\\\
> because \\ is reached first, then it will be replaced by '\'

No, you are demonstrating a gap in your understanding of shell quoting
rules.

echo \\\\
echo "\\\\"
echo '\\'
echo '\'"\\"

are all the same way to pass the two-character argument to echo.  That
two-character argument is a valid escape sequence, which in turn means
echo outputs a single \ character then a newline.

> character, immediately after that another \\ is reached, then it will
> be replaced by another '\' character. It turns out the ouput '\\'.

If you want two \ as output, you have to pass four characters (not two)
to echo, so your input has to be one of these (or other) valid quotings:

echo '\\\\'
echo "\\\\\\\\"
echo \\'\'"\\"\\'\'"\\"\\'\'
etc.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: echo "\\1"?

[Bosco]

Ahh, now I understood.
In command
  echo \\\\
first dash interprets the command line (and the arguments), then
internally echo interprets the arguments. But echo is part of dash, so
dash interprets twice the same argument. Sorry I didn't know that.

Thank you for your help and your time.
Bosco.

Re: echo "\\1"?

[Harald van Dijk]

On 27/07/2017 16:24, Eric Blake wrote:
> On 07/27/2017 08:13 AM, Bosco wrote:
>> On 27 July 2017 at 12:54, Eric Blake <eblake@redhat.com> wrote:
>>> Which man pages?  Echo is one of those programs that varies widely, and
>>> you are MUCH better off using printf(1) instead of echo(1) if you are
>>> trying to get newline suppression, trying to print something that might
>>> begin with -, or trying to print something that might contain \.
>>
>> Sorry, maybe I did't explain it correctly, I mean the man pages of the
>> dash source:
>> https://git.kernel.org/pub/scm/utils/dash/dash.git/tree/src/dash.1#n1202
>>
>> And because of this, I got an error compiling zziplib, you may see
>> https://github.com/gdraheim/zziplib/blob/v0.13.67/configure#L17542
> 
> Eww - storing generated files in git - that forces everyone that checks
> out your project to use the EXACT same version of autotools to avoid
> changing the generated files unintentionally.
> 
> Looking at those lines:
> 
>>    if test -f $ac_prefix_conf_INP ; then
>>      echo "s/^#undef  *\\([ABCDEFGHIJKLMNOPQRSTUVWXYZ_]\\)/#undef $ac_prefix_conf_UPP""_\\1/" > conftest.prefix
> 
> ac_prefix_conf_INP is not defined anywhere in autoconf 2.69 sources (and
> you really shouldn't use the ac_ prefix if you are writing code that is
> not part of autoconf proper).  I couldn't find mention of it at
> https://github.com/gdraheim/zziplib/blob/v0.13.67/configure.ac, but it
> may be in one of your other included files.  Can you pinpoint which part
> of your configure.ac results in that part of the generated configure
> file?  In all likelihood, you are using a buggy macro that is using
> autoconf primitives incorrectly, and thus resulting in non-portable
> code.  But without seeing the true source, I can't help you debug your
> problem.

This is coming from zziplib's outdated copy of ax_prefix_config_h.m4, 
for which I submitted a patch to make it work in dash, and which was 
accepted, back in 2010 (!): <https://savannah.gnu.org/patch/?7317>

zziplib should update to a current version of this file.

>>> Arguably, since it is not required by POSIX, we don't have to do it. But
>>> I also can't argue that POSIX forbids us to support \1 as an extension
>>> (it says nothing about whether implementations can have additional
>>> escape sequences).  So I'll argue that it is intentional as a dash
>>> extension.  But if you can make dash smaller by getting rid of the
>>> extension, that might be an acceptable patch.
>>
>> In that case, I think, the man page of dash should be modified with
>> that extension.
> 
> Indeed, or the fact that it is NOT documented means that it is an
> unintentional bug for providing the extension.

The man page of dash states:

   All other backslash sequences elicit undefined behaviour.

Implementation-wise, POSIX gives the same requirements for escape 
sequences for echo as for printf %b, therefore it makes sense to share 
the code between the two. Neither echo nor printf %b requires \1 to be 
treated as an escape sequence, but both allow it.

bash treats \1 as an escape sequence with printf %b. For compatibility 
with bash, it sort of makes sense to implement this in dash as well. Yet 
for some reason, bash *doesn't* treat \1 as an escape sequence with 
echo, not even with echo -e.

Keeping dash small, continuing to share the code between the two cases, 
can only mean behaving differently from bash in at least one of them.

Cheers,
Harald van Dijk