builtin read -r and backslash-char escape sequences

builtin read -r and backslash-char escape sequences

[John Marshall]

Problems with one of my scripts appear to have been caused by dash's read -r translating escape sequences (like \t) whereas several other shells read them literally.  For example:

$ printf '%s' '\a\t\x' > backslashes 
$ dash -c 'read -r foo < backslashes; echo "$foo"' | cat -t
^G^I\x
$ bash -c 'read -r foo < backslashes; echo "$foo"' | cat -t
\a\t\x
$ ksh -c 'read -r foo < backslashes; echo "$foo"' | cat -t
\a\t\x

POSIX says of -r, "Do not treat a <backslash> character in any special way. Consider each <backslash> to be part of the input line" [1].  Translating them as escape sequences doesn't appear to be particularly compatible with this, but conceivably the translation is occurring at some other stage.

Is this the intended behaviour?

Thanks,

    John

[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/read.html#tag_20_109_04

-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE. 

Re: builtin read -r and backslash-char escape sequences

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1313 bytes --]

On 08/03/2015 10:37 AM, John Marshall wrote:
> Problems with one of my scripts appear to have been caused by dash's read -r translating escape sequences (like \t) whereas several other shells read them literally.  For example:
> 
> $ printf '%s' '\a\t\x' > backslashes 
> $ dash -c 'read -r foo < backslashes; echo "$foo"' | cat -t
> ^G^I\x
> $ bash -c 'read -r foo < backslashes; echo "$foo"' | cat -t
> \a\t\x
> $ ksh -c 'read -r foo < backslashes; echo "$foo"' | cat -t
> \a\t\x
> 
> POSIX says of -r, "Do not treat a <backslash> character in any special way. Consider each <backslash> to be part of the input line" [1].  Translating them as escape sequences doesn't appear to be particularly compatible with this, but conceivably the translation is occurring at some other stage.

Bingo. The "some other stage" is your mistaken use of 'echo "$foo"'.
echo is not portable with backslashes.

$ bash -c 'shopt -s xpg_echo; read -r foo < backslashes; echo "$foo"' |
cat -t
^G^I\x
$ dash -c 'read -r foo < backslashes; printf %s\\n "$foo"' | cat -t
\a\t\x

> 
> Is this the intended behaviour?

No bug here except in your usage of echo when you should have been using
printf.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: builtin read -r and backslash-char escape sequences

[Harald van Dijk]

On 03/08/2015 18:37, John Marshall wrote:
> Problems with one of my scripts appear to have been caused by dash's read -r translating escape sequences (like \t) whereas several other shells read them literally.  For example:
>
> $ printf '%s' '\a\t\x' > backslashes
> $ dash -c 'read -r foo < backslashes; echo "$foo"' | cat -t
> ^G^I\x
> $ bash -c 'read -r foo < backslashes; echo "$foo"' | cat -t
> \a\t\x
> $ ksh -c 'read -r foo < backslashes; echo "$foo"' | cat -t
> \a\t\x

You're using echo to print what gets assigned to foo, but backslashes 
are not portable with echo. You probably noticed that already since 
you're using printf to determine what gets saved in the backslashes 
file. If you also use printf "%s\n" "$foo", you'll see that read works 
as you expect.

Cheers,
Harald van Dijk

Re: builtin read -r and backslash-char escape sequences

[John Marshall]

On 3 Aug 2015, at 18:03, Harald van Dijk wrote:
> On 03/08/2015 18:37, John Marshall wrote:
>> Problems with one of my scripts appear to have been caused by dash's read -r translating escape sequences (like \t) whereas several other shells read them literally.  For example:
>> 
>> $ printf '%s' '\a\t\x' > backslashes
>> $ dash -c 'read -r foo < backslashes; echo "$foo"' | cat -t
>> ^G^I\x
> 
> You're using echo to print what gets assigned to foo, but backslashes are not portable with echo. You probably noticed that already since you're using printf to determine what gets saved in the backslashes file.

D'oh!  Indeed I noticed that when the backslashes were right there, but to really internalise it I guess one has to find out the hard way -- which I have now done.  The real script eventually evals the line rather than echoing it, but there's a cmd=`echo $cmd | sed ...` along the way that does the damage.

Thanks to both of you,

    John

-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.