Re: dash drops exported bash functions

Re: dash drops exported bash functions

[Makarius]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1967 bytes --]

On 02/10/2016 08:54:40 Eric Blake wrote:

> That said, preserving any unusable environment variables unchanged,
> rather than scrubbing them, may be slightly nicer behavior, but I'm not
> sure it's worth the bloat to dash to do so.

> > Exporting bash functions via the environment might be a rarely used 
> > feature, but it is used in practice, unfortunately (otherwise I 
> > wouldn’t have noticed this).
>
> Exporting bash functions is only usable if you plan on directly invoking 
> bash.  Don't drag dash into the mess.  Inserting a dash child in between 
> a bash parent and grandchild means all bets are off for whether the 
> grandparent can export anything to the grandchild.

Eric,

I am a long-term user of GNU bash who is depending on the "export -f" 
feature of that shell (in an application that is on the free market for 
decades). The bash guys turn a shell function "foo" into the environment 
variable "BASH_FUNC_foo%%" and hope to be able to pick it up later on.

Dash gets into this game, because Ubuntu and Debian have chosen to make it 
the default for /bin/sh some years ago. This means that typical "system" 
invocations of Unix tools and libraries are now going through dash: 
/bin/sh -c is often seen in practice instead of more delicate execve 
invocations.

This means with /bin/sh -> dash users have no proper chance to avoid it. 
Sitting right there in the center /bin/sh, dash acquires special 
responsibilities to play nice with other shells.


Note that the issue has come up now due to this recent change in 
Debian-testing: https://tracker.debian.org/news/744916

Thus a the following change from 2012 became exposed to testers/users of 
Debian: 
http://git.kernel.org/cgit/utils/dash/dash.git/commit/?id=46d3c1a614f11f0d40a7e73376359618ff07abcd

In that change 46d3c1a614f, the original motivation was to make "export 
-p" work more robustly in dash, and not to cripple export -f in bash 
because of POSIX violations.


 	Makarius

Re: dash drops exported bash functions

[Makarius]

On Thu, 11 Feb 2016, Makarius wrote:

> I am a long-term user of GNU bash who is depending on the "export -f" feature 
> of that shell (in an application that is on the free market for decades). The 
> bash guys turn a shell function "foo" into the environment variable 
> "BASH_FUNC_foo%%" and hope to be able to pick it up later on.
>
> Dash gets into this game, because Ubuntu and Debian have chosen to make it 
> the default for /bin/sh some years ago. This means that typical "system" 
> invocations of Unix tools and libraries are now going through dash: /bin/sh 
> -c is often seen in practice instead of more delicate execve invocations.
>
> This means with /bin/sh -> dash users have no proper chance to avoid it. 
> Sitting right there in the center /bin/sh, dash acquires special 
> responsibilities to play nice with other shells.

After reading the official (!) sources of bash-4.3, I've found out the 
following: Not the bash guys are introducing this ill-formed name 
decoration "%%", but the Debian guys. See 
http://sourcesdev.debian.net/patches/bash/4.3-14/bash43-027.diff

This means dash is not working against bash, but against Debian. Or 
rather: Debian is working against themselves and their users.


 	Makarius

Re: dash drops exported bash functions

[Makarius]

On Thu, 11 Feb 2016, Makarius wrote:

> After reading the official (!) sources of bash-4.3, I've found out the 
> following: Not the bash guys are introducing this ill-formed name 
> decoration "%%", but the Debian guys. See 
> http://sourcesdev.debian.net/patches/bash/4.3-14/bash43-027.diff
>
> This means dash is not working against bash, but against Debian. Or 
> rather: Debian is working against themselves and their users.

Yet another update.

On another mailing list, someone else pointed out that this change might 
be due to the "shell shock" desaster from 2014: 
https://en.wikipedia.org/wiki/Shellshock_%28software_bug%29

If the malformedness of environment variables that are meant to be 
functions is important for bash43-027.diff above, the ball might be back 
on the side of dash.

In any case, the Debian guys need to get involved, but I am myself not a 
member of that society.


 	Makarius

Re: dash drops exported bash functions

[Olof Johansson]

On 2016-02-11 14:40 +0100, Makarius wrote:
> In any case, the Debian guys need to get involved, but I am myself
> not a member of that society.

Why? It's an upstream patch, it's not Debian specific.

 https://ftp.gnu.org/gnu/bash/bash-4.3-patches/bash43-027

Adapt your software instead?

-- 
Olof Johansson                            https://github.com/olof

Re: dash drops exported bash functions

[Makarius]

On Thu, 11 Feb 2016, Olof Johansson wrote:

> On 2016-02-11 14:40 +0100, Makarius wrote:
>> In any case, the Debian guys need to get involved, but I am myself
>> not a member of that society.
>
> Why? It's an upstream patch, it's not Debian specific.
>
> https://ftp.gnu.org/gnu/bash/bash-4.3-patches/bash43-027

Correct. It is also in bash-4.4-beta, where I should have looked first.


> Adapt your software instead?

I can certainly make evasive maneuvers.

The problem is a deeper one, though: export -f in bash no longer works for 
systems that have /bin/sh -> dash (i.e. Debian and Ubuntu).


 	Makarius

Re: dash drops exported bash functions

[Stephane Chazelas]

2016-02-11 15:30:59 +0100, Makarius:
[...]
> The problem is a deeper one, though: export -f in bash no longer
> works for systems that have /bin/sh -> dash (i.e. Debian and
> Ubuntu).
[...]

Note that pdksh/posh/mksh also strip those variables which it
can't map to shell variable names (and have for decades).

-- 
Stephane

Re: dash drops exported bash functions

[Joachim Breitner]

[-- Attachment #1: Type: text/plain, Size: 1717 bytes --]

Dear Eric,

Am Mittwoch, den 10.02.2016, 08:54 -0700 schrieb Eric Blake:
> On 02/10/2016 08:18 AM, Joachim Breitner wrote:
> > Dear dash developers,
> > 
> > a change in 0.5.8, very likely this one
> > http://git.kernel.org/cgit/utils/dash/dash.git/commit/?id=46d3c1a614f11f0d40a7e73376359618ff07abcd
> > broke the exporting of bash shell functions via the environment.
> 
> Not a bug. POSIX says that on shell startup, the behavior of any
> inherited environment variables that do not start with a proper shell
> name is undefined; and allows shells to scrub such items out of the
> environment on startup.  Just because bash does not scrub them (but
> instead treats them as shell function imports) does not mean dash has to
> behave the same.
> 
> That said, preserving any unusable environment variables unchanged,
> rather than scrubbing them, may be slightly nicer behavior, but I'm not
> sure it's worth the bloat to dash to do so.

I’m not versed in reading POSIX,, so I won’t argue with that (although
I could not find a reference), but let me point out 8.1
"8.1 Environment Variable Definition" of "The Open Group Base
Specifications Issue 7" says
    “Other characters may be permitted by an implementation;
    applications shall tolerate the presence of such names.”

But it is not a big deal now, as the application suffering from this
issue will likely have to work-around this (or fix this, depending on
the POV) anyways.


Greetings,
Joachim
-- 
Joachim “nomeata” Breitner
  mail@joachim-breitner.de • https://www.joachim-breitner.de/
  XMPP: nomeata@joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F
  Debian Developer: nomeata@debian.org


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: dash drops exported bash functions

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1409 bytes --]

On 02/10/2016 08:18 AM, Joachim Breitner wrote:
> Dear dash developers,
> 
> a change in 0.5.8, very likely this one
> http://git.kernel.org/cgit/utils/dash/dash.git/commit/?id=46d3c1a614f11f0d40a7e73376359618ff07abcd
> broke the exporting of bash shell functions via the environment.

Not a bug. POSIX says that on shell startup, the behavior of any
inherited environment variables that do not start with a proper shell
name is undefined; and allows shells to scrub such items out of the
environment on startup.  Just because bash does not scrub them (but
instead treats them as shell function imports) does not mean dash has to
behave the same.

That said, preserving any unusable environment variables unchanged,
rather than scrubbing them, may be slightly nicer behavior, but I'm not
sure it's worth the bloat to dash to do so.

> 
> Exporting bash functions via the environment might be a rarely used
> feature, but it is used in practice, unfortunately (otherwise I
> wouldn’t have noticed this).

Exporting bash functions is only usable if you plan on directly invoking
bash.  Don't drag dash into the mess.  Inserting a dash child in between
a bash parent and grandchild means all bets are off for whether the
grandparent can export anything to the grandchild.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

dash drops exported bash functions

[Joachim Breitner]

[-- Attachment #1.1: Type: text/plain, Size: 1027 bytes --]

Dear dash developers,

a change in 0.5.8, very likely this one
http://git.kernel.org/cgit/utils/dash/dash.git/commit/?id=46d3c1a614f11f0d40a7e73376359618ff07abcd
broke the exporting of bash shell functions via the environment.

The attached script shows the breakage, as it outputs

$ ./exporttest.sh 
exportest called ()
Setting up environment
Calling myself
exportest called (1)
function foo successfully called
Calling myself directly
exportest called (2)
function foo successfully called
Calling myself via dash
exportest called (3)
/tmp/exporttest.sh: Zeile 33: foo: Kommando nicht gefunden.
Done

Exporting bash functions via the environment might be a rarely used
feature, but it is used in practice, unfortunately (otherwise I
wouldn’t have noticed this).

Thanks,
Joachim

-- 
Joachim “nomeata” Breitner
  mail@joachim-breitner.de • https://www.joachim-breitner.de/
  XMPP: nomeata@joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F
  Debian Developer: nomeata@debian.org


[-- Attachment #1.2: exporttest.sh --]
[-- Type: application/x-shellscript, Size: 523 bytes --]

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]