All of
 help / color / mirror / Atom feed
From: Rob Landley <>
To: Linus Torvalds <>
Cc: Kees Cook <>,
	Linux Kernel Mailing List <>,,
Subject: Re: Regression: commit da029c11e6b1 broke toybox xargs.
Date: Fri, 3 Nov 2017 18:58:34 -0500	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On 11/02/2017 10:40 AM, Linus Torvalds wrote:
> On Wed, Nov 1, 2017 at 9:28 PM, Linus Torvalds
> <> wrote:
>> Behavior changed. Things that test particular limits will get different
>> results. That's not breakage.
>> Did an actual user application or script break?

Only due to getting the limit wrong. The actual failure's in the android
internal bugzilla I've never been able to read:

But it boils down to "got the limit wrong, the exec failed after the
fork(), dynamic recovery from which is awkward so I'm trying to figure
out the right limit".

> Ahh. I should have read that email more carefully. If xargs broke,
> that _will_ break actual scripts, yes. Do you actually set the stack
> limit to insane values? Anybody using toybox really shouldn't be doing
> 32MB stacks.

Toybox is the default command line of android since M, which went 64 bit
in L, and the Pixel 2 phone has 4 gigs of ram. My goal with toybox is to
turn android into a self-hosting development environment no longer
cross-compiled from a PC ( so I'm
trying to implement a command line that can run the entire AOSP build.

I.E. I have no idea what people will do with it, and try not to get in
their way.

My problem here is it's hard to figure out what exec size the limit
_is_. There's a sysconf(_SC_ARG_MAX) which bionic and glibc are
currently returning as stack_limit/4, which is now too big and exec()
will error out after the fork. Musl is returning the 131072 limit from
2011-ish, meaning "/bin/echo $(printf '%0*d' 131071)" works but
"printf '%0*d' 131071 | xargs" fails, an inconsistency I was trying to
avoid. Maybe I don't have that luxury...

Each argument has its own limit separate from the argv+envp total limit,
but there's only one "size" you can query through sysconf, so the
querying API is insufficient at the design level.

Meanwhile under bash you can allocate and dirty 256 megabytes from the
command line with:

  echo $(printf '%0*d' $((1<<28)))

Because it's a shell builtin so there's no actual exec. (And if ever gets fixed
it'll go back to allowing INT_MAX.)

Posix is its usual helpful self, read conservatively
says to break the line at 2048 bytes.

> So I still do wonder if this actually breaks anything real, or just a
> test-suite or something?

I've cc'd Elliott, who would know. (He's the Android base os userspace
maintainer, he knows everything. Or can at least decode
http://b/65818597 .)

But this just broke my _fix_, not the earlier deployed stuff. I removed
the size measuring code when the 131072 limit went away, the bug was
there's a new limit I need to not hit, I tried to figure out what the
limit is now, confirmed that the various libc implementations don't
agree, then the actual kernel limit changed again while I was looking at it.

>                    Linus

Should I just go back to hardwiring in 131072? It's no _less_ arbitrary
than 10 megs, and it sounds like getting it _right_ is unachievable.



  reply	other threads:[~2017-11-03 23:58 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-11-01 23:34 Regression: commit da029c11e6b1 broke toybox xargs Rob Landley
2017-11-02  3:30 ` Kees Cook
     [not found] ` <>
2017-11-02 15:40   ` Linus Torvalds
2017-11-03 23:58     ` Rob Landley [this message]
2017-11-04  0:03       ` [Toybox] " enh
2017-11-04  0:42       ` Kees Cook
2017-11-04  1:22         ` Linus Torvalds
2017-11-04  1:37           ` Kees Cook
2017-11-05  1:10             ` Rob Landley
2017-11-04  1:07       ` Linus Torvalds
2017-11-05  0:39         ` Rob Landley
2017-11-05 20:46           ` Linus Torvalds
2017-11-15 22:10             ` enh
2017-11-15 22:45               ` Linus Torvalds
2017-11-15 21:12           ` Pavel Machek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.