[Buildroot] [Bug 9456] New: mkusers script bash errors

[Buildroot] [Bug 9456] New: mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

            Bug ID: 9456
           Summary: mkusers script bash errors
           Product: buildroot
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: Other
          Assignee: unassigned at buildroot.uclibc.org
          Reporter: georgebrmz at oss3d.com
                CC: buildroot at uclibc.org
  Target Milestone: ---

This happens with the latest git as of 2016/11/28:

/buildroot/support/scripts/mkusers: line 417: [: -ge: unary operator expected
/buildroot/support/scripts/mkusers: line 426: [: -eq: unary operator expected

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #1 from Peter Korsgaard <jacmet@uclibc.org> ---
(In reply to George Y. from comment #0)

What is your .config? Have you added any custom packages? It sounds like wrong
values are passed to mkusers

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #2 from George Y. <georgebrmz@oss3d.com> ---
Created attachment 6811
  --> https://bugs.busybox.net/attachment.cgi?id=6811&action=edit
Buildroot config

The config is attached. Nothing specifically different.

By the way, the build process does not stop. It continues and ignores those
errors, so if it happens for you as well, you'll only see it from the messages,
but the building will continue and will give an impression of success.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #3 from Arnout Vandecappelle <arnout@mind.be> ---
Paul__ on IRC reported the same issue. I spent some time trying to debug it,
but I wasn't able to reproduce with the same users settings.

The only thing that changed in that area is pseudo, as far as I can see.
However, on 2016/11/28, pseudo was already reverted. That said, the .config
file that is attached to this bug says it is from 2016.11-rc2 so I'm not
entirely sure if the revert is already included. Please try again with
2016.11-rc3 - if it does work there, then we have identified the culprit.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #4 from George Y. <georgebrmz@oss3d.com> ---
Please disregard the version number in the config file, I'm using the same
config for a while without letting buildroot upgrade it, and I'm always doing a
clean git clone for the master branch. I'm also on OpenSuSE distribution, if
that matters.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #5 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> ---
George, do you still see the issue with the latest Buildroot 2017.02-rc1 ?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |NEEDINFO

--- Comment #6 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> ---
No feedback from George since end of November, so I'll close the bug as
NEEDINFO. George, if you still see the problem with the latest Buildroot
version, do not hesitate to reopen this bug with more details.

Thanks!

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #7 from George Y. <georgebrmz@oss3d.com> ---
We are not making buildroot builds too often. I'll attempt a build soon, and if
it still happens, I'll reopen the bug. Thanks!

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #8 from paulstewartis at gmail.com <paulstewartis@gmail.com> ---
(In reply to Thomas Petazzoni from comment #5)

Hello is Paul__ from the irc channel.

The issue is not resolved for me with 2017.02-rc3.  It is still reporting the
errors.  Because the problem occurs with the first line, my workaround is to
add a dummy user before the real users.  My user table looks like this:

dummy -1 users -1 * - - - Dummy user for mkusers script bug
paul 1000 users 100 =mypassword /home/paul /bin/sh ssh-user
git 1001 users 100 =gitpassword /home/git /usr/bin/git-shell ssh-user

If you remove the first line, it will not create the paul user.  The passwd
file will include the standard users followed by my git user and sshd (because
the openssh package is included in my buildroot .config).

This is what it looks like if you omit the dummy line above:

root:x:0:0:root:/root:/bin/sh
daemon:x:1:1:daemon:/usr/sbin:/bin/false
bin:x:2:2:bin:/bin:/bin/false
sys:x:3:3:sys:/dev:/bin/false
sync:x:4:100:sync:/bin:/bin/sync
mail:x:8:8:mail:/var/spool/mail:/bin/false
www-data:x:33:33:www-data:/var/www:/bin/false
operator:x:37:37:Operator:/var:/bin/false
nobody:x:65534:65534:nobody:/home:/bin/false
git:x:1001:100::/home/git:/usr/bin/git-shell
sshd:x:1000:1000:SSH drop priv user:/:/bin/false

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

Yann E. MORIN <yann.morin.1998@free.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |yann.morin.1998 at free.fr
             Status|RESOLVED                    |REOPENED
         Resolution|NEEDINFO                    |---

--- Comment #9 from Yann E. MORIN <yann.morin.1998@free.fr> ---
(In reply to paulstewartis at gmail.com from comment #8)

OK, let's see what I can do with that users table. Thanks for the use-case.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #10 from Yann E. MORIN <yann.morin.1998@free.fr> ---
(In reply to paulstewartis at gmail.com from comment #8)

Still no luck in trying to reproduce the issue with your users table
(minus the dummy user) and the following defconfig:

BR2_arm=y
BR2_cortex_a7=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
BR2_ROOTFS_USERS_TABLES="/home/ymorin/users.list"
# BR2_PACKAGE_BUSYBOX is not set

Here is the /etc/passwd the gets me:

root:x:0:0:root:/root:/bin/sh
daemon:x:1:1:daemon:/usr/sbin:/bin/false
bin:x:2:2:bin:/bin:/bin/false
sys:x:3:3:sys:/dev:/bin/false
sync:x:4:100:sync:/bin:/bin/sync
mail:x:8:8:mail:/var/spool/mail:/bin/false
www-data:x:33:33:www-data:/var/www:/bin/false
operator:x:37:37:Operator:/var:/bin/false
nobody:x:65534:65534:nobody:/home:/bin/false
paul:x:1000:100::/home/paul:/bin/sh
git:x:1001:100::/home/git:/usr/bin/git-shell

So as you can see, both the paul and git users are there.

What is your bash version:

/usr/bin/env bash --version

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #11 from paulstewartis at gmail.com <paulstewartis@gmail.com> ---
(In reply to Yann E. MORIN from comment #10)

paul at linux:~> /usr/bin/env bash --version
GNU bash, version 4.3.42(1)-release (x86_64-suse-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
paul at linux:~>

I can post my .config if you want too.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #12 from Yann E. MORIN <yann.morin.1998@free.fr> ---
(In reply to paulstewartis at gmail.com from comment #11)

OK, bash is not too old ( I have 4.3.46).

Wild guess: your users table file does not have CR-LF endings, does it?

I have absolutely no clue what's going on... :-/

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #13 from paulstewartis at gmail.com <paulstewartis@gmail.com> ---
(In reply to Yann E. MORIN from comment #12)

"Wild guess: your users table file does not have CR-LF endings, does it?"

LF only.  I tried both by the way.  No difference either way.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #14 from Arnout Vandecappelle <arnout@mind.be> ---
Perhaps there is some mojibake in the users file? Can you attach it as binary?

The bash errors make no sense at all, because exactly the same read commands
and LINES array is used a bit higher up in the mkusers script. Except of course
if one of the spaces is not a real space. Since mkusers is executed with
LANG=C, it will most likely not split on any of the > U+007F whitespace, and
you end up with e.g. uid = 1000\xc2\xa0users.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #15 from Paul Stewart <paulstewartis@gmail.com> ---
Created attachment 6906
  --> https://bugs.busybox.net/attachment.cgi?id=6906&action=edit
User table for paul buildroot config

Here is the actual file. The only difference from what I pasted before is the
passwords.  I have tried different, simple, short passwords and it did not make
a difference.  I will change these passwords, of course, now that they are
posted on the internet. :-)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #16 from Paul Stewart <paulstewartis@gmail.com> ---
(In reply to Arnout Vandecappelle from comment #14)

Looking at the Hexdump, it looks like 0x20 for the spaces:

00000000  64 75 6d 6d 79 20 2d 31  20 75 73 65 72 73 20 2d  |dummy -1 users -|
00000010  31 20 2a 20 2d 20 2d 20  2d 20 44 75 6d 6d 79 20  |1 * - - - Dummy |
<snip>
00000110  72 2f 62 69 6e 2f 67 69  74 2d 73 68 65 6c 6c 20  |r/bin/git-shell |
00000120  73 73 68 2d 75 73 65 72  0a                       |ssh-user.|
00000129

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #17 from Yann E. MORIN <yann.morin.1998@free.fr> ---
(In reply to Paul Stewart from comment #15)

So I downloaded that file and uses it:

  - as-is, and it creates the three users: dummy, paul and git;
  - without the dummy user, and it creates paul and git.

I'll try to instrument the script and attach a patch so that you can report the
output. Give me a moment...

Regards,
Yann E. MORIN.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #18 from Yann E. MORIN <yann.morin.1998@free.fr> ---
Created attachment 6911
  --> https://bugs.busybox.net/attachment.cgi?id=6911&action=edit
Debug mkusers

Paul,

Could you test with the attched patch and report the result, please?

Here is what I got:

line='paul 1000 users 100
=21248f7aac088184a7c9dd3a3cd8bd1bafecc5718174e9025bab7053756a013a /home/paul
/bin/sh ssh-user'
  username='paul'
  uid     ='1000'
  group   ='users'
  gid     ='100'
  paswd   ='=21248f7aac088184a7c9dd3a3cd8bd1bafecc5718174e9025bab7053756a013a'
  home    ='/home/paul'
  shell   ='/bin/sh'
  groups  ='ssh-user'
  comment =''
line='git 1001 users 100
=de5d4f76680ec42ff304682eebeaec1c4756e3d5de3e4ce6db747cb223d6037c /home/git
/usr/bin/git-shell ssh-user'
  username='git'
  uid     ='1001'
  group   ='users'
  gid     ='100'
  paswd   ='=de5d4f76680ec42ff304682eebeaec1c4756e3d5de3e4ce6db747cb223d6037c'
  home    ='/home/git'
  shell   ='/usr/bin/git-shell'
  groups  ='ssh-user'
  comment =''

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #19 from Paul Stewart <paulstewartis@gmail.com> ---
(In reply to Yann E. MORIN from comment #18)

line='paul 1000 users 100
=21248f7aac088184a7c9dd3a3cd8bd1bafecc5718174e9025bab7053756a013a /home/paul
/bin/sh ssh-user'
  username='paul'
  uid     ='1000'
  group   ='users'
  gid     ='100'
  paswd   ='=21248f7aac088184a7c9dd3a3cd8bd1bafecc5718174e9025bab7053756a013a'
  home    ='/home/paul'
  shell   ='/bin/sh'
  groups  ='ssh-user'
  comment =''
line='git 1001 users 100
=de5d4f76680ec42ff304682eebeaec1c4756e3d5de3e4ce6db747cb223d6037c /home/git
/usr/bin/git-shell ssh-user'
  username='git'
  uid     ='1001'
  group   ='users'
  gid     ='100'
  paswd   ='=de5d4f76680ec42ff304682eebeaec1c4756e3d5de3e4ce6db747cb223d6037c'
  home    ='/home/git'
  shell   ='/usr/bin/git-shell'
  groups  ='ssh-user'
  comment =''
line='sshd -1 sshd -1 * - - - SSH drop priv user'
  username='sshd'
  uid     ='-1'
  group   ='sshd'
  gid     ='-1'
  paswd   ='*'
  home    ='-'
  shell   ='-'
  groups  ='-'
  comment ='SSH drop priv user'

This is immediately followed by:
/home/paul/src/buildroot/support/scripts/mkusers: line 407: [: -eq: unary
operator expected
/home/paul/src/buildroot/support/scripts/mkusers: line 432: [: -ge: unary
operator expected
/home/paul/src/buildroot/support/scripts/mkusers: line 441: [: -eq: unary
operator expected

And the paul user is not created.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #20 from Paul Stewart <paulstewartis@gmail.com> ---
I created a default rpi 2 config and set it to use my user table.  And I get
the same issue.  So it isn't my .config file neither.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

Yann E. MORIN <yann.morin.1998@free.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #6911|0                           |1
        is obsolete|                            |
           Assignee|unassigned at buildroot.uclibc |yann.morin.1998 at free.fr
                   |.org                        |

--- Comment #21 from Yann E. MORIN <yann.morin.1998@free.fr> ---
Created attachment 6916
  --> https://bugs.busybox.net/attachment.cgi?id=6916&action=edit
Debug mkusers - paranoid

(In reply to Paul Stewart from comment #19)

Damn, that's uterly weird...

OK, so let's be a little bit more paranoiac. Can you test with the attached
patch, please?

The expected output is described in that patch, too.

Regards,
Yann E. MORIN.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #22 from Paul Stewart <paulstewartis@gmail.com> ---
Created attachment 6921
  --> https://bugs.busybox.net/attachment.cgi?id=6921&action=edit
Output from mkusers script with debug paranoid patch.

Attached output of paranoid patched mkusers.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #23 from Arnout Vandecappelle <arnout@mind.be> ---
D'oh!

From the bash man page:

LINES  Used by the select compound command to determine the column length for 
       printing selection lists.  Automatically  set  if  the  checkwinsize
       option  is enabled or in an interactive shell upon receipt of a
SIGWINCH.

Your bash is receiving a SIGWINCH from your terminal and updates the LINES
variable when it generates some output...

Yann, please rename LINES to a non-conflicting variable :-)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #24 from Yann E. MORIN <yann.morin.1998@free.fr> ---
(In reply to Arnout Vandecappelle from comment #23)

I'll do the change, for sanity sake.

However:
  - the checkwinsize option is disabled by default in a non interactive shell,
  - this is not interactive shell, so SIGWINCH should be ignored.

I'll paste a patch in a moment...

Regards,
Yann E. MORIN.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

Yann E. MORIN <yann.morin.1998@free.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #6916|0                           |1
        is obsolete|                            |
   Attachment #6921|0                           |1
        is obsolete|                            |

--- Comment #25 from Yann E. MORIN <yann.morin.1998@free.fr> ---
Created attachment 6926
  --> https://bugs.busybox.net/attachment.cgi?id=6926&action=edit
Rename variable, paranoid debug

Paul,

Could you please test this patch and report the output, please?

For the record, the output here contains:

    checkwinsize    off

Regards,
Yann E. MORIN.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #26 from Paul Stewart <paulstewartis@gmail.com> ---
(In reply to Yann E. MORIN from comment #25)

That produced the correct output.  The passwd, group and shadow files were all
created properly.  Yay!  Will post the output file momentarily.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #27 from Paul Stewart <paulstewartis@gmail.com> ---
Created attachment 6931
  --> https://bugs.busybox.net/attachment.cgi?id=6931&action=edit
Rename variable, paranoid debug output file

Here is the (successful) output from the renamed variable patch.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #28 from Yann E. MORIN <yann.morin.1998@free.fr> ---
(In reply to Paul Stewart from comment #27)

Thanks a lot for all the testing you did! :-)

This is really weird that your shell does have checkwinsize on. It
should be off by default in a non-interactive shell, and shell script
are running in non-interactive shells...

Anyway... I just sent the patch:

    http://lists.busybox.net/pipermail/buildroot/2017-March/185755.html

Thanks again!

Regards,
Yann E. MORIN.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #29 from Paul Stewart <paulstewartis@gmail.com> ---
(In reply to Yann E. MORIN from comment #28)

You're welcome.  Thank you for the fix!  Will this go into a 2017.02.1 bugfix
or will it wait for the 2017.05 release?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

--- Comment #30 from Yann E. MORIN <yann.morin.1998@free.fr> ---
(In reply to Paul Stewart from comment #29)

Since we decided to have 2017.02 be our first long-term maintainance branch,
I believe this will go in, yes.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9456] mkusers script bash errors

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9456

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|REOPENED                    |RESOLVED

--- Comment #31 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> ---
Fixed by
https://git.buildroot.org/buildroot/commit/?id=00d34e8a6f378653a384c66d68f9a65e13b8034f.
Thanks Paul, Arnout and Yann for the investigation!

-- 
You are receiving this mail because:
You are on the CC list for the bug.