All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v3] make: 4.2.1 -> 4.3
@ 2020-02-24 14:39 Jens Rehsack
  2020-02-25 20:07 ` Adrian Bunk
  2020-02-28  7:45 ` Richard Purdie
  0 siblings, 2 replies; 6+ messages in thread
From: Jens Rehsack @ 2020-02-24 14:39 UTC (permalink / raw)
  To: openembedded-core; +Cc: Jens Rehsack

Announcement: https://lists.gnu.org/archive/html/bug-make/2020-01/msg00057.html

1) Remove upstream provided patches 0001-glob-Do-not-assume-glibc-glob-internals.patch
   and 0002-glob-Do-not-assume-glibc-glob-internals.patch.

2) License has been changed to GPLv3 only

3) Important bug-fix is
   * https://lists.gnu.org/archive/html/bug-make/2018-09/msg00006.html

4) Backward-incompatibilities:
   * Number signs (#) appearing inside a macro reference or function invocation
     no longer introduce comments and should not be escaped with backslashes
   * Previously appending using '+=' to an empty variable would result in a value
     starting with a space.  Now the initial space is only added if the variable
     already contains some value.  Similarly, appending an empty string does not
     add a trailing space.

Fix incompatibility issues between gnulib bundled with updated make fix issues
in w32 compat sources.

Signed-off-by: Jens Rehsack <sno@netbsd.org>
---
 meta/recipes-devtools/make/make.inc           |  4 +-
 ...b-Do-not-assume-glibc-glob-internals.patch | 70 ----------------
 ...m4-restrict-AIX-specific-test-on-AIX.patch | 38 +++++++++
 ...rc-dir.c-fix-buffer-overflow-warning.patch | 41 ++++++++++
 ...b-Do-not-assume-glibc-glob-internals.patch | 38 ---------
 ...low-being-detected-by-importing-proj.patch | 33 ++++++++
 ...02-w32-compat-dirent.c-follow-header.patch | 36 +++++++++
 ...-posixfcn-fcntl-gnulib-make-emulated.patch | 79 +++++++++++++++++++
 meta/recipes-devtools/make/make_4.2.1.bb      | 12 ---
 meta/recipes-devtools/make/make_4.3.bb        | 18 +++++
 10 files changed, 246 insertions(+), 123 deletions(-)
 delete mode 100644 meta/recipes-devtools/make/make/0001-glob-Do-not-assume-glibc-glob-internals.patch
 create mode 100644 meta/recipes-devtools/make/make/0001-m4-getloadavg.m4-restrict-AIX-specific-test-on-AIX.patch
 create mode 100644 meta/recipes-devtools/make/make/0001-src-dir.c-fix-buffer-overflow-warning.patch
 delete mode 100644 meta/recipes-devtools/make/make/0002-glob-Do-not-assume-glibc-glob-internals.patch
 create mode 100644 meta/recipes-devtools/make/make/0002-modules-fcntl-allow-being-detected-by-importing-proj.patch
 create mode 100644 meta/recipes-devtools/make/make/0002-w32-compat-dirent.c-follow-header.patch
 create mode 100644 meta/recipes-devtools/make/make/0003-posixfcn-fcntl-gnulib-make-emulated.patch
 delete mode 100644 meta/recipes-devtools/make/make_4.2.1.bb
 create mode 100644 meta/recipes-devtools/make/make_4.3.bb

diff --git a/meta/recipes-devtools/make/make.inc b/meta/recipes-devtools/make/make.inc
index b8905bc6d3..4142cf23d3 100644
--- a/meta/recipes-devtools/make/make.inc
+++ b/meta/recipes-devtools/make/make.inc
@@ -5,9 +5,7 @@ called the makefile, which lists each of the non-source files and how to compute
 HOMEPAGE = "http://www.gnu.org/software/make/"
 SECTION = "devel"
 
-SRC_URI = "${GNU_MIRROR}/make/make-${PV}.tar.bz2 \
-           file://0001-glob-Do-not-assume-glibc-glob-internals.patch \
-           file://0002-glob-Do-not-assume-glibc-glob-internals.patch \
+SRC_URI = "${GNU_MIRROR}/make/make-${PV}.tar.lz \
            "
 
 inherit autotools gettext pkgconfig texinfo
diff --git a/meta/recipes-devtools/make/make/0001-glob-Do-not-assume-glibc-glob-internals.patch b/meta/recipes-devtools/make/make/0001-glob-Do-not-assume-glibc-glob-internals.patch
deleted file mode 100644
index 2b6e4d40c3..0000000000
--- a/meta/recipes-devtools/make/make/0001-glob-Do-not-assume-glibc-glob-internals.patch
+++ /dev/null
@@ -1,70 +0,0 @@
-From c90a7dda6c572f79b8e78da44b6ebf8704edef65 Mon Sep 17 00:00:00 2001
-From: Paul Eggert <eggert@cs.ucla.edu>
-Date: Sun, 24 Sep 2017 09:12:58 -0400
-Subject: [PATCH 1/2] glob: Do not assume glibc glob internals.
-
-It has been proposed that glibc glob start using gl_lstat,
-which the API allows it to do.  GNU 'make' should not get in
-the way of this.  See:
-https://sourceware.org/ml/libc-alpha/2017-09/msg00409.html
-
-* dir.c (local_lstat): New function, like local_stat.
-(dir_setup_glob): Use it to initialize gl_lstat too, as the API
-requires.
----
-Upstream-Status: Backport
-Signed-off-by: Khem Raj <raj.khem@gmail.com>
-
- dir.c | 29 +++++++++++++++++++++++++++--
- 1 file changed, 27 insertions(+), 2 deletions(-)
-
-diff --git a/dir.c b/dir.c
-index f34bbf5..12eef30 100644
---- a/dir.c
-+++ b/dir.c
-@@ -1299,15 +1299,40 @@ local_stat (const char *path, struct stat *buf)
- }
- #endif
- 
-+/* Similarly for lstat.  */
-+#if !defined(lstat) && !defined(WINDOWS32) || defined(VMS)
-+# ifndef VMS
-+#  ifndef HAVE_SYS_STAT_H
-+int lstat (const char *path, struct stat *sbuf);
-+#  endif
-+# else
-+    /* We are done with the fake lstat.  Go back to the real lstat */
-+#   ifdef lstat
-+#     undef lstat
-+#   endif
-+# endif
-+# define local_lstat lstat
-+#elif defined(WINDOWS32)
-+/* Windows doesn't support lstat().  */
-+# define local_lstat local_stat
-+#else
-+static int
-+local_lstat (const char *path, struct stat *buf)
-+{
-+  int e;
-+  EINTRLOOP (e, lstat (path, buf));
-+  return e;
-+}
-+#endif
-+
- void
- dir_setup_glob (glob_t *gl)
- {
-   gl->gl_opendir = open_dirstream;
-   gl->gl_readdir = read_dirstream;
-   gl->gl_closedir = free;
-+  gl->gl_lstat = local_lstat;
-   gl->gl_stat = local_stat;
--  /* We don't bother setting gl_lstat, since glob never calls it.
--     The slot is only there for compatibility with 4.4 BSD.  */
- }
- 
- void
--- 
-2.16.1
-
diff --git a/meta/recipes-devtools/make/make/0001-m4-getloadavg.m4-restrict-AIX-specific-test-on-AIX.patch b/meta/recipes-devtools/make/make/0001-m4-getloadavg.m4-restrict-AIX-specific-test-on-AIX.patch
new file mode 100644
index 0000000000..096bcfdf78
--- /dev/null
+++ b/meta/recipes-devtools/make/make/0001-m4-getloadavg.m4-restrict-AIX-specific-test-on-AIX.patch
@@ -0,0 +1,38 @@
+From 8309601775d9442416329a77f7dcfd8aa799e9a6 Mon Sep 17 00:00:00 2001
+From: Jens Rehsack <sno@netbsd.org>
+Date: Fri, 21 Feb 2020 17:39:56 +0100
+Subject: [PATCH 1/2] m4/getloadavg.m4: restrict AIX specific test on AIX
+
+When cross compiling for a system without getloadavg, do not try add
+additional linker paths unless it's absolutely necessary.
+
+Signed-off-by: Jens Rehsack <sno@netbsd.org>
+---
+Upstream-Status: Pending
+ m4/getloadavg.m4 | 3 +++
+ 1 file changed, 3 insertions(+)
+
+diff --git a/m4/getloadavg.m4 b/m4/getloadavg.m4
+index 3bd2a14..696c5de 100644
+--- a/m4/getloadavg.m4
++++ b/m4/getloadavg.m4
+@@ -42,6 +42,8 @@ AC_CHECK_FUNC([getloadavg], [],
+    fi
+ 
+    if test $gl_func_getloadavg_done = no; then
++     AS_CASE([$host_os],
++             [aix*], [
+      # There is a commonly available library for RS/6000 AIX.
+      # Since it is not a standard part of AIX, it might be installed locally.
+      gl_getloadavg_LIBS=$LIBS
+@@ -49,6 +51,7 @@ AC_CHECK_FUNC([getloadavg], [],
+      AC_CHECK_LIB([getloadavg], [getloadavg],
+                   [LIBS="-lgetloadavg $LIBS" gl_func_getloadavg_done=yes],
+                   [LIBS=$gl_getloadavg_LIBS])
++       ], [:])
+    fi
+ 
+    # Set up the replacement function if necessary.
+-- 
+2.17.1
+
diff --git a/meta/recipes-devtools/make/make/0001-src-dir.c-fix-buffer-overflow-warning.patch b/meta/recipes-devtools/make/make/0001-src-dir.c-fix-buffer-overflow-warning.patch
new file mode 100644
index 0000000000..57970824f6
--- /dev/null
+++ b/meta/recipes-devtools/make/make/0001-src-dir.c-fix-buffer-overflow-warning.patch
@@ -0,0 +1,41 @@
+From cd7091a7d88306004ca98c5dafcc40f44589b105 Mon Sep 17 00:00:00 2001
+From: Jens Rehsack <sno@netbsd.org>
+Date: Mon, 24 Feb 2020 10:52:21 +0100
+Subject: [PATCH 1/3] src/dir.c: fix buffer-overflow warning
+
+Fix compiler warning:
+	src/dir.c:1294:7: warning: 'strncpy' specified bound depends on the
+			  length of the source argument [-Wstringop-overflow=]
+
+The existing code assumes `path` will never exceed `MAXPATHLEN`. Also the
+size of the buffer is increased by 1 to hold a path with the length of
+`MAXPATHLEN` and trailing `0`.
+
+Signed-off-by: Jens Rehsack <sno@netbsd.org>
+---
+Upstream-Status: Pending (https://savannah.gnu.org/bugs/?57888)
+
+ src/dir.c | 6 +++---
+ 1 file changed, 3 insertions(+), 3 deletions(-)
+
+diff --git a/src/dir.c b/src/dir.c
+index 862a18e..cad4c4a 100644
+--- a/src/dir.c
++++ b/src/dir.c
+@@ -1289,10 +1289,10 @@ local_stat (const char *path, struct stat *buf)
+   if (plen > 1 && path[plen - 1] == '.'
+       && (path[plen - 2] == '/' || path[plen - 2] == '\\'))
+     {
+-      char parent[MAXPATHLEN];
++      char parent[MAXPATHLEN+1];
+ 
+-      strncpy (parent, path, plen - 2);
+-      parent[plen - 2] = '\0';
++      strncpy (parent, path, MAXPATHLEN);
++      parent[MIN(plen - 2, MAXPATHLEN)] = '\0';
+       if (stat (parent, buf) < 0 || !_S_ISDIR (buf->st_mode))
+         return -1;
+     }
+-- 
+2.17.1
+
diff --git a/meta/recipes-devtools/make/make/0002-glob-Do-not-assume-glibc-glob-internals.patch b/meta/recipes-devtools/make/make/0002-glob-Do-not-assume-glibc-glob-internals.patch
deleted file mode 100644
index d49acd9f6e..0000000000
--- a/meta/recipes-devtools/make/make/0002-glob-Do-not-assume-glibc-glob-internals.patch
+++ /dev/null
@@ -1,38 +0,0 @@
-From 9858702dbd1e137262c06765919937660879f63c Mon Sep 17 00:00:00 2001
-From: Paul Eggert <eggert@cs.ucla.edu>
-Date: Sun, 24 Sep 2017 09:12:58 -0400
-Subject: [PATCH 2/2] glob: Do not assume glibc glob internals.
-
-It has been proposed that glibc glob start using gl_lstat,
-which the API allows it to do.  GNU 'make' should not get in
-the way of this.  See:
-https://sourceware.org/ml/libc-alpha/2017-09/msg00409.html
-
-* dir.c (local_lstat): New function, like local_stat.
-(dir_setup_glob): Use it to initialize gl_lstat too, as the API
-requires.
----
-Upstream-Status: Backport
-
- configure.ac | 3 +--
- 1 file changed, 1 insertion(+), 2 deletions(-)
-
-diff --git a/configure.ac b/configure.ac
-index 64ec870..e87901c 100644
---- a/configure.ac
-+++ b/configure.ac
-@@ -399,10 +399,9 @@ AC_CACHE_CHECK([if system libc has GNU glob], [make_cv_sys_gnu_glob],
- #include <glob.h>
- #include <fnmatch.h>
- 
--#define GLOB_INTERFACE_VERSION 1
- #if !defined _LIBC && defined __GNU_LIBRARY__ && __GNU_LIBRARY__ > 1
- # include <gnu-versions.h>
--# if _GNU_GLOB_INTERFACE_VERSION == GLOB_INTERFACE_VERSION
-+if _GNU_GLOB_INTERFACE_VERSION == 1 || _GNU_GLOB_INTERFACE_VERSION == 2
-    gnu glob
- # endif
- #endif],
--- 
-2.16.1
-
diff --git a/meta/recipes-devtools/make/make/0002-modules-fcntl-allow-being-detected-by-importing-proj.patch b/meta/recipes-devtools/make/make/0002-modules-fcntl-allow-being-detected-by-importing-proj.patch
new file mode 100644
index 0000000000..b3d97f9a3a
--- /dev/null
+++ b/meta/recipes-devtools/make/make/0002-modules-fcntl-allow-being-detected-by-importing-proj.patch
@@ -0,0 +1,33 @@
+From fb8aaed3b040e589cd880fd714dda5ec00687217 Mon Sep 17 00:00:00 2001
+From: Jens Rehsack <sno@netbsd.org>
+Date: Mon, 24 Feb 2020 12:10:06 +0100
+Subject: [PATCH 2/2] modules: fcntl: allow being detected by importing
+ projects
+
+GNU project `make` relies on gnulib but provides some own compatibility
+functions - including an `fcntl`, which fails on mingw.
+The intension of gnulib is providing these functions and being wider tested,
+but silently injecting a function opens battle of compatibility layers.
+
+So adding a hint into target `config.h` to allow deciding whether using
+an own compatibility implementation or not.
+
+Signed-off-by: Jens Rehsack <sno@netbsd.org>
+---
+Upstream-Status: Pending
+
+ m4/gnulib-comp.m4 | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/m4/gnulib-comp.m4 b/m4/gnulib-comp.m4
+index 3ee0811..cf75541 100644
+--- a/m4/gnulib-comp.m4
++++ b/m4/gnulib-comp.m4
+@@ -147,6 +147,7 @@
+   gl_FUNC_FCNTL
+   if test $HAVE_FCNTL = 0 || test $REPLACE_FCNTL = 1; then
+     AC_LIBOBJ([fcntl])
++    AC_DEFINE(HAVE_GNULIB_FCNTL, 1, [Define to 1 if you have the `fcntl' function via gnulib.])
+   fi
+   gl_FCNTL_MODULE_INDICATOR([fcntl])
+   gl_FCNTL_H
diff --git a/meta/recipes-devtools/make/make/0002-w32-compat-dirent.c-follow-header.patch b/meta/recipes-devtools/make/make/0002-w32-compat-dirent.c-follow-header.patch
new file mode 100644
index 0000000000..9ecc44543e
--- /dev/null
+++ b/meta/recipes-devtools/make/make/0002-w32-compat-dirent.c-follow-header.patch
@@ -0,0 +1,36 @@
+From 4dd8b4f43aa0078707ad9a7932f4e137bc4383ed Mon Sep 17 00:00:00 2001
+From: Jens Rehsack <sno@netbsd.org>
+Date: Mon, 24 Feb 2020 11:12:43 +0100
+Subject: [PATCH 2/3] w32: compat: dirent.c: follow header
+
+src/w32/include/dirent.h completely delegates to mingw dirent implementation,
+gnulib detects it as fine and completely usable - trust in that.
+
+Signed-off-by: Jens Rehsack <sno@netbsd.org>
+---
+Upstream-Status: Pending (https://savannah.gnu.org/bugs/?57888)
+
+ src/w32/compat/dirent.c | 3 ++-
+ 1 file changed, 2 insertions(+), 1 deletion(-)
+
+diff --git a/src/w32/compat/dirent.c b/src/w32/compat/dirent.c
+index b8ec615..de80f72 100644
+--- a/src/w32/compat/dirent.c
++++ b/src/w32/compat/dirent.c
+@@ -23,7 +23,7 @@ this program.  If not, see <http://www.gnu.org/licenses/>.  */
+ #include <stdlib.h>
+ #include "dirent.h"
+ 
+-
++#ifndef __MINGW32__
+ DIR*
+ opendir(const char* pDirName)
+ {
+@@ -193,3 +193,4 @@ seekdir(DIR* pDir, long nPosition)
+ 
+         return;
+ }
++#endif  /* !__MINGW32__ */
+-- 
+2.17.1
+
diff --git a/meta/recipes-devtools/make/make/0003-posixfcn-fcntl-gnulib-make-emulated.patch b/meta/recipes-devtools/make/make/0003-posixfcn-fcntl-gnulib-make-emulated.patch
new file mode 100644
index 0000000000..70414c51f4
--- /dev/null
+++ b/meta/recipes-devtools/make/make/0003-posixfcn-fcntl-gnulib-make-emulated.patch
@@ -0,0 +1,79 @@
+From 3d074c8fca5fcf3e6b83d33788f35a8f1b3a44a2 Mon Sep 17 00:00:00 2001
+From: Jens Rehsack <sno@netbsd.org>
+Date: Fri, 21 Feb 2020 19:29:49 +0100
+Subject: [PATCH 3/3] posixfcn: fcntl: gnulib > make-emulated
+
+Rate the fcntl emulation from gnulib higher than the own one.
+
+Signed-off-by: Jens Rehsack <sno@netbsd.org>
+---
+Upstream-Status: Pending (https://savannah.gnu.org/bugs/?57888)
+
+ src/output.h              | 19 ++++++++++++++-----
+ src/w32/compat/posixfcn.c |  2 ++
+ 2 files changed, 16 insertions(+), 5 deletions(-)
+
+diff --git a/src/output.h b/src/output.h
+index a506505..d3ce6b7 100644
+--- a/src/output.h
++++ b/src/output.h
+@@ -67,14 +67,21 @@ void output_dump (struct output *out);
+ 
+ # ifdef WINDOWS32
+ /* For emulations in w32/compat/posixfcn.c.  */
+-#  define F_GETFD 1
+-#  define F_SETLKW 2
++#  ifndef F_GETFD
++#   define F_GETFD 1
++#  endif
++#  ifndef F_SETLKW
++#   define F_SETLKW 2
++#  endif
+ /* Implementation note: None of the values of l_type below can be zero
+    -- they are compared with a static instance of the struct, so zero
+    means unknown/invalid, see w32/compat/posixfcn.c. */
+-#  define F_WRLCK 1
+-#  define F_UNLCK 2
+-
++#  ifndef F_WRLCK
++#   define F_WRLCK 1
++#  endif
++#  ifndef F_UNLCK
++#   define F_UNLCK 2
++#  endif
+ struct flock
+   {
+     short l_type;
+@@ -89,7 +96,9 @@ struct flock
+ typedef intptr_t sync_handle_t;
+ 
+ /* Public functions emulated/provided in posixfcn.c.  */
++#  ifndef HAVE_GNULIB_FCNTL
+ int fcntl (intptr_t fd, int cmd, ...);
++#  endif
+ intptr_t create_mutex (void);
+ int same_stream (FILE *f1, FILE *f2);
+ 
+diff --git a/src/w32/compat/posixfcn.c b/src/w32/compat/posixfcn.c
+index 975dfb7..d337b9c 100644
+--- a/src/w32/compat/posixfcn.c
++++ b/src/w32/compat/posixfcn.c
+@@ -29,6 +29,7 @@ this program.  If not, see <http://www.gnu.org/licenses/>.  */
+ #ifndef NO_OUTPUT_SYNC
+ /* Support for OUTPUT_SYNC and related functionality.  */
+ 
++#ifndef HAVE_GNULIB_FCNTL
+ /* Emulation of fcntl that supports only F_GETFD and F_SETLKW.  */
+ int
+ fcntl (intptr_t fd, int cmd, ...)
+@@ -142,6 +143,7 @@ fcntl (intptr_t fd, int cmd, ...)
+         return -1;
+     }
+ }
++#endif /* GNULIB_TEST_FCNTL */
+ 
+ static intptr_t mutex_handle = -1;
+ 
+-- 
+2.17.1
+
diff --git a/meta/recipes-devtools/make/make_4.2.1.bb b/meta/recipes-devtools/make/make_4.2.1.bb
deleted file mode 100644
index c6e6a0cd58..0000000000
--- a/meta/recipes-devtools/make/make_4.2.1.bb
+++ /dev/null
@@ -1,12 +0,0 @@
-LICENSE = "GPLv3 & LGPLv2"
-LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \
-                    file://tests/COPYING;md5=d32239bcb673463ab874e80d47fae504 \
-                    file://glob/COPYING.LIB;md5=4a770b67e6be0f60da244beb2de0fce4"
-require make.inc
-
-EXTRA_OECONF += "--without-guile"
-
-SRC_URI[md5sum] = "15b012617e7c44c0ed482721629577ac"
-SRC_URI[sha256sum] = "d6e262bf3601b42d2b1e4ef8310029e1dcf20083c5446b4b7aa67081fdffc589"
-
-BBCLASSEXTEND = "native nativesdk"
diff --git a/meta/recipes-devtools/make/make_4.3.bb b/meta/recipes-devtools/make/make_4.3.bb
new file mode 100644
index 0000000000..70caf0ae16
--- /dev/null
+++ b/meta/recipes-devtools/make/make_4.3.bb
@@ -0,0 +1,18 @@
+LICENSE = "GPLv3"
+LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
+require make.inc
+
+SRC_URI += "\
+	file://0001-m4-getloadavg.m4-restrict-AIX-specific-test-on-AIX.patch \
+	file://0002-modules-fcntl-allow-being-detected-by-importing-proj.patch \
+	file://0001-src-dir.c-fix-buffer-overflow-warning.patch \
+	file://0002-w32-compat-dirent.c-follow-header.patch \
+	file://0003-posixfcn-fcntl-gnulib-make-emulated.patch \
+"
+
+EXTRA_OECONF += "--without-guile"
+
+SRC_URI[md5sum] = "d5c40e7bd1e97a7404f5d3be982f479a"
+SRC_URI[sha256sum] = "de1a441c4edf952521db30bfca80baae86a0ff1acd0a00402999344f04c45e82"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.17.1



^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH v3] make: 4.2.1 -> 4.3
  2020-02-24 14:39 [PATCH v3] make: 4.2.1 -> 4.3 Jens Rehsack
@ 2020-02-25 20:07 ` Adrian Bunk
  2020-02-28  7:45 ` Richard Purdie
  1 sibling, 0 replies; 6+ messages in thread
From: Adrian Bunk @ 2020-02-25 20:07 UTC (permalink / raw)
  To: Jens Rehsack; +Cc: openembedded-core

On Mon, Feb 24, 2020 at 03:39:20PM +0100, Jens Rehsack wrote:
>...
> -SRC_URI = "${GNU_MIRROR}/make/make-${PV}.tar.bz2 \
>...
> +SRC_URI = "${GNU_MIRROR}/make/make-${PV}.tar.lz \
>...

Please use the .tar.gz instead.

Building lzip-native just for being able to build make is not worth the 
small space savings, especially since this creates a bottleneck that
can slow down the whole build.

cu
Adrian


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH v3] make: 4.2.1 -> 4.3
  2020-02-24 14:39 [PATCH v3] make: 4.2.1 -> 4.3 Jens Rehsack
  2020-02-25 20:07 ` Adrian Bunk
@ 2020-02-28  7:45 ` Richard Purdie
  2020-02-29  7:55   ` Victor Kamensky (kamensky)
  1 sibling, 1 reply; 6+ messages in thread
From: Richard Purdie @ 2020-02-28  7:45 UTC (permalink / raw)
  To: Jens Rehsack, openembedded-core

On Mon, 2020-02-24 at 15:39 +0100, Jens Rehsack wrote:
> Announcement: https://lists.gnu.org/archive/html/bug-make/2020-01/msg00057.html
> 
> 1) Remove upstream provided patches 0001-glob-Do-not-assume-glibc-glob-internals.patch
>    and 0002-glob-Do-not-assume-glibc-glob-internals.patch.
> 
> 2) License has been changed to GPLv3 only
> 
> 3) Important bug-fix is
>    * https://lists.gnu.org/archive/html/bug-make/2018-09/msg00006.html
> 
> 4) Backward-incompatibilities:
>    * Number signs (#) appearing inside a macro reference or function invocation
>      no longer introduce comments and should not be escaped with backslashes
>    * Previously appending using '+=' to an empty variable would result in a value
>      starting with a space.  Now the initial space is only added if the variable
>      already contains some value.  Similarly, appending an empty string does not
>      add a trailing space.
> 
> Fix incompatibility issues between gnulib bundled with updated make fix issues
> in w32 compat sources.
> 
> Signed-off-by: Jens Rehsack <sno@netbsd.org

Unfortunately, crazy as this sounds, this is causing mips problems on
target with kernel module building. We only see the following failures
when this patch is included:

https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/1612 
https://autobuilder.yoctoproject.org/typhoon/#/builders/102/builds/315 
https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/1616

Cheers,

Richard



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH v3] make: 4.2.1 -> 4.3
  2020-02-28  7:45 ` Richard Purdie
@ 2020-02-29  7:55   ` Victor Kamensky (kamensky)
  2020-02-29 10:12     ` Richard Purdie
  0 siblings, 1 reply; 6+ messages in thread
From: Victor Kamensky (kamensky) @ 2020-02-29  7:55 UTC (permalink / raw)
  To: Richard Purdie, Jens Rehsack, openembedded-core

[-- Attachment #1: Type: text/plain, Size: 24646 bytes --]

[sorry for the top post: bad email client at work]

Hi Richard, Jens,

I was able to reproduce and debugged the mips new make issue. Workaround
patch attached.

In summary: kernelmodule.KernelModuleTest.test_kernel_module
fails with 'fatal error: opening dependency file scripts/basic/.fixdep.d: Permission denied'

New make-4.3 started using posix_spawn glibc call,
that happens to be broken on mips. On mips during posix_spawn call
__spawni_child function that is executed after clone makes
setresuid call with spurious/wrong value of ruid. Instead of -1
it passes 127, effectively switching real user id since test
runs under root. Subsequent gcc runs under wrong ruid and it
fails to write into /usr/src/kernel/scripts/basic directory.

__spawni_child uses direct system call invocation for setresuid
rather calling glibc function. And there is wrong code generation
issue somewhere in inline assembly for direct system call and/or
way how compiler processes it. I did not dig into it yet, it
looks as quite hard problem and needs more time.

For now I've made workaround patch that just through configure
ac_cv_func_posix_spawn=no parameter tells make configure not to use
posix_spawn on mips.

I will also send suggested patch through 'git send-mail'

Tested workaround on qemumips64, did not have time for
qemumips full test, verified build only and made sure that
posix_spawn is not used. But I believe it is the same issue.

Detailed debugging notes
=================

Running 'bitbake core-image-sato-sdk -c testimage' fails on qemumips64
----------------------------------------------------------------------

RESULTS - kernelmodule.KernelModuleTest.test_kernel_module: FAILED (36.71s)

Reproducing issue manually
--------------------------

root@qemumips64:~# cd /usr/src/kernel && make scripts prepare
  HOSTCC  scripts/basic/fixdep
scripts/basic/fixdep.c:410:1: fatal error: opening dependency file scripts/basic/.fixdep.d: Permission denied
  410 | }
      | ^
compilation terminated.
make[2]: *** [scripts/Makefile.host:107: scripts/basic/fixdep] Error 1
make[1]: *** [Makefile:500: scripts_basic] Error 2
make: *** [Makefile:678: include/config/auto.conf.cmd] Error 2
make: *** [include/config/auto.conf.cmd] Deleting file 'include/config/tristate.conf'
root@qemumips64:/usr/src/kernel# ls -al scripts/basic/.fixdep.d
ls: cannot access 'scripts/basic/.fixdep.d': No such file or directory

Calling the same gcc command manually from shell
------------------------------------------------

It works, no problem.

root@qemumips64:/usr/src/kernel# gcc -Wp,-MD,scripts/basic/.fixdep.d -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 -fomit-frame-pointer -std=gnu89 -o scripts/basic/fixdep scripts/basic/fixdep.c
root@qemumips64:/usr/src/kernel# 

Means make changes context in such way that prevents gcc
command execution.

Looking at strace result
------------------------

Run the same make command under strace. In log see that
getuid return 127! I.e it is different user. It does not
have permission to write into scripts/basic/.fixdep.d

1144  getuid()                          = 127
1144  getgid()                          = 127

and earlier:

1144  getuid()                          = 0
1144  setresuid(127, 0, -1)             = 0
1144  getgid()                          = 0
1144  setresgid(127, 0, -1)             = 0
1144  dup2(6, 1)                        = 1
1144  rt_sigprocmask(SIG_SETMASK, [], NULL, 16) = 0
1144  execve("/bin/sh", ["/bin/sh", "-c", "uname -m | sed -e s/i.86/x86/ -e"...], 0xfffbb2e1d0 /* 16 vars */ <unfinished ...>

There is call to setresuid that sets real user id to
127. Why?

Running make under gdb
----------------------

(gdb) set args scripts
(gdb) b main
Breakpoint 1 at 0x9d80: file ../make-4.3/src/main.c, line 1056.
(gdb) set follow-fork-mode child
(gdb) run
Starting program: /usr/bin/make scripts

Breakpoint 1, main (argc=2, argv=0xffffffac78, envp=0xffffffac90) at ../make-4.3/src/main.c:1056
1056	../make-4.3/src/main.c: No such file or directory.
(gdb) b getuid
Breakpoint 2 at 0xfff7eb5a00: file ../sysdeps/unix/syscall-template.S, line 59.
(gdb) b setresuid
Breakpoint 3 at 0xfff7eb5eb0: file ../sysdeps/unix/sysv/linux/setresuid.c, line 29.
(gdb) b execve
Breakpoint 4 at 0xfff7eb4f10: file ../sysdeps/unix/syscall-template.S, line 78.
(gdb) b clone
Breakpoint 5 at 0xfff7ef3c90: file ../sysdeps/unix/sysv/linux/mips/clone.S, line 45.
(gdb) b posix_spawn 
Breakpoint 6 at 0xfff7edd790: posix_spawn. (2 locations)
(gdb) b __spawni_child
Breakpoint 7 at 0xfff7eddac0: file ../sysdeps/unix/sysv/linux/spawni.c, line 123.
(gdb) c
Continuing.

Breakpoint 2, getuid () at ../sysdeps/unix/syscall-template.S:59
59	../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) bt
#0  getuid () at ../sysdeps/unix/syscall-template.S:59
#1  0x000000fff7edfa7c in __euidaccess (path=0xaaaab21b30 "/bin/sh", mode=1) at ../sysdeps/posix/euidaccess.c:159
#2  0x000000aaaaadecdc in find_in_given_path (progname=0xaaaab25440 "/bin/sh", path=<optimized out>, optimize_for_exec=<optimized out>)
    at ../../make-4.3/lib/findprog-in.c:137
#3  0x000000aaaaac7680 in child_execute_job (child=0xffffff8770, good_stdin=<optimized out>, argv=0xaaaab26cb0) at ../make-4.3/src/job.c:2384
#4  0x000000aaaaac2678 in func_shell_base (o=0xaaaab1e350 "", argv=<optimized out>, trim_newlines=<optimized out>) at ../make-4.3/src/function.c:1790
#5  0x000000aaaaac2c38 in handle_function (op=0xffffff8930, stringp=<optimized out>) at ../make-4.3/src/function.c:2544
#6  0x000000aaaaabaa10 in variable_expand_string (line=<optimized out>, string=<optimized out>, length=<optimized out>) at ../make-4.3/src/expand.c:262
#7  0x000000aaaaabb5b4 in allocated_variable_expand_for_file (line=<optimized out>, file=<optimized out>) at ../make-4.3/src/expand.c:566
#8  0x000000aaaaadc11c in do_variable_definition (flocp=0xffffff8d70, varname=0xaaaab1fca0 "SUBARCH", 
    value=0xaaaab24e6b "$(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ -e s/sun4u/sparc64/ -e s/arm.*/arm/ -e s/sa110/arm/ -e s/s390x/s390/ -e s/parisc64/parisc/ -e s/ppc.*/powerpc/ -e s/mips.*/mips/ -e s/sh[234].*/s"..., origin=<optimized out>, flavor=<optimized out>, target_var=<optimized out>) at ../make-4.3/src/variable.c:1187
#9  0x000000aaaaadccac in try_variable_definition (flocp=0xffffff8d70, line=<optimized out>, origin=<optimized out>, target_var=<optimized out>)
    at ../make-4.3/src/variable.c:1633
#10 0x000000aaaaad2dfc in eval (ebuf=<optimized out>, set_default=<optimized out>) at ../make-4.3/src/read.c:750
#11 0x000000aaaaad451c in eval_makefile (filename=<optimized out>, flags=<optimized out>) at ../make-4.3/src/read.c:436
#12 0x000000aaaaad3cc4 in eval (ebuf=<optimized out>, set_default=<optimized out>) at ../make-4.3/src/read.c:908
#13 0x000000aaaaad451c in eval_makefile (filename=<optimized out>, flags=<optimized out>) at ../make-4.3/src/read.c:436
#14 0x000000aaaaad4a70 in read_all_makefiles (makefiles=<optimized out>) at ../make-4.3/src/read.c:258
#15 0x000000aaaaab4f9c in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at ../make-4.3/src/main.c:1945
(gdb) c
Continuing.

Breakpoint 6, __GI___posix_spawn (pid=0xffffff84b4, path=0xaaaab25440 "/bin/sh", file_actions=0xffffff84b8, attrp=0xffffff8588, argv=0xaaaab26cb0, envp=0xffffffac90)
    at spawn.c:30
30	spawn.c: No such file or directory.
(gdb) bt
#0  __GI___posix_spawn (pid=0xffffff84b4, path=0xaaaab25440 "/bin/sh", file_actions=0xffffff84b8, attrp=0xffffff8588, argv=0xaaaab26cb0, envp=0xffffffac90) at spawn.c:30
#1  0x000000aaaaac76b0 in child_execute_job (child=0xffffff8770, good_stdin=<optimized out>, argv=0xaaaab26cb0) at ../make-4.3/src/job.c:2396
#2  0x000000aaaaac2678 in func_shell_base (o=0xaaaab1e350 "", argv=<optimized out>, trim_newlines=<optimized out>) at ../make-4.3/src/function.c:1790
#3  0x000000aaaaac2c38 in handle_function (op=0xffffff8930, stringp=<optimized out>) at ../make-4.3/src/function.c:2544
#4  0x000000aaaaabaa10 in variable_expand_string (line=<optimized out>, string=<optimized out>, length=<optimized out>) at ../make-4.3/src/expand.c:262
#5  0x000000aaaaabb5b4 in allocated_variable_expand_for_file (line=<optimized out>, file=<optimized out>) at ../make-4.3/src/expand.c:566
#6  0x000000aaaaadc11c in do_variable_definition (flocp=0xffffff8d70, varname=0xaaaab1fca0 "SUBARCH", 
    value=0xaaaab24e6b "$(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ -e s/sun4u/sparc64/ -e s/arm.*/arm/ -e s/sa110/arm/ -e s/s390x/s390/ -e s/parisc64/parisc/ -e s/ppc.*/powerpc/ -e s/mips.*/mips/ -e s/sh[234].*/s"..., origin=<optimized out>, flavor=<optimized out>, target_var=<optimized out>) at ../make-4.3/src/variable.c:1187
#7  0x000000aaaaadccac in try_variable_definition (flocp=0xffffff8d70, line=<optimized out>, origin=<optimized out>, target_var=<optimized out>)
    at ../make-4.3/src/variable.c:1633
#8  0x000000aaaaad2dfc in eval (ebuf=<optimized out>, set_default=<optimized out>) at ../make-4.3/src/read.c:750
#9  0x000000aaaaad451c in eval_makefile (filename=<optimized out>, flags=<optimized out>) at ../make-4.3/src/read.c:436
#10 0x000000aaaaad3cc4 in eval (ebuf=<optimized out>, set_default=<optimized out>) at ../make-4.3/src/read.c:908
#11 0x000000aaaaad451c in eval_makefile (filename=<optimized out>, flags=<optimized out>) at ../make-4.3/src/read.c:436
#12 0x000000aaaaad4a70 in read_all_makefiles (makefiles=<optimized out>) at ../make-4.3/src/read.c:258
#13 0x000000aaaaab4f9c in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at ../make-4.3/src/main.c:1945
(gdb) c
Continuing.

Breakpoint 5, clone () at ../sysdeps/unix/sysv/linux/mips/clone.S:45
45	../sysdeps/unix/sysv/linux/mips/clone.S: No such file or directory.
(gdb) bt
#0  clone () at ../sysdeps/unix/sysv/linux/mips/clone.S:45
#1  0x000000fff7edd938 in __spawnix (pid=0xffffff84b4, file=0xaaaab25440 "/bin/sh", file_actions=<optimized out>, attrp=<optimized out>, argv=0xaaaab26cb0, 
    envp=0xffffffac90, xflags=<optimized out>, exec=0xfff7eb4f10 <execve>) at ../sysdeps/unix/sysv/linux/spawni.c:382
#2  0x000000aaaaac76b0 in child_execute_job (child=0xffffff8770, good_stdin=<optimized out>, argv=0xaaaab26cb0) at ../make-4.3/src/job.c:2396
#3  0x000000aaaaac2678 in func_shell_base (o=0xaaaab1e350 "", argv=<optimized out>, trim_newlines=<optimized out>) at ../make-4.3/src/function.c:1790
#4  0x000000aaaaac2c38 in handle_function (op=0xffffff8930, stringp=<optimized out>) at ../make-4.3/src/function.c:2544
#5  0x000000aaaaabaa10 in variable_expand_string (line=<optimized out>, string=<optimized out>, length=<optimized out>) at ../make-4.3/src/expand.c:262
#6  0x000000aaaaabb5b4 in allocated_variable_expand_for_file (line=<optimized out>, file=<optimized out>) at ../make-4.3/src/expand.c:566
#7  0x000000aaaaadc11c in do_variable_definition (flocp=0xffffff8d70, varname=0xaaaab1fca0 "SUBARCH", 
    value=0xaaaab24e6b "$(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ -e s/sun4u/sparc64/ -e s/arm.*/arm/ -e s/sa110/arm/ -e s/s390x/s390/ -e s/parisc64/parisc/ -e s/ppc.*/powerpc/ -e s/mips.*/mips/ -e s/sh[234].*/s"..., origin=<optimized out>, flavor=<optimized out>, target_var=<optimized out>) at ../make-4.3/src/variable.c:1187
#8  0x000000aaaaadccac in try_variable_definition (flocp=0xffffff8d70, line=<optimized out>, origin=<optimized out>, target_var=<optimized out>)
    at ../make-4.3/src/variable.c:1633
#9  0x000000aaaaad2dfc in eval (ebuf=<optimized out>, set_default=<optimized out>) at ../make-4.3/src/read.c:750
#10 0x000000aaaaad451c in eval_makefile (filename=<optimized out>, flags=<optimized out>) at ../make-4.3/src/read.c:436
#11 0x000000aaaaad3cc4 in eval (ebuf=<optimized out>, set_default=<optimized out>) at ../make-4.3/src/read.c:908
#12 0x000000aaaaad451c in eval_makefile (filename=<optimized out>, flags=<optimized out>) at ../make-4.3/src/read.c:436
#13 0x000000aaaaad4a70 in read_all_makefiles (makefiles=<optimized out>) at ../make-4.3/src/read.c:258
#14 0x000000aaaaab4f9c in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at ../make-4.3/src/main.c:1945
(gdb) c
Continuing.
[Attaching after process 3441 vfork to child process 3444]
[New inferior 2 (process 3444)]
[Switching to process 3444]

Thread 2.1 "make" hit Breakpoint 7, __spawni_child (arguments=0xffffff8350) at ../sysdeps/unix/sysv/linux/spawni.c:123
123	../sysdeps/unix/sysv/linux/spawni.c: No such file or directory.
(gdb) bt
#0  __spawni_child (arguments=0xffffff8350) at ../sysdeps/unix/sysv/linux/spawni.c:123
#1  0x000000fff7ef3d28 in __thread_start () at ../sysdeps/unix/sysv/linux/mips/clone.S:135
(gdb) c
Continuing.

Thread 2.1 "make" hit Breakpoint 2, getuid () at ../sysdeps/unix/syscall-template.S:59
59	../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) bt
#0  getuid () at ../sysdeps/unix/syscall-template.S:59
#1  0x000000fff7eddedc in __spawni_child (arguments=0xffffff8350) at ../sysdeps/unix/sysv/linux/spawni.c:189
#2  0x000000fff7ef3d28 in __thread_start () at ../sysdeps/unix/sysv/linux/mips/clone.S:135
(gdb) c
Continuing.

Thread 2.1 "make" hit Breakpoint 4, 0x000000fff7eb4f10 in execve () at ../sysdeps/unix/syscall-template.S:78
78	../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) bt
#0  0x000000fff7eb4f10 in execve () at ../sysdeps/unix/syscall-template.S:78
#1  0x000000fff7eddd10 in __spawni_child (arguments=0xffffff8350) at ../sysdeps/unix/sysv/linux/spawni.c:292
#2  0x000000fff7ef3d28 in __thread_start () at ../sysdeps/unix/sysv/linux/mips/clone.S:135

It looks like we did not hit setresuid breakpoint
but reached execve ... strange

Looking at glibc __spawni_child code
------------------------------------

From cat -n ./2.31-r0/git/sysdeps/unix/sysv/linux/spawni.c

   187    /* Set the effective user and group IDs.  */
   188    if ((attr->__flags & POSIX_SPAWN_RESETIDS) != 0
   189        && (local_seteuid (__getuid ()) != 0
   190            || local_setegid (__getgid ()) != 0))
   191      goto fail;
   192  

What is local_seteuid
---------------------

From cat -n 2.31-r0/git/sysdeps/unix/sysv/linux/local-setxid.h

     8	# define local_seteuid(id) INLINE_SYSCALL (setresuid, 3, -1, id, -1)

    15  # define local_setegid(id) INLINE_SYSCALL (setresgid, 3, -1, id, -1)

That explains why setresuid breakpoint was not hit. Code calls
directly system call.


What is internal_syscall3
-------------------------

From cat -n 2.31-r0/git/sysdeps/unix/sysv/linux/mips/mips64/n64/sysdep.h

   181  #define internal_syscall3(v0_init, input, number, err,                  \
   182                            arg1, arg2, arg3)                             \
   183  ({                                                                      \
   184          long _sys_result;                                               \
   185                                                                          \
   186          {                                                               \
   187          register long __s0 asm ("$16") __attribute__ ((unused))         \
   188            = (number);                                                   \
   189          register long __v0 asm ("$2");                                  \
   190          register long __a0 asm ("$4") = (long) (arg1);                  \
   191          register long __a1 asm ("$5") = (long) (arg2);                  \
   192          register long __a2 asm ("$6") = (long) (arg3);                  \
   193          register long __a3 asm ("$7");                                  \
   194          __asm__ volatile (                                              \
   195          ".set\tnoreorder\n\t"                                           \
   196          v0_init                                                         \
   197          "syscall\n\t"                                                   \
   198          ".set\treorder"                                                 \
   199          : "=r" (__v0), "=r" (__a3)                                      \
   200          : input, "r" (__a0), "r" (__a1), "r" (__a2)                     \
   201          : __SYSCALL_CLOBBERS);                                          \
   202          err = __a3;                                                     \
   203          _sys_result = __v0;                                             \
   204          }                                                               \
   205          _sys_result;                                                    \
   206  })

In gdb again
------------

(gdb) bt
#0  getuid () at ../sysdeps/unix/syscall-template.S:59
#1  0x000000fff7eddedc in __spawni_child (arguments=0xffffff8350) at ../sysdeps/unix/sysv/linux/spawni.c:189
#2  0x000000fff7ef3d28 in __thread_start () at ../sysdeps/unix/sysv/linux/mips/clone.S:135
(gdb) up
#1  0x000000fff7eddedc in __spawni_child (arguments=0xffffff8350) at ../sysdeps/unix/sysv/linux/spawni.c:189
189	../sysdeps/unix/sysv/linux/spawni.c: No such file or directory.
(gdb) x /20i $pc - 20
   0xfff7eddec8 <__spawni_child+1032>:	nop
   0xfff7eddecc <__spawni_child+1036>:	b	0xfff7eddcc0 <__spawni_child+512>
   0xfff7edded0 <__spawni_child+1040>:	lw	a0,4(s1)
   0xfff7edded4 <__spawni_child+1044>:	jalr	t9
   0xfff7edded8 <__spawni_child+1048>:	nop
=> 0xfff7eddedc <__spawni_child+1052>:	li	a2,-1
   0xfff7eddee0 <__spawni_child+1056>:	dext	a1,v0,0x0,0x20
   0xfff7eddee4 <__spawni_child+1060>:	li	v0,5115
   0xfff7eddee8 <__spawni_child+1064>:	syscall
   0xfff7eddeec <__spawni_child+1068>:	bnez	a3,0xfff7ede000 <__spawni_child+1344>
   0xfff7eddef0 <__spawni_child+1072>:	nop
   0xfff7eddef4 <__spawni_child+1076>:	bnez	v0,0xfff7edde00 <__spawni_child+832>
   0xfff7eddef8 <__spawni_child+1080>:	ld	t9,-26712(gp)
   0xfff7eddefc <__spawni_child+1084>:	jalr	t9
   0xfff7eddf00 <__spawni_child+1088>:	nop
   0xfff7eddf04 <__spawni_child+1092>:	li	a2,-1
   0xfff7eddf08 <__spawni_child+1096>:	dext	a1,v0,0x0,0x20
   0xfff7eddf0c <__spawni_child+1100>:	li	v0,5117
   0xfff7eddf10 <__spawni_child+1104>:	syscall
   0xfff7eddf14 <__spawni_child+1108>:	bnez	a3,0xfff7ede000 <__spawni_child+1344>
   0xfff7eddf18 <__spawni_child+1112>:	nop
(gdb) display /i $pc
1: x/i $pc
=> 0xfff7eddedc <__spawni_child+1052>:	li	a2,-1
(gdb) si
0x000000fff7eb5a04	59	../sysdeps/unix/syscall-template.S: No such file or directory.
1: x/i $pc
=> 0xfff7eb5a04 <getuid+4>:	syscall
(gdb) info reg
                  zero               at               v0               v1
 R0   0000000000000000 000000001400a4e1 00000000000013ec 0000000000000000 
                    a0               a1               a2               a3
 R4   000000000000007f 000000fff7ff2d80 0000000000000000 0000000000000000 
                    a4               a5               a6               a7
 R8   0000000000000000 0000000000000000 0000000000000000 0000000000000000 
                    t0               t1               t2               t3
 R12  0000000000000000 ffffffff84080018 ffffffff801554b8 0000000000000000 
                    s0               s1               s2               s3
 R16  000000ffffff8588 000000ffffff84b8 0000000000000001 000000ffffff8350 
                    s4               s5               s6               s7
 R20  000000fff7e2ef80 000000fff7eb4f10 000000000000007f 000000000000007f 
                    t8               t9               k0               k1
 R24  0000000000000000 000000fff7eb5a00 0000000000000010 0000000000000000 
                    gp               sp               s8               ra
 R28  000000fff7f9f5e0 000000fff7ff2e20 000000fff7ff2e20 000000fff7eddedc 
                status               lo               hi         badvaddr
      000000000400a4f3 1b6ee11164e1e246 eb05325e15c10102 000000fff7ff2fe0 
                 cause               pc
      0000000010800024 000000fff7eb5a04 
                  fcsr              fir          restart
              00000000         007f0000 0000000000000000 
(gdb) p /d $a0
$1 = 127

a0 value is 127! But it should be -1 according to local_seteuid call.
Why is that?

This code looks quite wrong:

   0xfff7edded0 <__spawni_child+1040>:	lw	a0,4(s1)
   0xfff7edded4 <__spawni_child+1044>:	jalr	t9
   0xfff7edded8 <__spawni_child+1048>:	nop
=> 0xfff7eddedc <__spawni_child+1052>:	li	a2,-1
   0xfff7eddee0 <__spawni_child+1056>:	dext	a1,v0,0x0,0x20
   0xfff7eddee4 <__spawni_child+1060>:	li	v0,5115
   0xfff7eddee8 <__spawni_child+1064>:	syscall

There is not load of -1 into a0 before jumping to setresuid system call.

In disassembly at certain point one can see:

   0x000000fff7edde1c <+860>:	li	a0,127

It seems either internal_syscall3 is wrong or compiler does not
process it correctly ... quite bad ...

Looking at make make-4.3/src/job.c
----------------------------------

It seems that posix_spawn would be used only if USE_POSIX_SPAWN defined.

Test for USE_POSIX_SPAWN is in configure.ac
-------------------------------------------

From cat -n make-4.3/configure.ac

   382  AS_IF([test "$make_cv_posix_spawn" = yes],
   383    AC_CACHE_CHECK([for posix_spawn that fails synchronously],
   384      [make_cv_synchronous_posix_spawn],
   385      [make_cv_synchronous_posix_spawn=no
   386       AC_RUN_IFELSE([AC_LANG_SOURCE([[
   387         #include <spawn.h>
   388         #include <string.h>
   389  
   390         extern char **environ;
   391  
   392         int main() {
   393           char* path = strdup("./non-existent");
   394           char *argv[[2]];
   395           argv[[0]] = path;
   396           argv[[1]] =  0;
   397           return posix_spawn(0, path, 0, 0, argv, environ);
   398         }]])],
   399         [make_cv_synchronous_posix_spawn=no],
   400         [make_cv_synchronous_posix_spawn=yes],
   401         [make_cv_synchronous_posix_spawn="no (cross-compiling)"])]))
   402  
   403  AS_CASE([/$user_posix_spawn/$make_cv_posix_spawn/$make_cv_synchronous_posix_spawn/],
   404    [*/no/*], [make_cv_posix_spawn=no],
   405    [AC_DEFINE(USE_POSIX_SPAWN, 1, [Define to 1 to use posix_spawn().])
   406    ])

Note USE_POSIX_SPAWN code was not present in make-4.2.1
-------------------------------------------------------

Workaround
-----------

In case of mips disable use of posix_spawn through ac_cv_func_posix_spawn=no
configure option.

Testing
-------

Tested 'bitbake core-image-sato-sdk -c testimage' all tested are passed.

Thanks,
Victor

________________________________________
From: Richard Purdie <richard.purdie@linuxfoundation.org>
Sent: Thursday, February 27, 2020 11:45 PM
To: Jens Rehsack; openembedded-core@lists.openembedded.org
Cc: Bruce Ashfield; Victor Kamensky (kamensky)
Subject: Re: [OE-core] [PATCH v3] make: 4.2.1 -> 4.3

On Mon, 2020-02-24 at 15:39 +0100, Jens Rehsack wrote:
> Announcement: https://lists.gnu.org/archive/html/bug-make/2020-01/msg00057.html
>
> 1) Remove upstream provided patches 0001-glob-Do-not-assume-glibc-glob-internals.patch
>    and 0002-glob-Do-not-assume-glibc-glob-internals.patch.
>
> 2) License has been changed to GPLv3 only
>
> 3) Important bug-fix is
>    * https://lists.gnu.org/archive/html/bug-make/2018-09/msg00006.html
>
> 4) Backward-incompatibilities:
>    * Number signs (#) appearing inside a macro reference or function invocation
>      no longer introduce comments and should not be escaped with backslashes
>    * Previously appending using '+=' to an empty variable would result in a value
>      starting with a space.  Now the initial space is only added if the variable
>      already contains some value.  Similarly, appending an empty string does not
>      add a trailing space.
>
> Fix incompatibility issues between gnulib bundled with updated make fix issues
> in w32 compat sources.
>
> Signed-off-by: Jens Rehsack <sno@netbsd.org

Unfortunately, crazy as this sounds, this is causing mips problems on
target with kernel module building. We only see the following failures
when this patch is included:

https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/1612
https://autobuilder.yoctoproject.org/typhoon/#/builders/102/builds/315
https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/1616

Cheers,

Richard


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-make-disable-use-of-posix_spawn-on-mips.patch --]
[-- Type: text/x-patch; name="0001-make-disable-use-of-posix_spawn-on-mips.patch", Size: 1586 bytes --]

From 088ebd7b1c50babcf131cb0364cf8841b697863b Mon Sep 17 00:00:00 2001
From: Victor Kamensky <kamensky@cisco.com>
Date: Fri, 28 Feb 2020 22:26:11 -0800
Subject: [PATCH] make: disable use of posix_spawn on mips

After make-4.3 migration child_execute_job function started
using posix_spawn function, which happens to be broken on mips.

It manifests itself as when make executed by root, it switches
real user id to wrong value because of some issues with direct
setresuid system call done in glibc __spawni_child function
through inline assemble and/or gcc compiling it produces wrong
code. I.e instead of passing -1 posix_spawn function incorrectly
passes 127 as ruid. Subsequently job started by make can fail
with permission issue because they run under wrong user.

For now workaround is used by explicitly disabling posix_spawn
call use by make on mips through configure variable.

Signed-off-by: Victor Kamensky <kamensky@cisco.com>
---
 meta/recipes-devtools/make/make_4.3.bb | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-devtools/make/make_4.3.bb b/meta/recipes-devtools/make/make_4.3.bb
index 70caf0ae16..5fa7059169 100644
--- a/meta/recipes-devtools/make/make_4.3.bb
+++ b/meta/recipes-devtools/make/make_4.3.bb
@@ -12,6 +12,9 @@ SRC_URI += "\
 
 EXTRA_OECONF += "--without-guile"
 
+EXTRA_OECONF_append_mips=" ac_cv_func_posix_spawn=no"
+EXTRA_OECONF_append_mips64=" ac_cv_func_posix_spawn=no"
+
 SRC_URI[md5sum] = "d5c40e7bd1e97a7404f5d3be982f479a"
 SRC_URI[sha256sum] = "de1a441c4edf952521db30bfca80baae86a0ff1acd0a00402999344f04c45e82"
 
-- 
2.21.1


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH v3] make: 4.2.1 -> 4.3
  2020-02-29  7:55   ` Victor Kamensky (kamensky)
@ 2020-02-29 10:12     ` Richard Purdie
  2020-02-29 13:01       ` Richard Purdie
  0 siblings, 1 reply; 6+ messages in thread
From: Richard Purdie @ 2020-02-29 10:12 UTC (permalink / raw)
  To: Victor Kamensky (kamensky), Jens Rehsack, openembedded-core

Hi Victor,

On Sat, 2020-02-29 at 07:55 +0000, Victor Kamensky (kamensky) wrote:
> I was able to reproduce and debugged the mips new make issue.
> Workaround patch attached.
> 
> In summary: kernelmodule.KernelModuleTest.test_kernel_module
> fails with 'fatal error: opening dependency file
> scripts/basic/.fixdep.d: Permission denied'
> 
> New make-4.3 started using posix_spawn glibc call,
> that happens to be broken on mips. On mips during posix_spawn call
> __spawni_child function that is executed after clone makes
> setresuid call with spurious/wrong value of ruid. Instead of -1
> it passes 127, effectively switching real user id since test
> runs under root. Subsequent gcc runs under wrong ruid and it
> fails to write into /usr/src/kernel/scripts/basic directory.
> 
> __spawni_child uses direct system call invocation for setresuid
> rather calling glibc function. And there is wrong code generation
> issue somewhere in inline assembly for direct system call and/or
> way how compiler processes it. I did not dig into it yet, it
> looks as quite hard problem and needs more time.
> 
> For now I've made workaround patch that just through configure
> ac_cv_func_posix_spawn=no parameter tells make configure not to use
> posix_spawn on mips.
> 
> I will also send suggested patch through 'git send-mail'
> 
> Tested workaround on qemumips64, did not have time for
> qemumips full test, verified build only and made sure that
> posix_spawn is not used. But I believe it is the same issue.

Thanks for the quick turnaround on this, sounds like good progress on a
workaround! It does massively help with the preparation for 3.1 and is
much appreciated.

I've put the patch in for testing. It sounds like there may be an
upstream glibc patch which might be the fix of the underlying issue.
I'll probably look to merge the workaround and then we could look at
the glibc upgrade next.

I'll report back on how the testing goes.

Cheers,

Richard





^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH v3] make: 4.2.1 -> 4.3
  2020-02-29 10:12     ` Richard Purdie
@ 2020-02-29 13:01       ` Richard Purdie
  0 siblings, 0 replies; 6+ messages in thread
From: Richard Purdie @ 2020-02-29 13:01 UTC (permalink / raw)
  To: Victor Kamensky (kamensky), Jens Rehsack, openembedded-core

On Sat, 2020-02-29 at 10:12 +0000, Richard Purdie wrote:
> Hi Victor,
> 
> On Sat, 2020-02-29 at 07:55 +0000, Victor Kamensky (kamensky) wrote:
> > I was able to reproduce and debugged the mips new make issue.
> > Workaround patch attached.
> > 
> > In summary: kernelmodule.KernelModuleTest.test_kernel_module
> > fails with 'fatal error: opening dependency file
> > scripts/basic/.fixdep.d: Permission denied'
> > 
> > New make-4.3 started using posix_spawn glibc call,
> > that happens to be broken on mips. On mips during posix_spawn call
> > __spawni_child function that is executed after clone makes
> > setresuid call with spurious/wrong value of ruid. Instead of -1
> > it passes 127, effectively switching real user id since test
> > runs under root. Subsequent gcc runs under wrong ruid and it
> > fails to write into /usr/src/kernel/scripts/basic directory.
> > 
> > __spawni_child uses direct system call invocation for setresuid
> > rather calling glibc function. And there is wrong code generation
> > issue somewhere in inline assembly for direct system call and/or
> > way how compiler processes it. I did not dig into it yet, it
> > looks as quite hard problem and needs more time.
> > 
> > For now I've made workaround patch that just through configure
> > ac_cv_func_posix_spawn=no parameter tells make configure not to use
> > posix_spawn on mips.
> > 
> > I will also send suggested patch through 'git send-mail'
> > 
> > Tested workaround on qemumips64, did not have time for
> > qemumips full test, verified build only and made sure that
> > posix_spawn is not used. But I believe it is the same issue.
> 
> Thanks for the quick turnaround on this, sounds like good progress on
> a
> workaround! It does massively help with the preparation for 3.1 and
> is
> much appreciated.
> 
> I've put the patch in for testing. It sounds like there may be an
> upstream glibc patch which might be the fix of the underlying issue.
> I'll probably look to merge the workaround and then we could look at
> the glibc upgrade next.
> 
> I'll report back on how the testing goes.

Looks good, thanks!

I'll merge with a comment about the issue and likely fix when we can
drop it.

Cheers,

Richard



^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2020-02-29 13:01 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-24 14:39 [PATCH v3] make: 4.2.1 -> 4.3 Jens Rehsack
2020-02-25 20:07 ` Adrian Bunk
2020-02-28  7:45 ` Richard Purdie
2020-02-29  7:55   ` Victor Kamensky (kamensky)
2020-02-29 10:12     ` Richard Purdie
2020-02-29 13:01       ` Richard Purdie

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.