All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 1/4] python3: fix ftplib with TLS 1.3
@ 2018-09-10 13:31 Ross Burton
  2018-09-10 13:31 ` [PATCH 2/4] python3: remove redundant patch Ross Burton
                   ` (3 more replies)
  0 siblings, 4 replies; 9+ messages in thread
From: Ross Burton @ 2018-09-10 13:31 UTC (permalink / raw)
  To: openembedded-core

With OpenSSL 1.1.x TLS 1.3 can be used, so backport a patch from Python 3.6 to
fix the ftplib unit test.

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/recipes-devtools/python/python3/ftplib.patch | 57 +++++++++++++++++++++++
 meta/recipes-devtools/python/python3_3.5.5.bb     |  1 +
 2 files changed, 58 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3/ftplib.patch

diff --git a/meta/recipes-devtools/python/python3/ftplib.patch b/meta/recipes-devtools/python/python3/ftplib.patch
new file mode 100644
index 00000000000..8bb3cd02196
--- /dev/null
+++ b/meta/recipes-devtools/python/python3/ftplib.patch
@@ -0,0 +1,57 @@
+From cabe916dc694997d4892b58986e73a713d5a2f8d Mon Sep 17 00:00:00 2001
+From: "Miss Islington (bot)"
+ <31488909+miss-islington@users.noreply.github.com>
+Date: Thu, 16 Aug 2018 15:38:03 -0400
+Subject: [PATCH] [3.6] bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787)
+ (#8790)
+
+Read from data socket to avoid "[SSL] shutdown while in init" exception
+during shutdown of the dummy server.
+
+Signed-off-by: Christian Heimes <christian@python.org>
+
+
+<!-- issue-number: [bpo-34391](https://www.bugs.python.org/issue34391) -->
+https://bugs.python.org/issue34391
+<!-- /issue-number -->
+(cherry picked from commit 1590c393360df059160145e7475754427bfc6680)
+
+
+Co-authored-by: Christian Heimes <christian@python.org>
+---
+ Lib/test/test_ftplib.py                                         | 5 +++++
+ Misc/NEWS.d/next/Tests/2018-08-16-18-48-47.bpo-34391.ouNfxC.rst | 1 +
+ 2 files changed, 6 insertions(+)
+ create mode 100644 Misc/NEWS.d/next/Tests/2018-08-16-18-48-47.bpo-34391.ouNfxC.rst
+
+diff --git a/Lib/test/test_ftplib.py b/Lib/test/test_ftplib.py
+index 44dd73aeca..4ff2f71afb 100644
+--- a/Lib/test/test_ftplib.py
++++ b/Lib/test/test_ftplib.py
+@@ -876,18 +876,23 @@ class TestTLS_FTPClass(TestCase):
+         # clear text
+         with self.client.transfercmd('list') as sock:
+             self.assertNotIsInstance(sock, ssl.SSLSocket)
++            self.assertEqual(sock.recv(1024), LIST_DATA.encode('ascii'))
+         self.assertEqual(self.client.voidresp(), "226 transfer complete")
+ 
+         # secured, after PROT P
+         self.client.prot_p()
+         with self.client.transfercmd('list') as sock:
+             self.assertIsInstance(sock, ssl.SSLSocket)
++            # consume from SSL socket to finalize handshake and avoid
++            # "SSLError [SSL] shutdown while in init"
++            self.assertEqual(sock.recv(1024), LIST_DATA.encode('ascii'))
+         self.assertEqual(self.client.voidresp(), "226 transfer complete")
+ 
+         # PROT C is issued, the connection must be in cleartext again
+         self.client.prot_c()
+         with self.client.transfercmd('list') as sock:
+             self.assertNotIsInstance(sock, ssl.SSLSocket)
++            self.assertEqual(sock.recv(1024), LIST_DATA.encode('ascii'))
+         self.assertEqual(self.client.voidresp(), "226 transfer complete")
+ 
+     def test_login(self):
+-- 
+2.11.0
+
diff --git a/meta/recipes-devtools/python/python3_3.5.5.bb b/meta/recipes-devtools/python/python3_3.5.5.bb
index 40896dbc082..93878e4e45c 100644
--- a/meta/recipes-devtools/python/python3_3.5.5.bb
+++ b/meta/recipes-devtools/python/python3_3.5.5.bb
@@ -42,6 +42,7 @@ SRC_URI += "\
             file://Use-correct-CFLAGS-for-extensions-when-cross-compili.patch \
             file://0002-Makefile-add-target-to-split-profile-generation.patch \
             file://float-endian.patch \
+            file://ftplib.patch \
            "
 SRC_URI[md5sum] = "f3763edf9824d5d3a15f5f646083b6e0"
 SRC_URI[sha256sum] = "063d2c3b0402d6191b90731e0f735c64830e7522348aeb7ed382a83165d45009"
-- 
2.11.0



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 2/4] python3: remove redundant patch
  2018-09-10 13:31 [PATCH 1/4] python3: fix ftplib with TLS 1.3 Ross Burton
@ 2018-09-10 13:31 ` Ross Burton
  2018-09-10 16:12   ` Khem Raj
  2018-09-10 13:31 ` [PATCH 3/4] python3: remove patch that inadvertently disables xattrs Ross Burton
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 9+ messages in thread
From: Ross Burton @ 2018-09-10 13:31 UTC (permalink / raw)
  To: openembedded-core

None of the Python invocations that this changes are actually called, and
there's no need to provide a HOSTPGEN variable when the recipe can just override
PGEN directly.

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 .../python/python3-native_3.5.5.bb                 |  1 -
 .../python3/0001-cross-compile-support.patch       | 96 ----------------------
 meta/recipes-devtools/python/python3_3.5.5.bb      |  3 +-
 3 files changed, 1 insertion(+), 99 deletions(-)
 delete mode 100644 meta/recipes-devtools/python/python3/0001-cross-compile-support.patch

diff --git a/meta/recipes-devtools/python/python3-native_3.5.5.bb b/meta/recipes-devtools/python/python3-native_3.5.5.bb
index 3098027d803..7b940a16d28 100644
--- a/meta/recipes-devtools/python/python3-native_3.5.5.bb
+++ b/meta/recipes-devtools/python/python3-native_3.5.5.bb
@@ -7,7 +7,6 @@ DISTRO_SRC_URI_linuxstdbase = ""
 SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
 file://12-distutils-prefix-is-inside-staging-area.patch \
 file://python-config.patch \
-file://0001-cross-compile-support.patch \
 file://030-fixup-include-dirs.patch \
 file://070-dont-clean-ipkg-install.patch \
 file://080-distutils-dont_adjust_files.patch \
diff --git a/meta/recipes-devtools/python/python3/0001-cross-compile-support.patch b/meta/recipes-devtools/python/python3/0001-cross-compile-support.patch
deleted file mode 100644
index 7cd7e3b4906..00000000000
--- a/meta/recipes-devtools/python/python3/0001-cross-compile-support.patch
+++ /dev/null
@@ -1,96 +0,0 @@
-From ecde3ea170999a9ef734e8af4d7c25be5ba81697 Mon Sep 17 00:00:00 2001
-From: Alexander Kanavin <alex.kanavin@gmail.com>
-Date: Fri, 31 Mar 2017 15:42:46 +0300
-Subject: [PATCH] cross-compile support
-
-We cross compile python. This patch uses tools from host/native
-python instead of in-tree tools
-
--Khem
-
-Rebased on 3.5.4
-
-Upstream-Status: Inappropriate[Configuration Specific]
-Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
-Signed-off-by: Derek Straka <derek@asterius.io>
----
- Makefile.pre.in | 14 ++++++++------
- 1 file changed, 8 insertions(+), 6 deletions(-)
-
-diff --git a/Makefile.pre.in b/Makefile.pre.in
-index 144c1f8629..f252ac2417 100644
---- a/Makefile.pre.in
-+++ b/Makefile.pre.in
-@@ -223,6 +223,7 @@ LIBOBJS=	@LIBOBJS@
-
- PYTHON=		python$(EXE)
- BUILDPYTHON=	python$(BUILDEXE)
-+HOSTPYTHON=    $(BUILDPYTHON)
-
- PYTHON_FOR_REGEN=@PYTHON_FOR_REGEN@
- PYTHON_FOR_BUILD=@PYTHON_FOR_BUILD@
-@@ -277,6 +278,7 @@ LIBFFI_INCLUDEDIR=	@LIBFFI_INCLUDEDIR@
- ##########################################################################
- # Parser
- PGEN=		Parser/pgen$(EXE)
-+HOSTPGEN=	$(PGEN)$(EXE)
-
- PSRCS=		\
- 		Parser/acceler.c \
-@@ -478,7 +480,7 @@ build_all_generate_profile:
-
- run_profile_task:
- 	: # FIXME: can't run for a cross build
--	$(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true
-+	$(LLVM_PROF_FILE) $(RUNSHARED) $(HOSTPYTHON) $(PROFILE_TASK) || true
-
- build_all_merge_profile:
- 	$(LLVM_PROF_MERGER)
-@@ -772,7 +774,7 @@ regen-grammar: $(PGEN)
- 	# Regenerate Include/graminit.h and Python/graminit.c
- 	# from Grammar/Grammar using pgen
- 	@$(MKDIR_P) Include
--	$(PGEN) $(srcdir)/Grammar/Grammar \
-+	$(HOSTPGEN) $(srcdir)/Grammar/Grammar \
- 		$(srcdir)/Include/graminit.h \
- 		$(srcdir)/Python/graminit.c
-
-@@ -978,7 +980,7 @@ $(LIBRARY_OBJS) $(MODOBJS) Programs/python.o: $(PYTHON_HEADERS)
- ######################################################################
-
- TESTOPTS=	$(EXTRATESTOPTS)
--TESTPYTHON=	$(RUNSHARED) ./$(BUILDPYTHON) $(TESTPYTHONOPTS)
-+TESTPYTHON=	$(RUNSHARED) $(HOSTPYTHON) $(TESTPYTHONOPTS)
- TESTRUNNER=	$(TESTPYTHON) $(srcdir)/Tools/scripts/run_tests.py
- TESTTIMEOUT=	3600
-
-@@ -1470,7 +1472,7 @@ frameworkinstallstructure:	$(LDLIBRARY)
- 		fi; \
- 	done
- 	$(LN) -fsn include/python$(LDVERSION) $(DESTDIR)$(prefix)/Headers
--	sed 's/%VERSION%/'"`$(RUNSHARED) ./$(BUILDPYTHON) -c 'import platform; print(platform.python_version())'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist
-+	sed 's/%VERSION%/'"`$(RUNSHARED) $(HOSTPYTHON) -c 'import platform; print(platform.python_version())'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist
- 	$(LN) -fsn $(VERSION) $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/Versions/Current
- 	$(LN) -fsn Versions/Current/$(PYTHONFRAMEWORK) $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/$(PYTHONFRAMEWORK)
- 	$(LN) -fsn Versions/Current/Headers $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/Headers
-@@ -1543,7 +1545,7 @@ Python/dtoa.o: Python/dtoa.c
-
- # Run reindent on the library
- reindent:
--	./$(BUILDPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib
-+	$(HOSTPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib
-
- # Rerun configure with the same options as it was run last time,
- # provided the config.status script exists
-@@ -1678,7 +1680,7 @@ funny:
-
- # Perform some verification checks on any modified files.
- patchcheck: @DEF_MAKE_RULE@
--	$(RUNSHARED) ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
-+	$(RUNSHARED) $(HOSTPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
-
- # Dependencies
-
---
-2.11.0
-
diff --git a/meta/recipes-devtools/python/python3_3.5.5.bb b/meta/recipes-devtools/python/python3_3.5.5.bb
index 93878e4e45c..6598b486afe 100644
--- a/meta/recipes-devtools/python/python3_3.5.5.bb
+++ b/meta/recipes-devtools/python/python3_3.5.5.bb
@@ -11,7 +11,6 @@ DISTRO_SRC_URI ?= "file://sitecustomize.py"
 DISTRO_SRC_URI_linuxstdbase = ""
 SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
 file://python-config.patch \
-file://0001-cross-compile-support.patch \
 file://030-fixup-include-dirs.patch \
 file://070-dont-clean-ipkg-install.patch \
 file://080-distutils-dont_adjust_files.patch \
@@ -101,7 +100,7 @@ do_configure_append() {
 }
 
 run_make() {
-	oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
+	oe_runmake PGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
 		HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python3-native/python3 \
 		STAGING_LIBDIR=${STAGING_LIBDIR} \
 		STAGING_INCDIR=${STAGING_INCDIR} \
-- 
2.11.0



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 3/4] python3: remove patch that inadvertently disables xattrs
  2018-09-10 13:31 [PATCH 1/4] python3: fix ftplib with TLS 1.3 Ross Burton
  2018-09-10 13:31 ` [PATCH 2/4] python3: remove redundant patch Ross Burton
@ 2018-09-10 13:31 ` Ross Burton
  2018-09-10 13:31 ` [PATCH 4/4] python3: fix multiprocessing Ross Burton
  2018-09-10 14:02 ` ✗ patchtest: failure for "python3: fix ftplib with TLS 1..." and 3 more Patchwork
  3 siblings, 0 replies; 9+ messages in thread
From: Ross Burton @ 2018-09-10 13:31 UTC (permalink / raw)
  To: openembedded-core

This patch dates back to the addition of the Python 3 recipe to oe-core, and as
listxattr is never added to supports_follow_symlinks the extended attribute
support will never be enabled.

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/recipes-devtools/python/python3-native_3.5.5.bb    |  1 -
 .../python/python3/shutil-follow-symlink-fix.patch      | 17 -----------------
 meta/recipes-devtools/python/python3_3.5.5.bb           |  1 -
 3 files changed, 19 deletions(-)
 delete mode 100644 meta/recipes-devtools/python/python3/shutil-follow-symlink-fix.patch

diff --git a/meta/recipes-devtools/python/python3-native_3.5.5.bb b/meta/recipes-devtools/python/python3-native_3.5.5.bb
index 7b940a16d28..b231542f46f 100644
--- a/meta/recipes-devtools/python/python3-native_3.5.5.bb
+++ b/meta/recipes-devtools/python/python3-native_3.5.5.bb
@@ -15,7 +15,6 @@ file://150-fix-setupterm.patch \
 file://python-3.3-multilib.patch \
 file://03-fix-tkinter-detection.patch \
 file://avoid_warning_about_tkinter.patch \
-file://shutil-follow-symlink-fix.patch \
 file://0001-h2py-Fix-issue-13032-where-it-fails-with-UnicodeDeco.patch \
 file://sysroot-include-headers.patch \
 file://unixccompiler.patch \
diff --git a/meta/recipes-devtools/python/python3/shutil-follow-symlink-fix.patch b/meta/recipes-devtools/python/python3/shutil-follow-symlink-fix.patch
deleted file mode 100644
index 802b1c72038..00000000000
--- a/meta/recipes-devtools/python/python3/shutil-follow-symlink-fix.patch
+++ /dev/null
@@ -1,17 +0,0 @@
-shutils should consider symlinks
-
--Khem
-
-Upstream-Status: Pending
-
---- a/Lib/shutil.py	2013-01-29 12:31:06.926555779 -0800
-+++ b/Lib/shutil.py	2013-01-29 16:31:39.097554182 -0800
-@@ -132,7 +132,7 @@ def copymode(src, dst, *, follow_symlink
-     st = stat_func(src)
-     chmod_func(dst, stat.S_IMODE(st.st_mode))
- 
--if hasattr(os, 'listxattr'):
-+if hasattr(os, 'listxattr') and os.listxattr in os.supports_follow_symlinks:
-     def _copyxattr(src, dst, *, follow_symlinks=True):
-         """Copy extended filesystem attributes from `src` to `dst`.
- 
diff --git a/meta/recipes-devtools/python/python3_3.5.5.bb b/meta/recipes-devtools/python/python3_3.5.5.bb
index 6598b486afe..02decf470c8 100644
--- a/meta/recipes-devtools/python/python3_3.5.5.bb
+++ b/meta/recipes-devtools/python/python3_3.5.5.bb
@@ -28,7 +28,6 @@ SRC_URI += "\
             file://cgi_py.patch \
             file://host_include_contamination.patch \
             file://python-3.3-multilib.patch \
-            file://shutil-follow-symlink-fix.patch \
             file://sysroot-include-headers.patch \
             file://unixccompiler.patch \
             file://avoid-ncursesw-include-path.patch \
-- 
2.11.0



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 4/4] python3: fix multiprocessing
  2018-09-10 13:31 [PATCH 1/4] python3: fix ftplib with TLS 1.3 Ross Burton
  2018-09-10 13:31 ` [PATCH 2/4] python3: remove redundant patch Ross Burton
  2018-09-10 13:31 ` [PATCH 3/4] python3: remove patch that inadvertently disables xattrs Ross Burton
@ 2018-09-10 13:31 ` Ross Burton
  2018-09-10 14:02 ` ✗ patchtest: failure for "python3: fix ftplib with TLS 1..." and 3 more Patchwork
  3 siblings, 0 replies; 9+ messages in thread
From: Ross Burton @ 2018-09-10 13:31 UTC (permalink / raw)
  To: openembedded-core

glibc 2.28 slightly changed the behaviour of sigaddset() which broke
multiprocessing.  Backport a patch from Python 3.6 to solve this.

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/recipes-devtools/python/python3/signal.patch | 56 +++++++++++++++++++++++
 meta/recipes-devtools/python/python3_3.5.5.bb     |  1 +
 2 files changed, 57 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3/signal.patch

diff --git a/meta/recipes-devtools/python/python3/signal.patch b/meta/recipes-devtools/python/python3/signal.patch
new file mode 100644
index 00000000000..534a097771c
--- /dev/null
+++ b/meta/recipes-devtools/python/python3/signal.patch
@@ -0,0 +1,56 @@
+Upstream-Status: Backport
+Signed-off-by: Ross Burton <ross.burton@intel.com>
+
+From 4315389df3c4e8c1f94a18ab11a4b234762132b1 Mon Sep 17 00:00:00 2001
+From: Antoine Pitrou <pitrou@free.fr>
+Date: Mon, 23 Apr 2018 22:22:49 +0200
+Subject: [PATCH] [3.6] bpo-33329: Fix multiprocessing regression on newer
+ glibcs (GH-6575) (GH-6582)
+
+Starting with glibc 2.27.9000-xxx, sigaddset() can return EINVAL for some
+reserved signal numbers between 1 and NSIG.  The `range(1, NSIG)` idiom
+is commonly used to select all signals for blocking with `pthread_sigmask`.
+So we ignore the sigaddset() return value until we expose sigfillset()
+to provide a better idiom.
+(cherry picked from commit 25038ecfb665bef641abf8cb61afff7505b0e008)
+---
+ .../next/Library/2018-04-23-13-21-39.bpo-33329.lQ-Eod.rst  |  1 +
+ Modules/signalmodule.c                                     | 14 ++++++++------
+ 2 files changed, 9 insertions(+), 6 deletions(-)
+ create mode 100644 Misc/NEWS.d/next/Library/2018-04-23-13-21-39.bpo-33329.lQ-Eod.rst
+
+diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c
+index e0d06b434d..138e74e8a9 100644
+--- a/Modules/signalmodule.c
++++ b/Modules/signalmodule.c
+@@ -744,7 +744,6 @@ iterable_to_sigset(PyObject *iterable, sigset_t *mask)
+     int result = -1;
+     PyObject *iterator, *item;
+     long signum;
+-    int err;
+ 
+     sigemptyset(mask);
+ 
+@@ -766,11 +765,14 @@ iterable_to_sigset(PyObject *iterable, sigset_t *mask)
+         Py_DECREF(item);
+         if (signum == -1 && PyErr_Occurred())
+             goto error;
+-        if (0 < signum && signum < NSIG)
+-            err = sigaddset(mask, (int)signum);
+-        else
+-            err = 1;
+-        if (err) {
++        if (0 < signum && signum < NSIG) {
++            /* bpo-33329: ignore sigaddset() return value as it can fail
++             * for some reserved signals, but we want the `range(1, NSIG)`
++             * idiom to allow selecting all valid signals.
++             */
++            (void) sigaddset(mask, (int)signum);
++        }
++        else {
+             PyErr_Format(PyExc_ValueError,
+                          "signal number %ld out of range", signum);
+             goto error;
+-- 
+2.11.0
+
diff --git a/meta/recipes-devtools/python/python3_3.5.5.bb b/meta/recipes-devtools/python/python3_3.5.5.bb
index 02decf470c8..15ec7d7fda6 100644
--- a/meta/recipes-devtools/python/python3_3.5.5.bb
+++ b/meta/recipes-devtools/python/python3_3.5.5.bb
@@ -41,6 +41,7 @@ SRC_URI += "\
             file://0002-Makefile-add-target-to-split-profile-generation.patch \
             file://float-endian.patch \
             file://ftplib.patch \
+            file://signal.patch \
            "
 SRC_URI[md5sum] = "f3763edf9824d5d3a15f5f646083b6e0"
 SRC_URI[sha256sum] = "063d2c3b0402d6191b90731e0f735c64830e7522348aeb7ed382a83165d45009"
-- 
2.11.0



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* ✗ patchtest: failure for "python3: fix ftplib with TLS 1..." and 3 more
  2018-09-10 13:31 [PATCH 1/4] python3: fix ftplib with TLS 1.3 Ross Burton
                   ` (2 preceding siblings ...)
  2018-09-10 13:31 ` [PATCH 4/4] python3: fix multiprocessing Ross Burton
@ 2018-09-10 14:02 ` Patchwork
  3 siblings, 0 replies; 9+ messages in thread
From: Patchwork @ 2018-09-10 14:02 UTC (permalink / raw)
  To: Ross Burton; +Cc: openembedded-core

== Series Details ==

Series: "python3: fix ftplib with TLS 1..." and 3 more
Revision: 1
URL   : https://patchwork.openembedded.org/series/13994/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Series does not apply on top of target branch [test_series_merge_on_head] 
  Suggested fix    Rebase your series on top of targeted branch
  Targeted branch  master (currently at 0191456d81)

* Issue             Added patch file is missing Upstream-Status in the header [test_upstream_status_presence_format] 
  Suggested fix    Add Upstream-Status: <Valid status> to the header of meta/recipes-devtools/python/python3/ftplib.patch
  Standard format  Upstream-Status: <Valid status>
  Valid status     Pending, Accepted, Backport, Denied, Inappropriate [reason], Submitted [where]



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 2/4] python3: remove redundant patch
  2018-09-10 13:31 ` [PATCH 2/4] python3: remove redundant patch Ross Burton
@ 2018-09-10 16:12   ` Khem Raj
  2018-09-10 16:16     ` Burton, Ross
  0 siblings, 1 reply; 9+ messages in thread
From: Khem Raj @ 2018-09-10 16:12 UTC (permalink / raw)
  To: Burton, Ross; +Cc: Patches and discussions about the oe-core layer

On Mon, Sep 10, 2018 at 6:31 AM Ross Burton <ross.burton@intel.com> wrote:
>
> None of the Python invocations that this changes are actually called, and
> there's no need to provide a HOSTPGEN variable when the recipe can just override
> PGEN directly.
>

patch is also substituting ./$(BUILDPYTHON) with $(HOSTPYTHON), how is
that taken care of ?
maybe try to a build where hostarch != targetarch might catch the
problem. If its handled in other
ways then its probably good riddance.

> Signed-off-by: Ross Burton <ross.burton@intel.com>
> ---
>  .../python/python3-native_3.5.5.bb                 |  1 -
>  .../python3/0001-cross-compile-support.patch       | 96 ----------------------
>  meta/recipes-devtools/python/python3_3.5.5.bb      |  3 +-
>  3 files changed, 1 insertion(+), 99 deletions(-)
>  delete mode 100644 meta/recipes-devtools/python/python3/0001-cross-compile-support.patch
>
> diff --git a/meta/recipes-devtools/python/python3-native_3.5.5.bb b/meta/recipes-devtools/python/python3-native_3.5.5.bb
> index 3098027d803..7b940a16d28 100644
> --- a/meta/recipes-devtools/python/python3-native_3.5.5.bb
> +++ b/meta/recipes-devtools/python/python3-native_3.5.5.bb
> @@ -7,7 +7,6 @@ DISTRO_SRC_URI_linuxstdbase = ""
>  SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
>  file://12-distutils-prefix-is-inside-staging-area.patch \
>  file://python-config.patch \
> -file://0001-cross-compile-support.patch \
>  file://030-fixup-include-dirs.patch \
>  file://070-dont-clean-ipkg-install.patch \
>  file://080-distutils-dont_adjust_files.patch \
> diff --git a/meta/recipes-devtools/python/python3/0001-cross-compile-support.patch b/meta/recipes-devtools/python/python3/0001-cross-compile-support.patch
> deleted file mode 100644
> index 7cd7e3b4906..00000000000
> --- a/meta/recipes-devtools/python/python3/0001-cross-compile-support.patch
> +++ /dev/null
> @@ -1,96 +0,0 @@
> -From ecde3ea170999a9ef734e8af4d7c25be5ba81697 Mon Sep 17 00:00:00 2001
> -From: Alexander Kanavin <alex.kanavin@gmail.com>
> -Date: Fri, 31 Mar 2017 15:42:46 +0300
> -Subject: [PATCH] cross-compile support
> -
> -We cross compile python. This patch uses tools from host/native
> -python instead of in-tree tools
> -
> --Khem
> -
> -Rebased on 3.5.4
> -
> -Upstream-Status: Inappropriate[Configuration Specific]
> -Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
> -Signed-off-by: Derek Straka <derek@asterius.io>
> ----
> - Makefile.pre.in | 14 ++++++++------
> - 1 file changed, 8 insertions(+), 6 deletions(-)
> -
> -diff --git a/Makefile.pre.in b/Makefile.pre.in
> -index 144c1f8629..f252ac2417 100644
> ---- a/Makefile.pre.in
> -+++ b/Makefile.pre.in
> -@@ -223,6 +223,7 @@ LIBOBJS=   @LIBOBJS@
> -
> - PYTHON=               python$(EXE)
> - BUILDPYTHON=  python$(BUILDEXE)
> -+HOSTPYTHON=    $(BUILDPYTHON)
> -
> - PYTHON_FOR_REGEN=@PYTHON_FOR_REGEN@
> - PYTHON_FOR_BUILD=@PYTHON_FOR_BUILD@
> -@@ -277,6 +278,7 @@ LIBFFI_INCLUDEDIR= @LIBFFI_INCLUDEDIR@
> - ##########################################################################
> - # Parser
> - PGEN=         Parser/pgen$(EXE)
> -+HOSTPGEN=     $(PGEN)$(EXE)
> -
> - PSRCS=                \
> -               Parser/acceler.c \
> -@@ -478,7 +480,7 @@ build_all_generate_profile:
> -
> - run_profile_task:
> -       : # FIXME: can't run for a cross build
> --      $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true
> -+      $(LLVM_PROF_FILE) $(RUNSHARED) $(HOSTPYTHON) $(PROFILE_TASK) || true
> -
> - build_all_merge_profile:
> -       $(LLVM_PROF_MERGER)
> -@@ -772,7 +774,7 @@ regen-grammar: $(PGEN)
> -       # Regenerate Include/graminit.h and Python/graminit.c
> -       # from Grammar/Grammar using pgen
> -       @$(MKDIR_P) Include
> --      $(PGEN) $(srcdir)/Grammar/Grammar \
> -+      $(HOSTPGEN) $(srcdir)/Grammar/Grammar \
> -               $(srcdir)/Include/graminit.h \
> -               $(srcdir)/Python/graminit.c
> -
> -@@ -978,7 +980,7 @@ $(LIBRARY_OBJS) $(MODOBJS) Programs/python.o: $(PYTHON_HEADERS)
> - ######################################################################
> -
> - TESTOPTS=     $(EXTRATESTOPTS)
> --TESTPYTHON=   $(RUNSHARED) ./$(BUILDPYTHON) $(TESTPYTHONOPTS)
> -+TESTPYTHON=   $(RUNSHARED) $(HOSTPYTHON) $(TESTPYTHONOPTS)
> - TESTRUNNER=   $(TESTPYTHON) $(srcdir)/Tools/scripts/run_tests.py
> - TESTTIMEOUT=  3600
> -
> -@@ -1470,7 +1472,7 @@ frameworkinstallstructure:       $(LDLIBRARY)
> -               fi; \
> -       done
> -       $(LN) -fsn include/python$(LDVERSION) $(DESTDIR)$(prefix)/Headers
> --      sed 's/%VERSION%/'"`$(RUNSHARED) ./$(BUILDPYTHON) -c 'import platform; print(platform.python_version())'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist
> -+      sed 's/%VERSION%/'"`$(RUNSHARED) $(HOSTPYTHON) -c 'import platform; print(platform.python_version())'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist
> -       $(LN) -fsn $(VERSION) $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/Versions/Current
> -       $(LN) -fsn Versions/Current/$(PYTHONFRAMEWORK) $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/$(PYTHONFRAMEWORK)
> -       $(LN) -fsn Versions/Current/Headers $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/Headers
> -@@ -1543,7 +1545,7 @@ Python/dtoa.o: Python/dtoa.c
> -
> - # Run reindent on the library
> - reindent:
> --      ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib
> -+      $(HOSTPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib
> -
> - # Rerun configure with the same options as it was run last time,
> - # provided the config.status script exists
> -@@ -1678,7 +1680,7 @@ funny:
> -
> - # Perform some verification checks on any modified files.
> - patchcheck: @DEF_MAKE_RULE@
> --      $(RUNSHARED) ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
> -+      $(RUNSHARED) $(HOSTPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
> -
> - # Dependencies
> -
> ---
> -2.11.0
> -
> diff --git a/meta/recipes-devtools/python/python3_3.5.5.bb b/meta/recipes-devtools/python/python3_3.5.5.bb
> index 93878e4e45c..6598b486afe 100644
> --- a/meta/recipes-devtools/python/python3_3.5.5.bb
> +++ b/meta/recipes-devtools/python/python3_3.5.5.bb
> @@ -11,7 +11,6 @@ DISTRO_SRC_URI ?= "file://sitecustomize.py"
>  DISTRO_SRC_URI_linuxstdbase = ""
>  SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
>  file://python-config.patch \
> -file://0001-cross-compile-support.patch \
>  file://030-fixup-include-dirs.patch \
>  file://070-dont-clean-ipkg-install.patch \
>  file://080-distutils-dont_adjust_files.patch \
> @@ -101,7 +100,7 @@ do_configure_append() {
>  }
>
>  run_make() {
> -       oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
> +       oe_runmake PGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
>                 HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python3-native/python3 \
>                 STAGING_LIBDIR=${STAGING_LIBDIR} \
>                 STAGING_INCDIR=${STAGING_INCDIR} \
> --
> 2.11.0
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 2/4] python3: remove redundant patch
  2018-09-10 16:12   ` Khem Raj
@ 2018-09-10 16:16     ` Burton, Ross
  2018-09-10 16:23       ` Burton, Ross
  0 siblings, 1 reply; 9+ messages in thread
From: Burton, Ross @ 2018-09-10 16:16 UTC (permalink / raw)
  To: Khem Raj; +Cc: Patches and discussions about the oe-core layer

On 10 September 2018 at 17:12, Khem Raj <raj.khem@gmail.com> wrote:
> patch is also substituting ./$(BUILDPYTHON) with $(HOSTPYTHON), how is
> that taken care of ?
> maybe try to a build where hostarch != targetarch might catch the
> problem. If its handled in other
> ways then its probably good riddance.

Target binaries have a different loader so they won't run.  From what
I could tell none of the codepaths this touched were used in the
build.  I'll review the changes again and either demonstrate why I'm
right, or retract my patch.

Ross


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 2/4] python3: remove redundant patch
  2018-09-10 16:16     ` Burton, Ross
@ 2018-09-10 16:23       ` Burton, Ross
  2018-09-10 17:35         ` Khem Raj
  0 siblings, 1 reply; 9+ messages in thread
From: Burton, Ross @ 2018-09-10 16:23 UTC (permalink / raw)
  To: Khem Raj; +Cc: Patches and discussions about the oe-core layer

Line-by-line breakdown of the patch:

> diff --git a/Makefile.pre.in b/Makefile.pre.in
> index 144c1f8629..f252ac2417 100644
> --- a/Makefile.pre.in
> +++ b/Makefile.pre.in
> @@ -223,6 +223,7 @@ LIBOBJS= @LIBOBJS@
>  PYTHON= python$(EXE)
>  BUILDPYTHON= python$(BUILDEXE)
> +HOSTPYTHON=    $(BUILDPYTHON)

New variable added.

>  PGEN= Parser/pgen$(EXE)
> +HOSTPGEN= $(PGEN)$(EXE)

New variable added.  My removal simple sets PGEN=/path/to/pgen-native instead.

>  run_profile_task:
>   : # FIXME: can't run for a cross build
> - $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true
> + $(LLVM_PROF_FILE) $(RUNSHARED) $(HOSTPYTHON) $(PROFILE_TASK) || true

We never use run_profile_task.

> @@ -772,7 +774,7 @@ regen-grammar: $(PGEN)
>   # Regenerate Include/graminit.h and Python/graminit.c
>   # from Grammar/Grammar using pgen
>   @$(MKDIR_P) Include
> - $(PGEN) $(srcdir)/Grammar/Grammar \
> + $(HOSTPGEN) $(srcdir)/Grammar/Grammar \

As my patch changes PGEN directly, this is redundant.

> -TESTPYTHON= $(RUNSHARED) ./$(BUILDPYTHON) $(TESTPYTHONOPTS)
> +TESTPYTHON= $(RUNSHARED) $(HOSTPYTHON) $(TESTPYTHONOPTS)

TESTPYTHON is only used by the following targets:
- coverage-report
- test
- testall
- testuniversal
- quicktest

We don't invoke any of those during a build.

>  TESTRUNNER= $(TESTPYTHON) $(srcdir)/Tools/scripts/run_tests.py
>  TESTTIMEOUT= 3600
> @@ -1470,7 +1472,7 @@ frameworkinstallstructure: $(LDLIBRARY)
>   fi; \
>   done
>   $(LN) -fsn include/python$(LDVERSION) $(DESTDIR)$(prefix)/Headers
> - sed 's/%VERSION%/'"`$(RUNSHARED) ./$(BUILDPYTHON) -c 'import platform; print(platform.python_version())'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist
> + sed 's/%VERSION%/'"`$(RUNSHARED) $(HOSTPYTHON) -c 'import platform; print(platform.python_version())'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist

This is MacOS-specific.

>  # Run reindent on the library
>  reindent:
> - ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib
> + $(HOSTPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib

We never invoke reindent (as this is for maintainers).

> @@ -1678,7 +1680,7 @@ funny:
>  # Perform some verification checks on any modified files.
>  patchcheck: @DEF_MAKE_RULE@
> - $(RUNSHARED) ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
> + $(RUNSHARED) $(HOSTPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
>  # Dependencies

We never invoke patchcheck (as this is for maintainers).

Ross


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 2/4] python3: remove redundant patch
  2018-09-10 16:23       ` Burton, Ross
@ 2018-09-10 17:35         ` Khem Raj
  0 siblings, 0 replies; 9+ messages in thread
From: Khem Raj @ 2018-09-10 17:35 UTC (permalink / raw)
  To: Burton, Ross; +Cc: Patches and discussions about the oe-core layer

Thanks for the clarification. It seems to be fine then
On Mon, Sep 10, 2018 at 9:24 AM Burton, Ross <ross.burton@intel.com> wrote:
>
> Line-by-line breakdown of the patch:
>
> > diff --git a/Makefile.pre.in b/Makefile.pre.in
> > index 144c1f8629..f252ac2417 100644
> > --- a/Makefile.pre.in
> > +++ b/Makefile.pre.in
> > @@ -223,6 +223,7 @@ LIBOBJS= @LIBOBJS@
> >  PYTHON= python$(EXE)
> >  BUILDPYTHON= python$(BUILDEXE)
> > +HOSTPYTHON=    $(BUILDPYTHON)
>
> New variable added.
>
> >  PGEN= Parser/pgen$(EXE)
> > +HOSTPGEN= $(PGEN)$(EXE)
>
> New variable added.  My removal simple sets PGEN=/path/to/pgen-native instead.
>
> >  run_profile_task:
> >   : # FIXME: can't run for a cross build
> > - $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true
> > + $(LLVM_PROF_FILE) $(RUNSHARED) $(HOSTPYTHON) $(PROFILE_TASK) || true
>
> We never use run_profile_task.
>
> > @@ -772,7 +774,7 @@ regen-grammar: $(PGEN)
> >   # Regenerate Include/graminit.h and Python/graminit.c
> >   # from Grammar/Grammar using pgen
> >   @$(MKDIR_P) Include
> > - $(PGEN) $(srcdir)/Grammar/Grammar \
> > + $(HOSTPGEN) $(srcdir)/Grammar/Grammar \
>
> As my patch changes PGEN directly, this is redundant.
>
> > -TESTPYTHON= $(RUNSHARED) ./$(BUILDPYTHON) $(TESTPYTHONOPTS)
> > +TESTPYTHON= $(RUNSHARED) $(HOSTPYTHON) $(TESTPYTHONOPTS)
>
> TESTPYTHON is only used by the following targets:
> - coverage-report
> - test
> - testall
> - testuniversal
> - quicktest
>
> We don't invoke any of those during a build.
>
> >  TESTRUNNER= $(TESTPYTHON) $(srcdir)/Tools/scripts/run_tests.py
> >  TESTTIMEOUT= 3600
> > @@ -1470,7 +1472,7 @@ frameworkinstallstructure: $(LDLIBRARY)
> >   fi; \
> >   done
> >   $(LN) -fsn include/python$(LDVERSION) $(DESTDIR)$(prefix)/Headers
> > - sed 's/%VERSION%/'"`$(RUNSHARED) ./$(BUILDPYTHON) -c 'import platform; print(platform.python_version())'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist
> > + sed 's/%VERSION%/'"`$(RUNSHARED) $(HOSTPYTHON) -c 'import platform; print(platform.python_version())'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist
>
> This is MacOS-specific.
>
> >  # Run reindent on the library
> >  reindent:
> > - ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib
> > + $(HOSTPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib
>
> We never invoke reindent (as this is for maintainers).
>
> > @@ -1678,7 +1680,7 @@ funny:
> >  # Perform some verification checks on any modified files.
> >  patchcheck: @DEF_MAKE_RULE@
> > - $(RUNSHARED) ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
> > + $(RUNSHARED) $(HOSTPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
> >  # Dependencies
>
> We never invoke patchcheck (as this is for maintainers).
>
> Ross


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2018-09-10 17:36 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-09-10 13:31 [PATCH 1/4] python3: fix ftplib with TLS 1.3 Ross Burton
2018-09-10 13:31 ` [PATCH 2/4] python3: remove redundant patch Ross Burton
2018-09-10 16:12   ` Khem Raj
2018-09-10 16:16     ` Burton, Ross
2018-09-10 16:23       ` Burton, Ross
2018-09-10 17:35         ` Khem Raj
2018-09-10 13:31 ` [PATCH 3/4] python3: remove patch that inadvertently disables xattrs Ross Burton
2018-09-10 13:31 ` [PATCH 4/4] python3: fix multiprocessing Ross Burton
2018-09-10 14:02 ` ✗ patchtest: failure for "python3: fix ftplib with TLS 1..." and 3 more Patchwork

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.