From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.7 required=3.0 tests=BAYES_00,DATE_IN_PAST_12_24, FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 436ADC4338F for ; Thu, 5 Aug 2021 19:54:34 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D1C1760F02 for ; Thu, 5 Aug 2021 19:54:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D1C1760F02 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=free.fr Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=busybox.net Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 9919182871; Thu, 5 Aug 2021 19:54:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zsLMhtdL8_lg; Thu, 5 Aug 2021 19:54:32 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id E93E9827A9; Thu, 5 Aug 2021 19:54:31 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 74CFB1BF361 for ; Thu, 5 Aug 2021 19:49:57 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id E47F583BA8 for ; Thu, 5 Aug 2021 19:49:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SGMtAkSmk1T5 for ; Thu, 5 Aug 2021 19:49:51 +0000 (UTC) Received: from busybox.osuosl.org (busybox.osuosl.org [140.211.167.122]) by smtp1.osuosl.org (Postfix) with ESMTP id 684CC83B63 for ; Thu, 5 Aug 2021 19:49:51 +0000 (UTC) Received: by busybox.osuosl.org (Postfix, from userid 4052) id 32A4687DA7; Thu, 5 Aug 2021 19:49:39 +0000 (UTC) From: Yann E. MORIN To: buildroot@buildroot.org Date: Wed, 4 Aug 2021 22:04:37 +0200 X-Git-Refname: refs/heads/next X-Git-Oldrev: f23129ee1e3ceb2cdbe751efed9faedbfae67d56 X-Git-Newrev: e17946b409c84afd15c7de47df2c8ad4dd648fae X-Patchwork-Hint: ignore Message-Id: <20210805194939.32A4687DA7@busybox.osuosl.org> Subject: [Buildroot] [git commit branch/next] package/python3: add patch to fix highly parallel builds X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" commit: https://git.buildroot.net/buildroot/commit/?id=e17946b409c84afd15c7de47df2c8ad4dd648fae branch: https://git.buildroot.net/buildroot/commit/?id=refs/heads/next Highly parallel host-python3 builds sometimes fail with: Exception in thread Thread-1: Traceback (most recent call last): File "/tmp/instance-3/output-1/host/lib/python3.9/threading.py", line 973, in _bootstrap_inner self.run() File "/tmp/instance-3/output-1/host/lib/python3.9/concurrent/futures/process.py", line 317, in run result_item, is_broken, cause = self.wait_result_broken_or_wakeup() File "/tmp/instance-3/output-1/host/lib/python3.9/concurrent/futures/process.py", line 376, in wait_result_broken_or_wakeup worker_sentinels = [p.sentinel for p in self.processes.values()] File "/tmp/instance-3/output-1/host/lib/python3.9/concurrent/futures/process.py", line 376, in worker_sentinels = [p.sentinel for p in self.processes.values()] RuntimeError: dictionary changed size during iteration During the compile_all.py step of host-python3. This issue is reported upstream at https://bugs.python.org/issue43498, and while not yet fixed upstream, a PR was proposed with a possible fix for it. Seems the PR seems reasonable, let's give it a chance and see if it improves the situation. Hopefully Fixes: http://autobuild.buildroot.net/results/ae6c4ab292589a4e4442dfb0a1286349a9bf4d29/ Signed-off-by: Thomas Petazzoni Signed-off-by: Yann E. MORIN --- ...ary-iteration-error-in-_ExecutorManagerTh.patch | 29 ++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/package/python3/0036-Fix-dictionary-iteration-error-in-_ExecutorManagerTh.patch b/package/python3/0036-Fix-dictionary-iteration-error-in-_ExecutorManagerTh.patch new file mode 100644 index 0000000000..fbe44c7753 --- /dev/null +++ b/package/python3/0036-Fix-dictionary-iteration-error-in-_ExecutorManagerTh.patch @@ -0,0 +1,29 @@ +From 331966be70c371b268a4fcce9e97280cd869f137 Mon Sep 17 00:00:00 2001 +From: Jakub Kulik +Date: Mon, 15 Mar 2021 08:49:28 +0100 +Subject: [PATCH] Fix dictionary iteration error in _ExecutorManagerThread + +[Thomas: Taken from upstream pull request +https://github.com/python/cpython/pull/24868, which is aimed at fixing +https://bugs.python.org/issue43498] +Signed-off-by: Thomas Petazzoni +--- + Lib/concurrent/futures/process.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/Lib/concurrent/futures/process.py b/Lib/concurrent/futures/process.py +index 90bc98bf2ec..e3b36dff572 100644 +--- a/Lib/concurrent/futures/process.py ++++ b/Lib/concurrent/futures/process.py +@@ -373,7 +373,7 @@ class _ExecutorManagerThread(threading.Thread): + assert not self.thread_wakeup._closed + wakeup_reader = self.thread_wakeup._reader + readers = [result_reader, wakeup_reader] +- worker_sentinels = [p.sentinel for p in self.processes.values()] ++ worker_sentinels = [p.sentinel for p in self.processes.copy().values()] + ready = mp.connection.wait(readers + worker_sentinels) + + cause = None +-- +2.31.1 + _______________________________________________ buildroot mailing list buildroot@busybox.net http://lists.busybox.net/mailman/listinfo/buildroot