[PATCH] python3-dtc: fix packaging by using setuptools

[PATCH] python3-dtc: fix packaging by using setuptools

[Ross Burton]

Inheriting setuptools3-base doesn't actually define any tasks, so commit
1ef6050 to drop distutils3 had the side-effect of not building any code.

Backport a commit from upstream to switch to setuptools, inherit
setuptool3, and change DISTUTILS_SETUP_PATH to SETUPTOOLS_SETUP_PATH.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 recipes-kernel/dtc/files/setuptools.patch | 34 +++++++++++++++++++++++
 recipes-kernel/dtc/python3-dtc_1.6.1.bb   |  7 +++--
 2 files changed, 38 insertions(+), 3 deletions(-)
 create mode 100644 recipes-kernel/dtc/files/setuptools.patch

diff --git a/recipes-kernel/dtc/files/setuptools.patch b/recipes-kernel/dtc/files/setuptools.patch
new file mode 100644
index 0000000..3d37ebd
--- /dev/null
+++ b/recipes-kernel/dtc/files/setuptools.patch
@@ -0,0 +1,34 @@
+Upstream-Status: Backport
+Signed-off-by: Ross Burton <ross.burton@arm.com>
+
+From c691776ddb26acbd3674722caafacaf7b6e3e807 Mon Sep 17 00:00:00 2001
+From: Rob Herring <robh@kernel.org>
+Date: Wed, 10 Nov 2021 19:11:32 -0600
+Subject: [PATCH] pylibfdt: Use setuptools instead of distutils
+
+The use of setuptools is favored over distutils. setuptools is needed to
+support building Python 'wheels' and for pip support.
+
+Signed-off-by: Rob Herring <robh@kernel.org>
+Message-Id: <20211111011135.2386773-2-robh@kernel.org>
+Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
+---
+ pylibfdt/setup.py | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/pylibfdt/setup.py b/pylibfdt/setup.py
+index ef40f15..f065a59 100755
+--- a/pylibfdt/setup.py
++++ b/pylibfdt/setup.py
+@@ -10,7 +10,7 @@ Copyright (C) 2017 Google, Inc.
+ Written by Simon Glass <sjg@chromium.org>
+ """
+ 
+-from distutils.core import setup, Extension
++from setuptools import setup, Extension
+ import os
+ import re
+ import sys
+-- 
+2.25.1
+
diff --git a/recipes-kernel/dtc/python3-dtc_1.6.1.bb b/recipes-kernel/dtc/python3-dtc_1.6.1.bb
index e9b8499..6e02db6 100644
--- a/recipes-kernel/dtc/python3-dtc_1.6.1.bb
+++ b/recipes-kernel/dtc/python3-dtc_1.6.1.bb
@@ -7,6 +7,7 @@ LICENSE = "GPLv2 | BSD-2-Clause"
 DEPENDS = "flex-native bison-native swig-native libyaml dtc"
 
 SRC_URI = "git://git.kernel.org/pub/scm/utils/dtc/dtc.git;branch=master \
+           file://setuptools.patch \
            file://ssize.patch"
 
 UPSTREAM_CHECK_GITTAGREGEX = "v(?P<pver>\d+(\.\d+)+)"
@@ -17,13 +18,13 @@ SRCREV = "ecaeb97fec013973360e94888a7de645f084345c"
 
 S = "${WORKDIR}/git"
 
-inherit setuptools3-base pkgconfig
+inherit setuptools3 pkgconfig
 
-DISTUTILS_SETUP_PATH = "${S}/pylibfdt"
+SETUPTOOLS_SETUP_PATH = "${S}/pylibfdt"
 
 do_configure:prepend() {
     oe_runmake -C "${S}" version_gen.h
-    mv "${S}/version_gen.h" "${DISTUTILS_SETUP_PATH}/"
+    mv "${S}/version_gen.h" "${SETUPTOOLS_SETUP_PATH}/"
 }
 
 BBCLASSEXTEND = "native nativesdk"
-- 
2.25.1

Re: [meta-virtualization] [PATCH] python3-dtc: fix packaging by using setuptools

[Bruce Ashfield]

On Mon, Jan 17, 2022 at 7:58 AM Ross Burton <ross@burtonini.com> wrote:
>
> Inheriting setuptools3-base doesn't actually define any tasks, so commit
> 1ef6050 to drop distutils3 had the side-effect of not building any code.
>
> Backport a commit from upstream to switch to setuptools, inherit
> setuptool3, and change DISTUTILS_SETUP_PATH to SETUPTOOLS_SETUP_PATH.
>

I have a SRCREV bump ready to go for dtc, so I've applied this and
then the bump on top.

Bruce

> Signed-off-by: Ross Burton <ross.burton@arm.com>
> ---
>  recipes-kernel/dtc/files/setuptools.patch | 34 +++++++++++++++++++++++
>  recipes-kernel/dtc/python3-dtc_1.6.1.bb   |  7 +++--
>  2 files changed, 38 insertions(+), 3 deletions(-)
>  create mode 100644 recipes-kernel/dtc/files/setuptools.patch
>
> diff --git a/recipes-kernel/dtc/files/setuptools.patch b/recipes-kernel/dtc/files/setuptools.patch
> new file mode 100644
> index 0000000..3d37ebd
> --- /dev/null
> +++ b/recipes-kernel/dtc/files/setuptools.patch
> @@ -0,0 +1,34 @@
> +Upstream-Status: Backport
> +Signed-off-by: Ross Burton <ross.burton@arm.com>
> +
> +From c691776ddb26acbd3674722caafacaf7b6e3e807 Mon Sep 17 00:00:00 2001
> +From: Rob Herring <robh@kernel.org>
> +Date: Wed, 10 Nov 2021 19:11:32 -0600
> +Subject: [PATCH] pylibfdt: Use setuptools instead of distutils
> +
> +The use of setuptools is favored over distutils. setuptools is needed to
> +support building Python 'wheels' and for pip support.
> +
> +Signed-off-by: Rob Herring <robh@kernel.org>
> +Message-Id: <20211111011135.2386773-2-robh@kernel.org>
> +Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> +---
> + pylibfdt/setup.py | 2 +-
> + 1 file changed, 1 insertion(+), 1 deletion(-)
> +
> +diff --git a/pylibfdt/setup.py b/pylibfdt/setup.py
> +index ef40f15..f065a59 100755
> +--- a/pylibfdt/setup.py
> ++++ b/pylibfdt/setup.py
> +@@ -10,7 +10,7 @@ Copyright (C) 2017 Google, Inc.
> + Written by Simon Glass <sjg@chromium.org>
> + """
> +
> +-from distutils.core import setup, Extension
> ++from setuptools import setup, Extension
> + import os
> + import re
> + import sys
> +--
> +2.25.1
> +
> diff --git a/recipes-kernel/dtc/python3-dtc_1.6.1.bb b/recipes-kernel/dtc/python3-dtc_1.6.1.bb
> index e9b8499..6e02db6 100644
> --- a/recipes-kernel/dtc/python3-dtc_1.6.1.bb
> +++ b/recipes-kernel/dtc/python3-dtc_1.6.1.bb
> @@ -7,6 +7,7 @@ LICENSE = "GPLv2 | BSD-2-Clause"
>  DEPENDS = "flex-native bison-native swig-native libyaml dtc"
>
>  SRC_URI = "git://git.kernel.org/pub/scm/utils/dtc/dtc.git;branch=master \
> +           file://setuptools.patch \
>             file://ssize.patch"
>
>  UPSTREAM_CHECK_GITTAGREGEX = "v(?P<pver>\d+(\.\d+)+)"
> @@ -17,13 +18,13 @@ SRCREV = "ecaeb97fec013973360e94888a7de645f084345c"
>
>  S = "${WORKDIR}/git"
>
> -inherit setuptools3-base pkgconfig
> +inherit setuptools3 pkgconfig
>
> -DISTUTILS_SETUP_PATH = "${S}/pylibfdt"
> +SETUPTOOLS_SETUP_PATH = "${S}/pylibfdt"
>
>  do_configure:prepend() {
>      oe_runmake -C "${S}" version_gen.h
> -    mv "${S}/version_gen.h" "${DISTUTILS_SETUP_PATH}/"
> +    mv "${S}/version_gen.h" "${SETUPTOOLS_SETUP_PATH}/"
>  }
>
>  BBCLASSEXTEND = "native nativesdk"
> --
> 2.25.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#7007): https://lists.yoctoproject.org/g/meta-virtualization/message/7007
> Mute This Topic: https://lists.yoctoproject.org/mt/88482921/1050810
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] [PATCH] python3-dtc: fix packaging by using setuptools

[Bruce Ashfield]

On Mon, Jan 17, 2022 at 8:35 AM Bruce Ashfield via
lists.yoctoproject.org
<bruce.ashfield=gmail.com@lists.yoctoproject.org> wrote:
>
> On Mon, Jan 17, 2022 at 7:58 AM Ross Burton <ross@burtonini.com> wrote:
> >
> > Inheriting setuptools3-base doesn't actually define any tasks, so commit
> > 1ef6050 to drop distutils3 had the side-effect of not building any code.
> >
> > Backport a commit from upstream to switch to setuptools, inherit
> > setuptool3, and change DISTUTILS_SETUP_PATH to SETUPTOOLS_SETUP_PATH.
> >
>
> I have a SRCREV bump ready to go for dtc, so I've applied this and
> then the bump on top.

Your change + my update are now queued:

https://git.yoctoproject.org/meta-virtualization/commit/?h=master-next&id=ed4bd1de0d2e040c303a8497ac0cd961aa8b8a7b

If you have any cycles for test build, that would be great .. since
somehow my own build didn't notice the empty package, so I can't say I
fully trust it.

Bruce

>
> Bruce
>
> > Signed-off-by: Ross Burton <ross.burton@arm.com>
> > ---
> >  recipes-kernel/dtc/files/setuptools.patch | 34 +++++++++++++++++++++++
> >  recipes-kernel/dtc/python3-dtc_1.6.1.bb   |  7 +++--
> >  2 files changed, 38 insertions(+), 3 deletions(-)
> >  create mode 100644 recipes-kernel/dtc/files/setuptools.patch
> >
> > diff --git a/recipes-kernel/dtc/files/setuptools.patch b/recipes-kernel/dtc/files/setuptools.patch
> > new file mode 100644
> > index 0000000..3d37ebd
> > --- /dev/null
> > +++ b/recipes-kernel/dtc/files/setuptools.patch
> > @@ -0,0 +1,34 @@
> > +Upstream-Status: Backport
> > +Signed-off-by: Ross Burton <ross.burton@arm.com>
> > +
> > +From c691776ddb26acbd3674722caafacaf7b6e3e807 Mon Sep 17 00:00:00 2001
> > +From: Rob Herring <robh@kernel.org>
> > +Date: Wed, 10 Nov 2021 19:11:32 -0600
> > +Subject: [PATCH] pylibfdt: Use setuptools instead of distutils
> > +
> > +The use of setuptools is favored over distutils. setuptools is needed to
> > +support building Python 'wheels' and for pip support.
> > +
> > +Signed-off-by: Rob Herring <robh@kernel.org>
> > +Message-Id: <20211111011135.2386773-2-robh@kernel.org>
> > +Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > +---
> > + pylibfdt/setup.py | 2 +-
> > + 1 file changed, 1 insertion(+), 1 deletion(-)
> > +
> > +diff --git a/pylibfdt/setup.py b/pylibfdt/setup.py
> > +index ef40f15..f065a59 100755
> > +--- a/pylibfdt/setup.py
> > ++++ b/pylibfdt/setup.py
> > +@@ -10,7 +10,7 @@ Copyright (C) 2017 Google, Inc.
> > + Written by Simon Glass <sjg@chromium.org>
> > + """
> > +
> > +-from distutils.core import setup, Extension
> > ++from setuptools import setup, Extension
> > + import os
> > + import re
> > + import sys
> > +--
> > +2.25.1
> > +
> > diff --git a/recipes-kernel/dtc/python3-dtc_1.6.1.bb b/recipes-kernel/dtc/python3-dtc_1.6.1.bb
> > index e9b8499..6e02db6 100644
> > --- a/recipes-kernel/dtc/python3-dtc_1.6.1.bb
> > +++ b/recipes-kernel/dtc/python3-dtc_1.6.1.bb
> > @@ -7,6 +7,7 @@ LICENSE = "GPLv2 | BSD-2-Clause"
> >  DEPENDS = "flex-native bison-native swig-native libyaml dtc"
> >
> >  SRC_URI = "git://git.kernel.org/pub/scm/utils/dtc/dtc.git;branch=master \
> > +           file://setuptools.patch \
> >             file://ssize.patch"
> >
> >  UPSTREAM_CHECK_GITTAGREGEX = "v(?P<pver>\d+(\.\d+)+)"
> > @@ -17,13 +18,13 @@ SRCREV = "ecaeb97fec013973360e94888a7de645f084345c"
> >
> >  S = "${WORKDIR}/git"
> >
> > -inherit setuptools3-base pkgconfig
> > +inherit setuptools3 pkgconfig
> >
> > -DISTUTILS_SETUP_PATH = "${S}/pylibfdt"
> > +SETUPTOOLS_SETUP_PATH = "${S}/pylibfdt"
> >
> >  do_configure:prepend() {
> >      oe_runmake -C "${S}" version_gen.h
> > -    mv "${S}/version_gen.h" "${DISTUTILS_SETUP_PATH}/"
> > +    mv "${S}/version_gen.h" "${SETUPTOOLS_SETUP_PATH}/"
> >  }
> >
> >  BBCLASSEXTEND = "native nativesdk"
> > --
> > 2.25.1
> >
> >
> >
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#7008): https://lists.yoctoproject.org/g/meta-virtualization/message/7008
> Mute This Topic: https://lists.yoctoproject.org/mt/88482921/1050810
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] [PATCH] python3-dtc: fix packaging by using setuptools

[Ross Burton]

On Mon, 17 Jan 2022 at 18:07, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> Your change + my update are now queued:
>
> https://git.yoctoproject.org/meta-virtualization/commit/?h=master-next&id=ed4bd1de0d2e040c303a8497ac0cd961aa8b8a7b
>
> If you have any cycles for test build, that would be great .. since
> somehow my own build didn't notice the empty package, so I can't say I
> fully trust it.

SETUPTOOLS_SETUP_PATH defaults to S, so you can remove the assignment entirely.

Ross

Re: [meta-virtualization] [PATCH] python3-dtc: fix packaging by using setuptools

[Bruce Ashfield]

On Tue, Jan 18, 2022 at 6:18 AM Ross Burton <ross@burtonini.com> wrote:
>
> On Mon, 17 Jan 2022 at 18:07, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > Your change + my update are now queued:
> >
> > https://git.yoctoproject.org/meta-virtualization/commit/?h=master-next&id=ed4bd1de0d2e040c303a8497ac0cd961aa8b8a7b
> >
> > If you have any cycles for test build, that would be great .. since
> > somehow my own build didn't notice the empty package, so I can't say I
> > fully trust it.
>
> SETUPTOOLS_SETUP_PATH defaults to S, so you can remove the assignment entirely.
>

done. And pushed to master.

Bruce

> Ross



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] [PATCH] python3-dtc: fix packaging by using setuptools

[Peter Hoyes]

On 18/01/2022 13:52, Bruce Ashfield via lists.yoctoproject.org wrote:
> On Tue, Jan 18, 2022 at 6:18 AM Ross Burton <ross@burtonini.com> wrote:
>> On Mon, 17 Jan 2022 at 18:07, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>>> Your change + my update are now queued:
>>>
>>> https://git.yoctoproject.org/meta-virtualization/commit/?h=master-next&id=ed4bd1de0d2e040c303a8497ac0cd961aa8b8a7b
>>>
>>> If you have any cycles for test build, that would be great .. since
>>> somehow my own build didn't notice the empty package, so I can't say I
>>> fully trust it.
>> SETUPTOOLS_SETUP_PATH defaults to S, so you can remove the assignment entirely.
>>
> done. And pushed to master.
>
> Bruce
>
>> Ross
>
Hi Bruce,

I am now seeing the following failure on our build when attempting to 
use python3-dtc:

usr/lib/python3.10/site-packages/_libfdt.cpython-310-x86_64-linux-gnu.so: 
undefined symbol: fdt_overlay_target_offset

I think this is because the updated SRCREV includes 45f3d1a which 
requires libfdt to export the symbol above, but poky/dtc is still on an 
older revision. Maybe its necessary to roll back the SRCREV patch until 
poky updates?

Cheers,

Peter

Re: [meta-virtualization] [PATCH] python3-dtc: fix packaging by using setuptools

[Bruce Ashfield]

On Wed, Jan 19, 2022 at 7:52 AM Peter Hoyes <Peter.Hoyes@arm.com> wrote:
>
> On 18/01/2022 13:52, Bruce Ashfield via lists.yoctoproject.org wrote:
> > On Tue, Jan 18, 2022 at 6:18 AM Ross Burton <ross@burtonini.com> wrote:
> >> On Mon, 17 Jan 2022 at 18:07, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >>> Your change + my update are now queued:
> >>>
> >>> https://git.yoctoproject.org/meta-virtualization/commit/?h=master-next&id=ed4bd1de0d2e040c303a8497ac0cd961aa8b8a7b
> >>>
> >>> If you have any cycles for test build, that would be great .. since
> >>> somehow my own build didn't notice the empty package, so I can't say I
> >>> fully trust it.
> >> SETUPTOOLS_SETUP_PATH defaults to S, so you can remove the assignment entirely.
> >>
> > done. And pushed to master.
> >
> > Bruce
> >
> >> Ross
> >
> Hi Bruce,
>
> I am now seeing the following failure on our build when attempting to
> use python3-dtc:
>
> usr/lib/python3.10/site-packages/_libfdt.cpython-310-x86_64-linux-gnu.so:
> undefined symbol: fdt_overlay_target_offset
>
> I think this is because the updated SRCREV includes 45f3d1a which
> requires libfdt to export the symbol above, but poky/dtc is still on an
> older revision. Maybe its necessary to roll back the SRCREV patch until
> poky updates?

Hmm. That's a last resort, thrashing around the SRCREVs is never a
good idea.

I'll temporarily carry a newer dtc in meta-virt to get things working.

Bruce

>
> Cheers,
>
> Peter
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] [PATCH] python3-dtc: fix packaging by using setuptools

[Bruce Ashfield]

On Wed, Jan 19, 2022 at 8:32 AM Bruce Ashfield via
lists.yoctoproject.org
<bruce.ashfield=gmail.com@lists.yoctoproject.org> wrote:
>
> On Wed, Jan 19, 2022 at 7:52 AM Peter Hoyes <Peter.Hoyes@arm.com> wrote:
> >
> > On 18/01/2022 13:52, Bruce Ashfield via lists.yoctoproject.org wrote:
> > > On Tue, Jan 18, 2022 at 6:18 AM Ross Burton <ross@burtonini.com> wrote:
> > >> On Mon, 17 Jan 2022 at 18:07, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > >>> Your change + my update are now queued:
> > >>>
> > >>> https://git.yoctoproject.org/meta-virtualization/commit/?h=master-next&id=ed4bd1de0d2e040c303a8497ac0cd961aa8b8a7b
> > >>>
> > >>> If you have any cycles for test build, that would be great .. since
> > >>> somehow my own build didn't notice the empty package, so I can't say I
> > >>> fully trust it.
> > >> SETUPTOOLS_SETUP_PATH defaults to S, so you can remove the assignment entirely.
> > >>
> > > done. And pushed to master.
> > >
> > > Bruce
> > >
> > >> Ross
> > >
> > Hi Bruce,
> >
> > I am now seeing the following failure on our build when attempting to
> > use python3-dtc:
> >
> > usr/lib/python3.10/site-packages/_libfdt.cpython-310-x86_64-linux-gnu.so:
> > undefined symbol: fdt_overlay_target_offset
> >
> > I think this is because the updated SRCREV includes 45f3d1a which
> > requires libfdt to export the symbol above, but poky/dtc is still on an
> > older revision. Maybe its necessary to roll back the SRCREV patch until
> > poky updates?
>
> Hmm. That's a last resort, thrashing around the SRCREVs is never a
> good idea.
>
> I'll temporarily carry a newer dtc in meta-virt to get things working.

I split the difference.

The PV was wrong on the updated package anyway, so I've pushed a
change that uses SRCPV to show that I'm running ahead of oe-core DTC,
and I've reverted the one commit that introduces the problematic
symbol.

This gets me what I need for lopper, and I can now build and construct
xen-image-mininal for qemuarm64

Bruce


>
> Bruce
>
> >
> > Cheers,
> >
> > Peter
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#7026): https://lists.yoctoproject.org/g/meta-virtualization/message/7026
> Mute This Topic: https://lists.yoctoproject.org/mt/88482921/1050810
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II