* [PATCH v3] pip_install_wheel: improve wheel handling
@ 2022-03-01 9:42 Konrad Weihmann
2022-03-01 15:22 ` [OE-core] " Tim Orling
0 siblings, 1 reply; 11+ messages in thread
From: Konrad Weihmann @ 2022-03-01 9:42 UTC (permalink / raw)
To: openembedded-core; +Cc: Konrad Weihmann
- replace python3 prefix when guessing the wheel name
as there are still plenty of recipes out there that do use
python3 prefixes
- remove all previously generated wheels matching the glob
to avoid installing any outdated blob
- don't use PV in glob, as PV doesn't necessarily align with the
version used inside of the setuptools configuration.
this will avoid having the user set PYPA_WHEEL in a lot
of recipes
- respect SETUPTOOLS_SETUP_PATH in PIP_INSTALL_DIST_PATH
and use B as a fallback only (in case this class is inherited
without setuptools3 class being there as well).
recipes like python3-smbus run in a subfolder of the
workspace and were failing in before this adjustment
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
---
v3: remove in prepend to compile, not in install for obvious reasons
v2: fix python3 prefix string
meta/classes/pip_install_wheel.bbclass | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/meta/classes/pip_install_wheel.bbclass b/meta/classes/pip_install_wheel.bbclass
index 5b7e5cd706..789c25b68c 100644
--- a/meta/classes/pip_install_wheel.bbclass
+++ b/meta/classes/pip_install_wheel.bbclass
@@ -1,12 +1,13 @@
DEPENDS:append = " python3-pip-native"
def guess_pip_install_package_name(d):
+ import re
'''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
- return (d.getVar('PYPI_PACKAGE') or d.getVar('BPN')).replace('-', '_')
+ return (d.getVar('PYPI_PACKAGE') or re.sub(r"^python3-", "", d.getVar('BPN')).replace('-', '_'))
PIP_INSTALL_PACKAGE ?= "${@guess_pip_install_package_name(d)}"
-PIP_INSTALL_DIST_PATH ?= "${B}/dist"
-PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-${PV}-*.whl"
+PIP_INSTALL_DIST_PATH ?= "${@d.getVar('SETUPTOOLS_SETUP_PATH') or d.getVar('B')}/dist"
+PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-*-*.whl"
PIP_INSTALL_ARGS ?= "\
-vvvv \
@@ -45,4 +46,9 @@ pip_install_wheel_do_install () {
done
}
+do_compile:prepend() {
+ # cleanup previously generated files
+ rm -f ${PYPA_WHEEL}
+}
+
EXPORT_FUNCTIONS do_install
--
2.25.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [OE-core] [PATCH v3] pip_install_wheel: improve wheel handling
2022-03-01 9:42 [PATCH v3] pip_install_wheel: improve wheel handling Konrad Weihmann
@ 2022-03-01 15:22 ` Tim Orling
2022-03-01 18:48 ` Khem Raj
0 siblings, 1 reply; 11+ messages in thread
From: Tim Orling @ 2022-03-01 15:22 UTC (permalink / raw)
To: Konrad Weihmann; +Cc: openembedded-core
[-- Attachment #1: Type: text/plain, Size: 3072 bytes --]
On Tue, Mar 1, 2022 at 1:42 AM Konrad Weihmann <kweihmann@outlook.com>
wrote:
> - replace python3 prefix when guessing the wheel name
> as there are still plenty of recipes out there that do use
> python3 prefixes
> - remove all previously generated wheels matching the glob
> to avoid installing any outdated blob
> - don't use PV in glob, as PV doesn't necessarily align with the
> version used inside of the setuptools configuration.
> this will avoid having the user set PYPA_WHEEL in a lot
> of recipes
> - respect SETUPTOOLS_SETUP_PATH in PIP_INSTALL_DIST_PATH
> and use B as a fallback only (in case this class is inherited
> without setuptools3 class being there as well).
> recipes like python3-smbus run in a subfolder of the
> workspace and were failing in before this adjustment
>
> Thank you. These are patterns that were never seen during development, but
are painfully obvious now.
The wheel spec (PEP-427) allows for, for instance, dev/pre-release wheels
which we see with the
python3-dtc recipe from meta-virtualization.
> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Reviewed-by: Tim Orling <tim.orling@konsulko.com>
>
---
> v3: remove in prepend to compile, not in install for obvious reasons
> v2: fix python3 prefix string
>
> meta/classes/pip_install_wheel.bbclass | 12 +++++++++---
> 1 file changed, 9 insertions(+), 3 deletions(-)
>
> diff --git a/meta/classes/pip_install_wheel.bbclass
> b/meta/classes/pip_install_wheel.bbclass
> index 5b7e5cd706..789c25b68c 100644
> --- a/meta/classes/pip_install_wheel.bbclass
> +++ b/meta/classes/pip_install_wheel.bbclass
> @@ -1,12 +1,13 @@
> DEPENDS:append = " python3-pip-native"
>
> def guess_pip_install_package_name(d):
> + import re
> '''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
> - return (d.getVar('PYPI_PACKAGE') or d.getVar('BPN')).replace('-', '_')
> + return (d.getVar('PYPI_PACKAGE') or re.sub(r"^python3-", "",
> d.getVar('BPN')).replace('-', '_'))
>
> PIP_INSTALL_PACKAGE ?= "${@guess_pip_install_package_name(d)}"
> -PIP_INSTALL_DIST_PATH ?= "${B}/dist"
> -PYPA_WHEEL ??=
> "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-${PV}-*.whl"
> +PIP_INSTALL_DIST_PATH ?= "${@d.getVar('SETUPTOOLS_SETUP_PATH') or
> d.getVar('B')}/dist"
> +PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-*-*.whl"
>
> PIP_INSTALL_ARGS ?= "\
> -vvvv \
> @@ -45,4 +46,9 @@ pip_install_wheel_do_install () {
> done
> }
>
> +do_compile:prepend() {
> + # cleanup previously generated files
> + rm -f ${PYPA_WHEEL}
> +}
> +
> EXPORT_FUNCTIONS do_install
> --
> 2.25.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#162541):
> https://lists.openembedded.org/g/openembedded-core/message/162541
> Mute This Topic: https://lists.openembedded.org/mt/89472354/924729
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> ticotimo@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
[-- Attachment #2: Type: text/html, Size: 4943 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [OE-core] [PATCH v3] pip_install_wheel: improve wheel handling
2022-03-01 15:22 ` [OE-core] " Tim Orling
@ 2022-03-01 18:48 ` Khem Raj
2022-03-01 20:14 ` Ross Burton
0 siblings, 1 reply; 11+ messages in thread
From: Khem Raj @ 2022-03-01 18:48 UTC (permalink / raw)
To: Tim Orling; +Cc: Konrad Weihmann, openembedded-core
I am seeing this
| Created temporary directory: /tmp/pip-install-33iiigxv
| WARNING: Requirement
'/mnt/b/yoe/master/build/tmp/work/x86_64-linux/python3-pytest-runner-native/5.3.1-r0/pytest-runner-5.3.1/dist/pytest-runner-*-*.whl'
looks like a filename, but the file does not exist
| ERROR: pytest-runner-*-*.whl is not a valid wheel filename.
| Exception information:
| Traceback (most recent call last):
| File "/mnt/b/yoe/master/build/tmp/work/x86_64-linux/python3-pytest-runner-native/5.3.1-r0/recipe-sysroot-native/usr/lib/python3.10/site-packages/pip/_internal/cli/base_command.py",
line 167, in exc_logging_wrapper
where the wheel file is called
pytest_runner-5.3.1-py3-none-any.whl
On Tue, Mar 1, 2022 at 7:22 AM Tim Orling <ticotimo@gmail.com> wrote:
>
>
>
> On Tue, Mar 1, 2022 at 1:42 AM Konrad Weihmann <kweihmann@outlook.com> wrote:
>>
>> - replace python3 prefix when guessing the wheel name
>> as there are still plenty of recipes out there that do use
>> python3 prefixes
>> - remove all previously generated wheels matching the glob
>> to avoid installing any outdated blob
>> - don't use PV in glob, as PV doesn't necessarily align with the
>> version used inside of the setuptools configuration.
>> this will avoid having the user set PYPA_WHEEL in a lot
>> of recipes
>> - respect SETUPTOOLS_SETUP_PATH in PIP_INSTALL_DIST_PATH
>> and use B as a fallback only (in case this class is inherited
>> without setuptools3 class being there as well).
>> recipes like python3-smbus run in a subfolder of the
>> workspace and were failing in before this adjustment
>>
> Thank you. These are patterns that were never seen during development, but are painfully obvious now.
> The wheel spec (PEP-427) allows for, for instance, dev/pre-release wheels which we see with the
> python3-dtc recipe from meta-virtualization.
>
>>
>> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
>
> Reviewed-by: Tim Orling <tim.orling@konsulko.com>
>>
>>
>>
>> ---
>> v3: remove in prepend to compile, not in install for obvious reasons
>> v2: fix python3 prefix string
>>
>> meta/classes/pip_install_wheel.bbclass | 12 +++++++++---
>> 1 file changed, 9 insertions(+), 3 deletions(-)
>>
>> diff --git a/meta/classes/pip_install_wheel.bbclass b/meta/classes/pip_install_wheel.bbclass
>> index 5b7e5cd706..789c25b68c 100644
>> --- a/meta/classes/pip_install_wheel.bbclass
>> +++ b/meta/classes/pip_install_wheel.bbclass
>> @@ -1,12 +1,13 @@
>> DEPENDS:append = " python3-pip-native"
>>
>> def guess_pip_install_package_name(d):
>> + import re
>> '''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
>> - return (d.getVar('PYPI_PACKAGE') or d.getVar('BPN')).replace('-', '_')
>> + return (d.getVar('PYPI_PACKAGE') or re.sub(r"^python3-", "", d.getVar('BPN')).replace('-', '_'))
>>
>> PIP_INSTALL_PACKAGE ?= "${@guess_pip_install_package_name(d)}"
>> -PIP_INSTALL_DIST_PATH ?= "${B}/dist"
>> -PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-${PV}-*.whl"
>> +PIP_INSTALL_DIST_PATH ?= "${@d.getVar('SETUPTOOLS_SETUP_PATH') or d.getVar('B')}/dist"
>> +PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-*-*.whl"
>>
>> PIP_INSTALL_ARGS ?= "\
>> -vvvv \
>> @@ -45,4 +46,9 @@ pip_install_wheel_do_install () {
>> done
>> }
>>
>> +do_compile:prepend() {
>> + # cleanup previously generated files
>> + rm -f ${PYPA_WHEEL}
>> +}
>> +
>> EXPORT_FUNCTIONS do_install
>> --
>> 2.25.1
>>
>>
>>
>>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#162559): https://lists.openembedded.org/g/openembedded-core/message/162559
> Mute This Topic: https://lists.openembedded.org/mt/89472354/1997914
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [OE-core] [PATCH v3] pip_install_wheel: improve wheel handling
2022-03-01 18:48 ` Khem Raj
@ 2022-03-01 20:14 ` Ross Burton
2022-03-02 5:20 ` Khem Raj
0 siblings, 1 reply; 11+ messages in thread
From: Ross Burton @ 2022-03-01 20:14 UTC (permalink / raw)
To: Khem Raj; +Cc: Tim Orling, Konrad Weihmann, openembedded-core
On Tue, 1 Mar 2022 at 18:48, Khem Raj <raj.khem@gmail.com> wrote:
> where the wheel file is called
> pytest_runner-5.3.1-py3-none-any.whl
I'm thinking we should tell pip to install the wheels to a directory
we control and then just install *.whl.
IIRC the option was --dist-dir, and we can [cleandirs] that to ensure
it's clean on builds.
Ross
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [OE-core] [PATCH v3] pip_install_wheel: improve wheel handling
2022-03-01 20:14 ` Ross Burton
@ 2022-03-02 5:20 ` Khem Raj
2022-03-02 8:06 ` Konrad Weihmann
[not found] ` <16D881D0D78DCA29.25756@lists.openembedded.org>
0 siblings, 2 replies; 11+ messages in thread
From: Khem Raj @ 2022-03-02 5:20 UTC (permalink / raw)
To: Ross Burton; +Cc: Tim Orling, Konrad Weihmann, openembedded-core
this is causing 100+ packages to fail see
https://errors.yoctoproject.org/Errors/Build/142116/
I suggest to include meta-python for wider testing of such changes.
On Tue, Mar 1, 2022 at 12:14 PM Ross Burton <ross@burtonini.com> wrote:
>
> On Tue, 1 Mar 2022 at 18:48, Khem Raj <raj.khem@gmail.com> wrote:
> > where the wheel file is called
> > pytest_runner-5.3.1-py3-none-any.whl
>
> I'm thinking we should tell pip to install the wheels to a directory
> we control and then just install *.whl.
>
> IIRC the option was --dist-dir, and we can [cleandirs] that to ensure
> it's clean on builds.
>
> Ross
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [OE-core] [PATCH v3] pip_install_wheel: improve wheel handling
2022-03-02 5:20 ` Khem Raj
@ 2022-03-02 8:06 ` Konrad Weihmann
[not found] ` <16D881D0D78DCA29.25756@lists.openembedded.org>
1 sibling, 0 replies; 11+ messages in thread
From: Konrad Weihmann @ 2022-03-02 8:06 UTC (permalink / raw)
To: Khem Raj, Ross Burton; +Cc: Tim Orling, openembedded-core
My bad - one of the brackets in the name guessing slipped...
Will send a v4 soon
On 02.03.22 06:20, Khem Raj wrote:
> this is causing 100+ packages to fail see
>
> https://errors.yoctoproject.org/Errors/Build/142116/
>
> I suggest to include meta-python for wider testing of such changes.
>
> On Tue, Mar 1, 2022 at 12:14 PM Ross Burton <ross@burtonini.com> wrote:
>>
>> On Tue, 1 Mar 2022 at 18:48, Khem Raj <raj.khem@gmail.com> wrote:
>>> where the wheel file is called
>>> pytest_runner-5.3.1-py3-none-any.whl
>>
>> I'm thinking we should tell pip to install the wheels to a directory
>> we control and then just install *.whl.
>>
>> IIRC the option was --dist-dir, and we can [cleandirs] that to ensure
>> it's clean on builds.
>>
>> Ross
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [OE-core] [PATCH v3] pip_install_wheel: improve wheel handling
[not found] ` <16D881D0D78DCA29.25756@lists.openembedded.org>
@ 2022-03-02 9:21 ` Konrad Weihmann
2022-03-02 18:45 ` Khem Raj
0 siblings, 1 reply; 11+ messages in thread
From: Konrad Weihmann @ 2022-03-02 9:21 UTC (permalink / raw)
To: Khem Raj, Ross Burton; +Cc: Tim Orling, openembedded-core
v4 is out.
I tried to make it work with the dist-dir/bdist-dir option, but
apparently this isn't respected by setuptools in all of the tested recipes.
That's why I went with applying cleandirs on
${SETUPTOOLS_SETUP_PATH}/dist, that should do the trick as well.
v4 is tested against all core python recipes, all the python recipes in
my layers and a reasonable subset from meta-oe
On 02.03.22 09:06, Konrad Weihmann wrote:
> My bad - one of the brackets in the name guessing slipped...
> Will send a v4 soon
>
> On 02.03.22 06:20, Khem Raj wrote:
>> this is causing 100+ packages to fail see
>>
>> https://errors.yoctoproject.org/Errors/Build/142116/
>>
>> I suggest to include meta-python for wider testing of such changes.
>>
>> On Tue, Mar 1, 2022 at 12:14 PM Ross Burton <ross@burtonini.com> wrote:
>>>
>>> On Tue, 1 Mar 2022 at 18:48, Khem Raj <raj.khem@gmail.com> wrote:
>>>> where the wheel file is called
>>>> pytest_runner-5.3.1-py3-none-any.whl
>>>
>>> I'm thinking we should tell pip to install the wheels to a directory
>>> we control and then just install *.whl.
>>>
>>> IIRC the option was --dist-dir, and we can [cleandirs] that to ensure
>>> it's clean on builds.
>>>
>>> Ross
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#162595): https://lists.openembedded.org/g/openembedded-core/message/162595
> Mute This Topic: https://lists.openembedded.org/mt/89472354/3647476
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [kweihmann@outlook.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [OE-core] [PATCH v3] pip_install_wheel: improve wheel handling
2022-03-02 9:21 ` Konrad Weihmann
@ 2022-03-02 18:45 ` Khem Raj
2022-03-02 19:01 ` Konrad Weihmann
0 siblings, 1 reply; 11+ messages in thread
From: Khem Raj @ 2022-03-02 18:45 UTC (permalink / raw)
To: Konrad Weihmann; +Cc: Ross Burton, Tim Orling, openembedded-core
v4 is definitely better than v3, I see one case it could address as well e.g.
python3-pymetno produces PyMetno-0.9.0-py3-none-any.whl, so if
guessing code could address the case-sensitive piece here would be
good.
On Wed, Mar 2, 2022 at 1:21 AM Konrad Weihmann <kweihmann@outlook.com> wrote:
>
> v4 is out.
> I tried to make it work with the dist-dir/bdist-dir option, but
> apparently this isn't respected by setuptools in all of the tested recipes.
> That's why I went with applying cleandirs on
> ${SETUPTOOLS_SETUP_PATH}/dist, that should do the trick as well.
>
> v4 is tested against all core python recipes, all the python recipes in
> my layers and a reasonable subset from meta-oe
>
> On 02.03.22 09:06, Konrad Weihmann wrote:
> > My bad - one of the brackets in the name guessing slipped...
> > Will send a v4 soon
> >
> > On 02.03.22 06:20, Khem Raj wrote:
> >> this is causing 100+ packages to fail see
> >>
> >> https://errors.yoctoproject.org/Errors/Build/142116/
> >>
> >> I suggest to include meta-python for wider testing of such changes.
> >>
> >> On Tue, Mar 1, 2022 at 12:14 PM Ross Burton <ross@burtonini.com> wrote:
> >>>
> >>> On Tue, 1 Mar 2022 at 18:48, Khem Raj <raj.khem@gmail.com> wrote:
> >>>> where the wheel file is called
> >>>> pytest_runner-5.3.1-py3-none-any.whl
> >>>
> >>> I'm thinking we should tell pip to install the wheels to a directory
> >>> we control and then just install *.whl.
> >>>
> >>> IIRC the option was --dist-dir, and we can [cleandirs] that to ensure
> >>> it's clean on builds.
> >>>
> >>> Ross
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#162595): https://lists.openembedded.org/g/openembedded-core/message/162595
> > Mute This Topic: https://lists.openembedded.org/mt/89472354/3647476
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [kweihmann@outlook.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [OE-core] [PATCH v3] pip_install_wheel: improve wheel handling
2022-03-02 18:45 ` Khem Raj
@ 2022-03-02 19:01 ` Konrad Weihmann
2022-03-02 19:04 ` Khem Raj
0 siblings, 1 reply; 11+ messages in thread
From: Konrad Weihmann @ 2022-03-02 19:01 UTC (permalink / raw)
To: Khem Raj; +Cc: Ross Burton, Tim Orling, openembedded-core
On 02.03.22 19:45, Khem Raj wrote:
> v4 is definitely better than v3, I see one case it could address as well e.g.
>
> python3-pymetno produces PyMetno-0.9.0-py3-none-any.whl, so if
> guessing code could address the case-sensitive piece here would be
> good.
I'm torn on this one, I agree that it would really nice to have in terms
of a seamless transition, but it would require either some bash/shell
hacks to disable case sensitive filename globs or some "after compile"
renaming - I'm not sure how to tackle this.
If everyone else agrees I would vote for "after compile" renaming to a
fully lower case filename - any thoughts?
>
> On Wed, Mar 2, 2022 at 1:21 AM Konrad Weihmann <kweihmann@outlook.com> wrote:
>>
>> v4 is out.
>> I tried to make it work with the dist-dir/bdist-dir option, but
>> apparently this isn't respected by setuptools in all of the tested recipes.
>> That's why I went with applying cleandirs on
>> ${SETUPTOOLS_SETUP_PATH}/dist, that should do the trick as well.
>>
>> v4 is tested against all core python recipes, all the python recipes in
>> my layers and a reasonable subset from meta-oe
>>
>> On 02.03.22 09:06, Konrad Weihmann wrote:
>>> My bad - one of the brackets in the name guessing slipped...
>>> Will send a v4 soon
>>>
>>> On 02.03.22 06:20, Khem Raj wrote:
>>>> this is causing 100+ packages to fail see
>>>>
>>>> https://errors.yoctoproject.org/Errors/Build/142116/
>>>>
>>>> I suggest to include meta-python for wider testing of such changes.
>>>>
>>>> On Tue, Mar 1, 2022 at 12:14 PM Ross Burton <ross@burtonini.com> wrote:
>>>>>
>>>>> On Tue, 1 Mar 2022 at 18:48, Khem Raj <raj.khem@gmail.com> wrote:
>>>>>> where the wheel file is called
>>>>>> pytest_runner-5.3.1-py3-none-any.whl
>>>>>
>>>>> I'm thinking we should tell pip to install the wheels to a directory
>>>>> we control and then just install *.whl.
>>>>>
>>>>> IIRC the option was --dist-dir, and we can [cleandirs] that to ensure
>>>>> it's clean on builds.
>>>>>
>>>>> Ross
>>>
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#162595): https://lists.openembedded.org/g/openembedded-core/message/162595
>>> Mute This Topic: https://lists.openembedded.org/mt/89472354/3647476
>>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [kweihmann@outlook.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [OE-core] [PATCH v3] pip_install_wheel: improve wheel handling
2022-03-02 19:01 ` Konrad Weihmann
@ 2022-03-02 19:04 ` Khem Raj
2022-03-02 19:09 ` Konrad Weihmann
0 siblings, 1 reply; 11+ messages in thread
From: Khem Raj @ 2022-03-02 19:04 UTC (permalink / raw)
To: Konrad Weihmann; +Cc: Ross Burton, Tim Orling, openembedded-core
On Wed, Mar 2, 2022 at 11:01 AM Konrad Weihmann <kweihmann@outlook.com> wrote:
>
>
>
> On 02.03.22 19:45, Khem Raj wrote:
> > v4 is definitely better than v3, I see one case it could address as well e.g.
> >
> > python3-pymetno produces PyMetno-0.9.0-py3-none-any.whl, so if
> > guessing code could address the case-sensitive piece here would be
> > good.
>
> I'm torn on this one, I agree that it would really nice to have in terms
> of a seamless transition, but it would require either some bash/shell
> hacks to disable case sensitive filename globs or some "after compile"
> renaming - I'm not sure how to tackle this.
> If everyone else agrees I would vote for "after compile" renaming to a
> fully lower case filename - any thoughts?
>
I think the suggestion that Ross has is a good one to install the wheel file
intentionally to a known location using --dist-dir and then use a more
relaxed regexp to package it.
> >
> > On Wed, Mar 2, 2022 at 1:21 AM Konrad Weihmann <kweihmann@outlook.com> wrote:
> >>
> >> v4 is out.
> >> I tried to make it work with the dist-dir/bdist-dir option, but
> >> apparently this isn't respected by setuptools in all of the tested recipes.
> >> That's why I went with applying cleandirs on
> >> ${SETUPTOOLS_SETUP_PATH}/dist, that should do the trick as well.
> >>
> >> v4 is tested against all core python recipes, all the python recipes in
> >> my layers and a reasonable subset from meta-oe
> >>
> >> On 02.03.22 09:06, Konrad Weihmann wrote:
> >>> My bad - one of the brackets in the name guessing slipped...
> >>> Will send a v4 soon
> >>>
> >>> On 02.03.22 06:20, Khem Raj wrote:
> >>>> this is causing 100+ packages to fail see
> >>>>
> >>>> https://errors.yoctoproject.org/Errors/Build/142116/
> >>>>
> >>>> I suggest to include meta-python for wider testing of such changes.
> >>>>
> >>>> On Tue, Mar 1, 2022 at 12:14 PM Ross Burton <ross@burtonini.com> wrote:
> >>>>>
> >>>>> On Tue, 1 Mar 2022 at 18:48, Khem Raj <raj.khem@gmail.com> wrote:
> >>>>>> where the wheel file is called
> >>>>>> pytest_runner-5.3.1-py3-none-any.whl
> >>>>>
> >>>>> I'm thinking we should tell pip to install the wheels to a directory
> >>>>> we control and then just install *.whl.
> >>>>>
> >>>>> IIRC the option was --dist-dir, and we can [cleandirs] that to ensure
> >>>>> it's clean on builds.
> >>>>>
> >>>>> Ross
> >>>
> >>>
> >>> -=-=-=-=-=-=-=-=-=-=-=-
> >>> Links: You receive all messages sent to this group.
> >>> View/Reply Online (#162595): https://lists.openembedded.org/g/openembedded-core/message/162595
> >>> Mute This Topic: https://lists.openembedded.org/mt/89472354/3647476
> >>> Group Owner: openembedded-core+owner@lists.openembedded.org
> >>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [kweihmann@outlook.com]
> >>> -=-=-=-=-=-=-=-=-=-=-=-
> >>>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [OE-core] [PATCH v3] pip_install_wheel: improve wheel handling
2022-03-02 19:04 ` Khem Raj
@ 2022-03-02 19:09 ` Konrad Weihmann
0 siblings, 0 replies; 11+ messages in thread
From: Konrad Weihmann @ 2022-03-02 19:09 UTC (permalink / raw)
To: Khem Raj; +Cc: Ross Burton, Tim Orling, openembedded-core
On 02.03.22 20:04, Khem Raj wrote:
> On Wed, Mar 2, 2022 at 11:01 AM Konrad Weihmann <kweihmann@outlook.com> wrote:
>>
>>
>>
>> On 02.03.22 19:45, Khem Raj wrote:
>>> v4 is definitely better than v3, I see one case it could address as well e.g.
>>>
>>> python3-pymetno produces PyMetno-0.9.0-py3-none-any.whl, so if
>>> guessing code could address the case-sensitive piece here would be
>>> good.
>>
>> I'm torn on this one, I agree that it would really nice to have in terms
>> of a seamless transition, but it would require either some bash/shell
>> hacks to disable case sensitive filename globs or some "after compile"
>> renaming - I'm not sure how to tackle this.
>> If everyone else agrees I would vote for "after compile" renaming to a
>> fully lower case filename - any thoughts?
>>
>
> I think the suggestion that Ross has is a good one to install the wheel file
> intentionally to a known location using --dist-dir and then use a more
> relaxed regexp to package it.
As I mentioned in the v4 patch dist-dir doesn't work for me - somehow it
isn't respected by most of the python modules.
So my idea would be it remain on default "dist" directory, but run
something line rename 'y/A-Z/a-z/' ${PYPA_WHEEL} after compile - also
minding that in PIP_INSTALL_PACKAGE.
I'm happy to see dist-dir/bdist-dir work, but so far I failed to see any
successful run on my setup (Ubuntu 20.04 btw)
>
>>>
>>> On Wed, Mar 2, 2022 at 1:21 AM Konrad Weihmann <kweihmann@outlook.com> wrote:
>>>>
>>>> v4 is out.
>>>> I tried to make it work with the dist-dir/bdist-dir option, but
>>>> apparently this isn't respected by setuptools in all of the tested recipes.
>>>> That's why I went with applying cleandirs on
>>>> ${SETUPTOOLS_SETUP_PATH}/dist, that should do the trick as well.
>>>>
>>>> v4 is tested against all core python recipes, all the python recipes in
>>>> my layers and a reasonable subset from meta-oe
>>>>
>>>> On 02.03.22 09:06, Konrad Weihmann wrote:
>>>>> My bad - one of the brackets in the name guessing slipped...
>>>>> Will send a v4 soon
>>>>>
>>>>> On 02.03.22 06:20, Khem Raj wrote:
>>>>>> this is causing 100+ packages to fail see
>>>>>>
>>>>>> https://errors.yoctoproject.org/Errors/Build/142116/
>>>>>>
>>>>>> I suggest to include meta-python for wider testing of such changes.
>>>>>>
>>>>>> On Tue, Mar 1, 2022 at 12:14 PM Ross Burton <ross@burtonini.com> wrote:
>>>>>>>
>>>>>>> On Tue, 1 Mar 2022 at 18:48, Khem Raj <raj.khem@gmail.com> wrote:
>>>>>>>> where the wheel file is called
>>>>>>>> pytest_runner-5.3.1-py3-none-any.whl
>>>>>>>
>>>>>>> I'm thinking we should tell pip to install the wheels to a directory
>>>>>>> we control and then just install *.whl.
>>>>>>>
>>>>>>> IIRC the option was --dist-dir, and we can [cleandirs] that to ensure
>>>>>>> it's clean on builds.
>>>>>>>
>>>>>>> Ross
>>>>>
>>>>>
>>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>> Links: You receive all messages sent to this group.
>>>>> View/Reply Online (#162595): https://lists.openembedded.org/g/openembedded-core/message/162595
>>>>> Mute This Topic: https://lists.openembedded.org/mt/89472354/3647476
>>>>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>>>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [kweihmann@outlook.com]
>>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>>
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2022-03-02 19:09 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-01 9:42 [PATCH v3] pip_install_wheel: improve wheel handling Konrad Weihmann
2022-03-01 15:22 ` [OE-core] " Tim Orling
2022-03-01 18:48 ` Khem Raj
2022-03-01 20:14 ` Ross Burton
2022-03-02 5:20 ` Khem Raj
2022-03-02 8:06 ` Konrad Weihmann
[not found] ` <16D881D0D78DCA29.25756@lists.openembedded.org>
2022-03-02 9:21 ` Konrad Weihmann
2022-03-02 18:45 ` Khem Raj
2022-03-02 19:01 ` Konrad Weihmann
2022-03-02 19:04 ` Khem Raj
2022-03-02 19:09 ` Konrad Weihmann
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.