new meta-oe failures from master-next

new meta-oe failures from master-next

[Khem Raj]

Hi Richard

New master-next I see these fails

https://errors.yoctoproject.org/Errors/Build/91645/

First two are important one's we already know why
ti-display-sharing-fw fails to fetch

Re: new meta-oe failures from master-next

[Khem Raj]

here is AB failure link

https://autobuilder.yoctoproject.org/typhoon/#/builders/88/builds/56

On Wed, Oct 23, 2019 at 2:03 PM Khem Raj <raj.khem@gmail.com> wrote:
>
> Hi Richard
>
> New master-next I see these fails
>
> https://errors.yoctoproject.org/Errors/Build/91645/
>
> First two are important one's we already know why
> ti-display-sharing-fw fails to fetch

Re: new meta-oe failures from master-next

[Ross Burton]

On 23/10/2019 14:03, Khem Raj wrote:
> Hi Richard
> 
> New master-next I see these fails
> 
> https://errors.yoctoproject.org/Errors/Build/91645/
> 
> First two are important one's we already know why
> ti-display-sharing-fw fails to fetch

ERROR: QA Issue: libiio: Files/directories were installed but not 
shipped in any package:
   /usr/lib/python2.7
   /usr/lib/python2.7/site-packages
   /usr/lib/python2.7/site-packages/libiio-0.18-py2.7.egg-info
   /usr/lib/python2.7/site-packages/iio.pyc
   /usr/lib/python2.7/site-packages/iio.py

That may have been triggered by some change in oe-core but that's 
definitely an issue with libiio.

WARNING: 
TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-r0/temp/run.do_configure.7679:1 
exit 1 from 'NO_FETCH_BUILD=1 
TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-r0/recipe-sysroot-native/usr/bin/python3-native/python3 
setup.py clean'
ERROR: Execution of 
'TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-r0/temp/run.do_configure.7679' 
failed with exit code 1:
Traceback (most recent call last):
   File "setup.py", line 17, in <module>
     setup(**config['options'])
   File 
"TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-r0/recipe-sysroot-native/usr/lib/python3.7/site-packages/setuptools/__init__.py", 
line 145, in setup
     return distutils.core.setup(**attrs)

Looks like whatever astor is doing with setuptools doesn't agree with 
the upgrade to 41.4.0?

Ross

Re: new meta-oe failures from master-next

[akuster808]

On 10/23/19 6:05 AM, Khem Raj wrote:
> here is AB failure link
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/88/builds/56

Your builds are also archived on wiki

https://wiki.yoctoproject.org/wiki/BuildLog

>
> On Wed, Oct 23, 2019 at 2:03 PM Khem Raj <raj.khem@gmail.com> wrote:
>> Hi Richard
>>
>> New master-next I see these fails
>>
>> https://errors.yoctoproject.org/Errors/Build/91645/
>>
>> First two are important one's we already know why
>> ti-display-sharing-fw fails to fetch

Re: new meta-oe failures from master-next

[richard.purdie]

Hi Khem.

On Wed, 2019-10-23 at 14:03 +0100, Khem Raj wrote:
> Hi Richard
> 
> New master-next I see these fails
> 
> https://errors.yoctoproject.org/Errors/Build/91645/
> 
> First two are important one's we already know why
> ti-display-sharing-fw fails to fetch

Firstly, before you mention it, I started merging master-next earlier
today after multiple tries and finally a green build last night.

I paused as there was an issue, I dropped more patches and eventually
after I thought I'd handled all the remaining issues I knew about, I
merged things. I didn't see this report until now (afterwards) :(.

I guess I now need to bisect things and figure out where these failures
came from.

Cheers,

Richard

Re: new meta-oe failures from master-next

[Adrian Bunk]

On Wed, Oct 23, 2019 at 04:29:13PM +0100, Ross Burton wrote:
> On 23/10/2019 14:03, Khem Raj wrote:
> > Hi Richard
> > 
> > New master-next I see these fails
> > 
> > https://errors.yoctoproject.org/Errors/Build/91645/
> > 
> > First two are important one's we already know why
> > ti-display-sharing-fw fails to fetch
> 
> ERROR: QA Issue: libiio: Files/directories were installed but not shipped in
> any package:
>   /usr/lib/python2.7
>   /usr/lib/python2.7/site-packages
>   /usr/lib/python2.7/site-packages/libiio-0.18-py2.7.egg-info
>   /usr/lib/python2.7/site-packages/iio.pyc
>   /usr/lib/python2.7/site-packages/iio.py
> 
> That may have been triggered by some change in oe-core but that's definitely
> an issue with libiio.

I'd agree on that:

Python bindings were converted from python2 to python3 last week,
and it picks up the host python 2.7.

-inherit cmake pythonnative systemd
+inherit cmake python3native systemd

This does not in any way guarantee that there is no python2 interpreter 
available.

> WARNING: TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-r0/temp/run.do_configure.7679:1
> exit 1 from 'NO_FETCH_BUILD=1 TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-r0/recipe-sysroot-native/usr/bin/python3-native/python3
> setup.py clean'
> ERROR: Execution of 'TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-r0/temp/run.do_configure.7679'
> failed with exit code 1:
> Traceback (most recent call last):
>   File "setup.py", line 17, in <module>
>     setup(**config['options'])
>   File "TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-r0/recipe-sysroot-native/usr/lib/python3.7/site-packages/setuptools/__init__.py",
> line 145, in setup
>     return distutils.core.setup(**attrs)
> 
> Looks like whatever astor is doing with setuptools doesn't agree with the
> upgrade to 41.4.0?

The recipe is behind upstream, but even the latest version needs fixing:
https://github.com/berkerpeksag/astor/issues/162

Temporarily broken leaf package, not really important.

> Ross

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: new meta-oe failures from master-next

[Richard Purdie]

On Wed, 2019-10-23 at 16:29 +0100, Ross Burton wrote:
> On 23/10/2019 14:03, Khem Raj wrote:
> > Hi Richard
> > 
> > New master-next I see these fails
> > 
> > https://errors.yoctoproject.org/Errors/Build/91645/
> > 
> > First two are important one's we already know why
> > ti-display-sharing-fw fails to fetch
> 
> ERROR: QA Issue: libiio: Files/directories were installed but not 
> shipped in any package:
>    /usr/lib/python2.7
>    /usr/lib/python2.7/site-packages
>    /usr/lib/python2.7/site-packages/libiio-0.18-py2.7.egg-info
>    /usr/lib/python2.7/site-packages/iio.pyc
>    /usr/lib/python2.7/site-packages/iio.py
> 
> That may have been triggered by some change in oe-core but that's 
> definitely an issue with libiio.

Its from this cmake change:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=dbba06d9153ae588c9496b40e5db18b3602bfad1


> 
> WARNING: 
> TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-
> r0/temp/run.do_configure.7679:1 
> exit 1 from 'NO_FETCH_BUILD=1 
> TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-
> r0/recipe-sysroot-native/usr/bin/python3-native/python3 
> setup.py clean'
> ERROR: Execution of 
> 'TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-
> r0/temp/run.do_configure.7679' 
> failed with exit code 1:
> Traceback (most recent call last):
>    File "setup.py", line 17, in <module>
>      setup(**config['options'])
>    File 
> "TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-
> r0/recipe-sysroot-native/usr/lib/python3.7/site-
> packages/setuptools/__init__.py", 
> line 145, in setup
>      return distutils.core.setup(**attrs)
> 
> Looks like whatever astor is doing with setuptools doesn't agree
> with the upgrade to 41.4.0?

Yes, I can confirm that reverting 
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=b9703df2853e72b974a63ef1146f09a6c197990d 
locally does make it build again.

Cheers,

Richard

Re: new meta-oe failures from master-next

[Richard Purdie]

On Wed, 2019-10-23 at 22:23 +0300, Adrian Bunk wrote:
> On Wed, Oct 23, 2019 at 04:29:13PM +0100, Ross Burton wrote:
> > On 23/10/2019 14:03, Khem Raj wrote:
> > > Hi Richard
> > > 
> > > New master-next I see these fails
> > > 
> > > https://errors.yoctoproject.org/Errors/Build/91645/
> > > 
> > > First two are important one's we already know why
> > > ti-display-sharing-fw fails to fetch
> > 
> > ERROR: QA Issue: libiio: Files/directories were installed but not
> > shipped in
> > any package:
> >   /usr/lib/python2.7
> >   /usr/lib/python2.7/site-packages
> >   /usr/lib/python2.7/site-packages/libiio-0.18-py2.7.egg-info
> >   /usr/lib/python2.7/site-packages/iio.pyc
> >   /usr/lib/python2.7/site-packages/iio.py
> > 
> > That may have been triggered by some change in oe-core but that's
> > definitely
> > an issue with libiio.
> 
> I'd agree on that:
> 
> Python bindings were converted from python2 to python3 last week,
> and it picks up the host python 2.7.
> 
> -inherit cmake pythonnative systemd
> +inherit cmake python3native systemd
> 
> This does not in any way guarantee that there is no python2
> interpreter 
> available.
> 
> > WARNING: TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-
> > astor/0.7.1-r0/temp/run.do_configure.7679:1
> > exit 1 from 'NO_FETCH_BUILD=1 TOPDIR/build/tmp/work/mips32r2-yoe-
> > linux/python3-astor/0.7.1-r0/recipe-sysroot-native/usr/bin/python3-
> > native/python3
> > setup.py clean'
> > ERROR: Execution of 'TOPDIR/build/tmp/work/mips32r2-yoe-
> > linux/python3-astor/0.7.1-r0/temp/run.do_configure.7679'
> > failed with exit code 1:
> > Traceback (most recent call last):
> >   File "setup.py", line 17, in <module>
> >     setup(**config['options'])
> >   File "TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-
> > astor/0.7.1-r0/recipe-sysroot-native/usr/lib/python3.7/site-
> > packages/setuptools/__init__.py",
> > line 145, in setup
> >     return distutils.core.setup(**attrs)
> > 
> > Looks like whatever astor is doing with setuptools doesn't agree
> > with the
> > upgrade to 41.4.0?
> 
> The recipe is behind upstream, but even the latest version needs
> fixing:
> https://github.com/berkerpeksag/astor/issues/162
> 
> Temporarily broken leaf package, not really important.

Well, yes. The proposed policy is now that we should address these
kinds of issues if they get reported. Whilst I understand why and that
Khem doesn't get as much help as he needs with meta-oe, pushing the oe-
core maintainers to fix things likely won't scale. I would like to see
meta-oe quality/reliability improve though so I do understand the
desire.

Since I'd had to look into them, I've posted patches to oe-devel for
the above two issues.

Cheers,

Richard

Re: new meta-oe failures from master-next

[Adrian Bunk]

On Wed, Oct 23, 2019 at 10:20:29PM +0100, Richard Purdie wrote:
> 
> Well, yes. The proposed policy is now that we should address these
> kinds of issues if they get reported. Whilst I understand why and that
> Khem doesn't get as much help as he needs with meta-oe, pushing the oe-
> core maintainers to fix things likely won't scale. I would like to see
> meta-oe quality/reliability improve though so I do understand the
> desire.
>...

What is the proposed process for oe-core changes that are likely to 
break reverse dependencies in meta-oe?

Python 3.8 would be a good example for that - for astor your upgrade to 
the latest upstream version already contains Python 3.8 fixes, but other 
recipes in meta-oe might require fixing by someone.

> Cheers,
> 
> Richard

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: new meta-oe failures from master-next

[Khem Raj]

On Wed, Oct 23, 2019 at 8:31 PM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Wed, 2019-10-23 at 16:29 +0100, Ross Burton wrote:
> > On 23/10/2019 14:03, Khem Raj wrote:
> > > Hi Richard
> > >
> > > New master-next I see these fails
> > >
> > > https://errors.yoctoproject.org/Errors/Build/91645/
> > >
> > > First two are important one's we already know why
> > > ti-display-sharing-fw fails to fetch
> >
> > ERROR: QA Issue: libiio: Files/directories were installed but not
> > shipped in any package:
> >    /usr/lib/python2.7
> >    /usr/lib/python2.7/site-packages
> >    /usr/lib/python2.7/site-packages/libiio-0.18-py2.7.egg-info
> >    /usr/lib/python2.7/site-packages/iio.pyc
> >    /usr/lib/python2.7/site-packages/iio.py
> >
> > That may have been triggered by some change in oe-core but that's
> > definitely an issue with libiio.
>
> Its from this cmake change:
>
> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=dbba06d9153ae588c9496b40e5db18b3602bfad1
>

I think its causing more failures, where suddenly, the recipes are now
resorting to native python, is this patch correct I wonder ?

see
https://git.openembedded.org/meta-openembedded-contrib/commit/?h=yoe/mut&id=5d02e814bc7d30f925b2eeec85eb053cec516f26

https://git.openembedded.org/meta-openembedded-contrib/commit/?h=yoe/mut&id=3ab0af60e8512cbb718e8a0112d9c9aa98c620a4

>
> >
> > WARNING:
> > TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-
> > r0/temp/run.do_configure.7679:1
> > exit 1 from 'NO_FETCH_BUILD=1
> > TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-
> > r0/recipe-sysroot-native/usr/bin/python3-native/python3
> > setup.py clean'
> > ERROR: Execution of
> > 'TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-
> > r0/temp/run.do_configure.7679'
> > failed with exit code 1:
> > Traceback (most recent call last):
> >    File "setup.py", line 17, in <module>
> >      setup(**config['options'])
> >    File
> > "TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-
> > r0/recipe-sysroot-native/usr/lib/python3.7/site-
> > packages/setuptools/__init__.py",
> > line 145, in setup
> >      return distutils.core.setup(**attrs)
> >
> > Looks like whatever astor is doing with setuptools doesn't agree
> > with the upgrade to 41.4.0?
>
> Yes, I can confirm that reverting
> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=b9703df2853e72b974a63ef1146f09a6c197990d
> locally does make it build again.
>
> Cheers,
>
> Richard
>
>
>
>
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: new meta-oe failures from master-next

[Khem Raj]

On Wed, Oct 23, 2019 at 11:52 PM Khem Raj <raj.khem@gmail.com> wrote:
>
> On Wed, Oct 23, 2019 at 8:31 PM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> >
> > On Wed, 2019-10-23 at 16:29 +0100, Ross Burton wrote:
> > > On 23/10/2019 14:03, Khem Raj wrote:
> > > > Hi Richard
> > > >
> > > > New master-next I see these fails
> > > >
> > > > https://errors.yoctoproject.org/Errors/Build/91645/
> > > >
> > > > First two are important one's we already know why
> > > > ti-display-sharing-fw fails to fetch
> > >
> > > ERROR: QA Issue: libiio: Files/directories were installed but not
> > > shipped in any package:
> > >    /usr/lib/python2.7
> > >    /usr/lib/python2.7/site-packages
> > >    /usr/lib/python2.7/site-packages/libiio-0.18-py2.7.egg-info
> > >    /usr/lib/python2.7/site-packages/iio.pyc
> > >    /usr/lib/python2.7/site-packages/iio.py
> > >
> > > That may have been triggered by some change in oe-core but that's
> > > definitely an issue with libiio.
> >
> > Its from this cmake change:
> >
> > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=dbba06d9153ae588c9496b40e5db18b3602bfad1
> >
>
> I think its causing more failures, where suddenly, the recipes are now
> resorting to native python, is this patch correct I wonder ?
>
> see
> https://git.openembedded.org/meta-openembedded-contrib/commit/?h=yoe/mut&id=5d02e814bc7d30f925b2eeec85eb053cec516f26
>
> https://git.openembedded.org/meta-openembedded-contrib/commit/?h=yoe/mut&id=3ab0af60e8512cbb718e8a0112d9c9aa98c620a4

more fallouts
https://errors.yoctoproject.org/Errors/Details/274418/
>
> >
> > >
> > > WARNING:
> > > TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-
> > > r0/temp/run.do_configure.7679:1
> > > exit 1 from 'NO_FETCH_BUILD=1
> > > TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-
> > > r0/recipe-sysroot-native/usr/bin/python3-native/python3
> > > setup.py clean'
> > > ERROR: Execution of
> > > 'TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-
> > > r0/temp/run.do_configure.7679'
> > > failed with exit code 1:
> > > Traceback (most recent call last):
> > >    File "setup.py", line 17, in <module>
> > >      setup(**config['options'])
> > >    File
> > > "TOPDIR/build/tmp/work/mips32r2-yoe-linux/python3-astor/0.7.1-
> > > r0/recipe-sysroot-native/usr/lib/python3.7/site-
> > > packages/setuptools/__init__.py",
> > > line 145, in setup
> > >      return distutils.core.setup(**attrs)
> > >
> > > Looks like whatever astor is doing with setuptools doesn't agree
> > > with the upgrade to 41.4.0?
> >
> > Yes, I can confirm that reverting
> > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=b9703df2853e72b974a63ef1146f09a6c197990d
> > locally does make it build again.
> >
> > Cheers,
> >
> > Richard
> >
> >
> >
> >
> >
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: new meta-oe failures from master-next

[richard.purdie]

On Wed, 2019-10-23 at 23:52 +0100, Khem Raj wrote:
> On Wed, Oct 23, 2019 at 8:31 PM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Wed, 2019-10-23 at 16:29 +0100, Ross Burton wrote:
> > > On 23/10/2019 14:03, Khem Raj wrote:
> > > > Hi Richard
> > > > 
> > > > New master-next I see these fails
> > > > 
> > > > https://errors.yoctoproject.org/Errors/Build/91645/
> > > > 
> > > > First two are important one's we already know why
> > > > ti-display-sharing-fw fails to fetch
> > > 
> > > ERROR: QA Issue: libiio: Files/directories were installed but not
> > > shipped in any package:
> > >    /usr/lib/python2.7
> > >    /usr/lib/python2.7/site-packages
> > >    /usr/lib/python2.7/site-packages/libiio-0.18-py2.7.egg-info
> > >    /usr/lib/python2.7/site-packages/iio.pyc
> > >    /usr/lib/python2.7/site-packages/iio.py
> > > 
> > > That may have been triggered by some change in oe-core but that's
> > > definitely an issue with libiio.
> > 
> > Its from this cmake change:
> > 
> > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=dbba06d9153ae588c9496b40e5db18b3602bfad1
> > 
> 
> I think its causing more failures, where suddenly, the recipes are
> now
> resorting to native python, is this patch correct I wonder ?
> 
> see
> https://git.openembedded.org/meta-openembedded-contrib/commit/?h=yoe/mut&id=5d02e814bc7d30f925b2eeec85eb053cec516f26
> 
> https://git.openembedded.org/meta-openembedded-contrib/commit/?h=yoe/mut&id=3ab0af60e8512cbb718e8a0112d9c9aa98c620a4

The cmake change makes python visible from HOSTTOOLS. That isn't wrong
as such, it is a change in behaviour. If recipes don't set python
enabled/disabled explicitly, there is some fallout from that. I'd say
that is a bug in those recipes which aren't giving explicit
configuration to dependencies though.

Cheers,

RIchard

Re: new meta-oe failures from master-next

[Khem Raj]

On Thu, Oct 24, 2019 at 9:15 AM <richard.purdie@linuxfoundation.org> wrote:
>
> On Wed, 2019-10-23 at 23:52 +0100, Khem Raj wrote:
> > On Wed, Oct 23, 2019 at 8:31 PM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > > On Wed, 2019-10-23 at 16:29 +0100, Ross Burton wrote:
> > > > On 23/10/2019 14:03, Khem Raj wrote:
> > > > > Hi Richard
> > > > >
> > > > > New master-next I see these fails
> > > > >
> > > > > https://errors.yoctoproject.org/Errors/Build/91645/
> > > > >
> > > > > First two are important one's we already know why
> > > > > ti-display-sharing-fw fails to fetch
> > > >
> > > > ERROR: QA Issue: libiio: Files/directories were installed but not
> > > > shipped in any package:
> > > >    /usr/lib/python2.7
> > > >    /usr/lib/python2.7/site-packages
> > > >    /usr/lib/python2.7/site-packages/libiio-0.18-py2.7.egg-info
> > > >    /usr/lib/python2.7/site-packages/iio.pyc
> > > >    /usr/lib/python2.7/site-packages/iio.py
> > > >
> > > > That may have been triggered by some change in oe-core but that's
> > > > definitely an issue with libiio.
> > >
> > > Its from this cmake change:
> > >
> > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=dbba06d9153ae588c9496b40e5db18b3602bfad1
> > >
> >
> > I think its causing more failures, where suddenly, the recipes are
> > now
> > resorting to native python, is this patch correct I wonder ?
> >
> > see
> > https://git.openembedded.org/meta-openembedded-contrib/commit/?h=yoe/mut&id=5d02e814bc7d30f925b2eeec85eb053cec516f26
> >
> > https://git.openembedded.org/meta-openembedded-contrib/commit/?h=yoe/mut&id=3ab0af60e8512cbb718e8a0112d9c9aa98c620a4
>
> The cmake change makes python visible from HOSTTOOLS. That isn't wrong
> as such, it is a change in behaviour. If recipes don't set python
> enabled/disabled explicitly, there is some fallout from that. I'd say
> that is a bug in those recipes which aren't giving explicit
> configuration to dependencies though.
>

So now the behavior is interesting. It works fine if I build for a
machine where host-arch = target-arch
so if someone is building for x86_64 then they dont see the build
failures. IMO thats worse.

> Cheers,
>
> RIchard
>
>
>

Re: new meta-oe failures from master-next

[richard.purdie]

On Thu, 2019-10-24 at 09:20 +0100, Khem Raj wrote:
> On Thu, Oct 24, 2019 at 9:15 AM <richard.purdie@linuxfoundation.org>
> wrote:
> > On Wed, 2019-10-23 at 23:52 +0100, Khem Raj wrote:
> > > On Wed, Oct 23, 2019 at 8:31 PM Richard Purdie
> > > <richard.purdie@linuxfoundation.org> wrote:
> > > > On Wed, 2019-10-23 at 16:29 +0100, Ross Burton wrote:
> > > > > On 23/10/2019 14:03, Khem Raj wrote:
> > > > > > Hi Richard
> > > > > > 
> > > > > > New master-next I see these fails
> > > > > > 
> > > > > > https://errors.yoctoproject.org/Errors/Build/91645/
> > > > > > 
> > > > > > First two are important one's we already know why
> > > > > > ti-display-sharing-fw fails to fetch
> > > > > 
> > > > > ERROR: QA Issue: libiio: Files/directories were installed but
> > > > > not
> > > > > shipped in any package:
> > > > >    /usr/lib/python2.7
> > > > >    /usr/lib/python2.7/site-packages
> > > > >    /usr/lib/python2.7/site-packages/libiio-0.18-py2.7.egg-
> > > > > info
> > > > >    /usr/lib/python2.7/site-packages/iio.pyc
> > > > >    /usr/lib/python2.7/site-packages/iio.py
> > > > > 
> > > > > That may have been triggered by some change in oe-core but
> > > > > that's
> > > > > definitely an issue with libiio.
> > > > 
> > > > Its from this cmake change:
> > > > 
> > > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=dbba06d9153ae588c9496b40e5db18b3602bfad1
> > > > 
> > > 
> > > I think its causing more failures, where suddenly, the recipes
> > > are
> > > now
> > > resorting to native python, is this patch correct I wonder ?
> > > 
> > > see
> > > https://git.openembedded.org/meta-openembedded-contrib/commit/?h=yoe/mut&id=5d02e814bc7d30f925b2eeec85eb053cec516f26
> > > 
> > > https://git.openembedded.org/meta-openembedded-contrib/commit/?h=yoe/mut&id=3ab0af60e8512cbb718e8a0112d9c9aa98c620a4
> > 
> > The cmake change makes python visible from HOSTTOOLS. That isn't
> > wrong
> > as such, it is a change in behaviour. If recipes don't set python
> > enabled/disabled explicitly, there is some fallout from that. I'd
> > say
> > that is a bug in those recipes which aren't giving explicit
> > configuration to dependencies though.
> > 
> 
> So now the behavior is interesting. It works fine if I build for a
> machine where host-arch = target-arch
> so if someone is building for x86_64 then they dont see the build
> failures. IMO thats worse.

That is a general problem with cross compiling and applies to any
automatically detected dependency from the host. Python is in an odd
place here since we do need it on the host to run bitbake and there are
places where we need that exposed to code (e.g. they have python
scripts), there are places where we don't want it exposed as host
contamination.

Have you looked at the buildhistory logs to see what changed in meta-oe 
as that would likely highlight the changes.

Cheers,

Richard

Re: new meta-oe failures from master-next

[Khem Raj]

On Thu, Oct 24, 2019 at 9:25 AM <richard.purdie@linuxfoundation.org> wrote:
>
> On Thu, 2019-10-24 at 09:20 +0100, Khem Raj wrote:
> > On Thu, Oct 24, 2019 at 9:15 AM <richard.purdie@linuxfoundation.org>
> > wrote:
> > > On Wed, 2019-10-23 at 23:52 +0100, Khem Raj wrote:
> > > > On Wed, Oct 23, 2019 at 8:31 PM Richard Purdie
> > > > <richard.purdie@linuxfoundation.org> wrote:
> > > > > On Wed, 2019-10-23 at 16:29 +0100, Ross Burton wrote:
> > > > > > On 23/10/2019 14:03, Khem Raj wrote:
> > > > > > > Hi Richard
> > > > > > >
> > > > > > > New master-next I see these fails
> > > > > > >
> > > > > > > https://errors.yoctoproject.org/Errors/Build/91645/
> > > > > > >
> > > > > > > First two are important one's we already know why
> > > > > > > ti-display-sharing-fw fails to fetch
> > > > > >
> > > > > > ERROR: QA Issue: libiio: Files/directories were installed but
> > > > > > not
> > > > > > shipped in any package:
> > > > > >    /usr/lib/python2.7
> > > > > >    /usr/lib/python2.7/site-packages
> > > > > >    /usr/lib/python2.7/site-packages/libiio-0.18-py2.7.egg-
> > > > > > info
> > > > > >    /usr/lib/python2.7/site-packages/iio.pyc
> > > > > >    /usr/lib/python2.7/site-packages/iio.py
> > > > > >
> > > > > > That may have been triggered by some change in oe-core but
> > > > > > that's
> > > > > > definitely an issue with libiio.
> > > > >
> > > > > Its from this cmake change:
> > > > >
> > > > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=dbba06d9153ae588c9496b40e5db18b3602bfad1
> > > > >
> > > >
> > > > I think its causing more failures, where suddenly, the recipes
> > > > are
> > > > now
> > > > resorting to native python, is this patch correct I wonder ?
> > > >
> > > > see
> > > > https://git.openembedded.org/meta-openembedded-contrib/commit/?h=yoe/mut&id=5d02e814bc7d30f925b2eeec85eb053cec516f26
> > > >
> > > > https://git.openembedded.org/meta-openembedded-contrib/commit/?h=yoe/mut&id=3ab0af60e8512cbb718e8a0112d9c9aa98c620a4
> > >
> > > The cmake change makes python visible from HOSTTOOLS. That isn't
> > > wrong
> > > as such, it is a change in behaviour. If recipes don't set python
> > > enabled/disabled explicitly, there is some fallout from that. I'd
> > > say
> > > that is a bug in those recipes which aren't giving explicit
> > > configuration to dependencies though.
> > >
> >
> > So now the behavior is interesting. It works fine if I build for a
> > machine where host-arch = target-arch
> > so if someone is building for x86_64 then they dont see the build
> > failures. IMO thats worse.
>
> That is a general problem with cross compiling and applies to any
> automatically detected dependency from the host. Python is in an odd
> place here since we do need it on the host to run bitbake and there are
> places where we need that exposed to code (e.g. they have python
> scripts), there are places where we don't want it exposed as host
> contamination.
>
> Have you looked at the buildhistory logs to see what changed in meta-oe
> as that would likely highlight the changes.

well nothing has changed in meta-oe as such to cause this issue. Since
now we expand where
all cmake can look for development headers and libraries, it happily
finds then it native sysroot
earlier it was guarded from looking into native sysroot. So its cmake
not differentiating if its mixing
two sysroots now, which is not new for cmake, it has to be given certain paths.

So yes when headers and libraries for python are set to point to
target sysroot then you aid it to look
into right sysroot but I think it might become a pain point in future
where someone will test it on x86_64
and think it builds and works where as it will be implicitly wrong. We
can fix what we know but that is
not all.

>
> Cheers,
>
> Richard
>
>
>
>
>
>

Re: new meta-oe failures from master-next

[Adrian Bunk]

On Thu, Oct 24, 2019 at 09:15:01AM +0100, richard.purdie@linuxfoundation.org wrote:
> 
> The cmake change makes python visible from HOSTTOOLS. That isn't wrong
> as such, it is a change in behaviour. If recipes don't set python
> enabled/disabled explicitly, there is some fallout from that. I'd say
> that is a bug in those recipes which aren't giving explicit
> configuration to dependencies though.

The problem is not about enable/disable, it is more about not explicitly 
selecting the python version.

For the affected libiio the python2 -> python3 change of the python 
bindings last week [1] was basically:

-inherit cmake pythonnative systemd
+inherit cmake python3native systemd

This is very fragile, and I would consider it expected that either the 
old or the new case will break in some way when both python2 and python3
are available during the build.

> Cheers,
> 
> RIchard

cu
Adrian

[1] http://git.openembedded.org/meta-openembedded/commit/?id=6af155513506f6a0a3a44b66fb48897f22fb2c2f

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: new meta-oe failures from master-next

[richard.purdie]

On Thu, 2019-10-24 at 09:34 +0100, Khem Raj wrote:
> On Thu, Oct 24, 2019 at 9:25 AM <richard.purdie@linuxfoundation.org>
> wrote:
> > On Thu, 2019-10-24 at 09:20 +0100, Khem Raj wrote:
> > > On Thu, Oct 24, 2019 at 9:15 AM <
> > > richard.purdie@linuxfoundation.org>
> > > wrote:
> > > > On Wed, 2019-10-23 at 23:52 +0100, Khem Raj wrote:
> > > > > On Wed, Oct 23, 2019 at 8:31 PM Richard Purdie
> > > > > <richard.purdie@linuxfoundation.org> wrote:
> > > > > > On Wed, 2019-10-23 at 16:29 +0100, Ross Burton wrote:
> > > > > > > On 23/10/2019 14:03, Khem Raj wrote:
> > > > > > > > Hi Richard
> > > > > > > > 
> > > > > > > > New master-next I see these fails
> > > > > > > > 
> > > > > > > > https://errors.yoctoproject.org/Errors/Build/91645/
> > > > > > > > 
> > > > > > > > First two are important one's we already know why
> > > > > > > > ti-display-sharing-fw fails to fetch
> > > > > > > 
> > > > > > > ERROR: QA Issue: libiio: Files/directories were installed
> > > > > > > but
> > > > > > > not
> > > > > > > shipped in any package:
> > > > > > >    /usr/lib/python2.7
> > > > > > >    /usr/lib/python2.7/site-packages
> > > > > > >    /usr/lib/python2.7/site-packages/libiio-0.18-
> > > > > > > py2.7.egg-
> > > > > > > info
> > > > > > >    /usr/lib/python2.7/site-packages/iio.pyc
> > > > > > >    /usr/lib/python2.7/site-packages/iio.py
> > > > > > > 
> > > > > > > That may have been triggered by some change in oe-core
> > > > > > > but
> > > > > > > that's
> > > > > > > definitely an issue with libiio.
> > > > > > 
> > > > > > Its from this cmake change:
> > > > > > 
> > > > > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=dbba06d9153ae588c9496b40e5db18b3602bfad1
> > > > > > 
> > > > > 
> > > > > I think its causing more failures, where suddenly, the
> > > > > recipes
> > > > > are
> > > > > now
> > > > > resorting to native python, is this patch correct I wonder ?
> > > > > 
> > > > > see
> > > > > https://git.openembedded.org/meta-openembedded-contrib/commit/?h=yoe/mut&id=5d02e814bc7d30f925b2eeec85eb053cec516f26
> > > > > 
> > > > > https://git.openembedded.org/meta-openembedded-contrib/commit/?h=yoe/mut&id=3ab0af60e8512cbb718e8a0112d9c9aa98c620a4
> > > > 
> > > > The cmake change makes python visible from HOSTTOOLS. That
> > > > isn't
> > > > wrong
> > > > as such, it is a change in behaviour. If recipes don't set
> > > > python
> > > > enabled/disabled explicitly, there is some fallout from that.
> > > > I'd
> > > > say
> > > > that is a bug in those recipes which aren't giving explicit
> > > > configuration to dependencies though.
> > > > 
> > > 
> > > So now the behavior is interesting. It works fine if I build for
> > > a
> > > machine where host-arch = target-arch
> > > so if someone is building for x86_64 then they dont see the build
> > > failures. IMO thats worse.
> > 
> > That is a general problem with cross compiling and applies to any
> > automatically detected dependency from the host. Python is in an
> > odd
> > place here since we do need it on the host to run bitbake and there
> > are
> > places where we need that exposed to code (e.g. they have python
> > scripts), there are places where we don't want it exposed as host
> > contamination.
> > 
> > Have you looked at the buildhistory logs to see what changed in
> > meta-oe
> > as that would likely highlight the changes.
> 
> well nothing has changed in meta-oe as such to cause this issue.
> Since now we expand where all cmake can look for development headers
> and libraries, it happily finds then it native sysroot earlier it was
> guarded from looking into native sysroot. So its cmake not
> differentiating if its mixing two sysroots now, which is not new for
> cmake, it has to be given certain paths.

No, cmake has always looked in the native sysroot. What the patch
changes is whether HOSTTOOLS are seen or not. The exact change is:

-set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE} ${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH} ${EXTERNAL_TOOLCHAIN})
+set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE} ${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH} ${EXTERNAL_TOOLCHAIN} ${HOSTTOOLS_DIR})

It already looks at native, cross and external tools, just not
HOSTTOOLS_DIR. This does look very much like an omission and would
break ASSUME_PROVIDED usage for example so it does fix a real world
issue I can imagine people running into.

> So yes when headers and libraries for python are set to point to
> target sysroot then you aid it to look
> into right sysroot but I think it might become a pain point in future
> where someone will test it on x86_64
> and think it builds and works where as it will be implicitly wrong.
> We can fix what we know but that is not all.

Agreed, but that is a problem we already have and is why we do ask
people test another arch. Its also why we ask people to list
dependencies and explicitly enable or disable them.

I do understand what you mean but I don't think it changes much as its
the same general problem the project has always faced and continues to
face.

If you want to propose the patch be reverted, you can however I'm not
really seeing a case for it myself.

Cheers,

Richard

Re: new meta-oe failures from master-next

[richard.purdie]

On Thu, 2019-10-24 at 12:34 +0300, Adrian Bunk wrote:
> On Thu, Oct 24, 2019 at 09:15:01AM +0100, 
> richard.purdie@linuxfoundation.org wrote:
> > The cmake change makes python visible from HOSTTOOLS. That isn't
> > wrong
> > as such, it is a change in behaviour. If recipes don't set python
> > enabled/disabled explicitly, there is some fallout from that. I'd
> > say
> > that is a bug in those recipes which aren't giving explicit
> > configuration to dependencies though.
> 
> The problem is not about enable/disable, it is more about not
> explicitly selecting the python version.
> 
> For the affected libiio the python2 -> python3 change of the python 
> bindings last week [1] was basically:
> 
> -inherit cmake pythonnative systemd
> +inherit cmake python3native systemd
> 
> This is very fragile, and I would consider it expected that either
> the old or the new case will break in some way when both python2 and
> python3 are available during the build.

That isn't the complete story. We changed the search path in master:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=dbba06d9153ae588c9496b40e5db18b3602bfad1

which then meant libiio found the wrong python as "python" in PATH
became py2.

As you say, it doesn't specifically declare which python it wants so
things changed.

I admit I'd misread the recipes, both for the upgrade and
enable/disable, I should know better. I think this kind of proves my
point about trying to push more work onto people who are already
overworked though.

The key questions:

* is that OE-Core change is correct?
* should this recipe be more explicit about its configuration?
* are there other places this problem has been introduced?
* who is responsible for fixing any further issues?

While we remember we should document this for the 3.1 migration guide
at the very least.

Cheers,

Richard