Yocto Project support for Numeric/Scientific Python

Yocto Project support for Numeric/Scientific Python

[Smith, Virgil (US)]

[-- Attachment #1: Type: text/plain, Size: 1362 bytes --]

Is there a current or relatively recent recipe for SciPy and related libraries?

Further and more importantly, is having a maintainer for (recipes for) those libraries a priority for the active members of the Project?
(i.e. does interest rise above the general welcoming of participants to periodically asking "Hey has anyone put out a call to fill this slot?" if/when the slot is vacant).

BTW: If this is the wrong list for this query, please let me know.


Why?  We are trying to gauge community interest before making long term plans.
We would like to know if this horse is at all likely to have healthcare before betting on it (without sacrificing other patients to obtain the proper veterinary degree and keep up practice to treat it ourselves).


NOTE:  I see from the RRS emails that Derek Straka is currently maintaining the python-numpy recipe.  THANK YOU!


________________________________

Notice to recipient: This email is meant for only the intended recipient of the transmission, and may be a communication privileged by law, subject to export control restrictions or that otherwise contains proprietary information. If you receive this email by mistake, please notify us immediately by replying to this message and then destroy it and do not review, disclose, copy or distribute it. Thank you in advance for your cooperation.

[-- Attachment #2: Type: text/html, Size: 3774 bytes --]

Re: Yocto Project support for Numeric/Scientific Python

[Randy MacLeod]

On 1/23/19 2:54 PM, Smith, Virgil (US) wrote:
> Is there a current or relatively recent recipe for SciPy and related 
> libraries?

People have worked on it at least once before but found some problems
with blas and atlas:
 
https://lists.yoctoproject.org/pipermail/yocto/2018-March/thread.html#40348

I'd say that there is interest.
I CCed Peter who started one of the threads
and BCCed 5 other people who seemed to be interested
since I didn't want to drag them all into the thread.


> 
> Further and more importantly, is having a maintainer for (recipes for) 
> those libraries a priority for the active members of the Project?
> (i.e. does interest rise above the general welcoming of participants to 
> periodically asking “Hey has anyone put out a call to fill this slot?” 
> if/when the slot is vacant).

It's always nice to have a maintainer but community members sometimes
keep recipes up to date even if they aren't direct users.

> 
> BTW: If this is the wrong list for this query, please let me know.

It a reasonable list for general discussion.
If you get to a point where patches are being submitted,
it should probably go to another list such as:

> 
> Why?  We are trying to gauge community interest before making long term 
> plans.
> 
> We would like to know if this horse is at all likely to have healthcare 
> before betting on it (without sacrificing other patients to obtain the 
> proper veterinary degree and keep up practice to treat it ourselves).
heh.

Thanks!
../Randy

> 
> NOTE:  I see from the RRS emails that Derek Straka is currently 
> maintaining the python-numpy recipe.  THANK YOU!
> 
> 
> ------------------------------------------------------------------------
> 
> Notice to recipient: This email is meant for only the intended recipient 
> of the transmission, and may be a communication privileged by law, 
> subject to export control restrictions or that otherwise contains 
> proprietary information. If you receive this email by mistake, please 
> notify us immediately by replying to this message and then destroy it 
> and do not review, disclose, copy or distribute it. Thank you in advance 
> for your cooperation.
> 


-- 
# Randy MacLeod
# Wind River Linux

Re: Yocto Project support for Numeric/Scientific Python

[Philip Balister]

I care :)

On 01/23/2019 04:28 PM, Randy MacLeod wrote:
> On 1/23/19 2:54 PM, Smith, Virgil (US) wrote:
>> Is there a current or relatively recent recipe for SciPy and related
>> libraries?
> 
> People have worked on it at least once before but found some problems
> with blas and atlas:
> 
> https://lists.yoctoproject.org/pipermail/yocto/2018-March/thread.html#40348
> 
> I'd say that there is interest.
> I CCed Peter who started one of the threads
> and BCCed 5 other people who seemed to be interested
> since I didn't want to drag them all into the thread.
> 
> 
>>
>> Further and more importantly, is having a maintainer for (recipes for)
>> those libraries a priority for the active members of the Project?
>> (i.e. does interest rise above the general welcoming of participants
>> to periodically asking “Hey has anyone put out a call to fill this
>> slot?” if/when the slot is vacant).
> 
> It's always nice to have a maintainer but community members sometimes
> keep recipes up to date even if they aren't direct users.
> 
>>
>> BTW: If this is the wrong list for this query, please let me know.
> 
> It a reasonable list for general discussion.
> If you get to a point where patches are being submitted,
> it should probably go to another list such as:
> 
>>
>> Why?  We are trying to gauge community interest before making long
>> term plans.
>>
>> We would like to know if this horse is at all likely to have
>> healthcare before betting on it (without sacrificing other patients to
>> obtain the proper veterinary degree and keep up practice to treat it
>> ourselves).
> heh.
> 
> Thanks!
> ../Randy
> 
>>
>> NOTE:  I see from the RRS emails that Derek Straka is currently
>> maintaining the python-numpy recipe.  THANK YOU!
>>
>>
>> ------------------------------------------------------------------------
>>
>> Notice to recipient: This email is meant for only the intended
>> recipient of the transmission, and may be a communication privileged
>> by law, subject to export control restrictions or that otherwise
>> contains proprietary information. If you receive this email by
>> mistake, please notify us immediately by replying to this message and
>> then destroy it and do not review, disclose, copy or distribute it.
>> Thank you in advance for your cooperation.
>>
> 
> 

Re: Yocto Project support for Numeric/Scientific Python

[Randy MacLeod]

On 1/23/19 4:28 PM, Randy MacLeod wrote:
> It a reasonable list for general discussion.
> If you get to a point where patches are being submitted,
> it should probably go to another list such as:
oops... such as the meta-openembedded list:
   openembedded-devel@lists.openembedded.org
or whatever list the package(s) land in.

-- 
# Randy MacLeod
# Wind River Linux

Re: Yocto Project support for Numeric/Scientific Python

[Mike Looijmans]

+1

Got lapack to compile, but no such luck with any "blas" package (like 
openblas). And that's a requirement for octave, which was what I was aiming at.

I'll share some recipes, tomorrow or so (today is stuffed with other work).


On 23-01-19 22:39, Philip Balister wrote:
> I care :)
> 
> On 01/23/2019 04:28 PM, Randy MacLeod wrote:
>> On 1/23/19 2:54 PM, Smith, Virgil (US) wrote:
>>> Is there a current or relatively recent recipe for SciPy and related
>>> libraries?
>>
>> People have worked on it at least once before but found some problems
>> with blas and atlas:
>>
>> https://lists.yoctoproject.org/pipermail/yocto/2018-March/thread.html#40348
>>
>> I'd say that there is interest.
>> I CCed Peter who started one of the threads
>> and BCCed 5 other people who seemed to be interested
>> since I didn't want to drag them all into the thread.
>>
>>
>>>
>>> Further and more importantly, is having a maintainer for (recipes for)
>>> those libraries a priority for the active members of the Project?
>>> (i.e. does interest rise above the general welcoming of participants
>>> to periodically asking “Hey has anyone put out a call to fill this
>>> slot?” if/when the slot is vacant).
>>
>> It's always nice to have a maintainer but community members sometimes
>> keep recipes up to date even if they aren't direct users.
>>
>>>
>>> BTW: If this is the wrong list for this query, please let me know.
>>
>> It a reasonable list for general discussion.
>> If you get to a point where patches are being submitted,
>> it should probably go to another list such as:
>>
>>>
>>> Why?  We are trying to gauge community interest before making long
>>> term plans.
>>>
>>> We would like to know if this horse is at all likely to have
>>> healthcare before betting on it (without sacrificing other patients to
>>> obtain the proper veterinary degree and keep up practice to treat it
>>> ourselves).
>> heh.
>>
>> Thanks!
>> ../Randy
>>
>>>
>>> NOTE:  I see from the RRS emails that Derek Straka is currently
>>> maintaining the python-numpy recipe.  THANK YOU!
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> Notice to recipient: This email is meant for only the intended
>>> recipient of the transmission, and may be a communication privileged
>>> by law, subject to export control restrictions or that otherwise
>>> contains proprietary information. If you receive this email by
>>> mistake, please notify us immediately by replying to this message and
>>> then destroy it and do not review, disclose, copy or distribute it.
>>> Thank you in advance for your cooperation.
>>>
>>
>>

Re: Yocto Project support for Numeric/Scientific Python

[Philip Balister]

Sounds like we need a layer for packages that needs fortran enabled and
collect out work there.

Philip

On 01/24/2019 05:31 AM, Mike Looijmans wrote:
> +1
> 
> Got lapack to compile, but no such luck with any "blas" package (like 
> openblas). And that's a requirement for octave, which was what I was aiming at.
> 
> I'll share some recipes, tomorrow or so (today is stuffed with other work).
> 
> 
> On 23-01-19 22:39, Philip Balister wrote:
>> I care :)
>>
>> On 01/23/2019 04:28 PM, Randy MacLeod wrote:
>>> On 1/23/19 2:54 PM, Smith, Virgil (US) wrote:
>>>> Is there a current or relatively recent recipe for SciPy and related
>>>> libraries?
>>>
>>> People have worked on it at least once before but found some problems
>>> with blas and atlas:
>>>
>>> https://lists.yoctoproject.org/pipermail/yocto/2018-March/thread.html#40348
>>>
>>> I'd say that there is interest.
>>> I CCed Peter who started one of the threads
>>> and BCCed 5 other people who seemed to be interested
>>> since I didn't want to drag them all into the thread.
>>>
>>>
>>>>
>>>> Further and more importantly, is having a maintainer for (recipes for)
>>>> those libraries a priority for the active members of the Project?
>>>> (i.e. does interest rise above the general welcoming of participants
>>>> to periodically asking “Hey has anyone put out a call to fill this
>>>> slot?” if/when the slot is vacant).
>>>
>>> It's always nice to have a maintainer but community members sometimes
>>> keep recipes up to date even if they aren't direct users.
>>>
>>>>
>>>> BTW: If this is the wrong list for this query, please let me know.
>>>
>>> It a reasonable list for general discussion.
>>> If you get to a point where patches are being submitted,
>>> it should probably go to another list such as:
>>>
>>>>
>>>> Why?  We are trying to gauge community interest before making long
>>>> term plans.
>>>>
>>>> We would like to know if this horse is at all likely to have
>>>> healthcare before betting on it (without sacrificing other patients to
>>>> obtain the proper veterinary degree and keep up practice to treat it
>>>> ourselves).
>>> heh.
>>>
>>> Thanks!
>>> ../Randy
>>>
>>>>
>>>> NOTE:  I see from the RRS emails that Derek Straka is currently
>>>> maintaining the python-numpy recipe.  THANK YOU!
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> Notice to recipient: This email is meant for only the intended
>>>> recipient of the transmission, and may be a communication privileged
>>>> by law, subject to export control restrictions or that otherwise
>>>> contains proprietary information. If you receive this email by
>>>> mistake, please notify us immediately by replying to this message and
>>>> then destroy it and do not review, disclose, copy or distribute it.
>>>> Thank you in advance for your cooperation.
>>>>
>>>
>>>
> 

Re: Yocto Project support for Numeric/Scientific Python

[Mike Looijmans]

Just attempting to revive this dead horse again...

Anyone made any proress here?

Since cross-compiling turned out to be really really painful, I tried if 
compiling on the board would be an option. No such luck, apparently the 
Fortran compiler isn't being crosscompiled either.


On 26-01-19 20:01, Philip Balister wrote:
> Sounds like we need a layer for packages that needs fortran enabled and
> collect out work there.
> 
> Philip
> 
> On 01/24/2019 05:31 AM, Mike Looijmans wrote:
>> +1
>>
>> Got lapack to compile, but no such luck with any "blas" package (like
>> openblas). And that's a requirement for octave, which was what I was aiming at.
>>
>> I'll share some recipes, tomorrow or so (today is stuffed with other work).
>>
>>
>> On 23-01-19 22:39, Philip Balister wrote:
>>> I care :)
>>>
>>> On 01/23/2019 04:28 PM, Randy MacLeod wrote:
>>>> On 1/23/19 2:54 PM, Smith, Virgil (US) wrote:
>>>>> Is there a current or relatively recent recipe for SciPy and related
>>>>> libraries?
>>>>
>>>> People have worked on it at least once before but found some problems
>>>> with blas and atlas:
>>>>
>>>> https://lists.yoctoproject.org/pipermail/yocto/2018-March/thread.html#40348
>>>>
>>>> I'd say that there is interest.
>>>> I CCed Peter who started one of the threads
>>>> and BCCed 5 other people who seemed to be interested
>>>> since I didn't want to drag them all into the thread.
>>>>
>>>>
>>>>>
>>>>> Further and more importantly, is having a maintainer for (recipes for)
>>>>> those libraries a priority for the active members of the Project?
>>>>> (i.e. does interest rise above the general welcoming of participants
>>>>> to periodically asking “Hey has anyone put out a call to fill this
>>>>> slot?” if/when the slot is vacant).
>>>>
>>>> It's always nice to have a maintainer but community members sometimes
>>>> keep recipes up to date even if they aren't direct users.
>>>>
>>>>>
>>>>> BTW: If this is the wrong list for this query, please let me know.
>>>>
>>>> It a reasonable list for general discussion.
>>>> If you get to a point where patches are being submitted,
>>>> it should probably go to another list such as:
>>>>
>>>>>
>>>>> Why?  We are trying to gauge community interest before making long
>>>>> term plans.
>>>>>
>>>>> We would like to know if this horse is at all likely to have
>>>>> healthcare before betting on it (without sacrificing other patients to
>>>>> obtain the proper veterinary degree and keep up practice to treat it
>>>>> ourselves).
>>>> heh.
>>>>
>>>> Thanks!
>>>> ../Randy
>>>>
>>>>>
>>>>> NOTE:  I see from the RRS emails that Derek Straka is currently
>>>>> maintaining the python-numpy recipe.  THANK YOU!
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>> Notice to recipient: This email is meant for only the intended
>>>>> recipient of the transmission, and may be a communication privileged
>>>>> by law, subject to export control restrictions or that otherwise
>>>>> contains proprietary information. If you receive this email by
>>>>> mistake, please notify us immediately by replying to this message and
>>>>> then destroy it and do not review, disclose, copy or distribute it.
>>>>> Thank you in advance for your cooperation.
>>>>>
>>>>
>>>>
>>

Re: [EXTERNAL] Re: Yocto Project support for Numeric/Scientific Python

[Smith, Virgil (US)]

Mike Looijmans Wrote:
> Just attempting to revive this dead horse again...
>
> Anyone made any proress here?
>
> Since cross-compiling turned out to be really really painful, I tried if
> compiling on the board would be an option. No such luck, apparently
> the Fortran compiler isn't being crosscompiled either.

I had to do the following in my last attempt (working with the Sumo release branch).

Add the following to IMAGE_INSTALL:
gfortran  gfortran-symlinks  libgfortran-dev

AND add the following to local.conf
FORTRAN_forcevariable = ",fortran"

The latter is to adjust the build of gcc to include Fortran support.
In prior Yocto releases I did not have to use the _forcevariable construct, but I don't recall why I used that form for Sumo.

________________________________

Notice to recipient: This email is meant for only the intended recipient of the transmission, and may be a communication privileged by law, subject to export control restrictions or that otherwise contains proprietary information. If you receive this email by mistake, please notify us immediately by replying to this message and then destroy it and do not review, disclose, copy or distribute it. Thank you in advance for your cooperation.

Re: Yocto Project support for Numeric/Scientific Python

[Smith, Virgil (US)]

Discussing only the question of making a SciPy recipe today:

> > Is there a current or relatively recent recipe for SciPy and related libraries?
>
> People have worked on it at least once before but found some problems with blas and atlas:
> https://lists.yoctoproject.org/pipermail/yocto/2018-March/thread.html#40348
> # Randy MacLeod
> Wind River Linux

Thanks Randy, my own brief foray landed on the following thread as the most promising lead:
https://lists.yoctoproject.org/pipermail/yocto/2018-March/040450.html

I didn't have any success with those pieces, but the telling point in that discussion are the patches to numpy (numpy.distutils).  When you dig into SciPy's installation instructions and therefore its setup.py file, you find that it is using code packaged into the numpy library as effectively its own configure/build toolchain.  It is full of Linux vs Windows vs MacOS distinctions and path probing for recognized compilers/tools.  Patching that into something that can be tuned for cross compiling was beyond my allowances for an "investigative effort".

I did manage to *compile* OpenBlas just by blending its standard build instructions (pure make based) with oe_runmake, a libgfortran DEPENDS/RDEPENDS, (FORTRAN_forcevariable = ",fortran" in local.conf), and some regex mapping from TUNE_FEATURES to the defines used by OpenBlas.

If I ever find out it actually worked (once), I'll post it in case someone has bandwidth to add
checks for mapping other MACHINE/TUNE... values to OpenBlas "TargetList.txt" entries, testing that it works in newer Yocto releases (I'm using Sumo), and add update-alternatives integration.



________________________________

Notice to recipient: This email is meant for only the intended recipient of the transmission, and may be a communication privileged by law, subject to export control restrictions or that otherwise contains proprietary information. If you receive this email by mistake, please notify us immediately by replying to this message and then destroy it and do not review, disclose, copy or distribute it. Thank you in advance for your cooperation.