Installing Python app from source

Installing Python app from source

[Brian Zambrano]

[-- Attachment #1: Type: text/plain, Size: 1732 bytes --]

I'm trying to install my own Python application from source, and having a
hard time finding resources on how to accomplish it. I'm new to this
ecosystem and am probably missing something fundamental.

My recipe is structured like this...and my build is configure such that it
does actually find it.

├── firmware-1.0
│   ├── saunders_backend
│   └── saunders_frontend
└── firmware_1.0.bb

My application code which is for now just a "hello world" flask
application, is in saunders_backend. The setup.py file in there works as
expected when I use it in on my host machine.

├── firmware-1.0
│   ├── saunders_backend/
│   │   ├── bin/
│   │   │   └── server
│   │   ├── MANIFEST.in
│   │   ├── requirements.txt
│   │   ├── saunders_apollo/
│   │   │   ├── config.py
│   │   │   ├── __init__.py
│   │   │   ├── models.py
│   │   │   ├── static/
│   │   │   ├── templates/
│   │   │   └── views.py
│   │   └── setup.py
│   └── saunders_frontend/
└── firmware_1.0.bb

I've been hacking on the .bb files to get it to install this into the
resulting image. I do have IMAGE_INSTALL_append = " firmware" in the
top-level local.conf.

My primary confusion is that I don't know what I should be using for the
python application. do_install, do_install_append? What should SRC_UIR be?
In effect is that I want the exact same behavior of "cd saunders_backend &&
pip install ." My setup.py file has install_requires with a list of all
dependencies, which now is merely Flask.

Pointers would be appreciated.
BZ

[-- Attachment #2: Type: text/html, Size: 2611 bytes --]

Re: Installing Python app from source

[Shantanoo Desai]

[-- Attachment #1: Type: text/plain, Size: 2776 bytes --]

Hi Brian,

 

What you need in the .bb is a runtime-dependency. RDEPENDS_${PN} += “ ${PYTHON_PN}-flask ${PYTHON_PN}-jinja ${PYTHON_PN}-markdown” and so on. This will add these modules in the root-filesystem and will be available on the target. 

 

If you need some basic help with a python recipe, there is a git repository YoctoCookbook https://github.com/yoctocookbook/meta-custom/tree/master/recipes-python/python-helloworld to get you some idea.

 

Please not I am in no ways a Yocto expert on recipe creations but for the past few weeks this is what I have been able to discover. Due to lack of examples on custom recipes in python this is the best I can help you with.

 

 

Best Regards,

 

Shan

 

 

 

Von: yocto-bounces@yoctoproject.org <yocto-bounces@yoctoproject.org> Im Auftrag von Brian Zambrano
Gesendet: Donnerstag, 4. Oktober 2018 00:52
An: yocto@yoctoproject.org
Betreff: [yocto] Installing Python app from source

 

I'm trying to install my own Python application from source, and having a hard time finding resources on how to accomplish it. I'm new to this ecosystem and am probably missing something fundamental.

My recipe is structured like this...and my build is configure such that it does actually find it.

├── firmware-1.0
│   ├── saunders_backend
│   └── saunders_frontend

└── firmware_1.0.bb <http://firmware_1.0.bb> 

My application code which is for now just a "hello world" flask application, is in saunders_backend. The setup.py file in there works as expected when I use it in on my host machine.

├── firmware-1.0
│   ├── saunders_backend/
│   │   ├── bin/
│   │   │   └── server
│   │   ├── MANIFEST.in
│   │   ├── requirements.txt
│   │   ├── saunders_apollo/
│   │   │   ├── config.py
│   │   │   ├── __init__.py
│   │   │   ├── models.py
│   │   │   ├── static/
│   │   │   ├── templates/
│   │   │   └── views.py
│   │   └── setup.py

│   └── saunders_frontend/
└── firmware_1.0.bb <http://firmware_1.0.bb> 


I've been hacking on the .bb files to get it to install this into the resulting image. I do have IMAGE_INSTALL_append = " firmware" in the top-level local.conf.

 

My primary confusion is that I don't know what I should be using for the python application. do_install, do_install_append? What should SRC_UIR be? In effect is that I want the exact same behavior of "cd saunders_backend && pip install ." My setup.py file has install_requires with a list of all dependencies, which now is merely Flask.

 

Pointers would be appreciated.

BZ


[-- Attachment #2: Type: text/html, Size: 8336 bytes --]

Re: Installing Python app from source

[Brian Zambrano]

[-- Attachment #1: Type: text/plain, Size: 3502 bytes --]

Thanks very much for this Shan. I have that book and was working through it
yesterday, as a matter of fact.

That part which is confusing to me in that recipe is that the SRC_URI
points to every single file, there is a "inherit setuptools" line and also
a do_install_append. I'm not sure how each of these are working together to
get the application installed. I believe that the "inherit setuptools" will
automatically run "python setup.py install", but when then is the source
code manually copied over?

BZ

On Thu, Oct 4, 2018 at 2:13 AM Shantanoo Desai <desai@uni-bremen.de> wrote:

> Hi Brian,
>
>
>
> What you need in the .bb is a runtime-dependency. RDEPENDS_${PN} += “
> ${PYTHON_PN}-flask ${PYTHON_PN}-jinja ${PYTHON_PN}-markdown” and so on.
> This will add these modules in the root-filesystem and will be available on
> the target.
>
>
>
> If you need some basic help with a python recipe, there is a git
> repository YoctoCookbook
> https://github.com/yoctocookbook/meta-custom/tree/master/recipes-python/python-helloworld
> to get you some idea.
>
>
>
> Please not I am in no ways a Yocto expert on recipe creations but for the
> past few weeks this is what I have been able to discover. Due to lack of
> examples on custom recipes in python this is the best I can help you with.
>
>
>
>
>
> Best Regards,
>
>
>
> Shan
>
>
>
>
>
>
>
> *Von:* yocto-bounces@yoctoproject.org <yocto-bounces@yoctoproject.org> *Im
> Auft**rag von *Brian Zambrano
> *Gesendet:* Donnerstag, 4. Oktober 2018 00:52
> *An:* yocto@yoctoproject.org
> *Betreff:* [yocto] Installing Python app from source
>
>
>
> I'm trying to install my own Python application from source, and having a
> hard time finding resources on how to accomplish it. I'm new to this
> ecosystem and am probably missing something fundamental.
>
> My recipe is structured like this...and my build is configure such that it
> does actually find it.
>
> ├── firmware-1.0
> │   ├── saunders_backend
> │   └── saunders_frontend
>
> └── firmware_1.0.bb
>
> My application code which is for now just a "hello world" flask
> application, is in saunders_backend. The setup.py file in there works as
> expected when I use it in on my host machine.
>
> ├── firmware-1.0
> │   ├── saunders_backend/
> │   │   ├── bin/
> │   │   │   └── server
> │   │   ├── MANIFEST.in
> │   │   ├── requirements.txt
> │   │   ├── saunders_apollo/
> │   │   │   ├── config.py
> │   │   │   ├── __init__.py
> │   │   │   ├── models.py
> │   │   │   ├── static/
> │   │   │   ├── templates/
> │   │   │   └── views.py
> │   │   └── setup.py
>
> │   └── saunders_frontend/
> └── firmware_1.0.bb
>
>
> I've been hacking on the .bb files to get it to install this into the
> resulting image. I do have IMAGE_INSTALL_append = " firmware" in the
> top-level local.conf.
>
>
>
> My primary confusion is that I don't know what I should be using for the
> python application. do_install, do_install_append? What should SRC_UIR be?
> In effect is that I want the exact same behavior of "cd saunders_backend
> && pip install ." My setup.py file has install_requires with a list of
> all dependencies, which now is merely Flask.
>
>
>
> Pointers would be appreciated.
>
> BZ
>

[-- Attachment #2: Type: text/html, Size: 7882 bytes --]

Re: Installing Python app from source

[Shantanoo Desai]

[-- Attachment #1: Type: text/plain, Size: 4420 bytes --]

Hi Brian,

 

It is confusing, I agree. But I think it is essential because I assume your Target is not connected to the Internet. When you do pip install <package> normally the package if fetched from the internet and then installed. In this case your app is local so need to have those files first built natively and then need to be sent to the appropriate folders for python to access them which I guess what setuptools does. Hence you set all the files in the SRC_URI. 

 

Do_install_append is only when you have binaries which could be used as command line utilities. 


(But I could also be completely wrong, not a Yocto guru here)

 

Cheers

 

Shan

 

Von: Brian Zambrano <brianz@gmail.com> 
Gesendet: Donnerstag, 4. Oktober 2018 18:33
An: desai@uni-bremen.de
Cc: yocto@yoctoproject.org
Betreff: Re: [yocto] Installing Python app from source

 

Thanks very much for this Shan. I have that book and was working through it yesterday, as a matter of fact.

 

That part which is confusing to me in that recipe is that the SRC_URI points to every single file, there is a "inherit setuptools" line and also a do_install_append. I'm not sure how each of these are working together to get the application installed. I believe that the "inherit setuptools" will automatically run "python setup.py install", but when then is the source code manually copied over?

 

BZ

 

On Thu, Oct 4, 2018 at 2:13 AM Shantanoo Desai <desai@uni-bremen.de <mailto:desai@uni-bremen.de> > wrote:

Hi Brian,

 

What you need in the .bb is a runtime-dependency. RDEPENDS_${PN} += “ ${PYTHON_PN}-flask ${PYTHON_PN}-jinja ${PYTHON_PN}-markdown” and so on. This will add these modules in the root-filesystem and will be available on the target. 

 

If you need some basic help with a python recipe, there is a git repository YoctoCookbook https://github.com/yoctocookbook/meta-custom/tree/master/recipes-python/python-helloworld to get you some idea.

 

Please not I am in no ways a Yocto expert on recipe creations but for the past few weeks this is what I have been able to discover. Due to lack of examples on custom recipes in python this is the best I can help you with.

 

 

Best Regards,

 

Shan

 

 

 

Von: yocto-bounces@yoctoproject.org <mailto:yocto-bounces@yoctoproject.org>  <yocto-bounces@yoctoproject.org <mailto:yocto-bounces@yoctoproject.org> > Im Auftrag von Brian Zambrano
Gesendet: Donnerstag, 4. Oktober 2018 00:52
An: yocto@yoctoproject.org <mailto:yocto@yoctoproject.org> 
Betreff: [yocto] Installing Python app from source

 

I'm trying to install my own Python application from source, and having a hard time finding resources on how to accomplish it. I'm new to this ecosystem and am probably missing something fundamental.

My recipe is structured like this...and my build is configure such that it does actually find it.

├── firmware-1.0
│   ├── saunders_backend
│   └── saunders_frontend

└── firmware_1.0.bb <http://firmware_1.0.bb> 

My application code which is for now just a "hello world" flask application, is in saunders_backend. The setup.py file in there works as expected when I use it in on my host machine.

├── firmware-1.0
│   ├── saunders_backend/
│   │   ├── bin/
│   │   │   └── server
│   │   ├── MANIFEST.in
│   │   ├── requirements.txt
│   │   ├── saunders_apollo/
│   │   │   ├── config.py
│   │   │   ├── __init__.py
│   │   │   ├── models.py
│   │   │   ├── static/
│   │   │   ├── templates/
│   │   │   └── views.py
│   │   └── setup.py

│   └── saunders_frontend/
└── firmware_1.0.bb <http://firmware_1.0.bb> 


I've been hacking on the .bb files to get it to install this into the resulting image. I do have IMAGE_INSTALL_append = " firmware" in the top-level local.conf.

 

My primary confusion is that I don't know what I should be using for the python application. do_install, do_install_append? What should SRC_UIR be? In effect is that I want the exact same behavior of "cd saunders_backend && pip install ." My setup.py file has install_requires with a list of all dependencies, which now is merely Flask.

 

Pointers would be appreciated.

BZ


[-- Attachment #2: Type: text/html, Size: 13987 bytes --]

Re: Installing Python app from source

[Brian Zambrano]

[-- Attachment #1: Type: text/plain, Size: 5364 bytes --]

In a Python project with a setup.py file, you can actually do "pip install
." Of course you still need internet access for dependencies, which I have
during the build.  This is similar to running "python setup.py install" in
the same directory.

I've tried setting SRC_URI="saunders_backend" along with "import
setuptools".  That seems to run "python setup.py install" during the build
phase, however I get an error that setup.py cannot be found.

Also, my setup.py contains a section for scripts, which results on my
script being placed into the a bin directory on the host when installed
normally:

scripts=['bin/server'],

So, setup.py can do everything I need outside of bitbake/yocto...I just
can't figure out how to get it working during the bitbake build.

On Thu, Oct 4, 2018 at 11:08 AM Shantanoo Desai <desai@uni-bremen.de> wrote:

> Hi Brian,
>
>
>
> It is confusing, I agree. But I think it is essential because I assume
> your Target is not connected to the Internet. When you do pip install
> <package> normally the package if fetched from the internet and then
> installed. In this case your app is local so need to have those files first
> built natively and then need to be sent to the appropriate folders for
> python to access them which I guess what setuptools does. Hence you set all
> the files in the SRC_URI.
>
>
>
> Do_install_append is only when you have binaries which could be used as
> command line utilities.
>
>
> (But I could also be completely wrong, not a Yocto guru here)
>
>
>
> Cheers
>
>
>
> Shan
>
>
>
> *Von:* Brian Zambrano <brianz@gmail.com>
> *Gesendet:* Donnerstag, 4. Oktober 2018 18:33
> *An:* desai@uni-bremen.de
> *Cc:* yocto@yoctoproject.org
> *Betreff:* Re: [yocto] Installing Python app from source
>
>
>
> Thanks very much for this Shan. I have that book and was working through
> it yesterday, as a matter of fact.
>
>
>
> That part which is confusing to me in that recipe is that the SRC_URI
> points to every single file, there is a "inherit setuptools" line and
> also a do_install_append. I'm not sure how each of these are working
> together to get the application installed. I believe that the "inherit
> setuptools" will automatically run "python setup.py install", but when then
> is the source code manually copied over?
>
>
>
> BZ
>
>
>
> On Thu, Oct 4, 2018 at 2:13 AM Shantanoo Desai <desai@uni-bremen.de>
> wrote:
>
> Hi Brian,
>
>
>
> What you need in the .bb is a runtime-dependency. RDEPENDS_${PN} += “
> ${PYTHON_PN}-flask ${PYTHON_PN}-jinja ${PYTHON_PN}-markdown” and so on.
> This will add these modules in the root-filesystem and will be available on
> the target.
>
>
>
> If you need some basic help with a python recipe, there is a git
> repository YoctoCookbook
> https://github.com/yoctocookbook/meta-custom/tree/master/recipes-python/python-helloworld
> to get you some idea.
>
>
>
> Please not I am in no ways a Yocto expert on recipe creations but for the
> past few weeks this is what I have been able to discover. Due to lack of
> examples on custom recipes in python this is the best I can help you with.
>
>
>
>
>
> Best Regards,
>
>
>
> Shan
>
>
>
>
>
>
>
> *Von:* yocto-bounces@yoctoproject.org <yocto-bounces@yoctoproject.org> *Im
> Auft**rag von *Brian Zambrano
> *Gesendet:* Donnerstag, 4. Oktober 2018 00:52
> *An:* yocto@yoctoproject.org
> *Betreff:* [yocto] Installing Python app from source
>
>
>
> I'm trying to install my own Python application from source, and having a
> hard time finding resources on how to accomplish it. I'm new to this
> ecosystem and am probably missing something fundamental.
>
> My recipe is structured like this...and my build is configure such that it
> does actually find it.
>
> ├── firmware-1.0
> │   ├── saunders_backend
> │   └── saunders_frontend
>
> └── firmware_1.0.bb
>
> My application code which is for now just a "hello world" flask
> application, is in saunders_backend. The setup.py file in there works as
> expected when I use it in on my host machine.
>
> ├── firmware-1.0
> │   ├── saunders_backend/
> │   │   ├── bin/
> │   │   │   └── server
> │   │   ├── MANIFEST.in
> │   │   ├── requirements.txt
> │   │   ├── saunders_apollo/
> │   │   │   ├── config.py
> │   │   │   ├── __init__.py
> │   │   │   ├── models.py
> │   │   │   ├── static/
> │   │   │   ├── templates/
> │   │   │   └── views.py
> │   │   └── setup.py
>
> │   └── saunders_frontend/
> └── firmware_1.0.bb
>
>
> I've been hacking on the .bb files to get it to install this into the
> resulting image. I do have IMAGE_INSTALL_append = " firmware" in the
> top-level local.conf.
>
>
>
> My primary confusion is that I don't know what I should be using for the
> python application. do_install, do_install_append? What should SRC_UIR be?
> In effect is that I want the exact same behavior of "cd saunders_backend
> && pip install ." My setup.py file has install_requires with a list of
> all dependencies, which now is merely Flask.
>
>
>
> Pointers would be appreciated.
>
> BZ
>
>

[-- Attachment #2: Type: text/html, Size: 12356 bytes --]

Re: Installing Python app from source

[Outback Dingo]

On Fri, Oct 5, 2018 at 12:48 AM Brian Zambrano <brianz@gmail.com> wrote:
>
> In a Python project with a setup.py file, you can actually do "pip install ." Of course you still need internet access for dependencies, which I have during the build.  This is similar to running "python setup.py install" in the same directory.

curious where you found this innformatioon and what that recipe would
look like ?


>
> I've tried setting SRC_URI="saunders_backend" along with "import setuptools".  That seems to run "python setup.py install" during the build phase, however I get an error that setup.py cannot be found.
>
> Also, my setup.py contains a section for scripts, which results on my script being placed into the a bin directory on the host when installed normally:
>
> scripts=['bin/server'],
>
> So, setup.py can do everything I need outside of bitbake/yocto...I just can't figure out how to get it working during the bitbake build.
>
> On Thu, Oct 4, 2018 at 11:08 AM Shantanoo Desai <desai@uni-bremen.de> wrote:
>>
>> Hi Brian,
>>
>>
>>
>> It is confusing, I agree. But I think it is essential because I assume your Target is not connected to the Internet. When you do pip install <package> normally the package if fetched from the internet and then installed. In this case your app is local so need to have those files first built natively and then need to be sent to the appropriate folders for python to access them which I guess what setuptools does. Hence you set all the files in the SRC_URI.
>>
>>
>>
>> Do_install_append is only when you have binaries which could be used as command line utilities.
>>
>>
>> (But I could also be completely wrong, not a Yocto guru here)
>>
>>
>>
>> Cheers
>>
>>
>>
>> Shan
>>
>>
>>
>> Von: Brian Zambrano <brianz@gmail.com>
>> Gesendet: Donnerstag, 4. Oktober 2018 18:33
>> An: desai@uni-bremen.de
>> Cc: yocto@yoctoproject.org
>> Betreff: Re: [yocto] Installing Python app from source
>>
>>
>>
>> Thanks very much for this Shan. I have that book and was working through it yesterday, as a matter of fact.
>>
>>
>>
>> That part which is confusing to me in that recipe is that the SRC_URI points to every single file, there is a "inherit setuptools" line and also a do_install_append. I'm not sure how each of these are working together to get the application installed. I believe that the "inherit setuptools" will automatically run "python setup.py install", but when then is the source code manually copied over?
>>
>>
>>
>> BZ
>>
>>
>>
>> On Thu, Oct 4, 2018 at 2:13 AM Shantanoo Desai <desai@uni-bremen.de> wrote:
>>
>> Hi Brian,
>>
>>
>>
>> What you need in the .bb is a runtime-dependency. RDEPENDS_${PN} += “ ${PYTHON_PN}-flask ${PYTHON_PN}-jinja ${PYTHON_PN}-markdown” and so on. This will add these modules in the root-filesystem and will be available on the target.
>>
>>
>>
>> If you need some basic help with a python recipe, there is a git repository YoctoCookbook https://github.com/yoctocookbook/meta-custom/tree/master/recipes-python/python-helloworld to get you some idea.
>>
>>
>>
>> Please not I am in no ways a Yocto expert on recipe creations but for the past few weeks this is what I have been able to discover. Due to lack of examples on custom recipes in python this is the best I can help you with.
>>
>>
>>
>>
>>
>> Best Regards,
>>
>>
>>
>> Shan
>>
>>
>>
>>
>>
>>
>>
>> Von: yocto-bounces@yoctoproject.org <yocto-bounces@yoctoproject.org> Im Auftrag von Brian Zambrano
>> Gesendet: Donnerstag, 4. Oktober 2018 00:52
>> An: yocto@yoctoproject.org
>> Betreff: [yocto] Installing Python app from source
>>
>>
>>
>> I'm trying to install my own Python application from source, and having a hard time finding resources on how to accomplish it. I'm new to this ecosystem and am probably missing something fundamental.
>>
>> My recipe is structured like this...and my build is configure such that it does actually find it.
>>
>> ├── firmware-1.0
>> │   ├── saunders_backend
>> │   └── saunders_frontend
>>
>> └── firmware_1.0.bb
>>
>> My application code which is for now just a "hello world" flask application, is in saunders_backend. The setup.py file in there works as expected when I use it in on my host machine.
>>
>> ├── firmware-1.0
>> │   ├── saunders_backend/
>> │   │   ├── bin/
>> │   │   │   └── server
>> │   │   ├── MANIFEST.in
>> │   │   ├── requirements.txt
>> │   │   ├── saunders_apollo/
>> │   │   │   ├── config.py
>> │   │   │   ├── __init__.py
>> │   │   │   ├── models.py
>> │   │   │   ├── static/
>> │   │   │   ├── templates/
>> │   │   │   └── views.py
>> │   │   └── setup.py
>>
>> │   └── saunders_frontend/
>> └── firmware_1.0.bb
>>
>>
>> I've been hacking on the .bb files to get it to install this into the resulting image. I do have IMAGE_INSTALL_append = " firmware" in the top-level local.conf.
>>
>>
>>
>> My primary confusion is that I don't know what I should be using for the python application. do_install, do_install_append? What should SRC_UIR be? In effect is that I want the exact same behavior of "cd saunders_backend && pip install ." My setup.py file has install_requires with a list of all dependencies, which now is merely Flask.
>>
>>
>>
>> Pointers would be appreciated.
>>
>> BZ
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto