What can I share between projects?

What can I share between projects?

[Paul D. DeRocco]

If I'm doing multiple unrelated Yocto based projects, and they use the
same version of Yocto, and the same metadata (except for my own layers),
am I right in assuming that I can share everything in poky, downloads, and
sstate-cache, and I only need separate build directories? (I normally put
downloads and sstate-cache next to my build directory, rather than inside
it.)

-- 

Ciao,               Paul D. DeRocco
Paul                mailto:pderocco@ix.netcom.com

Re: What can I share between projects?

[Fabien Lahoudere]

On Thu, 2017-05-18 at 18:43 -0700, Paul D. DeRocco wrote:
> If I'm doing multiple unrelated Yocto based projects, and they use the
> same version of Yocto, and the same metadata (except for my own layers),
> am I right in assuming that I can share everything in poky, downloads, and
> sstate-cache, and I only need separate build directories? (I normally put
> downloads and sstate-cache next to my build directory, rather than inside
> it.)
> 

I do it successfully with poky and downloads.
I think the sstate-cache question is more complex and I cannot ensure it works fine in all case.

For example, What happen if you modify a software with different CFLAGS or build option?

> -- 
> 
> Ciao,               Paul D. DeRocco
> Paul                mailto:pderocco@ix.netcom.com
> 

Re: What can I share between projects?

[Mike Looijmans]

On 19-05-17 03:43, Paul D. DeRocco wrote:
> If I'm doing multiple unrelated Yocto based projects, and they use the
> same version of Yocto, and the same metadata (except for my own layers),
> am I right in assuming that I can share everything in poky, downloads, and
> sstate-cache, and I only need separate build directories? (I normally put
> downloads and sstate-cache next to my build directory, rather than inside
> it.)
>

You can share BOTH the downloads and sstate-cache. You can safely share 
sstate-cache between various versions of Yocto (OE) and distros, machines, 
etc., it was designed just for that.

Our build server keeps a single sstate-cache for about 30 projects, with 4 
versions of OE, 3 distros, and a dozen MACHINE configs. No problems encountered.


Kind regards,

Mike Looijmans
System Expert

TOPIC Products
Materiaalweg 4, NL-5681 RJ Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
E-mail: mike.looijmans@topicproducts.com
Website: www.topicproducts.com

Please consider the environment before printing this e-mail



Join our presentation at Electronics & Applications 2017:
FPGA for real-time data processing, subject “Hardware platform for industrial ultrasound steel plate Inspection” Topic Embedded Systems - Herman Kuster, 1st June 10 AM

Visit http://eabeurs.nl/author/612884/ for more information

Re: What can I share between projects?

[Mike Looijmans]

On 19-05-17 13:47, Mike Looijmans wrote:
> On 19-05-17 03:43, Paul D. DeRocco wrote:
>> If I'm doing multiple unrelated Yocto based projects, and they use the
>> same version of Yocto, and the same metadata (except for my own layers),
>> am I right in assuming that I can share everything in poky, downloads, 
>> and
>> sstate-cache, and I only need separate build directories? (I normally put
>> downloads and sstate-cache next to my build directory, rather than inside
>> it.)
>>
> 
> You can share BOTH the downloads and sstate-cache. You can safely share 
> sstate-cache between various versions of Yocto (OE) and distros, 
> machines, etc., it was designed just for that.
> 
> Our build server keeps a single sstate-cache for about 30 projects, with 
> 4 versions of OE, 3 distros, and a dozen MACHINE configs. No problems 
> encountered.

In addition to that, you can share the whole thing ("tmp") for projects 
that use the same local.conf and the same OE version. Building various 
images for about 20 different machines with over 4 different 
architectures (MIPS and ARM variants) in the same directory works just 
fine and drastically reduces build times since they can share about 
everything that's not unique to each machine.


Kind regards,

Mike Looijmans
System Expert

TOPIC Products
Materiaalweg 4, NL-5681 RJ Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
E-mail: mike.looijmans@topicproducts.com
Website: www.topicproducts.com

Please consider the environment before printing this e-mail



Join our presentation at Electronics & Applications 2017:
FPGA for real-time data processing, subject “Hardware platform for industrial ultrasound steel plate Inspection” Topic Embedded Systems - Herman Kuster, 1st June 10 AM

Visit http://eabeurs.nl/author/612884/ for more information

Re: What can I share between projects?

[Darcy Watkins]

[-- Attachment #1: Type: text/plain, Size: 3752 bytes --]

Hi,

What about different bblayer.conf stackups?

I was always under impression that as long as you use same layer stack, and the same local.conf (except perhaps switching the MACHINE per build variant), you can share the same tmp directory.  (For example, I could build same project for multiple I.MX6 machines, then for a Layerscape machine, and then for i586 machine).  But once the layer stack changes, you need a different build directory.  (For example some QorIQ SDKs used different layer stacks between ARM and PPC architectures).

And in our case, since we use 'repo' to manage checkouts, as soon as you use different Yocto version branches, you pretty much need a separate checkout, but could share common downloaded content.

Are you saying that the sstate cache is smart enough to handle exact same package for same arch but just built using different layer stack?  (Say one case builds package A with a bbappend, but the other uses a bbappend from a different layer, and say they are at same bbappend revision).

Thanks!



___

Regards,

Darcy

Darcy Watkins ::  Senior Staff Engineer, Firmware

SIERRA WIRELESS
Direct  +1 604 233 7989<tel:+1%20604%20233%207989>   ::  Fax  +1 604 231 1109<tel:+1%20604%20231%201109>  ::  Main  +1 604 231 1100<tel:+1%20604%20231%201100>
13811 Wireless Way  :: Richmond, BC Canada V6V 3A4<x-apple-data-detectors://3/1>
[M4]
dwatkins@sierrawireless.com<mailto:dwatkins@sierrawireless.com> :: www.sierrawireless.com<http://www.sierrawireless.com/> :: www.inmotiontechnology.com<http://www.inmotiontechnology.com/>


On May 20, 2017, at 2:54 AM, Mike Looijmans <mike.looijmans@topic.nl<mailto:mike.looijmans@topic.nl>> wrote:

On 19-05-17 13:47, Mike Looijmans wrote:
On 19-05-17 03:43, Paul D. DeRocco wrote:
If I'm doing multiple unrelated Yocto based projects, and they use the
same version of Yocto, and the same metadata (except for my own layers),
am I right in assuming that I can share everything in poky, downloads, and
sstate-cache, and I only need separate build directories? (I normally put
downloads and sstate-cache next to my build directory, rather than inside
it.)

You can share BOTH the downloads and sstate-cache. You can safely share sstate-cache between various versions of Yocto (OE) and distros, machines, etc., it was designed just for that.
Our build server keeps a single sstate-cache for about 30 projects, with 4 versions of OE, 3 distros, and a dozen MACHINE configs. No problems encountered.

In addition to that, you can share the whole thing ("tmp") for projects that use the same local.conf and the same OE version. Building various images for about 20 different machines with over 4 different architectures (MIPS and ARM variants) in the same directory works just fine and drastically reduces build times since they can share about everything that's not unique to each machine.


Kind regards,

Mike Looijmans
System Expert

TOPIC Products
Materiaalweg 4, NL-5681 RJ Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
E-mail: mike.looijmans@topicproducts.com<mailto:mike.looijmans@topicproducts.com>
Website: www.topicproducts.com<http://www.topicproducts.com>

Please consider the environment before printing this e-mail



Join our presentation at Electronics & Applications 2017:
FPGA for real-time data processing, subject “Hardware platform for industrial ultrasound steel plate Inspection” Topic Embedded Systems - Herman Kuster, 1st June 10 AM

Visit http://eabeurs.nl/author/612884/ for more information

--
_______________________________________________
yocto mailing list
yocto@yoctoproject.org<mailto:yocto@yoctoproject.org>
https://lists.yoctoproject.org/listinfo/yocto

[-- Attachment #2: Type: text/html, Size: 8432 bytes --]

Re: What can I share between projects?

[Khem Raj]

On Sat, May 20, 2017 at 8:16 AM, Darcy Watkins
<dwatkins@sierrawireless.com> wrote:
> Are you saying that the sstate cache is smart enough to handle exact same
> package for same arch but just built using different layer stack?  (Say one
> case builds package A with a bbappend, but the other uses a bbappend from a
> different layer, and say they are at same bbappend revision).

yes it should be, since it checksums the variables which are effective
during build of a given package.

Re: What can I share between projects?

[Darcy Watkins]

[-- Attachment #1: Type: text/plain, Size: 1572 bytes --]

Thanks!  That's a good clarification.

Now on other hand, I have experienced a few anomalies that are more at the final image build stage, especially when using RPM feed, and the PR service to auto bump them when rebuilt for dependencies.

The image build recipes must then not have visibility all the way back into the sstate cache, but only as far as the version decorations used in the RPM feed, right?


___

Regards,

Darcy

Darcy Watkins ::  Senior Staff Engineer, Firmware

SIERRA WIRELESS
Direct  +1 604 233 7989<tel:+1%20604%20233%207989>   ::  Fax  +1 604 231 1109<tel:+1%20604%20231%201109>  ::  Main  +1 604 231 1100<tel:+1%20604%20231%201100>
13811 Wireless Way  :: Richmond, BC Canada V6V 3A4<x-apple-data-detectors://3/1>
[M4]
dwatkins@sierrawireless.com<mailto:dwatkins@sierrawireless.com> :: www.sierrawireless.com<http://www.sierrawireless.com/> :: www.inmotiontechnology.com<http://www.inmotiontechnology.com/>


On May 20, 2017, at 8:54 AM, Khem Raj <raj.khem@gmail.com<mailto:raj.khem@gmail.com>> wrote:

On Sat, May 20, 2017 at 8:16 AM, Darcy Watkins
<dwatkins@sierrawireless.com<mailto:dwatkins@sierrawireless.com>> wrote:
Are you saying that the sstate cache is smart enough to handle exact same
package for same arch but just built using different layer stack?  (Say one
case builds package A with a bbappend, but the other uses a bbappend from a
different layer, and say they are at same bbappend revision).

yes it should be, since it checksums the variables which are effective
during build of a given package.

[-- Attachment #2: Type: text/html, Size: 4756 bytes --]

Re: What can I share between projects?

[Khem Raj]

On Sat, May 20, 2017 at 10:04 AM, Darcy Watkins
<dwatkins@sierrawireless.com> wrote:
> Thanks!  That's a good clarification.
>
> Now on other hand, I have experienced a few anomalies that are more at the
> final image build stage, especially when using RPM feed, and the PR service
> to auto bump them when rebuilt for dependencies.
>
> The image build recipes must then not have visibility all the way back into
> the sstate cache, but only as far as the version decorations used in the RPM
> feed, right?

They infact should. Since every image you are building follows exact
same process. It will recursively go down to package granularity and
PR is just one of inputs to determine rebuilding if its not
automatically set, if its set automatically then it will actually be
incremented for you

>
>
> ___
>
> Regards,
>
>
>
> Darcy
>
>
>
> Darcy Watkins ::  Senior Staff Engineer, Firmware
>
>
>
> SIERRA WIRELESS
>
> Direct  +1 604 233 7989   ::  Fax  +1 604 231 1109  ::  Main  +1 604 231
> 1100
>
> 13811 Wireless Way  :: Richmond, BC Canada V6V 3A4
>
> [M4]
>
> dwatkins@sierrawireless.com :: www.sierrawireless.com ::
> www.inmotiontechnology.com
>
>
>
>
> On May 20, 2017, at 8:54 AM, Khem Raj <raj.khem@gmail.com> wrote:
>
> On Sat, May 20, 2017 at 8:16 AM, Darcy Watkins
> <dwatkins@sierrawireless.com> wrote:
>
> Are you saying that the sstate cache is smart enough to handle exact same
>
> package for same arch but just built using different layer stack?  (Say one
>
> case builds package A with a bbappend, but the other uses a bbappend from a
>
> different layer, and say they are at same bbappend revision).
>
>
> yes it should be, since it checksums the variables which are effective
> during build of a given package.

Creating a directory in a recipe

[bahjat khan]

[-- Attachment #1.1: Type: text/plain, Size: 1226 bytes --]

I've searched the vast net for an answer to this but i've found NOTHING!  that works for me. All i want to do is a simple task, i want to create a new directory called "test" located in "/".
 My recipe is below

DESCRIPTION = "Hello world"
S = "${WORKDIR}"
SRC_URI="file://bye.sh"
LICENSE = "CLOSED"

LIC_FILES_CHKSUM = "file://${S}/README;md5=e584df0feead52ed7de9b60172adc0cf "

#INHIBIT_DEFAULT_DEPS = "1"

do_install() {
      install -d /test/
    install -m 0777 bye.sh /test/
}

 I can't seem to to do this, the error message is below
[cid:927d4972-8462-4e14-a645-1e674bf9e641]
I understand that there is a permissions issue with making a directory in "/" but i have no idea how to to get past this.

I feel it would be very useful to newbies (like me) if there was a tutorial which gave solutions to simple tasks like this. The reference manual breezes over most things and i understand that it can't cover everything but this does cause a massive barrier to entry when learning Yocto. Apologies if i'm sounding a bit harsh but i've been self teaching Yocto for 3 weeks in my spare time and still don't have any fruits to show for my labour :'(.

Kind Regards,
Bahjat
(Very stressed engineer)

[-- Attachment #1.2: Type: text/html, Size: 2655 bytes --]

[-- Attachment #2: pastedImage.png --]
[-- Type: image/png, Size: 40979 bytes --]

Re: Creating a directory in a recipe

[Joshua Watt]

[-- Attachment #1: Type: text/plain, Size: 437 bytes --]

On Sun, May 21, 2017 at 7:25 AM, bahjat khan <khanbahjat@hotmail.com> wrote:

> *do_install() {*
> *      install -d /test/*
>
> Should be: install -d ${D}/test/

>
> *    install -m 0777 bye.sh /test/*
>
> Should be: install -m 0777 bye.sh ${D}/test/

> *}*
>
> All of your install tasks need to install into the ${D} directory

See:
http://www.yoctoproject.org/docs/2.3/mega-manual/mega-manual.html#ref-tasks-install

[-- Attachment #2: Type: text/html, Size: 2326 bytes --]

Re: Creating a directory in a recipe

[Spriggs, Jim]

Hi Bahjat,


>do_install() {
>
>    install -d /test/
>    install -m 0777 bye.sh /test/
>}

Assuming you actually want to create "test"  in the root-dir of your *image*,
rather than in the root-dir of your host,
I think you'll find that "$(D}" is your friend, as in ...

install -d ${D}/test

etc.

--
jim

________________________________

RAYLASE AG
Argelsrieder Feld 2+4
82234 Wessling
Germany
Tel.: +49-(0)8153/88 98-0
Fax: +49-(0)8153/88 98-10
http://www.raylase.de

District Court Munich, HRB 131450

Board: Dr. Philipp Schön (Chairman), Berthold Dambacher, Christoph von Jan

Supervisory Board: Dr. Ulrich Lohmann (Chairman)


Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet.

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.