How to recreate rootfs image

How to recreate rootfs image

[Lenivyy Viktor]

> In your kernel recipe are you using SRC_URI to fetch from a git
> repository (e.g. git:// URI) or from a local directory?

This kernel is fetched from local directory.

> I guess that if you're using a local path, there can be either some
> uncommitted changes, or a stale git index.

No, because kernel built from sources in local directory doesn’t have “-dirty” in version string.

> You can try just for the experiment to add your current kernel sources
> to a test git repo and point the SRC_URI to it, so bitbake can clone
> the repo by git revision (SRCREV = "${AUTOREV}" will skip the need to
> update the recipe revision constantly during development). This should
> work fine, without the "-dirty" version suffix.

I can try this, but it doesn't answer main question:
how can I recreate rootfs image starting from the point after fetching Linux sources, so Yocto’s copy will remain intact?

Regards,
Nikolay

Re: How to recreate rootfs image

[Paul Eggleton]

Hi Viktor,

On Thursday 27 August 2015 08:25:38 Lenivyy Viktor wrote:
> > In your kernel recipe are you using SRC_URI to fetch from a git
> > repository (e.g. git:// URI) or from a local directory?
> 
> This kernel is fetched from local directory.
> 
> > I guess that if you're using a local path, there can be either some
> > uncommitted changes, or a stale git index.
> 
> No, because kernel built from sources in local directory doesn’t have
> “-dirty” in version string.
> > You can try just for the experiment to add your current kernel sources
> > to a test git repo and point the SRC_URI to it, so bitbake can clone
> > the repo by git revision (SRCREV = "${AUTOREV}" will skip the need to
> > update the recipe revision constantly during development). This should
> > work fine, without the "-dirty" version suffix.
> 
> I can try this, but it doesn't answer main question:
> how can I recreate rootfs image starting from the point after fetching Linux
> sources, so Yocto’s copy will remain intact?

Well one way would be:

 bitbake -C compile virtual/kernel <imagename>

(note the capital -C, not -c).

In the near future "devtool modify" should support the kind of workflow that it 
looks like you're attempting to get (where you want to modify the kernel 
sources locally and then build them and/or incorporate them in an image) - it 
already works well for non-kernel recipes.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

How to recreate rootfs image

[Lenivyy Viktor]

> Hi Viktor,
> 
> On Thursday 27 August 2015 08:25:38 Lenivyy Viktor wrote:
> > > In your kernel recipe are you using SRC_URI to fetch from a git 
> > > repository (e.g. git:// URI) or from a local directory?
> >
> > This kernel is fetched from local directory.
> >
> > > I guess that if you're using a local path, there can be either 
> > > some uncommitted changes, or a stale git index.
> >
> > No, because kernel built from sources in local directory doesn’t 
> > have “-dirty” in version string.
> > > You can try just for the experiment to add your current kernel 
> > > sources to a test git repo and point the SRC_URI to it, so bitbake 
> > > can clone the repo by git revision (SRCREV = "${AUTOREV}" will 
> > > skip the need to update the recipe revision constantly during 
> > > development). This should work fine, without the "-dirty" version suffix.
> >
> > I can try this, but it doesn't answer main question:
> > how can I recreate rootfs image starting from the point after 
> > fetching Linux sources, so Yocto’s copy will remain intact?
> 
> Well one way would be:
> 
>  bitbake -C compile virtual/kernel <imagename>
> 
> (note the capital -C, not -c).
> 

Hi, Paul. And thank you.
Looks like the most relevant answer.
IMO this "-C" flag is poorly documented. Thus if you don't know what it's doing, you can't figure out to use it.
Only some info in maillists found by Google can give a clue what "--clear-stamp" is intended for.

> In the near future "devtool modify" should support the kind of 
> workflow that it looks like you're attempting to get (where you want 
> to modify the kernel sources locally and then build them and/or 
> incorporate them in an image) - it already works well for non-kernel recipes.

I think this workflow is useful for many things, not only for workaround "-dirty", as in my case. It's not very convenient to wait for fetching sources every time you do some little modification.
Why we need separate command i.e. "devtool modify" if we can go without it?

Re: How to recreate rootfs image

[Paul Eggleton]

On Friday 28 August 2015 13:32:23 Lenivyy Viktor wrote:
> > On Thursday 27 August 2015 08:25:38 Lenivyy Viktor wrote:
> > > > In your kernel recipe are you using SRC_URI to fetch from a git
> > > > repository (e.g. git:// URI) or from a local directory?
> > >
> > > This kernel is fetched from local directory.
> > >
> > > > I guess that if you're using a local path, there can be either some
> > > > uncommitted changes, or a stale git index.
> > >
> > > No, because kernel built from sources in local directory doesn’t have
> > > “-dirty” in version string.
> > > 
> > > > You can try just for the experiment to add your current kernel
> > > > sources to a test git repo and point the SRC_URI to it, so bitbake
> > > > can clone the repo by git revision (SRCREV = "${AUTOREV}" will skip
> > > > the need to update the recipe revision constantly during
> > > > development). This should work fine, without the "-dirty" version
> > > > suffix.
> > >
> > > I can try this, but it doesn't answer main question:
> > > how can I recreate rootfs image starting from the point after fetching
> > > Linux sources, so Yocto’s copy will remain intact?
> > 
> > 
> > Well one way would be:
> > 
> > 
> >  bitbake -C compile virtual/kernel <imagename>
> > 
> > 
> > (note the capital -C, not -c).
> 
> Hi, Paul. And thank you.
> Looks like the most relevant answer.
> IMO this "-C" flag is poorly documented. Thus if you don't know what it's
> doing, you can't figure out to use it.
> Only some info in maillists found
> by Google can give a clue what "--clear-stamp" is intended for. 

Right, that may well be the case.
 
> > In the near future "devtool modify" should support the kind of workflow
> > that it looks like you're attempting to get (where you want to modify
> > the kernel sources locally and then build them and/or incorporate them in
> > an image) - it already works well for non-kernel recipes.
> 
> I think this workflow is useful for many things, not only for workaround
> "-dirty", as in my case. It's not very convenient to wait for fetching
> sources every time you do some little modification.
> Why we need separate
> command i.e. "devtool modify" if we can go without it?

The assumption is that you wouldn't normally be fetching the source from a 
local path as you are, you would fetch it from some repository (though that 
repository may be internal to your organisation). At some point you run 
"devtool modify" which is effectively a way of saying "I want to start working 
on the source for this particular recipe in this location" - at which point 
either you're specifying an existing source checkout that you've prepared, or 
you specify -x and it prepares it for you. Having run that, the fetch/unpack 
tasks are disabled, and the specified external source tree will be used in the 
build. Additionally, that external source tree is yours to keep should you 
wish - it will never be deleted by the build system; the same *cannot* be said 
of the build system's temporary source checkout under tmp/work - your work in 
progress can be lost in an instant if you run -c clean or have rm_work 
enabled. There really are two different modes here - normal build, where you 
want the sources as anyone else building the system would fetch, and 
development, where you want your own sources that you are in control of.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: How to recreate rootfs image

[Lenivyy Viktor]

> The assumption is that you wouldn't normally be fetching the source from a
> local path as you are, you would fetch it from some repository (though that
> repository may be internal to your organisation). At some point you run
> "devtool modify" which is effectively a way of saying "I want to start working on
> the source for this particular recipe in this location" - at which point either
> you're specifying an existing source checkout that you've prepared, or you
> specify -x and it prepares it for you. Having run that, the fetch/unpack tasks are
> disabled, and the specified external source tree will be used in the build.
> Additionally, that external source tree is yours to keep should you wish - it will
> never be deleted by the build system; the same *cannot* be said of the build
> system's temporary source checkout under tmp/work - your work in progress
> can be lost in an instant if you run -c clean or have rm_work enabled. There
> really are two different modes here - normal build, where you want the sources
> as anyone else building the system would fetch, and development, where you
> want your own sources that you are in control of.
> 
Thank you for detailed answer, Paul. It needs to be added to documentation.

Re: How to recreate rootfs image

[Khem Raj]

On Wed, Aug 26, 2015 at 9:13 AM, Lenivyy Viktor
<Viktor.Lenivyy@author.kiev.ua> wrote:
>> dirtypackage is the one which you want to rebuild and redeploy in the
>
>> rootfs. If the package source is local, cleanall won't do big harm, but
>
>> otherwise you're correct, it will clean the sources. You can use
>
>> "clean" instead.
>
>
>
>> Regards,
>
>> Nikolay
>
>
>
> I don’t know exactly which package I need to rebuild, but I guess it’s
> kernel-modules as modules folder have wrong version.
>
> After
>
> bitbake -c clean kernel-modules altera-image-minimal
>
>
>
> there is an error:
>
> …
>
> ERROR: Nothing PROVIDES 'kernel-modules'

kernel modules are part of virtual/kernel so you need to clean
virtual/kernel if thats what the issue is

>
> …
>
>
>
>
>
>
>
>
>
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>

How to recreate rootfs image

[Lenivyy Viktor]

[-- Attachment #1: Type: text/plain, Size: 535 bytes --]

> dirtypackage is the one which you want to rebuild and redeploy in the
> rootfs. If the package source is local, cleanall won't do big harm, but
> otherwise you're correct, it will clean the sources. You can use
> "clean" instead.

> Regards,
> Nikolay

I don’t know exactly which package I need to rebuild, but I guess it’s kernel-modules as modules folder have wrong version.
After
bitbake -c clean kernel-modules altera-image-minimal

there is an error:
…
ERROR: Nothing PROVIDES 'kernel-modules'
…





[-- Attachment #2: Type: text/html, Size: 5755 bytes --]

Re: How to recreate rootfs image

[Nikolay Dimitrov]

Hi Lenivyy Viktor,

On 08/26/2015 06:42 PM, Lenivyy Viktor wrote:
>>  Can you please try this and share if it works for you:
>
>>
>
>>  bitbake -c cleanall dirtypackage myimage
>
>>  bitbake myimage
>
>>
>
>>  Regards,
>
>>  Nikolay
>
> What is dirtypackage? Myimagewill be altera-image-minimal in my case, I
> guess.
>
> I’m wondering whether cleanall doesn’t remove Yocto’s copy of kernel
> sources?
>

dirtypackage is the one which you want to rebuild and redeploy in the
rootfs. If the package source is local, cleanall won't do big harm, but
otherwise you're correct, it will clean the sources. You can use
"clean" instead.

Regards,
Nikolay

How to recreate rootfs image

[Lenivyy Viktor]

[-- Attachment #1: Type: text/plain, Size: 318 bytes --]

> Can you please try this and share if it works for you:
>
> bitbake -c cleanall dirtypackage myimage
> bitbake myimage
>
> Regards,
> Nikolay

What is dirtypackage? Myimage will be altera-image-minimal in my case, I guess.
I’m wondering whether cleanall doesn’t remove Yocto’s copy of kernel sources?

[-- Attachment #2: Type: text/html, Size: 4964 bytes --]

Re: How to recreate rootfs image

[Nikolay Dimitrov]

Hi Lenivyy Viktor,

On 08/26/2015 05:37 PM, Lenivyy Viktor wrote:
> Hello.
>
> I have “altera-image-minimal” recipe which creates minimal rootfs based
> on Linux kernel provided by “linux-altera-local” recipe. This kernel is
> fetched from local directory.
>
> After first time building “altera-image-minimal”, it produced rootfs
> with modules version contained “-dirty” at the end. However kernel built
> from sources in local directory doesn’t have “-dirty” in version string.
> Thus kernel can’t run with produced rootfs.
>
> I opened directory which contains Yocto’s copy of Linux sources
>
> build/tmp/work/socfpga_socrates-poky-linux-gnueabi/linux-altera-local-1.0-r1/socrates_linux/
>
> then run “make ARCH=arm kernelrelease” to investigate kernel version.
> This command outputs the version with “-dirty”. I have found that
> “-dirty” string is appended from script in kernel sources. That script
> uses output from “git diff-index --name-only HEAD” to determine if
> working directory is “dirty”. To find changed files, I ran same command,
> it’s oddly but the output was empty. Then I rerun “make ARCH=arm
> kernelrelease”. This time it prints  the version without “-dirty”.
>
> Does someone have any clue why it can be so?
>
> My main question is: how can I recreate rootfs image starting from the
> point after fetching Linux sources, so Yocto’s copy will remain intact?
>
> I spent lot of time trying to figure it out by myself with no luck. My
> idea was to find sequence of commands used to create image and run only
> commands after fetching kernel sources. Unfortunately I didn’t manage to
> find the sequence anywhere. How such things need to be solved in Yocto?
>

Can you please try this and share if it works for you:

bitbake -c cleanall dirtypackage myimage
bitbake myimage

Regards,
Nikolay

How to recreate rootfs image

[Lenivyy Viktor]

[-- Attachment #1: Type: text/plain, Size: 1625 bytes --]

Hello.
I have “altera-image-minimal” recipe which creates minimal rootfs based on Linux kernel provided by “linux-altera-local” recipe. This kernel is fetched from local directory.
After first time building “altera-image-minimal”, it produced rootfs with modules version contained “-dirty” at the end. However kernel built from sources in local directory doesn’t have “-dirty” in version string. Thus kernel can’t run with produced rootfs.
I opened directory which contains Yocto’s copy of Linux sources
build/tmp/work/socfpga_socrates-poky-linux-gnueabi/linux-altera-local-1.0-r1/socrates_linux/
then run “make ARCH=arm kernelrelease” to investigate kernel version. This command outputs the version with “-dirty”. I have found that “-dirty” string is appended from script in kernel sources. That script uses output from “git diff-index --name-only HEAD” to determine if working directory is “dirty”. To find changed files, I ran same command, it’s oddly but the output was empty. Then I rerun “make ARCH=arm kernelrelease”. This time it prints  the version without “-dirty”.
Does someone have any clue why it can be so?

My main question is: how can I recreate rootfs image starting from the point after fetching Linux sources, so Yocto’s copy will remain intact?

I spent lot of time trying to figure it out by myself with no luck. My idea was to find sequence of commands used to create image and run only commands after fetching kernel sources. Unfortunately I didn’t manage to find the sequence anywhere. How such things need to be solved in Yocto?

[-- Attachment #2: Type: text/html, Size: 3928 bytes --]