[PATCH] archiver.bbclass: do checkout for kernel

[PATCH] archiver.bbclass: do checkout for kernel

[Jian Liu]

The repo of kernel is bare git repository and we need to checkout to get
the source code.

Signed-off-by: Jian Liu <jian.liu@windriver.com>

diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index 41a552c..7387fbf 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -66,6 +66,11 @@ python () {
     ar_recipe = d.getVarFlag('ARCHIVER_MODE', 'recipe', True)
 
     if ar_src == "original":
+        # For kernel, it is bare repo, we need to checkout
+        if bb.data.inherits_class('kernel-yocto', d):
+            bb.build.addtask('do_ar_original', 'do_kernel_metadata', 'do_validate_branches', d)
+        else:
+            bb.build.addtask('do_ar_original', None, 'do_unpack', d)
         d.appendVarFlag('do_deploy_archives', 'depends', ' %s:do_ar_original' % pn)
     elif ar_src == "patched":
         d.appendVarFlag('do_deploy_archives', 'depends', ' %s:do_ar_patched' % pn)
@@ -113,6 +118,12 @@ python do_ar_original() {
 
     ar_outdir = d.getVar('ARCHIVER_OUTDIR', True)
     bb.note('Archiving the original source...')
+    # For kernel, the source code is shared
+    if bb.data.inherits_class('kernel-yocto', d):
+        srcdir = d.getVar('S', True)
+        create_tarball(d, srcdir, 'original', ar_outdir)
+        return
+
     fetch = bb.fetch2.Fetch([], d)
     for url in fetch.urls:
         local = fetch.localpath(url).rstrip("/");
@@ -346,7 +357,6 @@ python do_deploy_archives_setscene () {
 do_deploy_archives[sstate-inputdirs] = "${ARCHIVER_TOPDIR}"
 do_deploy_archives[sstate-outputdirs] = "${DEPLOY_DIR_SRC}"
 
-addtask do_ar_original after do_unpack
 addtask do_ar_patched after do_unpack_and_patch
 addtask do_ar_configured after do_unpack_and_patch
 addtask do_dumpdata

Re: [PATCH] archiver.bbclass: do checkout for kernel

[Khem Raj]

On Sun, Nov 1, 2015 at 10:23 PM, Jian Liu <jian.liu@windriver.com> wrote:
> The repo of kernel is bare git repository and we need to checkout to get
> the source code.

it should be linux-yocto I think, or do we have all kernels as bare git repos ?

>
> Signed-off-by: Jian Liu <jian.liu@windriver.com>
>
> diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
> index 41a552c..7387fbf 100644
> --- a/meta/classes/archiver.bbclass
> +++ b/meta/classes/archiver.bbclass
> @@ -66,6 +66,11 @@ python () {
>      ar_recipe = d.getVarFlag('ARCHIVER_MODE', 'recipe', True)
>
>      if ar_src == "original":
> +        # For kernel, it is bare repo, we need to checkout
> +        if bb.data.inherits_class('kernel-yocto', d):
> +            bb.build.addtask('do_ar_original', 'do_kernel_metadata', 'do_validate_branches', d)
> +        else:
> +            bb.build.addtask('do_ar_original', None, 'do_unpack', d)
>          d.appendVarFlag('do_deploy_archives', 'depends', ' %s:do_ar_original' % pn)
>      elif ar_src == "patched":
>          d.appendVarFlag('do_deploy_archives', 'depends', ' %s:do_ar_patched' % pn)
> @@ -113,6 +118,12 @@ python do_ar_original() {
>
>      ar_outdir = d.getVar('ARCHIVER_OUTDIR', True)
>      bb.note('Archiving the original source...')
> +    # For kernel, the source code is shared
> +    if bb.data.inherits_class('kernel-yocto', d):

is this linux-yocto specific

> +        srcdir = d.getVar('S', True)
> +        create_tarball(d, srcdir, 'original', ar_outdir)
> +        return
> +
>      fetch = bb.fetch2.Fetch([], d)
>      for url in fetch.urls:
>          local = fetch.localpath(url).rstrip("/");
> @@ -346,7 +357,6 @@ python do_deploy_archives_setscene () {
>  do_deploy_archives[sstate-inputdirs] = "${ARCHIVER_TOPDIR}"
>  do_deploy_archives[sstate-outputdirs] = "${DEPLOY_DIR_SRC}"
>
> -addtask do_ar_original after do_unpack
>  addtask do_ar_patched after do_unpack_and_patch
>  addtask do_ar_configured after do_unpack_and_patch
>  addtask do_dumpdata
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH] archiver.bbclass: do checkout for kernel

[Jian Liu]

Hi,

I test linux-dummy linux-yocto-dev linux-yocto-rt linux-yocto-tiny and 
the result is as follows,

linux-dummy: this package just meets some dependency and there is no 
source code for it.
                       The patch does not impact it
linux-yocto, linux-yocto-dev, linux-yocto-rt, linux-yocto-tiny: The S 
variable of all packages is set to STAGING_KERNEK_DIR.
                        do_fetch and do_unpack just leave a bare git 
repo of kernel and do_kernel_checkout gets the source code

Thanks!
Jian

On 2015年11月03日 05:30, Khem Raj wrote:
> On Sun, Nov 1, 2015 at 10:23 PM, Jian Liu <jian.liu@windriver.com> wrote:
>> The repo of kernel is bare git repository and we need to checkout to get
>> the source code.
> it should be linux-yocto I think, or do we have all kernels as bare git repos ?
>
>> Signed-off-by: Jian Liu <jian.liu@windriver.com>
>>
>> diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
>> index 41a552c..7387fbf 100644
>> --- a/meta/classes/archiver.bbclass
>> +++ b/meta/classes/archiver.bbclass
>> @@ -66,6 +66,11 @@ python () {
>>       ar_recipe = d.getVarFlag('ARCHIVER_MODE', 'recipe', True)
>>
>>       if ar_src == "original":
>> +        # For kernel, it is bare repo, we need to checkout
>> +        if bb.data.inherits_class('kernel-yocto', d):
>> +            bb.build.addtask('do_ar_original', 'do_kernel_metadata', 'do_validate_branches', d)
>> +        else:
>> +            bb.build.addtask('do_ar_original', None, 'do_unpack', d)
>>           d.appendVarFlag('do_deploy_archives', 'depends', ' %s:do_ar_original' % pn)
>>       elif ar_src == "patched":
>>           d.appendVarFlag('do_deploy_archives', 'depends', ' %s:do_ar_patched' % pn)
>> @@ -113,6 +118,12 @@ python do_ar_original() {
>>
>>       ar_outdir = d.getVar('ARCHIVER_OUTDIR', True)
>>       bb.note('Archiving the original source...')
>> +    # For kernel, the source code is shared
>> +    if bb.data.inherits_class('kernel-yocto', d):
> is this linux-yocto specific
>
>> +        srcdir = d.getVar('S', True)
>> +        create_tarball(d, srcdir, 'original', ar_outdir)
>> +        return
>> +
>>       fetch = bb.fetch2.Fetch([], d)
>>       for url in fetch.urls:
>>           local = fetch.localpath(url).rstrip("/");
>> @@ -346,7 +357,6 @@ python do_deploy_archives_setscene () {
>>   do_deploy_archives[sstate-inputdirs] = "${ARCHIVER_TOPDIR}"
>>   do_deploy_archives[sstate-outputdirs] = "${DEPLOY_DIR_SRC}"
>>
>> -addtask do_ar_original after do_unpack
>>   addtask do_ar_patched after do_unpack_and_patch
>>   addtask do_ar_configured after do_unpack_and_patch
>>   addtask do_dumpdata
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>

Re: [PATCH] archiver.bbclass: do checkout for kernel

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 3187 bytes --]


> On Nov 3, 2015, at 1:22 AM, Jian Liu <jian.liu@windriver.com> wrote:
> 
> Hi,
> 
> I test linux-dummy linux-yocto-dev linux-yocto-rt linux-yocto-tiny and the result is as follows,
> 
> linux-dummy: this package just meets some dependency and there is no source code for it.
>                      The patch does not impact it
> linux-yocto, linux-yocto-dev, linux-yocto-rt, linux-yocto-tiny: The S variable of all packages is set to STAGING_KERNEK_DIR.
>                       do_fetch and do_unpack just leave a bare git repo of kernel and do_kernel_checkout gets the source code
> 

does it cover the case when kernel is not linux-yocto was basic question, I am not clear if thats covered in any of above

> Thanks!
> Jian
> 
> On 2015年11月03日 05:30, Khem Raj wrote:
>> On Sun, Nov 1, 2015 at 10:23 PM, Jian Liu <jian.liu@windriver.com> wrote:
>>> The repo of kernel is bare git repository and we need to checkout to get
>>> the source code.
>> it should be linux-yocto I think, or do we have all kernels as bare git repos ?
>> 
>>> Signed-off-by: Jian Liu <jian.liu@windriver.com>
>>> 
>>> diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
>>> index 41a552c..7387fbf 100644
>>> --- a/meta/classes/archiver.bbclass
>>> +++ b/meta/classes/archiver.bbclass
>>> @@ -66,6 +66,11 @@ python () {
>>>      ar_recipe = d.getVarFlag('ARCHIVER_MODE', 'recipe', True)
>>> 
>>>      if ar_src == "original":
>>> +        # For kernel, it is bare repo, we need to checkout
>>> +        if bb.data.inherits_class('kernel-yocto', d):
>>> +            bb.build.addtask('do_ar_original', 'do_kernel_metadata', 'do_validate_branches', d)
>>> +        else:
>>> +            bb.build.addtask('do_ar_original', None, 'do_unpack', d)
>>>          d.appendVarFlag('do_deploy_archives', 'depends', ' %s:do_ar_original' % pn)
>>>      elif ar_src == "patched":
>>>          d.appendVarFlag('do_deploy_archives', 'depends', ' %s:do_ar_patched' % pn)
>>> @@ -113,6 +118,12 @@ python do_ar_original() {
>>> 
>>>      ar_outdir = d.getVar('ARCHIVER_OUTDIR', True)
>>>      bb.note('Archiving the original source...')
>>> +    # For kernel, the source code is shared
>>> +    if bb.data.inherits_class('kernel-yocto', d):
>> is this linux-yocto specific
>> 
>>> +        srcdir = d.getVar('S', True)
>>> +        create_tarball(d, srcdir, 'original', ar_outdir)
>>> +        return
>>> +
>>>      fetch = bb.fetch2.Fetch([], d)
>>>      for url in fetch.urls:
>>>          local = fetch.localpath(url).rstrip("/");
>>> @@ -346,7 +357,6 @@ python do_deploy_archives_setscene () {
>>>  do_deploy_archives[sstate-inputdirs] = "${ARCHIVER_TOPDIR}"
>>>  do_deploy_archives[sstate-outputdirs] = "${DEPLOY_DIR_SRC}"
>>> 
>>> -addtask do_ar_original after do_unpack
>>>  addtask do_ar_patched after do_unpack_and_patch
>>>  addtask do_ar_configured after do_unpack_and_patch
>>>  addtask do_dumpdata
>>> --
>>> _______________________________________________
>>> Openembedded-core mailing list
>>> Openembedded-core@lists.openembedded.org
>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>> 
> 


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 211 bytes --]

Re: [PATCH] archiver.bbclass: do checkout for kernel

[Jian Liu]

Hi,

All kernels I list bellow will perform do_fetch(), do_unpack(), 
do_kernel_checkout().
do_unpack() will put a bare git repo under WORKDIR and then 
do_kernel_checkout()
will move the bare repo to bitbake/tmp/work-share/<board 
name>/kernel-source(value of variable S),
then performing "git checkout".

So do_ar_original must wait until do_kernel_checkout() is completed and 
then make tarball using the code under S.
Do I make that clear?


You refer to kernels other than linux-yocto. Can you give me an example?
I do not know how to build such kernel.

Thanks!
Jian

On 2015年11月04日 13:57, Khem Raj wrote:
>> On Nov 3, 2015, at 1:22 AM, Jian Liu<jian.liu@windriver.com>  wrote:
>> >
>> >Hi,
>> >
>> >I test linux-dummy linux-yocto-dev linux-yocto-rt linux-yocto-tiny and the result is as follows,
>> >
>> >linux-dummy: this package just meets some dependency and there is no source code for it.
>> >                      The patch does not impact it
>> >linux-yocto, linux-yocto-dev, linux-yocto-rt, linux-yocto-tiny: The S variable of all packages is set to STAGING_KERNEK_DIR.
>> >                       do_fetch and do_unpack just leave a bare git repo of kernel and do_kernel_checkout gets the source code
>> >
> does it cover the case when kernel is not linux-yocto was basic question, I am not clear if thats covered in any of above
>
>> >Thanks!

Re: [PATCH] archiver.bbclass: do checkout for kernel

[Khem Raj]

On Wed, Nov 4, 2015 at 6:14 PM, Jian Liu <jian.liu@windriver.com> wrote:
> Hi,
>
> All kernels I list bellow will perform do_fetch(), do_unpack(),
> do_kernel_checkout().
> do_unpack() will put a bare git repo under WORKDIR and then
> do_kernel_checkout()
> will move the bare repo to bitbake/tmp/work-share/<board
> name>/kernel-source(value of variable S),
> then performing "git checkout".
>
> So do_ar_original must wait until do_kernel_checkout() is completed and then
> make tarball using the code under S.
> Do I make that clear?
>
>
> You refer to kernels other than linux-yocto. Can you give me an example?
> I do not know how to build such kernel.

Add another BSP layer e.g. meta-raspberrypi to your bblayers.conf
and then MACHINE=raspberrypi2

and see what you are trying to do still holds.

>
> Thanks!
> Jian
>
>
> On 2015年11月04日 13:57, Khem Raj wrote:
>>>
>>> On Nov 3, 2015, at 1:22 AM, Jian Liu<jian.liu@windriver.com>  wrote:
>>> >
>>> >Hi,
>>> >
>>> >I test linux-dummy linux-yocto-dev linux-yocto-rt linux-yocto-tiny and
>>> > the result is as follows,
>>> >
>>> >linux-dummy: this package just meets some dependency and there is no
>>> > source code for it.
>>> >                      The patch does not impact it
>>> >linux-yocto, linux-yocto-dev, linux-yocto-rt, linux-yocto-tiny: The S
>>> > variable of all packages is set to STAGING_KERNEK_DIR.
>>> >                       do_fetch and do_unpack just leave a bare git repo
>>> > of kernel and do_kernel_checkout gets the source code
>>> >
>>
>> does it cover the case when kernel is not linux-yocto was basic question,
>> I am not clear if thats covered in any of above
>>
>>> >Thanks!
>
>

Re: [PATCH] archiver.bbclass: do checkout for kernel

[Jian Liu]

Hi Raj,

I tried meta-rasberrypi with my patch and it works well.
The tasks of the recipe linux-rasberrypi is similar with linux-yocto and 
linux-rasberrypi also inherits linux-yocto.bbclass

I read the codes of archiver and it seems that kernel is recognised by 
"bb.data.inherits_class('kernel-yocto', d)" when kernel needs to be 
handled specially

What do you think?

Thanks!
Jian

On 2015年11月05日 10:22, Khem Raj wrote:
> On Wed, Nov 4, 2015 at 6:14 PM, Jian Liu <jian.liu@windriver.com> wrote:
>> Hi,
>>
>> All kernels I list bellow will perform do_fetch(), do_unpack(),
>> do_kernel_checkout().
>> do_unpack() will put a bare git repo under WORKDIR and then
>> do_kernel_checkout()
>> will move the bare repo to bitbake/tmp/work-share/<board
>> name>/kernel-source(value of variable S),
>> then performing "git checkout".
>>
>> So do_ar_original must wait until do_kernel_checkout() is completed and then
>> make tarball using the code under S.
>> Do I make that clear?
>>
>>
>> You refer to kernels other than linux-yocto. Can you give me an example?
>> I do not know how to build such kernel.
> Add another BSP layer e.g. meta-raspberrypi to your bblayers.conf
> and then MACHINE=raspberrypi2
>
> and see what you are trying to do still holds.
>
>> Thanks!
>> Jian
>>
>>
>> On 2015年11月04日 13:57, Khem Raj wrote:
>>>> On Nov 3, 2015, at 1:22 AM, Jian Liu<jian.liu@windriver.com>  wrote:
>>>>> Hi,
>>>>>
>>>>> I test linux-dummy linux-yocto-dev linux-yocto-rt linux-yocto-tiny and
>>>>> the result is as follows,
>>>>>
>>>>> linux-dummy: this package just meets some dependency and there is no
>>>>> source code for it.
>>>>>                       The patch does not impact it
>>>>> linux-yocto, linux-yocto-dev, linux-yocto-rt, linux-yocto-tiny: The S
>>>>> variable of all packages is set to STAGING_KERNEK_DIR.
>>>>>                        do_fetch and do_unpack just leave a bare git repo
>>>>> of kernel and do_kernel_checkout gets the source code
>>>>>
>>> does it cover the case when kernel is not linux-yocto was basic question,
>>> I am not clear if thats covered in any of above
>>>
>>>>> Thanks!
>>

Re: [PATCH] archiver.bbclass: do checkout for kernel

[Jian Liu]

Ping

On 2015年11月13日 16:16, Jian Liu wrote:
> Hi Raj,
>
> I tried meta-rasberrypi with my patch and it works well.
> The tasks of the recipe linux-rasberrypi is similar with linux-yocto 
> and linux-rasberrypi also inherits linux-yocto.bbclass
>
> I read the codes of archiver and it seems that kernel is recognised by 
> "bb.data.inherits_class('kernel-yocto', d)" when kernel needs to be 
> handled specially
>
> What do you think?
>
> Thanks!
> Jian
>
> On 2015年11月05日 10:22, Khem Raj wrote:
>> On Wed, Nov 4, 2015 at 6:14 PM, Jian Liu <jian.liu@windriver.com> wrote:
>>> Hi,
>>>
>>> All kernels I list bellow will perform do_fetch(), do_unpack(),
>>> do_kernel_checkout().
>>> do_unpack() will put a bare git repo under WORKDIR and then
>>> do_kernel_checkout()
>>> will move the bare repo to bitbake/tmp/work-share/<board
>>> name>/kernel-source(value of variable S),
>>> then performing "git checkout".
>>>
>>> So do_ar_original must wait until do_kernel_checkout() is completed 
>>> and then
>>> make tarball using the code under S.
>>> Do I make that clear?
>>>
>>>
>>> You refer to kernels other than linux-yocto. Can you give me an 
>>> example?
>>> I do not know how to build such kernel.
>> Add another BSP layer e.g. meta-raspberrypi to your bblayers.conf
>> and then MACHINE=raspberrypi2
>>
>> and see what you are trying to do still holds.
>>
>>> Thanks!
>>> Jian
>>>
>>>
>>> On 2015年11月04日 13:57, Khem Raj wrote:
>>>>> On Nov 3, 2015, at 1:22 AM, Jian Liu<jian.liu@windriver.com>  wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I test linux-dummy linux-yocto-dev linux-yocto-rt 
>>>>>> linux-yocto-tiny and
>>>>>> the result is as follows,
>>>>>>
>>>>>> linux-dummy: this package just meets some dependency and there is no
>>>>>> source code for it.
>>>>>>                       The patch does not impact it
>>>>>> linux-yocto, linux-yocto-dev, linux-yocto-rt, linux-yocto-tiny: 
>>>>>> The S
>>>>>> variable of all packages is set to STAGING_KERNEK_DIR.
>>>>>>                        do_fetch and do_unpack just leave a bare 
>>>>>> git repo
>>>>>> of kernel and do_kernel_checkout gets the source code
>>>>>>
>>>> does it cover the case when kernel is not linux-yocto was basic 
>>>> question,
>>>> I am not clear if thats covered in any of above
>>>>
>>>>>> Thanks!
>>>
>

Re: [PATCH] archiver.bbclass: do checkout for kernel

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 400 bytes --]

On 2 November 2015 at 06:23, Jian Liu <jian.liu@windriver.com> wrote:

> The repo of kernel is bare git repository and we need to checkout to get
> the source code.
>

A recent patch to archiver ("fix previous issue regarding work-shared for
linux-yocto") claims to fix this, can you confirm if that fixes this
problem or if this fixes more edge cases regarding the kernel.

Cheers,
Ross

[-- Attachment #2: Type: text/html, Size: 847 bytes --]

Re: [PATCH] archiver.bbclass: do checkout for kernel

[Jian Liu]

[-- Attachment #1: Type: text/plain, Size: 1036 bytes --]

Hi Ross,

I test this patch. I seem to have no relationship with my issue. This 
patch just optimizes some tasks.

My problem seems to be gotten around by a patch 
523e4f6a6913b64453579d27a02467e14f7df42e submitted by Bruce.
"bareclone=1" is removed from SRC_URI. So kernel source is not a bare 
repo any more.

I do not know why "bareclone=1" is removed. This option seems to have no 
side effect on kernel.
So I thinks this should be a hidden trouble.

Do you have any ideas?

Thanks!

Jian

On 2015年12月17日 06:29, Burton, Ross wrote:
>
> On 2 November 2015 at 06:23, Jian Liu <jian.liu@windriver.com 
> <mailto:jian.liu@windriver.com>> wrote:
>
>     The repo of kernel is bare git repository and we need to checkout
>     to get
>     the source code.
>
>
> A recent patch to archiver ("fix previous issue regarding work-shared 
> for linux-yocto") claims to fix this, can you confirm if that fixes 
> this problem or if this fixes more edge cases regarding the kernel.
>
> Cheers,
> Ross


[-- Attachment #2: Type: text/html, Size: 2318 bytes --]