Y-AB PublishArtifacts toolchain wrong dir

Y-AB PublishArtifacts toolchain wrong dir

[gmane]

Hi,

I use the latest and greatest yocto autobuilder: 
f9b4e02730a5e712ee9085e2480f9a3b8b00f56d

It looks like Publishing Artifacts tries to find the toolchain instead 
of build/build/tmp/deploy/sdk (where it is built) here 
build/build/tmp/deploy/images/toolchain-x86_64

This patch [1] fixes the problem for me.

Regards,

Robert

https://github.com/RobertBerger/autobuilder-patches/blob/master/before-install/0004-PublichArtifacts-searches-toolchain-in-wrong-place.patch

Re: Y-AB PublishArtifacts toolchain wrong dir

[Bill Randle]

Hi Robert,

I see a couple of problems with this proposed patch. First, it's not
clear the existing code is broken. It calls getDeployNames() which
should detect the build artifact is "toolchain" and return a deploy
path of "deploy/sdk". If that is not working correctly, then that's
the code that should be fixed. Second, your patch dropped the "+
pipeline" from the command chain, which is used to set an error flag
in the event the copy failed. Third, you also dropped the portion of
the command chain that calculates the md5sums of the toolchain files.
(The fact that it appears, on the surface, that the md5sum files do
not get copied over would be a bug - although it seems like I tested
that at one time and it was working.)

    -Bill

On Fri, Oct 7, 2016 at 8:44 AM,  <gmane@reliableembeddedsystems.com> wrote:
> Hi,
>
> I use the latest and greatest yocto autobuilder:
> f9b4e02730a5e712ee9085e2480f9a3b8b00f56d
>
> It looks like Publishing Artifacts tries to find the toolchain instead of
> build/build/tmp/deploy/sdk (where it is built) here
> build/build/tmp/deploy/images/toolchain-x86_64
>
> This patch [1] fixes the problem for me.
>
> Regards,
>
> Robert
>
> https://github.com/RobertBerger/autobuilder-patches/blob/master/before-install/0004-PublichArtifacts-searches-toolchain-in-wrong-place.patch
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

Re: Y-AB PublishArtifacts toolchain wrong dir

[gmane]

Hi,

On 2016-10-07 22:05, Bill Randle wrote:
> Hi Robert,
> 
> I see a couple of problems with this proposed patch. First, it's not
> clear the existing code is broken.

Try this [1] to see the problem.

[1] 
https://github.com/RobertBerger/meta-mainline/blob/jethro-training-v4.4.x/multi-v7-ml/yocto-autobuilder/custom-jethro-multi-v7-core-image-minimal-sato-sdk-adt.conf

Regards,

Robert

Re: Y-AB PublishArtifacts toolchain wrong dir

[gmane]

Hi,

On 2016-10-07 22:05, Bill Randle wrote:
> Hi Robert,
> 
> I see a couple of problems with this proposed patch. First, it's not
> clear the existing code is broken.

Here[1] is a paste of the error.

cp: cannot stat 
'/tmp/yocto-autobuilder/yocto-autobuilder/yocto-worker/res-custom-morty-multi-v7-core-image-minimal-dev-sato-sdk-toolchain/build/build/tmp/deploy/sdk/poky-*i686-core-image*.sh': 
No such file or directory

That's[2] the .conf file which I used.

Note that I'm not building a i686 toolchain, but only the 64 bit 
version.

[1] http://pastebin.com/aLyer2fi
[2] 
https://github.com/RobertBerger/meta-mainline/blob/morty-training-v4.4.x/multi-v7-ml/yocto-autobuilder/res-custom-morty-multi-v7-core-image-minimal-dev-sato-sdk-toolchain.conf

Regards,

Robert

Re: Y-AB PublishArtifacts toolchain wrong dir

[Bill Randle]

On Fri, Oct 7, 2016 at 11:27 PM,  <gmane@reliableembeddedsystems.com> wrote:
> Hi,
>
> On 2016-10-07 22:05, Bill Randle wrote:
>>
>> Hi Robert,
>>
>> I see a couple of problems with this proposed patch. First, it's not
>> clear the existing code is broken.
>
>
> Here[1] is a paste of the error.
>
> cp: cannot stat
> '/tmp/yocto-autobuilder/yocto-autobuilder/yocto-worker/res-custom-morty-multi-v7-core-image-minimal-dev-sato-sdk-toolchain/build/build/tmp/deploy/sdk/poky-*i686-core-image*.sh':
> No such file or directory
>
> That's[2] the .conf file which I used.
>
> Note that I'm not building a i686 toolchain, but only the 64 bit version.

All the "standard" build configs produce both x86 and x86_64
toolchains - which makes sense, since it's not known what build host
the user is using and the AB needs to show both are buildable. In your
case, you know you have an x86_64 machine and don't want to bother
building for x86, so, it's seems like the thing to do is check for the
existence of a build package (x86 or x86_64) before attempting the
copy.

> [1] http://pastebin.com/aLyer2fi
> [2]
> https://github.com/RobertBerger/meta-mainline/blob/morty-training-v4.4.x/multi-v7-ml/yocto-autobuilder/res-custom-morty-multi-v7-core-image-minimal-dev-sato-sdk-toolchain.conf
>
> Regards,
>
> Robert

Re: Y-AB PublishArtifacts toolchain wrong dir

[gmane]

Hi,

On 2016-10-08 09:28, Bill Randle wrote:
> 
> All the "standard" build configs produce both x86 and x86_64
> toolchains - which makes sense, since it's not known what build host
> the user is using and the AB needs to show both are buildable. In your
> case, you know you have an x86_64 machine and don't want to bother
> building for x86, so, it's seems like the thing to do is check for the
> existence of a build package (x86 or x86_64) before attempting the
> copy.
> 

That's what I thought as well at first. So I introduced new toolchain 
artifacts like toolchain_x86_64 and toolchain_i686 where I did only the 
relevant thing in the relevant branch.
It did not solve my problem.

Regards,

Robert

Re: Y-AB PublishArtifacts toolchain wrong dir

[Bill Randle]

Hi,

I think I would leave the toolchain artifact as is, but just surround
the the 'cp' with an 'if' statement. E.g.,:
  if [ -e <path-to-x86-toolchain> ]; then
    cp <cp-args> <path-to-x86-toolchain> <destination>;
  fi;
  if [ -e <path-to-x86_64-toolchain> ]; then
    cp <cp-args> <path-to-x86_64-toolchain> <destination>;
  fi;

    -Bill

On Sat, Oct 8, 2016 at 2:52 PM,  <gmane@reliableembeddedsystems.com> wrote:
> Hi,
>
> On 2016-10-08 09:28, Bill Randle wrote:
>>
>>
>> All the "standard" build configs produce both x86 and x86_64
>> toolchains - which makes sense, since it's not known what build host
>> the user is using and the AB needs to show both are buildable. In your
>> case, you know you have an x86_64 machine and don't want to bother
>> building for x86, so, it's seems like the thing to do is check for the
>> existence of a build package (x86 or x86_64) before attempting the
>> copy.
>>
>
> That's what I thought as well at first. So I introduced new toolchain
> artifacts like toolchain_x86_64 and toolchain_i686 where I did only the
> relevant thing in the relevant branch.
> It did not solve my problem.
>
> Regards,
>
> Robert

Re: Y-AB PublishArtifacts toolchain wrong dir

[gmane]

Hi,

On 2016-10-08 18:17, Bill Randle wrote:
> Hi,
> 
> I think I would leave the toolchain artifact as is, but just surround
> the the 'cp' with an 'if' statement. E.g.,:
>   if [ -e <path-to-x86-toolchain> ]; then
>     cp <cp-args> <path-to-x86-toolchain> <destination>;
>   fi;
>   if [ -e <path-to-x86_64-toolchain> ]; then
>     cp <cp-args> <path-to-x86_64-toolchain> <destination>;
>   fi;
> 

This sounds like a brilliant idea. It looks like now I have the 
behaviour of the good old Y-AB plus md5sums and friends and being copied 
over.
What do you think about this patch? [1]


>     -Bill
> 

[1] 
https://github.com/RobertBerger/autobuilder-patches/blob/master/before-install/0004-deal-only-with-built-toolchains-cp-also-md5-and-mani.patch

Regards,

Robert

Re: Y-AB PublishArtifacts toolchain wrong dir

[Bill Randle]

On Sun, Oct 9, 2016 at 6:28 AM,  <gmane@reliableembeddedsystems.com> wrote:
> Hi,
>
> On 2016-10-08 18:17, Bill Randle wrote:
>>
>> Hi,
>>
>> I think I would leave the toolchain artifact as is, but just surround
>> the the 'cp' with an 'if' statement. E.g.,:
>>   if [ -e <path-to-x86-toolchain> ]; then
>>     cp <cp-args> <path-to-x86-toolchain> <destination>;
>>   fi;
>>   if [ -e <path-to-x86_64-toolchain> ]; then
>>     cp <cp-args> <path-to-x86_64-toolchain> <destination>;
>>   fi;
>>
>
> This sounds like a brilliant idea. It looks like now I have the behaviour of
> the good old Y-AB plus md5sums and friends and being copied over.
> What do you think about this patch? [1]

Much better. Now just move the mkdir into the same if block as the
copy like this (no point in checking the same thing twice):

+                    command=command+"if [ -e " +
os.path.join(deploy_image_dir, "poky-*i686-core-image*.sh ") + " ];
then " + \
+                                    "mkdir -p " + os.path.join(DEST,
X86TC_PUBLISH_DIR) + "; " + \
+                                    "cp -R --no-dereference
--preserve=links " + \
+                                    os.path.join(deploy_image_dir,
"poky-*i686-core-image* ") + \
+                                    os.path.join(DEST,
X86TC_PUBLISH_DIR) + pipeline + "; fi;"

and the same for the x86_64 case, of course.

Then submit to the mailing list in the standard format (look at other
patch submissions for examples) Also, include "[yocto-autobuilder]"
and file you are patching in the Subject line [1] and Cc:
joshua.g.lock@intel.com and pidge@toganlabs.com. Also, add a commit
message in the body of the email.

    -Bill

[1] E.g., Subject: [yocto-autobuilder][PATCH] PublishArtifacts.py:
deal only with built toolchains, cp also md5 and manifests

>
> [1]
> https://github.com/RobertBerger/autobuilder-patches/blob/master/before-install/0004-deal-only-with-built-toolchains-cp-also-md5-and-mani.patch
>
> Regards,
>
> Robert