Re: help needed: Top nightly failure

["Saran, Sajesh Kumar" <sajesh@ti.com>]

[-- Attachment #1: Type: text/plain, Size: 6694 bytes --]

This is a reproducible issue. I guess git cloning from arago-project is a problem.
I got another e-mail (attached) that cloning of linux-davinci is also have similar issue.

Regards,
Sajesh

-----Original Message-----
From: Cooper Jr., Franklin 
Sent: Monday, August 12, 2013 6:46 PM
To: Dmytriyenko, Denys; Nelson, Sam
Cc: meta-arago@arago-project.org; ares-team@list.ti.com - Raj Sivarajan's Team (May contain non-TIers)
Subject: [ARES] RE: [meta-arago] help needed: Top nightly failure

If this is blocking your release and your able to clone locally you could also workaround this issue by cloning it manually and then placing it in your downloads directory.
Based on your log bitbake is simply running the following command so you can run this command manually:
git clone --bare --mirror git://arago-project.org/git/projects/linux-tci6614.git /home/a0756824/sc_mcsdk/oe-layersetup-mcsdk/downloads/git2/arago-project.org.git.projects.linux-tci6614.git 



> -----Original Message-----
> From: meta-arago-bounces@arago-project.org [mailto:meta-arago- 
> bounces@arago-project.org] On Behalf Of Dmytriyenko, Denys
> Sent: Monday, August 12, 2013 5:31 PM
> To: Nelson, Sam
> Cc: meta-arago@arago-project.org; ares-team@list.ti.com - Raj 
> Sivarajan's Team (May contain non-TIers)
> Subject: Re: [meta-arago] help needed: Top nightly failure
> 
> Sam,
> 
> I have already mentioned many times that you should use a mirror...
> 
> I don't see any problems with the server and I'm cloning your trees 
> right now w/o problems so far. You may want to check your proxy setup as well.
> 
> BTW, did repacking the linux-keystone help for your nightlies? I can 
> do the same for linux-tci6614 too...
> 
> --
> Denys
> 
> 
> On Mon, Aug 12, 2013 at 06:22:11PM -0400, Nelson, Sam wrote:
> > Denys,
> > This is blocking the MCSDK 3.0.2 & SCMCSDK-2.2.0 releases and 
> > appreciate
> any
> > help in resolving this issue.
> >
> > With regards,
> > Sam
> >
> > -----Original Message-----
> > From: Nelson, Sam
> > Sent: Monday, August 12, 2013 12:27 PM
> > To: Dmytriyenko, Denys
> > Cc: 'meta-arago@arago-project.org'; Sivarajan, Raj; 
> > ares-team@list.ti.com -
> Raj Sivarajan's Team (May contain non-TIers)
> > Subject: RE: help needed: Top nightly failure
> >
> > Also  yocto builds using 
> > git://arago-project.org/git/projects/linux-tci6614.git
> also failing for the past 3 days.
> >
> > Looks like there is no issue in cloning the repos. But only the 
> > yocto fetch
> operation is failing.
> >
> > From the logs there is no information about what is the actual 
> > command ,
> which failed.
> > With regards,
> > Sam
> > -----Original Message-----
> > From: Nelson, Sam
> > Sent: Monday, August 12, 2013 10:57 AM
> > To: Dmytriyenko, Denys
> > Cc: meta-arago@arago-project.org; Sivarajan, Raj; 
> > ares-team@list.ti.com - Raj
> Sivarajan's Team (May contain non-TIers)
> > Subject: RE: help needed: Top nightly failure
> > Importance: High
> >
> > Hello Denys,
> > Now starting Thursday, our builds are failing now consistently.
> > Did something change in the repos?
> > WARNING: Failed to fetch URL 
> > git://arago-project.org/git/projects/linux-
> keystone.git;protocol=git;branch=current/master, attempting MIRRORS if 
> available
> > ERROR: Fetcher failure: Fetch command failed with exit code 128, output:
> > Cloning into bare repository '/home/a0862859/yocto/oe-layersetup-
> mcsdk/downloads/git2/arago-project.org.git.projects.linux-keystone.git'...
> >
> > fatal: The remote end hung up unexpectedly
> > fatal: early EOF
> > fatal: index-pack failed
> >
> > ERROR: Function failed: Fetcher failure for URL: 'git://arago-
> project.org/git/projects/linux-
> keystone.git;protocol=git;branch=current/master'. Unable to fetch URL 
> from any source.
> > ERROR: Logfile of failure stored in: 
> > /home/a0862859/yocto/oe-layersetup-
> mcsdk/build/arago-tmp-external-linaro-toolchain/work/keystone_evm-oe-
> linux-gnueabi/linux-keystone-3.8.4-r6a/temp/log.do_fetch.31058
> > ERROR: Task 684 (/home/a0862859/yocto/oe-layersetup-
> mcsdk/sources/meta-ti/recipes-kernel/linux/linux-keystone_3.8.4.bb, 
> do_fetch) failed with exit code '1'
> > NOTE: Tasks Summary: Attempted 3212 tasks of which 3211 didn't need 
> > to be
> rerun and 1 failed.
> > No currently running tasks (351 of 3340)
> >
> > With regards,
> > Sam
> >
> > -----Original Message-----
> > From: Dmytriyenko, Denys
> > Sent: Thursday, August 08, 2013 10:09 PM
> > To: Nelson, Sam
> > Cc: meta-arago@arago-project.org; Sivarajan, Raj
> > Subject: Re: help needed: Top nightly failure
> >
> > Sam,
> >
> > Please use a mirror and/or previously cloned repos from downloads/ 
> > directory
> - cloning the entire tree every night is not recommended and is rather abusive.
> >
> > Also, I've repacked the linux-keystone repository so it is now 
> > smaller - please
> let me know if you still see the problem.
> >
> > --
> > Denys
> >
> >
> > On Fri, Aug 02, 2013 at 06:20:08AM -0400, Nelson, Sam wrote:
> > >    Hello All
> > >
> > >
> > >
> > >    We are encountering a lot of nightly failures with fetch failure like the
> > >    following from the arago-project git.  The error happens randomly and
> > >    unpredictably.
> > >
> > >    Is this something to do with git servers going down/offline?
> > >
> > >    Any ideas in tracking this type of issue will be appreciated.
> > >
> > >
> > >
> > >    With regards,
> > >
> > >    Sam
> > >
> > >
> > >
> > >    ERROR: Fetcher failure: Fetch command failed with exit code 128, output:
> > >
> > >    Cloning into bare repository
> > >    '/home/gtbldadm/yocto/mcsdk-3_00_01_12/oe-layersetup-
> mcsdk/downloads/git2/arago-project.org.git.projects.linux-keystone.git'...
> > >
> > >
> > >
> > >    fatal: The remote end hung up unexpectedly
> > >
> > >    fatal: early EOF
> > >
> > >    fatal: index-pack failed
> > >
> > >
> > >
> > >    ERROR: Function failed: Fetcher failure for URL:
> > >    'git://arago-project.org/git/projects/linux-
> keystone.git;protocol=git;branch=july-2013b/master'.
> > >    Unable to fetch URL from any source.
> > >
> > >    ERROR: Logfile of failure stored in:
> > >    /home/gtbldadm/yocto/mcsdk-3_00_01_12/oe-layersetup-
> mcsdk/build/arago-tmp-external-linaro-toolchain/work/keystone_evm-oe-
> linux-gnueabi/linux-keystone-3.8.4-r6a/temp/log.do_fetch.14929
> > >
> > >
> _______________________________________________
> meta-arago mailing list
> meta-arago@arago-project.org
> http://arago-project.org/cgi-bin/mailman/listinfo/meta-arago

[-- Attachment #2: Type: message/rfc822, Size: 5117 bytes --]

From: "Wertz, Beverly" <beverly.wertz@ti.com>
To: "Zhang, Hao" <hzhang@ti.com>
Cc: "Nelson, Sam" <sam.nelson@ti.com>, "Saran, Sajesh Kumar" <sajesh@ti.com>
Subject: RE: MCSDK Build question
Date: Mon, 12 Aug 2013 22:46:35 +0000
Message-ID: <DA04BC11F659EF4E804F9788033FEB49D887AA@DFLE08.ent.ti.com>

Just to add, I’m also seeing a failure in cloning from arago-project.org.

ERROR: Function failed: Fetcher failure for URL: 'git://arago-project.org/git/projects/linux-davinci.git;protocol=git;branch=03.22.00.05'. Unable to fetch URL from any source.

This URL is failing locally too and I’ve had intermittent issues cloning from arago-project.org on a lab machine which I have serving up some boards for the OpenTest server.

The error I receive is identical to theirs:
beverlylocal@uda0220897a:~$ git clone git://arago-project.org/git/projects/linux-davinci.git
Cloning into 'linux-davinci'...
remote: Counting objects: 2797449, done.
remote: Compressing objects: 100% (442950/442950), done.
Receiving objects: 100% (2797449/2797449), 656.41 MiB | 1.50 MiB/s, done.
Resolving deltas: 100% (2327038/2327038), done.
fatal: The remote end hung up unexpectedly
fatal: error in sideband demultiplexer

-Beverly



From: Zhang, Hao
Sent: Monday, August 12, 2013 5:22 PM
To: Wertz, Beverly
Cc: Nelson, Sam; Saran, Sajesh Kumar
Subject: RE: MCSDK Build question

Looping Sam/Sajesh, who are facing similar issues during the past several days …

Beverly,

Here are our MCSDK 3.x Jenkins build proxy settings, they look matching with your settings.

export http_proxy="http://webproxy.ext.ti.com:80"
export ftp_proxy="http://webproxy.ext.ti.com:80"
export https_proxy="http://webproxy.ext.ti.com:80"
export no_proxy="ti.com"

Sam/Sajesh,

Are these fetching errors similar to what you observed in SC/MCSDK nightly?

Thanks,
Hao.

From: Wertz, Beverly
Sent: Monday, August 12, 2013 6:16 PM
To: Zhang, Hao
Subject: MCSDK Build question

Hao,

I was advised to reach out to you in regards to MCSDK scripts and an issue I was having with retrieving some files on the Jenkins VM build. Things seem to build just fine aside from our arago build, the issues I’m seeing appears to be fetching/firewall based. I’ve set the proxies as follows:


+ export http_proxy=http://webproxy.ext.ti.com:80<http://webproxy.ext.ti.com/>

+ export https_proxy=http://webproxy.ext.ti.com:80

+ export ftp_proxy=http://webproxy.ext.ti.com:80<http://webproxy.ext.ti.com/>

+ export no_proxy=.ti.com,localhost,127.0.0.0/8,172.24.0.0/16,local



But I’m still seeing fetch issues on the bitbake build. These files seem to be able to download locally and for our folks in India, is there anything wrong with my proxy setup to retrieve the files? Also, I’m able to retrieve file from other external sources so some sites get through the firewall settings without any issues. If you have any insight as to how you have resolved issues with your builds, we’d be happy to hear them.



-Beverly



Example issues:



ERROR: Function failed: Fetcher failure for URL: 'http://www.cpan.org/src/5.0/perl-5.14.2.tar.gz'. Unable to fetch URL from any source.

ERROR: Function failed: Fetcher failure for URL: 'ftp://ftp.gnu.org/gnu/gdb/gdb-7.5.tar.gz'. Unable to fetch URL from any source.

ERROR: Function failed: Fetcher failure for URL: 'http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2'. Unable to fetch URL from any source.

ERROR: Function failed: Fetcher failure for URL: 'ftp://ftp.gnu.org/gnu/gettext/gettext-0.18.1.1.tar.gz'. Unable to fetch URL from any source.

ERROR: Function failed: Fetcher failure for URL: 'ftp://ftp.gnu.org/gnu/binutils/binutils-2.20.1a.tar.bz2'. Unable to fetch URL from any source.

ERROR: Function failed: Fetcher failure for URL: 'http://download.oracle.com/berkeley-db/db-5.3.15.tar.gz'. Unable to fetch URL from any source.

ERROR: Function failed: Fetcher failure for URL: 'http://downloads.mysql.com/archives/mysql-5.1/mysql-5.1.40.tar.gz'. Unable to fetch URL from any source.




Beverly G. Wertz
DSP Software
Texas Instruments Inc.

Office:  214.479.3362