* Problems with sstate-cache in Artifactory @ 2019-07-03 10:08 Ankur Tyagi 2019-07-03 12:04 ` Burton, Ross 0 siblings, 1 reply; 8+ messages in thread From: Ankur Tyagi @ 2019-07-03 10:08 UTC (permalink / raw) To: openembedded-core Hi, I am trying to setup sstate-cache in Artifactory so that multiple Linux host systems can share it. Once build finished in one Linux machine, I uploaded all of sstate-cache directory to Artifactory Then I configured sstate mirrors in local.conf SSTATE_MIRRORS ?= "\ file://.* https://artifactory.com/yocto/sstate-cache/PATH;downloadfilename=PATH \ " But when the build started, it didn't used anything from the sstate mirror and ended up building everything Am I missing something? Thanks Ankur ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Problems with sstate-cache in Artifactory 2019-07-03 10:08 Problems with sstate-cache in Artifactory Ankur Tyagi @ 2019-07-03 12:04 ` Burton, Ross 2019-07-04 2:40 ` Ankur Tyagi 0 siblings, 1 reply; 8+ messages in thread From: Burton, Ross @ 2019-07-03 12:04 UTC (permalink / raw) To: Ankur Tyagi; +Cc: openembedded-core Check the log for the a particular setscene task and it will show what requests it made, and if they failed. Ross On Wed, 3 Jul 2019 at 11:09, Ankur Tyagi <ankur.tyagi85@gmail.com> wrote: > > Hi, > > I am trying to setup sstate-cache in Artifactory so that multiple > Linux host systems can share it. > > Once build finished in one Linux machine, I uploaded all of > sstate-cache directory to Artifactory > > Then I configured sstate mirrors in local.conf > SSTATE_MIRRORS ?= "\ > file://.* https://artifactory.com/yocto/sstate-cache/PATH;downloadfilename=PATH > \ > " > But when the build started, it didn't used anything from the sstate > mirror and ended up building everything > > Am I missing something? > > Thanks > Ankur > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Problems with sstate-cache in Artifactory 2019-07-03 12:04 ` Burton, Ross @ 2019-07-04 2:40 ` Ankur Tyagi [not found] ` <CY4PR12MB165398EE661D4CA2995CD0558BF60@CY4PR12MB1653.namprd12.prod.outlook.com> 0 siblings, 1 reply; 8+ messages in thread From: Ankur Tyagi @ 2019-07-04 2:40 UTC (permalink / raw) To: Burton, Ross; +Cc: openembedded-core Hi Ross, I can't see any "log.do_setscene" inside the tmp directory. When I start an image build, it does say executing setscene task but then no setscene task log for any recipe Initialising tasks: 100% |############################################################################################################################################| Time: 0:00:01 Checking sstate mirror object availability: 100% |####################################################################################################################| Time: 0:00:02 Sstate summary: Wanted 484 Found 0 Missed 968 Current 0 (0% match, 0% complete) NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks Regards Ankur On Thu, Jul 4, 2019 at 12:04 AM Burton, Ross <ross.burton@intel.com> wrote: > > Check the log for the a particular setscene task and it will show what > requests it made, and if they failed. > > Ross > > On Wed, 3 Jul 2019 at 11:09, Ankur Tyagi <ankur.tyagi85@gmail.com> wrote: > > > > Hi, > > > > I am trying to setup sstate-cache in Artifactory so that multiple > > Linux host systems can share it. > > > > Once build finished in one Linux machine, I uploaded all of > > sstate-cache directory to Artifactory > > > > Then I configured sstate mirrors in local.conf > > SSTATE_MIRRORS ?= "\ > > file://.* https://artifactory.com/yocto/sstate-cache/PATH;downloadfilename=PATH > > \ > > " > > But when the build started, it didn't used anything from the sstate > > mirror and ended up building everything > > > > Am I missing something? > > > > Thanks > > Ankur > > -- > > _______________________________________________ > > Openembedded-core mailing list > > Openembedded-core@lists.openembedded.org > > http://lists.openembedded.org/mailman/listinfo/openembedded-core ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <CY4PR12MB165398EE661D4CA2995CD0558BF60@CY4PR12MB1653.namprd12.prod.outlook.com>]
* Re: Problems with sstate-cache in Artifactory [not found] ` <CY4PR12MB165398EE661D4CA2995CD0558BF60@CY4PR12MB1653.namprd12.prod.outlook.com> @ 2019-07-08 21:16 ` Ankur Tyagi 2019-07-09 3:45 ` Ankur Tyagi 0 siblings, 1 reply; 8+ messages in thread From: Ankur Tyagi @ 2019-07-08 21:16 UTC (permalink / raw) To: chris.laplante; +Cc: openembedded-core Hi Chris, Thanks for your pointers, I will try them. Regarding using curl for sstate fetch, how to set SSTATE_MIRRORS ? Below is my current configuration SSTATE_MIRRORS ?= "\ file://.* https://artifactory.com/yocto/sstate-cache/PATH;downloadfilename=PATH \ " Thanks Ankur On Tue, Jul 9, 2019 at 3:37 AM chris.laplante@agilent.com <chris.laplante@agilent.com> wrote: > > > I can't see any "log.do_setscene" inside the tmp directory. > > When I start an image build, it does say executing setscene task but > > then no setscene task log for any recipe > > > > Initialising tasks: 100% > > |############################################################################################################## > > ##############################| > > Time: 0:00:01 > > Checking sstate mirror object availability: 100% > > |############################################################################################################## > > ######| > > Time: 0:00:02 > > Sstate summary: Wanted 484 Found 0 Missed 968 Current 0 (0% match, 0% complete) > > NOTE: Executing SetScene Tasks > > NOTE: Executing RunQueue Tasks > > We have Artifactory serving our sstate as well, so here's some tips. > > 1. Try seeing if you can fetch some sstate using curl directly. If you are authenticating to the Artifactory server using a .netrc server, make sure to pass "--netrc" to curl. > 2. You can diagnose the sstate mirror by passing -DDD to bitbake. It will spit out a ton of debug messages, so you might want to redirect it all to a file. E.g.: > > bitbake something -DDD >out.txt 2>&1 > > Start looking after the line that begins "DEBUG: SState using premirror of". It will dump out the URLs that it tries to fetch. > > 3. Within Artifactory, you can check the access log by going to: Admin > System Logs > access.log > > Look for "DENIED LOGIN" errors. > > Chris ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Problems with sstate-cache in Artifactory 2019-07-08 21:16 ` Ankur Tyagi @ 2019-07-09 3:45 ` Ankur Tyagi 2019-07-09 14:28 ` chris.laplante 0 siblings, 1 reply; 8+ messages in thread From: Ankur Tyagi @ 2019-07-09 3:45 UTC (permalink / raw) To: chris.laplante; +Cc: openembedded-core Hi Chris, 1) I am able to fetch sstate using curl, wget 2) Looking in bitbake -DDD output, url is correct as I can use that url to download it using curl, wget DEBUG: SState: Attempting to fetch file://5c/sstate:u-boot-ti-staging:wsl-linux-gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz DEBUG: Searching for 5c/sstate:u-boot-ti-staging:wsl-linux-gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz in paths: /home/nws/wsl/arago/downstream/ggl-oe-layersetup/build/sstate-cache DEBUG: Defaulting to /home/nws/wsl/arago/downstream/ggl-oe-layersetup/build/sstate-cache/5c/sstate:u-boot-ti-staging:wsl-linux-gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz for 5c/sstate:u-boot-ti-staging:wsl-linux-gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz DEBUG: Testing URL file://5c/sstate:u-boot-ti-staging:wsl-linux-gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz DEBUG: For url ['file', '', '5c/sstate:u-boot-ti-staging:wsl-linux-gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz', '', '', OrderedDict()] comparing ['file', '', '.*', '', '', OrderedDict()] to ['https', 'artifactory.local.gallagher.io', '/artifactory/rnd-yocto-packages/sstate-cache/wsl-thud/PATH', '', '', OrderedDict([('downloadfilename', 'PATH')])] DEBUG: For url file://5c/sstate:u-boot-ti-staging:wsl-linux-gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz returning https://artifactory.local.gallagher.io/artifactory/rnd-yocto-packages/sstate-cache/wsl-thud/5c/sstate%3Au-boot-ti-staging%3Awsl-linux-gnueabi%3A2019.01%2BgitAUTOINC%2B06e0b5babe%3Ar17%3Awsl%3A3%3A5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz;downloadfilename=5c/sstate:u-boot-ti-staging:wsl-linux-gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz DEBUG: checkstatus: trying again DEBUG: checkstatus: trying again DEBUG: checkstatus: trying again DEBUG: checkstatus: trying again DEBUG: checkstatus() urlopen failed: HTTP Error 404: Not Found Regards Ankur On Tue, Jul 9, 2019 at 9:16 AM Ankur Tyagi <ankur.tyagi85@gmail.com> wrote: > > Hi Chris, > > Thanks for your pointers, I will try them. > > Regarding using curl for sstate fetch, how to set SSTATE_MIRRORS ? > Below is my current configuration > > SSTATE_MIRRORS ?= "\ > file://.* https://artifactory.com/yocto/sstate-cache/PATH;downloadfilename=PATH > \ > " > Thanks > Ankur > > On Tue, Jul 9, 2019 at 3:37 AM chris.laplante@agilent.com > <chris.laplante@agilent.com> wrote: > > > > > I can't see any "log.do_setscene" inside the tmp directory. > > > When I start an image build, it does say executing setscene task but > > > then no setscene task log for any recipe > > > > > > Initialising tasks: 100% > > > |############################################################################################################## > > > ##############################| > > > Time: 0:00:01 > > > Checking sstate mirror object availability: 100% > > > |############################################################################################################## > > > ######| > > > Time: 0:00:02 > > > Sstate summary: Wanted 484 Found 0 Missed 968 Current 0 (0% match, 0% complete) > > > NOTE: Executing SetScene Tasks > > > NOTE: Executing RunQueue Tasks > > > > We have Artifactory serving our sstate as well, so here's some tips. > > > > 1. Try seeing if you can fetch some sstate using curl directly. If you are authenticating to the Artifactory server using a .netrc server, make sure to pass "--netrc" to curl. > > 2. You can diagnose the sstate mirror by passing -DDD to bitbake. It will spit out a ton of debug messages, so you might want to redirect it all to a file. E.g.: > > > > bitbake something -DDD >out.txt 2>&1 > > > > Start looking after the line that begins "DEBUG: SState using premirror of". It will dump out the URLs that it tries to fetch. > > > > 3. Within Artifactory, you can check the access log by going to: Admin > System Logs > access.log > > > > Look for "DENIED LOGIN" errors. > > > > Chris ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Problems with sstate-cache in Artifactory 2019-07-09 3:45 ` Ankur Tyagi @ 2019-07-09 14:28 ` chris.laplante 2019-07-10 1:39 ` Ankur Tyagi 0 siblings, 1 reply; 8+ messages in thread From: chris.laplante @ 2019-07-09 14:28 UTC (permalink / raw) To: Ankur Tyagi; +Cc: openembedded-core > DEBUG: For url file://5c/sstate:u-boot-ti-staging:wsl-linux- > gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz > returning https://artifactory.local.gallagher.io/artifactory/rnd-yocto-packages/sstate-cache/wsl-thud/5c/sstate%3Au-boot-ti- > staging%3Awsl-linux- > gnueabi%3A2019.01%2BgitAUTOINC%2B06e0b5babe%3Ar17%3Awsl%3A3%3A5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz; > downloadfilename=5c/sstate:u-boot-ti-staging:wsl-linux- > gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz > DEBUG: checkstatus: trying again > DEBUG: checkstatus: trying again > DEBUG: checkstatus: trying again > DEBUG: checkstatus: trying again > DEBUG: checkstatus() urlopen failed: HTTP Error 404: Not Found Is this mapping correct? Does using curl to download https://artifactory.local.gallagher.io/artifactory/rnd-yocto-packages/sstate-cache/wsl-thud/5c/sstate%3Au-boot-ti-staging%3Awsl-linux-gnueabi%3A2019.01%2BgitAUTOINC%2B06e0b5babe%3Ar17%3Awsl%3A3%3A5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz;downloadfilename=5c/sstate:u-boot-ti-staging:wsl-linux-gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz work? Chris ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Problems with sstate-cache in Artifactory 2019-07-09 14:28 ` chris.laplante @ 2019-07-10 1:39 ` Ankur Tyagi 2019-07-10 9:22 ` Ankur Tyagi 0 siblings, 1 reply; 8+ messages in thread From: Ankur Tyagi @ 2019-07-10 1:39 UTC (permalink / raw) To: chris.laplante; +Cc: openembedded-core On Wed, Jul 10, 2019 at 2:28 AM chris.laplante@agilent.com <chris.laplante@agilent.com> wrote: > > > DEBUG: For url file://5c/sstate:u-boot-ti-staging:wsl-linux- > > gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz > > returning https://artifactory.local.gallagher.io/artifactory/rnd-yocto-packages/sstate-cache/wsl-thud/5c/sstate%3Au-boot-ti- > > staging%3Awsl-linux- > > gnueabi%3A2019.01%2BgitAUTOINC%2B06e0b5babe%3Ar17%3Awsl%3A3%3A5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz; > > downloadfilename=5c/sstate:u-boot-ti-staging:wsl-linux- > > gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz > > DEBUG: checkstatus: trying again > > DEBUG: checkstatus: trying again > > DEBUG: checkstatus: trying again > > DEBUG: checkstatus: trying again > > DEBUG: checkstatus() urlopen failed: HTTP Error 404: Not Found > > Is this mapping correct? Does using curl to download https://artifactory.local.gallagher.io/artifactory/rnd-yocto-packages/sstate-cache/wsl-thud/5c/sstate%3Au-boot-ti-staging%3Awsl-linux-gnueabi%3A2019.01%2BgitAUTOINC%2B06e0b5babe%3Ar17%3Awsl%3A3%3A5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz;downloadfilename=5c/sstate:u-boot-ti-staging:wsl-linux-gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz work? > Yes, curl/wget can download it. What wrong do you suspect here? Thanks Ankur > Chris ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Problems with sstate-cache in Artifactory 2019-07-10 1:39 ` Ankur Tyagi @ 2019-07-10 9:22 ` Ankur Tyagi 0 siblings, 0 replies; 8+ messages in thread From: Ankur Tyagi @ 2019-07-10 9:22 UTC (permalink / raw) To: chris.laplante; +Cc: openembedded-core Hi Chris, I tried uploading sstate-cache again in Artifactory but this time used jfrog-cli (https://www.jfrog.com/confluence/display/CLI/CLI+for+JFrog+Artifactory) instead of curl and now it is working fine. regards Ankur On Wed, Jul 10, 2019 at 1:39 PM Ankur Tyagi <ankur.tyagi85@gmail.com> wrote: > > On Wed, Jul 10, 2019 at 2:28 AM chris.laplante@agilent.com > <chris.laplante@agilent.com> wrote: > > > > > DEBUG: For url file://5c/sstate:u-boot-ti-staging:wsl-linux- > > > gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz > > > returning https://artifactory.local.gallagher.io/artifactory/rnd-yocto-packages/sstate-cache/wsl-thud/5c/sstate%3Au-boot-ti- > > > staging%3Awsl-linux- > > > gnueabi%3A2019.01%2BgitAUTOINC%2B06e0b5babe%3Ar17%3Awsl%3A3%3A5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz; > > > downloadfilename=5c/sstate:u-boot-ti-staging:wsl-linux- > > > gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz > > > DEBUG: checkstatus: trying again > > > DEBUG: checkstatus: trying again > > > DEBUG: checkstatus: trying again > > > DEBUG: checkstatus: trying again > > > DEBUG: checkstatus() urlopen failed: HTTP Error 404: Not Found > > > > Is this mapping correct? Does using curl to download https://artifactory.local.gallagher.io/artifactory/rnd-yocto-packages/sstate-cache/wsl-thud/5c/sstate%3Au-boot-ti-staging%3Awsl-linux-gnueabi%3A2019.01%2BgitAUTOINC%2B06e0b5babe%3Ar17%3Awsl%3A3%3A5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz;downloadfilename=5c/sstate:u-boot-ti-staging:wsl-linux-gnueabi:2019.01+gitAUTOINC+06e0b5babe:r17:wsl:3:5cdf9380b9b3f8c94a706344cf19554a_packagedata.tgz work? > > > > Yes, curl/wget can download it. What wrong do you suspect here? > > Thanks > Ankur > > > Chris ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2019-07-10 9:22 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-03 10:08 Problems with sstate-cache in Artifactory Ankur Tyagi
2019-07-03 12:04 ` Burton, Ross
2019-07-04 2:40 ` Ankur Tyagi
[not found] ` <CY4PR12MB165398EE661D4CA2995CD0558BF60@CY4PR12MB1653.namprd12.prod.outlook.com>
2019-07-08 21:16 ` Ankur Tyagi
2019-07-09 3:45 ` Ankur Tyagi
2019-07-09 14:28 ` chris.laplante
2019-07-10 1:39 ` Ankur Tyagi
2019-07-10 9:22 ` Ankur Tyagi
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.