* Re: [review-request] 7414-cut the prefix from layer file paths for bitbake variables [not found] <011977994B5AE74ABA2BD08E983BAAABC1008B8C@ALA-MBA.corp.ad.wrs.com> @ 2015-03-13 14:16 ` Barros Pena, Belen 2015-03-16 14:14 ` Lerner, Dave 0 siblings, 1 reply; 11+ messages in thread From: Barros Pena, Belen @ 2015-03-13 14:16 UTC (permalink / raw) To: Lerner, David M (Wind River); +Cc: toaster Hi Dave, This is looking good. But I've realised I forgot about something. Not all files setting variables are inside a layer: some of them are configuration files inside the /build or /bitbake directories. I guess we should also cut those, showing only the path inside the root source directory. So something like /home/user/poky/build/conf/local.conf Would show as build/conf/local.conf I am not fully sure if this is the right thing to do, so I'd be interested in hearing what you and the others think. I also realise this was not explicitly explained in the original bug. If you want me to open a separate issue in Bugzilla, I can do so: just let me know. Thanks! Belén On 12/03/2015 15:48, "Lerner, Dave" <dave.lerner@windriver.com> wrote: >rebased. >Dave > >> -----Original Message----- >> From: Lerner, Dave >> Sent: Thursday, March 12, 2015 10:46 AM >> To: Barros Pena, Belen (belen.barros.pena@intel.com) >> Subject: RE: [review-request] 7414-cut the prefix from layer file paths >>for bitbake >> variables >> >> I don't think that I rebased (since Tuesday) before I pushed. Can you >>hold off pulling >> for 15 minutes? I'll send a note when rebased. >> Dave >> >> > -----Original Message----- >> > From: Lerner, Dave >> > Sent: Thursday, March 12, 2015 10:29 AM >> > To: Barros Pena, Belen (belen.barros.pena@intel.com); Damian, >>Alexandru >> > (alexandru.damian@intel.com) >> > Cc: toaster@yoctoproject.org >> > Subject: [review-request] 7414-cut the prefix from layer file paths >>for bitbake >> > variables >> > >> > >>http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/trim >>_varpath_7414 >> > >> > >> > Page Testing: >> > Create project using yocto master which includes openembedded-core, >>meta-yocto and >> meta- >> > yocto-bsp, then add layers meta-oe, meta-games. >> > Build the project. >> > >> > Open the build, configuration page, bitbake variables. >> > Note the following types of variable path names in the "Set in file" >>column and when >> > clicking the variable in the History. >> > >> > Variable: B >> > Set in file/history: meta/conf/bitbake.conf >> > >> > Variable: BBFILE_PATTERN_core >> > Set in file: full path to configure >> > History: first meta/conf/layer.conf, then >> > /path/_poky_master.toaster_cloned/bitbake/lib/bb/data_smart.py >> > >> > Variable: BBFILE_PATTTERN_games-layers >> > Set in file and History: >>meta-games_master.toaster_cloned/conf/layer.conf >> > >> > >> > HTML Validation: Pass >> > Dave Lerner ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [review-request] 7414-cut the prefix from layer file paths for bitbake variables 2015-03-13 14:16 ` [review-request] 7414-cut the prefix from layer file paths for bitbake variables Barros Pena, Belen @ 2015-03-16 14:14 ` Lerner, Dave 2015-03-16 15:41 ` Barros Pena, Belen 0 siblings, 1 reply; 11+ messages in thread From: Lerner, Dave @ 2015-03-16 14:14 UTC (permalink / raw) To: BARROS PENA, BELEN; +Cc: toaster Hi Belen, > -----Original Message----- > From: Barros Pena, Belen [mailto:belen.barros.pena@intel.com] > Sent: Friday, March 13, 2015 9:17 AM > To: Lerner, Dave > Cc: toaster@yoctoproject.org > Subject: Re: [review-request] 7414-cut the prefix from layer file paths for bitbake > variables > > Hi Dave, > > This is looking good. But I've realised I forgot about something. Not all > files setting variables are inside a layer: some of them are configuration > files inside the /build or /bitbake directories. > > I guess we should also cut those, showing only the path inside the root > source directory. So something like > > /home/user/poky/build/conf/local.conf > > Would show as > > build/conf/local.conf > > I am not fully sure if this is the right thing to do, so I'd be interested > in hearing what you and the others think. > > I also realise this was not explicitly explained in the original bug. If > you want me to open a separate issue in Bugzilla, I can do so: just let me > know. Please open a separate bug and give Alex the ok for 7414. Layer relative makes sense to me, active-build-directory-relative does not, too many questions "what is the relative to?" are raised. Dave > > Thanks! > > Belén > > On 12/03/2015 15:48, "Lerner, Dave" <dave.lerner@windriver.com> wrote: > > >rebased. > >Dave > > > >> -----Original Message----- > >> From: Lerner, Dave > >> Sent: Thursday, March 12, 2015 10:46 AM > >> To: Barros Pena, Belen (belen.barros.pena@intel.com) > >> Subject: RE: [review-request] 7414-cut the prefix from layer file paths > >>for bitbake > >> variables > >> > >> I don't think that I rebased (since Tuesday) before I pushed. Can you > >>hold off pulling > >> for 15 minutes? I'll send a note when rebased. > >> Dave > >> > >> > -----Original Message----- > >> > From: Lerner, Dave > >> > Sent: Thursday, March 12, 2015 10:29 AM > >> > To: Barros Pena, Belen (belen.barros.pena@intel.com); Damian, > >>Alexandru > >> > (alexandru.damian@intel.com) > >> > Cc: toaster@yoctoproject.org > >> > Subject: [review-request] 7414-cut the prefix from layer file paths > >>for bitbake > >> > variables > >> > > >> > > >>http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/trim > >>_varpath_7414 > >> > > >> > > >> > Page Testing: > >> > Create project using yocto master which includes openembedded-core, > >>meta-yocto and > >> meta- > >> > yocto-bsp, then add layers meta-oe, meta-games. > >> > Build the project. > >> > > >> > Open the build, configuration page, bitbake variables. > >> > Note the following types of variable path names in the "Set in file" > >>column and when > >> > clicking the variable in the History. > >> > > >> > Variable: B > >> > Set in file/history: meta/conf/bitbake.conf > >> > > >> > Variable: BBFILE_PATTERN_core > >> > Set in file: full path to configure > >> > History: first meta/conf/layer.conf, then > >> > /path/_poky_master.toaster_cloned/bitbake/lib/bb/data_smart.py > >> > > >> > Variable: BBFILE_PATTTERN_games-layers > >> > Set in file and History: > >>meta-games_master.toaster_cloned/conf/layer.conf > >> > > >> > > >> > HTML Validation: Pass > >> > Dave Lerner > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [review-request] 7414-cut the prefix from layer file paths for bitbake variables 2015-03-16 14:14 ` Lerner, Dave @ 2015-03-16 15:41 ` Barros Pena, Belen 2015-03-17 15:19 ` Damian, Alexandru 0 siblings, 1 reply; 11+ messages in thread From: Barros Pena, Belen @ 2015-03-16 15:41 UTC (permalink / raw) To: Lerner, David M (Wind River); +Cc: toaster On 16/03/2015 14:14, "Lerner, Dave" <dave.lerner@windriver.com> wrote: >Hi Belen, > >> -----Original Message----- >> From: Barros Pena, Belen [mailto:belen.barros.pena@intel.com] >> Sent: Friday, March 13, 2015 9:17 AM >> To: Lerner, Dave >> Cc: toaster@yoctoproject.org >> Subject: Re: [review-request] 7414-cut the prefix from layer file paths >>for bitbake >> variables >> >> Hi Dave, >> >> This is looking good. But I've realised I forgot about something. Not >>all >> files setting variables are inside a layer: some of them are >>configuration >> files inside the /build or /bitbake directories. >> >> I guess we should also cut those, showing only the path inside the root >> source directory. So something like >> >> /home/user/poky/build/conf/local.conf >> >> Would show as >> >> build/conf/local.conf >> >> I am not fully sure if this is the right thing to do, so I'd be >>interested >> in hearing what you and the others think. >> >> I also realise this was not explicitly explained in the original bug. If >> you want me to open a separate issue in Bugzilla, I can do so: just let >>me >> know. > >Please open a separate bug and give Alex the ok for 7414. Layer relative >makes sense to me, active-build-directory-relative does not, too many >questions "what is the relative to?" are raised. http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/trim_v arpath_7414 is good to go, then. I've opened this https://bugzilla.yoctoproject.org/show_bug.cgi?id=7463 for the configuration file paths. Cheers Belén > >Dave > >> >> Thanks! >> >> Belén >> >> On 12/03/2015 15:48, "Lerner, Dave" <dave.lerner@windriver.com> wrote: >> >> >rebased. >> >Dave >> > >> >> -----Original Message----- >> >> From: Lerner, Dave >> >> Sent: Thursday, March 12, 2015 10:46 AM >> >> To: Barros Pena, Belen (belen.barros.pena@intel.com) >> >> Subject: RE: [review-request] 7414-cut the prefix from layer file >>paths >> >>for bitbake >> >> variables >> >> >> >> I don't think that I rebased (since Tuesday) before I pushed. Can >>you >> >>hold off pulling >> >> for 15 minutes? I'll send a note when rebased. >> >> Dave >> >> >> >> > -----Original Message----- >> >> > From: Lerner, Dave >> >> > Sent: Thursday, March 12, 2015 10:29 AM >> >> > To: Barros Pena, Belen (belen.barros.pena@intel.com); Damian, >> >>Alexandru >> >> > (alexandru.damian@intel.com) >> >> > Cc: toaster@yoctoproject.org >> >> > Subject: [review-request] 7414-cut the prefix from layer file paths >> >>for bitbake >> >> > variables >> >> > >> >> > >> >>>>http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/tr >>>>im >> >>_varpath_7414 >> >> > >> >> > >> >> > Page Testing: >> >> > Create project using yocto master which includes openembedded-core, >> >>meta-yocto and >> >> meta- >> >> > yocto-bsp, then add layers meta-oe, meta-games. >> >> > Build the project. >> >> > >> >> > Open the build, configuration page, bitbake variables. >> >> > Note the following types of variable path names in the "Set in >>file" >> >>column and when >> >> > clicking the variable in the History. >> >> > >> >> > Variable: B >> >> > Set in file/history: meta/conf/bitbake.conf >> >> > >> >> > Variable: BBFILE_PATTERN_core >> >> > Set in file: full path to configure >> >> > History: first meta/conf/layer.conf, then >> >> > /path/_poky_master.toaster_cloned/bitbake/lib/bb/data_smart.py >> >> > >> >> > Variable: BBFILE_PATTTERN_games-layers >> >> > Set in file and History: >> >>meta-games_master.toaster_cloned/conf/layer.conf >> >> > >> >> > >> >> > HTML Validation: Pass >> >> > Dave Lerner >> > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [review-request] 7414-cut the prefix from layer file paths for bitbake variables 2015-03-16 15:41 ` Barros Pena, Belen @ 2015-03-17 15:19 ` Damian, Alexandru 2015-03-17 15:45 ` Lerner, Dave 2015-03-18 15:48 ` Lerner, Dave 0 siblings, 2 replies; 11+ messages in thread From: Damian, Alexandru @ 2015-03-17 15:19 UTC (permalink / raw) To: Barros Pena, Belen; +Cc: toaster [-- Attachment #1: Type: text/plain, Size: 5098 bytes --] Hi, Sorry for the significant delay in addressing this patch - I've been a bit overwhelmed. I am NOT taking this patch because of the way the layer matching is done. We already have local paths for layers present in the build in the Layer object; we also know _where_ the build took place, for we have "build.buildrequest.buildenvironment.builddir" variable that holds the path to the build directory. We must verify that the file path starts with one of the layer paths or the build environment builddir path, and strip that leading path from the full file path. I would also expect this to be done at data logging time instead of data display time (i.e. in the buildinfohelper.py), with the distinct advantage of speeding up page processing time. Thank you, Alex On Mon, Mar 16, 2015 at 3:41 PM, Barros Pena, Belen < belen.barros.pena@intel.com> wrote: > > > On 16/03/2015 14:14, "Lerner, Dave" <dave.lerner@windriver.com> wrote: > > >Hi Belen, > > > >> -----Original Message----- > >> From: Barros Pena, Belen [mailto:belen.barros.pena@intel.com] > >> Sent: Friday, March 13, 2015 9:17 AM > >> To: Lerner, Dave > >> Cc: toaster@yoctoproject.org > >> Subject: Re: [review-request] 7414-cut the prefix from layer file paths > >>for bitbake > >> variables > >> > >> Hi Dave, > >> > >> This is looking good. But I've realised I forgot about something. Not > >>all > >> files setting variables are inside a layer: some of them are > >>configuration > >> files inside the /build or /bitbake directories. > >> > >> I guess we should also cut those, showing only the path inside the root > >> source directory. So something like > >> > >> /home/user/poky/build/conf/local.conf > >> > >> Would show as > >> > >> build/conf/local.conf > >> > >> I am not fully sure if this is the right thing to do, so I'd be > >>interested > >> in hearing what you and the others think. > >> > >> I also realise this was not explicitly explained in the original bug. If > >> you want me to open a separate issue in Bugzilla, I can do so: just let > >>me > >> know. > > > >Please open a separate bug and give Alex the ok for 7414. Layer relative > >makes sense to me, active-build-directory-relative does not, too many > >questions "what is the relative to?" are raised. > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/trim_v > arpath_7414 > > is good to go, then. I've opened this > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=7463 > > for the configuration file paths. > > Cheers > > Belén > > > > > >Dave > > > >> > >> Thanks! > >> > >> Belén > >> > >> On 12/03/2015 15:48, "Lerner, Dave" <dave.lerner@windriver.com> wrote: > >> > >> >rebased. > >> >Dave > >> > > >> >> -----Original Message----- > >> >> From: Lerner, Dave > >> >> Sent: Thursday, March 12, 2015 10:46 AM > >> >> To: Barros Pena, Belen (belen.barros.pena@intel.com) > >> >> Subject: RE: [review-request] 7414-cut the prefix from layer file > >>paths > >> >>for bitbake > >> >> variables > >> >> > >> >> I don't think that I rebased (since Tuesday) before I pushed. Can > >>you > >> >>hold off pulling > >> >> for 15 minutes? I'll send a note when rebased. > >> >> Dave > >> >> > >> >> > -----Original Message----- > >> >> > From: Lerner, Dave > >> >> > Sent: Thursday, March 12, 2015 10:29 AM > >> >> > To: Barros Pena, Belen (belen.barros.pena@intel.com); Damian, > >> >>Alexandru > >> >> > (alexandru.damian@intel.com) > >> >> > Cc: toaster@yoctoproject.org > >> >> > Subject: [review-request] 7414-cut the prefix from layer file paths > >> >>for bitbake > >> >> > variables > >> >> > > >> >> > > >> > >>>> > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/tr > >>>>im > >> >>_varpath_7414 > >> >> > > >> >> > > >> >> > Page Testing: > >> >> > Create project using yocto master which includes openembedded-core, > >> >>meta-yocto and > >> >> meta- > >> >> > yocto-bsp, then add layers meta-oe, meta-games. > >> >> > Build the project. > >> >> > > >> >> > Open the build, configuration page, bitbake variables. > >> >> > Note the following types of variable path names in the "Set in > >>file" > >> >>column and when > >> >> > clicking the variable in the History. > >> >> > > >> >> > Variable: B > >> >> > Set in file/history: meta/conf/bitbake.conf > >> >> > > >> >> > Variable: BBFILE_PATTERN_core > >> >> > Set in file: full path to configure > >> >> > History: first meta/conf/layer.conf, then > >> >> > /path/_poky_master.toaster_cloned/bitbake/lib/bb/data_smart.py > >> >> > > >> >> > Variable: BBFILE_PATTTERN_games-layers > >> >> > Set in file and History: > >> >>meta-games_master.toaster_cloned/conf/layer.conf > >> >> > > >> >> > > >> >> > HTML Validation: Pass > >> >> > Dave Lerner > >> > > > > -- > _______________________________________________ > toaster mailing list > toaster@yoctoproject.org > https://lists.yoctoproject.org/listinfo/toaster > -- Alex Damian Yocto Project SSG / OTC [-- Attachment #2: Type: text/html, Size: 8415 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [review-request] 7414-cut the prefix from layer file paths for bitbake variables 2015-03-17 15:19 ` Damian, Alexandru @ 2015-03-17 15:45 ` Lerner, Dave 2015-03-17 15:51 ` Damian, Alexandru 2015-03-18 15:48 ` Lerner, Dave 1 sibling, 1 reply; 11+ messages in thread From: Lerner, Dave @ 2015-03-17 15:45 UTC (permalink / raw) To: DAMIAN, ALEXANDRU, BARROS PENA, BELEN; +Cc: toaster Hi Alex, Since you want this to be handled in the backend, I am reassigning the bug to you and changing the state to NEW. Dave > -----Original Message----- > From: Damian, Alexandru [mailto:alexandru.damian@intel.com] > Sent: Tuesday, March 17, 2015 10:19 AM > To: BARROS PENA, BELEN > Cc: Lerner, Dave; toaster@yoctoproject.org > Subject: Re: [Toaster] [review-request] 7414-cut the prefix from layer file paths for > bitbake variables > > Hi, > > > Sorry for the significant delay in addressing this patch - I've been a bit overwhelmed. > > > I am NOT taking this patch because of the way the layer matching is done. > > > We already have local paths for layers present in the build in the Layer object; we also > know _where_ the build took place, for we have > "build.buildrequest.buildenvironment.builddir" variable that holds the path to the build > directory. > > We must verify that the file path starts with one of the layer paths or the build > environment builddir path, and strip that leading path from the full file path. > > > I would also expect this to be done at data logging time instead of data display time > (i.e. in the buildinfohelper.py), with the distinct advantage of speeding up page > processing time. > > > Thank you, > > Alex > > > On Mon, Mar 16, 2015 at 3:41 PM, Barros Pena, Belen <belen.barros.pena@intel.com> wrote: > > > > > On 16/03/2015 14:14, "Lerner, Dave" <dave.lerner@windriver.com> wrote: > > >Hi Belen, > > > >> -----Original Message----- > >> From: Barros Pena, Belen [mailto:belen.barros.pena@intel.com] > >> Sent: Friday, March 13, 2015 9:17 AM > >> To: Lerner, Dave > >> Cc: toaster@yoctoproject.org > >> Subject: Re: [review-request] 7414-cut the prefix from layer file paths > >>for bitbake > >> variables > >> > >> Hi Dave, > >> > >> This is looking good. But I've realised I forgot about something. Not > >>all > >> files setting variables are inside a layer: some of them are > >>configuration > >> files inside the /build or /bitbake directories. > >> > >> I guess we should also cut those, showing only the path inside the root > >> source directory. So something like > >> > >> /home/user/poky/build/conf/local.conf > >> > >> Would show as > >> > >> build/conf/local.conf > >> > >> I am not fully sure if this is the right thing to do, so I'd be > >>interested > >> in hearing what you and the others think. > >> > >> I also realise this was not explicitly explained in the original bug. If > >> you want me to open a separate issue in Bugzilla, I can do so: just let > >>me > >> know. > > > >Please open a separate bug and give Alex the ok for 7414. Layer relative > >makes sense to me, active-build-directory-relative does not, too many > >questions "what is the relative to?" are raised. > > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/trim_v > arpath_7414 <http://git.yoctoproject.org/cgit/cgit.cgi/poky- > contrib/log/?h=lerner/trim_v arpath_7414> > > is good to go, then. I've opened this > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=7463 > > for the configuration file paths. > > Cheers > > Belén > > > > > > >Dave > > > >> > >> Thanks! > >> > >> Belén > >> > >> On 12/03/2015 15:48, "Lerner, Dave" <dave.lerner@windriver.com> wrote: > >> > >> >rebased. > >> >Dave > >> > > >> >> -----Original Message----- > >> >> From: Lerner, Dave > >> >> Sent: Thursday, March 12, 2015 10:46 AM > >> >> To: Barros Pena, Belen (belen.barros.pena@intel.com) > >> >> Subject: RE: [review-request] 7414-cut the prefix from layer file > >>paths > >> >>for bitbake > >> >> variables > >> >> > >> >> I don't think that I rebased (since Tuesday) before I pushed. Can > >>you > >> >>hold off pulling > >> >> for 15 minutes? I'll send a note when rebased. > >> >> Dave > >> >> > >> >> > -----Original Message----- > >> >> > From: Lerner, Dave > >> >> > Sent: Thursday, March 12, 2015 10:29 AM > >> >> > To: Barros Pena, Belen (belen.barros.pena@intel.com); Damian, > >> >>Alexandru > >> >> > (alexandru.damian@intel.com) > >> >> > Cc: toaster@yoctoproject.org > >> >> > Subject: [review-request] 7414-cut the prefix from layer file paths > >> >>for bitbake > >> >> > variables > >> >> > > >> >> > > >> > >>>>http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/tr > >>>>im > >> >>_varpath_7414 > >> >> > > >> >> > > >> >> > Page Testing: > >> >> > Create project using yocto master which includes openembedded-core, > >> >>meta-yocto and > >> >> meta- > >> >> > yocto-bsp, then add layers meta-oe, meta-games. > >> >> > Build the project. > >> >> > > >> >> > Open the build, configuration page, bitbake variables. > >> >> > Note the following types of variable path names in the "Set in > >>file" > >> >>column and when > >> >> > clicking the variable in the History. > >> >> > > >> >> > Variable: B > >> >> > Set in file/history: meta/conf/bitbake.conf > >> >> > > >> >> > Variable: BBFILE_PATTERN_core > >> >> > Set in file: full path to configure > >> >> > History: first meta/conf/layer.conf, then > >> >> > /path/_poky_master.toaster_cloned/bitbake/lib/bb/data_smart.py > >> >> > > >> >> > Variable: BBFILE_PATTTERN_games-layers > >> >> > Set in file and History: > >> >>meta-games_master.toaster_cloned/conf/layer.conf > >> >> > > >> >> > > >> >> > HTML Validation: Pass > >> >> > Dave Lerner > >> > > > > -- > _______________________________________________ > toaster mailing list > toaster@yoctoproject.org > https://lists.yoctoproject.org/listinfo/toaster > > > > > > -- > > Alex Damian > Yocto Project > > SSG / OTC ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [review-request] 7414-cut the prefix from layer file paths for bitbake variables 2015-03-17 15:45 ` Lerner, Dave @ 2015-03-17 15:51 ` Damian, Alexandru 0 siblings, 0 replies; 11+ messages in thread From: Damian, Alexandru @ 2015-03-17 15:51 UTC (permalink / raw) To: Lerner, David M (Wind River); +Cc: toaster [-- Attachment #1: Type: text/plain, Size: 7166 bytes --] This is fine ! :) Cheers, Alex On Tue, Mar 17, 2015 at 3:45 PM, Lerner, David M (Wind River) < dave.lerner@windriver.com> wrote: > Hi Alex, > Since you want this to be handled in the backend, I am reassigning the bug > to you and changing the state to NEW. > Dave > > > -----Original Message----- > > From: Damian, Alexandru [mailto:alexandru.damian@intel.com] > > Sent: Tuesday, March 17, 2015 10:19 AM > > To: BARROS PENA, BELEN > > Cc: Lerner, Dave; toaster@yoctoproject.org > > Subject: Re: [Toaster] [review-request] 7414-cut the prefix from layer > file paths for > > bitbake variables > > > > Hi, > > > > > > Sorry for the significant delay in addressing this patch - I've been a > bit overwhelmed. > > > > > > I am NOT taking this patch because of the way the layer matching is done. > > > > > > We already have local paths for layers present in the build in the Layer > object; we also > > know _where_ the build took place, for we have > > "build.buildrequest.buildenvironment.builddir" variable that holds the > path to the build > > directory. > > > > We must verify that the file path starts with one of the layer paths or > the build > > environment builddir path, and strip that leading path from the full > file path. > > > > > > I would also expect this to be done at data logging time instead of data > display time > > (i.e. in the buildinfohelper.py), with the distinct advantage of > speeding up page > > processing time. > > > > > > Thank you, > > > > Alex > > > > > > On Mon, Mar 16, 2015 at 3:41 PM, Barros Pena, Belen < > belen.barros.pena@intel.com> wrote: > > > > > > > > > > On 16/03/2015 14:14, "Lerner, Dave" <dave.lerner@windriver.com> > wrote: > > > > >Hi Belen, > > > > > >> -----Original Message----- > > >> From: Barros Pena, Belen [mailto:belen.barros.pena@intel.com] > > >> Sent: Friday, March 13, 2015 9:17 AM > > >> To: Lerner, Dave > > >> Cc: toaster@yoctoproject.org > > >> Subject: Re: [review-request] 7414-cut the prefix from layer > file paths > > >>for bitbake > > >> variables > > >> > > >> Hi Dave, > > >> > > >> This is looking good. But I've realised I forgot about > something. Not > > >>all > > >> files setting variables are inside a layer: some of them are > > >>configuration > > >> files inside the /build or /bitbake directories. > > >> > > >> I guess we should also cut those, showing only the path inside > the root > > >> source directory. So something like > > >> > > >> /home/user/poky/build/conf/local.conf > > >> > > >> Would show as > > >> > > >> build/conf/local.conf > > >> > > >> I am not fully sure if this is the right thing to do, so I'd be > > >>interested > > >> in hearing what you and the others think. > > >> > > >> I also realise this was not explicitly explained in the > original bug. If > > >> you want me to open a separate issue in Bugzilla, I can do so: > just let > > >>me > > >> know. > > > > > >Please open a separate bug and give Alex the ok for 7414. Layer > relative > > >makes sense to me, active-build-directory-relative does not, too > many > > >questions "what is the relative to?" are raised. > > > > > > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/trim_v > > arpath_7414 <http://git.yoctoproject.org/cgit/cgit.cgi/poky- > > contrib/log/?h=lerner/trim_v arpath_7414> > > > > is good to go, then. I've opened this > > > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=7463 > > > > for the configuration file paths. > > > > Cheers > > > > Belén > > > > > > > > > > > >Dave > > > > > >> > > >> Thanks! > > >> > > >> Belén > > >> > > >> On 12/03/2015 15:48, "Lerner, Dave" <dave.lerner@windriver.com> > wrote: > > >> > > >> >rebased. > > >> >Dave > > >> > > > >> >> -----Original Message----- > > >> >> From: Lerner, Dave > > >> >> Sent: Thursday, March 12, 2015 10:46 AM > > >> >> To: Barros Pena, Belen (belen.barros.pena@intel.com) > > >> >> Subject: RE: [review-request] 7414-cut the prefix from layer > file > > >>paths > > >> >>for bitbake > > >> >> variables > > >> >> > > >> >> I don't think that I rebased (since Tuesday) before I > pushed. Can > > >>you > > >> >>hold off pulling > > >> >> for 15 minutes? I'll send a note when rebased. > > >> >> Dave > > >> >> > > >> >> > -----Original Message----- > > >> >> > From: Lerner, Dave > > >> >> > Sent: Thursday, March 12, 2015 10:29 AM > > >> >> > To: Barros Pena, Belen (belen.barros.pena@intel.com); > Damian, > > >> >>Alexandru > > >> >> > (alexandru.damian@intel.com) > > >> >> > Cc: toaster@yoctoproject.org > > >> >> > Subject: [review-request] 7414-cut the prefix from layer > file paths > > >> >>for bitbake > > >> >> > variables > > >> >> > > > >> >> > > > >> > > >>>> > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/tr > > >>>>im > > >> >>_varpath_7414 > > >> >> > > > >> >> > > > >> >> > Page Testing: > > >> >> > Create project using yocto master which includes > openembedded-core, > > >> >>meta-yocto and > > >> >> meta- > > >> >> > yocto-bsp, then add layers meta-oe, meta-games. > > >> >> > Build the project. > > >> >> > > > >> >> > Open the build, configuration page, bitbake variables. > > >> >> > Note the following types of variable path names in the > "Set in > > >>file" > > >> >>column and when > > >> >> > clicking the variable in the History. > > >> >> > > > >> >> > Variable: B > > >> >> > Set in file/history: meta/conf/bitbake.conf > > >> >> > > > >> >> > Variable: BBFILE_PATTERN_core > > >> >> > Set in file: full path to configure > > >> >> > History: first meta/conf/layer.conf, then > > >> >> > > /path/_poky_master.toaster_cloned/bitbake/lib/bb/data_smart.py > > >> >> > > > >> >> > Variable: BBFILE_PATTTERN_games-layers > > >> >> > Set in file and History: > > >> >>meta-games_master.toaster_cloned/conf/layer.conf > > >> >> > > > >> >> > > > >> >> > HTML Validation: Pass > > >> >> > Dave Lerner > > >> > > > > > > > -- > > _______________________________________________ > > toaster mailing list > > toaster@yoctoproject.org > > https://lists.yoctoproject.org/listinfo/toaster > > > > > > > > > > > > -- > > > > Alex Damian > > Yocto Project > > > > SSG / OTC > -- Alex Damian Yocto Project SSG / OTC [-- Attachment #2: Type: text/html, Size: 11516 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [review-request] 7414-cut the prefix from layer file paths for bitbake variables 2015-03-17 15:19 ` Damian, Alexandru 2015-03-17 15:45 ` Lerner, Dave @ 2015-03-18 15:48 ` Lerner, Dave 2015-03-24 14:28 ` Damian, Alexandru 1 sibling, 1 reply; 11+ messages in thread From: Lerner, Dave @ 2015-03-18 15:48 UTC (permalink / raw) To: DAMIAN, ALEXANDRU, BARROS PENA, BELEN; +Cc: toaster Alex, I've thought about this some more... see below. Dave > -----Original Message----- > From: Damian, Alexandru [mailto:alexandru.damian@intel.com] > Sent: Tuesday, March 17, 2015 10:19 AM > To: BARROS PENA, BELEN > Cc: Lerner, Dave; toaster@yoctoproject.org > Subject: Re: [Toaster] [review-request] 7414-cut the prefix from layer file paths for > bitbake variables > > Hi, > > > Sorry for the significant delay in addressing this patch - I've been a bit overwhelmed. It's not a significant delay and the delay is not a problem. > > > I am NOT taking this patch because of the way the layer matching is done. > > > We already have local paths for layers present in the build in the Layer object; we also > know _where_ the build took place, for we have > "build.buildrequest.buildenvironment.builddir" variable that holds the path to the build > directory. Either there is a bug, or your claim isn't quite right regarding orm_layer.local_paths which are all empty or null for a yocto release build. That was my first choice examining the database, but not possible since data was missing. > We must verify that the file path starts with one of the layer paths or the build > environment builddir path, and strip that leading path from the full file path. > I agree that would make more sense, but that data isn't in the table, and I believe due to performance reasons, that it should be done at display-a-page time. > I would also expect this to be done at data logging time instead of data display time > (i.e. in the buildinfohelper.py), with the distinct advantage of speeding up page > processing time. Since build-time layer path extraction for the given database structures penalizes all build times for the low likelihood that some user wants the path to a layer, I would recommend adding just a little more processing time to the data logging as follows: a) Constraining Layer.local_path to non-null, and always populating that field, b) normalizing things a bit by creating a new Model Config which has: b1) the full path that was formerly in the (non-normalized) VariableHistory.file_name and b2) a foreign key to from Config to it's Layer, calculated once c) modifying (normalizing) orm_variablehistory c1) removing orm_variablehistory.file_name and c2) replacing that with a foreign key to it's defining Config model. Then display time could just join on demand to find the config file's layer's local path whenever the last bitbake path or history is required. Or else provide the display-time matching against a non-null Layer.file_name, which would be much faster since the set of layers is of order 10^ and the list to match is at most 150 variables(per page), still my first choice. Dave > > Thank you, > > Alex > My recommendations are: 1) to populate the Layer model with local_path in all cases, changing the local_path to NOT NULL 2) perform the trim of variable names at display time, which should be faster than penalizing builds with Config model updates, or variable defined file stripping for every new config file read. OR) 3) add the Config model, update each variable log with the key to the Config model. > > On Mon, Mar 16, 2015 at 3:41 PM, Barros Pena, Belen <belen.barros.pena@intel.com> wrote: > > > > > On 16/03/2015 14:14, "Lerner, Dave" <dave.lerner@windriver.com> wrote: > > >Hi Belen, > > > >> -----Original Message----- > >> From: Barros Pena, Belen [mailto:belen.barros.pena@intel.com] > >> Sent: Friday, March 13, 2015 9:17 AM > >> To: Lerner, Dave > >> Cc: toaster@yoctoproject.org > >> Subject: Re: [review-request] 7414-cut the prefix from layer file paths > >>for bitbake > >> variables > >> > >> Hi Dave, > >> > >> This is looking good. But I've realised I forgot about something. Not > >>all > >> files setting variables are inside a layer: some of them are > >>configuration > >> files inside the /build or /bitbake directories. > >> > >> I guess we should also cut those, showing only the path inside the root > >> source directory. So something like > >> > >> /home/user/poky/build/conf/local.conf > >> > >> Would show as > >> > >> build/conf/local.conf > >> > >> I am not fully sure if this is the right thing to do, so I'd be > >>interested > >> in hearing what you and the others think. > >> > >> I also realise this was not explicitly explained in the original bug. If > >> you want me to open a separate issue in Bugzilla, I can do so: just let > >>me > >> know. > > > >Please open a separate bug and give Alex the ok for 7414. Layer relative > >makes sense to me, active-build-directory-relative does not, too many > >questions "what is the relative to?" are raised. > > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/trim_v > arpath_7414 <http://git.yoctoproject.org/cgit/cgit.cgi/poky- > contrib/log/?h=lerner/trim_v arpath_7414> > > is good to go, then. I've opened this > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=7463 > > for the configuration file paths. > > Cheers > > Belén > > > > > > >Dave > > > >> > >> Thanks! > >> > >> Belén > >> > >> On 12/03/2015 15:48, "Lerner, Dave" <dave.lerner@windriver.com> wrote: > >> > >> >rebased. > >> >Dave > >> > > >> >> -----Original Message----- > >> >> From: Lerner, Dave > >> >> Sent: Thursday, March 12, 2015 10:46 AM > >> >> To: Barros Pena, Belen (belen.barros.pena@intel.com) > >> >> Subject: RE: [review-request] 7414-cut the prefix from layer file > >>paths > >> >>for bitbake > >> >> variables > >> >> > >> >> I don't think that I rebased (since Tuesday) before I pushed. Can > >>you > >> >>hold off pulling > >> >> for 15 minutes? I'll send a note when rebased. > >> >> Dave > >> >> > >> >> > -----Original Message----- > >> >> > From: Lerner, Dave > >> >> > Sent: Thursday, March 12, 2015 10:29 AM > >> >> > To: Barros Pena, Belen (belen.barros.pena@intel.com); Damian, > >> >>Alexandru > >> >> > (alexandru.damian@intel.com) > >> >> > Cc: toaster@yoctoproject.org > >> >> > Subject: [review-request] 7414-cut the prefix from layer file paths > >> >>for bitbake > >> >> > variables > >> >> > > >> >> > > >> > >>>>http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/tr > >>>>im > >> >>_varpath_7414 > >> >> > > >> >> > > >> >> > Page Testing: > >> >> > Create project using yocto master which includes openembedded-core, > >> >>meta-yocto and > >> >> meta- > >> >> > yocto-bsp, then add layers meta-oe, meta-games. > >> >> > Build the project. > >> >> > > >> >> > Open the build, configuration page, bitbake variables. > >> >> > Note the following types of variable path names in the "Set in > >>file" > >> >>column and when > >> >> > clicking the variable in the History. > >> >> > > >> >> > Variable: B > >> >> > Set in file/history: meta/conf/bitbake.conf > >> >> > > >> >> > Variable: BBFILE_PATTERN_core > >> >> > Set in file: full path to configure > >> >> > History: first meta/conf/layer.conf, then > >> >> > /path/_poky_master.toaster_cloned/bitbake/lib/bb/data_smart.py > >> >> > > >> >> > Variable: BBFILE_PATTTERN_games-layers > >> >> > Set in file and History: > >> >>meta-games_master.toaster_cloned/conf/layer.conf > >> >> > > >> >> > > >> >> > HTML Validation: Pass > >> >> > Dave Lerner > >> > > > > -- > _______________________________________________ > toaster mailing list > toaster@yoctoproject.org > https://lists.yoctoproject.org/listinfo/toaster > > > > > > -- > > Alex Damian > Yocto Project > > SSG / OTC ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [review-request] 7414-cut the prefix from layer file paths for bitbake variables 2015-03-18 15:48 ` Lerner, Dave @ 2015-03-24 14:28 ` Damian, Alexandru 2015-03-24 19:44 ` Lerner, Dave 0 siblings, 1 reply; 11+ messages in thread From: Damian, Alexandru @ 2015-03-24 14:28 UTC (permalink / raw) To: Lerner, David M (Wind River); +Cc: toaster [-- Attachment #1: Type: text/plain, Size: 10325 bytes --] Hi, After thinking about the issue, if we mangle at build time, we'll have trouble distinguishing between paths originating in the layer directories and paths in the build directory. So the good solution is that we mangle the path at display time, and I will be taking this patch. I have some more comments below, Cheers, Alex On Wed, Mar 18, 2015 at 3:48 PM, Lerner, David M (Wind River) < dave.lerner@windriver.com> wrote: > Alex, > I've thought about this some more... see below. > Dave > > > -----Original Message----- > > From: Damian, Alexandru [mailto:alexandru.damian@intel.com] > > Sent: Tuesday, March 17, 2015 10:19 AM > > To: BARROS PENA, BELEN > > Cc: Lerner, Dave; toaster@yoctoproject.org > > Subject: Re: [Toaster] [review-request] 7414-cut the prefix from layer > file paths for > > bitbake variables > > > > Hi, > > > > > > Sorry for the significant delay in addressing this patch - I've been a > bit overwhelmed. > > It's not a significant delay and the delay is not a problem. > > > > > > > I am NOT taking this patch because of the way the layer matching is done. > > > > > > We already have local paths for layers present in the build in the Layer > object; we also > > know _where_ the build took place, for we have > > "build.buildrequest.buildenvironment.builddir" variable that holds the > path to the build > > directory. > > Either there is a bug, or your claim isn't quite right regarding > orm_layer.local_paths which are all empty or null for a yocto release > build. That was my first choice examining the database, but not possible > since data was missing. > I should've been more clear than this. The layers which HAVE BEEN built have local_path set to a correct value. The layers that we know about because they are imported manually or from Layer Index, the local_path is NULL. The variables are always attached to a build, so the layers on which the variables are modified will have local_path set to something else than NULL. > > > > We must verify that the file path starts with one of the layer paths or > the build > > environment builddir path, and strip that leading path from the full > file path. > > > > I agree that would make more sense, but that data isn't in the table, and > I believe due to performance reasons, that it should be done at > display-a-page time. > > > I would also expect this to be done at data logging time instead of data > display time > > (i.e. in the buildinfohelper.py), with the distinct advantage of > speeding up page > > processing time. > > > Since build-time layer path extraction for the given database structures > penalizes all build times for the low likelihood that some user wants the > path to a layer, I would recommend adding just a little more processing > time to the data logging as follows: > > a) Constraining Layer.local_path to non-null, and always populating that > field, > This is impossible due to the fact that we keep info about layers that have not been checked-out in that table. > b) normalizing things a bit by creating a new Model Config which has: > b1) the full path that was formerly in the (non-normalized) > VariableHistory.file_name and > b2) a foreign key to from Config to it's Layer, calculated once > c) modifying (normalizing) orm_variablehistory > c1) removing orm_variablehistory.file_name and > c2) replacing that with a foreign key to it's defining Config model. > Yes, this makes sense. Still it wouldn't work 100% because some variables are modified outside any Layer checkout (i.e. in the build directory). Having foreign keys to different objects to re-compute the path in the "Config" model would be unworkable in the queries sense. > > Then display time could just join on demand to find the config file's > layer's local path whenever the last bitbake path or history is required. > > Or else provide the display-time matching against a non-null > Layer.file_name, which would be much faster since the set of layers is of > order 10^ and the list to match is at most 150 variables(per page), still > my first choice. > > Dave > > > > > Thank you, > > > > Alex > > > > My recommendations are: > 1) to populate the Layer model with local_path in all cases, changing the > local_path to NOT NULL > 2) perform the trim of variable names at display time, which should be > faster than penalizing builds with Config model updates, or variable > defined file stripping for every new config file read. > OR) > 3) add the Config model, update each variable log with the key to the > Config model. > > > > > > On Mon, Mar 16, 2015 at 3:41 PM, Barros Pena, Belen < > belen.barros.pena@intel.com> wrote: > > > > > > > > > > On 16/03/2015 14:14, "Lerner, Dave" <dave.lerner@windriver.com> > wrote: > > > > >Hi Belen, > > > > > >> -----Original Message----- > > >> From: Barros Pena, Belen [mailto:belen.barros.pena@intel.com] > > >> Sent: Friday, March 13, 2015 9:17 AM > > >> To: Lerner, Dave > > >> Cc: toaster@yoctoproject.org > > >> Subject: Re: [review-request] 7414-cut the prefix from layer > file paths > > >>for bitbake > > >> variables > > >> > > >> Hi Dave, > > >> > > >> This is looking good. But I've realised I forgot about > something. Not > > >>all > > >> files setting variables are inside a layer: some of them are > > >>configuration > > >> files inside the /build or /bitbake directories. > > >> > > >> I guess we should also cut those, showing only the path inside > the root > > >> source directory. So something like > > >> > > >> /home/user/poky/build/conf/local.conf > > >> > > >> Would show as > > >> > > >> build/conf/local.conf > > >> > > >> I am not fully sure if this is the right thing to do, so I'd be > > >>interested > > >> in hearing what you and the others think. > > >> > > >> I also realise this was not explicitly explained in the > original bug. If > > >> you want me to open a separate issue in Bugzilla, I can do so: > just let > > >>me > > >> know. > > > > > >Please open a separate bug and give Alex the ok for 7414. Layer > relative > > >makes sense to me, active-build-directory-relative does not, too > many > > >questions "what is the relative to?" are raised. > > > > > > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/trim_v > > arpath_7414 <http://git.yoctoproject.org/cgit/cgit.cgi/poky- > > contrib/log/?h=lerner/trim_v arpath_7414> > > > > is good to go, then. I've opened this > > > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=7463 > > > > for the configuration file paths. > > > > Cheers > > > > Belén > > > > > > > > > > > >Dave > > > > > >> > > >> Thanks! > > >> > > >> Belén > > >> > > >> On 12/03/2015 15:48, "Lerner, Dave" <dave.lerner@windriver.com> > wrote: > > >> > > >> >rebased. > > >> >Dave > > >> > > > >> >> -----Original Message----- > > >> >> From: Lerner, Dave > > >> >> Sent: Thursday, March 12, 2015 10:46 AM > > >> >> To: Barros Pena, Belen (belen.barros.pena@intel.com) > > >> >> Subject: RE: [review-request] 7414-cut the prefix from layer > file > > >>paths > > >> >>for bitbake > > >> >> variables > > >> >> > > >> >> I don't think that I rebased (since Tuesday) before I > pushed. Can > > >>you > > >> >>hold off pulling > > >> >> for 15 minutes? I'll send a note when rebased. > > >> >> Dave > > >> >> > > >> >> > -----Original Message----- > > >> >> > From: Lerner, Dave > > >> >> > Sent: Thursday, March 12, 2015 10:29 AM > > >> >> > To: Barros Pena, Belen (belen.barros.pena@intel.com); > Damian, > > >> >>Alexandru > > >> >> > (alexandru.damian@intel.com) > > >> >> > Cc: toaster@yoctoproject.org > > >> >> > Subject: [review-request] 7414-cut the prefix from layer > file paths > > >> >>for bitbake > > >> >> > variables > > >> >> > > > >> >> > > > >> > > >>>> > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/tr > > >>>>im > > >> >>_varpath_7414 > > >> >> > > > >> >> > > > >> >> > Page Testing: > > >> >> > Create project using yocto master which includes > openembedded-core, > > >> >>meta-yocto and > > >> >> meta- > > >> >> > yocto-bsp, then add layers meta-oe, meta-games. > > >> >> > Build the project. > > >> >> > > > >> >> > Open the build, configuration page, bitbake variables. > > >> >> > Note the following types of variable path names in the > "Set in > > >>file" > > >> >>column and when > > >> >> > clicking the variable in the History. > > >> >> > > > >> >> > Variable: B > > >> >> > Set in file/history: meta/conf/bitbake.conf > > >> >> > > > >> >> > Variable: BBFILE_PATTERN_core > > >> >> > Set in file: full path to configure > > >> >> > History: first meta/conf/layer.conf, then > > >> >> > > /path/_poky_master.toaster_cloned/bitbake/lib/bb/data_smart.py > > >> >> > > > >> >> > Variable: BBFILE_PATTTERN_games-layers > > >> >> > Set in file and History: > > >> >>meta-games_master.toaster_cloned/conf/layer.conf > > >> >> > > > >> >> > > > >> >> > HTML Validation: Pass > > >> >> > Dave Lerner > > >> > > > > > > > -- > > _______________________________________________ > > toaster mailing list > > toaster@yoctoproject.org > > https://lists.yoctoproject.org/listinfo/toaster > > > > > > > > > > > > -- > > > > Alex Damian > > Yocto Project > > > > SSG / OTC > -- Alex Damian Yocto Project SSG / OTC [-- Attachment #2: Type: text/html, Size: 15702 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [review-request] 7414-cut the prefix from layer file paths for bitbake variables 2015-03-24 14:28 ` Damian, Alexandru @ 2015-03-24 19:44 ` Lerner, Dave 2015-03-25 13:04 ` Damian, Alexandru 0 siblings, 1 reply; 11+ messages in thread From: Lerner, Dave @ 2015-03-24 19:44 UTC (permalink / raw) To: DAMIAN, ALEXANDRU; +Cc: toaster Hi Alex, I'm not sure if you want me to take back ownership and mark it Resolved/Fixed or if you are just suggesting a different alternative than my patch and will handle it. Dave > -----Original Message----- > From: Damian, Alexandru [mailto:alexandru.damian@intel.com] > Sent: Tuesday, March 24, 2015 10:29 AM > To: Lerner, Dave > Cc: BARROS PENA, BELEN; toaster@yoctoproject.org > Subject: Re: [Toaster] [review-request] 7414-cut the prefix from layer file paths for > bitbake variables > > Hi, > > > After thinking about the issue, if we mangle at build time, we'll have trouble > distinguishing between paths originating in the layer directories and paths in the build > directory. So the good solution is that we mangle the path at display time, and I will > be taking this patch. > > > I have some more comments below, > > > Cheers, > > Alex > > > On Wed, Mar 18, 2015 at 3:48 PM, Lerner, David M (Wind River) > <dave.lerner@windriver.com> wrote: > > > Alex, > I've thought about this some more... see below. > Dave > > > -----Original Message----- > > From: Damian, Alexandru [mailto:alexandru.damian@intel.com] > > Sent: Tuesday, March 17, 2015 10:19 AM > > To: BARROS PENA, BELEN > > Cc: Lerner, Dave; toaster@yoctoproject.org > > Subject: Re: [Toaster] [review-request] 7414-cut the prefix from layer file paths > for > > bitbake variables > > > > Hi, > > > > > > Sorry for the significant delay in addressing this patch - I've been a bit > overwhelmed. > > It's not a significant delay and the delay is not a problem. > > > > > > > I am NOT taking this patch because of the way the layer matching is done. > > > > > > We already have local paths for layers present in the build in the Layer object; > we also > > know _where_ the build took place, for we have > > "build.buildrequest.buildenvironment.builddir" variable that holds the path to > the build > > directory. > > Either there is a bug, or your claim isn't quite right regarding > orm_layer.local_paths which are all empty or null for a yocto release build. That was > my first choice examining the database, but not possible since data was missing. > > > > I should've been more clear than this. The layers which HAVE BEEN built have local_path > set to a correct value. The layers that we know about because they are imported manually > or from Layer Index, the local_path is NULL. The variables are always attached to a > build, so the layers on which the variables are modified will have local_path set to > something else than NULL. > > > > > > > > We must verify that the file path starts with one of the layer paths or the build > > environment builddir path, and strip that leading path from the full file path. > > > > I agree that would make more sense, but that data isn't in the table, and I believe > due to performance reasons, that it should be done at display-a-page time. > > > I would also expect this to be done at data logging time instead of data display > time > > (i.e. in the buildinfohelper.py), with the distinct advantage of speeding up page > > processing time. > > > Since build-time layer path extraction for the given database structures penalizes > all build times for the low likelihood that some user wants the path to a layer, I would > recommend adding just a little more processing time to the data logging as follows: > > a) Constraining Layer.local_path to non-null, and always populating that field, > > > > This is impossible due to the fact that we keep info about layers that have not been > checked-out in that table. > > > > > b) normalizing things a bit by creating a new Model Config which has: > b1) the full path that was formerly in the (non-normalized) > VariableHistory.file_name and > b2) a foreign key to from Config to it's Layer, calculated once > c) modifying (normalizing) orm_variablehistory > c1) removing orm_variablehistory.file_name and > c2) replacing that with a foreign key to it's defining Config model. > > > > > Yes, this makes sense. Still it wouldn't work 100% because some variables are modified > outside any Layer checkout (i.e. in the build directory). Having foreign keys to > different objects to re-compute the path in the "Config" model would be unworkable in > the queries sense. > > > > > Then display time could just join on demand to find the config file's layer's local > path whenever the last bitbake path or history is required. > > Or else provide the display-time matching against a non-null Layer.file_name, which > would be much faster since the set of layers is of order 10^ and the list to match is at > most 150 variables(per page), still my first choice. > > Dave > > > > > Thank you, > > > > Alex > > > > My recommendations are: > 1) to populate the Layer model with local_path in all cases, changing the > local_path to NOT NULL > 2) perform the trim of variable names at display time, which should be faster than > penalizing builds with Config model updates, or variable defined file stripping for > every new config file read. > OR) > 3) add the Config model, update each variable log with the key to the Config model. > > > > > > > On Mon, Mar 16, 2015 at 3:41 PM, Barros Pena, Belen <belen.barros.pena@intel.com> > wrote: > > > > > > > > > > On 16/03/2015 14:14, "Lerner, Dave" <dave.lerner@windriver.com> wrote: > > > > >Hi Belen, > > > > > >> -----Original Message----- > > >> From: Barros Pena, Belen [mailto:belen.barros.pena@intel.com] > > >> Sent: Friday, March 13, 2015 9:17 AM > > >> To: Lerner, Dave > > >> Cc: toaster@yoctoproject.org > > >> Subject: Re: [review-request] 7414-cut the prefix from layer file paths > > >>for bitbake > > >> variables > > >> > > >> Hi Dave, > > >> > > >> This is looking good. But I've realised I forgot about something. Not > > >>all > > >> files setting variables are inside a layer: some of them are > > >>configuration > > >> files inside the /build or /bitbake directories. > > >> > > >> I guess we should also cut those, showing only the path inside the root > > >> source directory. So something like > > >> > > >> /home/user/poky/build/conf/local.conf > > >> > > >> Would show as > > >> > > >> build/conf/local.conf > > >> > > >> I am not fully sure if this is the right thing to do, so I'd be > > >>interested > > >> in hearing what you and the others think. > > >> > > >> I also realise this was not explicitly explained in the original bug. If > > >> you want me to open a separate issue in Bugzilla, I can do so: just let > > >>me > > >> know. > > > > > >Please open a separate bug and give Alex the ok for 7414. Layer relative > > >makes sense to me, active-build-directory-relative does not, too many > > >questions "what is the relative to?" are raised. > > > > > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/trim_v > > > arpath_7414 <http://git.yoctoproject.org/cgit/cgit.cgi/poky- > > contrib/log/?h=lerner/trim_v arpath_7414> > > > > > is good to go, then. I've opened this > > > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=7463 > > > > for the configuration file paths. > > > > Cheers > > > > Belén > > > > > > > > > > > >Dave > > > > > >> > > >> Thanks! > > >> > > >> Belén > > >> > > >> On 12/03/2015 15:48, "Lerner, Dave" <dave.lerner@windriver.com> wrote: > > >> > > >> >rebased. > > >> >Dave > > >> > > > >> >> -----Original Message----- > > >> >> From: Lerner, Dave > > >> >> Sent: Thursday, March 12, 2015 10:46 AM > > >> >> To: Barros Pena, Belen (belen.barros.pena@intel.com) > > >> >> Subject: RE: [review-request] 7414-cut the prefix from layer file > > >>paths > > >> >>for bitbake > > >> >> variables > > >> >> > > >> >> I don't think that I rebased (since Tuesday) before I pushed. Can > > >>you > > >> >>hold off pulling > > >> >> for 15 minutes? I'll send a note when rebased. > > >> >> Dave > > >> >> > > >> >> > -----Original Message----- > > >> >> > From: Lerner, Dave > > >> >> > Sent: Thursday, March 12, 2015 10:29 AM > > >> >> > To: Barros Pena, Belen (belen.barros.pena@intel.com); Damian, > > >> >>Alexandru > > >> >> > (alexandru.damian@intel.com) > > >> >> > Cc: toaster@yoctoproject.org > > >> >> > Subject: [review-request] 7414-cut the prefix from layer file paths > > >> >>for bitbake > > >> >> > variables > > >> >> > > > >> >> > > > >> > > >>>>http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/tr > > >>>>im > > >> >>_varpath_7414 > > >> >> > > > >> >> > > > >> >> > Page Testing: > > >> >> > Create project using yocto master which includes openembedded-core, > > >> >>meta-yocto and > > >> >> meta- > > >> >> > yocto-bsp, then add layers meta-oe, meta-games. > > >> >> > Build the project. > > >> >> > > > >> >> > Open the build, configuration page, bitbake variables. > > >> >> > Note the following types of variable path names in the "Set in > > >>file" > > >> >>column and when > > >> >> > clicking the variable in the History. > > >> >> > > > >> >> > Variable: B > > >> >> > Set in file/history: meta/conf/bitbake.conf > > >> >> > > > >> >> > Variable: BBFILE_PATTERN_core > > >> >> > Set in file: full path to configure > > >> >> > History: first meta/conf/layer.conf, then > > >> >> > /path/_poky_master.toaster_cloned/bitbake/lib/bb/data_smart.py > > >> >> > > > >> >> > Variable: BBFILE_PATTTERN_games-layers > > >> >> > Set in file and History: > > >> >>meta-games_master.toaster_cloned/conf/layer.conf > > >> >> > > > >> >> > > > >> >> > HTML Validation: Pass > > >> >> > Dave Lerner > > >> > > > > > > > -- > > _______________________________________________ > > toaster mailing list > > toaster@yoctoproject.org > > https://lists.yoctoproject.org/listinfo/toaster > > > > > > > > > > > > -- > > > > Alex Damian > > Yocto Project > > > > SSG / OTC > > > > > > -- > > Alex Damian > Yocto Project > > SSG / OTC ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [review-request] 7414-cut the prefix from layer file paths for bitbake variables 2015-03-24 19:44 ` Lerner, Dave @ 2015-03-25 13:04 ` Damian, Alexandru 0 siblings, 0 replies; 11+ messages in thread From: Damian, Alexandru @ 2015-03-25 13:04 UTC (permalink / raw) To: Lerner, David M (Wind River); +Cc: toaster [-- Attachment #1: Type: text/plain, Size: 13279 bytes --] I've submitted your patch upstream after testing. Please wait until it's merged, and then mark Resolved/Fixed. Alex On Tue, Mar 24, 2015 at 7:44 PM, Lerner, David M (Wind River) < dave.lerner@windriver.com> wrote: > Hi Alex, > I'm not sure if you want me to take back ownership and mark it > Resolved/Fixed or if you are just suggesting a different alternative than > my patch and will handle it. > Dave > > > -----Original Message----- > > From: Damian, Alexandru [mailto:alexandru.damian@intel.com] > > Sent: Tuesday, March 24, 2015 10:29 AM > > To: Lerner, Dave > > Cc: BARROS PENA, BELEN; toaster@yoctoproject.org > > Subject: Re: [Toaster] [review-request] 7414-cut the prefix from layer > file paths for > > bitbake variables > > > > Hi, > > > > > > After thinking about the issue, if we mangle at build time, we'll have > trouble > > distinguishing between paths originating in the layer directories and > paths in the build > > directory. So the good solution is that we mangle the path at display > time, and I will > > be taking this patch. > > > > > > I have some more comments below, > > > > > > Cheers, > > > > Alex > > > > > > On Wed, Mar 18, 2015 at 3:48 PM, Lerner, David M (Wind River) > > <dave.lerner@windriver.com> wrote: > > > > > > Alex, > > I've thought about this some more... see below. > > Dave > > > > > -----Original Message----- > > > From: Damian, Alexandru [mailto:alexandru.damian@intel.com] > > > Sent: Tuesday, March 17, 2015 10:19 AM > > > To: BARROS PENA, BELEN > > > Cc: Lerner, Dave; toaster@yoctoproject.org > > > Subject: Re: [Toaster] [review-request] 7414-cut the prefix from > layer file paths > > for > > > bitbake variables > > > > > > Hi, > > > > > > > > > Sorry for the significant delay in addressing this patch - I've > been a bit > > overwhelmed. > > > > It's not a significant delay and the delay is not a problem. > > > > > > > > > > > I am NOT taking this patch because of the way the layer matching > is done. > > > > > > > > > We already have local paths for layers present in the build in > the Layer object; > > we also > > > know _where_ the build took place, for we have > > > "build.buildrequest.buildenvironment.builddir" variable that > holds the path to > > the build > > > directory. > > > > Either there is a bug, or your claim isn't quite right regarding > > orm_layer.local_paths which are all empty or null for a yocto release > build. That was > > my first choice examining the database, but not possible since data was > missing. > > > > > > > > I should've been more clear than this. The layers which HAVE BEEN > built have local_path > > set to a correct value. The layers that we know about because they are > imported manually > > or from Layer Index, the local_path is NULL. The variables are always > attached to a > > build, so the layers on which the variables are modified will have > local_path set to > > something else than NULL. > > > > > > > > > > > > > > > We must verify that the file path starts with one of the layer > paths or the build > > > environment builddir path, and strip that leading path from the > full file path. > > > > > > > I agree that would make more sense, but that data isn't in the > table, and I believe > > due to performance reasons, that it should be done at display-a-page > time. > > > > > I would also expect this to be done at data logging time instead > of data display > > time > > > (i.e. in the buildinfohelper.py), with the distinct advantage of > speeding up page > > > processing time. > > > > > > Since build-time layer path extraction for the given database > structures penalizes > > all build times for the low likelihood that some user wants the path to > a layer, I would > > recommend adding just a little more processing time to the data logging > as follows: > > > > a) Constraining Layer.local_path to non-null, and always > populating that field, > > > > > > > > This is impossible due to the fact that we keep info about layers that > have not been > > checked-out in that table. > > > > > > > > > > b) normalizing things a bit by creating a new Model Config which > has: > > b1) the full path that was formerly in the (non-normalized) > > VariableHistory.file_name and > > b2) a foreign key to from Config to it's Layer, calculated once > > c) modifying (normalizing) orm_variablehistory > > c1) removing orm_variablehistory.file_name and > > c2) replacing that with a foreign key to it's defining Config > model. > > > > > > > > > > Yes, this makes sense. Still it wouldn't work 100% because some > variables are modified > > outside any Layer checkout (i.e. in the build directory). Having foreign > keys to > > different objects to re-compute the path in the "Config" model would be > unworkable in > > the queries sense. > > > > > > > > > > Then display time could just join on demand to find the config > file's layer's local > > path whenever the last bitbake path or history is required. > > > > Or else provide the display-time matching against a non-null > Layer.file_name, which > > would be much faster since the set of layers is of order 10^ and the > list to match is at > > most 150 variables(per page), still my first choice. > > > > Dave > > > > > > > > Thank you, > > > > > > Alex > > > > > > > My recommendations are: > > 1) to populate the Layer model with local_path in all cases, > changing the > > local_path to NOT NULL > > 2) perform the trim of variable names at display time, which > should be faster than > > penalizing builds with Config model updates, or variable defined file > stripping for > > every new config file read. > > OR) > > 3) add the Config model, update each variable log with the key to > the Config model. > > > > > > > > > > > > On Mon, Mar 16, 2015 at 3:41 PM, Barros Pena, Belen < > belen.barros.pena@intel.com> > > wrote: > > > > > > > > > > > > > > > On 16/03/2015 14:14, "Lerner, Dave" < > dave.lerner@windriver.com> wrote: > > > > > > >Hi Belen, > > > > > > > >> -----Original Message----- > > > >> From: Barros Pena, Belen [mailto: > belen.barros.pena@intel.com] > > > >> Sent: Friday, March 13, 2015 9:17 AM > > > >> To: Lerner, Dave > > > >> Cc: toaster@yoctoproject.org > > > >> Subject: Re: [review-request] 7414-cut the prefix from > layer file paths > > > >>for bitbake > > > >> variables > > > >> > > > >> Hi Dave, > > > >> > > > >> This is looking good. But I've realised I forgot about > something. Not > > > >>all > > > >> files setting variables are inside a layer: some of > them are > > > >>configuration > > > >> files inside the /build or /bitbake directories. > > > >> > > > >> I guess we should also cut those, showing only the path > inside the root > > > >> source directory. So something like > > > >> > > > >> /home/user/poky/build/conf/local.conf > > > >> > > > >> Would show as > > > >> > > > >> build/conf/local.conf > > > >> > > > >> I am not fully sure if this is the right thing to do, > so I'd be > > > >>interested > > > >> in hearing what you and the others think. > > > >> > > > >> I also realise this was not explicitly explained in the > original bug. If > > > >> you want me to open a separate issue in Bugzilla, I can > do so: just let > > > >>me > > > >> know. > > > > > > > >Please open a separate bug and give Alex the ok for > 7414. Layer relative > > > >makes sense to me, active-build-directory-relative does > not, too many > > > >questions "what is the relative to?" are raised. > > > > > > > > > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/trim_v > > > > > arpath_7414 < > http://git.yoctoproject.org/cgit/cgit.cgi/poky- > > > contrib/log/?h=lerner/trim_v arpath_7414> > > > > > > > > is good to go, then. I've opened this > > > > > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=7463 > > > > > > for the configuration file paths. > > > > > > Cheers > > > > > > Belén > > > > > > > > > > > > > > > > >Dave > > > > > > > >> > > > >> Thanks! > > > >> > > > >> Belén > > > >> > > > >> On 12/03/2015 15:48, "Lerner, Dave" < > dave.lerner@windriver.com> wrote: > > > >> > > > >> >rebased. > > > >> >Dave > > > >> > > > > >> >> -----Original Message----- > > > >> >> From: Lerner, Dave > > > >> >> Sent: Thursday, March 12, 2015 10:46 AM > > > >> >> To: Barros Pena, Belen (belen.barros.pena@intel.com) > > > >> >> Subject: RE: [review-request] 7414-cut the prefix > from layer file > > > >>paths > > > >> >>for bitbake > > > >> >> variables > > > >> >> > > > >> >> I don't think that I rebased (since Tuesday) before > I pushed. Can > > > >>you > > > >> >>hold off pulling > > > >> >> for 15 minutes? I'll send a note when rebased. > > > >> >> Dave > > > >> >> > > > >> >> > -----Original Message----- > > > >> >> > From: Lerner, Dave > > > >> >> > Sent: Thursday, March 12, 2015 10:29 AM > > > >> >> > To: Barros Pena, Belen ( > belen.barros.pena@intel.com); Damian, > > > >> >>Alexandru > > > >> >> > (alexandru.damian@intel.com) > > > >> >> > Cc: toaster@yoctoproject.org > > > >> >> > Subject: [review-request] 7414-cut the prefix from > layer file paths > > > >> >>for bitbake > > > >> >> > variables > > > >> >> > > > > >> >> > > > > >> > > > >>>> > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/tr > > > >>>>im > > > >> >>_varpath_7414 > > > >> >> > > > > >> >> > > > > >> >> > Page Testing: > > > >> >> > Create project using yocto master which includes > openembedded-core, > > > >> >>meta-yocto and > > > >> >> meta- > > > >> >> > yocto-bsp, then add layers meta-oe, meta-games. > > > >> >> > Build the project. > > > >> >> > > > > >> >> > Open the build, configuration page, bitbake > variables. > > > >> >> > Note the following types of variable path names in > the "Set in > > > >>file" > > > >> >>column and when > > > >> >> > clicking the variable in the History. > > > >> >> > > > > >> >> > Variable: B > > > >> >> > Set in file/history: meta/conf/bitbake.conf > > > >> >> > > > > >> >> > Variable: BBFILE_PATTERN_core > > > >> >> > Set in file: full path to configure > > > >> >> > History: first meta/conf/layer.conf, then > > > >> >> > > /path/_poky_master.toaster_cloned/bitbake/lib/bb/data_smart.py > > > >> >> > > > > >> >> > Variable: BBFILE_PATTTERN_games-layers > > > >> >> > Set in file and History: > > > >> >>meta-games_master.toaster_cloned/conf/layer.conf > > > >> >> > > > > >> >> > > > > >> >> > HTML Validation: Pass > > > >> >> > Dave Lerner > > > >> > > > > > > > > > > -- > > > _______________________________________________ > > > toaster mailing list > > > toaster@yoctoproject.org > > > https://lists.yoctoproject.org/listinfo/toaster > > > > > > > > > > > > > > > > > > -- > > > > > > Alex Damian > > > Yocto Project > > > > > > SSG / OTC > > > > > > > > > > > > -- > > > > Alex Damian > > Yocto Project > > > > SSG / OTC > -- Alex Damian Yocto Project SSG / OTC [-- Attachment #2: Type: text/html, Size: 19806 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* [review-request] 7414-cut the prefix from layer file paths for bitbake variables @ 2015-03-12 15:28 Lerner, Dave 0 siblings, 0 replies; 11+ messages in thread From: Lerner, Dave @ 2015-03-12 15:28 UTC (permalink / raw) To: BARROS PENA, BELEN, DAMIAN, ALEXANDRU; +Cc: toaster http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/trim_varpath_7414 Page Testing: Create project using yocto master which includes openembedded-core, meta-yocto and meta-yocto-bsp, then add layers meta-oe, meta-games. Build the project. Open the build, configuration page, bitbake variables. Note the following types of variable path names in the "Set in file" column and when clicking the variable in the History. Variable: B Set in file/history: meta/conf/bitbake.conf Variable: BBFILE_PATTERN_core Set in file: full path to configure History: first meta/conf/layer.conf, then /path/_poky_master.toaster_cloned/bitbake/lib/bb/data_smart.py Variable: BBFILE_PATTTERN_games-layers Set in file and History: meta-games_master.toaster_cloned/conf/layer.conf HTML Validation: Pass Dave Lerner ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2015-03-25 13:04 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <011977994B5AE74ABA2BD08E983BAAABC1008B8C@ALA-MBA.corp.ad.wrs.com>
2015-03-13 14:16 ` [review-request] 7414-cut the prefix from layer file paths for bitbake variables Barros Pena, Belen
2015-03-16 14:14 ` Lerner, Dave
2015-03-16 15:41 ` Barros Pena, Belen
2015-03-17 15:19 ` Damian, Alexandru
2015-03-17 15:45 ` Lerner, Dave
2015-03-17 15:51 ` Damian, Alexandru
2015-03-18 15:48 ` Lerner, Dave
2015-03-24 14:28 ` Damian, Alexandru
2015-03-24 19:44 ` Lerner, Dave
2015-03-25 13:04 ` Damian, Alexandru
2015-03-12 15:28 Lerner, Dave
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.