configvar: Sorting on the last "set-by-file" column

configvar: Sorting on the last "set-by-file" column

[Reyna, David]

Hi Belén and Alex,

> 2) "But the Set in file heading should be sortable"	

I have been looking into this. I cannot find the ability to supply a custom sort function for querysets, nor do I think we should do that given the complexity of the compare logic.

Since the values of the "set-by-file" is known and static, I think that we should just add a "vhistory_last" field to the Variable table so that all displays and sorts become simple. Specifically, I think we should add it as a link from the Variable table to the VariableHistory entry (if any) that represents the last file value. This would only cost us an id value per Variable.

I have been prototyping this and have some intermediate solutions, but I have also hit some unexpected problems.

1) I started by adding the column "vhistory_last" to the Variable model structure, using a simple string type (while I work out the details of class forward references and all that). I then updated "toaster/bitbake/lib/bb/ui/buildinfohelper.py" to set the new value. I also stopped Toaster and deleted the database, to force a new instance so that the schema would be correct, and then re-started Toaster and built a target.

However, while the Django interface saw this new column, I got an error because the column was not also created in the database. I looked at the database via "sqlite3" and confirmed that fact. Once I manually created the column in the Variable table, things worked.

So, is there a trick to re-create the database correctly when you add a column?

2) I also added some sample variables to the project's "conf/local.conf" to insure that these values were being picked up. I however found that Toaster did not always get the latest values. That is a worry some error.

This is how I created new target runs, which I hoped had enough complexity to populate the database.

  $ vi conf/local.conf   #update a local variable
  $ bitbake -c cleansstate base-files
  $ bitbake core-image-minimal

So, is there some extra caching (by bitbake I guess) of the "local.conf" values that my procedure does not trigger?

- David

Re: configvar: Sorting on the last "set-by-file" column

[Barros Pena, Belen]

On 16/02/2014 07:31, "Reyna, David" <david.reyna@windriver.com> wrote:

>Hi Belén and Alex,
>
>> 2) "But the Set in file heading should be sortable"	
>
>I have been looking into this. I cannot find the ability to supply a
>custom sort function for querysets, nor do I think we should do that
>given the complexity of the compare logic.
>
>Since the values of the "set-by-file" is known and static, I think that
>we should just add a "vhistory_last" field to the Variable table so that
>all displays and sorts become simple.

I was going to suggest something like this. If we cannot get it to work,
we'll need to disable the Set by file sorting.

>Specifically, I think we should add it as a link from the Variable table
>to the VariableHistory entry (if any) that represents the last file
>value. This would only cost us an id value per Variable.
>
>I have been prototyping this and have some intermediate solutions, but I
>have also hit some unexpected problems.
>
>1) I started by adding the column "vhistory_last" to the Variable model
>structure, using a simple string type (while I work out the details of
>class forward references and all that). I then updated
>"toaster/bitbake/lib/bb/ui/buildinfohelper.py" to set the new value. I
>also stopped Toaster and deleted the database, to force a new instance so
>that the schema would be correct, and then re-started Toaster and built a
>target.
>
>However, while the Django interface saw this new column, I got an error
>because the column was not also created in the database. I looked at the
>database via "sqlite3" and confirmed that fact. Once I manually created
>the column in the Variable table, things worked.
>
>So, is there a trick to re-create the database correctly when you add a
>column?
>
>2) I also added some sample variables to the project's "conf/local.conf"
>to insure that these values were being picked up. I however found that
>Toaster did not always get the latest values. That is a worry some error.
>
>This is how I created new target runs, which I hoped had enough
>complexity to populate the database.
>
>  $ vi conf/local.conf   #update a local variable
>  $ bitbake -c cleansstate base-files
>  $ bitbake core-image-minimal
>
>So, is there some extra caching (by bitbake I guess) of the "local.conf"
>values that my procedure does not trigger?

I have been told by Paul Eggleton that the changes should be be picked up,
but have you tried running the oe-init-build-env script after you make
your changes to local.conf?

>
>- David
>

Re: configvar: Sorting on the last "set-by-file" column

[Damian, Alexandru]

[-- Attachment #1: Type: text/plain, Size: 3265 bytes --]

About the Django database layer -

- Modifications to the models have to be manually put into the database.
Normally, Django supports this with the "syncdb" command. Our application
uses South though, so it requires two steps - first telling South to pick
up changes in the models, and second applying these changes to the
database. Second step is automated when you start the server, first you
have to do it manually.
Details here. http://south.readthedocs.org/en/latest/tutorial/part1.html

About changes to the local.conf -
- you have to restart the system, the bitbake server doesn't pick up
changes made on the fly; fixing this isn't a high priority at the moment
because we target the "fire and forget" functionality and not optimizing
for interactive builds. The toaster build control system will actually
shutdown the BB server after each build.


Hope this helps,
Alex


On Sun, Feb 16, 2014 at 7:31 AM, Reyna, David <david.reyna@windriver.com>wrote:

> Hi Belén and Alex,
>
> > 2) "But the Set in file heading should be sortable"
>
> I have been looking into this. I cannot find the ability to supply a
> custom sort function for querysets, nor do I think we should do that given
> the complexity of the compare logic.
>
> Since the values of the "set-by-file" is known and static, I think that we
> should just add a "vhistory_last" field to the Variable table so that all
> displays and sorts become simple. Specifically, I think we should add it as
> a link from the Variable table to the VariableHistory entry (if any) that
> represents the last file value. This would only cost us an id value per
> Variable.
>
> I have been prototyping this and have some intermediate solutions, but I
> have also hit some unexpected problems.
>
> 1) I started by adding the column "vhistory_last" to the Variable model
> structure, using a simple string type (while I work out the details of
> class forward references and all that). I then updated
> "toaster/bitbake/lib/bb/ui/buildinfohelper.py" to set the new value. I also
> stopped Toaster and deleted the database, to force a new instance so that
> the schema would be correct, and then re-started Toaster and built a target.
>
> However, while the Django interface saw this new column, I got an error
> because the column was not also created in the database. I looked at the
> database via "sqlite3" and confirmed that fact. Once I manually created the
> column in the Variable table, things worked.
>
> So, is there a trick to re-create the database correctly when you add a
> column?
>
> 2) I also added some sample variables to the project's "conf/local.conf"
> to insure that these values were being picked up. I however found that
> Toaster did not always get the latest values. That is a worry some error.
>
> This is how I created new target runs, which I hoped had enough complexity
> to populate the database.
>
>   $ vi conf/local.conf   #update a local variable
>   $ bitbake -c cleansstate base-files
>   $ bitbake core-image-minimal
>
> So, is there some extra caching (by bitbake I guess) of the "local.conf"
> values that my procedure does not trigger?
>
> - David
>
>


-- 
Alex Damian
Yocto Project
SSG / OTC

[-- Attachment #2: Type: text/html, Size: 4701 bytes --]