testing branch 2010-11-12

testing branch 2010-11-12

[Cliff Brake]

testing-next is ready for clean builds.  Not many builds succeeded
last week--I suspect people have been busy:

http://cgit.openembedded.org/cgit.cgi/openembedded/tag/?id=tested_2010-11-04

An overview of the testing process:
http://wiki.openembedded.net/index.php/Testing

Thanks,
Cliff

-- 
=================
http://bec-systems.com

Re: testing branch 2010-11-12

[Frans Meulenbroeks]

2010/11/12 Cliff Brake <cliff.brake@gmail.com>:
> testing-next is ready for clean builds.  Not many builds succeeded
> last week--I suspect people have been busy:

Actually my builds also succeeded (apart from the neek one which seem
to require some more love). The autobuilder did it's job quite nicely,
but I was just not around to verify the results and update the page.
The neek requires at least the os2 patch.

Testing 2010-11-12 gives the same results. updated the testing page.

Frans

Re: testing branch 2010-11-12

[Tom Rini]

On 11/14/2010 11:11 AM, Frans Meulenbroeks wrote:
> 2010/11/12 Cliff Brake<cliff.brake@gmail.com>:
>> testing-next is ready for clean builds.  Not many builds succeeded
>> last week--I suspect people have been busy:
>
> Actually my builds also succeeded (apart from the neek one which seem
> to require some more love). The autobuilder did it's job quite nicely,
> but I was just not around to verify the results and update the page.
> The neek requires at least the os2 patch.
>
> Testing 2010-11-12 gives the same results. updated the testing page.

After my first week of doing some testing branch test builds I was 
thinking, would some form of either automated result submitting or 
form-based result submitting be a good idea?  We could try and use a 
google doc spreadsheet / form, for example.  Thoughts?

-- 
Tom Rini
Mentor Graphics Corporation

Re: testing branch 2010-11-12

[Yury Bushmelev]

2010/11/14 Tom Rini <tom_rini@mentor.com>:
> On 11/14/2010 11:11 AM, Frans Meulenbroeks wrote:
>>
>> 2010/11/12 Cliff Brake<cliff.brake@gmail.com>:
>>>
>>> testing-next is ready for clean builds.  Not many builds succeeded
>>> last week--I suspect people have been busy:
>>
>> Actually my builds also succeeded (apart from the neek one which seem
>> to require some more love). The autobuilder did it's job quite nicely,
>> but I was just not around to verify the results and update the page.
>> The neek requires at least the os2 patch.
>>
>> Testing 2010-11-12 gives the same results. updated the testing page.
>
> After my first week of doing some testing branch test builds I was thinking,
> would some form of either automated result submitting or form-based result
> submitting be a good idea?  We could try and use a google doc spreadsheet /
> form, for example.  Thoughts?

I'll prefer to use our oestats-server to auto-publish builds data.
Oestats have complete information about builds.

-- 
Yury Bushmelev

Re: testing branch 2010-11-12

[Tom Rini]

On 11/14/2010 12:23 PM, Yury Bushmelev wrote:
> 2010/11/14 Tom Rini<tom_rini@mentor.com>:
>> On 11/14/2010 11:11 AM, Frans Meulenbroeks wrote:
>>>
>>> 2010/11/12 Cliff Brake<cliff.brake@gmail.com>:
>>>>
>>>> testing-next is ready for clean builds.  Not many builds succeeded
>>>> last week--I suspect people have been busy:
>>>
>>> Actually my builds also succeeded (apart from the neek one which seem
>>> to require some more love). The autobuilder did it's job quite nicely,
>>> but I was just not around to verify the results and update the page.
>>> The neek requires at least the os2 patch.
>>>
>>> Testing 2010-11-12 gives the same results. updated the testing page.
>>
>> After my first week of doing some testing branch test builds I was thinking,
>> would some form of either automated result submitting or form-based result
>> submitting be a good idea?  We could try and use a google doc spreadsheet /
>> form, for example.  Thoughts?
>
> I'll prefer to use our oestats-server to auto-publish builds data.
> Oestats have complete information about builds.

So some form of linkage between the two?  Or is there already an easy 
way to see what builds on the tinderbox view have been done this way 
that I don't see (also just started poking that stuff).  Thanks!

-- 
Tom Rini
Mentor Graphics Corporation

Re: testing branch 2010-11-12

[Gary Thomas]

On 11/14/2010 12:23 PM, Yury Bushmelev wrote:
> 2010/11/14 Tom Rini<tom_rini@mentor.com>:
>> On 11/14/2010 11:11 AM, Frans Meulenbroeks wrote:
>>>
>>> 2010/11/12 Cliff Brake<cliff.brake@gmail.com>:
>>>>
>>>> testing-next is ready for clean builds.  Not many builds succeeded
>>>> last week--I suspect people have been busy:
>>>
>>> Actually my builds also succeeded (apart from the neek one which seem
>>> to require some more love). The autobuilder did it's job quite nicely,
>>> but I was just not around to verify the results and update the page.
>>> The neek requires at least the os2 patch.
>>>
>>> Testing 2010-11-12 gives the same results. updated the testing page.
>>
>> After my first week of doing some testing branch test builds I was thinking,
>> would some form of either automated result submitting or form-based result
>> submitting be a good idea?  We could try and use a google doc spreadsheet /
>> form, for example.  Thoughts?
>
> I'll prefer to use our oestats-server to auto-publish builds data.
> Oestats have complete information about builds.

Except when they don't :-(

I've tried to use this many times and there have been times when the oestats
results are stale or incorrect.  Sometimes a build completes, but the status
is not updated as such.  Similarly, when a build fails, that may or may not
show up as well.

I reported this to the list a couple of weeks ago, but there was no response
at all.  It's not all that important to me (so I moved on), but if you're
going to rely on it to be able to summarize the results from a given set of
builds, it may be important that it works reliably.


-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: testing branch 2010-11-12

[Cliff Brake]

On Sun, Nov 14, 2010 at 2:23 PM, Yury Bushmelev <jay4mail@gmail.com> wrote:
> 2010/11/14 Tom Rini <tom_rini@mentor.com>:
>> On 11/14/2010 11:11 AM, Frans Meulenbroeks wrote:
>>>
>>> 2010/11/12 Cliff Brake<cliff.brake@gmail.com>:
>>>>
>>>> testing-next is ready for clean builds.  Not many builds succeeded
>>>> last week--I suspect people have been busy:
>>>
>>> Actually my builds also succeeded (apart from the neek one which seem
>>> to require some more love). The autobuilder did it's job quite nicely,
>>> but I was just not around to verify the results and update the page.
>>> The neek requires at least the os2 patch.
>>>
>>> Testing 2010-11-12 gives the same results. updated the testing page.
>>
>> After my first week of doing some testing branch test builds I was thinking,
>> would some form of either automated result submitting or form-based result
>> submitting be a good idea?  We could try and use a google doc spreadsheet /
>> form, for example.  Thoughts?
>
> I'll prefer to use our oestats-server to auto-publish builds data.
> Oestats have complete information about builds.

If someone has the time to update the oestats-server to provide
summary data in a tabular form, that would be great.

Alternatively, using forms on a google spreadsheet is an interesting
idea.  I had not thought of forms as a way that we could get anonymous
build data with requiring a google account.  Then again, I'm not sure
how we would keep the form from being spammed, other than perhaps
hiding it.

Cliff

-- 
=================
http://bec-systems.com

Re: testing branch 2010-11-12

[Tom Rini]

On 11/15/2010 08:47 AM, Cliff Brake wrote:
> On Sun, Nov 14, 2010 at 2:23 PM, Yury Bushmelev<jay4mail@gmail.com>  wrote:
>> 2010/11/14 Tom Rini<tom_rini@mentor.com>:
>>> On 11/14/2010 11:11 AM, Frans Meulenbroeks wrote:
>>>>
>>>> 2010/11/12 Cliff Brake<cliff.brake@gmail.com>:
>>>>>
>>>>> testing-next is ready for clean builds.  Not many builds succeeded
>>>>> last week--I suspect people have been busy:
>>>>
>>>> Actually my builds also succeeded (apart from the neek one which seem
>>>> to require some more love). The autobuilder did it's job quite nicely,
>>>> but I was just not around to verify the results and update the page.
>>>> The neek requires at least the os2 patch.
>>>>
>>>> Testing 2010-11-12 gives the same results. updated the testing page.
>>>
>>> After my first week of doing some testing branch test builds I was thinking,
>>> would some form of either automated result submitting or form-based result
>>> submitting be a good idea?  We could try and use a google doc spreadsheet /
>>> form, for example.  Thoughts?
>>
>> I'll prefer to use our oestats-server to auto-publish builds data.
>> Oestats have complete information about builds.
>
> If someone has the time to update the oestats-server to provide
> summary data in a tabular form, that would be great.
>
> Alternatively, using forms on a google spreadsheet is an interesting
> idea.  I had not thought of forms as a way that we could get anonymous
> build data with requiring a google account.  Then again, I'm not sure
> how we would keep the form from being spammed, other than perhaps
> hiding it.

I do have to admit that I hadn't given it more than a few seconds 
thought, but can't we perform some sanity checks on the input?  That 
should really cut down on the spam, I would think (ie make sure the 
tested tag field starts with testing_).  If this sounds like a possibly 
decent idea I can try and play with it a little bit.

-- 
Tom Rini
Mentor Graphics Corporation

Re: testing branch 2010-11-12

[Yuri Bushmelev]

Hello!

> >> After my first week of doing some testing branch test builds I was
> >> thinking, would some form of either automated result submitting or
> >> form-based result submitting be a good idea?  We could try and use a
> >> google doc spreadsheet / form, for example.  Thoughts?
> > 
> > I'll prefer to use our oestats-server to auto-publish builds data.
> > Oestats have complete information about builds.
> 
> If someone has the time to update the oestats-server to provide
> summary data in a tabular form, that would be great.

I would to spend some time to look into oestats-server/client code, but it 
will happens after release anyway.

> Alternatively, using forms on a google spreadsheet is an interesting
> idea.  I had not thought of forms as a way that we could get anonymous
> build data with requiring a google account.  Then again, I'm not sure
> how we would keep the form from being spammed, other than perhaps
> hiding it.

My opinion is to leave things as is up to release (about 2-3 weeks).
IMHO, we should concentrate now on intensive build testing and testing on 
target machines.

-- 
Yury Bushmelev

Re: testing branch 2010-11-12

[Cliff Brake]

Last week's test run was a success with 38 combinations passing.
Thanks to all running tests:

http://cgit.openembedded.org/cgit.cgi/openembedded/tag/?id=tested_2010-11-12

Due to the OE 2010.12 release effort, we are going to give the
testing-next effort a break for a couple weeks and ask people doing
testing to focus on the
release-2010.12 branch that Khem is working on.

Thanks,
Cliff


On Fri, Nov 12, 2010 at 12:17 AM, Cliff Brake <cliff.brake@gmail.com> wrote:
> testing-next is ready for clean builds.  Not many builds succeeded
> last week--I suspect people have been busy:
>
> http://cgit.openembedded.org/cgit.cgi/openembedded/tag/?id=tested_2010-11-04
>
> An overview of the testing process:
> http://wiki.openembedded.net/index.php/Testing
>
> Thanks,
> Cliff
>
> --
> =================
> http://bec-systems.com
>



-- 
=================
http://bec-systems.com

Re: testing branch 2010-11-12

[Petr Štetiar]

Cliff Brake <cliff.brake@gmail.com> [2010-11-18 17:13:17]:

> Last week's test run was a success with 38 combinations passing.  Thanks to
> all running tests:
> 
> http://cgit.openembedded.org/cgit.cgi/openembedded/tag/?id=tested_2010-11-12
> 
> Due to the OE 2010.12 release effort, we are going to give the testing-next
> effort a break for a couple weeks and ask people doing testing to focus on
> the release-2010.12 branch that Khem is working on.

Hi Cliff,

it looks like, you've included my failed builds also :-) I don't know if it's
intention, but I think, that only successful builds should be in the commit
message, right? Is it my mistake, to report failed builds on the Testing page?

I'm really sorry, that I can only report my build results without fixing them,
but I don't have any spare time now...

-- ynezz

Re: testing branch 2010-11-12

[Frans Meulenbroeks]

2010/11/19 Petr Štetiar <ynezz@true.cz>:
> Cliff Brake <cliff.brake@gmail.com> [2010-11-18 17:13:17]:
>
>> Last week's test run was a success with 38 combinations passing.  Thanks to
>> all running tests:
>>
>> http://cgit.openembedded.org/cgit.cgi/openembedded/tag/?id=tested_2010-11-12
>>
>> Due to the OE 2010.12 release effort, we are going to give the testing-next
>> effort a break for a couple weeks and ask people doing testing to focus on
>> the release-2010.12 branch that Khem is working on.
>
> Hi Cliff,
>
> it looks like, you've included my failed builds also :-) I don't know if it's
> intention, but I think, that only successful builds should be in the commit
> message, right? Is it my mistake, to report failed builds on the Testing page?
>
> I'm really sorry, that I can only report my build results without fixing them,
> but I don't have any spare time now...
>
> -- ynezz

Failure should be reported as it is also an indicator of quality.
That's why the remarks column is there.
Preferably also indicate failure cause and tinderbox link.

Frans

Re: testing branch 2010-11-12

[Yury Bushmelev]

2010/11/19 Frans Meulenbroeks <fransmeulenbroeks@gmail.com>:
> 2010/11/19 Petr Štetiar <ynezz@true.cz>:
>> Cliff Brake <cliff.brake@gmail.com> [2010-11-18 17:13:17]:
>>
>>> Last week's test run was a success with 38 combinations passing.  Thanks to
>>> all running tests:
>>>
>>> http://cgit.openembedded.org/cgit.cgi/openembedded/tag/?id=tested_2010-11-12
>>>
>>> Due to the OE 2010.12 release effort, we are going to give the testing-next
>>> effort a break for a couple weeks and ask people doing testing to focus on
>>> the release-2010.12 branch that Khem is working on.
>>
>> Hi Cliff,
>>
>> it looks like, you've included my failed builds also :-) I don't know if it's
>> intention, but I think, that only successful builds should be in the commit
>> message, right? Is it my mistake, to report failed builds on the Testing page?
>>
>> I'm really sorry, that I can only report my build results without fixing them,
>> but I don't have any spare time now...
>>
>> -- ynezz
>
> Failure should be reported as it is also an indicator of quality.
> That's why the remarks column is there.
> Preferably also indicate failure cause and tinderbox link.

I've used '<del>opie-image</del>' wiki syntax to mark failed
opie-image and package name with tinderbox link in 'current issues'
field.

-- 
Yury Bushmelev

Re: testing branch 2010-11-12

[Petr Štetiar]

Frans Meulenbroeks <fransmeulenbroeks@gmail.com> [2010-11-19 10:43:45]:

> Preferably also indicate failure cause and tinderbox link.

Yep, did both.

-- ynezz

Re: testing branch 2010-11-12

[Cliff Brake]

On Fri, Nov 19, 2010 at 4:11 AM, Petr Štetiar <ynezz@true.cz> wrote:
> Cliff Brake <cliff.brake@gmail.com> [2010-11-18 17:13:17]:
>
>> Last week's test run was a success with 38 combinations passing.  Thanks to
>> all running tests:
>>
>> http://cgit.openembedded.org/cgit.cgi/openembedded/tag/?id=tested_2010-11-12
>>
>> Due to the OE 2010.12 release effort, we are going to give the testing-next
>> effort a break for a couple weeks and ask people doing testing to focus on
>> the release-2010.12 branch that Khem is working on.
>
> Hi Cliff,
>
> it looks like, you've included my failed builds also :-) I don't know if it's
> intention, but I think, that only successful builds should be in the commit
> message, right? Is it my mistake, to report failed builds on the Testing page?

My process right now is to export the table to a spreadsheet, and sort
by last successful build column.  If this field has a data that
matches the current testing branch, then I include it.  If you have a
build that does not work, then just leave this field blank, or at the
last successful build.  I could have very easily messed something up
as well.

Thanks,
Cliff

-- 
=================
http://bec-systems.com

Re: testing branch 2010-11-12

[Steffen Sledz]

Am 18.11.2010 23:13, schrieb Cliff Brake:
> Last week's test run was a success with 38 combinations passing.
> Thanks to all running tests:
> 
> http://cgit.openembedded.org/cgit.cgi/openembedded/tag/?id=tested_2010-11-12

Just a short question (i was "offline" for some weeks).

The Testing Log [1] ends with "tested_2010-10-29". Is this indented?

Steffen

[1] http://wiki.openembedded.net/index.php/Testing#Testing_Log

Re: testing branch 2010-11-12

[Cliff Brake]

On Tue, Jan 4, 2011 at 5:36 AM, Steffen Sledz <sledz@dresearch.de> wrote:
> Am 18.11.2010 23:13, schrieb Cliff Brake:
>> Last week's test run was a success with 38 combinations passing.
>> Thanks to all running tests:
>>
>> http://cgit.openembedded.org/cgit.cgi/openembedded/tag/?id=tested_2010-11-12
>
> Just a short question (i was "offline" for some weeks).
>
> The Testing Log [1] ends with "tested_2010-10-29". Is this indented?
>
> Steffen
>
> [1] http://wiki.openembedded.net/index.php/Testing#Testing_Log

Looks like I neglected to update this list.  Since cgit provides a
nice list of tags, I decided to simply reference cgit for the tags.

Thanks,
Cliff

-- 
=================
http://bec-systems.com