[U-Boot] livetime of boards

[U-Boot] livetime of boards

[Heiko Schocher]

Hello all,

We have the real problem, that we have a lot of old boards, which
are unmaintained in U-Boot, and we have no chance to find out, if this
boards are longer used/tested ...

So, the question raises, should we introduce a column in boards.cfg,
which shows the "livetime" of a board support in U-Boot?

If we introduce this, we init the livetime column in boards.cfg with
a value of n for all current boards, or if we add a new board support.
(n releases valid) I think n = 4 would be a good starting point.

Every release cycle the livetime value gets decremented by one.
(Can be done by a script sitting in the tools directory?)

livetime == 0 -> EMail to board maintainer, that board livetime ends.
                  please test and send an update, if board support should
                  remain in U-Boot.

livetime == -1 -> board gets deleted in next release

If a board maintainer gets the above EMail, or of course whenever
he really try a current release on a real hardware, he can send a patch,
which updates the livetime column again back to n ...

So, the hope is, we have after n releases only real tested and used
boards in U-Boot...

What do others think?

short ToDo list:
- make an initial patch for boards.cfg
- make a script, which decrements the "livetime" column in boards.cfg
   and send an EMail for the "livetime == 0" case to the boardmaintainer
   and make a delete patch for boards.cfg if "livetime == -1" ?
- add a doc/README.livetime

bye,
heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] livetime of boards

[Tom Rini]

On Tue, Nov 05, 2013 at 02:05:28PM +0100, Heiko Schocher wrote:

> Hello all,
> 
> We have the real problem, that we have a lot of old boards, which
> are unmaintained in U-Boot, and we have no chance to find out, if this
> boards are longer used/tested ...

We also have a feature, lots of hardware support because lots of things
don't change drastically, frequently.  That's not to say that I wouldn't
mind dropping old platforms (even that ones I have sentimental feelings
towards), and would certainly like to see more and frequent sanity
testing.

This problem comes up when we talk about doing big changes, and I think
that's the right time to talk about things.  And I think the answer
should be, we try and convert things forward and when it's not obvious
if things will still work correctly, or how to do it, that's when we
need to make a hard push on the board maintainers to find some time to
work on things.

It's not that being a U-Boot board maintainer is a near zero effort,
when someone steps up to own a board, but to be frank, it also shouldn't
be a frequent high-touch area either.

> So, the question raises, should we introduce a column in boards.cfg,
> which shows the "livetime" of a board support in U-Boot?

I sense a lot of conflicting patches.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20131105/b05cdae8/attachment.pgp>

[U-Boot] livetime of boards

[Wolfgang Denk]

Dear Tom,

In message <20131105203736.GM5925@bill-the-cat> you wrote:
> 
> > We have the real problem, that we have a lot of old boards, which
> > are unmaintained in U-Boot, and we have no chance to find out, if this
> > boards are longer used/tested ...
> 
> We also have a feature, lots of hardware support because lots of things
> don't change drastically, frequently.  That's not to say that I wouldn't
> mind dropping old platforms (even that ones I have sentimental feelings
> towards), and would certainly like to see more and frequent sanity
> testing.

I think Heiko's idea of documenting test reports is pretty cool - but
of course we need to discuss in detail how to implement this, and
also decide wether we use this for (semi-automatic) code cleanup (by
removing boards that have not been tested for a long time).

> This problem comes up when we talk about doing big changes, and I think
> that's the right time to talk about things.  And I think the answer
> should be, we try and convert things forward and when it's not obvious
> if things will still work correctly, or how to do it, that's when we
> need to make a hard push on the board maintainers to find some time to
> work on things.

Agreed. And here information how recently (or maybe even how
frequently) a board has been tested (build tested, run on actual
hardware) would come in really handy.  we can probbaly automate build
testing one way or another, but for actual runtime tests we will lways
depend on the board maintainers, or board users.

> > So, the question raises, should we introduce a column in boards.cfg,
> > which shows the "livetime" of a board support in U-Boot?
> 
> I sense a lot of conflicting patches.

Again I agree.  Also, I fear that  boards.cfg  is becoming more and
more unreadable by adding even more stuff.  If I see this correctly,
the maximum line length in  boards.cfg  already exceeds 360 characters
:-(


So nstead of adding this information to  boards.cfg  we could probably
use separate files for such information.  We could provide tools to
make test reports really easy, say something like

	scripts/build_test
	scripts/run_test

which the user would just call with a "passed" or "failed" argument;
the scripts could then auto-detect which configuration and which exact
U-Boot version were in use, and send an email.  Whether that would be
a patch against the source code or something that get's auto-added to
a wiki page is just an implementation detail.  But if we had something
like this, we could get a muchbetter understanding how actively boards
are being tested.

So when you're once again doing some change that requires touching
files for some othe rboards, you could simply check that database.  If
you see that 3 out of the last 5 releases have reported succesful
run-time tests you will probably decide to accept the needed efforts,
but when you see the last test report is more than 5 years old, you
will probably rather decide to initiate a code removal process.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
A Chairman was as necessary to a Board planet  as  the  zero  was  in
mathematics, but being a zero had big disadvantages...
                         - Terry Pratchett, _The Dark Side of the Sun_

[U-Boot] livetime of boards

[Heiko Schocher]

Hello Wolfgang, Tom,

Am 06.11.2013 08:50, schrieb Wolfgang Denk:
> Dear Tom,
>
> In message<20131105203736.GM5925@bill-the-cat>  you wrote:
>>
>>> We have the real problem, that we have a lot of old boards, which
>>> are unmaintained in U-Boot, and we have no chance to find out, if this
>>> boards are longer used/tested ...
>>
>> We also have a feature, lots of hardware support because lots of things
>> don't change drastically, frequently.  That's not to say that I wouldn't
>> mind dropping old platforms (even that ones I have sentimental feelings
>> towards), and would certainly like to see more and frequent sanity
>> testing.

Yep, thats the reason I had in mind. We must have more real tests
on real hardware (or at least, get back the info, that this is done
somewhere ...) ... thats the idea behind to force the board maintainers
to give back us such info.

> I think Heiko's idea of documenting test reports is pretty cool - but
> of course we need to discuss in detail how to implement this, and
> also decide wether we use this for (semi-automatic) code cleanup (by
> removing boards that have not been tested for a long time).

I vote for removing boards from which we get no such test info back.
The board support code is just availiable, just not for current releases.

If code does not change drastically, such a "compile and try it on the
hardware, give feedback to mainline" should not cost much time the board
maintainer, and maybe we add a script, which the board maintainer just
have to call, to send an "board tested EMail" to the mailinglist ...

Maybe a lot of boardmaintainers do this tests, and this info is
important for mainline, but we have no mechanism to collect this
info!

And if code changes drastically (which it currently does and will do
in future I think) a board maintainer must decide, if it makes sense to
sync the board with current mainline or the board get dropped from
mainline ...

And I have this problem currently with i2c subsystem ... a lot of
boards to convert to new framework, and I cannot decide, which are
really maintained... or if currently, the converted boards, really
work. (And I think, not only the i2c subsystem has this problem)

>> This problem comes up when we talk about doing big changes, and I think
>> that's the right time to talk about things.  And I think the answer
>> should be, we try and convert things forward and when it's not obvious
>> if things will still work correctly, or how to do it, that's when we
>> need to make a hard push on the board maintainers to find some time to
>> work on things.
>
> Agreed. And here information how recently (or maybe even how
> frequently) a board has been tested (build tested, run on actual
> hardware) would come in really handy.  we can probbaly automate build
> testing one way or another, but for actual runtime tests we will lways
> depend on the board maintainers, or board users.

Hmm.. Is it always obvious, if changes are big? Do we really get
feedback when doing such big changes? I just reworking the i2c framework,
and I have not really much feedback from board maintainers for their
boards I changed ... Does this change really work on all boards I
converted? I have no chance to get this info. But if we have such a
livetime feature, I can be sure, that after n releases all boards in
mainline are tested ... automatically ... I want such a feature ...

>>> So, the question raises, should we introduce a column in boards.cfg,
>>> which shows the "livetime" of a board support in U-Boot?
>>
>> I sense a lot of conflicting patches.
>
> Again I agree.  Also, I fear that  boards.cfg  is becoming more and
> more unreadable by adding even more stuff.  If I see this correctly,
> the maximum line length in  boards.cfg  already exceeds 360 characters
> :-(

Right, boards.cfg gets unhandy ... Hmm .. what with the column
"Staus" ... instead of "Active" it would be more informative to have
there the livetime counter, and a single digit saves some characters ;-)

But you are right, that approach leads in a lot of conflicting
patches ... but I think, we just pooled board information in boards.cfg,
so this would be the right place in my eyes ...

Maybe we get such Information "a Boards is tested with current mainline"
inform of an EMail with an Text "Board xy tested with commit mm. Please
update livetime" ... and we can add a script, which updates the
livetime for this board, so we can prevent conflicting patches ... ?

> So nstead of adding this information to  boards.cfg  we could probably
> use separate files for such information.  We could provide tools to
> make test reports really easy, say something like
>
> 	scripts/build_test
> 	scripts/run_test
>
> which the user would just call with a "passed" or "failed" argument;
> the scripts could then auto-detect which configuration and which exact
> U-Boot version were in use, and send an email.  Whether that would be
> a patch against the source code or something that get's auto-added to
> a wiki page is just an implementation detail.  But if we had something
> like this, we could get a muchbetter understanding how actively boards
> are being tested.

Yes, that sound also good. I want to see the test information in the
sourcecode, not somewhere on a wiki...

> So when you're once again doing some change that requires touching
> files for some othe rboards, you could simply check that database.  If
> you see that 3 out of the last 5 releases have reported succesful
> run-time tests you will probably decide to accept the needed efforts,

Hmm.. that works, if you have to touch some (some < 5) boards. But
If you have to touch > 5 boards, this gets unhandy...

> but when you see the last test report is more than 5 years old, you
> will probably rather decide to initiate a code removal process.

If we decide to delete older boards after n release cycles without
testreports, we must not decide nor look in a database. We are
sure, we have only "good and working" boards ... and we just
do the necessary work for new features ... and we are sure, that
we get back testreports within n release cycles ...

So let us decide first, if we want to go this way ...

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] livetime of boards

[Andreas Bießmann]

Hello all together,

On 11/07/2013 09:17 AM, Heiko Schocher wrote:
> Am 06.11.2013 08:50, schrieb Wolfgang Denk:
>> In message<20131105203736.GM5925@bill-the-cat>  you wrote:

<snip problem description>

Full ACK, we need some way to track which board is working with the
current ToT or at least on a release basis.

>>>> So, the question raises, should we introduce a column in boards.cfg,
>>>> which shows the "livetime" of a board support in U-Boot?
>>>
>>> I sense a lot of conflicting patches.
>>
>> Again I agree.  Also, I fear that  boards.cfg  is becoming more and
>> more unreadable by adding even more stuff.  If I see this correctly,
>> the maximum line length in  boards.cfg  already exceeds 360 characters
>> :-(
> 
> Right, boards.cfg gets unhandy ... Hmm .. what with the column
> "Staus" ... instead of "Active" it would be more informative to have
> there the livetime counter, and a single digit saves some characters ;-)

I can't understand the status field at all, just for the record ;)

> But you are right, that approach leads in a lot of conflicting
> patches ... but I think, we just pooled board information in boards.cfg,
> so this would be the right place in my eyes ...
> 
> Maybe we get such Information "a Boards is tested with current mainline"
> inform of an EMail with an Text "Board xy tested with commit mm. Please
> update livetime" ... and we can add a script, which updates the
> livetime for this board, so we can prevent conflicting patches ... ?

I agree here with Tom. Beside the possibility of conflicting pahces I
see another problem here.
We will get a lot of patches just for increasing the tested counter for
a single board. These patches needs to be handled in some way. If we
shift to some integrated system (gerrit comes to mind) this could be
easier than today, but it will bind resources anyways.
Therefore I think it is a bad idea to save a such often changing
information in the source code repository.

>> So nstead of adding this information to  boards.cfg  we could probably
>> use separate files for such information.  We could provide tools to
>> make test reports really easy, say something like
>>
>>     scripts/build_test
>>     scripts/run_test
>>
>> which the user would just call with a "passed" or "failed" argument;
>> the scripts could then auto-detect which configuration and which exact
>> U-Boot version were in use, and send an email.  Whether that would be
>> a patch against the source code or something that get's auto-added to
>> a wiki page is just an implementation detail.  But if we had something
>> like this, we could get a muchbetter understanding how actively boards
>> are being tested.
> 
> Yes, that sound also good. I want to see the test information in the
> sourcecode, not somewhere on a wiki...

I think another place than the source code repository would be best for
gathering such frequently changing information. Why not use some wiki
other other web service for that purpose?

I don't want to search a web page for the information 'board X is not
tested since ...' either. But we could easily write some scripts and add
them to the source code repository to provide it.

>> So when you're once again doing some change that requires touching
>> files for some othe rboards, you could simply check that database.  If
>> you see that 3 out of the last 5 releases have reported succesful
>> run-time tests you will probably decide to accept the needed efforts,
> 
> Hmm.. that works, if you have to touch some (some < 5) boards. But
> If you have to touch > 5 boards, this gets unhandy...

How about:

MAKEALL --check-boards -s at91

;)

>> but when you see the last test report is more than 5 years old, you
>> will probably rather decide to initiate a code removal process.

Why not save the SHA1 with the build-/runtime-tested information? Then
we could easily build helper scripts to query that database when this
board was last tested.

> If we decide to delete older boards after n release cycles without
> testreports, we must not decide nor look in a database. We are
> sure, we have only "good and working" boards ... and we just
> do the necessary work for new features ... and we are sure, that
> we get back testreports within n release cycles ...
> 
> So let us decide first, if we want to go this way ...

Yes, we should introduce some mechanism to check when a specific board
was last runtime tested. But I fear the overhead with patches that
update a tested counter.

Best regards

Andreas Bie?mann

[U-Boot] livetime of boards

[Heiko Schocher]

Hello Andreas,

Am 07.11.2013 10:37, schrieb Andreas Bie?mann:
> Hello all together,
>
> On 11/07/2013 09:17 AM, Heiko Schocher wrote:
>> Am 06.11.2013 08:50, schrieb Wolfgang Denk:
>>> In message<20131105203736.GM5925@bill-the-cat>   you wrote:
>
> <snip problem description>
>
> Full ACK, we need some way to track which board is working with the
> current ToT or at least on a release basis.
>
>>>>> So, the question raises, should we introduce a column in boards.cfg,
>>>>> which shows the "livetime" of a board support in U-Boot?
>>>>
>>>> I sense a lot of conflicting patches.
>>>
>>> Again I agree.  Also, I fear that  boards.cfg  is becoming more and
>>> more unreadable by adding even more stuff.  If I see this correctly,
>>> the maximum line length in  boards.cfg  already exceeds 360 characters
>>> :-(
>>
>> Right, boards.cfg gets unhandy ... Hmm .. what with the column
>> "Staus" ... instead of "Active" it would be more informative to have
>> there the livetime counter, and a single digit saves some characters ;-)
>
> I can't understand the status field at all, just for the record ;)

Hmm.. good question ...

>> But you are right, that approach leads in a lot of conflicting
>> patches ... but I think, we just pooled board information in boards.cfg,
>> so this would be the right place in my eyes ...
>>
>> Maybe we get such Information "a Boards is tested with current mainline"
>> inform of an EMail with an Text "Board xy tested with commit mm. Please
>> update livetime" ... and we can add a script, which updates the
>> livetime for this board, so we can prevent conflicting patches ... ?
>
> I agree here with Tom. Beside the possibility of conflicting pahces I
> see another problem here.
> We will get a lot of patches just for increasing the tested counter for
> a single board. These patches needs to be handled in some way. If we
> shift to some integrated system (gerrit comes to mind) this could be
> easier than today, but it will bind resources anyways.
> Therefore I think it is a bad idea to save a such often changing
> information in the source code repository.

I see this info just changing once when releasing a new U-Boot version.

I not see anymore a patch for updating the livetime counter for every
board, instead we should have a script which a board maintainer can call,
after he did a board test, which sends an EMail to the U-Boot ML with
a special format, saying:
------------------------------------
subject: livetime: board name

Tested-by: ...

with commit ...
------------------------------------
On the mailserver a script scans all incoming EMails for his Subject,
(Is this possible?)
and collect the infos, for which boards, such EMails arrived. When
releasing a new U-Boot version, this collected info can be used to
update this livetime counter through another script saying
"collect_livetime_info" (also this script can automatically send
EMails to board maintainers, for boards which had reached end of
livetime, outputs a delete board lists ...)

So (in current case Tom) should, before releasing a new U-Boot
version, first call this script "collect_livetime_info" and he get:

-> one livetime counter patch for current release
-> one list for boards which reach end of life
-> one list for boards, which should be deleted

All Infos are "release info" I think, and fully fit in the commit
for the new release ...

... maybe "deleting boards" can be done automatically, but this is
not a trivial job ...

So, with such a solution, I see no big additional cost for adding
such a feature (except the task "deleting old boards", which is maybe
not trivial)

Do not understand me wrong, if we find another solution, I am
happy also ... just spinning around ...

>>> So nstead of adding this information to  boards.cfg  we could probably
>>> use separate files for such information.  We could provide tools to
>>> make test reports really easy, say something like
>>>
>>>      scripts/build_test
>>>      scripts/run_test
>>>
>>> which the user would just call with a "passed" or "failed" argument;
>>> the scripts could then auto-detect which configuration and which exact
>>> U-Boot version were in use, and send an email.  Whether that would be
>>> a patch against the source code or something that get's auto-added to
>>> a wiki page is just an implementation detail.  But if we had something
>>> like this, we could get a muchbetter understanding how actively boards
>>> are being tested.
>>
>> Yes, that sound also good. I want to see the test information in the
>> sourcecode, not somewhere on a wiki...
>
> I think another place than the source code repository would be best for
> gathering such frequently changing information. Why not use some wiki
> other other web service for that purpose?

See above explanation, I see this info not frequently changing, just
always with a new U-Boot release ... and can nearly (except the "delete
old boards" case) automatised ...

> I don't want to search a web page for the information 'board X is not
> tested since ...' either. But we could easily write some scripts and add
> them to the source code repository to provide it.

Ok, fine with me too. I just thinking about this problem, and how
we can fix it ;-)

>>> So when you're once again doing some change that requires touching
>>> files for some othe rboards, you could simply check that database.  If
>>> you see that 3 out of the last 5 releases have reported succesful
>>> run-time tests you will probably decide to accept the needed efforts,
>>
>> Hmm.. that works, if you have to touch some (some<  5) boards. But
>> If you have to touch>  5 boards, this gets unhandy...
>
> How about:
>
> MAKEALL --check-boards -s at91
>
> ;)

;-)

>>> but when you see the last test report is more than 5 years old, you
>>> will probably rather decide to initiate a code removal process.
>
> Why not save the SHA1 with the build-/runtime-tested information? Then
> we could easily build helper scripts to query that database when this
> board was last tested.

Ok, also good idea.

>> If we decide to delete older boards after n release cycles without
>> testreports, we must not decide nor look in a database. We are
>> sure, we have only "good and working" boards ... and we just
>> do the necessary work for new features ... and we are sure, that
>> we get back testreports within n release cycles ...
>>
>> So let us decide first, if we want to go this way ...
>
> Yes, we should introduce some mechanism to check when a specific board
> was last runtime tested. But I fear the overhead with patches that
> update a tested counter.

I thought with "decide": Do we want to delete "old boards"?
With this, we do not need a "MAKEALL --check-boards -s at91" when
we introduce new features, as all boards in mainline are in a well
tested shape ...

Ok, two decisions:

- Do we want to collect board testinginformation?
- Do we want to delete old boards automatically after we do not get
   some test reports after a time intervall?

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] livetime of boards

[Albert ARIBAUD]

Hi Heiko,

On Thu, 07 Nov 2013 11:39:08 +0100, Heiko Schocher <hs@denx.de> wrote:


> >> Right, boards.cfg gets unhandy ... Hmm .. what with the column
> >> "Staus" ... instead of "Active" it would be more informative to have
> >> there the livetime counter, and a single digit saves some characters ;-)

I'm not sure we need to save characters from boards.cfg.

Note also that one goal of boards.cfg is to not have multiple files
around that have to remain consistent.

Still :

> > I can't understand the status field at all, just for the record ;)
> 
> Hmm.. good question ...

There has been some discussion on this already; indeed, the status
field is not really aptly named or defined and should be reworked.

We could, for instance, have a "Last tested" field instead of the
"Active" field.

> Ok, two decisions:
> 
> - Do we want to collect board testinginformation?

I'd say we might want to, if only because it tells us if a board
maintainer is still active or not, instead of finding out long later.

> - Do we want to delete old boards automatically after we do not get
>    some test reports after a time intervall?

I don't, at least, not without a long enough period during which the
board remains in "unmaintained" state. E.g., for each release, the list
of "unmaintained" boards is produced along with the release notes, and
boards which are still "unmaintained" when nearing the next release
(roughly 90 days later) get deleted right before the new release.

> bye,
> Heiko

Amicalement,
-- 
Albert.

[U-Boot] livetime of boards

[Andreas Bießmann]

Hello Heiko,

On 11/07/2013 11:39 AM, Heiko Schocher wrote:
> Am 07.11.2013 10:37, schrieb Andreas Bie?mann:
>> On 11/07/2013 09:17 AM, Heiko Schocher wrote:
>>> Am 06.11.2013 08:50, schrieb Wolfgang Denk:
>>>> In message<20131105203736.GM5925@bill-the-cat>   you wrote:

<snip>

>>> But you are right, that approach leads in a lot of conflicting
>>> patches ... but I think, we just pooled board information in boards.cfg,
>>> so this would be the right place in my eyes ...
>>>
>>> Maybe we get such Information "a Boards is tested with current mainline"
>>> inform of an EMail with an Text "Board xy tested with commit mm. Please
>>> update livetime" ... and we can add a script, which updates the
>>> livetime for this board, so we can prevent conflicting patches ... ?
>>
>> I agree here with Tom. Beside the possibility of conflicting pahces I
>> see another problem here.
>> We will get a lot of patches just for increasing the tested counter for
>> a single board. These patches needs to be handled in some way. If we
>> shift to some integrated system (gerrit comes to mind) this could be
>> easier than today, but it will bind resources anyways.
>> Therefore I think it is a bad idea to save a such often changing
>> information in the source code repository.
> 
> I see this info just changing once when releasing a new U-Boot version.

The saved information how often a board was runtime tested with the
correct SHA1 of the u-boot/master could be quite useful.
In the end just the last tested commit will be interesting but it could
give some information how often that specific board is used. The
information must not be generated by a board maintainer ... the
maintainer could then see if he needs to pull out a board or if one else
run the test before.

If we would save this in the repository we do not have this information
in time.
If we send the information to a list we need to parse it or use some
other tool to provide the information.
Beside that we will pollute the list with status updates about boards
being tested. It could be hard to find real patches in that information
flood then.

<snip mail proposal>

> So (in current case Tom) should, before releasing a new U-Boot
> version, first call this script "collect_livetime_info" and he get:
> 
> -> one livetime counter patch for current release
> -> one list for boards which reach end of life
> -> one list for boards, which should be deleted

Good idea, but the information could also be saved on a website or in
another database.
It should be easily filled by the tester and also easily queried by
wherever is interested in.

> All Infos are "release info" I think, and fully fit in the commit
> for the new release ...

I also think that should be done on release only.

> ... maybe "deleting boards" can be done automatically, but this is
> not a trivial job ...

I think deleting should be done in next release then to give the board
maintainer some time to check the boards. On a new release the board
maintainer should be mailed that in the next release the board will be
removed. We should also store this somewhere in the code (status in
boards.cfg?).

Next question is what to do if the mail bounces ;)

> So, with such a solution, I see no big additional cost for adding
> such a feature (except the task "deleting old boards", which is maybe
> not trivial)
> 
> Do not understand me wrong, if we find another solution, I am
> happy also ... just spinning around ...

Me too.

<snip>

>>> If we decide to delete older boards after n release cycles without
>>> testreports, we must not decide nor look in a database. We are
>>> sure, we have only "good and working" boards ... and we just
>>> do the necessary work for new features ... and we are sure, that
>>> we get back testreports within n release cycles ...
>>>
>>> So let us decide first, if we want to go this way ...
>>
>> Yes, we should introduce some mechanism to check when a specific board
>> was last runtime tested. But I fear the overhead with patches that
>> update a tested counter.
> 
> I thought with "decide": Do we want to delete "old boards"?
> With this, we do not need a "MAKEALL --check-boards -s at91" when
> we introduce new features, as all boards in mainline are in a well
> tested shape ...
> 
> Ok, two decisions:
> 
> - Do we want to collect board testinginformation?

I think we should do that i none way or another.

> - Do we want to delete old boards automatically after we do not get
>   some test reports after a time intervall?

And we should delete 'unmaintained' boards, when is to be discussed. I'm
currently fiddling with at91 gpio and ask myself if I should adopt all
the boards or just let them fail ...

Best regards

Andreas Bie?mann

[U-Boot] livetime of boards

[Heiko Schocher]

Hello Albert,

Am 07.11.2013 12:13, schrieb Albert ARIBAUD:
> Hi Heiko,
>
> On Thu, 07 Nov 2013 11:39:08 +0100, Heiko Schocher<hs@denx.de>  wrote:
>
>
>>>> Right, boards.cfg gets unhandy ... Hmm .. what with the column
>>>> "Staus" ... instead of "Active" it would be more informative to have
>>>> there the livetime counter, and a single digit saves some characters ;-)
>
> I'm not sure we need to save characters from boards.cfg.
>
> Note also that one goal of boards.cfg is to not have multiple files
> around that have to remain consistent.

Yep, exactly. Thats why I think we could collect this in boards.cfg.

> Still :
>
>>> I can't understand the status field at all, just for the record ;)
>>
>> Hmm.. good question ...
>
> There has been some discussion on this already; indeed, the status
> field is not really aptly named or defined and should be reworked.
>
> We could, for instance, have a "Last tested" field instead of the
> "Active" field.

Yep.

>> Ok, two decisions:
>>
>> - Do we want to collect board testinginformation?
>
> I'd say we might want to, if only because it tells us if a board
> maintainer is still active or not, instead of finding out long later.

Yes.

>> - Do we want to delete old boards automatically after we do not get
>>     some test reports after a time intervall?
>
> I don't, at least, not without a long enough period during which the
> board remains in "unmaintained" state. E.g., for each release, the list
> of "unmaintained" boards is produced along with the release notes, and
> boards which are still "unmaintained" when nearing the next release
> (roughly 90 days later) get deleted right before the new release.

I proposed:

livetime = 4 releases cycle without testing
If livetime = 0 -> EMail to board maintainer, please test or board
get deleted.

If livetime -1 -> board deleted....

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] livetime of boards

[Heiko Schocher]

Hello Andreas,

Am 07.11.2013 12:24, schrieb Andreas Bie?mann:
> Hello Heiko,
>
> On 11/07/2013 11:39 AM, Heiko Schocher wrote:
>> Am 07.11.2013 10:37, schrieb Andreas Bie?mann:
>>> On 11/07/2013 09:17 AM, Heiko Schocher wrote:
>>>> Am 06.11.2013 08:50, schrieb Wolfgang Denk:
>>>>> In message<20131105203736.GM5925@bill-the-cat>    you wrote:
>
> <snip>
>
>>>> But you are right, that approach leads in a lot of conflicting
>>>> patches ... but I think, we just pooled board information in boards.cfg,
>>>> so this would be the right place in my eyes ...
>>>>
>>>> Maybe we get such Information "a Boards is tested with current mainline"
>>>> inform of an EMail with an Text "Board xy tested with commit mm. Please
>>>> update livetime" ... and we can add a script, which updates the
>>>> livetime for this board, so we can prevent conflicting patches ... ?
>>>
>>> I agree here with Tom. Beside the possibility of conflicting pahces I
>>> see another problem here.
>>> We will get a lot of patches just for increasing the tested counter for
>>> a single board. These patches needs to be handled in some way. If we
>>> shift to some integrated system (gerrit comes to mind) this could be
>>> easier than today, but it will bind resources anyways.
>>> Therefore I think it is a bad idea to save a such often changing
>>> information in the source code repository.
>>
>> I see this info just changing once when releasing a new U-Boot version.
>
> The saved information how often a board was runtime tested with the
> correct SHA1 of the u-boot/master could be quite useful.
> In the end just the last tested commit will be interesting but it could
> give some information how often that specific board is used. The
> information must not be generated by a board maintainer ... the
> maintainer could then see if he needs to pull out a board or if one else
> run the test before.
>
> If we would save this in the repository we do not have this information
> in time.
> If we send the information to a list we need to parse it or use some
> other tool to provide the information.
> Beside that we will pollute the list with status updates about boards
> being tested. It could be hard to find real patches in that information
> flood then.

Hmm... I hope we get a lot of such EMails ... and think, this is not
a big problem ... Or, maybe, if we get a lot of such EMails, maybe we
open a u-boot-testing list?

> <snip mail proposal>
>
>> So (in current case Tom) should, before releasing a new U-Boot
>> version, first call this script "collect_livetime_info" and he get:
>>
>> ->  one livetime counter patch for current release
>> ->  one list for boards which reach end of life
>> ->  one list for boards, which should be deleted
>
> Good idea, but the information could also be saved on a website or in
> another database.
> It should be easily filled by the tester and also easily queried by
> wherever is interested in.

Ok, if we have this info, we can show it wherever we want ...

>> All Infos are "release info" I think, and fully fit in the commit
>> for the new release ...
>
> I also think that should be done on release only.

Yep! But collecting this infos can be done all the time.

>> ... maybe "deleting boards" can be done automatically, but this is
>> not a trivial job ...
>
> I think deleting should be done in next release then to give the board
> maintainer some time to check the boards. On a new release the board
> maintainer should be mailed that in the next release the board will be
> removed. We should also store this somewhere in the code (status in
> boards.cfg?).

See my proposal for the livetime counter:

livetime init value n (n=4)
livetimer decrement on every new release
livetimer set to n, if in release cycle comes a test report
livetimer == 0 -> EMail to board maintainer, board reached end of live in
                   mainline, please send test report.
livetimer == -1 -> board get deleted

So all info is in boards.cfg availiable ...

> Next question is what to do if the mail bounces ;)

Board gets deleted, as board maintainer didn;t send an update patch
for boards.cfg ...

>> So, with such a solution, I see no big additional cost for adding
>> such a feature (except the task "deleting old boards", which is maybe
>> not trivial)
>>
>> Do not understand me wrong, if we find another solution, I am
>> happy also ... just spinning around ...
>
> Me too.
>
> <snip>
>
>>>> If we decide to delete older boards after n release cycles without
>>>> testreports, we must not decide nor look in a database. We are
>>>> sure, we have only "good and working" boards ... and we just
>>>> do the necessary work for new features ... and we are sure, that
>>>> we get back testreports within n release cycles ...
>>>>
>>>> So let us decide first, if we want to go this way ...
>>>
>>> Yes, we should introduce some mechanism to check when a specific board
>>> was last runtime tested. But I fear the overhead with patches that
>>> update a tested counter.
>>
>> I thought with "decide": Do we want to delete "old boards"?
>> With this, we do not need a "MAKEALL --check-boards -s at91" when
>> we introduce new features, as all boards in mainline are in a well
>> tested shape ...
>>
>> Ok, two decisions:
>>
>> - Do we want to collect board testinginformation?
>
> I think we should do that i none way or another.

Yep.

>> - Do we want to delete old boards automatically after we do not get
>>    some test reports after a time intervall?
>
> And we should delete 'unmaintained' boards, when is to be discussed. I'm
> currently fiddling with at91 gpio and ask myself if I should adopt all
> the boards or just let them fail ...

You do not have this problem when we descide to delete old boards!

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] livetime of boards

[Wolfgang Denk]

Dear "Andreas Bie?mann",

In message <527B7883.1080302@gmail.com> you wrote:
> 
> The saved information how often a board was runtime tested with the
> correct SHA1 of the u-boot/master could be quite useful.
> In the end just the last tested commit will be interesting but it could
> give some information how often that specific board is used. The
> information must not be generated by a board maintainer ... the

s/must not/need not/   (faux ami in action, I guess)

> maintainer could then see if he needs to pull out a board or if one else
> run the test before.

I fully agree - everybody should be able to provide such test
information.  Actually it would be a big help to board maintainers as
well if these would get test reports from actual users of the
hardware.

> If we would save this in the repository we do not have this information
> in time.
> If we send the information to a list we need to parse it or use some
> other tool to provide the information.
> Beside that we will pollute the list with status updates about boards
> being tested. It could be hard to find real patches in that information
> flood then.

Agreed too. I doubt if a mailing list makes sense to collect such
data. It would probably be more efficient to provide a web based
service for this. It just has to be easy to submit reports, and to
query the status for boards.

> > -> one livetime counter patch for current release
> > -> one list for boards which reach end of life
> > -> one list for boards, which should be deleted
> 
> Good idea, but the information could also be saved on a website or in
> another database.
> It should be easily filled by the tester and also easily queried by
> wherever is interested in.

Agreed.  I definitely do not want to see such trafic on the regular
U-Boot mailing list.

> > All Infos are "release info" I think, and fully fit in the commit
> > for the new release ...
> 
> I also think that should be done on release only.

Why?  To me it makes a lot of sense to also collect information on
intermediate snapshots.

> I think deleting should be done in next release then to give the board
> maintainer some time to check the boards. On a new release the board
> maintainer should be mailed that in the next release the board will be
> removed. We should also store this somewhere in the code (status in
> boards.cfg?).
> 
> Next question is what to do if the mail bounces ;)

Mail bounces (and new address of maintainer cannot be found, and no
other user volunteers to take over maintenance) => board is
unmaintained => board gets removed.

> > - Do we want to delete old boards automatically after we do not get
> >   some test reports after a time intervall?
> 
> And we should delete 'unmaintained' boards, when is to be discussed. I'm
> currently fiddling with at91 gpio and ask myself if I should adopt all
> the boards or just let them fail ...

I hesitate to automatically remove existing boards.  Why would we want
to do that?  To reduce efforts, right?  So I vote to keep boards as
long as they are either maintained, or they at least "do not hurt".
If a board just builds fine and does not cause any additional efforts
we should keep it, no matter if there is an active maintainer or test
reports or not.  Only when a board becomes a pain to somebody - say,
because it develops build errors, or wit would require efforts to
adapt it to some new feature, _then_ we would check if this is one of
the "precious" boards we want to keep or if it is just old cruft
nobody cares about anyway.  And only then I would remove it.


Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Simplicity boils down to two steps: Identify the essential. Eliminate
the rest.                                               - Leo Babauta

[U-Boot] livetime of boards

[Wolfgang Denk]

Dear Heiko Schocher,

In message <527B7CB0.6040707@denx.de> you wrote:
> 
> > Note also that one goal of boards.cfg is to not have multiple files
> > around that have to remain consistent.
> 
> Yep, exactly. Thats why I think we could collect this in boards.cfg.

No, we really want to have a database here, which collects more than
just a timestamp.  I would lile to be able to see the history as well
as information which exact commit ID has been tested (and I strongly
vote to allow testing at any time, not only for a end-of-release-
cycle version).  we should probably also collect information about the
build environment (at least versions of make, gcc, bintuils), etc.

This by far exceeds what could be done in boards.cfg, and it exceeds
anything that could be run over the mailing list.


Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
The important thing about being a leader is not being right or wrong,
but being *certain*.                    - Terry Pratchett, _Truckers_

[U-Boot] livetime of boards

[Wolfgang Denk]

Dear Heiko,

In message <527B7EE6.4030307@denx.de> you wrote:
> 
> Hmm... I hope we get a lot of such EMails ... and think, this is not
> a big problem ... Or, maybe, if we get a lot of such EMails, maybe we
> open a u-boot-testing list?

NAK.  Mailing lists are good for some kind of information - especially
for such information that is read by humans.

All you want to do here is feed a database with data.  This is not
what mailing lists were made for, so we should really use a more
appropriate interface.

> See my proposal for the livetime counter:
> 
> livetime init value n (n=4)
> livetimer decrement on every new release
> livetimer set to n, if in release cycle comes a test report
> livetimer == 0 -> EMail to board maintainer, board reached end of live in
>                    mainline, please send test report.
> livetimer == -1 -> board get deleted

This is too simple in one respect (as you are not including
information that may be vital, like which exact commit ID has been
tested, or which tool chain has been used to build it).

Also we should probably not only allow for positive test reports, but
also allow to report failures (to mark board support as broken).

All this cannot be done in  boards.cfg .

And there is no need to as long as we provide tools to query for any
information we might be interested in.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Every living thing wants to survive.
	-- Spock, "The Ultimate Computer", stardate 4731.3

[U-Boot] livetime of boards

[Andreas Bießmann]

Dear Wolfgang Denk,

On 11/07/2013 01:01 PM, Wolfgang Denk wrote:
> In message <527B7883.1080302@gmail.com> you wrote:
>>
>> The saved information how often a board was runtime tested with the
>> correct SHA1 of the u-boot/master could be quite useful.
>> In the end just the last tested commit will be interesting but it could
>> give some information how often that specific board is used. The
>> information must not be generated by a board maintainer ... the
> 
> s/must not/need not/   (faux ami in action, I guess)

You're right

<snip>

>>> All Infos are "release info" I think, and fully fit in the commit
>>> for the new release ...
>>
>> I also think that should be done on release only.
> 
> Why?  To me it makes a lot of sense to also collect information on
> intermediate snapshots.

Well, I think we should query that database on release and transform the
testing information to some information maybe stored in release code and
to trigger board maintainers to do tests.
Maybe the proposed lifetime counter is the correct tooling here.

We should introduce some service to gather testing information which
should have quite high throughput.
But we should also install some measures to see the 'liveliness' of some
board's tests in the released source code.

>> I think deleting should be done in next release then to give the board
>> maintainer some time to check the boards. On a new release the board
>> maintainer should be mailed that in the next release the board will be
>> removed. We should also store this somewhere in the code (status in
>> boards.cfg?).
>>
>> Next question is what to do if the mail bounces ;)
> 
> Mail bounces (and new address of maintainer cannot be found, and no
> other user volunteers to take over maintenance) => board is
> unmaintained => board gets removed.

Sounds good, but doesn't fit to your next statement ...

>>> - Do we want to delete old boards automatically after we do not get
>>>   some test reports after a time intervall?
>>
>> And we should delete 'unmaintained' boards, when is to be discussed. I'm
>> currently fiddling with at91 gpio and ask myself if I should adopt all
>> the boards or just let them fail ...
> 
> I hesitate to automatically remove existing boards.  Why would we want
> to do that?  To reduce efforts, right?  So I vote to keep boards as
> long as they are either maintained, or they at least "do not hurt".
> If a board just builds fine and does not cause any additional efforts
> we should keep it, no matter if there is an active maintainer or test
> reports or not.  Only when a board becomes a pain to somebody - say,
> because it develops build errors, or wit would require efforts to
> adapt it to some new feature, _then_ we would check if this is one of
> the "precious" boards we want to keep or if it is just old cruft
> nobody cares about anyway.  And only then I would remove it.

Sounds good.

Best regards

Andreas Bie?mann

[U-Boot] livetime of boards

[Heiko Schocher]

Hello Wolfgang,

Am 07.11.2013 13:06, schrieb Wolfgang Denk:
> Dear Heiko Schocher,
>
> In message<527B7CB0.6040707@denx.de>  you wrote:
>>
>>> Note also that one goal of boards.cfg is to not have multiple files
>>> around that have to remain consistent.
>>
>> Yep, exactly. Thats why I think we could collect this in boards.cfg.
>
> No, we really want to have a database here, which collects more than
> just a timestamp.  I would lile to be able to see the history as well
> as information which exact commit ID has been tested (and I strongly
> vote to allow testing at any time, not only for a end-of-release-
> cycle version).  we should probably also collect information about the
> build environment (at least versions of make, gcc, bintuils), etc.
>
> This by far exceeds what could be done in boards.cfg, and it exceeds
> anything that could be run over the mailing list.

Ok, if we need all such information, it breaks boards.cfg, correct.

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] livetime of boards

[Heiko Schocher]

Hello Wolfgang,

Am 07.11.2013 13:01, schrieb Wolfgang Denk:
> Dear "Andreas Bie?mann",
>
> In message<527B7883.1080302@gmail.com>  you wrote:
>>
[...]

>> maintainer could then see if he needs to pull out a board or if one else
>> run the test before.
>
> I fully agree - everybody should be able to provide such test
> information.  Actually it would be a big help to board maintainers as
> well if these would get test reports from actual users of the
> hardware.

Yep.

>> If we would save this in the repository we do not have this information
>> in time.
>> If we send the information to a list we need to parse it or use some
>> other tool to provide the information.
>> Beside that we will pollute the list with status updates about boards
>> being tested. It could be hard to find real patches in that information
>> flood then.
>
> Agreed too. I doubt if a mailing list makes sense to collect such
> data. It would probably be more efficient to provide a web based
> service for this. It just has to be easy to submit reports, and to
> query the status for boards.

Ok, how would look like such a web based service?

>>> ->  one livetime counter patch for current release
>>> ->  one list for boards which reach end of life
>>> ->  one list for boards, which should be deleted
>>
>> Good idea, but the information could also be saved on a website or in
>> another database.
>> It should be easily filled by the tester and also easily queried by
>> wherever is interested in.
>
> Agreed.  I definitely do not want to see such trafic on the regular
> U-Boot mailing list.

Oh, ok ... then we must look for another way.

>>> All Infos are "release info" I think, and fully fit in the commit
>>> for the new release ...
>>
>> I also think that should be done on release only.
>
> Why?  To me it makes a lot of sense to also collect information on
> intermediate snapshots.

Hmm.. I thought we get a "test report" (however this looks like in the
end) based on a commit id in u-boot/master ... isn;t this enough?

>> I think deleting should be done in next release then to give the board
>> maintainer some time to check the boards. On a new release the board
>> maintainer should be mailed that in the next release the board will be
>> removed. We should also store this somewhere in the code (status in
>> boards.cfg?).
>>
>> Next question is what to do if the mail bounces ;)
>
> Mail bounces (and new address of maintainer cannot be found, and no
> other user volunteers to take over maintenance) =>  board is
> unmaintained =>  board gets removed.

Full Ack.

>>> - Do we want to delete old boards automatically after we do not get
>>>    some test reports after a time intervall?
>>
>> And we should delete 'unmaintained' boards, when is to be discussed. I'm
>> currently fiddling with at91 gpio and ask myself if I should adopt all
>> the boards or just let them fail ...
>
> I hesitate to automatically remove existing boards.  Why would we want
> to do that?  To reduce efforts, right?  So I vote to keep boards as
> long as they are either maintained, or they at least "do not hurt".

Why the "not hurt case"? Is it really good to have a lot of boards,
which compile clean, but we do not know, if the code really works?

I prefer to have in current mainline only boards, which really work
or at least maintained... if a board maintainer did the work to
bring it into mainline, it should be interested in "stay in" mainline.
If this interest is lost and no other volunteers ... board is useless
in mainline.

Code for old boards is not lost.

> If a board just builds fine and does not cause any additional efforts
> we should keep it, no matter if there is an active maintainer or test
> reports or not.  Only when a board becomes a pain to somebody - say,
> because it develops build errors, or wit would require efforts to
> adapt it to some new feature, _then_ we would check if this is one of
> the "precious" boards we want to keep or if it is just old cruft
> nobody cares about anyway.  And only then I would remove it.

Ok, that is a way to go, if we have for all boards the information,
in which maintainstate it is ... but think of for example the new
i2c framework, how much boards use I2C? I have to check now all
this boards, decide to delete it or convert it  ... very time consuming
frustrating work ... and maybe for a lot of "do not hurt" boards only
waste of time ... :-(

if we have a clean mainline state, where I know all boards are working
the decision is clear, convert all ... and board maintainers will test
it ... and we have always a clean compile state for mainline

And if we have this test/delete cycle, I can be sure, that after a
defined time all boards in mainline are working!

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] livetime of boards

[Heiko Schocher]

Hello Wolfgang,

Am 07.11.2013 13:12, schrieb Wolfgang Denk:
> Dear Heiko,
>
> In message<527B7EE6.4030307@denx.de>  you wrote:
>>
>> Hmm... I hope we get a lot of such EMails ... and think, this is not
>> a big problem ... Or, maybe, if we get a lot of such EMails, maybe we
>> open a u-boot-testing list?
>
> NAK.  Mailing lists are good for some kind of information - especially
> for such information that is read by humans.
>
> All you want to do here is feed a database with data.  This is not
> what mailing lists were made for, so we should really use a more
> appropriate interface.

Ok. But this status report can be in readable text format ;-)

>> See my proposal for the livetime counter:
>>
>> livetime init value n (n=4)
>> livetimer decrement on every new release
>> livetimer set to n, if in release cycle comes a test report
>> livetimer == 0 ->  EMail to board maintainer, board reached end of live in
>>                     mainline, please send test report.
>> livetimer == -1 ->  board get deleted
>
> This is too simple in one respect (as you are not including
> information that may be vital, like which exact commit ID has been
> tested, or which tool chain has been used to build it).

Ok.

> Also we should probably not only allow for positive test reports, but
> also allow to report failures (to mark board support as broken).

Yep, this will set livetime = 0 -> EMail to board maintainer ...

> All this cannot be done in  boards.cfg .
>
> And there is no need to as long as we provide tools to query for any
> information we might be interested in.

Ok, then we must define this way ...

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] livetime of boards

[Tom Rini]

On Thu, Nov 07, 2013 at 10:37:24AM +0100, Andreas Bie?mann wrote:
> Hello all together,
> 
> On 11/07/2013 09:17 AM, Heiko Schocher wrote:
> > Am 06.11.2013 08:50, schrieb Wolfgang Denk:
[snip]
> >> So when you're once again doing some change that requires touching
> >> files for some othe rboards, you could simply check that database.  If
> >> you see that 3 out of the last 5 releases have reported succesful
> >> run-time tests you will probably decide to accept the needed efforts,
> > 
> > Hmm.. that works, if you have to touch some (some < 5) boards. But
> > If you have to touch > 5 boards, this gets unhandy...
> 
> How about:
> 
> MAKEALL --check-boards -s at91
> 
> ;)

I feel this is the hard part of the problem, and what we're glossing
over.  What has to be tested by the board maintainer?  What are we going
to leave to their discretion?  Will am335x_evm not count if I don't dig
up the NOR cape for it?

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20131107/79a77722/attachment.pgp>

[U-Boot] livetime of boards

[Andreas Bießmann]

Dear Tom Rini,

On 11/07/2013 02:31 PM, Tom Rini wrote:
> On Thu, Nov 07, 2013 at 10:37:24AM +0100, Andreas Bie?mann wrote:
>> Hello all together,
>>
>> On 11/07/2013 09:17 AM, Heiko Schocher wrote:
>>> Am 06.11.2013 08:50, schrieb Wolfgang Denk:
> [snip]
>>>> So when you're once again doing some change that requires touching
>>>> files for some othe rboards, you could simply check that database.  If
>>>> you see that 3 out of the last 5 releases have reported succesful
>>>> run-time tests you will probably decide to accept the needed efforts,
>>>
>>> Hmm.. that works, if you have to touch some (some < 5) boards. But
>>> If you have to touch > 5 boards, this gets unhandy...
>>
>> How about:
>>
>> MAKEALL --check-boards -s at91
>>
>> ;)
> 
> I feel this is the hard part of the problem, and what we're glossing
> over.  What has to be tested by the board maintainer?  What are we going
> to leave to their discretion?  Will am335x_evm not count if I don't dig
> up the NOR cape for it?

for the time being I'd glad to see reports of (un)successful boot with
configured bootm command.

But I see your point, there is another input vector for the tests. I
think this could only be defined on a per board basis.
To pick up your example, I think it is worth to know that one tested the
am335x_evm to boot via NAND. At least for the maintainer, so he can skip
that and just test the NOR booting.

Best regards

Andreas Bie?mann

[U-Boot] livetime of boards

[Wolfgang Denk]

Dear Heiko,

In message <527B8BA7.2070605@denx.de> you wrote:
> 
> > Agreed too. I doubt if a mailing list makes sense to collect such
> > data. It would probably be more efficient to provide a web based
> > service for this. It just has to be easy to submit reports, and to
> > query the status for boards.
> 
> Ok, how would look like such a web based service?

I have no idea.  Not yet.  Let's first define what we want to have,
then think about how to implement it.

> >> I also think that should be done on release only.
> >
> > Why?  To me it makes a lot of sense to also collect information on
> > intermediate snapshots.
> 
> Hmm.. I thought we get a "test report" (however this looks like in the
> end) based on a commit id in u-boot/master ... isn;t this enough?

Yes, this is perfectly fine.  I just want to allow this at _any_ time,
not only once per release (near the end of the release cycle).
especially for releases where bigger changes get merged it may be
precious information to know when the code stopped working.

> > I hesitate to automatically remove existing boards.  Why would we want
> > to do that?  To reduce efforts, right?  So I vote to keep boards as
> > long as they are either maintained, or they at least "do not hurt".
> 
> Why the "not hurt case"? Is it really good to have a lot of boards,
> which compile clean, but we do not know, if the code really works?

Well, one reason is efficiency.  If the code builds fine, and does not
cause efforts druing any of the ongoing work, it is more efficient to
just keep it than to actively remove it, which would require active
work.  I. e. I want to reduce the work load on the maintainer(s) such
that they only have to become active if such action saves even more
effort.  Actually it may even seem more efficient in some cases to
perform minor fixing even of unmaintained boards than to remove them.

> I prefer to have in current mainline only boards, which really work
> or at least maintained... if a board maintainer did the work to
> bring it into mainline, it should be interested in "stay in" mainline.
> If this interest is lost and no other volunteers ... board is useless
> in mainline.

But should we not also try to minimize efforts, especially on the
custodians?  If a board does not cause any trouble, we should not have
to invent additional efforts for it.

> Ok, that is a way to go, if we have for all boards the information,
> in which maintainstate it is ... but think of for example the new
> i2c framework, how much boards use I2C? I have to check now all
> this boards, decide to delete it or convert it  ... very time consuming
> frustrating work ... and maybe for a lot of "do not hurt" boards only
> waste of time ... :-(

I don't really understand this argument. The I2C code is either
hardware independent (say, the command line interface code), or it is
platform specific (say, the driver code for the I2C controller on a
specific SoC), or it is board dendent (say, some specific twiddeling
with I2C devices to perform some magic operations on a board).

In the first two cases the work wil have to be done in any case
(except for the realy rare case that there are only old, unmaintained
boards left using this SoC).  An for the board specific code you can
check if you need to adapt it by checking the board's test status.  If
the board is unmaintained, then we enter the "it hurts" branch, and
drop the board.

> if we have a clean mainline state, where I know all boards are working
> the decision is clear, convert all ... and board maintainers will test
> it ... and we have always a clean compile state for mainline

Again: you don't need any knowledge about boards that are not affected
by your I2C changes.

> And if we have this test/delete cycle, I can be sure, that after a
> defined time all boards in mainline are working!

Yes, but the cost is additional efforts, and being more aggressive
than necessary.  I dislike both.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Life would be so much easier if everyone read the manual.

[U-Boot] livetime of boards

[Wolfgang Denk]

Dear Heiko Schocher,

In message <527B8C7F.6060503@denx.de> you wrote:
> 
> > All you want to do here is feed a database with data.  This is not
> > what mailing lists were made for, so we should really use a more
> > appropriate interface.
> 
> Ok. But this status report can be in readable text format ;-)

Yes, it can. So do you want to see 250 "test passed" messages for the
BeagleBone or the Rasperry Pi for every few commits that get merged?
I don't.  In a database, we can automatically filter redundant
information.

You, you can use a mailing list for submitting such information, but I
doubt that it would be efficient.  And I definitely do not want to see
this on the current U-Boot ML.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"You ain't experienced..." "Well, nor are you." "That's true. But the
point is ... the point is ... the point is we've been not experienced
for a lot longer than you. We've got  a  lot  of  experience  of  not
having any experience."           - Terry Pratchett, _Witches Abroad_

[U-Boot] livetime of boards

[Wolfgang Denk]

Dear Tom,

In message <20131107133159.GR5925@bill-the-cat> you wrote:
> 
> I feel this is the hard part of the problem, and what we're glossing
> over.  What has to be tested by the board maintainer?  What are we going
> to leave to their discretion?  Will am335x_evm not count if I don't dig
> up the NOR cape for it?

Good question.  Eventually this is something that develops over time.

Intially, we might be satisfied with a very basic "it works" message,
which may just mean that this specific version booted on the actual
hardware.

In the long run, we might provide a more detailed questionaire to the
reported.  I could for example imagine a tool that parses the board's
config file and then provides some checkboxes - if there is NOR flash
configured on the board, ask if NOR has been tested; similar for
network, MMC, USB, ... other features.

One day we might even have more developers using automatic test tools
so we could generate information on a per-command base.

I know that I'm just dreaming, but we should try to just be open for
any such future extensions, even if we start really small now.

I think we all agree that _any_ kind of test information will be
better than none.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
You see things; and you say ``Why?'' But I dream  things  that  never
were; and I say ``Why not?''
       - George Bernard Shaw _Back to Methuselah_ (1921) pt. 1, act 1

[U-Boot] livetime of boards

[Tom Rini]

On Thu, Nov 07, 2013 at 08:26:57PM +0100, Wolfgang Denk wrote:
> Dear Tom,
> 
> In message <20131107133159.GR5925@bill-the-cat> you wrote:
> > 
> > I feel this is the hard part of the problem, and what we're glossing
> > over.  What has to be tested by the board maintainer?  What are we going
> > to leave to their discretion?  Will am335x_evm not count if I don't dig
> > up the NOR cape for it?
> 
> Good question.  Eventually this is something that develops over time.
> 
> Intially, we might be satisfied with a very basic "it works" message,
> which may just mean that this specific version booted on the actual
> hardware.
> 
> In the long run, we might provide a more detailed questionaire to the
> reported.  I could for example imagine a tool that parses the board's
> config file and then provides some checkboxes - if there is NOR flash
> configured on the board, ask if NOR has been tested; similar for
> network, MMC, USB, ... other features.
> 
> One day we might even have more developers using automatic test tools
> so we could generate information on a per-command base.
> 
> I know that I'm just dreaming, but we should try to just be open for
> any such future extensions, even if we start really small now.
> 
> I think we all agree that _any_ kind of test information will be
> better than none.

What we need to be careful of here is making sure whatever we grow is
both useful and not overly complicated.  What I honestly wonder about is
automated testing for commands (crc32 pops to mind only because I just
fixed things) but otherwise having things broken down into a front end
where people select what they did "Booted a ___ into ___ via ___",
provide some output from a command (maybe add just a touch more info to
'version') and cover non-boot testing with copy/paste'able drop-downs.

I know automated testing is The Thing, but given N frameworks, everyone
of them has issues because frankly, every SoC family has its own quirks
about how boot and load and what is and is not even feasible, especially
for the bootloader.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20131107/66b988cc/attachment.pgp>

[U-Boot] livetime of boards

[Wolfgang Denk]

Dear Tom,

In message <20131107205133.GT5925@bill-the-cat> you wrote:
> 
> What we need to be careful of here is making sure whatever we grow is
> both useful and not overly complicated.  What I honestly wonder about is
> automated testing for commands (crc32 pops to mind only because I just
> fixed things) but otherwise having things broken down into a front end
> where people select what they did "Booted a ___ into ___ via ___",
> provide some output from a command (maybe add just a touch more info to
> 'version') and cover non-boot testing with copy/paste'able drop-downs.
> 
> I know automated testing is The Thing, but given N frameworks, everyone
> of them has issues because frankly, every SoC family has its own quirks
> about how boot and load and what is and is not even feasible, especially
> for the bootloader.

Agreed.  In the end, you will probably define a specific set of test
cases that can be run automatically on a board.  We will never have
100% coverage, nor any generic set of tests that fits all boards.

OK, a pretty large sub-set of common functionality can be tested in
the sandbox, which is a great achievement ...

I still firmly belive that the approach we've taken a long time ago,
i. e. to use a test framework (DUTS) in combination with a (today wiki
based) documentation framework (DULG) is something that can meet a
wide range of requirements.

Unfortunately, we haven't been able yet to find a test framework that
makes it easy for the average user to add new test cases.  I'm not
even talking here about the core of the framework, that has to deal
with things like attaching to the console of a board, control power
and/or reset state, detect if a JTAG debugger is attached and such
things.  In the current state, our code is expect based (and thus
implemented in tcl), and it's really a PITA to add new test cases to
it.

BUt it would require significant efforts to rwrite this in a more
modern context...

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
In C we had to code our own bugs, in C++ we can inherit them.

[U-Boot] livetime of boards

[Heiko Schocher]

Hello Tom,

Am 07.11.2013 14:31, schrieb Tom Rini:
> On Thu, Nov 07, 2013 at 10:37:24AM +0100, Andreas Bie?mann wrote:
>> Hello all together,
>>
>> On 11/07/2013 09:17 AM, Heiko Schocher wrote:
>>> Am 06.11.2013 08:50, schrieb Wolfgang Denk:
> [snip]
>>>> So when you're once again doing some change that requires touching
>>>> files for some othe rboards, you could simply check that database.  If
>>>> you see that 3 out of the last 5 releases have reported succesful
>>>> run-time tests you will probably decide to accept the needed efforts,
>>>
>>> Hmm.. that works, if you have to touch some (some<  5) boards. But
>>> If you have to touch>  5 boards, this gets unhandy...
>>
>> How about:
>>
>> MAKEALL --check-boards -s at91
>>
>> ;)
>
> I feel this is the hard part of the problem, and what we're glossing
> over.  What has to be tested by the board maintainer?  What are we going
> to leave to their discretion?  Will am335x_evm not count if I don't dig
> up the NOR cape for it?

Good point! What gets tested on a board can only decide the person who
execute the test on the board. We must trust, that it covers hopefully
all functionality ...

Hmm... would we want to introduce testscripts to U-Boot, which should
be executed on boards? Hehe... I see, this discussion breaks my fast
thought, of introducing just a small and fast livetimer mechanism ;-)

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] livetime of boards

[Heiko Schocher]

Hello Wolfgang,

Am 07.11.2013 20:15, schrieb Wolfgang Denk:
> Dear Heiko,
>
> In message<527B8BA7.2070605@denx.de>  you wrote:
>>
>>> Agreed too. I doubt if a mailing list makes sense to collect such
>>> data. It would probably be more efficient to provide a web based
>>> service for this. It just has to be easy to submit reports, and to
>>> query the status for boards.
>>
>> Ok, how would look like such a web based service?
>
> I have no idea.  Not yet.  Let's first define what we want to have,
> then think about how to implement it.

Ok.

>>>> I also think that should be done on release only.
>>>
>>> Why?  To me it makes a lot of sense to also collect information on
>>> intermediate snapshots.
>>
>> Hmm.. I thought we get a "test report" (however this looks like in the
>> end) based on a commit id in u-boot/master ... isn;t this enough?
>
> Yes, this is perfectly fine.  I just want to allow this at _any_ time,
> not only once per release (near the end of the release cycle).
> especially for releases where bigger changes get merged it may be
> precious information to know when the code stopped working.

With "once", I only meant, we once update the "livetimer" and collect
all the time test reports!

>>> I hesitate to automatically remove existing boards.  Why would we want
>>> to do that?  To reduce efforts, right?  So I vote to keep boards as
>>> long as they are either maintained, or they at least "do not hurt".
>>
>> Why the "not hurt case"? Is it really good to have a lot of boards,
>> which compile clean, but we do not know, if the code really works?
>
> Well, one reason is efficiency.  If the code builds fine, and does not
> cause efforts druing any of the ongoing work, it is more efficient to
> just keep it than to actively remove it, which would require active
> work.  I. e. I want to reduce the work load on the maintainer(s) such
> that they only have to become active if such action saves even more
> effort.  Actually it may even seem more efficient in some cases to
> perform minor fixing even of unmaintained boards than to remove them.

But in the long term I think, we have more work with old, unmaintained
boards, then removing it. And if we remove boards actively, we maybe
force more the board maintainers (for examples corporates who have
interest that the board stays in mainline) to really test them ;-)

>> I prefer to have in current mainline only boards, which really work
>> or at least maintained... if a board maintainer did the work to
>> bring it into mainline, it should be interested in "stay in" mainline.
>> If this interest is lost and no other volunteers ... board is useless
>> in mainline.
>
> But should we not also try to minimize efforts, especially on the
> custodians?  If a board does not cause any trouble, we should not have
> to invent additional efforts for it.

Hmm... I am not really sure, if it is more work for removing a
board or fixing compiling issues ...

>> Ok, that is a way to go, if we have for all boards the information,
>> in which maintainstate it is ... but think of for example the new
>> i2c framework, how much boards use I2C? I have to check now all
>> this boards, decide to delete it or convert it  ... very time consuming
>> frustrating work ... and maybe for a lot of "do not hurt" boards only
>> waste of time ... :-(
>
> I don't really understand this argument. The I2C code is either
> hardware independent (say, the command line interface code), or it is
> platform specific (say, the driver code for the I2C controller on a
> specific SoC), or it is board dendent (say, some specific twiddeling
> with I2C devices to perform some magic operations on a board).
>
> In the first two cases the work wil have to be done in any case
> (except for the realy rare case that there are only old, unmaintained
> boards left using this SoC).  An for the board specific code you can
 >
> check if you need to adapt it by checking the board's test status.  If
> the board is unmaintained, then we enter the "it hurts" branch, and
> drop the board.

Hmm... :

pollux:u-boot hs [master] $ grep -lr HARD_I2C arch/powerpc/
[...]
arch/powerpc/cpu/mpc8xx/i2c.c
[...]
arch/powerpc/cpu/mpc8260/i2c.c
arch/powerpc/cpu/mpc5xxx/i2c.c
arch/powerpc/cpu/mpc824x/drivers/i2c/i2c.c
arch/powerpc/cpu/mpc512x/i2c.c
pollux:u-boot hs [master] $

Is it worth to move the drivers to drivers/i2c/ and convert them to
the new framework? Or is it better to delete them and the boards,
which use them ... Ok, if we had such a "livetimer" without deleting
old boards, you are right, I could do now:

- search the boards, which use the drivers
- search the state of this board
- decide, dropping or convert ...
   On which criteria whould this decision be done? I have no idea
   if this boards exist anymore ...

If we have only good boards, I have only one way -> convert. No
time wasted for the above steps ...

So, yes, I think we do not need to delete the boards, just mark
it broken ...

My hope with "delete old boards from mainline" is, that we become
more testreports from board maintainers, as they (should) have
interest that the board support stays in mainline. If they know,
that the board is only marked as broken, old or whatever ... they
maybe do not reserve the time for testing a board once a year ...

>> if we have a clean mainline state, where I know all boards are working
>> the decision is clear, convert all ... and board maintainers will test
>> it ... and we have always a clean compile state for mainline
>
> Again: you don't need any knowledge about boards that are not affected
> by your I2C changes.
>
>> And if we have this test/delete cycle, I can be sure, that after a
>> defined time all boards in mainline are working!
>
> Yes, but the cost is additional efforts, and being more aggressive
> than necessary.  I dislike both.

I dislike additional efforts too (But the question is, needs removing
old boards more effort than fixing them all the time maybe uselessly?).
Is it really aggressive to want once a year a testreport from
board maintainers? They get a lot of effort from the mainline for
free...

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] livetime of boards

[Heiko Schocher]

Hello Wolfgang,

Am 07.11.2013 20:19, schrieb Wolfgang Denk:
> Dear Heiko Schocher,
>
> In message<527B8C7F.6060503@denx.de>  you wrote:
>>
>>> All you want to do here is feed a database with data.  This is not
>>> what mailing lists were made for, so we should really use a more
>>> appropriate interface.
>>
>> Ok. But this status report can be in readable text format ;-)
>
> Yes, it can. So do you want to see 250 "test passed" messages for the
> BeagleBone or the Rasperry Pi for every few commits that get merged?
> I don't.  In a database, we can automatically filter redundant
> information.

Ok, thats a good point!

> You, you can use a mailing list for submitting such information, but I
> doubt that it would be efficient.  And I definitely do not want to see
> this on the current U-Boot ML.

Ok, let us discuss the way we collect such information, if we decided
that we want testreports ...

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] livetime of boards

[Heiko Schocher]

Hello Wolfgang,

Am 07.11.2013 20:26, schrieb Wolfgang Denk:
> Dear Tom,
>
> In message<20131107133159.GR5925@bill-the-cat>  you wrote:
>>
>> I feel this is the hard part of the problem, and what we're glossing
>> over.  What has to be tested by the board maintainer?  What are we going
>> to leave to their discretion?  Will am335x_evm not count if I don't dig
>> up the NOR cape for it?
>
> Good question.  Eventually this is something that develops over time.

Yes, let us go step by step.

> Intially, we might be satisfied with a very basic "it works" message,
> which may just mean that this specific version booted on the actual
> hardware.
>
> In the long run, we might provide a more detailed questionaire to the
> reported.  I could for example imagine a tool that parses the board's
> config file and then provides some checkboxes - if there is NOR flash
> configured on the board, ask if NOR has been tested; similar for
> network, MMC, USB, ... other features.

Cool... and we can maybe start testscripts based on this information...
(Huch... I am sure I am sitting in Front of my PC and not sleeping
  anymore)

> One day we might even have more developers using automatic test tools
> so we could generate information on a per-command base.
>
> I know that I'm just dreaming, but we should try to just be open for
> any such future extensions, even if we start really small now.

Maybe we add in tools a directory, where we add such tests?
But this would be another big discussion...

> I think we all agree that _any_ kind of test information will be
> better than none.

Yes.

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] livetime of boards

[Wolfgang Denk]

Dear Heiko,

In message <527C767D.3070700@denx.de> you wrote:
> 
> > Yes, this is perfectly fine.  I just want to allow this at _any_ time,
> > not only once per release (near the end of the release cycle).
> > especially for releases where bigger changes get merged it may be
> > precious information to know when the code stopped working.
> 
> With "once", I only meant, we once update the "livetimer" and collect
> all the time test reports!

But with your suggestion, the only stoprage for such information is
the life timer entry in the boards.cfg file.   Or would you suggest to
store additional information elsewhere?  Then we don't need the (then
redundant) counter in  boards.cfg ...

> > Well, one reason is efficiency.  If the code builds fine, and does not
> > cause efforts druing any of the ongoing work, it is more efficient to
> > just keep it than to actively remove it, which would require active
> > work.  I. e. I want to reduce the work load on the maintainer(s) such
> > that they only have to become active if such action saves even more
> > effort.  Actually it may even seem more efficient in some cases to
> > perform minor fixing even of unmaintained boards than to remove them.
> 
> But in the long term I think, we have more work with old, unmaintained
> boards, then removing it. And if we remove boards actively, we maybe

YOu misunderstand.  My intention is to keep them as long as they do
NOT cause any efforts.  If they do, they are candidates for checking
for removal.

> force more the board maintainers (for examples corporates who have
> interest that the board stays in mainline) to really test them ;-)

Yes, but you organize more work for all of us.

> > But should we not also try to minimize efforts, especially on the
> > custodians?  If a board does not cause any trouble, we should not have
> > to invent additional efforts for it.
> 
> Hmm... I am not really sure, if it is more work for removing a
> board or fixing compiling issues ...

If we have to fix compilation issues, that means that the board _IS_ a
candidate for removal.  I suggested to keep them as long as they do
"not cause any trouble" !

> > I don't really understand this argument. The I2C code is either
> > hardware independent (say, the command line interface code), or it is
> > platform specific (say, the driver code for the I2C controller on a
> > specific SoC), or it is board dendent (say, some specific twiddeling
> > with I2C devices to perform some magic operations on a board).
> >
> > In the first two cases the work wil have to be done in any case
> > (except for the realy rare case that there are only old, unmaintained
> > boards left using this SoC).  An for the board specific code you can
>  >
> > check if you need to adapt it by checking the board's test status.  If
> > the board is unmaintained, then we enter the "it hurts" branch, and
> > drop the board.
> 
> Hmm... :
> 
> pollux:u-boot hs [master] $ grep -lr HARD_I2C arch/powerpc/
> [...]
> arch/powerpc/cpu/mpc8xx/i2c.c
> [...]
> arch/powerpc/cpu/mpc8260/i2c.c
> arch/powerpc/cpu/mpc5xxx/i2c.c
> arch/powerpc/cpu/mpc824x/drivers/i2c/i2c.c
> arch/powerpc/cpu/mpc512x/i2c.c
> pollux:u-boot hs [master] $

THi sis all architecure support.  None of these files are board
specific.  None of them would be removed if you crap old boards.

> Is it worth to move the drivers to drivers/i2c/ and convert them to
> the new framework? Or is it better to delete them and the boards,
> which use them ... Ok, if we had such a "livetimer" without deleting
> old boards, you are right, I could do now:

How do you know which boards use these drivers?

> - search the boards, which use the drivers
> - search the state of this board
> - decide, dropping or convert ...
>    On which criteria whould this decision be done? I have no idea
>    if this boards exist anymore ...

There is no conflict with what I wrote - if you can find that only
"old" boards are using these drivers, then both the boards and the
drivers are candidates for removalk.  But you need to find this
correlation first, and verify that there are no "good" boards left
that use trhem do.

All of this is totally different of the mechanism how to determine
what are "good" or "old" boards.

> If we have only good boards, I have only one way -> convert. No
> time wasted for the above steps ...

It does not really matter when you have to check if there are "good"
users of this code left.  You have to do this in both cases - either
when removing the boards automatically, or when touching that code.

With your solution, you will probably have to do it twice, as you
cannot be really sure that the guy who removed the board support
verified that he was the last user of some global architecture support
code.

> I dislike additional efforts too (But the question is, needs removing
> old boards more effort than fixing them all the time maybe uselessly?).

Can you please re-read what I wrote?  I say we keep them as long as
they do NOT need ANY fixing.  If they do, this is a toalluy new game.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
The following statement is not true.  The previous statement is true.

[U-Boot] livetime of boards

[Heiko Schocher]

Hello Wolfgang,

Am 08.11.2013 07:20, schrieb Wolfgang Denk:
> Dear Heiko,
>
> In message<527C767D.3070700@denx.de>  you wrote:
>>
>>> Yes, this is perfectly fine.  I just want to allow this at _any_ time,
>>> not only once per release (near the end of the release cycle).
>>> especially for releases where bigger changes get merged it may be
>>> precious information to know when the code stopped working.
>>
>> With "once", I only meant, we once update the "livetimer" and collect
>> all the time test reports!
>
> But with your suggestion, the only stoprage for such information is
> the life timer entry in the boards.cfg file.   Or would you suggest to
> store additional information elsewhere?  Then we don't need the (then
> redundant) counter in  boards.cfg ...

Yes, if we store more information then the "livetime", we need another
approach. But with storing this somewhere else, we must take care that
the number and names of the boards in the external database are in
sync with the boards in mainline.

>>> Well, one reason is efficiency.  If the code builds fine, and does not
>>> cause efforts druing any of the ongoing work, it is more efficient to
>>> just keep it than to actively remove it, which would require active
>>> work.  I. e. I want to reduce the work load on the maintainer(s) such
>>> that they only have to become active if such action saves even more
>>> effort.  Actually it may even seem more efficient in some cases to
>>> perform minor fixing even of unmaintained boards than to remove them.
>>
>> But in the long term I think, we have more work with old, unmaintained
>> boards, then removing it. And if we remove boards actively, we maybe
>
> YOu misunderstand.  My intention is to keep them as long as they do
> NOT cause any efforts.  If they do, they are candidates for checking
> for removal.

Ah, ok!

>> force more the board maintainers (for examples corporates who have
>> interest that the board stays in mainline) to really test them ;-)
>
> Yes, but you organize more work for all of us.

That would be bad.

>>> But should we not also try to minimize efforts, especially on the
>>> custodians?  If a board does not cause any trouble, we should not have
>>> to invent additional efforts for it.
>>
>> Hmm... I am not really sure, if it is more work for removing a
>> board or fixing compiling issues ...
>
> If we have to fix compilation issues, that means that the board _IS_ a
> candidate for removal.  I suggested to keep them as long as they do
> "not cause any trouble" !

Ok, got it, that sounds good. So, if we introduce new features we
have to check for boards which are affected:

- board is in maintained state -> convert to new feature
- board is in unmaintained state -> drop it ...

>>> I don't really understand this argument. The I2C code is either
>>> hardware independent (say, the command line interface code), or it is
>>> platform specific (say, the driver code for the I2C controller on a
>>> specific SoC), or it is board dendent (say, some specific twiddeling
>>> with I2C devices to perform some magic operations on a board).
>>>
>>> In the first two cases the work wil have to be done in any case
>>> (except for the realy rare case that there are only old, unmaintained
>>> boards left using this SoC).  An for the board specific code you can
>>   >
>>> check if you need to adapt it by checking the board's test status.  If
>>> the board is unmaintained, then we enter the "it hurts" branch, and
>>> drop the board.
>>
>> Hmm... :
>>
>> pollux:u-boot hs [master] $ grep -lr HARD_I2C arch/powerpc/
>> [...]
>> arch/powerpc/cpu/mpc8xx/i2c.c
>> [...]
>> arch/powerpc/cpu/mpc8260/i2c.c
>> arch/powerpc/cpu/mpc5xxx/i2c.c
>> arch/powerpc/cpu/mpc824x/drivers/i2c/i2c.c
>> arch/powerpc/cpu/mpc512x/i2c.c
>> pollux:u-boot hs [master] $
>
> THi sis all architecure support.  None of these files are board
> specific.  None of them would be removed if you crap old boards.

Hmm.. a U-Boot rule is, remove unnused code, or? If we remove all
boards, who use this drivers, we remove the drivers also ...

>> Is it worth to move the drivers to drivers/i2c/ and convert them to
>> the new framework? Or is it better to delete them and the boards,
>> which use them ... Ok, if we had such a "livetimer" without deleting
>> old boards, you are right, I could do now:
>
> How do you know which boards use these drivers?

I must find out ...

>> - search the boards, which use the drivers
>> - search the state of this board
>> - decide, dropping or convert ...
>>     On which criteria whould this decision be done? I have no idea
>>     if this boards exist anymore ...
>
> There is no conflict with what I wrote - if you can find that only
> "old" boards are using these drivers, then both the boards and the
> drivers are candidates for removalk.  But you need to find this
> correlation first, and verify that there are no "good" boards left
> that use trhem do.

Yes.

But there is a difference! With having old boards in mainline, I must
do the above step "search the state, and decide what to do". With
having only "good" boards in mainline, I do not need this steps.

But I aggree with you, if we have the rule:
"unmaintained board and compile error" -> remove it

This decision is easy and I am fine with it.

> All of this is totally different of the mechanism how to determine
> what are "good" or "old" boards.
>
>> If we have only good boards, I have only one way ->  convert. No
>> time wasted for the above steps ...
>
> It does not really matter when you have to check if there are "good"
> users of this code left.  You have to do this in both cases - either
> when removing the boards automatically, or when touching that code.
>
> With your solution, you will probably have to do it twice, as you
> cannot be really sure that the guy who removed the board support
> verified that he was the last user of some global architecture support
> code.

Yes for the driver code. As I said in this discusion, removing boards
is no trivial job.

>> I dislike additional efforts too (But the question is, needs removing
>> old boards more effort than fixing them all the time maybe uselessly?).
>
> Can you please re-read what I wrote?  I say we keep them as long as
> they do NOT need ANY fixing.  If they do, this is a toalluy new game.

I got it now ;-)

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] livetime of boards

[Wolfgang Denk]

Dear Heiko,

In message <527C7716.8060308@denx.de> you wrote:
> 
> > You, you can use a mailing list for submitting such information, but I
> > doubt that it would be efficient.  And I definitely do not want to see
> > this on the current U-Boot ML.
> 
> Ok, let us discuss the way we collect such information, if we decided
> that we want testreports ...

I think there is a general agreement that we do want such reports.

We just need to get a better understanding, what exactly we want, and
then how we can get this.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Time is an illusion perpetrated by the manufacturers of space.