[U-Boot] Removing the need for boards.cfg

[U-Boot] Removing the need for boards.cfg

[Simon Glass]

Hi,

At present, as a work-around, we generate boards.cfg if needed. This
is quite a slow process since each board config must be fully
processed.

What can we do to improve this? We only need a small number of options
in order to start buildman - things like CONFIG_SYS_ARCH,
CONFIG_SYS_CPU, etc.

I wonder if we could run a script which adds these to the defconfigs
for each board and then apply a patch to mainline? Would that require
removing the options from the config.h files? Or could we do that
later as a separate step?

Regards,
Simon

[U-Boot] Removing the need for boards.cfg

[Masahiro Yamada]

Hi.



On Fri, 1 Aug 2014 12:48:44 +0100
Simon Glass <sjg@chromium.org> wrote:

> Hi,
> 
> At present, as a work-around, we generate boards.cfg if needed. This
> is quite a slow process since each board config must be fully
> processed.
> 
> What can we do to improve this? We only need a small number of options
> in order to start buildman - things like CONFIG_SYS_ARCH,
> CONFIG_SYS_CPU, etc.
> 
> I wonder if we could run a script which adds these to the defconfigs
> for each board and then apply a patch to mainline? Would that require
> removing the options from the config.h files? Or could we do that
> later as a separate step?
> 


We cannot add CONFIG_SYS_ARCH, CONFIG_SYS_CPU, etc. to defconfigs
because they are not user-editable options.
(They must be always selected in correct combination.)

This is already discussed when I posted the RFC version of the Kconfig series.


We can easily do

[1] Choose boards by arch

  Just grep  CONFIG_ARM=y, CONFIG_MIPS=y, CONFIG_SANDBOX=y  in defconfigs

[2] Choose boards by board name

  It is the file name of defconfigs




My question is, do we really need to choose by CPU, Vendor, SoC ?
I know they are useful.
For ex. nvidia developers probably want to build only Tegra boards.



One possible idea I come up with is to have 'board group' in .buildman file
so people can build only boards they are interested in.


Something like this:

---------->8---------------

[board-alisa]
tegra20:  harmony seaboard ventana
tegra30:  beaver
tegra114: dalmore
tegra124: venice2
tegra:    tegra20 tegra30 tegra114 tegra124

-----------8<---------------


With the settings above,


"tools/buildman/buildman  tegra20"
is the same as
"tools/buildman/buildman   harmony seaboard ventana"


"tools/buildman/buildman   tegra"
is the same as
"tools/buildman/buildman    tegra20 tegra30 tegra114 tegra124"
and also the same as
"tools/buildman/buildman    harmony seaboard ventana beaver dalmore venice2"



We have to setup  ~/.buildman@the first run.
(and modify it when adding new boards)

Better to provide a tool to auto-generate .buildman file?



Best Regards
Masahiro Yamada

[U-Boot] Removing the need for boards.cfg

[Tom Rini]

On Fri, Aug 01, 2014 at 09:22:42PM +0900, Masahiro Yamada wrote:
> Hi.
> 
> 
> 
> On Fri, 1 Aug 2014 12:48:44 +0100
> Simon Glass <sjg@chromium.org> wrote:
> 
> > Hi,
> > 
> > At present, as a work-around, we generate boards.cfg if needed. This
> > is quite a slow process since each board config must be fully
> > processed.
> > 
> > What can we do to improve this? We only need a small number of options
> > in order to start buildman - things like CONFIG_SYS_ARCH,
> > CONFIG_SYS_CPU, etc.
> > 
> > I wonder if we could run a script which adds these to the defconfigs
> > for each board and then apply a patch to mainline? Would that require
> > removing the options from the config.h files? Or could we do that
> > later as a separate step?
> > 
> 
> 
> We cannot add CONFIG_SYS_ARCH, CONFIG_SYS_CPU, etc. to defconfigs
> because they are not user-editable options.
> (They must be always selected in correct combination.)
> 
> This is already discussed when I posted the RFC version of the Kconfig series.
> 
> 
> We can easily do
> 
> [1] Choose boards by arch
> 
>   Just grep  CONFIG_ARM=y, CONFIG_MIPS=y, CONFIG_SANDBOX=y  in defconfigs
> 
> [2] Choose boards by board name
> 
>   It is the file name of defconfigs
> 
> 
> 
> 
> My question is, do we really need to choose by CPU, Vendor, SoC ?
> I know they are useful.
> For ex. nvidia developers probably want to build only Tegra boards.

For me, I don't see CPU or Vendor as super important but SoC is.  I'll
often be doing some TI work for say am335x or am43xx and be checking
over those two board configs but once I'm done (or nearing done) I need
to build for SoC==am33xx to catch all of the other vendors that have
boards in the family.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140801/879f1d4a/attachment.pgp>

[U-Boot] Removing the need for boards.cfg

[Masahiro Yamada]

Simon, Tom,



On Fri, 1 Aug 2014 09:38:03 -0400
Tom Rini <trini@ti.com> wrote:

> On Fri, Aug 01, 2014 at 09:22:42PM +0900, Masahiro Yamada wrote:
> > Hi.
> > 
> > 
> > 
> > On Fri, 1 Aug 2014 12:48:44 +0100
> > Simon Glass <sjg@chromium.org> wrote:
> > 
> > > Hi,
> > > 
> > > At present, as a work-around, we generate boards.cfg if needed. This
> > > is quite a slow process since each board config must be fully
> > > processed.
> > > 
> > > What can we do to improve this? We only need a small number of options
> > > in order to start buildman - things like CONFIG_SYS_ARCH,
> > > CONFIG_SYS_CPU, etc.
> > > 
> > > I wonder if we could run a script which adds these to the defconfigs
> > > for each board and then apply a patch to mainline? Would that require
> > > removing the options from the config.h files? Or could we do that
> > > later as a separate step?
> > > 
> > 
> > 
> > We cannot add CONFIG_SYS_ARCH, CONFIG_SYS_CPU, etc. to defconfigs
> > because they are not user-editable options.
> > (They must be always selected in correct combination.)
> > 
> > This is already discussed when I posted the RFC version of the Kconfig series.
> > 
> > 
> > We can easily do
> > 
> > [1] Choose boards by arch
> > 
> >   Just grep  CONFIG_ARM=y, CONFIG_MIPS=y, CONFIG_SANDBOX=y  in defconfigs
> > 
> > [2] Choose boards by board name
> > 
> >   It is the file name of defconfigs
> > 
> > 
> > 
> > 
> > My question is, do we really need to choose by CPU, Vendor, SoC ?
> > I know they are useful.
> > For ex. nvidia developers probably want to build only Tegra boards.
> 
> For me, I don't see CPU or Vendor as super important but SoC is.  I'll
> often be doing some TI work for say am335x or am43xx and be checking
> over those two board configs but once I'm done (or nearing done) I need
> to build for SoC==am33xx to catch all of the other vendors that have
> boards in the family.
> 


If MAKEALL/buildman is invoked without any arguments
(which means build all the board),
boards.cfg is not necessary in the first place.

It that case, can we skip generating boards.cfg ?
(We can get the board list by 'ls configs' )


We need a database only when we want to select boards by CPU, Vendor, SoC.


Best Regards
Masahiro Yamada

[U-Boot] Removing the need for boards.cfg

[Wolfgang Denk]

Dear Masahiro,

In message <20140801225339.D541.AA925319@jp.panasonic.com> you wrote:
> 
> It that case, can we skip generating boards.cfg ?
> (We can get the board list by 'ls configs' )

This would neither give all build target names no=r do the needed
translation of names (like upper case -> lower case translations),
etc.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Microsoft Compatibility:
     your old Windows 3.11 application crash exactly as the new ones.

[U-Boot] Removing the need for boards.cfg

[York Sun]

On 08/01/2014 06:53 AM, Masahiro Yamada wrote:
> 
> If MAKEALL/buildman is invoked without any arguments
> (which means build all the board),
> boards.cfg is not necessary in the first place.
> 
> It that case, can we skip generating boards.cfg ?
> (We can get the board list by 'ls configs' )
> 
> 
> We need a database only when we want to select boards by CPU, Vendor, SoC.
> 

Dear Masahiro,

What is the best way to replace "./MAKEALL -a powerpc" or "./MAKEALL -a arm"?
You must have verified compiling for all targets. I am wondering if you can
share your setup. I still heavily rely on MAKEALL to build all targets I care.

York

[U-Boot] Removing the need for boards.cfg

[Simon Glass]

Hi,

On 1 August 2014 12:31, Wolfgang Denk <wd@denx.de> wrote:
> Dear Masahiro,
>
> In message <20140801225339.D541.AA925319@jp.panasonic.com> you wrote:
>>
>> It that case, can we skip generating boards.cfg ?
>> (We can get the board list by 'ls configs' )
>
> This would neither give all build target names no=r do the needed
> translation of names (like upper case -> lower case translations),
> etc.

Skipping in an optimised case doesn't buy us much. I normally want to
build 'sandbox' at least before I go ahead and build everything.

I suppose what I'm asking is whether there is a fast way for buildman
to detect all these fields without having to run 'make xxx_defconfig'
on every board. For example, a simple read and scan of a few config
files, or worse case a C preprocessor run?

Regards,
Simon

[U-Boot] Removing the need for boards.cfg

[Masahiro Yamada]

Hi York,



On Mon, 4 Aug 2014 11:58:56 -0700
York Sun <yorksun@freescale.com> wrote:

> On 08/01/2014 06:53 AM, Masahiro Yamada wrote:
> > 
> > If MAKEALL/buildman is invoked without any arguments
> > (which means build all the board),
> > boards.cfg is not necessary in the first place.
> > 
> > It that case, can we skip generating boards.cfg ?
> > (We can get the board list by 'ls configs' )
> > 
> > 
> > We need a database only when we want to select boards by CPU, Vendor, SoC.
> > 
> 
> Dear Masahiro,
> 
> What is the best way to replace "./MAKEALL -a powerpc" or "./MAKEALL -a arm"?
> You must have verified compiling for all targets. I am wondering if you can
> share your setup. I still heavily rely on MAKEALL to build all targets I care.
> 
> York


I think "./MAKEALL -a powerpc" or "./MAKEALL -a arm"
is still working as it was.

Isn't it on your machine?


Best Regards
Masahiro Yamada

[U-Boot] Removing the need for boards.cfg

[York Sun]

It is. But it just takes long to generate boards.cfg. I am wondering if you have a better way.

York

________________________________
From: Masahiro Yamada
Sent: Mon, 04/08/2014 18:22
To: Sun York-R58495 <yorksun@freescale.com>
CC: Tom Rini <trini@ti.com>; Simon Glass <sjg@chromium.org>; U-Boot Mailing List <u-boot@lists.denx.de>
Subject: Re: [U-Boot] Removing the need for boards.cfg


Hi York,



On Mon, 4 Aug 2014 11:58:56 -0700
York Sun <yorksun@freescale.com> wrote:

> On 08/01/2014 06:53 AM, Masahiro Yamada wrote:
> >
> > If MAKEALL/buildman is invoked without any arguments
> > (which means build all the board),
> > boards.cfg is not necessary in the first place.
> >
> > It that case, can we skip generating boards.cfg ?
> > (We can get the board list by 'ls configs' )
> >
> >
> > We need a database only when we want to select boards by CPU, Vendor, SoC.
> >
>
> Dear Masahiro,
>
> What is the best way to replace "./MAKEALL -a powerpc" or "./MAKEALL -a arm"?
> You must have verified compiling for all targets. I am wondering if you can
> share your setup. I still heavily rely on MAKEALL to build all targets I care.
>
> York


I think "./MAKEALL -a powerpc" or "./MAKEALL -a arm"
is still working as it was.

Isn't it on your machine?


Best Regards
Masahiro Yamada

[U-Boot] Removing the need for boards.cfg

[York Sun]

On 08/04/2014 06:22 PM, Masahiro Yamada wrote:
> Hi York,
> 
> 
> 
> On Mon, 4 Aug 2014 11:58:56 -0700
> York Sun <yorksun@freescale.com> wrote:
> 
>> On 08/01/2014 06:53 AM, Masahiro Yamada wrote:
>>>
>>> If MAKEALL/buildman is invoked without any arguments
>>> (which means build all the board),
>>> boards.cfg is not necessary in the first place.
>>>
>>> It that case, can we skip generating boards.cfg ?
>>> (We can get the board list by 'ls configs' )
>>>
>>>
>>> We need a database only when we want to select boards by CPU, Vendor, SoC.
>>>
>>
>> Dear Masahiro,
>>
>> What is the best way to replace "./MAKEALL -a powerpc" or "./MAKEALL -a arm"?
>> You must have verified compiling for all targets. I am wondering if you can
>> share your setup. I still heavily rely on MAKEALL to build all targets I care.
>>
>> York
> 
> 
> I think "./MAKEALL -a powerpc" or "./MAKEALL -a arm"
> is still working as it was.

Dear Masahiro,

Generating boards.cfg gives me another trouble. I use Jenkins
(http://jenkins-ci.org/) for internal development. Even I can manually run
MAKEALL to generate boards.cfg, the same script under Jenkins cannot. I have
this error


./MAKEALL -c mpc83xx -c mpc85xx -c mpc86xx
Could not find boards.cfg
Generating boards.cfg ...  (jobs: 8)
Traceback (most recent call last):
  File "tools/genboardscfg.py", line 504, in <module>
    main()
  File "tools/genboardscfg.py", line 501, in main
    gen_boards_cfg(jobs)
  File "tools/genboardscfg.py", line 471, in gen_boards_cfg
    __gen_boards_cfg(jobs)
  File "tools/genboardscfg.py", line 437, in __gen_boards_cfg
    indicator = Indicator(len(defconfigs))
  File "tools/genboardscfg.py", line 384, in __init__
    width = get_terminal_columns()
  File "tools/genboardscfg.py", line 59, in get_terminal_columns
    ret = fcntl.ioctl(sys.stdout.fileno(), termios.TIOCGWINSZ, arg)
IOError: [Errno 22] Invalid argument
Failed to generate boards.cfg

Looks like the script is trying to get the terminal window size. There is no
terminal for my case. Checking variable TERM, it is "dumb". I can make it work
by removing the two lines but I don't think that's the solution.

--- a/tools/genboardscfg.py
+++ b/tools/genboardscfg.py
@@ -58,8 +58,6 @@ def get_terminal_columns():
         try:
             ret = fcntl.ioctl(sys.stdout.fileno(), termios.TIOCGWINSZ, arg)
         except IOError as exception:
-            if exception.errno != errno.ENOTTY:
-                raise

I am not an expert on python. Can you take a deep look? The host has ubuntu
12.04.3 with python 2.7.3.

York

[U-Boot] Removing the need for boards.cfg

[Jeroen Hofstee]

Hello York,

On 07-08-14 22:52, York Sun wrote:
> On 08/04/2014 06:22 PM, Masahiro Yamada wrote:
>> Hi York,
>>
>>
>>
>> On Mon, 4 Aug 2014 11:58:56 -0700
>> York Sun <yorksun@freescale.com> wrote:
>>
>>> On 08/01/2014 06:53 AM, Masahiro Yamada wrote:
>>>> If MAKEALL/buildman is invoked without any arguments
>>>> (which means build all the board),
>>>> boards.cfg is not necessary in the first place.
>>>>
>>>> It that case, can we skip generating boards.cfg ?
>>>> (We can get the board list by 'ls configs' )
>>>>
>>>>
>>>> We need a database only when we want to select boards by CPU, Vendor, SoC.
>>>>
>>> Dear Masahiro,
>>>
>>> What is the best way to replace "./MAKEALL -a powerpc" or "./MAKEALL -a arm"?
>>> You must have verified compiling for all targets. I am wondering if you can
>>> share your setup. I still heavily rely on MAKEALL to build all targets I care.
>>>
>>> York
>>
>> I think "./MAKEALL -a powerpc" or "./MAKEALL -a arm"
>> is still working as it was.
> Dear Masahiro,
>
> Generating boards.cfg gives me another trouble. I use Jenkins
> (http://jenkins-ci.org/) for internal development. Even I can manually run
> MAKEALL to generate boards.cfg, the same script under Jenkins cannot. I have
> this error
>
>
> ./MAKEALL -c mpc83xx -c mpc85xx -c mpc86xx
> Could not find boards.cfg
> Generating boards.cfg ...  (jobs: 8)
> Traceback (most recent call last):
>    File "tools/genboardscfg.py", line 504, in <module>
>      main()
>    File "tools/genboardscfg.py", line 501, in main
>      gen_boards_cfg(jobs)
>    File "tools/genboardscfg.py", line 471, in gen_boards_cfg
>      __gen_boards_cfg(jobs)
>    File "tools/genboardscfg.py", line 437, in __gen_boards_cfg
>      indicator = Indicator(len(defconfigs))
>    File "tools/genboardscfg.py", line 384, in __init__
>      width = get_terminal_columns()
>    File "tools/genboardscfg.py", line 59, in get_terminal_columns
>      ret = fcntl.ioctl(sys.stdout.fileno(), termios.TIOCGWINSZ, arg)
> IOError: [Errno 22] Invalid argument
> Failed to generate boards.cfg
>
> Looks like the script is trying to get the terminal window size. There is no
> terminal for my case. Checking variable TERM, it is "dumb". I can make it work
> by removing the two lines but I don't think that's the solution.
>
> --- a/tools/genboardscfg.py
> +++ b/tools/genboardscfg.py
> @@ -58,8 +58,6 @@ def get_terminal_columns():
>           try:
>               ret = fcntl.ioctl(sys.stdout.fileno(), termios.TIOCGWINSZ, arg)
>           except IOError as exception:
> -            if exception.errno != errno.ENOTTY:
> -                raise
>
> I am not an expert on python. Can you take a deep look? The host has ubuntu
> 12.04.3 with python 2.7.3.
>

I can't parse the python either, but is this not the same issue?
(just from the looks of it) https://patchwork.ozlabs.org/patch/377897/

Regards,
Jeroen

[U-Boot] Removing the need for boards.cfg

[York Sun]

On 08/07/2014 01:57 PM, Jeroen Hofstee wrote:
> I can't parse the python either, but is this not the same issue?
> (just from the looks of it) https://patchwork.ozlabs.org/patch/377897/
> 

It is the same solution/workaround, but I'm not sure if Roger has the same issue.

York

[U-Boot] Removing the need for boards.cfg

[Simon Glass]

Hi Masahiro,

On 1 August 2014 06:22, Masahiro Yamada <yamada.m@jp.panasonic.com> wrote:
> Hi.
>
>
>
> On Fri, 1 Aug 2014 12:48:44 +0100
> Simon Glass <sjg@chromium.org> wrote:
>
>> Hi,
>>
>> At present, as a work-around, we generate boards.cfg if needed. This
>> is quite a slow process since each board config must be fully
>> processed.
>>
>> What can we do to improve this? We only need a small number of options
>> in order to start buildman - things like CONFIG_SYS_ARCH,
>> CONFIG_SYS_CPU, etc.
>>
>> I wonder if we could run a script which adds these to the defconfigs
>> for each board and then apply a patch to mainline? Would that require
>> removing the options from the config.h files? Or could we do that
>> later as a separate step?
>>
>
>
> We cannot add CONFIG_SYS_ARCH, CONFIG_SYS_CPU, etc. to defconfigs
> because they are not user-editable options.
> (They must be always selected in correct combination.)
>
> This is already discussed when I posted the RFC version of the Kconfig series.

My understanding is that if you have a 'select' option in Kconfig then
it will be selected always, but it will still appear in the defconfig.
I think that is the way the kernel works. So there are options that
cannot be changed, and don't appear in the menu, but they are selected
just the same.

>
>
> We can easily do
>
> [1] Choose boards by arch
>
>   Just grep  CONFIG_ARM=y, CONFIG_MIPS=y, CONFIG_SANDBOX=y  in defconfigs

OK.

>
> [2] Choose boards by board name
>
>   It is the file name of defconfigs

Almost, as Wolfgang points out.

>
>
>
>
> My question is, do we really need to choose by CPU, Vendor, SoC ?
> I know they are useful.
> For ex. nvidia developers probably want to build only Tegra boards.
>
>
>
> One possible idea I come up with is to have 'board group' in .buildman file
> so people can build only boards they are interested in.
>
>
> Something like this:
>
> ---------->8---------------
>
> [board-alisa]
> tegra20:  harmony seaboard ventana
> tegra30:  beaver
> tegra114: dalmore
> tegra124: venice2
> tegra:    tegra20 tegra30 tegra114 tegra124
>
> -----------8<---------------
>
>
> With the settings above,
>
>
> "tools/buildman/buildman  tegra20"
> is the same as
> "tools/buildman/buildman   harmony seaboard ventana"
>
>
> "tools/buildman/buildman   tegra"
> is the same as
> "tools/buildman/buildman    tegra20 tegra30 tegra114 tegra124"
> and also the same as
> "tools/buildman/buildman    harmony seaboard ventana beaver dalmore venice2"
>
>
>
> We have to setup  ~/.buildman at the first run.
> (and modify it when adding new boards)
>
> Better to provide a tool to auto-generate .buildman file?

That doesn't sound very different to having a boards.cfg file again.
If we really need it, it seems better to me to have an auto-generated
boards.cfg which is checked into the source tree, at least for now.
When people add boards they can re-run the generation script.

But I'd like to avoid that if possible, so will await your thoughts on
the above Kconfig question.

Regards,
Simon

[U-Boot] Removing the need for boards.cfg

[Tom Rini]

On Fri, Aug 08, 2014 at 05:06:03AM -0600, Simon Glass wrote:
> Hi Masahiro,
> 
> On 1 August 2014 06:22, Masahiro Yamada <yamada.m@jp.panasonic.com> wrote:
> > Hi.
> >
> >
> >
> > On Fri, 1 Aug 2014 12:48:44 +0100
> > Simon Glass <sjg@chromium.org> wrote:
> >
> >> Hi,
> >>
> >> At present, as a work-around, we generate boards.cfg if needed. This
> >> is quite a slow process since each board config must be fully
> >> processed.
> >>
> >> What can we do to improve this? We only need a small number of options
> >> in order to start buildman - things like CONFIG_SYS_ARCH,
> >> CONFIG_SYS_CPU, etc.
> >>
> >> I wonder if we could run a script which adds these to the defconfigs
> >> for each board and then apply a patch to mainline? Would that require
> >> removing the options from the config.h files? Or could we do that
> >> later as a separate step?
> >>
> >
> >
> > We cannot add CONFIG_SYS_ARCH, CONFIG_SYS_CPU, etc. to defconfigs
> > because they are not user-editable options.
> > (They must be always selected in correct combination.)
> >
> > This is already discussed when I posted the RFC version of the Kconfig series.
> 
> My understanding is that if you have a 'select' option in Kconfig then
> it will be selected always, but it will still appear in the defconfig.
> I think that is the way the kernel works. So there are options that
> cannot be changed, and don't appear in the menu, but they are selected
> just the same.

Maybe we came up with the wrong answer during RFC on Kconfig then.  It's
good if selecting say CONFIG_TARGET_AM335X_EVM results in
CONFIG_SYS_ARCH="arm", CONFIG_SYS_CPU="armv7", etc.

I _think_ the problem here is what I ran into with doing CONS_INDEX
patches.  You cannot do 'select SYS_CPU="armv7"'.  So we would instead
need to get rid of CONFIG_SYS_CPU and instead do CONFIG_SYS_CPU_ARMV7,
etc, etc.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140808/30a66f7e/attachment.pgp>

[U-Boot] Removing the need for boards.cfg

[Tom Rini]

On Fri, Aug 01, 2014 at 12:48:44PM +0100, Simon Glass wrote:

> Hi,
> 
> At present, as a work-around, we generate boards.cfg if needed. This
> is quite a slow process since each board config must be fully
> processed.
> 
> What can we do to improve this? We only need a small number of options
> in order to start buildman - things like CONFIG_SYS_ARCH,
> CONFIG_SYS_CPU, etc.
> 
> I wonder if we could run a script which adds these to the defconfigs
> for each board and then apply a patch to mainline? Would that require
> removing the options from the config.h files? Or could we do that
> later as a separate step?

I wonder (and I haven't had my coffee nor dug into this much more yet)
about using say https://github.com/ulfalizer/Kconfiglib to have buildman
parse all configs/ and then use that information to build what's been
asked (depending on how long the reading+parsing takes).  If it's
relatively quick, that would be a big argument in favour of dropping
MAKEALL.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140808/0aecf168/attachment.pgp>

[U-Boot] Removing the need for boards.cfg

[Simon Glass]

Hi Tom,

On 8 August 2014 05:59, Tom Rini <trini@ti.com> wrote:
> On Fri, Aug 08, 2014 at 05:06:03AM -0600, Simon Glass wrote:
>> Hi Masahiro,
>>
>> On 1 August 2014 06:22, Masahiro Yamada <yamada.m@jp.panasonic.com> wrote:
>> > Hi.
>> >
>> >
>> >
>> > On Fri, 1 Aug 2014 12:48:44 +0100
>> > Simon Glass <sjg@chromium.org> wrote:
>> >
>> >> Hi,
>> >>
>> >> At present, as a work-around, we generate boards.cfg if needed. This
>> >> is quite a slow process since each board config must be fully
>> >> processed.
>> >>
>> >> What can we do to improve this? We only need a small number of options
>> >> in order to start buildman - things like CONFIG_SYS_ARCH,
>> >> CONFIG_SYS_CPU, etc.
>> >>
>> >> I wonder if we could run a script which adds these to the defconfigs
>> >> for each board and then apply a patch to mainline? Would that require
>> >> removing the options from the config.h files? Or could we do that
>> >> later as a separate step?
>> >>
>> >
>> >
>> > We cannot add CONFIG_SYS_ARCH, CONFIG_SYS_CPU, etc. to defconfigs
>> > because they are not user-editable options.
>> > (They must be always selected in correct combination.)
>> >
>> > This is already discussed when I posted the RFC version of the Kconfig series.
>>
>> My understanding is that if you have a 'select' option in Kconfig then
>> it will be selected always, but it will still appear in the defconfig.
>> I think that is the way the kernel works. So there are options that
>> cannot be changed, and don't appear in the menu, but they are selected
>> just the same.
>
> Maybe we came up with the wrong answer during RFC on Kconfig then.  It's
> good if selecting say CONFIG_TARGET_AM335X_EVM results in
> CONFIG_SYS_ARCH="arm", CONFIG_SYS_CPU="armv7", etc.
>
> I _think_ the problem here is what I ran into with doing CONS_INDEX
> patches.  You cannot do 'select SYS_CPU="armv7"'.  So we would instead
> need to get rid of CONFIG_SYS_CPU and instead do CONFIG_SYS_CPU_ARMV7,
> etc, etc.
>
> --
> Tom

[U-Boot] Removing the need for boards.cfg

[Simon Glass]

Hi Tom,

On 8 August 2014 05:59, Tom Rini <trini@ti.com> wrote:
> On Fri, Aug 08, 2014 at 05:06:03AM -0600, Simon Glass wrote:
>> Hi Masahiro,
>>
>> On 1 August 2014 06:22, Masahiro Yamada <yamada.m@jp.panasonic.com> wrote:
>> > Hi.
>> >
>> >
>> >
>> > On Fri, 1 Aug 2014 12:48:44 +0100
>> > Simon Glass <sjg@chromium.org> wrote:
>> >
>> >> Hi,
>> >>
>> >> At present, as a work-around, we generate boards.cfg if needed. This
>> >> is quite a slow process since each board config must be fully
>> >> processed.
>> >>
>> >> What can we do to improve this? We only need a small number of options
>> >> in order to start buildman - things like CONFIG_SYS_ARCH,
>> >> CONFIG_SYS_CPU, etc.
>> >>
>> >> I wonder if we could run a script which adds these to the defconfigs
>> >> for each board and then apply a patch to mainline? Would that require
>> >> removing the options from the config.h files? Or could we do that
>> >> later as a separate step?
>> >>
>> >
>> >
>> > We cannot add CONFIG_SYS_ARCH, CONFIG_SYS_CPU, etc. to defconfigs
>> > because they are not user-editable options.
>> > (They must be always selected in correct combination.)
>> >
>> > This is already discussed when I posted the RFC version of the Kconfig series.
>>
>> My understanding is that if you have a 'select' option in Kconfig then
>> it will be selected always, but it will still appear in the defconfig.
>> I think that is the way the kernel works. So there are options that
>> cannot be changed, and don't appear in the menu, but they are selected
>> just the same.
>
> Maybe we came up with the wrong answer during RFC on Kconfig then.  It's
> good if selecting say CONFIG_TARGET_AM335X_EVM results in
> CONFIG_SYS_ARCH="arm", CONFIG_SYS_CPU="armv7", etc.
>
> I _think_ the problem here is what I ran into with doing CONS_INDEX
> patches.  You cannot do 'select SYS_CPU="armv7"'.  So we would instead
> need to get rid of CONFIG_SYS_CPU and instead do CONFIG_SYS_CPU_ARMV7,
> etc, etc.

Well I think we should run this one to ground first - if a 'select'
item doesn't appear in Kconfig then I think we have a bit of a
problem. Will await Masahiro's thoughts.

Regards,
Simon

[U-Boot] Removing the need for boards.cfg

[Simon Glass]

Hi Tom,

On 8 August 2014 07:04, Tom Rini <trini@ti.com> wrote:
> On Fri, Aug 01, 2014 at 12:48:44PM +0100, Simon Glass wrote:
>
>> Hi,
>>
>> At present, as a work-around, we generate boards.cfg if needed. This
>> is quite a slow process since each board config must be fully
>> processed.
>>
>> What can we do to improve this? We only need a small number of options
>> in order to start buildman - things like CONFIG_SYS_ARCH,
>> CONFIG_SYS_CPU, etc.
>>
>> I wonder if we could run a script which adds these to the defconfigs
>> for each board and then apply a patch to mainline? Would that require
>> removing the options from the config.h files? Or could we do that
>> later as a separate step?
>
> I wonder (and I haven't had my coffee nor dug into this much more yet)
> about using say https://github.com/ulfalizer/Kconfiglib to have buildman
> parse all configs/ and then use that information to build what's been
> asked (depending on how long the reading+parsing takes).  If it's
> relatively quick, that would be a big argument in favour of dropping
> MAKEALL.

Well I've had my wine but it seems to parse the Kconfig files rather
than the #includes that are part of U-Boot's system. Unless I'm
missing something, we would still need to find out things like the CPU
type, even with that tool.

Regards,
Simon

[U-Boot] Removing the need for boards.cfg

[Tom Rini]

On Sun, Aug 10, 2014 at 08:16:43PM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On 8 August 2014 05:59, Tom Rini <trini@ti.com> wrote:
> > On Fri, Aug 08, 2014 at 05:06:03AM -0600, Simon Glass wrote:
> >> Hi Masahiro,
> >>
> >> On 1 August 2014 06:22, Masahiro Yamada <yamada.m@jp.panasonic.com> wrote:
> >> > Hi.
> >> >
> >> >
> >> >
> >> > On Fri, 1 Aug 2014 12:48:44 +0100
> >> > Simon Glass <sjg@chromium.org> wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> At present, as a work-around, we generate boards.cfg if needed. This
> >> >> is quite a slow process since each board config must be fully
> >> >> processed.
> >> >>
> >> >> What can we do to improve this? We only need a small number of options
> >> >> in order to start buildman - things like CONFIG_SYS_ARCH,
> >> >> CONFIG_SYS_CPU, etc.
> >> >>
> >> >> I wonder if we could run a script which adds these to the defconfigs
> >> >> for each board and then apply a patch to mainline? Would that require
> >> >> removing the options from the config.h files? Or could we do that
> >> >> later as a separate step?
> >> >>
> >> >
> >> >
> >> > We cannot add CONFIG_SYS_ARCH, CONFIG_SYS_CPU, etc. to defconfigs
> >> > because they are not user-editable options.
> >> > (They must be always selected in correct combination.)
> >> >
> >> > This is already discussed when I posted the RFC version of the Kconfig series.
> >>
> >> My understanding is that if you have a 'select' option in Kconfig then
> >> it will be selected always, but it will still appear in the defconfig.
> >> I think that is the way the kernel works. So there are options that
> >> cannot be changed, and don't appear in the menu, but they are selected
> >> just the same.
> >
> > Maybe we came up with the wrong answer during RFC on Kconfig then.  It's
> > good if selecting say CONFIG_TARGET_AM335X_EVM results in
> > CONFIG_SYS_ARCH="arm", CONFIG_SYS_CPU="armv7", etc.
> >
> > I _think_ the problem here is what I ran into with doing CONS_INDEX
> > patches.  You cannot do 'select SYS_CPU="armv7"'.  So we would instead
> > need to get rid of CONFIG_SYS_CPU and instead do CONFIG_SYS_CPU_ARMV7,
> > etc, etc.
> 
> Well I think we should run this one to ground first - if a 'select'
> item doesn't appear in Kconfig then I think we have a bit of a
> problem. Will await Masahiro's thoughts.

It's not that we can't select, it's:
board/ti/am335x/Kconfig:27:warning: 'CONS_INDEX' has wrong type.
'select' only accept arguments of boolean and tristate type

So we'd have to modify Kconfig to accept int/hex/string as well so we
could do:
config SYS_CPU
	default "armv7" if SOCA || SOCB || ...
	default "ppc4xx' if SOCC || SOCD || ..

And that also seems like a conflict-generating nightmare down the line
too.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140811/abf152a1/attachment.pgp>

[U-Boot] Removing the need for boards.cfg

[Tom Rini]

On Sun, Aug 10, 2014 at 08:22:35PM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On 8 August 2014 07:04, Tom Rini <trini@ti.com> wrote:
> > On Fri, Aug 01, 2014 at 12:48:44PM +0100, Simon Glass wrote:
> >
> >> Hi,
> >>
> >> At present, as a work-around, we generate boards.cfg if needed. This
> >> is quite a slow process since each board config must be fully
> >> processed.
> >>
> >> What can we do to improve this? We only need a small number of options
> >> in order to start buildman - things like CONFIG_SYS_ARCH,
> >> CONFIG_SYS_CPU, etc.
> >>
> >> I wonder if we could run a script which adds these to the defconfigs
> >> for each board and then apply a patch to mainline? Would that require
> >> removing the options from the config.h files? Or could we do that
> >> later as a separate step?
> >
> > I wonder (and I haven't had my coffee nor dug into this much more yet)
> > about using say https://github.com/ulfalizer/Kconfiglib to have buildman
> > parse all configs/ and then use that information to build what's been
> > asked (depending on how long the reading+parsing takes).  If it's
> > relatively quick, that would be a big argument in favour of dropping
> > MAKEALL.
> 
> Well I've had my wine but it seems to parse the Kconfig files rather
> than the #includes that are part of U-Boot's system. Unless I'm
> missing something, we would still need to find out things like the CPU
> type, even with that tool.

Well, my thought was:
foreach Kconfig file, read to build state machine
foreach defconfig, evaluate config, spl/config, tpl/config, store object

Now we can poke those objects for everything with CONFIG_SYS_CPU
whatever, and so forth.  Or maybe that all takes longer than what we do
today.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140811/7feb2617/attachment.pgp>

[U-Boot] Removing the need for boards.cfg

[Simon Glass]

+Masahiro

Hi Tom,

On 11 August 2014 07:37, Tom Rini <trini@ti.com> wrote:
> On Sun, Aug 10, 2014 at 08:16:43PM -0600, Simon Glass wrote:
>> Hi Tom,
>>
>> On 8 August 2014 05:59, Tom Rini <trini@ti.com> wrote:
>> > On Fri, Aug 08, 2014 at 05:06:03AM -0600, Simon Glass wrote:
>> >> Hi Masahiro,
>> >>
>> >> On 1 August 2014 06:22, Masahiro Yamada <yamada.m@jp.panasonic.com> wrote:
>> >> > Hi.
>> >> >
>> >> >
>> >> >
>> >> > On Fri, 1 Aug 2014 12:48:44 +0100
>> >> > Simon Glass <sjg@chromium.org> wrote:
>> >> >
>> >> >> Hi,
>> >> >>
>> >> >> At present, as a work-around, we generate boards.cfg if needed. This
>> >> >> is quite a slow process since each board config must be fully
>> >> >> processed.
>> >> >>
>> >> >> What can we do to improve this? We only need a small number of options
>> >> >> in order to start buildman - things like CONFIG_SYS_ARCH,
>> >> >> CONFIG_SYS_CPU, etc.
>> >> >>
>> >> >> I wonder if we could run a script which adds these to the defconfigs
>> >> >> for each board and then apply a patch to mainline? Would that require
>> >> >> removing the options from the config.h files? Or could we do that
>> >> >> later as a separate step?
>> >> >>
>> >> >
>> >> >
>> >> > We cannot add CONFIG_SYS_ARCH, CONFIG_SYS_CPU, etc. to defconfigs
>> >> > because they are not user-editable options.
>> >> > (They must be always selected in correct combination.)
>> >> >
>> >> > This is already discussed when I posted the RFC version of the Kconfig series.
>> >>
>> >> My understanding is that if you have a 'select' option in Kconfig then
>> >> it will be selected always, but it will still appear in the defconfig.
>> >> I think that is the way the kernel works. So there are options that
>> >> cannot be changed, and don't appear in the menu, but they are selected
>> >> just the same.
>> >
>> > Maybe we came up with the wrong answer during RFC on Kconfig then.  It's
>> > good if selecting say CONFIG_TARGET_AM335X_EVM results in
>> > CONFIG_SYS_ARCH="arm", CONFIG_SYS_CPU="armv7", etc.
>> >
>> > I _think_ the problem here is what I ran into with doing CONS_INDEX
>> > patches.  You cannot do 'select SYS_CPU="armv7"'.  So we would instead
>> > need to get rid of CONFIG_SYS_CPU and instead do CONFIG_SYS_CPU_ARMV7,
>> > etc, etc.
>>
>> Well I think we should run this one to ground first - if a 'select'
>> item doesn't appear in Kconfig then I think we have a bit of a
>> problem. Will await Masahiro's thoughts.
>
> It's not that we can't select, it's:
> board/ti/am335x/Kconfig:27:warning: 'CONS_INDEX' has wrong type.
> 'select' only accept arguments of boolean and tristate type
>
> So we'd have to modify Kconfig to accept int/hex/string as well so we
> could do:
> config SYS_CPU
>         default "armv7" if SOCA || SOCB || ...
>         default "ppc4xx' if SOCC || SOCD || ..
>
> And that also seems like a conflict-generating nightmare down the line
> too.

We currently have this in the Kconfig for snow:

if TARGET_SNOW

config SYS_CPU
string
default "armv7"

config SYS_BOARD
string
default "smdk5250"

config SYS_VENDOR
string
default "samsung"

config SYS_SOC
string
default "exynos"

config SYS_CONFIG_NAME
string
default "snow"

endif


so now I'm wondering why we can't just pick this up somehow. Sure we
might need to read all the Kconfig files (using the library you
mentioned) but then we could perhaps very quickly work out the CPU,
BOARD, etc. based on the target name. It makes me wonder if
genboardscfg.py could be a lot more efficient?

Regards,
Simon