special handle of scripts/kconfig/zconf.tab.o

special handle of scripts/kconfig/zconf.tab.o

[Cao jin]

Masahiro-san,

I have a question about make *config. In scripts/kconfig/Makefile, there
is following statement:

$(obj)/zconf.tab.o: $(obj)/zconf.lex.c $(obj)/zconf.hash.c

and the $(obj)/zconf.{tab,hash,lex}.c match the rule in Makefile.lib:

$(obj)/%: $(src)/%_shipped
        $(call cmd,shipped)

and cmd_shipped just transform the _shipped file to .c via `cat`.

And zconf.tab.c includes several *other* .c files which make the whole
process a little obscure, because there are not corresponding .o files
for the *other* .c files.

My questions is: Does this special handling has other meanings that I
may miss? Or just legacy.

Because a straightforward way in my mind would be:

rename zconf.{tab,hash,lex}.c_shipped to zconf.{tab,hash,lex}.c, then
has following in the Makefile

common-objs := zconf.tab.o zconf.hash.o zconf.lex.o util.o etc...
conf-objs := conf.o $(common-objs)

-- 
Sincerely,
Cao jin

Re: special handle of scripts/kconfig/zconf.tab.o

[Masahiro Yamada]

Hi.
(+CC Sam)

2017-08-15 20:02 GMT+09:00 Cao jin <caoj.fnst@cn.fujitsu.com>:
> Masahiro-san,
>
> I have a question about make *config. In scripts/kconfig/Makefile, there
> is following statement:
>
> $(obj)/zconf.tab.o: $(obj)/zconf.lex.c $(obj)/zconf.hash.c
>
> and the $(obj)/zconf.{tab,hash,lex}.c match the rule in Makefile.lib:
>
> $(obj)/%: $(src)/%_shipped
>         $(call cmd,shipped)
>
> and cmd_shipped just transform the _shipped file to .c via `cat`.
>
> And zconf.tab.c includes several *other* .c files which make the whole
> process a little obscure, because there are not corresponding .o files
> for the *other* .c files.
>
> My questions is: Does this special handling has other meanings that I
> may miss? Or just legacy.


This convention had existed before I joined the kernel development,
so I am not sure about the historical background.

The following is my understanding.
(I hope Sam and Michal will correct me if I am wrong.)


The difference between with/without _shipped is,
zconf.{tab,hash,lex}.c_shipped reside under $(srctree),
whereas zconf.{tab,hash,lex}.c under $(objtree).

zconf.{tab,hash,lex}.c are generated by
bison, gperf, flex, respectively.
(zconf.y, zconf.gperf, zconf.l are *real* source files)

So, Kbuild likes to copy zconf.{tab,hash,lex}.c to $(objtree)
to treat them as generated files, I think.




> Because a straightforward way in my mind would be:
>
> rename zconf.{tab,hash,lex}.c_shipped to zconf.{tab,hash,lex}.c, then
> has following in the Makefile
>
> common-objs := zconf.tab.o zconf.hash.o zconf.lex.o util.o etc...
> conf-objs := conf.o $(common-objs)
>

This is possible.

I think Kbuild developers chose to put them in $(objtree)
because they are not essentially source files.

The location of C files affect the include search path.


See line 208 of scripts/kconfig/Makefile:

# generated files seem to need this to find local include files
HOSTCFLAGS_zconf.lex.o  := -I$(src)
HOSTCFLAGS_zconf.tab.o  := -I$(src)




Here, we have one more question.


Can we generate zconf.{tab,hash,lex}.c from zconf.{y,gperf,l}
instead of from *_shipped?

This is also possible, technically.
But, I do not know if it is acceptable to have
more external tool dependencies.
(So, I sent RFC patches to hear more opinions.)




-- 
Best Regards
Masahiro Yamada

Re: special handle of scripts/kconfig/zconf.tab.o

[Sam Ravnborg]

Hi Masahiro

On Sat, Aug 19, 2017 at 09:42:40PM +0900, Masahiro Yamada wrote:
> Hi.
> (+CC Sam)
> 
> 2017-08-15 20:02 GMT+09:00 Cao jin <caoj.fnst@cn.fujitsu.com>:
> > Masahiro-san,
> >
> > I have a question about make *config. In scripts/kconfig/Makefile, there
> > is following statement:
> >
> > $(obj)/zconf.tab.o: $(obj)/zconf.lex.c $(obj)/zconf.hash.c
> >
> > and the $(obj)/zconf.{tab,hash,lex}.c match the rule in Makefile.lib:
> >
> > $(obj)/%: $(src)/%_shipped
> >         $(call cmd,shipped)
> >
> > and cmd_shipped just transform the _shipped file to .c via `cat`.
> >
> > And zconf.tab.c includes several *other* .c files which make the whole
> > process a little obscure, because there are not corresponding .o files
> > for the *other* .c files.
> >
> > My questions is: Does this special handling has other meanings that I
> > may miss? Or just legacy.
> 
> 
> This convention had existed before I joined the kernel development,
> so I am not sure about the historical background.
> 
> The following is my understanding.
> (I hope Sam and Michal will correct me if I am wrong.)
> 
> 
> The difference between with/without _shipped is,
> zconf.{tab,hash,lex}.c_shipped reside under $(srctree),
> whereas zconf.{tab,hash,lex}.c under $(objtree).

The whole idea behind the *_shipped file was to have less dependencies
on the host system buildign the kernel.
So we did not rely on users having all of flex, bison, gperf
installed in the right versions.

So this is all about less dependencies on the build host,
and we did not back then consider so much $(srctree) versus $(objtree).
I think that support for _shipped predates when we added support for
"make O=..." - but I may be wrong - did not check the history.

I recall we have had to updated support for _shipped a few times
due to some smaller issues, and maybe we never got around to actually
document this in Documentation/kbuild/

	Sam

Re: special handle of scripts/kconfig/zconf.tab.o

[Sam Ravnborg]

> > The difference between with/without _shipped is,
> > zconf.{tab,hash,lex}.c_shipped reside under $(srctree),
> > whereas zconf.{tab,hash,lex}.c under $(objtree).
> 
> The whole idea behind the *_shipped file was to have less dependencies
> on the host system buildign the kernel.
> So we did not rely on users having all of flex, bison, gperf
> installed in the right versions.
Another reason to use the _shipped files was to make
the "make *config" step faster for a clean kernel.

It is much faster to copy a file than to runn the tools.
The build machines has become significant faster since
then so maybe this is less than an issue today.

	Sam

Re: special handle of scripts/kconfig/zconf.tab.o

[Cao jin]

Hi,
  Thank you both for those valuable info.

On 08/19/2017 08:42 PM, Masahiro Yamada wrote:
> Hi.
> (+CC Sam)
> 
> 2017-08-15 20:02 GMT+09:00 Cao jin <caoj.fnst@cn.fujitsu.com>:
>> Masahiro-san,
>>
>> I have a question about make *config. In scripts/kconfig/Makefile, there
>> is following statement:
>>
>> $(obj)/zconf.tab.o: $(obj)/zconf.lex.c $(obj)/zconf.hash.c
>>
>> and the $(obj)/zconf.{tab,hash,lex}.c match the rule in Makefile.lib:
>>
>> $(obj)/%: $(src)/%_shipped
>>         $(call cmd,shipped)
>>
>> and cmd_shipped just transform the _shipped file to .c via `cat`.
>>
>> And zconf.tab.c includes several *other* .c files which make the whole
>> process a little obscure, because there are not corresponding .o files
>> for the *other* .c files.
>>
>> My questions is: Does this special handling has other meanings that I
>> may miss? Or just legacy.
> 

> 
> 
>> Because a straightforward way in my mind would be:
>>
>> rename zconf.{tab,hash,lex}.c_shipped to zconf.{tab,hash,lex}.c, then
>> has following in the Makefile
>>
>> common-objs := zconf.tab.o zconf.hash.o zconf.lex.o util.o etc...
>> conf-objs := conf.o $(common-objs)
>>
> 

> 
> 
> Here, we have one more question.
> 
> 
> Can we generate zconf.{tab,hash,lex}.c from zconf.{y,gperf,l}
> instead of from *_shipped?
> 
> This is also possible, technically.
> But, I do not know if it is acceptable to have
> more external tool dependencies.
> (So, I sent RFC patches to hear more opinions.)
> 

I think this is good for the new comers who don't know those tools
before. When I come to here. I just noticed that kbuild will cat
*_shipped file to the corresponding .c file, and miss the part that the
.c file is actually created by the tools.

And another new-comer friendly improvement may be something like: remove
"include *.c" lines in zconf.y, and modify makefile like I said above.
 So I will not get confused when I see symbols within certain .c file is
used while don't see the certain .o file.

-- 
Sincerely,
Cao jin