[Buildroot] how BR download the linux kernel now

[Buildroot] how BR download the linux kernel now

[raymond zhao]

Hi,
I checked out the latest version of the buildroot, and noticed that
the linux.mk was changed a lot. Thomas checked in a version with log
message "linux: convert to the GENTARGETS infrastructure". And, from
there, I am confused. How the BR download the linux kernel now? I
mean, before this version, there are download, extraction and patch.
But they are removed and there is only LINUX_DOWNLOAD_PATCHES in the
new version. Where is the script for BR to download the kernel source
code now?

thanks,

Raymond

[Buildroot] how BR download the linux kernel now

[Yann E. MORIN]

Raymond, All,

On Tuesday 12 July 2011 22:26:49 raymond zhao wrote:
> I checked out the latest version of the buildroot, and noticed that
> the linux.mk was changed a lot. Thomas checked in a version with log
> message "linux: convert to the GENTARGETS infrastructure". And, from
> there, I am confused. How the BR download the linux kernel now? I
> mean, before this version, there are download, extraction and patch.
> But they are removed and there is only LINUX_DOWNLOAD_PATCHES in the
> new version. Where is the script for BR to download the kernel source
> code now?

In a word, it's automagic, now! :-)

In a hundred words, look at linux/linux.mk:

- on line 9-28, we set the two variables LINUX_SITE and LINUX_TARBALL
  depending on the user's options
- on line 180, we call GENTARGETS that generates all the reuired rules

- the variable LINUX_SITE tells where the kernel is to be get from
- the variable LINUX_TARBALL tells the tarball's file name
- the function GENTARGETS instantiates download+extract+build+install rules
  for the specified package

GENTARGETS is defined in package/Makefile.package.in
It's the basis for the generic package infrastructure. It's a bit complex,
but that Makefile is rather pretty well commented. Also, you can have a
look at the doc:
  http://buildroot.org/buildroot.html#add_packages
  http://buildroot.org/buildroot.html#generic-reference

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20110712/75eb5552/attachment.html>

[Buildroot] how BR download the linux kernel now

[raymond zhao]

On Tue, Jul 12, 2011 at 4:59 PM, Yann E. MORIN <
yann.morin.1998@anciens.enib.fr> wrote:

> **
>
> Raymond, All,
>
> On Tuesday 12 July 2011 22:26:49 raymond zhao wrote:
>
> > I checked out the latest version of the buildroot, and noticed that
>
> > the linux.mk was changed a lot. Thomas checked in a version with log
>
> > message "linux: convert to the GENTARGETS infrastructure". And, from
>
> > there, I am confused. How the BR download the linux kernel now? I
>
> > mean, before this version, there are download, extraction and patch.
>
> > But they are removed and there is only LINUX_DOWNLOAD_PATCHES in the
>
> > new version. Where is the script for BR to download the kernel source
>
> > code now?
>
> In a word, it's automagic, now! :-)
>
> In a hundred words, look at linux/linux.mk:
>
> - on line 9-28, we set the two variables LINUX_SITE and LINUX_TARBALL
>
> depending on the user's options
>
> - on line 180, we call GENTARGETS that generates all the reuired rules
>
> - the variable LINUX_SITE tells where the kernel is to be get from
>
> - the variable LINUX_TARBALL tells the tarball's file name
>
> - the function GENTARGETS instantiates download+extract+build+install rules
>
> for the specified package
>
> GENTARGETS is defined in package/Makefile.package.in
>
> It's the basis for the generic package infrastructure. It's a bit complex,
>
> but that Makefile is rather pretty well commented. Also, you can have a
>
> look at the doc:
>
> http://buildroot.org/buildroot.html#add_packages
>
> http://buildroot.org/buildroot.html#generic-reference
>
> Regards,
>
> Yann E. MORIN.
>
> --
>
>
> .-----------------.--------------------.------------------.--------------------.
>
> | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics'
> conspiracy: |
>
> | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
>
> | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
>
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
>
>
> '------------------------------^-------^------------------^--------------------'
>
>
Thanks, Yann
I do not know the initial motive of this movement, but I prefer the old
approach.
The linux kernel is a special "package". Lots of embedded projects (if not
all of them) will start from a certain version of the kernel and hack the
kernel from there, such as add device drivers etc. Then, put the source code
into a local git server to do version control. Some people argued to use
original tarball plus the patch, but it makes the development procedure very
painful. In the old approach, it is very easy to hack the linux.mk to
checkout the kernel source code directly into the output directory. But, in
the new one. Looks it becomes more complex. The kernel is very big. Check
out with the git, archive it to a tarball, and then extract it to the output
directory for building will waste lots of time. Is there a official solution
(or suggestion) for my situation?

again, thanks a lot.

Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20110712/d36f142d/attachment.html>

[Buildroot] how BR download the linux kernel now

[Thomas Petazzoni]

Le Tue, 12 Jul 2011 18:03:27 -0400,
raymond zhao <raymond.zhao.ml@gmail.com> a ?crit :

> I do not know the initial motive of this movement, but I prefer the
> old approach.

The motivation is to benefit from the common features of the package
infrastructure (git/svn/bzr download, common way of tracking package
construction, etc.)

> The linux kernel is a special "package". Lots of embedded projects
> (if not all of them) will start from a certain version of the kernel
> and hack the kernel from there, such as add device drivers etc. Then,
> put the source code into a local git server to do version control.
> Some people argued to use original tarball plus the patch, but it
> makes the development procedure very painful. In the old approach, it
> is very easy to hack the linux.mk to checkout the kernel source code
> directly into the output directory. But, in the new one. Looks it
> becomes more complex. The kernel is very big. Check out with the git,
> archive it to a tarball, and then extract it to the output directory
> for building will waste lots of time. Is there a official solution
> (or suggestion) for my situation?

Yes, I am currently working on extending the package infrastructure so
that any package (not only the Linux kernel, but any package) can be
sourced from an existing directory, instead of being downloaded through
git/svn/http.

I have already published preliminary versions of this work (see "[RFC]
Override source directories", posted May, 18th 2011), and I intend to
continue. The basic mechanism is working, I still have issues
implementing "make source" and "make external-deps" with this mechanism.

It'd be really nice if you could have a look at what I did to see if it
would solve your use case as well.

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] how BR download the linux kernel now

[raymond zhao]

>> The linux kernel is a special "package". Lots of embedded projects

>> (if not all of them) will start from a certain version of the kernel

>> and hack the kernel from there, such as add device drivers etc. Then,

>> put the source code into a local git server to do version control.

>> Some people argued to use original tarball plus the patch, but it

>> makes the development procedure very painful. In the old approach, it

>> is very easy to hack the linux.mk to checkout the kernel source code

>> directly into the output directory. But, in the new one. Looks it
>
>> becomes more complex. The kernel is very big. Check out with the git,
>
>> archive it to a tarball, and then extract it to the output directory
>
>> for building will waste lots of time. Is there a official solution
>
>> (or suggestion) for my situation?
>
>Yes, I am currently working on extending the package infrastructure so
>
>that any package (not only the Linux kernel, but any package) can be
>
>sourced from an existing directory, instead of being downloaded through
>
>git/svn/http.
>
>I have already published preliminary versions of this work (see "[RFC]
>
>Override source directories", posted May, 18th 2011), and I intend to
>
>continue. The basic mechanism is working, I still have issues
>
>implementing "make source" and "make external-deps" with this mechanism.
>
>It'd be really nice if you could have a look at what I did to see if it
>
>would solve your use case as well.

That's great. I will definitely check it. If we solved this problem. I
believe BR will be #1 choice for most embedded linux projects.

thanks,
raymond

[Buildroot] how BR download the linux kernel now

[Thomas Petazzoni]

Le Wed, 13 Jul 2011 09:31:10 -0400,
raymond zhao <raymond.zhao.ml@gmail.com> a ?crit :

> That's great. I will definitely check it. If we solved this problem. I
> believe BR will be #1 choice for most embedded linux projects.

Thanks. I indeed think that this feature would be a really nice
addition to Buildroot. I miss it in many of the projects for which I
use Buildroot.

For the moment, my workaround is to not use Buildroot to build the part
of the system I'm actively working on. When I'm doing kernel
development, I build the kernel outside of Buildroot. When I develop an
application, I build the application outside of Buildroot. Buildroot is
here to build all the rest, and to provide a reproducible mechanism to
build the final system image.

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] how BR download the linux kernel now

[raymond zhao]

On Wed, Jul 13, 2011 at 9:31 AM, raymond zhao <raymond.zhao.ml@gmail.com> wrote:
>>> The linux kernel is a special "package". Lots of embedded projects
>
>>> (if not all of them) will start from a certain version of the kernel
>
>>> and hack the kernel from there, such as add device drivers etc. Then,
>
>>> put the source code into a local git server to do version control.
>
>>> Some people argued to use original tarball plus the patch, but it
>
>>> makes the development procedure very painful. In the old approach, it
>
>>> is very easy to hack the linux.mk to checkout the kernel source code
>
>>> directly into the output directory. But, in the new one. Looks it
>>
>>> becomes more complex. The kernel is very big. Check out with the git,
>>
>>> archive it to a tarball, and then extract it to the output directory
>>
>>> for building will waste lots of time. Is there a official solution
>>
>>> (or suggestion) for my situation?
>>
>>Yes, I am currently working on extending the package infrastructure so
>>
>>that any package (not only the Linux kernel, but any package) can be
>>
>>sourced from an existing directory, instead of being downloaded through
>>
>>git/svn/http.
>>
>>I have already published preliminary versions of this work (see "[RFC]
>>
>>Override source directories", posted May, 18th 2011), and I intend to
>>
>>continue. The basic mechanism is working, I still have issues
>>
>>implementing "make source" and "make external-deps" with this mechanism.
>>
>>It'd be really nice if you could have a look at what I did to see if it
>>
>>would solve your use case as well.
>
> That's great. I will definitely check it. If we solved this problem. I
> believe BR will be #1 choice for most embedded linux projects.
>
> thanks,
> raymond
>

BR 2011.08 rc was released and I did not see this feature in there? Is
it still going to happened in 2011.08 release?
Please add this feature if possible, it is really useful.
thanks,
raymond

[Buildroot] how BR download the linux kernel now

[Thomas Petazzoni]

Hello Raymond,

Le Tue, 23 Aug 2011 16:30:29 -0400,
raymond zhao <raymond.zhao.ml@gmail.com> a ?crit :

> BR 2011.08 rc was released and I did not see this feature in there? Is
> it still going to happened in 2011.08 release?
> Please add this feature if possible, it is really useful.

The feature that allows to override the source directory for a package
hasn't been integrated in 2011.08 because it was too late in the
development cycle to integrate it.

However, I was posted yesterday a new version of the patches and I hope
to get them merged for the 2011.11. At least this time I am not late: I
posted the patches the exact day 2011.08 was released, so I couldn't do
earlier than that in the development cycle :-)

If you have the opportunity to test those patches, it would be great.

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com