[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[Adam Duskett]

Just like the kernel menuconfig, this allows for a user to
specify a local directory for the uboot source code.

Also had to change ifeq ($(UBOOT_VERSION),custom) to
ifeq ($(BR2_TARGET_UBOOT_CUSTOM_TARBALL),y) in uboot.mk,
this is also just like the kernel's make file as well.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
---
 boot/uboot/Config.in | 15 +++++++++++++++
 boot/uboot/uboot.mk  |  5 ++++-
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/boot/uboot/Config.in b/boot/uboot/Config.in
index 9ffbb51..5a91461 100644
--- a/boot/uboot/Config.in
+++ b/boot/uboot/Config.in
@@ -57,6 +57,12 @@ config BR2_TARGET_UBOOT_CUSTOM_HG
 config BR2_TARGET_UBOOT_CUSTOM_SVN
 	bool "Custom Subversion repository"
 
+config BR2_TARGET_UBOOT_CUSTOM_LOCAL
+	bool "Local directory"
+	help
+	  This option allows Buildroot to get the Linux kernel source
+	  code from a local directory.
+
 endchoice
 
 config BR2_TARGET_UBOOT_CUSTOM_VERSION_VALUE
@@ -84,6 +90,13 @@ config BR2_TARGET_UBOOT_CUSTOM_REPO_VERSION
 
 endif
 
+config BR2_TARGET_UBOOT_CUSTOM_LOCAL_PATH
+	string "Path to the local directory"
+	depends on BR2_TARGET_UBOOT_CUSTOM_LOCAL
+	help
+	  Path to the local directory with the uboot source code.
+
+
 config BR2_TARGET_UBOOT_VERSION
 	string
 	default "2016.05"	if BR2_TARGET_UBOOT_LATEST_VERSION
@@ -92,9 +105,11 @@ config BR2_TARGET_UBOOT_VERSION
 	default "custom"	if BR2_TARGET_UBOOT_CUSTOM_TARBALL
 	default BR2_TARGET_UBOOT_CUSTOM_REPO_VERSION \
 		if BR2_TARGET_UBOOT_CUSTOM_GIT || BR2_TARGET_UBOOT_CUSTOM_HG || BR2_TARGET_UBOOT_CUSTOM_SVN
+	default "custom" if BR2_TARGET_UBOOT_CUSTOM_LOCAL
 
 config BR2_TARGET_UBOOT_PATCH
 	string "Custom U-Boot patches"
+	depends on !BR2_TARGET_UBOOT_CUSTOM_LOCAL
 	help
 	  A space-separated list of patches to apply to U-Boot.
 	  Each patch can be described as an URL, a local file path,
diff --git a/boot/uboot/uboot.mk b/boot/uboot/uboot.mk
index a2274ee..86671bd 100644
--- a/boot/uboot/uboot.mk
+++ b/boot/uboot/uboot.mk
@@ -12,12 +12,15 @@ UBOOT_LICENSE_FILES = Licenses/gpl-2.0.txt
 
 UBOOT_INSTALL_IMAGES = YES
 
-ifeq ($(UBOOT_VERSION),custom)
+ifeq ($(BR2_TARGET_UBOOT_CUSTOM_TARBALL),y)
 # Handle custom U-Boot tarballs as specified by the configuration
 UBOOT_TARBALL = $(call qstrip,$(BR2_TARGET_UBOOT_CUSTOM_TARBALL_LOCATION))
 UBOOT_SITE = $(patsubst %/,%,$(dir $(UBOOT_TARBALL)))
 UBOOT_SOURCE = $(notdir $(UBOOT_TARBALL))
 BR_NO_CHECK_HASH_FOR += $(UBOOT_SOURCE)
+else ifeq ($(BR2_TARGET_UBOOT_CUSTOM_LOCAL),y)
+UBOOT_SITE = $(call qstrip,$(BR2_TARGET_UBOOT_CUSTOM_LOCAL_PATH))
+UBOOT_SITE_METHOD = local
 else ifeq ($(BR2_TARGET_UBOOT_CUSTOM_GIT),y)
 UBOOT_SITE = $(call qstrip,$(BR2_TARGET_UBOOT_CUSTOM_REPO_URL))
 UBOOT_SITE_METHOD = git
-- 
2.7.4

[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[Yann E. MORIN]

Adam, All,

(Peter, Thomas, Arnout: questions for you, below.)

On 2016-06-26 22:25 -0400, Adam Duskett spake thusly:
> Just like the kernel menuconfig, this allows for a user to
> specify a local directory for the uboot source code.

I'm not sure this is a great idea... I'm not sure it was a great idea to
allow it for the Linux kernel either, mind you.

The problem with local sources is that they make the build
non-reproducible.

If your concern is to be able to use a local source during development,
we have the override-srcdir mechanism just for that. See in the
menuconfig, in the "Build options" sub-menu:

    ($(CONFIG_DIR)/local.mk) location of a package override file

So, by default, Buildroot will look for a file named "local.mk" in the
same directory with the .config file.  You can change the location where
to find that file. If it does not exist, that's not an error. If it
exists, Buildroot will parse it.

The local.mk file is suposed to include override-srcdir directives, like
so:

    MY_PACKAGE_OVERRIDE_SRCDIR = /path/to/my/local/source/for/my/package

For example:

    LINUX_OVERRIDE_SRCDIR = /path/to/my/linux
    UBOOT_OVERRIDE_SRCDIR = /path/to/my/uboot

With those, Buildroot will conclude that it should download the affected
package, and just use the sources it finds in the specified directory.
Note that Buildroot will not apply any patches in this case; you are
responsbile for propagating whatever patch you need. Buildroot jsut use
the sources as-is.

And looking at the commit that introduced it for linux, it is exactly
the case that would have been covered by an override-srcdir:

    commit 73da2ff6f718f2889e3c5024d899f8d58f502863
    Author: Rafal Fabich <rafal.fabich@gmail.com>
    Date:   Sat Feb 23 19:03:30 2013 +0100

        Added local directory as source of kernel code
    
        Add the option to use a local directory as the source for
        building the Linux kernel, which can be useful during
        kernel development.
    
        Signed-off-by: Rafal Fabich <rafal.fabich@gmail.com>
        Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
        Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
        Signed-off-by: Peter Korsgaard <peter@korsgaard.com>

Peter, any reason that you applied this patch at the time?
Arnout, anything you remember for giving your ACK?

So, I would think that:
  1- we don't want this for U-Boot;
  2- we should drop it for Linux.

Peter, Thomas, Arnout: what do you think?

Regards,
Yann E. MORIN.

> Also had to change ifeq ($(UBOOT_VERSION),custom) to
> ifeq ($(BR2_TARGET_UBOOT_CUSTOM_TARBALL),y) in uboot.mk,
> this is also just like the kernel's make file as well.
> 
> Signed-off-by: Adam Duskett <Aduskett@gmail.com>
> ---
>  boot/uboot/Config.in | 15 +++++++++++++++
>  boot/uboot/uboot.mk  |  5 ++++-
>  2 files changed, 19 insertions(+), 1 deletion(-)
> 
> diff --git a/boot/uboot/Config.in b/boot/uboot/Config.in
> index 9ffbb51..5a91461 100644
> --- a/boot/uboot/Config.in
> +++ b/boot/uboot/Config.in
> @@ -57,6 +57,12 @@ config BR2_TARGET_UBOOT_CUSTOM_HG
>  config BR2_TARGET_UBOOT_CUSTOM_SVN
>  	bool "Custom Subversion repository"
>  
> +config BR2_TARGET_UBOOT_CUSTOM_LOCAL
> +	bool "Local directory"
> +	help
> +	  This option allows Buildroot to get the Linux kernel source
> +	  code from a local directory.
> +
>  endchoice
>  
>  config BR2_TARGET_UBOOT_CUSTOM_VERSION_VALUE
> @@ -84,6 +90,13 @@ config BR2_TARGET_UBOOT_CUSTOM_REPO_VERSION
>  
>  endif
>  
> +config BR2_TARGET_UBOOT_CUSTOM_LOCAL_PATH
> +	string "Path to the local directory"
> +	depends on BR2_TARGET_UBOOT_CUSTOM_LOCAL
> +	help
> +	  Path to the local directory with the uboot source code.
> +
> +
>  config BR2_TARGET_UBOOT_VERSION
>  	string
>  	default "2016.05"	if BR2_TARGET_UBOOT_LATEST_VERSION
> @@ -92,9 +105,11 @@ config BR2_TARGET_UBOOT_VERSION
>  	default "custom"	if BR2_TARGET_UBOOT_CUSTOM_TARBALL
>  	default BR2_TARGET_UBOOT_CUSTOM_REPO_VERSION \
>  		if BR2_TARGET_UBOOT_CUSTOM_GIT || BR2_TARGET_UBOOT_CUSTOM_HG || BR2_TARGET_UBOOT_CUSTOM_SVN
> +	default "custom" if BR2_TARGET_UBOOT_CUSTOM_LOCAL
>  
>  config BR2_TARGET_UBOOT_PATCH
>  	string "Custom U-Boot patches"
> +	depends on !BR2_TARGET_UBOOT_CUSTOM_LOCAL
>  	help
>  	  A space-separated list of patches to apply to U-Boot.
>  	  Each patch can be described as an URL, a local file path,
> diff --git a/boot/uboot/uboot.mk b/boot/uboot/uboot.mk
> index a2274ee..86671bd 100644
> --- a/boot/uboot/uboot.mk
> +++ b/boot/uboot/uboot.mk
> @@ -12,12 +12,15 @@ UBOOT_LICENSE_FILES = Licenses/gpl-2.0.txt
>  
>  UBOOT_INSTALL_IMAGES = YES
>  
> -ifeq ($(UBOOT_VERSION),custom)
> +ifeq ($(BR2_TARGET_UBOOT_CUSTOM_TARBALL),y)
>  # Handle custom U-Boot tarballs as specified by the configuration
>  UBOOT_TARBALL = $(call qstrip,$(BR2_TARGET_UBOOT_CUSTOM_TARBALL_LOCATION))
>  UBOOT_SITE = $(patsubst %/,%,$(dir $(UBOOT_TARBALL)))
>  UBOOT_SOURCE = $(notdir $(UBOOT_TARBALL))
>  BR_NO_CHECK_HASH_FOR += $(UBOOT_SOURCE)
> +else ifeq ($(BR2_TARGET_UBOOT_CUSTOM_LOCAL),y)
> +UBOOT_SITE = $(call qstrip,$(BR2_TARGET_UBOOT_CUSTOM_LOCAL_PATH))
> +UBOOT_SITE_METHOD = local
>  else ifeq ($(BR2_TARGET_UBOOT_CUSTOM_GIT),y)
>  UBOOT_SITE = $(call qstrip,$(BR2_TARGET_UBOOT_CUSTOM_REPO_URL))
>  UBOOT_SITE_METHOD = git
> -- 
> 2.7.4
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[Peter Korsgaard]

>>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:

 > Adam, All,
 > (Peter, Thomas, Arnout: questions for you, below.)

 > On 2016-06-26 22:25 -0400, Adam Duskett spake thusly:
 >> Just like the kernel menuconfig, this allows for a user to
 >> specify a local directory for the uboot source code.

 > I'm not sure this is a great idea... I'm not sure it was a great idea to
 > allow it for the Linux kernel either, mind you.

 > The problem with local sources is that they make the build
 > non-reproducible.

Indeed.

 >     commit 73da2ff6f718f2889e3c5024d899f8d58f502863
 >     Author: Rafal Fabich <rafal.fabich@gmail.com>
 >     Date:   Sat Feb 23 19:03:30 2013 +0100

 >         Added local directory as source of kernel code
    
 >         Add the option to use a local directory as the source for
 >         building the Linux kernel, which can be useful during
 >         kernel development.
    
 >         Signed-off-by: Rafal Fabich <rafal.fabich@gmail.com>
 >         Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
 >         Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
 >         Signed-off-by: Peter Korsgaard <peter@korsgaard.com>

 > Peter, any reason that you applied this patch at the time?
 > Arnout, anything you remember for giving your ACK?

Sorry, I don't remember. Most likely because Arnout acked it. I have
never personally used this feature.

 > So, I would think that:
 >   1- we don't want this for U-Boot;
 >   2- we should drop it for Linux.

 > Peter, Thomas, Arnout: what do you think?

Ok from me unless somebody can explain a use case for it.

-- 
Venlig hilsen,
Peter Korsgaard 

[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[Arnout Vandecappelle]

On 27-06-16 22:57, Yann E. MORIN wrote:
[snip]
> And looking at the commit that introduced it for linux, it is exactly
> the case that would have been covered by an override-srcdir:
> 
>     commit 73da2ff6f718f2889e3c5024d899f8d58f502863
>     Author: Rafal Fabich <rafal.fabich@gmail.com>
>     Date:   Sat Feb 23 19:03:30 2013 +0100
> 
>         Added local directory as source of kernel code
>     
>         Add the option to use a local directory as the source for
>         building the Linux kernel, which can be useful during
>         kernel development.
>     
>         Signed-off-by: Rafal Fabich <rafal.fabich@gmail.com>
>         Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
>         Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
>         Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
> 
> Peter, any reason that you applied this patch at the time?
> Arnout, anything you remember for giving your ACK?

 I think at the time I considered using _OVERRIDE_SRCDIR for something like this
too clunky. However, now I'm more convinced that every realistic project should
have an automatic _OVERRIDE_SRCDIR for everything in $(BR2_EXTERNAL)/src. So
then indeed this option is not so relevant anymore.

 It's still a bit clunky, though, that your (def)config has to specify
_something_ for the kernel version, but that it will never be used in practice.

 I propose to reject this patch but keep the existing LINUX_CUSTOM_LOCAL around
for a bit still.

 Regards,
 Arnout

> 
> So, I would think that:
>   1- we don't want this for U-Boot;
>   2- we should drop it for Linux.
> 
> Peter, Thomas, Arnout: what do you think?
[snip]
-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[Yann E. MORIN]

Arnout, All,

On 2016-06-28 00:27 +0200, Arnout Vandecappelle spake thusly:
> On 27-06-16 22:57, Yann E. MORIN wrote:
> [snip]
> > And looking at the commit that introduced it for linux, it is exactly
> > the case that would have been covered by an override-srcdir:
> > 
> >     commit 73da2ff6f718f2889e3c5024d899f8d58f502863
> >     Author: Rafal Fabich <rafal.fabich@gmail.com>
> >     Date:   Sat Feb 23 19:03:30 2013 +0100
> > 
> >         Added local directory as source of kernel code
> >     
> >         Add the option to use a local directory as the source for
> >         building the Linux kernel, which can be useful during
> >         kernel development.
> >     
> >         Signed-off-by: Rafal Fabich <rafal.fabich@gmail.com>
> >         Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> >         Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> >         Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
> > 
> > Peter, any reason that you applied this patch at the time?
> > Arnout, anything you remember for giving your ACK?
> 
>  I think at the time I considered using _OVERRIDE_SRCDIR for something like this
> too clunky. However, now I'm more convinced that every realistic project should
> have an automatic _OVERRIDE_SRCDIR for everything in $(BR2_EXTERNAL)/src. So
> then indeed this option is not so relevant anymore.

Well, any two projects would have different workflows, so suggesting a
"realistic project" uses an automated override-srcdir is just pushing
the limits too much, IMHO.

Besides, my opinion is that having the sources for packages in the
br2-external tree is bad, and that each package should have its own
git/hg/svn/bzr tree. But that varies between projects! ;-)

>  It's still a bit clunky, though, that your (def)config has to specify
> _something_ for the kernel version, but that it will never be used in practice.

Well, that would be the same for any package for which we have a version
choice when an override-srcdir is in use for that package...

And anyway, any override-srcdir should not be versioned I think. It
should be purely for local development.

>  I propose to reject this patch but keep the existing LINUX_CUSTOM_LOCAL around
> for a bit still.

Well, I saw your mail after I sent the patch, so... ;-)

Regards,
Yann E. MORIN.

>  Regards,
>  Arnout
> 
> > 
> > So, I would think that:
> >   1- we don't want this for U-Boot;
> >   2- we should drop it for Linux.
> > 
> > Peter, Thomas, Arnout: what do you think?
> [snip]
> -- 
> Arnout Vandecappelle                          arnout at mind be
> Senior Embedded Software Architect            +32-16-286500
> Essensium/Mind                                http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[Arnout Vandecappelle]

On 28-06-16 00:39, Yann E. MORIN wrote:
> Arnout, All,
> 
> On 2016-06-28 00:27 +0200, Arnout Vandecappelle spake thusly:
>> On 27-06-16 22:57, Yann E. MORIN wrote:
>> [snip]
>>> And looking at the commit that introduced it for linux, it is exactly
>>> the case that would have been covered by an override-srcdir:
>>>
>>>     commit 73da2ff6f718f2889e3c5024d899f8d58f502863
>>>     Author: Rafal Fabich <rafal.fabich@gmail.com>
>>>     Date:   Sat Feb 23 19:03:30 2013 +0100
>>>
>>>         Added local directory as source of kernel code
>>>     
>>>         Add the option to use a local directory as the source for
>>>         building the Linux kernel, which can be useful during
>>>         kernel development.
>>>     
>>>         Signed-off-by: Rafal Fabich <rafal.fabich@gmail.com>
>>>         Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
>>>         Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
>>>         Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
>>>
>>> Peter, any reason that you applied this patch at the time?
>>> Arnout, anything you remember for giving your ACK?
>>
>>  I think at the time I considered using _OVERRIDE_SRCDIR for something like this
>> too clunky. However, now I'm more convinced that every realistic project should
>> have an automatic _OVERRIDE_SRCDIR for everything in $(BR2_EXTERNAL)/src. So
>> then indeed this option is not so relevant anymore.
> 
> Well, any two projects would have different workflows, so suggesting a
> "realistic project" uses an automated override-srcdir is just pushing
> the limits too much, IMHO.

 Of course. But it's true that the same workflows are covered by
_OVERRIDE_SRCDIR and _CUSTOM_LOCAL.

> 
> Besides, my opinion is that having the sources for packages in the
> br2-external tree is bad, and that each package should have its own
> git/hg/svn/bzr tree. But that varies between projects! ;-)

 They are git submodules, of course.

 In my experience referring to tags or shas from a defconfig is really clunky,
it makes updating extremely inflexible. I've implemented scripts to automate the
tagging for some projects; in the end they didn't use it, because the submodule
tag already covers it.


> 
>>  It's still a bit clunky, though, that your (def)config has to specify
>> _something_ for the kernel version, but that it will never be used in practice.
> 
> Well, that would be the same for any package for which we have a version
> choice when an override-srcdir is in use for that package...

 ... which is why someone had the idea of adding this option for U-Boot :-)


 Regards,
 Arnout

> 
> And anyway, any override-srcdir should not be versioned I think. It
> should be purely for local development.
> 
>>  I propose to reject this patch but keep the existing LINUX_CUSTOM_LOCAL around
>> for a bit still.
> 
> Well, I saw your mail after I sent the patch, so... ;-)
> 
> Regards,
> Yann E. MORIN.
> 
>>  Regards,
>>  Arnout
>>
>>>
>>> So, I would think that:
>>>   1- we don't want this for U-Boot;
>>>   2- we should drop it for Linux.
>>>
>>> Peter, Thomas, Arnout: what do you think?
>> [snip]
>> -- 
>> Arnout Vandecappelle                          arnout at mind be
>> Senior Embedded Software Architect            +32-16-286500
>> Essensium/Mind                                http://www.mind.be
>> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
>> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
>> GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[aduskett at gmail.com]

Interesting, as I use the feature quite a bit!? I have my linux source as a submodule of my project. I guess I could just point buildroot to the git project of the kernel source as well and achieve the same results. 





On Mon, Jun 27, 2016 at 6:50 PM -0400, "Arnout Vandecappelle" <arnout@mind.be> wrote:












On 28-06-16 00:39, Yann E. MORIN wrote:
> Arnout, All,
> 
> On 2016-06-28 00:27 +0200, Arnout Vandecappelle spake thusly:
>> On 27-06-16 22:57, Yann E. MORIN wrote:
>> [snip]
>>> And looking at the commit that introduced it for linux, it is exactly
>>> the case that would have been covered by an override-srcdir:
>>>
>>>     commit 73da2ff6f718f2889e3c5024d899f8d58f502863
>>>     Author: Rafal Fabich 
>>>     Date:   Sat Feb 23 19:03:30 2013 +0100
>>>
>>>         Added local directory as source of kernel code
>>>     
>>>         Add the option to use a local directory as the source for
>>>         building the Linux kernel, which can be useful during
>>>         kernel development.
>>>     
>>>         Signed-off-by: Rafal Fabich 
>>>         Acked-by: Arnout Vandecappelle (Essensium/Mind) 
>>>         Tested-by: Arnout Vandecappelle (Essensium/Mind) 
>>>         Signed-off-by: Peter Korsgaard 
>>>
>>> Peter, any reason that you applied this patch at the time?
>>> Arnout, anything you remember for giving your ACK?
>>
>>  I think at the time I considered using _OVERRIDE_SRCDIR for something like this
>> too clunky. However, now I'm more convinced that every realistic project should
>> have an automatic _OVERRIDE_SRCDIR for everything in $(BR2_EXTERNAL)/src. So
>> then indeed this option is not so relevant anymore.
> 
> Well, any two projects would have different workflows, so suggesting a
> "realistic project" uses an automated override-srcdir is just pushing
> the limits too much, IMHO.

 Of course. But it's true that the same workflows are covered by
_OVERRIDE_SRCDIR and _CUSTOM_LOCAL.

> 
> Besides, my opinion is that having the sources for packages in the
> br2-external tree is bad, and that each package should have its own
> git/hg/svn/bzr tree. But that varies between projects! ;-)

 They are git submodules, of course.

 In my experience referring to tags or shas from a defconfig is really clunky,
it makes updating extremely inflexible. I've implemented scripts to automate the
tagging for some projects; in the end they didn't use it, because the submodule
tag already covers it.


> 
>>  It's still a bit clunky, though, that your (def)config has to specify
>> _something_ for the kernel version, but that it will never be used in practice.
> 
> Well, that would be the same for any package for which we have a version
> choice when an override-srcdir is in use for that package...

 ... which is why someone had the idea of adding this option for U-Boot :-)


 Regards,
 Arnout

> 
> And anyway, any override-srcdir should not be versioned I think. It
> should be purely for local development.
> 
>>  I propose to reject this patch but keep the existing LINUX_CUSTOM_LOCAL around
>> for a bit still.
> 
> Well, I saw your mail after I sent the patch, so... ;-)
> 
> Regards,
> Yann E. MORIN.
> 
>>  Regards,
>>  Arnout
>>
>>>
>>> So, I would think that:
>>>   1- we don't want this for U-Boot;
>>>   2- we should drop it for Linux.
>>>
>>> Peter, Thomas, Arnout: what do you think?
>> [snip]
>> -- 
>> Arnout Vandecappelle                          arnout at mind be
>> Senior Embedded Software Architect            +32-16-286500
>> Essensium/Mind                                http://www.mind.be
>> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
>> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
>> GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20160628/30e2a8fa/attachment.html>

[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

Hi,

 >> Well, any two projects would have different workflows, so suggesting a
 >> "realistic project" uses an automated override-srcdir is just pushing
 >> the limits too much, IMHO.

 >  Of course. But it's true that the same workflows are covered by
 > _OVERRIDE_SRCDIR and _CUSTOM_LOCAL.

Yes, except that _CUSTOM_LOCAL ends up in the .config, so it easily gets
committed to version version control, whereas you have to go out of your
way to shoot yourself in the foot with _OVERRIDE_SRCDIR.

 >> 
 >> Besides, my opinion is that having the sources for packages in the
 >> br2-external tree is bad, and that each package should have its own
 >> git/hg/svn/bzr tree. But that varies between projects! ;-)

 >  They are git submodules, of course.

 >  In my experience referring to tags or shas from a defconfig is really clunky,
 > it makes updating extremely inflexible. I've implemented scripts to automate the
 > tagging for some projects; in the end they didn't use it, because the submodule
 > tag already covers it.

What is the problem with referring to shas from the defconfig? That
sounds to me to be no more work than changing the submodule version. Or
are you just referring to the effort to tag the individual components
for a release?

-- 
Venlig hilsen,
Peter Korsgaard 

[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[Arnout Vandecappelle]

On 28-06-16 08:23, Peter Korsgaard wrote:
>>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:
> 
> Hi,
> 
>  >> Well, any two projects would have different workflows, so suggesting a
>  >> "realistic project" uses an automated override-srcdir is just pushing
>  >> the limits too much, IMHO.
> 
>  >  Of course. But it's true that the same workflows are covered by
>  > _OVERRIDE_SRCDIR and _CUSTOM_LOCAL.
> 
> Yes, except that _CUSTOM_LOCAL ends up in the .config, so it easily gets
> committed to version version control, whereas you have to go out of your
> way to shoot yourself in the foot with _OVERRIDE_SRCDIR.

 Except, of course, when you really want to refer to a separately managed local
checkout. Though in that case putting local.mk under version control is equally
easy.

> 
>  >> 
>  >> Besides, my opinion is that having the sources for packages in the
>  >> br2-external tree is bad, and that each package should have its own
>  >> git/hg/svn/bzr tree. But that varies between projects! ;-)
> 
>  >  They are git submodules, of course.
> 
>  >  In my experience referring to tags or shas from a defconfig is really clunky,
>  > it makes updating extremely inflexible. I've implemented scripts to automate the
>  > tagging for some projects; in the end they didn't use it, because the submodule
>  > tag already covers it.
> 
> What is the problem with referring to shas from the defconfig? That
> sounds to me to be no more work than changing the submodule version. Or
> are you just referring to the effort to tag the individual components
> for a release?

 To update the sha, you have to:

- commit in the component repo;
- generate a tag;
- update the .config;
- run a test build;
- make savedefconfig;
- git diff and check that it's OK to commit this;
- commit;
- generate a tag for integration repo;
- push both repos.

=> This really requires a script.


 While with submodules and OVERRIDE_SRCDIR, you do:

- run a test build;
- commit in the component repo;
- commit in the integration repo;
- git push --recurse-submodules

i.e. only the steps that you really want to do by hand.

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

Hi,

 >> Yes, except that _CUSTOM_LOCAL ends up in the .config, so it easily gets
 >> committed to version version control, whereas you have to go out of your
 >> way to shoot yourself in the foot with _OVERRIDE_SRCDIR.

 >  Except, of course, when you really want to refer to a separately managed local
 > checkout. Though in that case putting local.mk under version control is equally
 > easy.

Exactly.

>> >  In my experience referring to tags or shas from a defconfig is really clunky,
 >> > it makes updating extremely inflexible. I've implemented scripts to automate the
 >> > tagging for some projects; in the end they didn't use it, because the submodule
 >> > tag already covers it.
 >> 
 >> What is the problem with referring to shas from the defconfig? That
 >> sounds to me to be no more work than changing the submodule version. Or
 >> are you just referring to the effort to tag the individual components
 >> for a release?

 >  To update the sha, you have to:

 > - commit in the component repo;
 > - generate a tag;

Why? You can just select the sha in the .config.

 > - update the .config;
 > - run a test build;

Or just let the continous integration system do this. That way you are
also sure it is clean build from pristine sources.


 > - make savedefconfig;
 > - git diff and check that it's OK to commit this;
 > - commit;
 > - generate a tag for integration repo;

Why the tag?

> - push both repos.

I think people would normally push the component changes earlier/more
often than the buildroot .config is changed, but ok.

 > => This really requires a script.

I don't really think so, but ok - Some might. It will end up being quite
a lot of scripts if you have many local components.


 >  While with submodules and OVERRIDE_SRCDIR, you do:

 > - run a test build;
 > - commit in the component repo;
 > - commit in the integration repo;
 > - git push --recurse-submodules

Without the tags above, this looks pretty similar to me. You don't
manually need to adjust the git hash in the .config, but you instead
gain the complications of sub modules and you basically need a buildroot
branch per project.

But anyway, both work flows are supported with override - So we're fine.

-- 
Venlig hilsen,
Peter Korsgaard 

[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[Adam Duskett]

On Jun 27, 2016 16:57, "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:
>
> Adam, All,
>
> (Peter, Thomas, Arnout: questions for you, below.)
>
> On 2016-06-26 22:25 -0400, Adam Duskett spake thusly:
> > Just like the kernel menuconfig, this allows for a user to
> > specify a local directory for the uboot source code.
>
> I'm not sure this is a great idea... I'm not sure it was a great idea to
> allow it for the Linux kernel either, mind you.
>
> The problem with local sources is that they make the build
> non-reproducible.
>
> If your concern is to be able to use a local source during development,
> we have the override-srcdir mechanism just for that. See in the
> menuconfig, in the "Build options" sub-menu:
>
>     ($(CONFIG_DIR)/local.mk) location of a package override file
>
Interesting!  However I was not aware of this feature and I would suspect
most users won't know about it because it's not in menuconfig.  For local
development why would this be a bad thing?

> So, by default, Buildroot will look for a file named "local.mk" in the
> same directory with the .config file.  You can change the location where
> to find that file. If it does not exist, that's not an error. If it
> exists, Buildroot will parse it.
>
> The local.mk file is suposed to include override-srcdir directives, like
> so:
>
>     MY_PACKAGE_OVERRIDE_SRCDIR = /path/to/my/local/source/for/my/package
>
> For example:
>
>     LINUX_OVERRIDE_SRCDIR = /path/to/my/linux
>     UBOOT_OVERRIDE_SRCDIR = /path/to/my/uboot
>
> With those, Buildroot will conclude that it should download the affected
> package, and just use the sources it finds in the specified directory.
> Note that Buildroot will not apply any patches in this case; you are
> responsbile for propagating whatever patch you need. Buildroot jsut use
> the sources as-is.
>
Again, while this is good; I believe it isn't advertised well in the
documentation. I had to go hunt it.  Perhaps a revision to the online
documentation to inform users of this feature other than putting it in the
"using buildroot for development" section?  Perhaps just a subsection in
that section?

> And looking at the commit that introduced it for linux, it is exactly
> the case that would have been covered by an override-srcdir:
>
>     commit 73da2ff6f718f2889e3c5024d899f8d58f502863
>     Author: Rafal Fabich <rafal.fabich@gmail.com>
>     Date:   Sat Feb 23 19:03:30 2013 +0100
>
>         Added local directory as source of kernel code
>
>         Add the option to use a local directory as the source for
>         building the Linux kernel, which can be useful during
>         kernel development.
>
>         Signed-off-by: Rafal Fabich <rafal.fabich@gmail.com>
>         Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
>         Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
>         Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
>
> Peter, any reason that you applied this patch at the time?
> Arnout, anything you remember for giving your ACK?
>
> So, I would think that:
>   1- we don't want this for U-Boot;
>   2- we should drop it for Linux.
>
> Peter, Thomas, Arnout: what do you think?
>
> Regards,
> Yann E. MORIN.
>
> > Also had to change ifeq ($(UBOOT_VERSION),custom) to
> > ifeq ($(BR2_TARGET_UBOOT_CUSTOM_TARBALL),y) in uboot.mk,
> > this is also just like the kernel's make file as well.
> >
> > Signed-off-by: Adam Duskett <Aduskett@gmail.com>
> > ---
> >  boot/uboot/Config.in | 15 +++++++++++++++
> >  boot/uboot/uboot.mk  |  5 ++++-
> >  2 files changed, 19 insertions(+), 1 deletion(-)
> >
> > diff --git a/boot/uboot/Config.in b/boot/uboot/Config.in
> > index 9ffbb51..5a91461 100644
> > --- a/boot/uboot/Config.in
> > +++ b/boot/uboot/Config.in
> > @@ -57,6 +57,12 @@ config BR2_TARGET_UBOOT_CUSTOM_HG
> >  config BR2_TARGET_UBOOT_CUSTOM_SVN
> >       bool "Custom Subversion repository"
> >
> > +config BR2_TARGET_UBOOT_CUSTOM_LOCAL
> > +     bool "Local directory"
> > +     help
> > +       This option allows Buildroot to get the Linux kernel source
> > +       code from a local directory.
> > +
> >  endchoice
> >
> >  config BR2_TARGET_UBOOT_CUSTOM_VERSION_VALUE
> > @@ -84,6 +90,13 @@ config BR2_TARGET_UBOOT_CUSTOM_REPO_VERSION
> >
> >  endif
> >
> > +config BR2_TARGET_UBOOT_CUSTOM_LOCAL_PATH
> > +     string "Path to the local directory"
> > +     depends on BR2_TARGET_UBOOT_CUSTOM_LOCAL
> > +     help
> > +       Path to the local directory with the uboot source code.
> > +
> > +
> >  config BR2_TARGET_UBOOT_VERSION
> >       string
> >       default "2016.05"       if BR2_TARGET_UBOOT_LATEST_VERSION
> > @@ -92,9 +105,11 @@ config BR2_TARGET_UBOOT_VERSION
> >       default "custom"        if BR2_TARGET_UBOOT_CUSTOM_TARBALL
> >       default BR2_TARGET_UBOOT_CUSTOM_REPO_VERSION \
> >               if BR2_TARGET_UBOOT_CUSTOM_GIT ||
BR2_TARGET_UBOOT_CUSTOM_HG || BR2_TARGET_UBOOT_CUSTOM_SVN
> > +     default "custom" if BR2_TARGET_UBOOT_CUSTOM_LOCAL
> >
> >  config BR2_TARGET_UBOOT_PATCH
> >       string "Custom U-Boot patches"
> > +     depends on !BR2_TARGET_UBOOT_CUSTOM_LOCAL
> >       help
> >         A space-separated list of patches to apply to U-Boot.
> >         Each patch can be described as an URL, a local file path,
> > diff --git a/boot/uboot/uboot.mk b/boot/uboot/uboot.mk
> > index a2274ee..86671bd 100644
> > --- a/boot/uboot/uboot.mk
> > +++ b/boot/uboot/uboot.mk
> > @@ -12,12 +12,15 @@ UBOOT_LICENSE_FILES = Licenses/gpl-2.0.txt
> >
> >  UBOOT_INSTALL_IMAGES = YES
> >
> > -ifeq ($(UBOOT_VERSION),custom)
> > +ifeq ($(BR2_TARGET_UBOOT_CUSTOM_TARBALL),y)
> >  # Handle custom U-Boot tarballs as specified by the configuration
> >  UBOOT_TARBALL = $(call
qstrip,$(BR2_TARGET_UBOOT_CUSTOM_TARBALL_LOCATION))
> >  UBOOT_SITE = $(patsubst %/,%,$(dir $(UBOOT_TARBALL)))
> >  UBOOT_SOURCE = $(notdir $(UBOOT_TARBALL))
> >  BR_NO_CHECK_HASH_FOR += $(UBOOT_SOURCE)
> > +else ifeq ($(BR2_TARGET_UBOOT_CUSTOM_LOCAL),y)
> > +UBOOT_SITE = $(call qstrip,$(BR2_TARGET_UBOOT_CUSTOM_LOCAL_PATH))
> > +UBOOT_SITE_METHOD = local
> >  else ifeq ($(BR2_TARGET_UBOOT_CUSTOM_GIT),y)
> >  UBOOT_SITE = $(call qstrip,$(BR2_TARGET_UBOOT_CUSTOM_REPO_URL))
> >  UBOOT_SITE_METHOD = git
> > --
> > 2.7.4
> >
> > _______________________________________________
> > buildroot mailing list
> > buildroot at busybox.net
> > http://lists.busybox.net/mailman/listinfo/buildroot
>
> --
>
.-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics'
conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___
     |
> | +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There
is no  |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v
 conspiracy.  |
>
'------------------------------^-------^------------------^--------------------'
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20160627/0a1e6bad/attachment.html>

[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[Peter Korsgaard]

>>>>> "Adam" == Adam Duskett <aduskett@gmail.com> writes:

Hi,

 >> If your concern is to be able to use a local source during development,
 >> we have the override-srcdir mechanism just for that. See in the
 >> menuconfig, in the "Build options" sub-menu:
 >> 
 >> ($(CONFIG_DIR)/local.mk) location of a package override file
 >> 
 > Interesting!  However I was not aware of this feature and I would suspect
 > most users won't know about it because it's not in menuconfig.  For local
 > development why would this be a bad thing?

I agree that it isn't an obvious feature and that it is easy to miss. As
Yann said, systems like Buildroot are really about reproducability, so
referring to something local on the developers machine doesn't match
very well.


 >> With those, Buildroot will conclude that it should download the affected
 >> package, and just use the sources it finds in the specified directory.
 >> Note that Buildroot will not apply any patches in this case; you are
 >> responsbile for propagating whatever patch you need. Buildroot jsut use
 >> the sources as-is.
 >> 
 > Again, while this is good; I believe it isn't advertised well in the
 > documentation. I had to go hunt it.  Perhaps a revision to the online
 > documentation to inform users of this feature other than putting it in the
 > "using buildroot for development" section?  Perhaps just a subsection in
 > that section?

Yes, can you please send a patch? It is always very nice if "new" people
can help us improve the documentation. Once you have used Buildroot for
years it becomes hard to know what is missing from the documentation. 

Thanks!

-- 
Venlig hilsen,
Peter Korsgaard 

[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[Thomas Petazzoni]

Hello,

On Tue, 28 Jun 2016 08:14:39 +0200, Peter Korsgaard wrote:

>  >> If your concern is to be able to use a local source during development,
>  >> we have the override-srcdir mechanism just for that. See in the
>  >> menuconfig, in the "Build options" sub-menu:
>  >> 
>  >> ($(CONFIG_DIR)/local.mk) location of a package override file
>  >>   
>  > Interesting!  However I was not aware of this feature and I would suspect
>  > most users won't know about it because it's not in menuconfig.  For local
>  > development why would this be a bad thing?  
> 
> I agree that it isn't an obvious feature and that it is easy to miss. As
> Yann said, systems like Buildroot are really about reproducability, so
> referring to something local on the developers machine doesn't match
> very well.

This is only if you use Buildroot solely as a "final integration" tool.
But lots of people use Buildroot even during active development on a
given software component. In such a situation, people are ready to
trade "reproducibility" against "ease of building/testing a new
change". You certainly don't want to push to your kernel tree a new
commit, change your Buildroot configuration, pay the price of a
completely new kernel clone + build, to test each and every change you
make to your kernel.

The kernel, U-Boot and other bootloaders are somewhat special
components, as you often need HW-specific versions of those. This is
why we have the possibility of specifying custom Git versions, custom
tarballs, etc. From that perspective, having a way to specify a local
directory is also not a bad idea.

To me the _OVERRIDE_SRCDIR / local.mk thing is meant to be _local_ to
the developer machine. It's meant to only be used while hacking on a
given package, and at the end, produce a patch, or commit your changes.
I am not a big fan of suggesting people to version control their
local.mk, but maybe it's just a matter of taste (and improper name for
this local.mk).

So in the end, I am not as clear cut as you and Yann are about
rejecting the U-Boot "local directory" patch and removing the similar
functionality from Linux.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [v1 1/1] uboot: Add local directory option to menuconfig

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

Hi,

 >> I agree that it isn't an obvious feature and that it is easy to miss. As
 >> Yann said, systems like Buildroot are really about reproducability, so
 >> referring to something local on the developers machine doesn't match
 >> very well.

 > This is only if you use Buildroot solely as a "final integration" tool.
 > But lots of people use Buildroot even during active development on a
 > given software component. In such a situation, people are ready to
 > trade "reproducibility" against "ease of building/testing a new
 > change". You certainly don't want to push to your kernel tree a new
 > commit, change your Buildroot configuration, pay the price of a
 > completely new kernel clone + build, to test each and every change you
 > make to your kernel.

You as a local developer might not, but for integration purposes
(E.G. driven through a continous integration tool like
Jenkins/buildbot/..) it becomes very important, especially as teams grow
in size. The fact that we don't track reverse dependencies means that
clean rebuilds are very important, and I would very much argue that this
is a daily thing rather than "final" integration.

People might do their development outside Buildroot (E.G. build by hand)
or use override if they prefer to let Buildroot build it, but finally
the tested changes need to get committed and the buildroot config
adjusted to use the new version.


 > The kernel, U-Boot and other bootloaders are somewhat special
 > components, as you often need HW-specific versions of those. This is
 > why we have the possibility of specifying custom Git versions, custom
 > tarballs, etc. From that perspective, having a way to specify a local
 > directory is also not a bad idea.

But the only advantage of local directory over override (besides
documentation / ease of discovery) is that the change is "permanent"
(E.G. listed in .config). That to me seems like a big disadvantage.


 > To me the _OVERRIDE_SRCDIR / local.mk thing is meant to be _local_ to
 > the developer machine. It's meant to only be used while hacking on a
 > given package, and at the end, produce a patch, or commit your changes.

Indeed, isn't that the use case you describe above?

 > I am not a big fan of suggesting people to version control their
 > local.mk, but maybe it's just a matter of taste (and improper name for
 > this local.mk).

But why would you want to make the development setup override permanent?

-- 
Venlig hilsen,
Peter Korsgaard