All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCHv2] core: enhance printvars
@ 2017-03-28 17:40 Yann E. MORIN
  2017-03-28 20:52 ` Arnout Vandecappelle
  0 siblings, 1 reply; 3+ messages in thread
From: Yann E. MORIN @ 2017-03-28 17:40 UTC (permalink / raw)
  To: buildroot

Currently, the output of printvars copntains the name of the variable,
its expanded value and its un-expanded value.

However, most of the time, we need the actual, expanded value, so it can
be re-used from a (non-Buildroot) infrastructure script, like a
post-build script, or a build-farm driver (e.g. a Jenkins job...)

Add two options that a user may set to change the output of printvars:
  - QUOTED_VARS, if set, will quote the value
  - RAW_VARS, if set, will print the unexpanded value

The new output by default only prints the expanded value now.

So that it can be used as such:

    $ make -s printvars VARS=BUSYBOX_VERSION
    BUSYBOX_VERSION=1.26.2

    $ make -s printvars VARS=BUSYBOX_RDEPENDENCIES QUOTED_VARS=YES
    BUSYBOX_RDEPENDENCIES='ncurses util-linux'

    $ make -s printvars VARS=BUSYBOX_FINAL_PATCH_DEPENDENCIES RAW_VARS=YES
    BUSYBOX_FINAL_PATCH_DEPENDENCIES=$(sort $(BUSYBOX_PATCH_DEPENDENCIES))

And it is even possible to directly evaluate it in a shell script:

    eval $(make -s printvars VARS=BUSYBOX_VERSION QUOTED_VARS=YES)

Backward compatibility of the output is not maintained. It is believed
that scripts that depended on the previous output were very fragile to
begin with, because they had to filter the non-formatted output
(splitting on spaces or braces was not really possible, because values
could contain either).

Document printvars and its options in the output of 'make help' and in
the manual.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>

---
Changes v1 -> v2:
  - add the documentation  (Thomas, Arnout)
  - change the meaning of conmfig variables  (Thomas, Arnout)
  - don't maintain backward compatibility  (Thomas, Arnout)
---
 Makefile                  |  3 ++-
 docs/manual/make-tips.txt | 53 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index cceae92..d3530ad 100644
--- a/Makefile
+++ b/Makefile
@@ -940,7 +940,7 @@ printvars:
 		$(sort $(if $(VARS),$(filter $(VARS),$(.VARIABLES)),$(.VARIABLES))), \
 		$(if $(filter-out environment% default automatic, \
 				$(origin $V)), \
-		$(info $V=$($V) ($(value $V)))))
+		$(info $V=$(if $(QUOTED_VARS),')$(if $(RAW_VARS),$(value $V),$($V))$(if $(QUOTED_VARS),'))))
 
 clean:
 	rm -rf $(TARGET_DIR) $(BINARIES_DIR) $(HOST_DIR) \
@@ -1018,6 +1018,7 @@ help:
 	@echo '  source-check           - check selected packages for valid download URLs'
 	@echo '  external-deps          - list external packages used'
 	@echo '  legal-info             - generate info about license compliance'
+	@echo '  printvars              - dump all the variables known to make'
 	@echo
 	@echo '  make V=0|1             - 0 => quiet build (default), 1 => verbose build'
 	@echo '  make O=dir             - Locate all output files in "dir", including .config'
diff --git a/docs/manual/make-tips.txt b/docs/manual/make-tips.txt
index 97a3302..691e25f 100644
--- a/docs/manual/make-tips.txt
+++ b/docs/manual/make-tips.txt
@@ -77,3 +77,56 @@ To delete all build products as well as the configuration:
 If +ccache+ is enabled, running +make clean+ or +distclean+ does
 not empty the compiler cache used by Buildroot. To delete it, refer
 to xref:ccache[].
+
+.Dumping the internal make variables:
+
+One can dump all the variables known to make, along with their values:
+
+----
+ $ make -s printvars
+ VARIABLE=value_of_variable
+ ...
+----
+
+It is possible to tweak the output using some variables:
+
+- +VARS+ will limit the listing to variables which names match the
+  specified make-pattern
+- +QUOTED_VARS+, if set to +YES+, will single-quote the value
+- +RAW_VARS+, if set to +YES+, will print the unexpanded value
+
+For example:
+
+----
+ $ make -s printvars VARS=BUSYBOX_%DEPENDENCIES
+ BUSYBOX_DEPENDENCIES=skeleton toolchain
+ BUSYBOX_FINAL_ALL_DEPENDENCIES=skeleton toolchain
+ BUSYBOX_FINAL_DEPENDENCIES=skeleton toolchain
+ BUSYBOX_FINAL_PATCH_DEPENDENCIES=
+ BUSYBOX_RDEPENDENCIES=ncurses util-linux
+----
+
+----
+ $ make -s printvars VARS=BUSYBOX_%DEPENDENCIES QUOTED_VARS=YES
+ BUSYBOX_DEPENDENCIES='skeleton toolchain'
+ BUSYBOX_FINAL_ALL_DEPENDENCIES='skeleton toolchain'
+ BUSYBOX_FINAL_DEPENDENCIES='skeleton toolchain'
+ BUSYBOX_FINAL_PATCH_DEPENDENCIES=''
+ BUSYBOX_RDEPENDENCIES='ncurses util-linux'
+----
+
+----
+ $ make -s printvars VARS=BUSYBOX_%DEPENDENCIES RAW_VARS=YES
+ BUSYBOX_DEPENDENCIES=skeleton toolchain
+ BUSYBOX_FINAL_ALL_DEPENDENCIES=$(sort $(BUSYBOX_FINAL_DEPENDENCIES) $(BUSYBOX_FINAL_PATCH_DEPENDENCIES))
+ BUSYBOX_FINAL_DEPENDENCIES=$(sort $(BUSYBOX_DEPENDENCIES))
+ BUSYBOX_FINAL_PATCH_DEPENDENCIES=$(sort $(BUSYBOX_PATCH_DEPENDENCIES))
+ BUSYBOX_RDEPENDENCIES=ncurses util-linux
+----
+
+The output of quoited raw variables can be reused in shell scripts,
+for example:
+
+----
+ eval $(make -s printvars VARS=BUSYBOX_VERSION RAW_VARS=YES QUOTED_VARS=YES)
+----
-- 
2.9.3

^ permalink raw reply related	[flat|nested] 3+ messages in thread

* [Buildroot] [PATCHv2] core: enhance printvars
  2017-03-28 17:40 [Buildroot] [PATCHv2] core: enhance printvars Yann E. MORIN
@ 2017-03-28 20:52 ` Arnout Vandecappelle
  2017-03-29  9:47   ` Yann E. MORIN
  0 siblings, 1 reply; 3+ messages in thread
From: Arnout Vandecappelle @ 2017-03-28 20:52 UTC (permalink / raw)
  To: buildroot



On 28-03-17 19:40, Yann E. MORIN wrote:
> Currently, the output of printvars copntains the name of the variable,
> its expanded value and its un-expanded value.
> 
> However, most of the time, we need the actual, expanded value, so it can
> be re-used from a (non-Buildroot) infrastructure script, like a
> post-build script, or a build-farm driver (e.g. a Jenkins job...)
> 
> Add two options that a user may set to change the output of printvars:
>   - QUOTED_VARS, if set, will quote the value
>   - RAW_VARS, if set, will print the unexpanded value
> 
> The new output by default only prints the expanded value now.
> 
> So that it can be used as such:
> 
>     $ make -s printvars VARS=BUSYBOX_VERSION
>     BUSYBOX_VERSION=1.26.2
> 
>     $ make -s printvars VARS=BUSYBOX_RDEPENDENCIES QUOTED_VARS=YES
>     BUSYBOX_RDEPENDENCIES='ncurses util-linux'
> 
>     $ make -s printvars VARS=BUSYBOX_FINAL_PATCH_DEPENDENCIES RAW_VARS=YES
>     BUSYBOX_FINAL_PATCH_DEPENDENCIES=$(sort $(BUSYBOX_PATCH_DEPENDENCIES))
> 
> And it is even possible to directly evaluate it in a shell script:
> 
>     eval $(make -s printvars VARS=BUSYBOX_VERSION QUOTED_VARS=YES)
> 
> Backward compatibility of the output is not maintained. It is believed
> that scripts that depended on the previous output were very fragile to
> begin with, because they had to filter the non-formatted output
> (splitting on spaces or braces was not really possible, because values
> could contain either).
> 
> Document printvars and its options in the output of 'make help' and in
> the manual.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
> Cc: Arnout Vandecappelle <arnout@mind.be>

 Two nits, one which can be fixed while applying, the other that is probably
better in a follow-up patch. So

Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

> 
> ---
> Changes v1 -> v2:
>   - add the documentation  (Thomas, Arnout)
>   - change the meaning of conmfig variables  (Thomas, Arnout)
>   - don't maintain backward compatibility  (Thomas, Arnout)
> ---
>  Makefile                  |  3 ++-
>  docs/manual/make-tips.txt | 53 +++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 55 insertions(+), 1 deletion(-)
> 
> diff --git a/Makefile b/Makefile
> index cceae92..d3530ad 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -940,7 +940,7 @@ printvars:
>  		$(sort $(if $(VARS),$(filter $(VARS),$(.VARIABLES)),$(.VARIABLES))), \
>  		$(if $(filter-out environment% default automatic, \
>  				$(origin $V)), \
> -		$(info $V=$($V) ($(value $V)))))
> +		$(info $V=$(if $(QUOTED_VARS),')$(if $(RAW_VARS),$(value $V),$($V))$(if $(QUOTED_VARS),'))))

 Unfortunately, this doesn't fully protect the variable for shell evaluation,
because the value may contain quotes. Try e.g. 'make VARS=BMON_CONF_ENV
QUOTED_VARS=YES'.

 The solution is to add $(subst ','\'',...). But of course only in case of
QUOTED_VARS. Which makes it a little annoying... Probably best to use
$(eval value := $(if $(RAW_VARS),$(value $V),$($V))) \
$(info $V=$(if $(QUOTED_VARS),'$(value)',$(value))

 Since that's pretty complicated, it's better in a separate patch. Anyway a
value that has quotes probably doesn't work well in the shell eval (because the
quotes will normally *not* be interpreted when the variable is used later on).

[snip]
> +
> +The output of quoited raw variables can be reused in shell scripts,
                    ^

 Regards,
 Arnout

> +for example:
> +
> +----
> + eval $(make -s printvars VARS=BUSYBOX_VERSION RAW_VARS=YES QUOTED_VARS=YES)
> +----
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

^ permalink raw reply	[flat|nested] 3+ messages in thread

* [Buildroot] [PATCHv2] core: enhance printvars
  2017-03-28 20:52 ` Arnout Vandecappelle
@ 2017-03-29  9:47   ` Yann E. MORIN
  0 siblings, 0 replies; 3+ messages in thread
From: Yann E. MORIN @ 2017-03-29  9:47 UTC (permalink / raw)
  To: buildroot

Arnout, All,

On 2017-03-28 22:52 +0200, Arnout Vandecappelle spake thusly:
> On 28-03-17 19:40, Yann E. MORIN wrote:
> > Currently, the output of printvars copntains the name of the variable,
> > its expanded value and its un-expanded value.
> > 
> > However, most of the time, we need the actual, expanded value, so it can
> > be re-used from a (non-Buildroot) infrastructure script, like a
> > post-build script, or a build-farm driver (e.g. a Jenkins job...)
> > 
> > Add two options that a user may set to change the output of printvars:
> >   - QUOTED_VARS, if set, will quote the value
> >   - RAW_VARS, if set, will print the unexpanded value
> > 
> > The new output by default only prints the expanded value now.
> > 
> > So that it can be used as such:
> > 
> >     $ make -s printvars VARS=BUSYBOX_VERSION
> >     BUSYBOX_VERSION=1.26.2
> > 
> >     $ make -s printvars VARS=BUSYBOX_RDEPENDENCIES QUOTED_VARS=YES
> >     BUSYBOX_RDEPENDENCIES='ncurses util-linux'
> > 
> >     $ make -s printvars VARS=BUSYBOX_FINAL_PATCH_DEPENDENCIES RAW_VARS=YES
> >     BUSYBOX_FINAL_PATCH_DEPENDENCIES=$(sort $(BUSYBOX_PATCH_DEPENDENCIES))
> > 
> > And it is even possible to directly evaluate it in a shell script:
> > 
> >     eval $(make -s printvars VARS=BUSYBOX_VERSION QUOTED_VARS=YES)
> > 
> > Backward compatibility of the output is not maintained. It is believed
> > that scripts that depended on the previous output were very fragile to
> > begin with, because they had to filter the non-formatted output
> > (splitting on spaces or braces was not really possible, because values
> > could contain either).
> > 
> > Document printvars and its options in the output of 'make help' and in
> > the manual.
> > 
> > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> > Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
> > Cc: Arnout Vandecappelle <arnout@mind.be>
> 
>  Two nits, one which can be fixed while applying, the other that is probably
> better in a follow-up patch. So

Seeing the issue you poiinted at, this means the quoting feature is
barely usefull. So I repsun the patch with your suggestion to subst the
single quotes. In the end it was not too complex, at the cost of a bit
of duplication.

> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

Thanks. But I did not carry it in v3 of the patch, because too much has
changed.

> > ---
> > Changes v1 -> v2:
> >   - add the documentation  (Thomas, Arnout)
> >   - change the meaning of conmfig variables  (Thomas, Arnout)
> >   - don't maintain backward compatibility  (Thomas, Arnout)
> > ---
> >  Makefile                  |  3 ++-
> >  docs/manual/make-tips.txt | 53 +++++++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 55 insertions(+), 1 deletion(-)
> > 
> > diff --git a/Makefile b/Makefile
> > index cceae92..d3530ad 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -940,7 +940,7 @@ printvars:
> >  		$(sort $(if $(VARS),$(filter $(VARS),$(.VARIABLES)),$(.VARIABLES))), \
> >  		$(if $(filter-out environment% default automatic, \
> >  				$(origin $V)), \
> > -		$(info $V=$($V) ($(value $V)))))
> > +		$(info $V=$(if $(QUOTED_VARS),')$(if $(RAW_VARS),$(value $V),$($V))$(if $(QUOTED_VARS),'))))
> 
>  Unfortunately, this doesn't fully protect the variable for shell evaluation,
> because the value may contain quotes. Try e.g. 'make VARS=BMON_CONF_ENV
> QUOTED_VARS=YES'.

Damned, good catch.

>  The solution is to add $(subst ','\'',...). But of course only in case of
> QUOTED_VARS. Which makes it a little annoying... Probably best to use
> $(eval value := $(if $(RAW_VARS),$(value $V),$($V))) \
> $(info $V=$(if $(QUOTED_VARS),'$(value)',$(value))
> 
>  Since that's pretty complicated, it's better in a separate patch. Anyway a
> value that has quotes probably doesn't work well in the shell eval (because the
> quotes will normally *not* be interpreted when the variable is used later on).
> 
> [snip]
> > +
> > +The output of quoited raw variables can be reused in shell scripts,
>                     ^

What, you do not like typoes? ;-)

Regards,
Yann E. MORIN.

>  Regards,
>  Arnout
> 
> > +for example:
> > +
> > +----
> > + eval $(make -s printvars VARS=BUSYBOX_VERSION RAW_VARS=YES QUOTED_VARS=YES)
> > +----
> > 
> 
> -- 
> Arnout Vandecappelle                          arnout at mind be
> Senior Embedded Software Architect            +32-16-286500
> Essensium/Mind                                http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2017-03-29  9:47 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-28 17:40 [Buildroot] [PATCHv2] core: enhance printvars Yann E. MORIN
2017-03-28 20:52 ` Arnout Vandecappelle
2017-03-29  9:47   ` Yann E. MORIN

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.