[Buildroot] [PATCH 0/6] Misc graphs enhancements (branch yem/graphs)

[Buildroot] [PATCH 0/6] Misc graphs enhancements (branch yem/graphs)

[Yann E. MORIN]

From: "Yann E. MORIN" <yann.morin.1998@free.fr>

Hello All!

Here is a quick series with enhancements to generated graphs:
  - graph-depends
    - new option to limit the depth of the graph
    - toolchain is only shown as a dependency of of 'All', not
      every packages
  - build-time
    - embiggen the graph with lots of packages

This should hopefully be a step to present nicer graphs. Not yet
really perfect, but still.

Regards,
Yann E. MORIN.


The following changes since commit fae7cc33a117d5408019b705d76ebb4549593f88:

  mtd: fix build against uClibc (2014-04-13 21:44:06 +0200)

are available in the git repository at:

  git://gitorious.org/buildroot/buildroot.git yem/graphs

for you to fetch changes up to 3f8fcc1e8205eb851227c1b085355089a9368faf:

  graphs: make graphs with lots of packages nicer to look at (2014-04-13 22:37:45 +0200)

----------------------------------------------------------------
Yann E. MORIN (6):
      Makefile: rename non-user-facing variable
      support/graph-depends: use argparse to parse argv[]
      support/graph-depends: add option to limit the depth of the graph
      manual: document BR2_GRAPH_DEPTH
      support/graph-depends: don't show toolchain dependency for all packages
      graphs: make graphs with lots of packages nicer to look at

 Makefile                         | 11 +++---
 docs/manual/common-usage.txt     |  2 +
 package/pkg-generic.mk           |  4 +-
 support/scripts/graph-build-time | 17 ++++++---
 support/scripts/graph-depends    | 79 ++++++++++++++++++++++++++++------------
 5 files changed, 76 insertions(+), 37 deletions(-)

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/6] Makefile: rename non-user-facing variable

[Yann E. MORIN]

From: "Yann E. MORIN" <yann.morin.1998@free.fr>

Variables should be prefixed with BR_ when they are not user-facing.

As a side effect, the new variable is prettier than the previous one. :-)

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 Makefile               | 8 ++++----
 package/pkg-generic.mk | 2 +-
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/Makefile b/Makefile
index 41c51c6..b0eec7f 100644
--- a/Makefile
+++ b/Makefile
@@ -149,7 +149,7 @@ endif
 
 # Need that early, before we scan packages
 # Avoids doing the $(or...) everytime
-_BR2_GRAPH_OUT := $(or $(BR2_GRAPH_OUT),pdf)
+BR_GRAPH_OUT := $(or $(BR2_GRAPH_OUT),pdf)
 
 BUILD_DIR:=$(BASE_DIR)/build
 BINARIES_DIR:=$(BASE_DIR)/images
@@ -662,18 +662,18 @@ graph-build: $(O)/build/build-time.log
 	@install -d $(O)/graphs
 	$(foreach o,name build duration,./support/scripts/graph-build-time \
 					--type=histogram --order=$(o) --input=$(<) \
-					--output=$(O)/graphs/build.hist-$(o).$(_BR2_GRAPH_OUT) \
+					--output=$(O)/graphs/build.hist-$(o).$(BR_GRAPH_OUT) \
 					$(if $(BR2_GRAPH_ALT),--alternate-colors)$(sep))
 	$(foreach t,packages steps,./support/scripts/graph-build-time \
 				   --type=pie-$(t) --input=$(<) \
-				   --output=$(O)/graphs/build.pie-$(t).$(_BR2_GRAPH_OUT) \
+				   --output=$(O)/graphs/build.pie-$(t).$(BR_GRAPH_OUT) \
 				   $(if $(BR2_GRAPH_ALT),--alternate-colors)$(sep))
 
 graph-depends:
 	@$(INSTALL) -d $(O)/graphs
 	@cd "$(CONFIG_DIR)"; \
 	$(TOPDIR)/support/scripts/graph-depends \
-	|dot -T$(_BR2_GRAPH_OUT) -o $(O)/graphs/$(@).$(_BR2_GRAPH_OUT)
+	|dot -T$(BR_GRAPH_OUT) -o $(O)/graphs/$(@).$(BR_GRAPH_OUT)
 
 else # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
 
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 339c3eb..b3a4c17 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -496,7 +496,7 @@ $(1)-graph-depends:
 			@$(INSTALL) -d $(O)/graphs
 			@cd "$(CONFIG_DIR)"; \
 			$(TOPDIR)/support/scripts/graph-depends $(1) \
-			|dot -T$(_BR2_GRAPH_OUT) -o $(O)/graphs/$$(@).$(_BR2_GRAPH_OUT)
+			|dot -T$(BR_GRAPH_OUT) -o $(O)/graphs/$$(@).$(BR_GRAPH_OUT)
 
 $(1)-dirclean:		$$($(2)_TARGET_DIRCLEAN)
 
-- 
1.8.3.2

[Buildroot] [PATCH 2/6] support/graph-depends: use argparse to parse argv[]

[Yann E. MORIN]

From: "Yann E. MORIN" <yann.morin.1998@free.fr>

Currently, we are using a crude, ad-hoc parsing of argv[].
This is a limiting factor to adding new options.

Use argparse instead, and introduce a single argument for now:
  --package, -p PACKAGE

In the (near) future, we'll be able to add more option arguments,
such as depth-limiting for big graphs.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/pkg-generic.mk        |  2 +-
 support/scripts/graph-depends | 23 +++++++++++++----------
 2 files changed, 14 insertions(+), 11 deletions(-)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index b3a4c17..080ee56 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -495,7 +495,7 @@ $(1)-show-depends:
 $(1)-graph-depends:
 			@$(INSTALL) -d $(O)/graphs
 			@cd "$(CONFIG_DIR)"; \
-			$(TOPDIR)/support/scripts/graph-depends $(1) \
+			$(TOPDIR)/support/scripts/graph-depends -p $(1) \
 			|dot -T$(BR_GRAPH_OUT) -o $(O)/graphs/$$(@).$(BR_GRAPH_OUT)
 
 $(1)-dirclean:		$$($(2)_TARGET_DIRCLEAN)
diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
index ac24086..fc3cadd 100755
--- a/support/scripts/graph-depends
+++ b/support/scripts/graph-depends
@@ -1,13 +1,13 @@
 #!/usr/bin/python
 
 # Usage (the graphviz package must be installed in your distribution)
-#  ./support/scripts/graph-depends [package-name] > test.dot
+#  ./support/scripts/graph-depends [-p package-name] > test.dot
 #  dot -Tpdf test.dot -o test.pdf
 #
 # With no arguments, graph-depends will draw a complete graph of
-# dependencies for the current configuration. With an argument,
-# graph-depends will draw a graph of dependencies for the given
-# package name.
+# dependencies for the current configuration.
+# If '-p <package-name>' is specified, graph-depends will draw a graph
+# of dependencies for the given package name.
 #
 # Limitations
 #
@@ -21,6 +21,7 @@
 
 import sys
 import subprocess
+import argparse
 
 # In FULL_MODE, we draw the full dependency graph for all selected
 # packages
@@ -31,14 +32,16 @@ PKG_MODE  = 2
 
 mode = 0
 
-if len(sys.argv) == 1:
+parser = argparse.ArgumentParser(description="Graph pacakges dependencies")
+parser.add_argument("--package", '-p', metavar="PACKAGE",
+                    help="Graph the dependencies of PACKAGE")
+args = parser.parse_args()
+
+if args.package == None:
     mode = FULL_MODE
-elif len(sys.argv) == 2:
-    mode = PKG_MODE
-    rootpkg  = sys.argv[1]
 else:
-    print "Usage: graph-depends [package-name]"
-    sys.exit(1)
+    mode = PKG_MODE
+    rootpkg = args.package
 
 allpkgs = []
 
-- 
1.8.3.2

[Buildroot] [PATCH 3/6] support/graph-depends: add option to limit the depth of the graph

[Yann E. MORIN]

From: "Yann E. MORIN" <yann.morin.1998@free.fr>

Currently, the complete dependency chain of a package is used to
generate the dependency graph. When this dependency chain is long,
the generated graph becomes almost unreadable.

However, it is often sufficient to get the first few levels of
dependency of a package.

Add a new variable BR2_GRAPH_DEPTH, that the user can set to limit
the depth of the dependency list.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 Makefile                      |  3 ++-
 package/pkg-generic.mk        |  2 +-
 support/scripts/graph-depends | 49 ++++++++++++++++++++++++++++++++-----------
 3 files changed, 40 insertions(+), 14 deletions(-)

diff --git a/Makefile b/Makefile
index b0eec7f..467e21c 100644
--- a/Makefile
+++ b/Makefile
@@ -150,6 +150,7 @@ endif
 # Need that early, before we scan packages
 # Avoids doing the $(or...) everytime
 BR_GRAPH_OUT := $(or $(BR2_GRAPH_OUT),pdf)
+BR_GRAPH_DEPTH := $(or $(BR2_GRAPH_DEPTH),0)
 
 BUILD_DIR:=$(BASE_DIR)/build
 BINARIES_DIR:=$(BASE_DIR)/images
@@ -672,7 +673,7 @@ graph-build: $(O)/build/build-time.log
 graph-depends:
 	@$(INSTALL) -d $(O)/graphs
 	@cd "$(CONFIG_DIR)"; \
-	$(TOPDIR)/support/scripts/graph-depends \
+	$(TOPDIR)/support/scripts/graph-depends -d $(BR_GRAPH_DEPTH) \
 	|dot -T$(BR_GRAPH_OUT) -o $(O)/graphs/$(@).$(BR_GRAPH_OUT)
 
 else # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 080ee56..c162902 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -495,7 +495,7 @@ $(1)-show-depends:
 $(1)-graph-depends:
 			@$(INSTALL) -d $(O)/graphs
 			@cd "$(CONFIG_DIR)"; \
-			$(TOPDIR)/support/scripts/graph-depends -p $(1) \
+			$(TOPDIR)/support/scripts/graph-depends -p $(1) -d $(BR_GRAPH_DEPTH) \
 			|dot -T$(BR_GRAPH_OUT) -o $(O)/graphs/$$(@).$(BR_GRAPH_OUT)
 
 $(1)-dirclean:		$$($(2)_TARGET_DIRCLEAN)
diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
index fc3cadd..ebf511b 100755
--- a/support/scripts/graph-depends
+++ b/support/scripts/graph-depends
@@ -8,6 +8,8 @@
 # dependencies for the current configuration.
 # If '-p <package-name>' is specified, graph-depends will draw a graph
 # of dependencies for the given package name.
+# If '-d <depth>' is specified, graph-depends will limit the depth of
+# the dependency graph to 'depth' levels.
 #
 # Limitations
 #
@@ -31,10 +33,13 @@ FULL_MODE = 1
 PKG_MODE  = 2
 
 mode = 0
+max_depth = 0
 
 parser = argparse.ArgumentParser(description="Graph pacakges dependencies")
 parser.add_argument("--package", '-p', metavar="PACKAGE",
                     help="Graph the dependencies of PACKAGE")
+parser.add_argument("--depth", '-d', metavar="DEPTH",
+                    help="Limit the dependency graph to DEPTH levels")
 args = parser.parse_args()
 
 if args.package == None:
@@ -43,6 +48,9 @@ else:
     mode = PKG_MODE
     rootpkg = args.package
 
+if args.depth != None:
+    max_depth = int(args.depth)
+
 allpkgs = []
 
 # Execute the "make show-targets" command to get the list of the main
@@ -193,6 +201,7 @@ if mode == FULL_MODE:
     deps = get_all_depends(filtered_targets)
     if deps != None:
         dependencies += deps
+    rootpkg = 'all'
 
 # In pkg mode, start directly with get_all_depends() on the requested
 # package
@@ -201,26 +210,42 @@ elif mode == PKG_MODE:
 
 dependencies = remove_redundant_deps(dependencies)
 
-# Start printing the graph data
-print "digraph G {"
-
-# First, the dependencies. Usage of set allows to remove duplicated
-# dependencies in the graph
-for dep in set(dependencies):
-    print "%s -> %s" % (pkg_node_name(dep[0]), pkg_node_name(dep[1]))
+# Make the dependencies a dictionnary { 'pkg':[dep1, dep2, ...] }
+dict_deps = {}
+for dep in dependencies:
+    if not dict_deps.has_key(dep[0]):
+        dict_deps[dep[0]] = []
+    dict_deps[dep[0]].append(dep[1])
 
-# Then, the node attributes: color, style and label.
-for pkg in allpkgs:
+# Print the attributes of a node: label and fill-color
+def print_attrs(pkg):
     if pkg == 'all':
         print "all [label = \"ALL\"]"
         print "all [color=lightblue,style=filled]"
-        continue
-
+        return
     print "%s [label = \"%s\"]" % (pkg_node_name(pkg), pkg)
-
     if mode == PKG_MODE and pkg == rootpkg:
         print "%s [color=lightblue,style=filled]" % pkg_node_name(rootpkg)
     else:
         print "%s [color=grey,style=filled]" % pkg_node_name(pkg)
 
+# Print the dependency graph of a package
+def print_pkg_deps(depth, pkg):
+    if pkg in done_deps:
+        return
+    done_deps.append(pkg)
+    print_attrs(pkg)
+    if not dict_deps.has_key(pkg):
+        return
+    if max_depth == 0 or depth < max_depth:
+        for d in dict_deps[pkg]:
+            print "%s -> %s" % (pkg_node_name(pkg), pkg_node_name(d))
+            print_pkg_deps(depth+1, d)
+
+# Start printing the graph data
+print "digraph G {"
+
+done_deps = []
+print_pkg_deps(0, rootpkg)
+
 print "}"
-- 
1.8.3.2

[Buildroot] [PATCH 4/6] manual: document BR2_GRAPH_DEPTH

[Yann E. MORIN]

From: "Yann E. MORIN" <yann.morin.1998@free.fr>

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 docs/manual/common-usage.txt | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/docs/manual/common-usage.txt b/docs/manual/common-usage.txt
index d9de598..c9d5eb9 100644
--- a/docs/manual/common-usage.txt
+++ b/docs/manual/common-usage.txt
@@ -96,6 +96,8 @@ to +make+ or set in the environment:
   build-time graphs
 * +BR2_GRAPH_OUT+ to set the filetype of generated graphs, either +pdf+ (the
   default), or +png+.
+* +BR2_GRAPH_DEPTH+ (an integer) to limit the depth of the dependency graph.
+  The default, +0+, is to not limit the depth.
 
 An example that uses config files located in the toplevel directory and
 in your $HOME:
-- 
1.8.3.2

[Buildroot] [PATCH 5/6] support/graph-depends: don't show toolchain dependency for all packages

[Yann E. MORIN]

From: "Yann E. MORIN" <yann.morin.1998@free.fr>

Al packages depend on 'toolchain'. Currently, 'graph-depends' graphs this
dependency. The resulting graph is thus cluttered with less-than-useful
information.

Instead, do not graph the 'toolchain' dependency for any package, save
for the fake 'all' package. The graph is now a bit more readable.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 support/scripts/graph-depends | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
index ebf511b..4f532ba 100755
--- a/support/scripts/graph-depends
+++ b/support/scripts/graph-depends
@@ -162,13 +162,16 @@ def has_redundant_deps(deps, pkg):
 def remove_redundant_deps(deps):
     newdeps = []
     for dep in deps:
-        if dep[0] != "all":
+        if dep[0] == "all" and dep[1] == "toolchain":
+            newdeps.append(dep)
+            continue
+        if dep[0] != "all" and dep[1] != "toolchain":
             newdeps.append(dep)
             continue
         if not has_redundant_deps(deps, dep[1]):
             newdeps.append(dep)
             continue
-        sys.stderr.write("Removing redundant dep all -> %s\n" % dep[1])
+        sys.stderr.write("Removing redundant dep %s -> %s\n" % (dep[0],dep[1]))
     return newdeps
 
 TARGET_EXCEPTIONS = [
-- 
1.8.3.2

[Buildroot] [PATCH 6/6] graphs: make graphs with lots of packages nicer to look at

[Yann E. MORIN]

From: "Yann E. MORIN" <yann.morin.1998@free.fr>

Some magic numbers obtained with trial-and-error and successive
iterations, to eventually get a nice graph.

Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
 support/scripts/graph-build-time | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/support/scripts/graph-build-time b/support/scripts/graph-build-time
index 00c2349..59aec5f 100755
--- a/support/scripts/graph-build-time
+++ b/support/scripts/graph-build-time
@@ -126,12 +126,17 @@ def pkg_histogram(data, output, order="build"):
     # Draw the package names
     plt.xticks(ind + .6, [ p.name for p in data ], rotation=-60, rotation_mode="anchor", fontsize=8, ha='left')
 
-    # Adjust size of graph (double the width)
-    sz = plt.gcf().get_size_inches()
-    plt.gcf().set_size_inches(sz[0] * 2, sz[1])
-
-    # Add more space for the package names at the bottom
-    plt.gcf().subplots_adjust(bottom=0.2)
+    # Adjust size of graph depending on the number of packages
+    # Ensure a minimal size twice as the default
+    # Magic Numbers do Magic Layout!
+    ratio = max( ( ( n_pkgs + 10 ) / 48 , 2) )
+    borders = 0.1 / ratio
+    sz = plt.gcf().get_figwidth()
+    plt.gcf().set_figwidth(sz * ratio)
+
+    # Adjust space at borders, add more space for the
+    # package names at the bottom
+    plt.gcf().subplots_adjust(bottom=0.2, left=borders, right=1-borders)
 
     # Remove ticks in the graph for each package
     axes = plt.gcf().gca()
-- 
1.8.3.2

[Buildroot] [PATCH 5/6] support/graph-depends: don't show toolchain dependency for all packages

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Sun, 13 Apr 2014 22:42:41 +0200, Yann E. MORIN wrote:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
> 
> Al packages depend on 'toolchain'. Currently, 'graph-depends' graphs this
> dependency. The resulting graph is thus cluttered with less-than-useful
> information.
> 
> Instead, do not graph the 'toolchain' dependency for any package, save
> for the fake 'all' package. The graph is now a bit more readable.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  support/scripts/graph-depends | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)

Are you sure this works? I applied your patch series, and I still see
all packages having a dependency on the toolchain. The PDF result is
http://free-electrons.com/~thomas/pub/bug-toolchain-graph-depends.pdf.

My defconfig is:

BR2_arm=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-2014.02-rc1.tar.bz2"
BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_13=y
BR2_TOOLCHAIN_EXTERNAL_LARGEFILE=y
BR2_TOOLCHAIN_EXTERNAL_INET_IPV6=y
BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
BR2_TOOLCHAIN_EXTERNAL_INET_RPC=y
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_INIT_NONE=y
# BR2_PACKAGE_BUSYBOX is not set
BR2_PACKAGE_XORG7=y
BR2_PACKAGE_XSERVER_XORG_SERVER=y
BR2_PACKAGE_JQUERY_UI=y
# BR2_TARGET_ROOTFS_TAR is not set

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 5/6] support/graph-depends: don't show toolchain dependency for all packages

[Thomas Petazzoni]

Yann,

On Mon, 14 Apr 2014 21:00:03 +0200, Thomas Petazzoni wrote:

> Are you sure this works? I applied your patch series, and I still see
> all packages having a dependency on the toolchain. The PDF result is
> http://free-electrons.com/~thomas/pub/bug-toolchain-graph-depends.pdf.

Forget it: I screwed up when applying your patches, and only the first
four patches were applied, while it's the fifth patch that was
implementing this feature.

With all your patches applied, it just works.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 1/6] Makefile: rename non-user-facing variable

[Samuel Martin]

On Sun, Apr 13, 2014 at 10:42 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>
> Variables should be prefixed with BR_ when they are not user-facing.
>
> As a side effect, the new variable is prettier than the previous one. :-)
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Acked-by: Samuel Martin <s.martin49@gmail.com

Regards,

-- 
Samuel

[Buildroot] [PATCH 2/6] support/graph-depends: use argparse to parse argv[]

[Samuel Martin]

Hi Yann,

On Sun, Apr 13, 2014 at 10:42 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>
> Currently, we are using a crude, ad-hoc parsing of argv[].
> This is a limiting factor to adding new options.
>
> Use argparse instead, and introduce a single argument for now:
>   --package, -p PACKAGE

Is an option really needed for package? I mean it could be the
remaining argument(s); or maybe it does not fit in developments you
have in mind ;).

>
> In the (near) future, we'll be able to add more option arguments,
> such as depth-limiting for big graphs.
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  package/pkg-generic.mk        |  2 +-
>  support/scripts/graph-depends | 23 +++++++++++++----------
>  2 files changed, 14 insertions(+), 11 deletions(-)
>
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index b3a4c17..080ee56 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -495,7 +495,7 @@ $(1)-show-depends:
>  $(1)-graph-depends:
>                         @$(INSTALL) -d $(O)/graphs
>                         @cd "$(CONFIG_DIR)"; \
> -                       $(TOPDIR)/support/scripts/graph-depends $(1) \
> +                       $(TOPDIR)/support/scripts/graph-depends -p $(1) \
>                         |dot -T$(BR_GRAPH_OUT) -o $(O)/graphs/$$(@).$(BR_GRAPH_OUT)
>
>  $(1)-dirclean:         $$($(2)_TARGET_DIRCLEAN)
> diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
> index ac24086..fc3cadd 100755
> --- a/support/scripts/graph-depends
> +++ b/support/scripts/graph-depends
> @@ -1,13 +1,13 @@
>  #!/usr/bin/python
>
>  # Usage (the graphviz package must be installed in your distribution)
> -#  ./support/scripts/graph-depends [package-name] > test.dot
> +#  ./support/scripts/graph-depends [-p package-name] > test.dot
>  #  dot -Tpdf test.dot -o test.pdf
>  #
>  # With no arguments, graph-depends will draw a complete graph of
> -# dependencies for the current configuration. With an argument,
> -# graph-depends will draw a graph of dependencies for the given
> -# package name.
> +# dependencies for the current configuration.
> +# If '-p <package-name>' is specified, graph-depends will draw a graph
> +# of dependencies for the given package name.
>  #
>  # Limitations
>  #
> @@ -21,6 +21,7 @@
>
>  import sys
>  import subprocess
> +import argparse
>
>  # In FULL_MODE, we draw the full dependency graph for all selected
>  # packages
> @@ -31,14 +32,16 @@ PKG_MODE  = 2
>
>  mode = 0
>
> -if len(sys.argv) == 1:
> +parser = argparse.ArgumentParser(description="Graph pacakges dependencies")
> +parser.add_argument("--package", '-p', metavar="PACKAGE",

Any reason to mixed double and single quotes?

> +                    help="Graph the dependencies of PACKAGE")
> +args = parser.parse_args()
> +
> +if args.package == None:

In Python, None is a singleton, and it is recommended to use "is" or
"is not" for testing them [1].

>      mode = FULL_MODE
> -elif len(sys.argv) == 2:
> -    mode = PKG_MODE
> -    rootpkg  = sys.argv[1]
>  else:
> -    print "Usage: graph-depends [package-name]"
> -    sys.exit(1)
> +    mode = PKG_MODE
> +    rootpkg = args.package
>
>  allpkgs = []
>
> --
> 1.8.3.2
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

Regards,

[1] http://legacy.python.org/dev/peps/pep-0008/#programming-recommendations

-- 
Samuel

[Buildroot] [PATCH 3/6] support/graph-depends: add option to limit the depth of the graph

[Samuel Martin]

Hi Yann,

Same comments as for patch 2/6 ;)

On Sun, Apr 13, 2014 at 10:42 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>
> Currently, the complete dependency chain of a package is used to
> generate the dependency graph. When this dependency chain is long,
> the generated graph becomes almost unreadable.
>
> However, it is often sufficient to get the first few levels of
> dependency of a package.
>
> Add a new variable BR2_GRAPH_DEPTH, that the user can set to limit
> the depth of the dependency list.
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  Makefile                      |  3 ++-
>  package/pkg-generic.mk        |  2 +-
>  support/scripts/graph-depends | 49 ++++++++++++++++++++++++++++++++-----------
>  3 files changed, 40 insertions(+), 14 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index b0eec7f..467e21c 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -150,6 +150,7 @@ endif
>  # Need that early, before we scan packages
>  # Avoids doing the $(or...) everytime
>  BR_GRAPH_OUT := $(or $(BR2_GRAPH_OUT),pdf)
> +BR_GRAPH_DEPTH := $(or $(BR2_GRAPH_DEPTH),0)
>
>  BUILD_DIR:=$(BASE_DIR)/build
>  BINARIES_DIR:=$(BASE_DIR)/images
> @@ -672,7 +673,7 @@ graph-build: $(O)/build/build-time.log
>  graph-depends:
>         @$(INSTALL) -d $(O)/graphs
>         @cd "$(CONFIG_DIR)"; \
> -       $(TOPDIR)/support/scripts/graph-depends \
> +       $(TOPDIR)/support/scripts/graph-depends -d $(BR_GRAPH_DEPTH) \
>         |dot -T$(BR_GRAPH_OUT) -o $(O)/graphs/$(@).$(BR_GRAPH_OUT)
>
>  else # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index 080ee56..c162902 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -495,7 +495,7 @@ $(1)-show-depends:
>  $(1)-graph-depends:
>                         @$(INSTALL) -d $(O)/graphs
>                         @cd "$(CONFIG_DIR)"; \
> -                       $(TOPDIR)/support/scripts/graph-depends -p $(1) \
> +                       $(TOPDIR)/support/scripts/graph-depends -p $(1) -d $(BR_GRAPH_DEPTH) \
>                         |dot -T$(BR_GRAPH_OUT) -o $(O)/graphs/$$(@).$(BR_GRAPH_OUT)
>
>  $(1)-dirclean:         $$($(2)_TARGET_DIRCLEAN)
> diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
> index fc3cadd..ebf511b 100755
> --- a/support/scripts/graph-depends
> +++ b/support/scripts/graph-depends
> @@ -8,6 +8,8 @@
>  # dependencies for the current configuration.
>  # If '-p <package-name>' is specified, graph-depends will draw a graph
>  # of dependencies for the given package name.
> +# If '-d <depth>' is specified, graph-depends will limit the depth of
> +# the dependency graph to 'depth' levels.
>  #
>  # Limitations
>  #
> @@ -31,10 +33,13 @@ FULL_MODE = 1
>  PKG_MODE  = 2
>
>  mode = 0
> +max_depth = 0
>
>  parser = argparse.ArgumentParser(description="Graph pacakges dependencies")
>  parser.add_argument("--package", '-p', metavar="PACKAGE",
>                      help="Graph the dependencies of PACKAGE")
> +parser.add_argument("--depth", '-d', metavar="DEPTH",

Mixed quote style.

> +                    help="Limit the dependency graph to DEPTH levels")
>  args = parser.parse_args()
>
>  if args.package == None:
> @@ -43,6 +48,9 @@ else:
>      mode = PKG_MODE
>      rootpkg = args.package
>
> +if args.depth != None:

s/!=/is not/

> +    max_depth = int(args.depth)
> +
>  allpkgs = []
>
>  # Execute the "make show-targets" command to get the list of the main
> @@ -193,6 +201,7 @@ if mode == FULL_MODE:
>      deps = get_all_depends(filtered_targets)
>      if deps != None:
>          dependencies += deps
> +    rootpkg = 'all'
>
>  # In pkg mode, start directly with get_all_depends() on the requested
>  # package
> @@ -201,26 +210,42 @@ elif mode == PKG_MODE:
>
>  dependencies = remove_redundant_deps(dependencies)
>
> -# Start printing the graph data
> -print "digraph G {"
> -
> -# First, the dependencies. Usage of set allows to remove duplicated
> -# dependencies in the graph
> -for dep in set(dependencies):
> -    print "%s -> %s" % (pkg_node_name(dep[0]), pkg_node_name(dep[1]))
> +# Make the dependencies a dictionnary { 'pkg':[dep1, dep2, ...] }
> +dict_deps = {}
> +for dep in dependencies:
> +    if not dict_deps.has_key(dep[0]):
> +        dict_deps[dep[0]] = []
> +    dict_deps[dep[0]].append(dep[1])
>
> -# Then, the node attributes: color, style and label.
> -for pkg in allpkgs:
> +# Print the attributes of a node: label and fill-color
> +def print_attrs(pkg):
>      if pkg == 'all':
>          print "all [label = \"ALL\"]"
>          print "all [color=lightblue,style=filled]"
> -        continue
> -
> +        return
>      print "%s [label = \"%s\"]" % (pkg_node_name(pkg), pkg)
> -
>      if mode == PKG_MODE and pkg == rootpkg:
>          print "%s [color=lightblue,style=filled]" % pkg_node_name(rootpkg)
>      else:
>          print "%s [color=grey,style=filled]" % pkg_node_name(pkg)
>
> +# Print the dependency graph of a package
> +def print_pkg_deps(depth, pkg):
> +    if pkg in done_deps:
> +        return
> +    done_deps.append(pkg)
> +    print_attrs(pkg)
> +    if not dict_deps.has_key(pkg):
> +        return
> +    if max_depth == 0 or depth < max_depth:
> +        for d in dict_deps[pkg]:
> +            print "%s -> %s" % (pkg_node_name(pkg), pkg_node_name(d))
> +            print_pkg_deps(depth+1, d)
> +
> +# Start printing the graph data
> +print "digraph G {"
> +
> +done_deps = []
> +print_pkg_deps(0, rootpkg)
> +
>  print "}"
> --
> 1.8.3.2
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

Regards,

-- 
Samuel

[Buildroot] [PATCH 4/6] manual: document BR2_GRAPH_DEPTH

[Samuel Martin]

On Sun, Apr 13, 2014 at 10:42 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Acked-by: Samuel Martin <s.martin49@gmail.com>

> ---
>  docs/manual/common-usage.txt | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/docs/manual/common-usage.txt b/docs/manual/common-usage.txt
> index d9de598..c9d5eb9 100644
> --- a/docs/manual/common-usage.txt
> +++ b/docs/manual/common-usage.txt
> @@ -96,6 +96,8 @@ to +make+ or set in the environment:
>    build-time graphs
>  * +BR2_GRAPH_OUT+ to set the filetype of generated graphs, either +pdf+ (the
>    default), or +png+.
> +* +BR2_GRAPH_DEPTH+ (an integer) to limit the depth of the dependency graph.
> +  The default, +0+, is to not limit the depth.
>
>  An example that uses config files located in the toplevel directory and
>  in your $HOME:
> --
> 1.8.3.2
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

Regards,

-- 
Samuel

[Buildroot] [PATCH 5/6] support/graph-depends: don't show toolchain dependency for all packages

[Samuel Martin]

Hi Yann,

On Sun, Apr 13, 2014 at 10:42 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>
> Al packages depend on 'toolchain'. Currently, 'graph-depends' graphs this
s/Al/All/

> dependency. The resulting graph is thus cluttered with less-than-useful
> information.
>
> Instead, do not graph the 'toolchain' dependency for any package, save
> for the fake 'all' package. The graph is now a bit more readable.
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  support/scripts/graph-depends | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
> index ebf511b..4f532ba 100755
> --- a/support/scripts/graph-depends
> +++ b/support/scripts/graph-depends
> @@ -162,13 +162,16 @@ def has_redundant_deps(deps, pkg):
>  def remove_redundant_deps(deps):
>      newdeps = []
>      for dep in deps:
> -        if dep[0] != "all":
> +        if dep[0] == "all" and dep[1] == "toolchain":
> +            newdeps.append(dep)
> +            continue
> +        if dep[0] != "all" and dep[1] != "toolchain":
>              newdeps.append(dep)
>              continue
>          if not has_redundant_deps(deps, dep[1]):
>              newdeps.append(dep)
>              continue
> -        sys.stderr.write("Removing redundant dep all -> %s\n" % dep[1])
> +        sys.stderr.write("Removing redundant dep %s -> %s\n" % (dep[0],dep[1]))
>      return newdeps
>
>  TARGET_EXCEPTIONS = [
> --
> 1.8.3.2
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

In this cset, I don't see anything wrong.
However, I'm not sure to well understand this function and the
subtleties of these if conditions. I'll need to spend more time on
this ;)

Regards,

-- 
Samuel

[Buildroot] [PATCH 2/6] support/graph-depends: use argparse to parse argv[]

[Yann E. MORIN]

Samuel, All,

On 2014-04-14 21:20 +0200, Samuel Martin spake thusly:
> On Sun, Apr 13, 2014 at 10:42 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > From: "Yann E. MORIN" <yann.morin.1998@free.fr>
> >
> > Currently, we are using a crude, ad-hoc parsing of argv[].
> > This is a limiting factor to adding new options.
> >
> > Use argparse instead, and introduce a single argument for now:
> >   --package, -p PACKAGE
> 
> Is an option really needed for package? I mean it could be the
> remaining argument(s); or maybe it does not fit in developments you
> have in mind ;).

It's just that I don't know how to do that with Python's argparse.
I just duplicated the code in:
    support/scripts/graph-nuild-time

[--SNIP--]
> > diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
> > index ac24086..fc3cadd 100755
> > --- a/support/scripts/graph-depends
> > +++ b/support/scripts/graph-depends
> > @@ -31,14 +32,16 @@ PKG_MODE  = 2
> >
> >  mode = 0
> >
> > -if len(sys.argv) == 1:
> > +parser = argparse.ArgumentParser(description="Graph pacakges dependencies")
> > +parser.add_argument("--package", '-p', metavar="PACKAGE",
> 
> Any reason to mixed double and single quotes?

Again, I just duplicated what was already in:
    support/scripts/graph-build-time

> > +                    help="Graph the dependencies of PACKAGE")
> > +args = parser.parse_args()
> > +
> > +if args.package == None:
> 
> In Python, None is a singleton, and it is recommended to use "is" or
> "is not" for testing them [1].

OK, I'll send an follow-up patch to fix that.

Thanks!

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 3/6] support/graph-depends: add option to limit the depth of the graph

[Yann E. MORIN]

Samuel, All,

On 2014-04-14 21:29 +0200, Samuel Martin spake thusly:
> Same comments as for patch 2/6 ;)

Same answers! ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 0/6] Misc graphs enhancements (branch yem/graphs)

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Sun, 13 Apr 2014 22:42:37 +0200, Yann E. MORIN wrote:

> Yann E. MORIN (6):
>       Makefile: rename non-user-facing variable
>       support/graph-depends: use argparse to parse argv[]
>       support/graph-depends: add option to limit the depth of the graph
>       manual: document BR2_GRAPH_DEPTH
>       support/graph-depends: don't show toolchain dependency for all packages
>       graphs: make graphs with lots of packages nicer to look at

Applied, thanks. In the last patch, I've removed excessive spaces in
the expression:

	max( ( ( n_pkgs + 10 ) / 48 , 2) )

If you have other fixes to do (such as things suggested by Samuel),
please send followup patches as appropriate.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 0/6] Misc graphs enhancements (branch yem/graphs)

[Yann E. MORIN]

Thomas, All,

On 2014-04-14 22:15 +0200, Thomas Petazzoni spake thusly:
> On Sun, 13 Apr 2014 22:42:37 +0200, Yann E. MORIN wrote:
> 
> > Yann E. MORIN (6):
> >       Makefile: rename non-user-facing variable
> >       support/graph-depends: use argparse to parse argv[]
> >       support/graph-depends: add option to limit the depth of the graph
> >       manual: document BR2_GRAPH_DEPTH
> >       support/graph-depends: don't show toolchain dependency for all packages
> >       graphs: make graphs with lots of packages nicer to look at
> 
> Applied, thanks. In the last patch, I've removed excessive spaces in
> the expression:
> 
> 	max( ( ( n_pkgs + 10 ) / 48 , 2) )

Yes, that's my default formatting style, which I use since I find it
more readable. I forgot to re-hash it to Buildroot's style.

Thanks for fixing.

> If you have other fixes to do (such as things suggested by Samuel),
> please send followup patches as appropriate.

Yep, doing so right now! :-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'