[Buildroot] [V2 1/2] graph-depends: display virtual package with italic style

[Buildroot] [V2 1/2] graph-depends: display virtual package with italic style

[Francois Perrad]

virtual packages are found by their version,
so we retrieve the version of all packages

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
---
 package/pkg-generic.mk        |  3 +++
 support/scripts/graph-depends | 30 +++++++++++++++++++++++++++++-
 2 files changed, 32 insertions(+), 1 deletion(-)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 9643a30..87d8dd8 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -565,6 +565,9 @@ $(1)-rsync:		$$($(2)_TARGET_RSYNC)
 $(1)-source:		$$($(2)_TARGET_RSYNC_SOURCE)
 endif
 
+$(1)-show-version:
+			@echo $$($(2)_VERSION)
+
 $(1)-show-depends:
 			@echo $$($(2)_FINAL_DEPENDENCIES)
 
diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
index 1ecfeda..28fe2f0 100755
--- a/support/scripts/graph-depends
+++ b/support/scripts/graph-depends
@@ -76,6 +76,28 @@ host_colour = colours[2]
 
 allpkgs = []
 
+# Execute the "make <pkg>-show-version" command to get the version of a given
+# list of packages, and return the version formatted as a Python dictionary.
+def get_version(pkgs):
+    sys.stderr.write("Getting version for %s\n" % pkgs)
+    cmd = ["make", "-s", "--no-print-directory" ]
+    for pkg in pkgs:
+        cmd.append("%s-show-version" % pkg)
+    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
+    output = p.communicate()[0]
+    if p.returncode != 0:
+        sys.stderr.write("Error getting version %s\n" % pkgs)
+        sys.exit(1)
+    output = output.split("\n")
+    if len(output) != len(pkgs) + 1:
+        sys.stderr.write("Error getting version\n")
+        sys.exit(1)
+    version = {}
+    for i in range(0, len(pkgs)):
+        pkg = pkgs[i]
+        version[pkg] = output[i]
+    return version
+
 # Execute the "make show-targets" command to get the list of the main
 # Buildroot TARGETS and return it formatted as a Python list. This
 # list is used as the starting point for full dependency graphs
@@ -257,6 +279,8 @@ def remove_extra_deps(deps):
     return deps
 
 dict_deps = remove_extra_deps(dict_deps)
+dict_version = get_version([pkg for pkg in allpkgs
+                                if pkg != "all" and not pkg.startswith("root")])
 
 # Print the attributes of a node: label and fill-color
 def print_attrs(pkg):
@@ -274,7 +298,11 @@ def print_attrs(pkg):
             color = host_colour
         else:
             color = target_colour
-    print("%s [label = \"%s\"]" % (name, label))
+    version = dict_version.get(pkg)
+    if version == "virtual":
+        print("%s [label = <<I>%s</I>>]" % (name, label))
+    else:
+        print("%s [label = \"%s\"]" % (name, label))
     print("%s [color=%s,style=filled]" % (name, color))
 
 # Print the dependency graph of a package
-- 
2.1.0

[Buildroot] [V2 2/2] graph-depends: add an option --stop-on-virtual

[Francois Perrad]

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
---
 docs/manual/common-usage.txt  |  3 +++
 support/scripts/graph-depends | 11 +++++++++++
 2 files changed, 14 insertions(+)

diff --git a/docs/manual/common-usage.txt b/docs/manual/common-usage.txt
index 89cd9fe..966c694 100644
--- a/docs/manual/common-usage.txt
+++ b/docs/manual/common-usage.txt
@@ -215,6 +215,9 @@ The +graph-depends+ behaviour can be controlled by setting options in the
   root package (+R+), the target packages (+T+) and the host packages
   (+H+). Defaults to: +lightblue,grey,gainsboro+
 
+* +--stop-on-virtual+, +--dont-stop-on-virtual+, to follow (or not)
+  the dependencies of virtual package
+
 --------------------------------
 BR2_GRAPH_DEPS_OPTS='-d 3 --no-transitive --colours=red,green,blue' make graph-depends
 --------------------------------
diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
index 28fe2f0..4e12351 100755
--- a/support/scripts/graph-depends
+++ b/support/scripts/graph-depends
@@ -36,6 +36,9 @@ max_depth = 0
 # Whether to draw the transitive dependencies
 transitive = True
 
+# Stop dependency on virtual package
+stop_on_virtual = False
+
 parser = argparse.ArgumentParser(description="Graph pacakges dependencies")
 parser.add_argument("--package", '-p', metavar="PACKAGE",
                     help="Graph the dependencies of PACKAGE")
@@ -51,6 +54,10 @@ parser.add_argument("--transitive", dest="transitive", action='store_true',
                     default=False)
 parser.add_argument("--no-transitive", dest="transitive", action='store_false',
                     help="Draw (do not draw) transitive dependencies")
+parser.add_argument("--stop-on-virtual", dest="stop_on_virtual", action='store_true',
+                    default=False)
+parser.add_argument("--dont-stop-on-virtual", dest="stop_on_virtual", action='store_false',
+                    help="Stop (or do not stop) on virtual package")
 args = parser.parse_args()
 
 if args.package is None:
@@ -63,6 +70,8 @@ max_depth = args.depth
 
 transitive = args.transitive
 
+stop_on_virtual = args.stop_on_virtual
+
 # Get the colours: we need exactly three colours,
 # so no need not split more than 4
 # We'll let 'dot' validate the colours...
@@ -313,6 +322,8 @@ def print_pkg_deps(depth, pkg):
     print_attrs(pkg)
     if pkg not in dict_deps:
         return
+    if stop_on_virtual and dict_version.get(pkg) == "virtual":
+        return
     if max_depth == 0 or depth < max_depth:
         for d in dict_deps[pkg]:
             print("%s -> %s" % (pkg_node_name(pkg), pkg_node_name(d)))
-- 
2.1.0

[Buildroot] [V2 1/2] graph-depends: display virtual package with italic style

[Yann E. MORIN]

Fran?ois, All,

On 2015-01-03 15:29 +0100, Francois Perrad spake thusly:
> virtual packages are found by their version,
> so we retrieve the version of all packages
> 
> Signed-off-by: Francois Perrad <francois.perrad@gadz.org>

Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Regards,
Yann E. MORIN.

> ---
>  package/pkg-generic.mk        |  3 +++
>  support/scripts/graph-depends | 30 +++++++++++++++++++++++++++++-
>  2 files changed, 32 insertions(+), 1 deletion(-)
> 
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index 9643a30..87d8dd8 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -565,6 +565,9 @@ $(1)-rsync:		$$($(2)_TARGET_RSYNC)
>  $(1)-source:		$$($(2)_TARGET_RSYNC_SOURCE)
>  endif
>  
> +$(1)-show-version:
> +			@echo $$($(2)_VERSION)
> +
>  $(1)-show-depends:
>  			@echo $$($(2)_FINAL_DEPENDENCIES)
>  
> diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
> index 1ecfeda..28fe2f0 100755
> --- a/support/scripts/graph-depends
> +++ b/support/scripts/graph-depends
> @@ -76,6 +76,28 @@ host_colour = colours[2]
>  
>  allpkgs = []
>  
> +# Execute the "make <pkg>-show-version" command to get the version of a given
> +# list of packages, and return the version formatted as a Python dictionary.
> +def get_version(pkgs):
> +    sys.stderr.write("Getting version for %s\n" % pkgs)
> +    cmd = ["make", "-s", "--no-print-directory" ]
> +    for pkg in pkgs:
> +        cmd.append("%s-show-version" % pkg)
> +    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
> +    output = p.communicate()[0]
> +    if p.returncode != 0:
> +        sys.stderr.write("Error getting version %s\n" % pkgs)
> +        sys.exit(1)
> +    output = output.split("\n")
> +    if len(output) != len(pkgs) + 1:
> +        sys.stderr.write("Error getting version\n")
> +        sys.exit(1)
> +    version = {}
> +    for i in range(0, len(pkgs)):
> +        pkg = pkgs[i]
> +        version[pkg] = output[i]
> +    return version
> +
>  # Execute the "make show-targets" command to get the list of the main
>  # Buildroot TARGETS and return it formatted as a Python list. This
>  # list is used as the starting point for full dependency graphs
> @@ -257,6 +279,8 @@ def remove_extra_deps(deps):
>      return deps
>  
>  dict_deps = remove_extra_deps(dict_deps)
> +dict_version = get_version([pkg for pkg in allpkgs
> +                                if pkg != "all" and not pkg.startswith("root")])
>  
>  # Print the attributes of a node: label and fill-color
>  def print_attrs(pkg):
> @@ -274,7 +298,11 @@ def print_attrs(pkg):
>              color = host_colour
>          else:
>              color = target_colour
> -    print("%s [label = \"%s\"]" % (name, label))
> +    version = dict_version.get(pkg)
> +    if version == "virtual":
> +        print("%s [label = <<I>%s</I>>]" % (name, label))
> +    else:
> +        print("%s [label = \"%s\"]" % (name, label))
>      print("%s [color=%s,style=filled]" % (name, color))
>  
>  # Print the dependency graph of a package
> -- 
> 2.1.0
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [V2 1/2] graph-depends: display virtual package with italic style

[Luca Ceresoli]

Dear Francois,

Francois Perrad wrote:
> virtual packages are found by their version,
> so we retrieve the version of all packages
>
> Signed-off-by: Francois Perrad <francois.perrad@gadz.org>

Tested-by: Luca Ceresoli <luca@lucaceresoli.net>

-- 
Luca

[Buildroot] [V2 1/2] graph-depends: display virtual package with italic style

[Thomas Petazzoni]

Dear Francois Perrad,

On Sat,  3 Jan 2015 15:29:12 +0100, Francois Perrad wrote:
> virtual packages are found by their version,
> so we retrieve the version of all packages
> 
> Signed-off-by: Francois Perrad <francois.perrad@gadz.org>

Applied, thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [V2 2/2] graph-depends: add an option --stop-on-virtual

[Thomas Petazzoni]

Dear Francois Perrad,

On Sat,  3 Jan 2015 15:29:13 +0100, Francois Perrad wrote:
> Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
> ---
>  docs/manual/common-usage.txt  |  3 +++
>  support/scripts/graph-depends | 11 +++++++++++
>  2 files changed, 14 insertions(+)

I'm still not convinced by this one. Shouldn't we instead have a more
generic mechanism such as --exclude=<foo>,<bar>, which would allow to
exclude packages? And maybe one of the possible values would be a
magical value to exclude virtual packages?

There are some cases where ignoring host-pkgconf, or
host-automake/host-autoconf/host-libtool may be interesting, for
example.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [V2 2/2] graph-depends: add an option --stop-on-virtual

[François Perrad]

2015-03-08 22:18 GMT+01:00 Thomas Petazzoni
<thomas.petazzoni@free-electrons.com>:
> Dear Francois Perrad,
>
> On Sat,  3 Jan 2015 15:29:13 +0100, Francois Perrad wrote:
>> Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
>> ---
>>  docs/manual/common-usage.txt  |  3 +++
>>  support/scripts/graph-depends | 11 +++++++++++
>>  2 files changed, 14 insertions(+)
>
> I'm still not convinced by this one. Shouldn't we instead have a more
> generic mechanism such as --exclude=<foo>,<bar>, which would allow to
> exclude packages? And maybe one of the possible values would be a
> magical value to exclude virtual packages?
>
> There are some cases where ignoring host-pkgconf, or
> host-automake/host-autoconf/host-libtool may be interesting, for
> example.
>

I agree with you.
The graphs become quickly too large.
The current option --depth seems to be only a workaround against an
infinite recursion when the graph has a cycle.
But at this time, we haven't found the good way (or the good use
cases) to limit the size of the graph.

Fran?ois

> Best regards,
>
> Thomas
> --
> Thomas Petazzoni, CTO, Free Electrons
> Embedded Linux, Kernel and Android engineering
> http://free-electrons.com
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] [V2 2/2] graph-depends: add an option --stop-on-virtual

[Thomas Petazzoni]

Dear Fran?ois Perrad,

On Mon, 9 Mar 2015 20:32:16 +0100, Fran?ois Perrad wrote:

> I agree with you.
> The graphs become quickly too large.
> The current option --depth seems to be only a workaround against an
> infinite recursion when the graph has a cycle.

Well, I don't really think --depth is meant to avoid infinite
recursion: I don't think it's possible to have infinite recursion since
we can't have cyclic dependencies in Buildroot.

> But at this time, we haven't found the good way (or the good use
> cases) to limit the size of the graph.

I believe being able to exclude certain packages (and their dependency
tree) would be useful.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [V2 2/2] graph-depends: add an option --stop-on-virtual

[Yann E. MORIN]

Thomas, Fran?ois, All,

On 2015-03-09 21:15 +0100, Thomas Petazzoni spake thusly:
> On Mon, 9 Mar 2015 20:32:16 +0100, Fran?ois Perrad wrote:
> > The current option --depth seems to be only a workaround against an
> > infinite recursion when the graph has a cycle.
> 
> Well, I don't really think --depth is meant to avoid infinite
> recursion: I don't think it's possible to have infinite recursion since
> we can't have cyclic dependencies in Buildroot.

No, -depth was never meant to be a stop-gap for recursion: we can *not*
have recursive depedencies.

What I introduced --depth for, is because often only the first few level
of dependencies of a given package are of interest. --depth is
essentially meant for use when graphing the dependencies of a single
package, like so:

    BR2_GRAPH_DEPS_OPTS='--depth 2' make foo-graph-depends

that would limit graphinh the dependencies of 'foo' down to two levels.

> > But at this time, we haven't found the good way (or the good use
> > cases) to limit the size of the graph.
> 
> I believe being able to exclude certain packages (and their dependency
> tree) would be useful.

What about something like:

    BR2_GRAPH_DEPS_OPTS='--stop-on PKGS' make graph-depends

where 'PKGS' would be a comma-separated list of packages, possibly a
glob or regexp, or even the keyword 'virtual' to stop on virtual
packages?

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [V2 2/2] graph-depends: add an option --stop-on-virtual

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Sat, 14 Mar 2015 17:26:57 +0100, Yann E. MORIN wrote:

> What about something like:
> 
>     BR2_GRAPH_DEPS_OPTS='--stop-on PKGS' make graph-depends
> 
> where 'PKGS' would be a comma-separated list of packages, possibly a
> glob or regexp, or even the keyword 'virtual' to stop on virtual
> packages?

Sounds like a good idea to me.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com