[PATCH] opkg-utils: Add opkg-graph-deps

* [PATCH] opkg-utils: Add opkg-graph-deps
@ 2016-01-04 23:45 Haris Okanovic
  2016-01-07  0:01 ` Alejandro del Castillo
  0 siblings, 1 reply; 2+ messages in thread
From: Haris Okanovic @ 2016-01-04 23:45 UTC (permalink / raw)
  To: yocto; +Cc: Haris Okanovic, Ken Sharp

Usage: opkg-graph-deps [-h] [-o feed.dot] [-u <base feed URL>] <paths to
Packages>

Generates a dot formatted dependency graph of an IPK feed.

The feed is specified by a list of IPK index (Packages) files, which
are sourced in the order specified to build a dependency graph. Last
index to declare a package wins, but also generates a warning to stderr.

Possible warnings:
 Duplicate package: package appears in more than one index.
 Broken dependency: no package satisfies a declared dependency.
 Replacing alias: package is replacing another package.
 Self alias: package declares an alias on it's own name.

If a base feed URL is specified, each package node includes an 'href'
to the associated IPK file. It's assumes that the specified base
feed URL hosts the current working directory, so the resulting
hrefs are generated by joining the base and a relative IPK path.

The resulting feed graph is written to './feed.dot' or an alternate
path specified by the caller. Nodes represent real packages (not
aliases)
and edges represent dependencies.

Node attributes:
 (node name): Package name from feed index (without version or arch)
 label: [Package name] [ipkArchitecture] [ipkVersion]
 ipkArchitecture: Architecture name from feed index
 ipkVersion: The full version number from feed index
 ipkMissing: Set to "1" when the ipk is not actually in feed, but has
  one or inbound dependencies.
 href: URL to the IPK file. Only if optional base URL is specified.

Edge attributes:
 (from) The package name declaring a dependency
 (to) The (de-aliased) package name (from) depends on
 ipkAlias: The alias of (to) which (from) depends on. Only set when
  the alias != (to).
 ipkBrokenDep: Set to "1" if (to) is missing from the feed.

Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Cc: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Cc: Paul Barker <paul@paulbarker.me.uk>
Cc: Ken Sharp <ken.sharp@ni.com>
Cc: Richard Tollerton <rich.tollerton@ni.com>
---
 opkg-graph-deps | 246 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 246 insertions(+)
 create mode 100755 opkg-graph-deps

diff --git a/opkg-graph-deps b/opkg-graph-deps
new file mode 100755
index 0000000..c42e7ce
--- /dev/null
+++ b/opkg-graph-deps
@@ -0,0 +1,246 @@
+#!/usr/bin/env python
+
+import sys
+import os
+import getopt
+import pydot
+import opkg
+
+def usage(more=False):
+    print >>sys.stderr, ( 'Usage: opkg-graph-deps '
+        '[-h] [-o feed.dot] '
+        '[-u <base feed URL>] '
+        '<paths to Packages>' )
+    if more:
+        print >>sys.stderr, '\n'.join( [
+'',
+'Generates a dot formatted dependency graph of an IPK feed.',
+'',
+'The feed is specified by a list of IPK index (Packages) files, which',
+'are sourced in the order specified to build a dependency graph. Last',
+'index to declare a package wins, but also generates a warning to stderr.',
+'',
+'Possible warnings:',
+' Duplicate package: package appears in more than one index.',
+' Broken dependency: no package satisfies a declared dependency.',
+' Replacing alias: package is replacing another package.',
+' Self alias: package declares an alias on it\'s own name.',
+'',
+'If a base feed URL is specified, each package node includes an \'href\'',
+'to the associated IPK file. It\'s assumes that the specified base',
+'feed URL hosts the current working directory, so the resulting',
+'hrefs are generated by joining the base and a relative IPK path.',
+'',
+'The resulting feed graph is written to \'./feed.dot\' or an alternate',
+'path specified by the caller. Nodes represent real packages (not aliases)',
+'and edges represent dependencies.',
+'',
+'Node attributes:',
+' (node name): Package name from feed index (without version or arch)',
+' label: [Package name] [ipkArchitecture] [ipkVersion]',
+' ipkArchitecture: Architecture name from feed index',
+' ipkVersion: The full version number from feed index',
+' ipkMissing: Set to "1" when the ipk is not actually in feed, but has',
+'  one or inbound dependencies.',
+' href: URL to the IPK file. Only if optional base URL is specified.',
+'',
+'Edge attributes:',
+' (from) The package name declaring a dependency',
+' (to) The (de-aliased) package name (from) depends on',
+' ipkAlias: The alias of (to) which (from) depends on. Only set when',
+'  the alias != (to).',
+' ipkBrokenDep: Set to "1" if (to) is missing from the feed.',
+'',
+        ] )
+    exit(1)
+
+# optional args
+dot_filename = "feed.dot"
+feed_url = None
+
+(opts, index_files) = getopt.getopt(sys.argv[1:], "ho:u:")
+for (optkey, optval) in opts:
+    if optkey == '-h': 
+        usage(more=True)
+    elif optkey == '-o': 
+        dot_filename = optval
+    elif optkey == '-u':
+        feed_url = optval
+
+if not index_files:
+    print >>sys.stderr, 'Must specify a path to at least one Packages file'
+    usage()
+
+def split_dep_list(lst):
+    '''
+    Splits a comma-space delimited list, retuning only the first item.
+    E.g. 'foo (>= 1.2), bar, lab (x)' yields ['foo', 'bar', 'lab']
+    '''
+    if not lst:
+        lst = ''
+
+    res = []
+
+    splitLst = lst.split(',')
+    for itm in splitLst:
+        itm = itm.strip()
+        if not itm:
+            continue
+        itmSplit = itm.split()
+        res.append(itmSplit[0])
+
+    return res
+
+# define the graph
+graph = pydot.Dot(graph_name='ipkFeed', graph_type='digraph')
+graph.set_node_defaults(shape='rectangle', style='solid', color='black')
+graph.set_edge_defaults(style='solid', color='black')
+
+def pkg_architectcture(pkg):
+    return str(pkg.architecture or '?')
+
+def pkg_label(pkg, includeArch=True, includeVersion=False, includePath=False, multiLine=False):
+    label = str(pkg.package or '?')
+    if multiLine:
+        label += '\\n'
+    if includeArch:
+        label += '[%s]' % pkg_architectcture(pkg)
+    if includeVersion:
+        label += '[%s]' % (pkg.version or '?')
+    if includePath:
+        label += '[%s]' % (pkg.fn or '?')
+    return label
+
+def add_package_to_graph(pkg, missing=False):
+    if not pkg.package:
+        raise Exception('Invalid package name')
+
+    node = pydot.Node(pkg.package)
+
+    node.set('label', pkg_label(pkg,
+        includeVersion=(not missing),
+        includeArch=(not missing),
+        multiLine=True) )
+
+    if missing:
+        node.set('ipkMissing', '1')
+        node.set('style', 'dotted')
+        node.set('color', 'red')
+
+    node.set('ipkVersion', pkg.version or 'none')
+    node.set('ipkArchitecture', pkg_architectcture(pkg))
+
+    if feed_url and pkg.filename:
+        node.set('href', '%s/%s' % (feed_url, pkg.fn) )
+
+    graph.add_node(node)
+
+def add_dependency_to_graph(fromPkg, toPkg, alias=None, broken=False):
+        edge = pydot.Edge(fromPkg.package, toPkg.package)
+
+        if alias:
+            edge.set('ipkAlias', alias)
+            edge.set('style', 'dashed')
+
+        if broken:
+            edge.set('ipkBrokenDep', '1')
+            edge.set('style', 'dotted')
+            edge.set('color', 'red')
+
+        graph.add_edge(edge)
+
+# the feed
+pkg_map = {}
+alias_pkg_map = {}
+missing_pkg_map = {}
+real_pkg_replace_count = 0
+broken_dep_count = 0
+
+# populate pkg_map with all real packages defined in the indexes
+#  do this first for all indexes before updating alias_pkg_map and
+#  adding nodes to apply any replacements
+for indexFilePath in index_files:
+    feedDir = os.path.dirname(indexFilePath)
+    feedDir = os.path.relpath(feedDir, start=os.getcwd())
+
+    packages = opkg.Packages()
+    packages.read_packages_file(indexFilePath)
+
+    # add each package
+    for pkgKey in packages.keys():
+        pkg = packages[pkgKey]
+
+        # save package filename relative to sub-feed dir
+        pkg.fn = os.path.join(feedDir, pkg.filename)
+
+        if pkg.package in pkg_map:
+            # pkg is being replaced
+            replacedPkg = pkg_map[pkg.package]
+
+            real_pkg_replace_count = real_pkg_replace_count + 1
+            print >>sys.stderr, "Duplicate package: Replacing %s with %s" % (
+                pkg_label(replacedPkg, includePath=True),
+                pkg_label(pkg, includePath=True) )
+
+        pkg_map[pkg.package] = pkg
+
+# populate alias_pkg_map with all real+alias packages defined in the indexes
+# add nodes to graph
+for pkgKey, pkg in pkg_map.iteritems():
+    # Add the real package
+    alias_pkg_map[pkg.package] = pkg
+
+    # Add any aliases
+    for alias in split_dep_list(pkg.provides):
+        if alias in alias_pkg_map:
+            oldAliasPkg = alias_pkg_map[alias]
+
+            if pkg == oldAliasPkg:
+                # weird, not an error, but worth documenting
+                print >>sys.stderr, "Self alias: %s" % pkg_label(pkg)
+            else:
+                print >>sys.stderr, "Replacing alias: %s (%s) with %s" % (
+                    alias, pkg_label(oldAliasPkg), pkg_label(pkg) )
+
+        alias_pkg_map[alias] = pkg
+
+    add_package_to_graph(pkg)
+
+# create stub packages in alias_pkg_map for broken deps
+# add them to missing_pkg_map
+# add nodes to graph
+for pkgKey, pkg in pkg_map.iteritems():
+    for depName in split_dep_list(pkg.depends):
+        if not depName in alias_pkg_map:
+            broken_dep_count = broken_dep_count + 1
+            print >>sys.stderr, "Broken dependency: %s --> %s (missing)" % (
+                pkg_label(pkg), depName )
+
+            stub = opkg.Package()
+            stub.package = depName
+
+            # don't update pkg_map, stub is not a real package
+            alias_pkg_map[stub.package] = stub
+            missing_pkg_map[stub.package] = stub
+
+            add_package_to_graph(stub, missing=True)
+
+# process dependencies
+# add edges to graph
+for pkgKey, pkg in pkg_map.iteritems():
+    for depName in split_dep_list(pkg.depends):
+        depPkg = alias_pkg_map[depName]
+
+        add_dependency_to_graph(pkg, depPkg,
+            alias=(depName if (depName != depPkg.package) else None),
+            broken=(depPkg.package in missing_pkg_map) )
+
+# Results
+print "%s packages" % len(pkg_map.keys())
+print "%s aliases" % len(alias_pkg_map)
+print "%s replaced packages" % real_pkg_replace_count
+print "%s broken dependencies" % broken_dep_count
+
+# Write the graph
+graph.write(path=dot_filename)
+print "Graphed at %s" % dot_filename
-- 
2.6.2



^ permalink raw reply related	[flat|nested] 2+ messages in thread