All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH 01/14] support/scripts/pkg-stats: URL checking support
@ 2018-09-20 13:24 Matt Weber
  2018-09-20 13:24 ` [Buildroot] [PATCH 02/14] boot/at91bootstrap: URL update Matt Weber
                   ` (13 more replies)
  0 siblings, 14 replies; 44+ messages in thread
From: Matt Weber @ 2018-09-20 13:24 UTC (permalink / raw)
  To: buildroot

 - Adds support to check if a package has a URL and if that URL
   is valid by doing a header request.
 - Reports this information as part of the generated html output

This check helps ensure the URLs are valid and can be used
for other scripting purposes as the product's home site/URL.
CPE XML generation is an example of a case that could use this
product URL as part of an automated update generation script.

Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
---
 support/scripts/pkg-stats | 37 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 36 insertions(+), 1 deletion(-)

diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats
index b7b00e8..c83545b 100755
--- a/support/scripts/pkg-stats
+++ b/support/scripts/pkg-stats
@@ -24,6 +24,8 @@ from collections import defaultdict
 import re
 import subprocess
 import sys
+import time
+import requests  # URL checking
 
 INFRA_RE = re.compile("\$\(eval \$\(([a-z-]*)-package\)\)")
 
@@ -43,10 +45,30 @@ class Package:
         self.patch_count = 0
         self.warnings = 0
         self.current_version = None
+        self.url = None
 
     def pkgvar(self):
         return self.name.upper().replace("-", "_")
 
+    def set_url(self):
+        """
+        Fills in the .url field
+        """
+        in_help_section = False
+        self.url = "No Config"
+        for filename in os.listdir(os.path.dirname(self.path)):
+            if fnmatch.fnmatch(filename, 'Config.*'):
+                fp = open(os.path.join(os.path.dirname(self.path), filename), "r")
+                for config_line in fp:
+                    if config_line.strip() == "help":
+                        in_help_section = True
+                    if in_help_section and re.match("(.*)https?://", config_line):
+                        self.url = ''.join(config_line.split())
+                        fp.close()
+                        return
+                self.url = "Missing Entry"
+                fp.close()
+
     def set_infra(self):
         """
         Fills in the .infras field
@@ -356,7 +378,19 @@ def boolean_str(b):
 
 def dump_html_pkg(f, pkg):
     f.write(" <tr>\n")
-    f.write("  <td>%s</td>\n" % pkg.path[2:])
+    url_status = "Ok"
+    if str(pkg.url) == "Missing Entry":
+        f.write("  <td>%s<br>    (URL: Missing URL)</td>\n" % pkg.path[2:])
+    elif str(pkg.url) == "No Config":
+        f.write("  <td>%s<br>    (URL: No Config File)</td>\n" % pkg.path[2:])
+    else:
+        try:
+            url_status_code = requests.head(pkg.url, verify=False).status_code
+            if url_status_code > 308:
+                url_status = "Error(" + str(url_status_code) + ")"
+        except requests.exceptions.RequestException as e:
+            url_status = e
+        f.write("  <td>%s<br>    (<a href=%s>URL: %s</a>)</td>\n" % (pkg.path[2:], str(pkg.url), url_status))
 
     # Patch count
     td_class = ["centered"]
@@ -511,6 +545,7 @@ def __main__():
     package_init_make_info()
     print("Getting package details ...")
     for pkg in packages:
+        pkg.set_url()
         pkg.set_infra()
         pkg.set_license()
         pkg.set_hash_info()
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 44+ messages in thread
end of thread, other threads:[~2018-10-05 11:51 UTC | newest]

Thread overview: 44+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-09-20 13:24 [Buildroot] [PATCH 01/14] support/scripts/pkg-stats: URL checking support Matt Weber
2018-09-20 13:24 ` [Buildroot] [PATCH 02/14] boot/at91bootstrap: URL update Matt Weber
2018-09-20 20:12   ` Thomas Petazzoni
2018-09-20 20:35     ` Matthew Weber
2018-09-20 21:10       ` Thomas Petazzoni
2018-09-20 13:24 ` [Buildroot] [PATCH 03/14] boot/at91bootstrap3: " Matt Weber
2018-09-20 20:11   ` Thomas Petazzoni
2018-10-05 11:48   ` Peter Korsgaard
2018-09-20 13:24 ` [Buildroot] [PATCH 04/14] boot/at91dataflashboot: " Matt Weber
2018-09-20 13:24 ` [Buildroot] [PATCH 05/14] boot/xloader: " Matt Weber
2018-09-20 20:28   ` Thomas Petazzoni
2018-10-05 11:49   ` Peter Korsgaard
2018-09-20 13:24 ` [Buildroot] [PATCH 06/14] package/android-tools: " Matt Weber
2018-09-20 20:21   ` Thomas Petazzoni
2018-10-05 11:48   ` Peter Korsgaard
2018-09-20 13:24 ` [Buildroot] [PATCH 07/14] package/arp-scan: " Matt Weber
2018-09-20 20:21   ` Thomas Petazzoni
2018-10-05 11:48   ` Peter Korsgaard
2018-09-20 13:24 ` [Buildroot] [PATCH 08/14] package/bandwidthd: URL move to newline Matt Weber
2018-09-20 20:21   ` Thomas Petazzoni
2018-10-05 11:49   ` Peter Korsgaard
2018-09-20 13:24 ` [Buildroot] [PATCH 09/14] package/connman: " Matt Weber
2018-09-20 19:29   ` Thomas Petazzoni
2018-10-05 11:51   ` Peter Korsgaard
2018-09-20 13:24 ` [Buildroot] [PATCH 10/14] package/glibc: URL update Matt Weber
2018-09-20 19:30   ` Thomas Petazzoni
2018-09-20 20:05     ` Matthew Weber
2018-09-20 20:17       ` Thomas Petazzoni
2018-09-20 20:31         ` Matthew Weber
2018-09-20 13:24 ` [Buildroot] [PATCH 11/14] package/tidsp-binaries: " Matt Weber
2018-09-20 19:39   ` Thomas Petazzoni
2018-09-20 21:38     ` Matthew Weber
2018-09-20 13:24 ` [Buildroot] [PATCH 12/14] boot/vexpress-firmware: " Matt Weber
2018-09-20 19:41   ` Thomas Petazzoni
2018-10-05 11:47   ` Peter Korsgaard
2018-09-20 13:24 ` [Buildroot] [PATCH 13/14] package/alljoyn-base: " Matt Weber
2018-09-20 20:04   ` Thomas Petazzoni
2018-09-20 20:44     ` Matthew Weber
2018-09-20 21:12       ` Thomas Petazzoni
2018-09-20 13:24 ` [Buildroot] [PATCH 14/14] package/alljoyn-tcl-base: " Matt Weber
2018-09-20 20:58 ` [Buildroot] [PATCH 01/14] support/scripts/pkg-stats: URL checking support Thomas Petazzoni
2018-09-21 12:56   ` Matthew Weber
2018-09-21 13:08     ` Thomas Petazzoni
2018-09-21 13:15       ` Matthew Weber

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.