[OSSTEST PATCH 1/4] line wrapping: Provide cr-fold-long-lines script

[OSSTEST PATCH 1/4] line wrapping: Provide cr-fold-long-lines script

[Ian Jackson]

This is a reversible transformation which usually just introduces a \
where it splits lines.

We are going to use this to wrap the lines in our emails.  SMTP has a
999-byte length limit (including a CR-LF pair).  This can cause our
emails to go astray.  We don't really want our messages to be q-p or
base64-encoded if we can avoid it, and MTAs don't do that anyway (so
we would have to organise it).  So instead, we will simply wrap any
long lines that occur.

This transformation is not suitable for headers, but we don't intend
or want to generate long lines which would need further wrapping.  (A
reversible transformation suitable for headers would be quite ugly and
would only be right for a subset of headers anyway...)

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 cr-fold-long-lines | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100755 cr-fold-long-lines

diff --git a/cr-fold-long-lines b/cr-fold-long-lines
new file mode 100755
index 00000000..821c46a1
--- /dev/null
+++ b/cr-fold-long-lines
@@ -0,0 +1,22 @@
+#!/usr/bin/perl -wp
+#
+# long lines are broken by inserting \ plus spaces/tabs, somewhere
+# lines already ending with \ and maybe some $s have a $ postpended
+#   so they do not look broken
+#
+# unparsing:
+#      ooo\ | <SPC>bar     =>      ooobar   (<SPC> is >=0 tab/space)
+# then:
+#     ooo\$ | any          =>      ooo\ | any
+#    ooo\$$ | any          =>     ooo\$ | any
+#   ooo\$$$ | any          =>    ooo\$$ | any     etc.
+# others unchanged, specifically:
+#      ooo$ | any          =>      ooo$ | any
+#     ooo$$ | any          =>     ooo$$ | any
+#    ooo$$$ | any          =>    ooo$$$ | any     etc.
+# (where in these diagrams ` | ' means a newline)
+
+use strict;
+
+s{\\(\$*)\n}{\\$1\$\n}; # $-stuffing, now does not end in backslash
+s{.{500}}{$&\\\n }g; # insert \ \n SPC as needed
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST PATCH 2/4] line wrapping: Use tmp/$flight.report in two extra places

[Ian Jackson]

The output from sg-report-flight might in principle contain long
lines, although this is not expected.  So we are going to want to feed
it through the new cr-fold-long-lines.

Rather than piping, we are going to keep a copy of the .report file,
like is done in mg-execute-flight.  So for now, just make that change.

No overall change other than to leave behind the tmp/$flight.report
file.  It will be tidied up by the usual cleanup processes.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 cri-args-hostlists | 3 ++-
 cri-bisect         | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/cri-args-hostlists b/cri-args-hostlists
index 7d230875..a788e095 100644
--- a/cri-args-hostlists
+++ b/cri-args-hostlists
@@ -119,7 +119,8 @@ start_email () {
 
 	./sg-report-flight --html-dir=$flight_html_dir/$flight/ \
 		--allow=allow.all --allow=allow.$branch \
-		$sgr_args $flight
+		$sgr_args $flight >tmp/$flight.report
+	cat tmp/$flight.report
 
 	mkdir -p $host_html_dir
 	with-lock-ex -w $globallockdir/report-lock \
diff --git a/cri-bisect b/cri-bisect
index 28a8659a..f8fc55a8 100644
--- a/cri-bisect
+++ b/cri-bisect
@@ -107,7 +107,8 @@ END
 		if [ "x$OSSTEST_HTMLPUB_DIR" != x ]; then
 			sgr_args+=" --html-dir=$OSSTEST_HTMLPUB_DIR/$flight/"
 		fi
-                ./sg-report-flight $sgr_args >&3 $flight
+                ./sg-report-flight $sgr_args $flight >tmp/$flight.report
+		cat tmp/$flight.report >&3
                 ./cr-publish-flight-logs --push-harness $flight
         fi
         exec 3>&-
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST PATCH 3/4] line wrapping: Replace many uses of cat with cr-fold-long-lines

[Ian Jackson]

These are the nicely uniform bits of this change.

This arranges that many of the places where stuff gets put into emails
has their lines wrapped.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 cr-daily-branch    |  2 +-
 cri-args-hostlists |  2 +-
 cri-bisect         | 10 +++++-----
 mg-execute-flight  |  4 ++--
 4 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/cr-daily-branch b/cr-daily-branch
index e1528d49..49b8ad8e 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -547,7 +547,7 @@ if [ -f $revlog ]; then
     revlog_lines=`wc -l <$revlog`
     if [ $revlog_lines -lt 300 ]; then
         echo ------------------------------------------------------------
-        cat $revlog
+        ./cr-fold-long-lines $revlog
     else
         echo "(No revision log; it would be" $revlog_lines "lines long.)"
     fi
diff --git a/cri-args-hostlists b/cri-args-hostlists
index a788e095..3578fe1c 100644
--- a/cri-args-hostlists
+++ b/cri-args-hostlists
@@ -120,7 +120,7 @@ start_email () {
 	./sg-report-flight --html-dir=$flight_html_dir/$flight/ \
 		--allow=allow.all --allow=allow.$branch \
 		$sgr_args $flight >tmp/$flight.report
-	cat tmp/$flight.report
+	./cr-fold-long-lines tmp/$flight.report
 
 	mkdir -p $host_html_dir
 	with-lock-ex -w $globallockdir/report-lock \
diff --git a/cri-bisect b/cri-bisect
index f8fc55a8..576e7511 100644
--- a/cri-bisect
+++ b/cri-bisect
@@ -90,25 +90,25 @@ job $job
 testid $testid
 
 END
-        cat >&3 $summaryfile
-        if $bisgraph; then cat >&3 <<END; fi
+        ./cr-fold-long-lines >&3 $summaryfile
+        if $bisgraph; then ./cr-fold-long-lines >&3 <<END; fi
 
 For bisection revision-tuple graph see:
    $OSSTEST_HTML_URL/$bisleaf.html
 Revision IDs in each graph node refer, respectively, to the Trees above.
 END
-        cat >&3 <<END
+        ./cr-fold-long-lines >&3 <<END
 
 ----------------------------------------
 END
-        cat >&3 $reportfile
+        ./cr-fold-long-lines >&3 $reportfile
         if $flight_executed; then
                 echo >&3 ----------------------------------------
 		if [ "x$OSSTEST_HTMLPUB_DIR" != x ]; then
 			sgr_args+=" --html-dir=$OSSTEST_HTMLPUB_DIR/$flight/"
 		fi
                 ./sg-report-flight $sgr_args $flight >tmp/$flight.report
-		cat tmp/$flight.report >&3
+		./cr-fold-long-lines tmp/$flight.report >&3
                 ./cr-publish-flight-logs --push-harness $flight
         fi
         exec 3>&-
diff --git a/mg-execute-flight b/mg-execute-flight
index 98aca45d..349880b0 100755
--- a/mg-execute-flight
+++ b/mg-execute-flight
@@ -97,10 +97,10 @@ END
 git log -n1 --pretty=format:'harness %h: %s' | perl -pe 's/(.{70}).+/$1.../'
 echo
 
-cat <tmp/$flight.report
+./cr-fold-long-lines <tmp/$flight.report
 
 echo ============================================================
-cat <tmp/$flight.transcript
+./cr-fold-long-lines <tmp/$flight.transcript
 
 exec >&2
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST PATCH 4/4] line wrapping: In one place pipe through cr-fold-long-lines

[Ian Jackson]

This is the remaining place where long lines might get into emails.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-execute-flight | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/mg-execute-flight b/mg-execute-flight
index 349880b0..b3cdf431 100755
--- a/mg-execute-flight
+++ b/mg-execute-flight
@@ -94,7 +94,8 @@ $email_std_headers
 $OSSTEST_RESOURCE_PREINFO
 END
 
-git log -n1 --pretty=format:'harness %h: %s' | perl -pe 's/(.{70}).+/$1.../'
+git log -n1 --pretty=format:'harness %h: %s' | perl -pe 's/(.{70}).+/$1.../' \
+	| ./cr-fold-long-lines
 echo
 
 ./cr-fold-long-lines <tmp/$flight.report
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel