[PATCH -perfbook 0/7] Build script updates

[PATCH -perfbook 0/7] Build script updates

[Akira Yokosawa]

Hi Paul,

Here is a set of updates to build scripts.

Patch 1/7 is to avoid possible build errors when previous build
is interrupted in the middle of converting figures with some
intermediate temporary eps and/or pdf file(s) left. 

Example of such an error (long message not folded):

CodeSamples/datastruct/hash/data/hps.perf.2020.11.26a/zoocatall.eps --> .pdf
make: *** No rule to make target 'CodeSamples/datastruct/hash/data/hps.perf.2020.11.26a/zoocatall__.eps', needed by 'CodeSamples/datastruct/hash/data/hps.perf.2020.11.26a/zoocatall__.pdf'.  Stop.

After this type of an error (or errors after a parallel build),
one needs to run "make clean" (or "make neatfreak"), and run
"make" again.

By adding commands in Makefile to ignore file names of "*__.eps",
this mode of error can be avoided.

The added pattern in .gitignore resolves false "Untracked files"
from "git status".

Patch 2/7 fixes redundant use of filter-out function in Makefile.

Patch 3/7 updates help texts regarding EBook-size builds in
Makefile.  They don't need (WIP) anymore.

Patch 4/7 is to allow use of LaTeX commands other than "pdflatex".
In vanilla TeX Live, there is an alternative command of
"pdflatex-dev", which is used to test development-phase
latex implementation.  With this change, I am allowed to test
it by saying "make LATEX=pdflatex-dev".

Patch 5/7 is to catch typo in the LATEX variable.

Patch 6/7 catches an error exit of LATEX command and print log
messages to the terminal.

Patch 7/7 is a cosmetic tweak improving the later stages of
progress reports from parallel builds.

        Thanks, Akira
--
Akira Yokosawa (7):
  Makefile: Ignore temporary .eps files
  Makefile: Remove redundant nest of filter-out function
  Makefile: Let ebook-size builds graduate from 'WIP'
  run{first}latex.sh: Use variable for 'pdflatex' command
  Makefile: Test $(LATEX) before invoking runfirstlatex.sh
  run(first)latex.sh: Catch error exit of LATEX
  runlatex.sh: Align status report messages

 .gitignore                            |   1 +
 Makefile                              |  27 ++++-
 Makefile.2022.01.25                   |   4 +-
 utilities/runfirstlatex.2022.01.25.sh |  98 +++++++++++++++
 utilities/runfirstlatex.sh            |  16 ++-
 utilities/runlatex.2022.01.25.sh      | 164 ++++++++++++++++++++++++++
 utilities/runlatex.sh                 |  23 ++--
 7 files changed, 314 insertions(+), 19 deletions(-)
 create mode 100644 utilities/runfirstlatex.2022.01.25.sh
 create mode 100644 utilities/runlatex.2022.01.25.sh


base-commit: 87d3513257e793b90f53ebd4b138ab2371aaf25c
-- 
2.25.1

[PATCH -perfbook 1/7] Makefile: Ignore temporary .eps files

[Akira Yokosawa]

Recipes in epstopdf-rule.mk and a2ping-rule.mk use *__.eps and
*__.pdf as temporary files for fixing fonts and cropping.

Those files are removed afterwards in normal builds, but if you
interrupt "make", they can escape from removals.

They may cause a build error in subsequent "make", which might not
be resolved by "make clean" but needs "make neatfreak".

Add variable EPSSOURCES_TMP in Makefile and filter them out from
EPSSOURCES.

Add a pattern in .gitignore for them as well.

Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
---
 .gitignore |  1 +
 Makefile   | 11 +++++++++--
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/.gitignore b/.gitignore
index 3711e0c6..1604a4d8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -17,6 +17,7 @@
 *.swo
 *_backup.svg
 *.epsi
+*__.eps
 *.svgi
 *.fcv
 *.ltms
diff --git a/Makefile b/Makefile
index e09ba32d..a605487a 100644
--- a/Makefile
+++ b/Makefile
@@ -67,13 +67,20 @@ EPSSOURCES_DUP := \
 	$(EPSSOURCES_FROM_DOT) \
 	$(EPSSOURCES_FROM_FIG)
 
+EPSSOURCES_TMP := \
+	$(wildcard */*__.eps) \
+	$(wildcard */*/*__.eps) \
+	$(wildcard */*/*/*__.eps) \
+	$(wildcard */*/*/*/*__.eps) \
+	$(wildcard */*/*/*/*/*__.eps) \
+
 EPSSOURCES_OLD := \
 	$(wildcard CodeSamples/*/*/OLD-*/*.eps) \
 	$(wildcard CodeSamples/*/*/OLD-*/*/*.eps) \
 	$(wildcard CodeSamples/*/*/*/OLD-*/*.eps) \
 	$(wildcard CodeSamples/*/*/*/OLD-*/*/*.eps)
 
-EPSSOURCES := $(sort $(filter-out $(EPSSOURCES_OLD),$(filter-out $(OBSOLETE_FILES),$(EPSSOURCES_DUP))))
+EPSSOURCES := $(sort $(filter-out $(EPSSOURCES_OLD),$(filter-out $(OBSOLETE_FILES) $(EPSSOURCES_TMP),$(EPSSOURCES_DUP))))
 
 PDFTARGETS_OF_EPS := $(EPSSOURCES:%.eps=%.pdf)
 
@@ -564,7 +571,7 @@ clean:
 	rm -f perfbook*.glg perfbook*.glo perfbook*.gls perfbook*.glsdefs
 	rm -f CodeSamples/snippets.d
 	rm -f *.synctex*
-	@rm -f $(OBSOLETE_FILES)
+	@rm -f $(OBSOLETE_FILES) $(EPSSOURCES_TMP)
 
 paper-clean:
 	rm -f $(BASE_DEPENDS)
-- 
2.25.1

[PATCH -perfbook 2/7] Makefile: Remove redundant nest of filter-out function

[Akira Yokosawa]

Simplify the use of filter-out function.
This simpler list of variables in its first argument should have
been used in commit afd0c61f9cd2 ("Makefile: Ignore .eps files
under CodeSamples/.../OLD-*/").

Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index a605487a..ff8a6a26 100644
--- a/Makefile
+++ b/Makefile
@@ -80,7 +80,7 @@ EPSSOURCES_OLD := \
 	$(wildcard CodeSamples/*/*/*/OLD-*/*.eps) \
 	$(wildcard CodeSamples/*/*/*/OLD-*/*/*.eps)
 
-EPSSOURCES := $(sort $(filter-out $(EPSSOURCES_OLD),$(filter-out $(OBSOLETE_FILES) $(EPSSOURCES_TMP),$(EPSSOURCES_DUP))))
+EPSSOURCES := $(sort $(filter-out $(EPSSOURCES_OLD) $(OBSOLETE_FILES) $(EPSSOURCES_TMP),$(EPSSOURCES_DUP)))
 
 PDFTARGETS_OF_EPS := $(EPSSOURCES:%.eps=%.pdf)
 
-- 
2.25.1

[PATCH -perfbook 3/7] Makefile: Let ebook-size builds graduate from 'WIP'

[Akira Yokosawa]

Now that ebook-size builds are mostly in good shape, let them
graduate from "WIP" by removing such markers in help texts.

Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
---
 Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index ff8a6a26..fb899fa0 100644
--- a/Makefile
+++ b/Makefile
@@ -516,7 +516,7 @@ help-paper:
 	@echo "  perfbook-lt.pdf,   lt:   2c layout on letterpaper"
 	@echo "  perfbook-hb.pdf,   hb:   2c layout for hard cover book"
 	@echo "  perfbook-a4.pdf,   a4:   2c layout on a4paper"
-	@echo "  perfbook-eb.pdf,   eb:   1c layout for ebook reader (WIP)"
+	@echo "  perfbook-eb.pdf,   eb:   1c layout for ebook reader"
 
 help: help-official help-paper
 	@echo
@@ -533,7 +533,7 @@ help-prefixed:
 	@echo "Prefixed targets:"
 	@echo "  \"1c*\" such as \"1cnq\", \"1csf\", and \"1cix\" are for 1c-layout."
 	@echo "  \"ebnq\", \"ebsf\", \"ebsfnq\", \"ebix\", and \"ebdf\" are for ebook-size 1c-layout,"
-	@echo "     independent of PERFBOOK_PAPER. (WIP)"
+	@echo "     independent of PERFBOOK_PAPER."
 
 help-experimental:
 	@echo
-- 
2.25.1

[PATCH -perfbook 4/7] run{first}latex.sh: Use variable for 'pdflatex' command

[Akira Yokosawa]

In testing a dev version of upstream LaTeX, "pdflatex-dev" is used
in place of "pdflatex".

Using a variable named "LATEX", whose default is pdflatex, in the
scripts helps in such cases.

Update Makefile as well so that the dev version can be specified
as follows:

    make LATEX=pdflatex-dev

Note 1: The trick for setting a default value to a shell variable
using ":" is borrowed from an answer in unix.stackexchange.com:

    https://unix.stackexchange.com/questions/122845/

Note 2: To keep the test of utilities/parallel-latex-regress.sh
working, make copies of the scripts before this change as
utilities/run{first}latex.2022.01.25.sh.

Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
---
 Makefile                              |   5 +-
 Makefile.2022.01.25                   |   4 +-
 utilities/runfirstlatex.2022.01.25.sh |  98 +++++++++++++++
 utilities/runfirstlatex.sh            |   6 +-
 utilities/runlatex.2022.01.25.sh      | 164 ++++++++++++++++++++++++++
 utilities/runlatex.sh                 |  12 +-
 6 files changed, 278 insertions(+), 11 deletions(-)
 create mode 100644 utilities/runfirstlatex.2022.01.25.sh
 create mode 100644 utilities/runlatex.2022.01.25.sh

diff --git a/Makefile b/Makefile
index fb899fa0..4ab54130 100644
--- a/Makefile
+++ b/Makefile
@@ -1,4 +1,5 @@
 SHELL = /bin/bash
+LATEX ?= pdflatex
 
 GITREFSTAGS := $(shell ls -d .git/refs/tags 2>/dev/null)
 
@@ -224,7 +225,7 @@ qqmsg: perfbook.pdf
 	@echo "built as perfbook.pdf."
 
 $(PDFTARGETS): %.pdf: %.tex %.bbl
-	sh utilities/runlatex.sh $(basename $@)
+	LATEX=$(LATEX) sh utilities/runlatex.sh $(basename $@)
 
 $(PDFTARGETS:.pdf=.bbl): %.bbl: %.aux $(BIBSOURCES)
 	bibtex $(basename $@)
@@ -233,7 +234,7 @@ $(PDFTARGETS:.pdf=.aux): %.aux: %.tex $(LATEXGENERATED)
 ifeq ($(NEWTXTEXT),)
 	$(error Font package 'newtx' not found. See #9 in FAQ-BUILD.txt)
 endif
-	sh utilities/runfirstlatex.sh $(basename $@)
+	LATEX=$(LATEX) sh utilities/runfirstlatex.sh $(basename $@)
 
 autodate.tex: $(LATEXSOURCES) $(BIBSOURCES) $(LST_SOURCES) \
     $(PDFTARGETS_OF_EPS) $(PDFTARGETS_OF_SVG) $(FCVSNIPPETS) $(FCVSNIPPETS_VIA_LTMS) \
diff --git a/Makefile.2022.01.25 b/Makefile.2022.01.25
index 6ee849fd..9e4b7ecd 100644
--- a/Makefile.2022.01.25
+++ b/Makefile.2022.01.25
@@ -217,7 +217,7 @@ qqmsg: perfbook.pdf
 	@echo "built as perfbook.pdf."
 
 $(PDFTARGETS): %.pdf: %.tex %.bbl
-	sh utilities/runlatex.sh $(basename $@)
+	sh utilities/runlatex.2022.01.25.sh $(basename $@)
 
 $(PDFTARGETS:.pdf=.bbl): %.bbl: %.aux $(BIBSOURCES)
 	bibtex $(basename $@)
@@ -226,7 +226,7 @@ $(PDFTARGETS:.pdf=.aux): $(LATEXGENERATED) $(LATEXSOURCES) $(LST_SOURCES)
 ifeq ($(NEWTXTEXT),)
 	$(error Font package 'newtx' not found. See #9 in FAQ-BUILD.txt)
 endif
-	sh utilities/runfirstlatex.sh $(basename $@)
+	sh utilities/runfirstlatex.2022.01.25.sh $(basename $@)
 
 autodate.tex: perfbook-lt.tex $(LATEXSOURCES) $(BIBSOURCES) \
     $(PDFTARGETS_OF_EPS) $(PDFTARGETS_OF_SVG) $(FCVSNIPPETS) $(FCVSNIPPETS_VIA_LTMS) \
diff --git a/utilities/runfirstlatex.2022.01.25.sh b/utilities/runfirstlatex.2022.01.25.sh
new file mode 100644
index 00000000..81b66dc3
--- /dev/null
+++ b/utilities/runfirstlatex.2022.01.25.sh
@@ -0,0 +1,98 @@
+#!/bin/sh
+#
+# Run the first round of pdflatex.
+# It is assumed to be used together with runlatex.sh and invoked from
+# 'make' command.
+#
+# Usage: sh runfirstlatex.sh file[.tex]
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2012-2019
+# Copyright (C) Facebook, 2019
+# Copyright (C) Akira Yokosawa, 2016
+#
+# Authors: Paul E. McKenney <paulmck@us.ibm.com>
+#          Akira Yokosawa <akiyks@gmail.com>
+#
+# Saved as of 2022.01.25 for regression testing.
+
+if test -z "$1"
+then
+	echo No latex file specified, aborting.
+	exit 1
+fi
+
+if ! sh utilities/mpostcheck.sh
+then
+	exit 1
+fi
+
+# newtx font package version check (newer or equal to Ubuntu 14.04)
+NEWTXTEXT=`kpsewhich newtxtext.sty`
+NEWTXTEXT_DATE=`grep filedate $NEWTXTEXT | grep -o -E "[/0-9]*"`
+# We need TeX Live 2013/Debian (Ubuntu 14.04) or later
+if [ "$NEWTXTEXT_DATE" \< "2014/02/12" ]
+then
+	echo "############################################################"
+	echo "### Old version of font package 'newtx' is detected.     ###"
+	echo "### You need to upgrade your TeX Live installation.      ###"
+	echo "### See item 9 in FAQ-BUILD.txt for further info.        ###"
+	echo "############################################################"
+	exit 1
+fi
+
+DETECTED_BUGGY=0
+# listings package version check (TeX Live 2014 and 2015 had buggy ones)
+if grep -F "fileversion" `kpsewhich listings.sty` | grep -q -E "1.5[cde]"
+then
+	echo "############################################################"
+	echo "### Buggy version of LaTeX package 'listings' detected!! ###"
+	echo "### (Known issue in TeX Live 2014 and 2015)              ###"
+	echo "### Please install a latest version.                     ###"
+	echo "### See item 10 in FAQ-BUILD.txt for further info.       ###"
+	echo "############################################################"
+	DETECTED_BUGGY=1
+fi
+
+basename=`echo $1 | sed -e 's/\.tex$//'`
+
+echo "pdflatex 1 for $basename.pdf"
+pdflatex $LATEX_OPT $basename > /dev/null 2>&1 < /dev/null || :
+if grep -q 'LaTeX Warning: You have requested' $basename.log
+then
+	grep -A 4 'LaTeX Warning: You have requested' $basename.log
+	echo "### Incompatible package(s) detected. See $basename.log for details. ###"
+	echo "### See items 9 and 10 in FAQ-BUILD.txt for how to update.          ###"
+	exit 1
+fi
+if [ $DETECTED_BUGGY -eq 1 ]; then
+	exit 1
+fi
+if grep -q '! Emergency stop.' $basename.log
+then
+	grep -B 15 -A 5 '! Emergency stop.' $basename.log
+	echo "----- Fatal latex error, see $basename.log for details. -----"
+	exit 1
+fi
+if grep -q '!pdfTeX error:' $basename.log
+then
+	grep -A 2 '!pdfTeX error:' $basename.log
+	echo "----- Fatal latex error, see $basename.log for details. -----"
+	exit 1
+fi
+grep 'LaTeX Warning:' $basename.log > $basename-warning.log
+touch $basename-first.log
+exit 0
diff --git a/utilities/runfirstlatex.sh b/utilities/runfirstlatex.sh
index 22bafc2a..f5504f88 100644
--- a/utilities/runfirstlatex.sh
+++ b/utilities/runfirstlatex.sh
@@ -67,8 +67,10 @@ fi
 
 basename=`echo $1 | sed -e 's/\.tex$//'`
 
-echo "pdflatex 1 for $basename.pdf"
-pdflatex $LATEX_OPT $basename > /dev/null 2>&1 < /dev/null || :
+: ${LATEX:-pdflatex}
+
+echo "$LATEX 1 for $basename.pdf"
+$LATEX $LATEX_OPT $basename > /dev/null 2>&1 < /dev/null || :
 if grep -q 'LaTeX Warning: You have requested' $basename.log
 then
 	grep -A 4 'LaTeX Warning: You have requested' $basename.log
diff --git a/utilities/runlatex.2022.01.25.sh b/utilities/runlatex.2022.01.25.sh
new file mode 100644
index 00000000..22aca75f
--- /dev/null
+++ b/utilities/runlatex.2022.01.25.sh
@@ -0,0 +1,164 @@
+#!/bin/sh
+#
+# Run pdflatex on the specified file.
+# Attempt to avoid useless repeats.
+# This version is heavily customized to be used for perfbook.
+# It is assumed to be used together with runfirstlatex.sh
+# and Makefile of perfbook.
+#
+# Usage: sh runlatex.sh file[.tex]
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2012-2019
+# Copyright (C) Facebook, 2019
+# Copyright (C) Akira Yokosawa, 2016, 2017
+#
+# Authors: Paul E. McKenney <paulmck@us.ibm.com>
+#          Akira Yokosawa <akiyks@gmail.com>
+#
+# Saved as of 2022.01.25 for regression testing.
+
+diff_warning () {
+	if diff -q $basename-warning.log $basename-warning-prev.log >/dev/null
+	then
+		echo "No more improvement is expected, giving up."
+		return 0 ;
+	else
+#		echo "Some improvements are observed, continuing."
+		return 1 ;
+	fi
+}
+
+identical_warnings () {
+	if test -r $basename-warning-prev.log
+	then
+		if test "$iter" -gt "$min_iter" && diff_warning
+		then
+			return 0 ;
+		fi
+	fi
+	return 1 ;
+}
+
+exerpt_warnings () {
+	if grep -q "LaTeX Warning:" $basename.log
+	then
+		echo "----- Excerpt around remaining warning messages -----"
+		grep -B 8 -A 5 "LaTeX Warning:" $basename.log | tee $basename-warning.log
+		echo "----- You can see $basename-warning.log for the warnings above. -----"
+		echo "----- If you need to, see $basename.log for details. -----"
+		rm -f $basename-warning-prev.log
+		exit 1
+	fi
+}
+
+iterate_latex () {
+	perl utilities/adjustindexformat.pl $basename.idx > $basename-adjust.idx
+	cp -f $basename-adjust.idx $basename.idx
+	makeindex $basename.idx > /dev/null 2>&1
+	makeindex $basename-api.idx > /dev/null 2>&1
+	if grep -q '## Warning' $basename.ilg $basename-api.ilg
+	then
+		echo "----- Warning in makeindex, see .ilg log files. -----"
+		exit 1
+	fi
+	makeglossaries $basename > /dev/null 2>&1
+	pdflatex $LATEX_OPT $basename > /dev/null 2>&1 < /dev/null || :
+	if grep -q '! Emergency stop.' $basename.log
+	then
+		grep -B 15 -A 5 '! Emergency stop.' $basename.log
+		echo "----- Fatal latex error, see $basename.log for details. -----"
+		exit 1
+	fi
+	if grep -q '!pdfTeX error:' $basename.log
+	then
+		grep -A 2 '!pdfTeX error:' $basename.log
+		echo "----- Fatal latex error, see $basename.log for details. -----"
+		exit 1
+	fi
+	if test -r $basename-warning.log
+	then
+		mv -f $basename-warning.log $basename-warning-prev.log
+	fi
+	grep 'LaTeX Warning:' $basename.log > $basename-warning.log
+	return 0 ;
+}
+
+if test -z "$1"
+then
+	echo No latex file specified, aborting.
+	exit 1
+fi
+
+basename=`echo $1 | sed -e 's/\.tex$//'`
+
+if ! test -r $basename-first.log
+then
+	echo "No need to update aux and bbl files."
+	echo "pdflatex 1 for $basename.pdf"
+	iter=1
+else
+	rm -f $basename-first.log
+	echo "pdflatex 2 for $basename.pdf # for possible bib update"
+	iter=2
+fi
+iterate_latex
+min_iter=2
+while grep -q 'LaTeX Warning: There were undefined references' $basename.log
+do
+	if test $undefined_refs
+	then
+		echo "Undefined refs remain, giving up."
+		exerpt_warnings
+	fi
+	if identical_warnings
+	then
+		break
+	fi
+	iter=`expr $iter + 1`
+	echo "pdflatex $iter for $basename.pdf # remaining undefined refs"
+	undefined_refs=1
+	iterate_latex
+done
+min_iter=3
+while grep -q 'LaTeX Warning: Label(s) may have changed' $basename.log
+do
+	if identical_warnings
+	then
+		break
+	fi
+	iter=`expr $iter + 1`
+	echo "pdflatex $iter for $basename.pdf # label(s) may have changed"
+	iterate_latex
+done
+exerpt_warnings
+rm -f $basename-warning.log $basename-warning-prev.log
+echo "'$basename.pdf' is ready."
+# cleveref version check (Ubuntu 18.04 LTS has buggy one
+if grep -q -F "packageversion{0.21.1}" `kpsewhich cleveref.sty`
+then
+	echo "############################################################"
+	echo "### Buggy version of LaTeX package 'cleveref' detected!! ###"
+	echo "### (Known issue on Ubuntu 18.04 LTS)                    ###"
+	echo "### Please consider installing a stable version.         ###"
+	echo "### See item 10 in FAQ-BUILD.txt for further info.       ###"
+	echo "############################################################"
+fi
+# to avoid redundant run of bibtex and pdflatex
+touch $basename.bbl
+touch $basename.pdf
+sh utilities/mpostcheck.sh
+exit 0
diff --git a/utilities/runlatex.sh b/utilities/runlatex.sh
index 48761fd0..db196ad2 100644
--- a/utilities/runlatex.sh
+++ b/utilities/runlatex.sh
@@ -29,6 +29,8 @@
 # Authors: Paul E. McKenney <paulmck@us.ibm.com>
 #          Akira Yokosawa <akiyks@gmail.com>
 
+: ${LATEX:-pdflatex}
+
 diff_warning () {
 	if diff -q $basename-warning.log $basename-warning-prev.log >/dev/null
 	then
@@ -74,7 +76,7 @@ iterate_latex () {
 		exit 1
 	fi
 	makeglossaries $basename > /dev/null 2>&1
-	pdflatex $LATEX_OPT $basename > /dev/null 2>&1 < /dev/null || :
+	$LATEX $LATEX_OPT $basename > /dev/null 2>&1 < /dev/null || :
 	if grep -q '! Emergency stop.' $basename.log
 	then
 		grep -B 15 -A 5 '! Emergency stop.' $basename.log
@@ -106,11 +108,11 @@ basename=`echo $1 | sed -e 's/\.tex$//'`
 if ! test -r $basename-first.log
 then
 	echo "No need to update aux and bbl files."
-	echo "pdflatex 1 for $basename.pdf"
+	echo "$LATEX 1 for $basename.pdf"
 	iter=1
 else
 	rm -f $basename-first.log
-	echo "pdflatex 2 for $basename.pdf # for possible bib update"
+	echo "$LATEX 2 for $basename.pdf # for possible bib update"
 	iter=2
 fi
 iterate_latex
@@ -127,7 +129,7 @@ do
 		break
 	fi
 	iter=`expr $iter + 1`
-	echo "pdflatex $iter for $basename.pdf # remaining undefined refs"
+	echo "$LATEX $iter for $basename.pdf # remaining undefined refs"
 	undefined_refs=1
 	iterate_latex
 done
@@ -139,7 +141,7 @@ do
 		break
 	fi
 	iter=`expr $iter + 1`
-	echo "pdflatex $iter for $basename.pdf # label(s) may have changed"
+	echo "$LATEX $iter for $basename.pdf # label(s) may have changed"
 	iterate_latex
 done
 exerpt_warnings
-- 
2.25.1

[PATCH -perfbook 5/7] Makefile: Test $(LATEX) before invoking runfirstlatex.sh

[Akira Yokosawa]

The LATEX variable can have a typo.  Check if it is available
before invoking it.

Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
---
 Makefile | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/Makefile b/Makefile
index 4ab54130..abcd7e2c 100644
--- a/Makefile
+++ b/Makefile
@@ -101,6 +101,7 @@ PDFTARGETS_OF_EPSOTHER := $(filter-out $(PDFTARGETS_OF_EPSORIG) $(PDFTARGETS_OF_
 BIBSOURCES := bib/*.bib alphapf.bst
 
 # required commands
+LATEX_CMD := $(shell which $(LATEX) 2>/dev/null)
 DOT := $(shell which dot 2>/dev/null)
 FIG2EPS := $(shell which fig2eps 2>/dev/null)
 INKSCAPE := $(shell which inkscape 2>/dev/null)
@@ -225,6 +226,9 @@ qqmsg: perfbook.pdf
 	@echo "built as perfbook.pdf."
 
 $(PDFTARGETS): %.pdf: %.tex %.bbl
+ifeq ($(LATEX_CMD),)
+	$(error LaTeX engine "$(LATEX)" not found.)
+endif
 	LATEX=$(LATEX) sh utilities/runlatex.sh $(basename $@)
 
 $(PDFTARGETS:.pdf=.bbl): %.bbl: %.aux $(BIBSOURCES)
@@ -233,6 +237,9 @@ $(PDFTARGETS:.pdf=.bbl): %.bbl: %.aux $(BIBSOURCES)
 $(PDFTARGETS:.pdf=.aux): %.aux: %.tex $(LATEXGENERATED)
 ifeq ($(NEWTXTEXT),)
 	$(error Font package 'newtx' not found. See #9 in FAQ-BUILD.txt)
+endif
+ifeq ($(LATEX_CMD),)
+	$(error LaTeX engine "$(LATEX)" not found.)
 endif
 	LATEX=$(LATEX) sh utilities/runfirstlatex.sh $(basename $@)
 
-- 
2.25.1

[PATCH -perfbook 6/7] run(first)latex.sh: Catch error exit of LATEX

[Akira Yokosawa]

LATEX command can abort in its early stages of execution due to
(1) mis-configuration of TeX Live, (2) regression in TeX Live updates,
or (3) missing fonts.
Catch exit code of LATEX command and print the tail of .log when
LATEX command aborted.

Also increment the exit code of these script when some error condition
is detected in the .log file.

  exit code 0: success
  exit code 1: remaining warning(s) in .log
  exit code 2: error detected in .log
  other: exit code from LATEX

Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
---
 utilities/runfirstlatex.sh | 12 +++++++++---
 utilities/runlatex.sh      | 12 +++++++++---
 2 files changed, 18 insertions(+), 6 deletions(-)

diff --git a/utilities/runfirstlatex.sh b/utilities/runfirstlatex.sh
index f5504f88..b92895f3 100644
--- a/utilities/runfirstlatex.sh
+++ b/utilities/runfirstlatex.sh
@@ -70,7 +70,13 @@ basename=`echo $1 | sed -e 's/\.tex$//'`
 : ${LATEX:-pdflatex}
 
 echo "$LATEX 1 for $basename.pdf"
-$LATEX $LATEX_OPT $basename > /dev/null 2>&1 < /dev/null || :
+$LATEX $LATEX_OPT $basename > /dev/null 2>&1 < /dev/null
+exitcode=$?
+if [ $exitcode -ne 0 ]; then
+	tail -n 20 $basename.log
+	echo "\n!!! $LATEX aborted !!!"
+	exit $exitcode
+fi
 if grep -q 'LaTeX Warning: You have requested' $basename.log
 then
 	grep -A 4 'LaTeX Warning: You have requested' $basename.log
@@ -85,13 +91,13 @@ if grep -q '! Emergency stop.' $basename.log
 then
 	grep -B 15 -A 5 '! Emergency stop.' $basename.log
 	echo "----- Fatal latex error, see $basename.log for details. -----"
-	exit 1
+	exit 2
 fi
 if grep -q '!pdfTeX error:' $basename.log
 then
 	grep -A 2 '!pdfTeX error:' $basename.log
 	echo "----- Fatal latex error, see $basename.log for details. -----"
-	exit 1
+	exit 2
 fi
 grep 'LaTeX Warning:' $basename.log > $basename-warning.log
 touch $basename-first.log
diff --git a/utilities/runlatex.sh b/utilities/runlatex.sh
index db196ad2..b7d553ab 100644
--- a/utilities/runlatex.sh
+++ b/utilities/runlatex.sh
@@ -76,18 +76,24 @@ iterate_latex () {
 		exit 1
 	fi
 	makeglossaries $basename > /dev/null 2>&1
-	$LATEX $LATEX_OPT $basename > /dev/null 2>&1 < /dev/null || :
+	$LATEX $LATEX_OPT $basename > /dev/null 2>&1 < /dev/null
+	exitcode=$?
+	if [ $exitcode -ne 0 ]; then
+		tail -n 20 $basename.log
+		echo "\n!!! $LATEX aborted !!!"
+		exit $exitcode
+	fi
 	if grep -q '! Emergency stop.' $basename.log
 	then
 		grep -B 15 -A 5 '! Emergency stop.' $basename.log
 		echo "----- Fatal latex error, see $basename.log for details. -----"
-		exit 1
+		exit 2
 	fi
 	if grep -q '!pdfTeX error:' $basename.log
 	then
 		grep -A 2 '!pdfTeX error:' $basename.log
 		echo "----- Fatal latex error, see $basename.log for details. -----"
-		exit 1
+		exit 2
 	fi
 	if test -r $basename-warning.log
 	then
-- 
2.25.1

[PATCH -perfbook 7/7] runlatex.sh: Align status report messages

[Akira Yokosawa]

In parallel builds, status-report messages from different length
PDF file names look jagged.

Align those messages by using the "printf" command and left-adjusting
the file names in a 19 character field.

Improved messages from later iterations (make -j3 2c 1cnq ebsfnq):

    ...
    pdflatex 3 for perfbook.pdf        # remaining undefined refs
    pdflatex 3 for perfbook-1cnq.pdf   # remaining undefined refs
    pdflatex 3 for perfbook-ebsfnq.pdf # remaining undefined refs
    pdflatex 4 for perfbook.pdf        # label(s) may have changed
    pdflatex 4 for perfbook-1cnq.pdf   # label(s) may have changed
    pdflatex 4 for perfbook-ebsfnq.pdf # label(s) may have changed
    'perfbook.pdf' is ready.
    'perfbook-1cnq.pdf' is ready.
    'perfbook-ebsfnq.pdf' is ready.

Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
---
 utilities/runlatex.sh | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/utilities/runlatex.sh b/utilities/runlatex.sh
index b7d553ab..d4c9baf8 100644
--- a/utilities/runlatex.sh
+++ b/utilities/runlatex.sh
@@ -110,6 +110,7 @@ then
 fi
 
 basename=`echo $1 | sed -e 's/\.tex$//'`
+pdfname=`env printf "%-19s" $basename.pdf`
 
 if ! test -r $basename-first.log
 then
@@ -118,7 +119,7 @@ then
 	iter=1
 else
 	rm -f $basename-first.log
-	echo "$LATEX 2 for $basename.pdf # for possible bib update"
+	echo "$LATEX 2 for $pdfname # for possible bib update"
 	iter=2
 fi
 iterate_latex
@@ -135,7 +136,7 @@ do
 		break
 	fi
 	iter=`expr $iter + 1`
-	echo "$LATEX $iter for $basename.pdf # remaining undefined refs"
+	echo "$LATEX $iter for $pdfname # remaining undefined refs"
 	undefined_refs=1
 	iterate_latex
 done
@@ -147,7 +148,7 @@ do
 		break
 	fi
 	iter=`expr $iter + 1`
-	echo "$LATEX $iter for $basename.pdf # label(s) may have changed"
+	echo "$LATEX $iter for $pdfname # label(s) may have changed"
 	iterate_latex
 done
 exerpt_warnings
-- 
2.25.1

Re: [PATCH -perfbook 0/7] Build script updates

[Paul E. McKenney]

On Mon, Apr 11, 2022 at 04:49:01PM +0900, Akira Yokosawa wrote:
> Hi Paul,
> 
> Here is a set of updates to build scripts.
> 
> Patch 1/7 is to avoid possible build errors when previous build
> is interrupted in the middle of converting figures with some
> intermediate temporary eps and/or pdf file(s) left. 
> 
> Example of such an error (long message not folded):
> 
> CodeSamples/datastruct/hash/data/hps.perf.2020.11.26a/zoocatall.eps --> .pdf
> make: *** No rule to make target 'CodeSamples/datastruct/hash/data/hps.perf.2020.11.26a/zoocatall__.eps', needed by 'CodeSamples/datastruct/hash/data/hps.perf.2020.11.26a/zoocatall__.pdf'.  Stop.
> 
> After this type of an error (or errors after a parallel build),
> one needs to run "make clean" (or "make neatfreak"), and run
> "make" again.
> 
> By adding commands in Makefile to ignore file names of "*__.eps",
> this mode of error can be avoided.
> 
> The added pattern in .gitignore resolves false "Untracked files"
> from "git status".
> 
> Patch 2/7 fixes redundant use of filter-out function in Makefile.
> 
> Patch 3/7 updates help texts regarding EBook-size builds in
> Makefile.  They don't need (WIP) anymore.
> 
> Patch 4/7 is to allow use of LaTeX commands other than "pdflatex".
> In vanilla TeX Live, there is an alternative command of
> "pdflatex-dev", which is used to test development-phase
> latex implementation.  With this change, I am allowed to test
> it by saying "make LATEX=pdflatex-dev".
> 
> Patch 5/7 is to catch typo in the LATEX variable.
> 
> Patch 6/7 catches an error exit of LATEX command and print log
> messages to the terminal.
> 
> Patch 7/7 is a cosmetic tweak improving the later stages of
> progress reports from parallel builds.

Queued and pushed, thank you!

							Thanx, Paul

>         Thanks, Akira
> --
> Akira Yokosawa (7):
>   Makefile: Ignore temporary .eps files
>   Makefile: Remove redundant nest of filter-out function
>   Makefile: Let ebook-size builds graduate from 'WIP'
>   run{first}latex.sh: Use variable for 'pdflatex' command
>   Makefile: Test $(LATEX) before invoking runfirstlatex.sh
>   run(first)latex.sh: Catch error exit of LATEX
>   runlatex.sh: Align status report messages
> 
>  .gitignore                            |   1 +
>  Makefile                              |  27 ++++-
>  Makefile.2022.01.25                   |   4 +-
>  utilities/runfirstlatex.2022.01.25.sh |  98 +++++++++++++++
>  utilities/runfirstlatex.sh            |  16 ++-
>  utilities/runlatex.2022.01.25.sh      | 164 ++++++++++++++++++++++++++
>  utilities/runlatex.sh                 |  23 ++--
>  7 files changed, 314 insertions(+), 19 deletions(-)
>  create mode 100644 utilities/runfirstlatex.2022.01.25.sh
>  create mode 100644 utilities/runlatex.2022.01.25.sh
> 
> 
> base-commit: 87d3513257e793b90f53ebd4b138ab2371aaf25c
> -- 
> 2.25.1
>