[PATCH 0/5] Add a script to check for Sphinx install requirements

[PATCH 0/5] Add a script to check for Sphinx install requirements

[Mauro Carvalho Chehab]

Sphinx installation is not trivial, as not all versions are supported,
and it requires a lot of stuff for math, images and PDF/LaTeX output
to work.

So, add a script that checks if everything is fine, providing
distro-specific hints about what's needed for it to work.

I posted already RFC patches for the first patch in this series along
this weekend at linux-doc ML. The current version of the script
will provide hints for some popular distributions. I tested myself
on a couple of release-based and rolling popular distributions:

            - Fedora 25 and 26;
            - Ubuntu 17.04;
            - OpenSuse Tumbleweed;
            - Arch Linux;
            - Gentoo.

I tested compilation with both htmldocs and pdfdocs targets[1].
Yet, tests (and fix patches) are welcomed :-)

[1] There's an unrelated bug with the Kernel's sphinx extension
kimage: when parsing GraphViz graphs, it uses "-Tpdf" argument,
in order to generate a PDF image. That doesn't work on some
distros, as GraphViz doesn't support PDF images. 

Mauro Carvalho Chehab (5):
  scripts/sphinx-pre-install: add a script to check Sphinx install
  sphinx-pre-install: detect an existing virtualenv
  sphinx-pre-install: use a requirements file
  sphinx.rst: document scripts/sphinx-pre-install script
  doc: Makefile: if sphinx is not found, run a check script

 Documentation/Makefile             |   2 +
 Documentation/doc-guide/sphinx.rst |  48 +++-
 scripts/sphinx-pre-install         | 524 +++++++++++++++++++++++++++++++++++++
 3 files changed, 570 insertions(+), 4 deletions(-)
 create mode 100755 scripts/sphinx-pre-install

-- 
2.13.3

[PATCH 1/5] scripts/sphinx-pre-install: add a script to check Sphinx install

[Mauro Carvalho Chehab]

Solving Sphinx dependencies can be painful. Add a script to
check if everything is ok.

Tested on:
	- Fedora 25 and 26;
	- Ubuntu 17.04;
	- OpenSuse Tumbleweed;
	- Arch Linux;
	- Gentoo.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 scripts/sphinx-pre-install | 517 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 517 insertions(+)
 create mode 100755 scripts/sphinx-pre-install

diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
new file mode 100755
index 000000000000..5fe3b4a86174
--- /dev/null
+++ b/scripts/sphinx-pre-install
@@ -0,0 +1,517 @@
+#!/usr/bin/perl
+use strict;
+
+# Copyright (c) 2017 Mauro Carvalho Chehab <mchehab@kernel.org>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+#
+# Static vars
+#
+
+my %missing;
+my $system_release;
+my $need = 0;
+my $optional = 0;
+my $need_symlink = 0;
+my $need_sphinx = 0;
+my $install = "";
+
+#
+# Command line arguments
+#
+
+my $pdf = 1;
+my $virtualenv = 1;
+
+#
+# List of required texlive packages on Fedora and OpenSuse
+#
+
+my %texlive = (
+	'adjustbox.sty'      => 'texlive-adjustbox',
+	'amsfonts.sty'       => 'texlive-amsfonts',
+	'amsmath.sty'        => 'texlive-amsmath',
+	'amssymb.sty'        => 'texlive-amsfonts',
+	'amsthm.sty'         => 'texlive-amscls',
+	'anyfontsize.sty'    => 'texlive-anyfontsize',
+	'atbegshi.sty'       => 'texlive-oberdiek',
+	'bm.sty'             => 'texlive-tools',
+	'capt-of.sty'        => 'texlive-capt-of',
+	'cmap.sty'           => 'texlive-cmap',
+	'ecrm1000.tfm'       => 'texlive-ec',
+	'eqparbox.sty'       => 'texlive-eqparbox',
+	'eu1enc.def'         => 'texlive-euenc',
+	'fancybox.sty'       => 'texlive-fancybox',
+	'fancyvrb.sty'       => 'texlive-fancyvrb',
+	'float.sty'          => 'texlive-float',
+	'fncychap.sty'       => 'texlive-fncychap',
+	'footnote.sty'       => 'texlive-mdwtools',
+	'framed.sty'         => 'texlive-framed',
+	'luatex85.sty'       => 'texlive-luatex85',
+	'multirow.sty'       => 'texlive-multirow',
+	'needspace.sty'      => 'texlive-needspace',
+	'palatino.sty'       => 'texlive-psnfss',
+	'parskip.sty'        => 'texlive-parskip',
+	'polyglossia.sty'    => 'texlive-polyglossia',
+	'tabulary.sty'       => 'texlive-tabulary',
+	'threeparttable.sty' => 'texlive-threeparttable',
+	'titlesec.sty'       => 'texlive-titlesec',
+	'ucs.sty'            => 'texlive-ucs',
+	'upquote.sty'        => 'texlive-upquote',
+	'wrapfig.sty'        => 'texlive-wrapfig',
+);
+
+#
+# Subroutines that checks if a feature exists
+#
+
+sub check_missing(%)
+{
+	my %map = %{$_[0]};
+
+	foreach my $prog (sort keys %missing) {
+		my $is_optional = $missing{$prog};
+
+		if ($is_optional) {
+			print "Warning: better to also install \"$prog\".\n";
+		} else {
+			print "ERROR: please install \"$prog\", otherwise, build won't work.\n";
+		}
+		if (defined($map{$prog})) {
+			$install .= " " . $map{$prog};
+		} else {
+			$install .= " " . $prog;
+		}
+	}
+
+	$install =~ s/^\s//;
+}
+
+sub add_package($$)
+{
+	my $package = shift;
+	my $is_optional = shift;
+
+	$missing{$package} = $is_optional;
+	if ($is_optional) {
+		$optional++;
+	} else {
+		$need++;
+	}
+}
+
+sub check_missing_file($$$)
+{
+	my $file = shift;
+	my $package = shift;
+	my $is_optional = shift;
+
+	return if(-e $file);
+
+	add_package($package, $is_optional);
+}
+
+sub findprog($)
+{
+	foreach(split(/:/, $ENV{PATH})) {
+		return "$_/$_[0]" if(-x "$_/$_[0]");
+	}
+}
+
+sub check_program($$)
+{
+	my $prog = shift;
+	my $is_optional = shift;
+
+	return if findprog($prog);
+
+	add_package($prog, $is_optional);
+}
+
+sub check_perl_module($$)
+{
+	my $prog = shift;
+	my $is_optional = shift;
+
+	my $err = system("perl -M$prog -e 1 2>/dev/null /dev/null");
+	return if ($err == 0);
+
+	add_package($prog, $is_optional);
+}
+
+sub check_python_module($$)
+{
+	my $prog = shift;
+	my $is_optional = shift;
+
+	my $err = system("python3 -c 'import $prog' 2>/dev/null /dev/null");
+	return if ($err == 0);
+	my $err = system("python -c 'import $prog' 2>/dev/null /dev/null");
+	return if ($err == 0);
+
+	add_package($prog, $is_optional);
+}
+
+sub check_rpm_missing($$)
+{
+	my @pkgs = @{$_[0]};
+	my $is_optional = $_[1];
+
+	foreach my $prog(@pkgs) {
+		my $err = system("rpm -q '$prog' 2>/dev/null >/dev/null");
+		add_package($prog, $is_optional) if ($err);
+	}
+}
+
+sub check_pacman_missing($$)
+{
+	my @pkgs = @{$_[0]};
+	my $is_optional = $_[1];
+
+	foreach my $prog(@pkgs) {
+		my $err = system("pacman -Q '$prog' 2>/dev/null >/dev/null");
+		add_package($prog, $is_optional) if ($err);
+	}
+}
+
+sub check_missing_tex($)
+{
+	my $is_optional = shift;
+	my $kpsewhich = findprog("kpsewhich");
+
+	foreach my $prog(keys %texlive) {
+		my $package = $texlive{$prog};
+		if (!$kpsewhich) {
+			add_package($package, $is_optional);
+			next;
+		}
+		my $file = qx($kpsewhich $prog);
+		add_package($package, $is_optional) if ($file =~ /^\s*$/);
+	}
+}
+
+sub check_sphinx()
+{
+	return if findprog("sphinx-build");
+
+	if (findprog("sphinx-build-3")) {
+		$need_symlink = 1;
+		return;
+	}
+
+	if ($virtualenv) {
+		check_program("virtualenv", 0) if (!findprog("virtualenv-3"));
+		check_program("pip", 0) if (!findprog("pip3"));
+		$need_sphinx = 1;
+	} else {
+		add_package("python-sphinx", 0);
+	}
+}
+
+#
+# Ancillary subroutines
+#
+
+sub catcheck($)
+{
+  my $res = "";
+  $res = qx(cat $_[0]) if (-r $_[0]);
+  return $res;
+}
+
+sub which($)
+{
+	my $file = shift;
+	my @path = split ":", $ENV{PATH};
+
+	foreach my $dir(@path) {
+		my $name = $dir.'/'.$file;
+		return $name if (-x $name );
+	}
+	return undef;
+}
+
+#
+# Subroutines that check distro-specific hints
+#
+
+sub give_debian_hints()
+{
+	my %map = (
+		"python-sphinx"		=> "python3-sphinx",
+		"sphinx_rtd_theme"	=> "python3-sphinx-rtd-theme",
+		"virtualenv"		=> "virtualenv",
+		"pip"			=> "python3-pip",
+		"dot"			=> "graphviz",
+		"convert"		=> "imagemagick",
+		"Pod::Usage"		=> "perl-modules",
+		"xelatex"		=> "texlive-xetex",
+	);
+
+	if ($pdf) {
+		check_missing_file("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
+				   "fonts-dejavu", 1);
+	}
+
+	check_program("dvipng", 1) if ($pdf);
+	check_missing(\%map);
+
+	return if (!$need && !$optional);
+	printf("You should run:\n\n\tsudo apt-get install $install\n");
+}
+
+sub give_redhat_hints()
+{
+	my %map = (
+		"python-sphinx"		=> "python3-sphinx",
+		"sphinx_rtd_theme"	=> "python3-sphinx_rtd_theme",
+		"virtualenv"		=> "python3-virtualenv",
+		"pip"			=> "python3-pip",
+		"dot"			=> "graphviz",
+		"convert"		=> "ImageMagick",
+		"Pod::Usage"		=> "perl-Pod-Usage",
+		"xelatex"		=> "texlive-xetex-bin",
+	);
+
+	my @fedora_tex_pkgs = (
+		"texlive-collection-fontsrecommended",
+		"texlive-collection-latex",
+		"dejavu-sans-fonts",
+		"dejavu-serif-fonts",
+		"dejavu-sans-mono-fonts",
+	);
+
+	check_rpm_missing(\@fedora_tex_pkgs, 1) if ($pdf);
+	check_missing_tex(1) if ($pdf);
+	check_missing(\%map);
+
+	return if (!$need && !$optional);
+	printf("You should run:\n\n\tsudo dnf install -y $install\n");
+}
+
+sub give_opensuse_hints()
+{
+	my %map = (
+		"python-sphinx"		=> "python3-sphinx",
+		"sphinx_rtd_theme"	=> "python3-sphinx_rtd_theme",
+		"virtualenv"		=> "python3-virtualenv",
+		"pip"			=> "python3-pip",
+		"dot"			=> "graphviz",
+		"convert"		=> "ImageMagick",
+		"Pod::Usage"		=> "perl-Pod-Usage",
+		"xelatex"		=> "texlive-xetex-bin",
+	);
+
+	my @suse_tex_pkgs = (
+		"texlive-babel-english",
+		"texlive-caption",
+		"texlive-colortbl",
+		"texlive-courier",
+		"texlive-dvips",
+		"texlive-helvetic",
+		"texlive-makeindex",
+		"texlive-metafont",
+		"texlive-metapost",
+		"texlive-palatino",
+		"texlive-preview",
+		"texlive-times",
+		"texlive-zapfchan",
+		"texlive-zapfding",
+	);
+
+	check_rpm_missing(\@suse_tex_pkgs, 1) if ($pdf);
+	check_missing_tex(1) if ($pdf);
+	check_missing(\%map);
+
+	return if (!$need && !$optional);
+	printf("You should run:\n\n\tsudo zypper install --no-recommends $install\n");
+}
+
+sub give_arch_linux_hints()
+{
+	my %map = (
+		"sphinx_rtd_theme"	=> "python-sphinx_rtd_theme",
+		"virtualenv"		=> "python-virtualenv",
+		"pip"			=> "python-pip",
+		"dot"			=> "graphviz",
+		"convert"		=> "imagemagick",
+		"xelatex"		=> "texlive-bin",
+	);
+
+	my @archlinux_tex_pkgs = (
+		"texlive-core",
+		"texlive-latexextra",
+		"ttf-dejavu",
+	);
+	check_pacman_missing(\@archlinux_tex_pkgs, 1) if ($pdf);
+	check_missing(\%map);
+
+	return if (!$need && !$optional);
+	printf("You should run:\n\n\tsudo pacman -S $install\n");
+}
+
+sub give_gentoo_hints()
+{
+	my %map = (
+		"sphinx_rtd_theme"	=> "dev-python/sphinx_rtd_theme",
+		"virtualenv"		=> "dev-python/virtualenv",
+		"pip"			=> "dev-python/pip",
+		"dot"			=> "media-gfx/graphviz",
+		"convert"		=> "media-gfx/imagemagick",
+		"xelatex"		=> "dev-texlive/texlive-xetex media-fonts/dejavu",
+	);
+
+	check_missing_file("/usr/share/fonts/dejavu/DejaVuSans.ttf",
+			   "media-fonts/dejavu", 1) if ($pdf);
+
+	check_missing(\%map);
+
+	return if (!$need && !$optional);
+	printf("You should run:\n\n\tsudo emerge --ask $install\n");
+}
+
+sub check_distros()
+{
+	# Distro-specific hints
+	if ($system_release =~ /Red Hat Enterprise Linux/) {
+		give_redhat_hints;
+		return;
+	}
+	if ($system_release =~ /Fedora/) {
+		give_redhat_hints;
+		return;
+	}
+	if ($system_release =~ /Ubuntu/) {
+		give_debian_hints;
+		return;
+	}
+	if ($system_release =~ /Debian/) {
+		give_debian_hints;
+		return;
+	}
+	if ($system_release =~ /openSUSE/) {
+		give_opensuse_hints;
+		return;
+	}
+	if ($system_release =~ /Arch Linux/) {
+		give_arch_linux_hints;
+		return;
+	}
+	if ($system_release =~ /Gentoo/) {
+		give_gentoo_hints;
+		return;
+	}
+
+	#
+	# Fall-back to generic hint code for other distros
+	# That's far from ideal, specially for LaTeX dependencies.
+	#
+	my %map = (
+		"sphinx-build" => "sphinx"
+	);
+	check_missing_tex(1) if ($pdf);
+	check_missing(\%map);
+	print "I don't know distro $system_release.\n";
+	print "So, I can't provide you a hint with the install procedure.\n";
+	print "There are likely missing dependencies.\n";
+}
+
+#
+# Common dependencies
+#
+
+sub check_needs()
+{
+	if ($system_release) {
+		print "Checking if the needed tools for $system_release are available\n";
+	} else {
+		print "Checking if the needed tools are present\n";
+	}
+
+	# Check for needed programs/tools
+	check_sphinx();
+	check_perl_module("Pod::Usage", 0);
+	check_program("make", 0);
+	check_program("gcc", 0);
+	check_python_module("sphinx_rtd_theme", 1) if (!$virtualenv);
+	check_program("xelatex", 1) if ($pdf);
+	check_program("dot", 1);
+	check_program("convert", 1);
+
+	check_distros();
+
+	if ($need_symlink) {
+		printf "\tsudo ln -sf %s /usr/bin/sphinx-build\n\n",
+		       which("sphinx-build-3");
+	}
+	if ($need_sphinx) {
+		my $virtualenv = findprog("virtualenv-3");
+		$virtualenv = findprog("virtualenv") if (!$virtualenv);
+		$virtualenv = "virtualenv" if (!$virtualenv);
+
+		printf "\t$virtualenv sphinx_1.4\n";
+		printf "\t. sphinx_1.4/bin/activate\n";
+		printf "\tpip install 'docutils==0.12'\n";
+		printf "\tpip install 'Sphinx==1.4.9'\n";
+		printf "\tpip install sphinx_rtd_theme\n";
+		$need++;
+	}
+	printf "\n";
+
+	print "All optional dependenties are met.\n" if (!$optional);
+
+	if ($need == 1) {
+		die "Can't build as $need mandatory dependency is missing";
+	} elsif ($need) {
+		die "Can't build as $need mandatory dependencies are missing";
+	}
+
+	print "Needed package dependencies are met.\n";
+}
+
+#
+# Main
+#
+
+while (@ARGV) {
+	my $arg = shift(@ARGV);
+
+	if ($arg eq "--no-virtualenv") {
+		$virtualenv = 0;
+	} elsif ($arg eq "--no-pdf"){
+		$pdf = 0;
+	} else {
+		print "Usage:\n\t$0 <--no-virtualenv> <--no-pdf>\n\n";
+		exit -1;
+	}
+}
+
+#
+# Determine the system type. There's no standard unique way that would
+# work with all distros with a minimal package install. So, several
+# methods are used here.
+#
+# By default, it will use lsb_release function. If not available, it will
+# fail back to reading the known different places where the distro name
+# is stored
+#
+
+$system_release = qx(lsb_release -d) if which("lsb_release");
+$system_release =~ s/Description:\s*// if ($system_release);
+$system_release = catcheck("/etc/system-release") if !$system_release;
+$system_release = catcheck("/etc/redhat-release") if !$system_release;
+$system_release = catcheck("/etc/lsb-release") if !$system_release;
+$system_release = catcheck("/etc/gentoo-release") if !$system_release;
+$system_release = catcheck("/etc/issue") if !$system_release;
+$system_release =~ s/\s+$//;
+
+check_needs;
-- 
2.13.3

[PATCH 2/5] sphinx-pre-install: detect an existing virtualenv

[Mauro Carvalho Chehab]

Detect if the script runs after creating the virtualenv,
printing the command line commands to enable the virtualenv.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 scripts/sphinx-pre-install | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)

diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 5fe3b4a86174..bcd6f7978f8d 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -13,6 +13,8 @@ use strict;
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 
+my $virtenv_dir = "sphinx_1.4";
+
 #
 # Static vars
 #
@@ -454,16 +456,22 @@ sub check_needs()
 		       which("sphinx-build-3");
 	}
 	if ($need_sphinx) {
-		my $virtualenv = findprog("virtualenv-3");
-		$virtualenv = findprog("virtualenv") if (!$virtualenv);
-		$virtualenv = "virtualenv" if (!$virtualenv);
+		my $activate = "$virtenv_dir/bin/activate";
+		if (-e "$ENV{'PWD'}/$activate") {
+			printf "\nNeed to activate virtualenv with:\n";
+			printf "\t. $activate\n";
+		} else {
+			my $virtualenv = findprog("virtualenv-3");
+			$virtualenv = findprog("virtualenv") if (!$virtualenv);
+			$virtualenv = "virtualenv" if (!$virtualenv);
 
-		printf "\t$virtualenv sphinx_1.4\n";
-		printf "\t. sphinx_1.4/bin/activate\n";
-		printf "\tpip install 'docutils==0.12'\n";
-		printf "\tpip install 'Sphinx==1.4.9'\n";
-		printf "\tpip install sphinx_rtd_theme\n";
-		$need++;
+			printf "\t$virtualenv $virtenv_dir\n";
+			printf "\t. $activate\n";
+			printf "\tpip install 'docutils==0.12'\n";
+			printf "\tpip install 'Sphinx==1.4.9'\n";
+			printf "\tpip install sphinx_rtd_theme\n";
+			$need++;
+		}
 	}
 	printf "\n";
 
-- 
2.13.3

[PATCH 3/5] sphinx-pre-install: use a requirements file

[Mauro Carvalho Chehab]

Instead of using 3 commands to install a virtualenv, use
a single one, reading the requirements from this file:

	Documentation/sphinx/requirements.txt

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 scripts/sphinx-pre-install | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index bcd6f7978f8d..230d9cc7ee1d 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -14,6 +14,7 @@ use strict;
 # GNU General Public License for more details.
 
 my $virtenv_dir = "sphinx_1.4";
+my $requirement_file = "Documentation/sphinx/requirements.txt";
 
 #
 # Static vars
@@ -467,9 +468,7 @@ sub check_needs()
 
 			printf "\t$virtualenv $virtenv_dir\n";
 			printf "\t. $activate\n";
-			printf "\tpip install 'docutils==0.12'\n";
-			printf "\tpip install 'Sphinx==1.4.9'\n";
-			printf "\tpip install sphinx_rtd_theme\n";
+			printf "\tpip install -r $requirement_file\n";
 			$need++;
 		}
 	}
-- 
2.13.3

[PATCH 4/5] sphinx.rst: document scripts/sphinx-pre-install script

[Mauro Carvalho Chehab]

Now that we have a script to check for Sphinx dependencies,
document it.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 Documentation/doc-guide/sphinx.rst | 48 ++++++++++++++++++++++++++++++++++----
 1 file changed, 44 insertions(+), 4 deletions(-)

diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index 5521213efa68..8faafb9b2d86 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -28,6 +28,9 @@ The ReST markups currently used by the Documentation/ files are meant to be
 built with ``Sphinx`` version 1.3 or upper. If you're desiring to build
 PDF outputs, it is recommended to use version 1.4.6 or upper.
 
+There's a script that checks for the Spinx requirements. Please see
+:ref:`sphinx-pre-install` for further details.
+
 Most distributions are shipped with Sphinx, but its toolchain is fragile,
 and it is not uncommon that upgrading it or some other Python packages
 on your machine would cause the documentation build to break.
@@ -47,13 +50,15 @@ or ``virtualenv``, depending on how your distribution packaged Python 3.
       on the Sphinx version, it should be installed  in separate,
       with ``pip install sphinx_rtd_theme``.
 
+   #) Some ReST pages contain math expressions. Due to the way Sphinx work,
+      those expressions are written using LaTeX notation. It needs texlive
+      installed with amdfonts and amsmath in order to evaluate them.
+
 In summary, if you want to install Sphinx version 1.4.9, you should do::
 
        $ virtualenv sphinx_1.4
        $ . sphinx_1.4/bin/activate
-       (sphinx_1.4) $ pip install 'docutils==0.12'
-       (sphinx_1.4) $ pip install 'Sphinx==1.4.9'
-       (sphinx_1.4) $ pip install sphinx_rtd_theme
+       (sphinx_1.4) $ pip install -r Documentation/sphinx/requirements.txt
 
 After running ``. sphinx_1.4/bin/activate``, the prompt will change,
 in order to indicate that you're using the new environment. If you
@@ -83,7 +88,42 @@ For PDF and LaTeX output, you'll also need ``XeLaTeX`` version 3.14159265.
 
 Depending on the distribution, you may also need to install a series of
 ``texlive`` packages that provide the minimal set of functionalities
-required for ``XeLaTex`` to work.
+required for ``XeLaTeX`` to work.
+
+.. _sphinx-pre-install:
+
+Checking for Sphinx dependencies
+--------------------------------
+
+There's a script that automatically check for Sphinx dependencies. If it can
+recognize your distribution, it will also give a hint about the install
+command line options for your distro::
+
+	$ ./scripts/sphinx-pre-install
+	Checking if the needed tools for Fedora release 26 (Twenty Six) are available
+	Warning: better to also install "texlive-luatex85".
+	You should run:
+
+		sudo dnf install -y texlive-luatex85
+		/usr/bin/virtualenv sphinx_1.4
+		. sphinx_1.4/bin/activate
+		pip install -r Documentation/sphinx/requirements.txt
+
+	Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468.
+
+By default, it checks all the requirements for both html and PDF, including
+the requirements for images, math expressions and LaTeX build, and assumes
+that a virtual Python environment will be used. The ones needed for html
+builds are assumed to be mandatory; the others to be optional.
+
+It supports two optional parameters:
+
+``--no-pdf``
+	Disable checks for PDF;
+
+``--no-virtualenv``
+	Use OS packaging for Sphinx instead of Python virtual environment.
+
 
 Sphinx Build
 ============
-- 
2.13.3

[PATCH 5/5] doc: Makefile: if sphinx is not found, run a check script

[Mauro Carvalho Chehab]

Right now, if the building system doesn't find Sphinx, it
bails out, without providing any instructions about what
should be done.

Instead, run a script, providing some guidance about the
steps needed for Sphinx build to work.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 Documentation/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index d75c00e3aadb..85f7856f0092 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -22,6 +22,8 @@ ifeq ($(HAVE_SPHINX),0)
 
 .DEFAULT:
 	$(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.)
+	@echo
+	@./scripts/sphinx-pre-install
 	@echo "  SKIP    Sphinx $@ target."
 
 else # HAVE_SPHINX
-- 
2.13.3

Re: [PATCH 3/5] sphinx-pre-install: use a requirements file

[Markus Heiser]

> Am 17.07.2017 um 00:08 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> 
> Instead of using 3 commands to install a virtualenv, use
> a single one, reading the requirements from this file:
> 
> 	Documentation/sphinx/requirements.txt
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

Hi Mauro,

I get a ..

 fatal: sha1 information is lacking or useless (Documentation/doc-guide/sphinx.rst)

and  I miss the Documentation/doc-guide/sphinx.rst file in this patch.

Did it only happened to me?

-- Markus --

Re: [PATCH 0/5] Add a script to check for Sphinx install requirements

[Markus Heiser]

> Am 17.07.2017 um 00:08 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> 
> [1] There's an unrelated bug with the Kernel's sphinx extension
> kimage: when parsing GraphViz graphs, it uses "-Tpdf" argument,
> in order to generate a PDF image. That doesn't work on some
> distros, as GraphViz doesn't support PDF images. 
> 

Hmm .. http://www.graphviz.org/content/output-formats#dpdf

Is it a problem of the GraphViz version?

See http://www.graphviz.org/News.php ..

  """New Release 2.32 (1 August 2013)

     ... In addition, if the poppler library is available, Graphviz
     can now use PDF files as images."""

I suppose that 2.32 is in your distro .. it's from 2013

It seems, that the graphiz homepage is not up to date. I found 
the repo at github. There is a issue about PDF

  https://github.com/ellson/graphviz/issues/1180

So I guess graphviz is compiled without HAVE_PANGOCAIRO in your
distros?

-- Markus --

Re: [PATCH 0/5] Add a script to check for Sphinx install requirements

[Mauro Carvalho Chehab]

Em Mon, 17 Jul 2017 10:15:26 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> > Am 17.07.2017 um 00:08 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> > 
> > [1] There's an unrelated bug with the Kernel's sphinx extension
> > kimage: when parsing GraphViz graphs, it uses "-Tpdf" argument,
> > in order to generate a PDF image. That doesn't work on some
> > distros, as GraphViz doesn't support PDF images. 
> >   
> 
> Hmm .. http://www.graphviz.org/content/output-formats#dpdf
> 
> Is it a problem of the GraphViz version?
> 
> See http://www.graphviz.org/News.php ..
> 
>   """New Release 2.32 (1 August 2013)
> 
>      ... In addition, if the poppler library is available, Graphviz
>      can now use PDF files as images."""
> 
> I suppose that 2.32 is in your distro .. it's from 2013

Fedora 26 is shipped with version 2.40:

	$ rpm -q graphviz
	graphviz-2.40.1-4.fc26.x86_64

Yet:

$ dot -Thelp|grep pdf
Format: "help" not recognized. Use one of: canon cmap cmapx cmapx_np dot dot_json eps fig gv imap imap_np ismap json json0 mp pic plain plain-ext pov ps ps2 svg svgz tk vdx vml vmlz xdot xdot1.2 xdot1.4 xdot_json

I actually noticed this issue first on OpenSuse, where GraphViz is
packaged on several packages, but I was unable to find one package
there with pdf suport. I'm pretty sure pdf used to work on F25.

> 
> It seems, that the graphiz homepage is not up to date. I found 
> the repo at github. There is a issue about PDF
> 
>   https://github.com/ellson/graphviz/issues/1180
> 
> So I guess graphviz is compiled without HAVE_PANGOCAIRO in your
> distros?


>From Fedora source package, it is requiring both cairo and pango:

	BuildRequires:		cairo-devel >= 1.1.10, pango-devel, gmp-devel, lua-devel, gtk2-devel, libgnomeui-devel

Building it gets:

	checking for PANGOCAIRO... yes

Still, no PDF support. I'll try to investigate it further, as I
found myself useful to produce PDF directly from GraphViz.

Anyway, I guess we should modify kfigure.py to check if PDF is
available, falling back to SVG, using ImageMagick to convert
from SVG to PDF.


Thanks,
Mauro

Re: [PATCH 3/5] sphinx-pre-install: use a requirements file

[Mauro Carvalho Chehab]

Em Mon, 17 Jul 2017 08:00:07 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> > Am 17.07.2017 um 00:08 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> > 
> > Instead of using 3 commands to install a virtualenv, use
> > a single one, reading the requirements from this file:
> > 
> > 	Documentation/sphinx/requirements.txt
> > 
> > Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>  
> 
> Hi Mauro,
> 
> I get a ..
> 
>  fatal: sha1 information is lacking or useless (Documentation/doc-guide/sphinx.rst)
> 
> and  I miss the Documentation/doc-guide/sphinx.rst file in this patch.
> 
> Did it only happened to me?

Sorry, I forgot to mention. This patch depends on the patch series I
posted before adding information about the Sphinx install procedures:

	[PATCH v2 0/6] Better document Sphinx install instructions
	https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1444295.html


Thanks,
Mauro

Re: [PATCH 0/5] Add a script to check for Sphinx install requirements

[Mauro Carvalho Chehab]

Em Mon, 17 Jul 2017 06:33:03 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:

> Em Mon, 17 Jul 2017 10:15:26 +0200
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
> 
> > > Am 17.07.2017 um 00:08 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> > > 
> > > [1] There's an unrelated bug with the Kernel's sphinx extension
> > > kimage: when parsing GraphViz graphs, it uses "-Tpdf" argument,
> > > in order to generate a PDF image. That doesn't work on some
> > > distros, as GraphViz doesn't support PDF images. 
> > >     
> > 
> > Hmm .. http://www.graphviz.org/content/output-formats#dpdf
> > 
> > Is it a problem of the GraphViz version?
> > 
> > See http://www.graphviz.org/News.php ..
> > 
> >   """New Release 2.32 (1 August 2013)
> > 
> >      ... In addition, if the poppler library is available, Graphviz
> >      can now use PDF files as images."""
> > 
> > I suppose that 2.32 is in your distro .. it's from 2013  
> 
> Fedora 26 is shipped with version 2.40:
> 
> 	$ rpm -q graphviz
> 	graphviz-2.40.1-4.fc26.x86_64
> 
> Yet:
> 
> $ dot -Thelp|grep pdf
> Format: "help" not recognized. Use one of: canon cmap cmapx cmapx_np dot dot_json eps fig gv imap imap_np ismap json json0 mp pic plain plain-ext pov ps ps2 svg svgz tk vdx vml vmlz xdot xdot1.2 xdot1.4 xdot_json
> 
> I actually noticed this issue first on OpenSuse, where GraphViz is
> packaged on several packages, but I was unable to find one package
> there with pdf suport. I'm pretty sure pdf used to work on F25.
> 
> > 
> > It seems, that the graphiz homepage is not up to date. I found 
> > the repo at github. There is a issue about PDF
> > 
> >   https://github.com/ellson/graphviz/issues/1180
> > 
> > So I guess graphviz is compiled without HAVE_PANGOCAIRO in your
> > distros?  
> 
> 
> From Fedora source package, it is requiring both cairo and pango:
> 
> 	BuildRequires:		cairo-devel >= 1.1.10, pango-devel, gmp-devel, lua-devel, gtk2-devel, libgnomeui-devel
> 
> Building it gets:
> 
> 	checking for PANGOCAIRO... yes
> 
> Still, no PDF support. I'll try to investigate it further, as I
> found myself useful to produce PDF directly from GraphViz.
> 
> Anyway, I guess we should modify kfigure.py to check if PDF is
> available, falling back to SVG, using ImageMagick to convert
> from SVG to PDF.

Ok, I discovered that, on Fedora, support for pdf, png and some other
formats are packaged on a separate package: graphviz-gd.x86_64

With it installed, it now shows "pdf":

$ dot -Thelp
Format: "help" not recognized. Use one of: bmp canon cmap cmapx cmapx_np dot dot_json eps fig gd gd2 gif gtk gv ico imap imap_np ismap jpe jpeg jpg json json0 mp pdf pic plain plain-ext png pov ps ps2 svg svgz tif tiff tk vdx vml vmlz vrml wbmp x11 xdot xdot1.2 xdot1.4 xdot_json xlib

Thanks,
Mauro

Re: [PATCH 0/5] Add a script to check for Sphinx install requirements

[Markus Heiser]

> Am 17.07.2017 um 12:09 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> 
>> 
>> Anyway, I guess we should modify kfigure.py to check if PDF is
>> available, falling back to SVG, using ImageMagick to convert
>> from SVG to PDF.
> 
> Ok, I discovered that, on Fedora, support for pdf, png and some other
> formats are packaged on a separate package: graphviz-gd.x86_64
> 
> With it installed, it now shows "pdf":
> 

Ah, OK .. thanks a lot for clarifying.

-- Markus --

Re: [PATCH 0/5] Add a script to check for Sphinx install requirements

[Mauro Carvalho Chehab]

Em Mon, 17 Jul 2017 07:09:46 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:

> Em Mon, 17 Jul 2017 06:33:03 -0300
> Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:
> 
> > Em Mon, 17 Jul 2017 10:15:26 +0200
> > Markus Heiser <markus.heiser@darmarit.de> escreveu:
> >   
> > > > Am 17.07.2017 um 00:08 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> > > > 
> > > > [1] There's an unrelated bug with the Kernel's sphinx extension
> > > > kimage: when parsing GraphViz graphs, it uses "-Tpdf" argument,
> > > > in order to generate a PDF image. That doesn't work on some
> > > > distros, as GraphViz doesn't support PDF images. 
> > > >       
> > > 
> > > Hmm .. http://www.graphviz.org/content/output-formats#dpdf
> > > 
> > > Is it a problem of the GraphViz version?
> > > 
> > > See http://www.graphviz.org/News.php ..
> > > 
> > >   """New Release 2.32 (1 August 2013)
> > > 
> > >      ... In addition, if the poppler library is available, Graphviz
> > >      can now use PDF files as images."""
> > > 
> > > I suppose that 2.32 is in your distro .. it's from 2013    
> > 
> > Fedora 26 is shipped with version 2.40:
> > 
> > 	$ rpm -q graphviz
> > 	graphviz-2.40.1-4.fc26.x86_64
> > 
> > Yet:
> > 
> > $ dot -Thelp|grep pdf
> > Format: "help" not recognized. Use one of: canon cmap cmapx cmapx_np dot dot_json eps fig gv imap imap_np ismap json json0 mp pic plain plain-ext pov ps ps2 svg svgz tk vdx vml vmlz xdot xdot1.2 xdot1.4 xdot_json
> > 
> > I actually noticed this issue first on OpenSuse, where GraphViz is
> > packaged on several packages, but I was unable to find one package
> > there with pdf suport. I'm pretty sure pdf used to work on F25.
> >   
> > > 
> > > It seems, that the graphiz homepage is not up to date. I found 
> > > the repo at github. There is a issue about PDF
> > > 
> > >   https://github.com/ellson/graphviz/issues/1180
> > > 
> > > So I guess graphviz is compiled without HAVE_PANGOCAIRO in your
> > > distros?    
> > 
> > 
> > From Fedora source package, it is requiring both cairo and pango:
> > 
> > 	BuildRequires:		cairo-devel >= 1.1.10, pango-devel, gmp-devel, lua-devel, gtk2-devel, libgnomeui-devel
> > 
> > Building it gets:
> > 
> > 	checking for PANGOCAIRO... yes
> > 
> > Still, no PDF support. I'll try to investigate it further, as I
> > found myself useful to produce PDF directly from GraphViz.
> > 
> > Anyway, I guess we should modify kfigure.py to check if PDF is
> > available, falling back to SVG, using ImageMagick to convert
> > from SVG to PDF.  
> 
> Ok, I discovered that, on Fedora, support for pdf, png and some other
> formats are packaged on a separate package: graphviz-gd.x86_
> 
> With it installed, it now shows "pdf":
> 
> $ dot -Thelp
> Format: "help" not recognized. Use one of: bmp canon cmap cmapx cmapx_np dot dot_json eps fig gd gd2 gif gtk gv ico imap imap_np ismap jpe jpeg jpg json json0 mp pdf pic plain plain-ext png pov ps ps2 svg svgz tif tiff tk vdx vml vmlz vrml wbmp x11 xdot xdot1.2 xdot1.4 xdot_json xlib

Double-checked on all VMs I used to test this pachset:

- on Fedora 25, Debian 17.04, Gentoo and ArchLinux, just graphviz
  is enough for PDF support.

- on Fedora 26, graphviz-gd is needed for PDF support;

- on OpenSuse Tumbleweed, I was unable to discover a package that
  would make GraphViz to support pdf. There, it uses graphviz
  version 2.38, with is the same version as used on Fedora 25.

That is with just graphviz:

	mchehab@OpenSuse:~/docs> dot -Thelp
	Format: "help" not recognized. Use one of: canon cmap cmapx cmapx_np dot eps fig gv imap imap_np ismap pic plain plain-ext pov ps ps2 svg svgz tk vml vmlz xdot xdot1.2 xdot1.4

And that is with graphviz-gd:

	mchehab@OpenSuse:~/docs> dot -Thelp
	Format: "help" not recognized. Use one of: canon cmap cmapx cmapx_np dot eps fig gd gd2 gif gv imap imap_np ismap jpe jpeg jpg pic plain plain-ext png pov ps ps2 svg svgz tk vml vmlz vrml wbmp xdot xdot1.2 xdot1.4

So, we need a way for kfigure to fallback when distros don't
have such feature.

Regards,
Mauro

Re: [PATCH 0/5] Add a script to check for Sphinx install requirements

[Markus Heiser]

> Am 17.07.2017 um 12:57 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> 
> So, we need a way for kfigure to fallback when distros don't
> have such feature.

Hm .. I'am not very happy to implement where distros packaging fail.
But .. if there is really a need for, I will do so.

Lets see how this series goes upstream and feel free to remind
on my words if I completely suppressed the idea after upstream. :) 

-- Markus --