[PATCH 0/5] Improvements to the documentation build system

[PATCH 0/5] Improvements to the documentation build system

[Mauro Carvalho Chehab]

The first three patches here address some issues with the script which
detects the requirements for running Sphinx, making it a smarter and
running it all the times, in order to allow recommending version
updates.

The 4th patch adds "-jauto" by default, if Sphinx version supports it
(e. g. version 1.7 or upper).

The final patch changes the recommendation to use at least Sphinx
version 1.7.9, as discussed at linux-doc ML.

Mauro Carvalho Chehab (5):
  scripts/sphinx-pre-install: make activate hint smarter
  scripts/sphinx-pre-install: get rid of RHEL7 explicity check
  scripts/sphinx-pre-install: always check if version is compatible with
    build
  docs: by default, build docs a lot faster with Sphinx >= 1.7
  docs: requirements.txt: recommend Sphinx 1.7.9

 Documentation/Makefile                |  7 +++
 Documentation/doc-guide/sphinx.rst    | 17 +++---
 Documentation/sphinx/requirements.txt |  4 +-
 scripts/sphinx-pre-install            | 75 +++++++++++++++------------
 4 files changed, 58 insertions(+), 45 deletions(-)

-- 
2.21.0

[PATCH 1/5] scripts/sphinx-pre-install: make activate hint smarter

[Mauro Carvalho Chehab]

It is possible that multiple Sphinx virtualenvs are installed
on a given kernel tree. Change the logic to get the latest
version of those, as this is probably what the user wants.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
 scripts/sphinx-pre-install | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 8c2d1bcf2e02..11239eb29695 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -1,7 +1,7 @@
 #!/usr/bin/perl
 use strict;
 
-# Copyright (c) 2017 Mauro Carvalho Chehab <mchehab@kernel.org>
+# Copyright (c) 2017-2019 Mauro Carvalho Chehab <mchehab@kernel.org>
 #
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
@@ -15,6 +15,7 @@ use strict;
 
 my $conf = "Documentation/conf.py";
 my $requirement_file = "Documentation/sphinx/requirements.txt";
+my $virtenv_prefix = "sphinx_";
 
 #
 # Static vars
@@ -28,7 +29,8 @@ my $need_symlink = 0;
 my $need_sphinx = 0;
 my $rec_sphinx_upgrade = 0;
 my $install = "";
-my $virtenv_dir = "sphinx_";
+my $virtenv_dir = "";
+my $min_version;
 
 #
 # Command line arguments
@@ -229,7 +231,6 @@ sub get_sphinx_fname()
 
 sub check_sphinx()
 {
-	my $min_version;
 	my $rec_version;
 	my $cur_version;
 
@@ -255,7 +256,7 @@ sub check_sphinx()
 
 	die "Can't get recommended sphinx version from $requirement_file" if (!$min_version);
 
-	$virtenv_dir .= $rec_version;
+	$virtenv_dir = $virtenv_prefix . $rec_version;
 
 	my $sphinx = get_sphinx_fname();
 	return if ($sphinx eq "");
@@ -612,18 +613,23 @@ sub check_needs()
 		       which("sphinx-build-3");
 	}
 	if ($need_sphinx || $rec_sphinx_upgrade) {
-		my $activate = "$virtenv_dir/bin/activate";
-		if (-e "$ENV{'PWD'}/$activate") {
+		my $min_activate = "$ENV{'PWD'}/${virtenv_prefix}${min_version}/bin/activate";
+                my @activates = glob "$ENV{'PWD'}/${virtenv_prefix}*/bin/activate";
+
+                @activates = sort {$b cmp $a} @activates;
+
+		if (scalar @activates > 0 && $activates[0] ge $min_activate) {
 			printf "\nNeed to activate virtualenv with:\n";
-			printf "\t. $activate\n";
+			printf "\t. $activates[0]\n";
 		} else {
+			my $rec_activate = "$virtenv_dir/bin/activate";
 			my $virtualenv = findprog("virtualenv-3");
 			$virtualenv = findprog("virtualenv-3.5") if (!$virtualenv);
 			$virtualenv = findprog("virtualenv") if (!$virtualenv);
 			$virtualenv = "virtualenv" if (!$virtualenv);
 
 			printf "\t$virtualenv $virtenv_dir\n";
-			printf "\t. $activate\n";
+			printf "\t. $rec_activate\n";
 			printf "\tpip install -r $requirement_file\n";
 
 			$need++ if (!$rec_sphinx_upgrade);
-- 
2.21.0

[PATCH 2/5] scripts/sphinx-pre-install: get rid of RHEL7 explicity check

[Mauro Carvalho Chehab]

RHEL8 was already launched. This test won't get it, and will
do the wrong thing. Ok, we could fix it, but now we check
Sphinx version to ensure that it matches the minimal (1.3),
so there's no need for an explicit check there.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
 scripts/sphinx-pre-install | 13 -------------
 1 file changed, 13 deletions(-)

diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 11239eb29695..ded3e2ef3f8d 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -581,19 +581,6 @@ sub check_needs()
 		print "Unknown OS\n";
 	}
 
-	# RHEL 7.x and clones have Sphinx version 1.1.x and incomplete texlive
-	if (($system_release =~ /Red Hat Enterprise Linux/) ||
-	    ($system_release =~ /CentOS/) ||
-	    ($system_release =~ /Scientific Linux/) ||
-	    ($system_release =~ /Oracle Linux Server/)) {
-		$virtualenv = 1;
-		$pdf = 0;
-
-		printf("NOTE: On this distro, Sphinx and TexLive shipped versions are incompatible\n");
-		printf("with doc build. So, use Sphinx via a Python virtual environment.\n\n");
-		printf("This script can't install a TexLive version that would provide PDF.\n");
-	}
-
 	# Check for needed programs/tools
 	check_sphinx();
 	check_perl_module("Pod::Usage", 0);
-- 
2.21.0

[PATCH 3/5] scripts/sphinx-pre-install: always check if version is compatible with build

[Mauro Carvalho Chehab]

Call the script every time a make docs target is selected, on
a simplified check mode.

With this change, the script will set two vars:

$min_version - obtained from `needs_sphinx` var inside
	       conf.py (currently, '1.3')

$rec_version - obtained from sphinx/requirements.txt.

With those changes, a target like "make htmldocs" will do:

1) If no sphinx-build/sphinx-build3 is found, it will run
   the script on normal mode as before, checking for all
   system dependencies and providing install hints for the
   needed programs and will abort the build;

2) If no sphinx-build/sphinx-build3 is found, but there is
   a sphinx_${VER}/bin/activate file, and if
   ${VER} >= $min_version (string comparation), it will
   run in full mode, and will recommend to activate the
   virtualenv. If there are multiple virtualenvs, it
   will string sort the versions, recommending the
   highest version and will abort the build;

3) If Sphinx is detected but has a version lower than
   $min_version, it will run in full mode - with will
   recommend creating a virtual env using sphinx/requirements.txt,
   and will abort the build.

4) If Sphinx is detected and version is lower than
   $rec_version, it will run in full mode and will
   recommend creating a virtual env using sphinx/requirements.txt.

   In this case, it **won't** abort the build.

5) If Sphinx is detected and version is equal or righer than
   $rec_version it will return just after detecting the
   version ("quick mode"), not checking if are there any
   missing dependencies.

Just like before, if one wants to install Sphinx from the
distro, it has to call the script manually and use `--no-virtualenv`
argument to get the hints for his OS:

    You should run:

	sudo dnf install -y python3-sphinx python3-sphinx_rtd_theme

While here, add a small help for the three optional arguments
for the script.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
 Documentation/Makefile     |  5 +++++
 scripts/sphinx-pre-install | 46 +++++++++++++++++++++++++-------------
 2 files changed, 35 insertions(+), 16 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index e889e7cb8511..380e24053d6f 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -70,12 +70,14 @@ quiet_cmd_sphinx = SPHINX  $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
 	$(abspath $(BUILDDIR)/$3/$4)
 
 htmldocs:
+	@./scripts/sphinx-pre-install --version-check
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))
 
 linkcheckdocs:
 	@$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,linkcheck,$(var),,$(var)))
 
 latexdocs:
+	@./scripts/sphinx-pre-install --version-check
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,latex,$(var),latex,$(var)))
 
 ifeq ($(HAVE_PDFLATEX),0)
@@ -87,14 +89,17 @@ pdfdocs:
 else # HAVE_PDFLATEX
 
 pdfdocs: latexdocs
+	@./scripts/sphinx-pre-install --version-check
 	$(foreach var,$(SPHINXDIRS), $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" -C $(BUILDDIR)/$(var)/latex || exit;)
 
 endif # HAVE_PDFLATEX
 
 epubdocs:
+	@./scripts/sphinx-pre-install --version-check
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var)))
 
 xmldocs:
+	@./scripts/sphinx-pre-install --version-check
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,xml,$(var),xml,$(var)))
 
 endif # HAVE_SPHINX
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index ded3e2ef3f8d..f001fc2fcf12 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -38,6 +38,7 @@ my $min_version;
 
 my $pdf = 1;
 my $virtualenv = 1;
+my $version_check = 0;
 
 #
 # List of required texlive packages on Fedora and OpenSuse
@@ -277,20 +278,22 @@ sub check_sphinx()
 
 	die "$sphinx didn't return its version" if (!$cur_version);
 
-	printf "Sphinx version %s (minimal: %s, recommended >= %s)\n",
-		$cur_version, $min_version, $rec_version;
-
 	if ($cur_version lt $min_version) {
-		print "Warning: Sphinx version should be >= $min_version\n\n";
+		printf "ERROR: Sphinx version is %s. It should be >= %s (recommended >= %s)\n",
+		       $cur_version, $min_version, $rec_version;;
 		$need_sphinx = 1;
 		return;
 	}
 
 	if ($cur_version lt $rec_version) {
+		printf "Sphinx version %s\n", $cur_version;
 		print "Warning: It is recommended at least Sphinx version $rec_version.\n";
-		print "         To upgrade, use:\n\n";
 		$rec_sphinx_upgrade = 1;
+		return;
 	}
+
+	# On version check mode, just assume Sphinx has all mandatory deps
+	exit (0) if ($version_check);
 }
 
 #
@@ -575,14 +578,18 @@ sub check_distros()
 
 sub check_needs()
 {
-	if ($system_release) {
-		print "Detected OS: $system_release.\n";
-	} else {
-		print "Unknown OS\n";
-	}
-
 	# Check for needed programs/tools
 	check_sphinx();
+
+	if ($system_release) {
+		print "Detected OS: $system_release.\n\n";
+	} else {
+		print "Unknown OS\n\n";
+	}
+
+	print "To upgrade Sphinx, use:\n\n" if ($rec_sphinx_upgrade);
+
+	# Check for needed programs/tools
 	check_perl_module("Pod::Usage", 0);
 	check_program("make", 0);
 	check_program("gcc", 0);
@@ -601,13 +608,14 @@ sub check_needs()
 	}
 	if ($need_sphinx || $rec_sphinx_upgrade) {
 		my $min_activate = "$ENV{'PWD'}/${virtenv_prefix}${min_version}/bin/activate";
-                my @activates = glob "$ENV{'PWD'}/${virtenv_prefix}*/bin/activate";
+		my @activates = glob "$ENV{'PWD'}/${virtenv_prefix}*/bin/activate";
 
-                @activates = sort {$b cmp $a} @activates;
+		@activates = sort {$b cmp $a} @activates;
 
-		if (scalar @activates > 0 && $activates[0] ge $min_activate) {
-			printf "\nNeed to activate virtualenv with:\n";
+		if ($need_sphinx && scalar @activates > 0 && $activates[0] ge $min_activate) {
+			printf "\nNeed to activate a compatible Sphinx version on virtualenv with:\n";
 			printf "\t. $activates[0]\n";
+			exit (1);
 		} else {
 			my $rec_activate = "$virtenv_dir/bin/activate";
 			my $virtualenv = findprog("virtualenv-3");
@@ -646,8 +654,14 @@ while (@ARGV) {
 		$virtualenv = 0;
 	} elsif ($arg eq "--no-pdf"){
 		$pdf = 0;
+	} elsif ($arg eq "--version-check"){
+		$version_check = 1;
 	} else {
-		print "Usage:\n\t$0 <--no-virtualenv> <--no-pdf>\n\n";
+		print "Usage:\n\t$0 <--no-virtualenv> <--no-pdf> <--version-check>\n\n";
+		print "Where:\n";
+		print "\t--no-virtualenv\t- Recommend installing Sphinx instead of using a virtualenv\n";
+		print "\t--version-check\t- if version is compatible, don't check for missing dependencies\n";
+		print "\t--no-pdf\t- don't check for dependencies required to build PDF docs\n\n";
 		exit -1;
 	}
 }
-- 
2.21.0

[PATCH 4/5] docs: by default, build docs a lot faster with Sphinx >= 1.7

[Mauro Carvalho Chehab]

Since Sphinx version 1.7, it is possible to use "-jauto" in
order to speedup documentation builds. On older versions,
while -j was already supported, one would need to set the
number of threads manually.

So, if SPHINXOPTS is not provided, add -jauto, in order to
speed up the build. That makes it *a lot* times faster than
without -j.

If one really wants to slow things down, it can just use:

	make SPHINXOPTS=-j1 htmldocs

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
 Documentation/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 380e24053d6f..794233d05789 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -28,6 +28,8 @@ ifeq ($(HAVE_SPHINX),0)
 
 else # HAVE_SPHINX
 
+SPHINXOPTS = $(shell perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto" if ($$1 >= "1.7") } ;} close IN')
+
 # User-friendly check for pdflatex and latexmk
 HAVE_PDFLATEX := $(shell if which $(PDFLATEX) >/dev/null 2>&1; then echo 1; else echo 0; fi)
 HAVE_LATEXMK := $(shell if which latexmk >/dev/null 2>&1; then echo 1; else echo 0; fi)
-- 
2.21.0

[PATCH 5/5] docs: requirements.txt: recommend Sphinx 1.7.9

[Mauro Carvalho Chehab]

As discussed at the linux-doc ML, while we'll still support
version 1.3, it is time to recommend a more modern version.

So, let's switch the minimal requirements to Sphinx 1.7.9,
as it has the "-jauto" flag, with makes a lot faster when
building documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
 Documentation/doc-guide/sphinx.rst    | 17 ++++++++---------
 Documentation/sphinx/requirements.txt |  4 ++--
 2 files changed, 10 insertions(+), 11 deletions(-)

diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index c039224b404e..4ba081f43e98 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -27,8 +27,7 @@ Sphinx Install
 ==============
 
 The ReST markups currently used by the Documentation/ files are meant to be
-built with ``Sphinx`` version 1.3 or higher. If you desire to build
-PDF output, it is recommended to use version 1.4.6 or higher.
+built with ``Sphinx`` version 1.3 or higher.
 
 There's a script that checks for the Sphinx requirements. Please see
 :ref:`sphinx-pre-install` for further details.
@@ -56,13 +55,13 @@ or ``virtualenv``, depending on how your distribution packaged Python 3.
       those expressions are written using LaTeX notation. It needs texlive
       installed with amdfonts and amsmath in order to evaluate them.
 
-In summary, if you want to install Sphinx version 1.4.9, you should do::
+In summary, if you want to install Sphinx version 1.7.9, you should do::
 
-       $ virtualenv sphinx_1.4
-       $ . sphinx_1.4/bin/activate
-       (sphinx_1.4) $ pip install -r Documentation/sphinx/requirements.txt
+       $ virtualenv sphinx_1.7.9
+       $ . sphinx_1.7.9/bin/activate
+       (sphinx_1.7.9) $ pip install -r Documentation/sphinx/requirements.txt
 
-After running ``. sphinx_1.4/bin/activate``, the prompt will change,
+After running ``. sphinx_1.7.9/bin/activate``, the prompt will change,
 in order to indicate that you're using the new environment. If you
 open a new shell, you need to rerun this command to enter again at
 the virtual environment before building the documentation.
@@ -105,8 +104,8 @@ command line options for your distro::
 	You should run:
 
 		sudo dnf install -y texlive-luatex85
-		/usr/bin/virtualenv sphinx_1.4
-		. sphinx_1.4/bin/activate
+		/usr/bin/virtualenv sphinx_1.7.9
+		. sphinx_1.7.9/bin/activate
 		pip install -r Documentation/sphinx/requirements.txt
 
 	Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468.
diff --git a/Documentation/sphinx/requirements.txt b/Documentation/sphinx/requirements.txt
index 742be3e12619..14e29a0ae480 100644
--- a/Documentation/sphinx/requirements.txt
+++ b/Documentation/sphinx/requirements.txt
@@ -1,3 +1,3 @@
-docutils==0.12
-Sphinx==1.4.9
+docutils
+Sphinx==1.7.9
 sphinx_rtd_theme
-- 
2.21.0

Re: [PATCH 4/5] docs: by default, build docs a lot faster with Sphinx >= 1.7

[Jonathan Corbet]

On Mon, 27 May 2019 08:07:40 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:

> Since Sphinx version 1.7, it is possible to use "-jauto" in
> order to speedup documentation builds. On older versions,
> while -j was already supported, one would need to set the
> number of threads manually.
> 
> So, if SPHINXOPTS is not provided, add -jauto, in order to
> speed up the build. That makes it *a lot* times faster than
> without -j.
> 
> If one really wants to slow things down, it can just use:
> 
> 	make SPHINXOPTS=-j1 htmldocs
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> ---
>  Documentation/Makefile | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/Documentation/Makefile b/Documentation/Makefile
> index 380e24053d6f..794233d05789 100644
> --- a/Documentation/Makefile
> +++ b/Documentation/Makefile
> @@ -28,6 +28,8 @@ ifeq ($(HAVE_SPHINX),0)
>  
>  else # HAVE_SPHINX
>  
> +SPHINXOPTS = $(shell perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto" if ($$1 >= "1.7") } ;} close IN')
> +

So this totally fails to work for me with any version of sphinx, and I'm
not enough of a Perl person to figure it out.  Sometimes I'll see the
sphinx-build output, i.e.:

    sphinx-build 1.8.4

and sometimes (like with 2.0) I don't, but I never get -jauto regardless.
Not sure what's going on here?

Thanks,

jon

Re: [PATCH 4/5] docs: by default, build docs a lot faster with Sphinx >= 1.7

[Mauro Carvalho Chehab]

Em Wed, 29 May 2019 17:02:02 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> On Mon, 27 May 2019 08:07:40 -0300
> Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> 
> > Since Sphinx version 1.7, it is possible to use "-jauto" in
> > order to speedup documentation builds. On older versions,
> > while -j was already supported, one would need to set the
> > number of threads manually.
> > 
> > So, if SPHINXOPTS is not provided, add -jauto, in order to
> > speed up the build. That makes it *a lot* times faster than
> > without -j.
> > 
> > If one really wants to slow things down, it can just use:
> > 
> > 	make SPHINXOPTS=-j1 htmldocs
> > 
> > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> > ---
> >  Documentation/Makefile | 2 ++
> >  1 file changed, 2 insertions(+)
> > 
> > diff --git a/Documentation/Makefile b/Documentation/Makefile
> > index 380e24053d6f..794233d05789 100644
> > --- a/Documentation/Makefile
> > +++ b/Documentation/Makefile
> > @@ -28,6 +28,8 @@ ifeq ($(HAVE_SPHINX),0)
> >  
> >  else # HAVE_SPHINX
> >  
> > +SPHINXOPTS = $(shell perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto" if ($$1 >= "1.7") } ;} close IN')
> > +  
> 
> So this totally fails to work for me with any version of sphinx, and I'm
> not enough of a Perl person to figure it out.  Sometimes I'll see the
> sphinx-build output, i.e.:
> 
>     sphinx-build 1.8.4
> 
> and sometimes (like with 2.0) I don't, but I never get -jauto regardless.

Hmm... with 2.0.0 --version prints the version.

	$ sphinx-build --version
	sphinx-build 2.0.0

Afaikt, only too old versions will output it with a different format:


	$ sphinx-build --version
	Sphinx (sphinx-build) 1.2


> Not sure what's going on here?

Do you have SPHINXOPTS already set on your environment? If so, Makefile
will not override the existing environment.

Here, if I call it by hand (replacing $$1 by $1), it does the right
thing. For example:

1.8.4:

	$ sphinx-build --version
	sphinx-build 1.8.4
	$ perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
	-jauto

2.0.0:
	$ sphinx-build --version
	sphinx-build 2.0.0
	(sphinx_2.0) $ perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
-jauto


Thanks,
Mauro

Re: [PATCH 4/5] docs: by default, build docs a lot faster with Sphinx >= 1.7

[Jonathan Corbet]

On Wed, 29 May 2019 20:20:05 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:

> > So this totally fails to work for me with any version of sphinx, and I'm
> > not enough of a Perl person to figure it out.  Sometimes I'll see the
> > sphinx-build output, i.e.:
> > 
> >     sphinx-build 1.8.4
> > 
> > and sometimes (like with 2.0) I don't, but I never get -jauto regardless.  
> 
> Hmm... with 2.0.0 --version prints the version.
> 
> 	$ sphinx-build --version
> 	sphinx-build 2.0.0

Yup.  The point is that I see the sphinx-build output *in the docs-build
output", not when I run it standalone (where it does the expected thing). 

> > Not sure what's going on here?  
> 
> Do you have SPHINXOPTS already set on your environment? If so, Makefile
> will not override the existing environment.

Yeah, I had it set to -j1 because I want to wait as long as possible for my
docs builds :)

No, I didn't have it set separately, made a point of that.

> Here, if I call it by hand (replacing $$1 by $1), it does the right
> thing. For example:
> 
> 1.8.4:
> 
> 	$ sphinx-build --version
> 	sphinx-build 1.8.4
> 	$ perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
> 	-jauto

$ perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
sphinx-build 1.8.4
$

It works properly with 2.0.1 - but only on the command line; I still don't
get the right behavior in a docs build.

Most weird.

This is an Fedora 30 system, FWIW.

jon

Re: [PATCH 4/5] docs: by default, build docs a lot faster with Sphinx >= 1.7

[Mauro Carvalho Chehab]

Em Wed, 29 May 2019 17:47:16 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> On Wed, 29 May 2019 20:20:05 -0300
> Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> 
> > > So this totally fails to work for me with any version of sphinx, and I'm
> > > not enough of a Perl person to figure it out.  Sometimes I'll see the
> > > sphinx-build output, i.e.:
> > > 
> > >     sphinx-build 1.8.4
> > > 
> > > and sometimes (like with 2.0) I don't, but I never get -jauto regardless.    
> > 
> > Hmm... with 2.0.0 --version prints the version.
> > 
> > 	$ sphinx-build --version
> > 	sphinx-build 2.0.0  
> 
> Yup.  The point is that I see the sphinx-build output *in the docs-build
> output", not when I run it standalone (where it does the expected thing).

Weird... could some versions of Sphinx be redirecting the output of
--version to stderr instead of stdout?

If so, something like:

	perl -e 'open IN,"sphinx-build --version 2>&1 |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'

would make it print "-jauto" with those other versions you're trying.
 
> 
> > > Not sure what's going on here?    
> > 
> > Do you have SPHINXOPTS already set on your environment? If so, Makefile
> > will not override the existing environment.  
> 
> Yeah, I had it set to -j1 because I want to wait as long as possible for my
> docs builds :)
> 
> No, I didn't have it set separately, made a point of that.
> 
> > Here, if I call it by hand (replacing $$1 by $1), it does the right
> > thing. For example:
> > 
> > 1.8.4:
> > 
> > 	$ sphinx-build --version
> > 	sphinx-build 1.8.4
> > 	$ perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
> > 	-jauto  
> 
> $ perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
> sphinx-build 1.8.4
> $
> 
> It works properly with 2.0.1 - but only on the command line; I still don't
> get the right behavior in a docs build.
> 
> Most weird.
> 
> This is an Fedora 30 system, FWIW.

Yeah, really weird. Here I'm using Fedora 30 too:

	python3-sphinx_rtd_theme-0.4.3-1.fc30.noarch
	python3-sphinx-1.8.4-1.fc30.noarch

It works with both installed version and pip3 virtualenvs.

I didn't try the python2 versions, though.


> 
> jon



Thanks,
Mauro

Re: [PATCH 4/5] docs: by default, build docs a lot faster with Sphinx >= 1.7

[Jonathan Corbet]

On Wed, 29 May 2019 22:53:05 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:

> > Yup.  The point is that I see the sphinx-build output *in the docs-build
> > output", not when I run it standalone (where it does the expected thing).  
> 
> Weird... could some versions of Sphinx be redirecting the output of
> --version to stderr instead of stdout?
> 
> If so, something like:
> 
> 	perl -e 'open IN,"sphinx-build --version 2>&1 |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
> 
> would make it print "-jauto" with those other versions you're trying.

That does improve the behavior from the command line; it seems that
sphinx-build is indeed writing to stderr.  BUT that still doesn't fix the
docs build!  To get the option to take effect, I also have to explicitly
export SPHINXOPTS.  So the winning combination is:

  export SPHINXOPTS = $(shell perl -e 'open IN,"sphinx-build --version
  2>&1 |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto" if ($$1 >= "1.7") } ;} close IN')

I don't have any weird version of make, so I'm not sure why you see
different results than I do here.

I can apply those tweaks to your patch if it's OK with you.

> I didn't try the python2 versions, though.

Interestingly, I would appear to have both versions installed, with
python2 winning in $PATH.

jon

Re: [PATCH 4/5] docs: by default, build docs a lot faster with Sphinx >= 1.7

[Mauro Carvalho Chehab]

Em Thu, 30 May 2019 08:54:04 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> On Wed, 29 May 2019 22:53:05 -0300
> Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> 
> > > Yup.  The point is that I see the sphinx-build output *in the docs-build
> > > output", not when I run it standalone (where it does the expected thing).    
> > 
> > Weird... could some versions of Sphinx be redirecting the output of
> > --version to stderr instead of stdout?
> > 
> > If so, something like:
> > 
> > 	perl -e 'open IN,"sphinx-build --version 2>&1 |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
> > 
> > would make it print "-jauto" with those other versions you're trying.  
> 
> That does improve the behavior from the command line; it seems that
> sphinx-build is indeed writing to stderr.  BUT that still doesn't fix the
> docs build!  To get the option to take effect, I also have to explicitly
> export SPHINXOPTS.  So the winning combination is:
> 
>   export SPHINXOPTS = $(shell perl -e 'open IN,"sphinx-build --version
>   2>&1 |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto" if ($$1 >= "1.7") } ;} close IN')  
> 
> I don't have any weird version of make, so I'm not sure why you see
> different results than I do here.
> 
> I can apply those tweaks to your patch if it's OK with you.

Yeah, sure! With those changes it work fine here too.

So, feel free to apply the changes.

> 
> > I didn't try the python2 versions, though.  
> 
> Interestingly, I would appear to have both versions installed, with
> python2 winning in $PATH.

It sounds that Fedora 30 is conservative with regards to python :-)

The Sphinx version detection script takes it into account,
suggesting pip3 instead of pip - or when called like:

	$ ./scripts/sphinx-pre-install --no-virtualenv
	Detected OS: Fedora release 30 (Thirty).

	ERROR: please install "python-sphinx", otherwise, build won't work.
	Warning: better to also install "sphinx_rtd_theme".
	You should run:

		sudo dnf install -y python3-sphinx python3-sphinx_rtd_theme

It seeks for the python3 packages on Fedora.

Regards,
Mauro