[PATCH v3 0/2] docs: conf.py: Reduce texlive dependency

[PATCH v3 0/2] docs: conf.py: Reduce texlive dependency

[Akira Yokosawa]

Hi,

This is v3 which addresses Jon's comments to v2 [1], but slightly
different ways from Jon's suggestions.

Changes since v2 [1]:
 - Drop Patch 3/3, which is in docs-fixes now.
 - Patch 1/2:
     o Simplify find_command() by using shutil.which(). For guessing
       available commands, it should be good enough.
     o Mention find_command() in the changelog.
 - Patch 2/2:
     o Instead of silently ignoring unknown setting of SPHINX_IMGMATH,
       print a warning msg.

Acked-bys from Mauro are kept, as I assume these changes wouldn't
affect the purpose of each patch.

Changes since v1 [2]:
 - Patch 2/3: Rename LOAD_IMGMATH -> SPHINX_IMGMATH (Mauro).
 - For the series: Acked-bys from Mauro.

Amended coverletter:

This was inspired from the discussion on the expected behavior of
sphinx-pre-install [3].

There was a mismatch between Mauro's intent and my expectation for
the --no-pdf option of sphinx-pre-install.

My thought was if I installed all the packages suggested from
"./scripts/sphinx-pre-install --no-pdf", "make htmldocs" should run
free of complaints of missing commands or packages.

However, I got this warning when I tried the procedure on a debian-
based container image:

    WARNING: LaTeX command 'latex' cannot be run (needed for math display),
    check the imgmath_latex setting

, or:

    WARNING: dvipng command 'dvipng' cannot be run (needed for math display),
    check the imgmath_dvipng setting

Mauro's response to my complaint was this:

> The idea of using --no-pdf is to setup an environment without LaTeX,
> meaning that math tags would only be partially parsed: basically, the
> output would be html with LaTeX-like math expressions (at least last
> time I tried).

The mismatch can be resolved by using "mathjax" for math rendering
and making "make htmldocs" be free of texlive packages.

mathjax is the default math renderer since Sphinx 1.8.  It delegates
math rendering to web browsers.  

As Mauro has pointed out, "make epubdocs" requires imgmath.

So this patch set treats mathjax as a fallback math renderer for html
docs when imgmath requirements are not met.
Existing systems which meet imgmath requirements are not affected by
this change.

Summary of math rendering in html:

         dvipng, browser             before           after
    ==========================  ===============  ================
    dvipng                      imgmath (png)    <--
    no divpng, with javascript  raw math:: code  mathjax
    no dvipng, w/o javascript   raw math:: code  raw mathjax code

Patch 1/2 adds code in conf.py so that for html docs, the imgmath
extension will be loaded only when both latex and dvipng are available.
For epub docs, imgmath will always be loaded (no change).

Patch 2/2 adds code respecting a new env variable "SPHINX_IMGMATH" which
will override the math renderer choice. This variable can be helpful
on distros such as Arch linux, Mageia, etc. whose packaging policy is
coarse-grained.

E.g., to test math rendering by mathjax, run:
    make SPHINX_IMGMATH=no htmldocs

I mentioned in the thread of [3] that imgmath can generate scalable
math images in SVG.

My plan was to implement that option as well.  But during tests under
Fedora/CentOS/openSUSE, I encountered a couple of warnings from dvisvgm.
That would be regressions on existing systems which happen to have
not-working dvisvgm along with working dvipng.  I'm thinking of adding
the SVG option later if I can figure out the minimal requirement for
dvisvgm under imgmath.

[1] v2: https://lore.kernel.org/r/c41cab17-afd6-bc99-56a1-e4e73b8c1ef6@gmail.com/
[2] v1: https://lore.kernel.org/r/12d078f5-6995-b039-7076-bdb1f372a799@gmail.com/
[3]: https://lore.kernel.org/r/3ba5a52e-cab6-05cf-a66e-adc58c467e1f@gmail.com/

        Thanks, Akira
--
Akira Yokosawa (2):
  docs/conf.py: Treat mathjax as fallback math renderer
  docs/conf.py: Respect env variable SPHINX_IMGMATH

 Documentation/conf.py | 41 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 40 insertions(+), 1 deletion(-)


base-commit: e1a0cc8865e3152e9fe43c6436b44e64c0359dfb
-- 
2.25.1

[PATCH v3 1/2] docs/conf.py: Treat mathjax as fallback math renderer

[Akira Yokosawa]

Currently, math expressions using the "math::" directive or
the ":math:" role of Sphinx need the imgmath extension for proper
rendering in html and epub builds.
imgmath requires dvipng (and latex).
Otherwise, "make htmldocs" will complain of missing commands.

As a matter of fact, the mathjax extension is loaded by default since
Sphinx v1.8 and it is good enough for html docs without any dependency
on texlive packages.

Stop loading the imgmath extension for html docs unless requirements
for imgmath are met.

To find out whether required commands are available, add a helper
find_command(), which is a wrapper of shutil.which().

For epub docs, keep the same behavior of always loading imgmath.

Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
Acked-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
Changes since v2:
 - Mention find_command() in the changelog.
 - Simplify find_command() by using shutil.which().

Changes sinve v1:
 - Acked-by from Mauro
--
 Documentation/conf.py | 29 ++++++++++++++++++++++++++++-
 1 file changed, 28 insertions(+), 1 deletion(-)

diff --git a/Documentation/conf.py b/Documentation/conf.py
index 255384d094bf..c4aaedd96220 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -15,6 +15,18 @@
 import sys
 import os
 import sphinx
+import shutil
+
+# helper
+# ------
+
+def have_command(cmd):
+    """Search ``cmd`` in the ``PATH`` environment.
+
+    If found, return True.
+    If not found, return False.
+    """
+    return shutil.which(cmd) is not None
 
 # Get Sphinx version
 major, minor, patch = sphinx.version_info[:3]
@@ -107,7 +119,22 @@ else:
 autosectionlabel_prefix_document = True
 autosectionlabel_maxdepth = 2
 
-extensions.append("sphinx.ext.imgmath")
+# Load math renderer:
+# For html builder, load imgmath only when its dependencies are met.
+# mathjax is the default math renderer since Sphinx 1.8.
+have_latex =  have_command('latex')
+have_dvipng = have_command('dvipng')
+load_imgmath = ((have_latex and have_dvipng)
+                or (major == 1 and minor < 8)
+                or 'epub' in sys.argv)
+
+if load_imgmath:
+    extensions.append("sphinx.ext.imgmath")
+    math_renderer = 'imgmath'
+else:
+    math_renderer = 'mathjax'
+
+sys.stderr.write("math_renderer: %s\n" % math_renderer)
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
-- 
2.25.1

[PATCH v3 2/2] docs/conf.py: Respect env variable SPHINX_IMGMATH

[Akira Yokosawa]

On some distros with coarse-grained packaging policy, dvipng is
installed along with latex.  In such cases, math rendering will
use imgmath by default.  It is possible to override the choice by
specifying the option string of "-D html_math_renderer='mathjax'"
to sphinx-build (Sphinx >= 1.8).

To provide developers an easier-to-use knob, add code for an env
variable "SPHINX_IMGMATH" which overrides the automatic choice
of math renderer for html docs.

    SPHINX_IMGMATH=yes : Load imgmath even if dvipng is not found
    SPHINX_IMGMATH=no  : Don't load imgmath (fall back to mathjax)

Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
Acked-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
Changes since v2:
 - Emit warning on unknown setting of SPHINX_IMGMATH.

Changes since v1:
 - No logical changes.
 - Renames:
     LOAD_IMGMATH -> SPHINX_IMGMATH (Mauro),
     opt_load_imgmath -> env_sphinx_imgmath.
 - Acked-by from Mauro.
--
 Documentation/conf.py | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/Documentation/conf.py b/Documentation/conf.py
index c4aaedd96220..9678a8215457 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -124,8 +124,20 @@ autosectionlabel_maxdepth = 2
 # mathjax is the default math renderer since Sphinx 1.8.
 have_latex =  have_command('latex')
 have_dvipng = have_command('dvipng')
-load_imgmath = ((have_latex and have_dvipng)
-                or (major == 1 and minor < 8)
+load_imgmath = have_latex and have_dvipng
+
+# Respect SPHINX_IMGMATH (for html docs only)
+if 'SPHINX_IMGMATH' in os.environ:
+    env_sphinx_imgmath = os.environ['SPHINX_IMGMATH']
+    if 'yes' in env_sphinx_imgmath:
+        load_imgmath = True
+    elif 'no' in env_sphinx_imgmath:
+        load_imgmath = False
+    else:
+        sys.stderr.write("Unknown env SPHINX_IMGMATH=%s ignored.\n" % env_sphinx_imgmath)
+
+# Always load imgmath for Sphinx <1.8 or for epub docs
+load_imgmath = (load_imgmath or (major == 1 and minor < 8)
                 or 'epub' in sys.argv)
 
 if load_imgmath:
-- 
2.25.1

Re: [PATCH v3 2/2] docs/conf.py: Respect env variable SPHINX_IMGMATH

[Randy Dunlap]

Hi Akira,

On 8/26/22 21:38, Akira Yokosawa wrote:
> On some distros with coarse-grained packaging policy, dvipng is
> installed along with latex.  In such cases, math rendering will
> use imgmath by default.  It is possible to override the choice by
> specifying the option string of "-D html_math_renderer='mathjax'"
> to sphinx-build (Sphinx >= 1.8).
> 
> To provide developers an easier-to-use knob, add code for an env
> variable "SPHINX_IMGMATH" which overrides the automatic choice
> of math renderer for html docs.
> 
>     SPHINX_IMGMATH=yes : Load imgmath even if dvipng is not found
>     SPHINX_IMGMATH=no  : Don't load imgmath (fall back to mathjax)

Please add SPHINX_IMGMATH to Documentation/doc-guide/sphinx.rst
(I guess), where other Sphinx environment variables are listed.

> Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
> Acked-by: Mauro Carvalho Chehab <mchehab@kernel.org>
> ---
> Changes since v2:
>  - Emit warning on unknown setting of SPHINX_IMGMATH.
> 
> Changes since v1:
>  - No logical changes.
>  - Renames:
>      LOAD_IMGMATH -> SPHINX_IMGMATH (Mauro),
>      opt_load_imgmath -> env_sphinx_imgmath.
>  - Acked-by from Mauro.
> --
>  Documentation/conf.py | 16 ++++++++++++++--
>  1 file changed, 14 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index c4aaedd96220..9678a8215457 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -124,8 +124,20 @@ autosectionlabel_maxdepth = 2
>  # mathjax is the default math renderer since Sphinx 1.8.
>  have_latex =  have_command('latex')
>  have_dvipng = have_command('dvipng')
> -load_imgmath = ((have_latex and have_dvipng)
> -                or (major == 1 and minor < 8)
> +load_imgmath = have_latex and have_dvipng
> +
> +# Respect SPHINX_IMGMATH (for html docs only)
> +if 'SPHINX_IMGMATH' in os.environ:
> +    env_sphinx_imgmath = os.environ['SPHINX_IMGMATH']
> +    if 'yes' in env_sphinx_imgmath:
> +        load_imgmath = True
> +    elif 'no' in env_sphinx_imgmath:
> +        load_imgmath = False
> +    else:
> +        sys.stderr.write("Unknown env SPHINX_IMGMATH=%s ignored.\n" % env_sphinx_imgmath)
> +
> +# Always load imgmath for Sphinx <1.8 or for epub docs
> +load_imgmath = (load_imgmath or (major == 1 and minor < 8)
>                  or 'epub' in sys.argv)
>  
>  if load_imgmath:

thanks.
-- 
~Randy

Re: [PATCH v3 2/2] docs/conf.py: Respect env variable SPHINX_IMGMATH

[Akira Yokosawa]

Hi Randy,  thank you for looking into this!

On Sat, 27 Aug 2022 17:38:41 -0700, Randy Dunlap wrote:
> Hi Akira,
> 
> On 8/26/22 21:38, Akira Yokosawa wrote:
>> On some distros with coarse-grained packaging policy, dvipng is
>> installed along with latex.  In such cases, math rendering will
>> use imgmath by default.  It is possible to override the choice by
>> specifying the option string of "-D html_math_renderer='mathjax'"
>> to sphinx-build (Sphinx >= 1.8).
>>
>> To provide developers an easier-to-use knob, add code for an env
>> variable "SPHINX_IMGMATH" which overrides the automatic choice
>> of math renderer for html docs.
>>
>>     SPHINX_IMGMATH=yes : Load imgmath even if dvipng is not found
>>     SPHINX_IMGMATH=no  : Don't load imgmath (fall back to mathjax)
> 
> Please add SPHINX_IMGMATH to Documentation/doc-guide/sphinx.rst
> (I guess), where other Sphinx environment variables are listed.

Right.
As I mentioned in the coverletter, (quoted below):

> I mentioned in the thread of [3] that imgmath can generate scalable
> math images in SVG.
> 
> My plan was to implement that option as well.  But during tests under
> Fedora/CentOS/openSUSE, I encountered a couple of warnings from dvisvgm.
> That would be regressions on existing systems which happen to have
> not-working dvisvgm along with working dvipng.  I'm thinking of adding
> the SVG option later if I can figure out the minimal requirement for
> dvisvgm under imgmath.

, I'm working on follow-up changes related to SPHINX_IMGMATH.

My plan is to update related docs and help text in Makefile when I manage
the SVG option. At that time, SPHINX_IMGMATH will have additional options
like 'svg' and 'png'.

Hopefully, such follow-up changes can make the v6.1 merge window.

Does this plan work for you?

        Thanks, Akira
> 
>> Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
>> Acked-by: Mauro Carvalho Chehab <mchehab@kernel.org>
>> ---
>> Changes since v2:
>>  - Emit warning on unknown setting of SPHINX_IMGMATH.
>>
>> Changes since v1:
>>  - No logical changes.
>>  - Renames:
>>      LOAD_IMGMATH -> SPHINX_IMGMATH (Mauro),
>>      opt_load_imgmath -> env_sphinx_imgmath.
>>  - Acked-by from Mauro.
>> --
>>  Documentation/conf.py | 16 ++++++++++++++--
>>  1 file changed, 14 insertions(+), 2 deletions(-)
>>
>> diff --git a/Documentation/conf.py b/Documentation/conf.py
>> index c4aaedd96220..9678a8215457 100644
>> --- a/Documentation/conf.py
>> +++ b/Documentation/conf.py
>> @@ -124,8 +124,20 @@ autosectionlabel_maxdepth = 2
>>  # mathjax is the default math renderer since Sphinx 1.8.
>>  have_latex =  have_command('latex')
>>  have_dvipng = have_command('dvipng')
>> -load_imgmath = ((have_latex and have_dvipng)
>> -                or (major == 1 and minor < 8)
>> +load_imgmath = have_latex and have_dvipng
>> +
>> +# Respect SPHINX_IMGMATH (for html docs only)
>> +if 'SPHINX_IMGMATH' in os.environ:
>> +    env_sphinx_imgmath = os.environ['SPHINX_IMGMATH']
>> +    if 'yes' in env_sphinx_imgmath:
>> +        load_imgmath = True
>> +    elif 'no' in env_sphinx_imgmath:
>> +        load_imgmath = False
>> +    else:
>> +        sys.stderr.write("Unknown env SPHINX_IMGMATH=%s ignored.\n" % env_sphinx_imgmath)
>> +
>> +# Always load imgmath for Sphinx <1.8 or for epub docs
>> +load_imgmath = (load_imgmath or (major == 1 and minor < 8)
>>                  or 'epub' in sys.argv)
>>  
>>  if load_imgmath:
> 
> thanks.

Re: [PATCH v3 2/2] docs/conf.py: Respect env variable SPHINX_IMGMATH

[Randy Dunlap]

On 8/27/22 20:47, Akira Yokosawa wrote:
> Hi Randy,  thank you for looking into this!
> 
> On Sat, 27 Aug 2022 17:38:41 -0700, Randy Dunlap wrote:
>> Hi Akira,
>>
>> On 8/26/22 21:38, Akira Yokosawa wrote:
>>> On some distros with coarse-grained packaging policy, dvipng is
>>> installed along with latex.  In such cases, math rendering will
>>> use imgmath by default.  It is possible to override the choice by
>>> specifying the option string of "-D html_math_renderer='mathjax'"
>>> to sphinx-build (Sphinx >= 1.8).
>>>
>>> To provide developers an easier-to-use knob, add code for an env
>>> variable "SPHINX_IMGMATH" which overrides the automatic choice
>>> of math renderer for html docs.
>>>
>>>     SPHINX_IMGMATH=yes : Load imgmath even if dvipng is not found
>>>     SPHINX_IMGMATH=no  : Don't load imgmath (fall back to mathjax)
>>
>> Please add SPHINX_IMGMATH to Documentation/doc-guide/sphinx.rst
>> (I guess), where other Sphinx environment variables are listed.
> 
> Right.
> As I mentioned in the coverletter, (quoted below):
> 
>> I mentioned in the thread of [3] that imgmath can generate scalable
>> math images in SVG.
>>
>> My plan was to implement that option as well.  But during tests under
>> Fedora/CentOS/openSUSE, I encountered a couple of warnings from dvisvgm.
>> That would be regressions on existing systems which happen to have
>> not-working dvisvgm along with working dvipng.  I'm thinking of adding
>> the SVG option later if I can figure out the minimal requirement for
>> dvisvgm under imgmath.
> 
> , I'm working on follow-up changes related to SPHINX_IMGMATH.
> 
> My plan is to update related docs and help text in Makefile when I manage
> the SVG option. At that time, SPHINX_IMGMATH will have additional options
> like 'svg' and 'png'.
> 
> Hopefully, such follow-up changes can make the v6.1 merge window.
> 
> Does this plan work for you?

Ok, yes, sure, no problem.

Thanks.

-- 
~Randy

Re: [PATCH v3 0/2] docs: conf.py: Reduce texlive dependency

[Jonathan Corbet]

Akira Yokosawa <akiyks@gmail.com> writes:

> Hi,
>
> This is v3 which addresses Jon's comments to v2 [1], but slightly
> different ways from Jon's suggestions.
>
> Changes since v2 [1]:
>  - Drop Patch 3/3, which is in docs-fixes now.
>  - Patch 1/2:
>      o Simplify find_command() by using shutil.which(). For guessing
>        available commands, it should be good enough.
>      o Mention find_command() in the changelog.
>  - Patch 2/2:
>      o Instead of silently ignoring unknown setting of SPHINX_IMGMATH,
>        print a warning msg.
>
> Acked-bys from Mauro are kept, as I assume these changes wouldn't
> affect the purpose of each patch.
>
> Changes since v1 [2]:
>  - Patch 2/3: Rename LOAD_IMGMATH -> SPHINX_IMGMATH (Mauro).
>  - For the series: Acked-bys from Mauro.

Set applied, thanks.  I did make one change to patch 1, see my reply
there.

jon

Re: [PATCH v3 1/2] docs/conf.py: Treat mathjax as fallback math renderer

[Jonathan Corbet]

Akira Yokosawa <akiyks@gmail.com> writes:

> Currently, math expressions using the "math::" directive or
> the ":math:" role of Sphinx need the imgmath extension for proper
> rendering in html and epub builds.
> imgmath requires dvipng (and latex).
> Otherwise, "make htmldocs" will complain of missing commands.
>
> As a matter of fact, the mathjax extension is loaded by default since
> Sphinx v1.8 and it is good enough for html docs without any dependency
> on texlive packages.
>
> Stop loading the imgmath extension for html docs unless requirements
> for imgmath are met.
>
> To find out whether required commands are available, add a helper
> find_command(), which is a wrapper of shutil.which().
>
> For epub docs, keep the same behavior of always loading imgmath.
>
> Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
> Acked-by: Mauro Carvalho Chehab <mchehab@kernel.org>
> ---
> Changes since v2:
>  - Mention find_command() in the changelog.
>  - Simplify find_command() by using shutil.which().
>
> Changes sinve v1:
>  - Acked-by from Mauro
> --
>  Documentation/conf.py | 29 ++++++++++++++++++++++++++++-
>  1 file changed, 28 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index 255384d094bf..c4aaedd96220 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -15,6 +15,18 @@
>  import sys
>  import os
>  import sphinx
> +import shutil
> +
> +# helper
> +# ------
> +
> +def have_command(cmd):
> +    """Search ``cmd`` in the ``PATH`` environment.
> +
> +    If found, return True.
> +    If not found, return False.
> +    """
> +    return shutil.which(cmd) is not None
>  
>  # Get Sphinx version
>  major, minor, patch = sphinx.version_info[:3]
> @@ -107,7 +119,22 @@ else:
>  autosectionlabel_prefix_document = True
>  autosectionlabel_maxdepth = 2
>  
> -extensions.append("sphinx.ext.imgmath")
> +# Load math renderer:
> +# For html builder, load imgmath only when its dependencies are met.
> +# mathjax is the default math renderer since Sphinx 1.8.
> +have_latex =  have_command('latex')
> +have_dvipng = have_command('dvipng')
> +load_imgmath = ((have_latex and have_dvipng)
> +                or (major == 1 and minor < 8)
> +                or 'epub' in sys.argv)
> +
> +if load_imgmath:
> +    extensions.append("sphinx.ext.imgmath")
> +    math_renderer = 'imgmath'
> +else:
> +    math_renderer = 'mathjax'
> +
> +sys.stderr.write("math_renderer: %s\n" % math_renderer)

I've taken this line out as I applied the patch.  The docs build is
verbose enough as it is, and this line isn't really going to add
anything for most people.  If it needs to be there, I think, it should
only show up on a V=1 build.

Thanks,

jon