[PATCH 00/13] Address issues with PDF output at media uAPI docs

[PATCH 00/13] Address issues with PDF output at media uAPI docs

[Mauro Carvalho Chehab]

Keeping LaTeX/PDF output working is hard, as Sphinx require lots of
manual adjusts, as it has several troubles:

- It can't compute properly table cell sizes when literals are used;
- It doesn't break long literal words with hiphens when needed;
- It easily write things out of tables and out of pages;
- It doesn't have any tag that would allow changing the font size;
- It doesn't re-scale tables.

So, from time to time, a manual (hard) work is needed in order to
check what broke, fixing them.

This series address most of such issues that happen at the uAPI
media documents.

Jon,

Patch 1 actually fixes a bug at conf.py.  The remaining patches
depend on it. My plan is to have this series merged for 5.11.

IMO, the best would be to have this patch applied via my tree,
but I'm OK if you prefer to merge this one via yours.

Regards,
Mauro

Mauro Carvalho Chehab (13):
  docs: conf.py: fix sphinx version detection for margin set
  media: colorspaces-details.rst: drop tabularcolumns
  media: control.rst: use a table for V4L2_CID_POWER_LINE
  media: docs: sliced-vbi: fix V4L2_SLICED_WSS_625 docs
  media: ext-ctrls-codec-stateless.rst: change a FWHT flag description
  media: ext-ctrls-codec.rst: add a missing profile description
  media: ext-ctrls-codec.rst: simplify a few tables
  media: ext-ctrls-jpeg.rst: cleanup V4L2_CID_JPEG_COMPRESSION_QUALITY
    text
  media: docs: pixfmt: use section titles for bayer formats
  media: buffer.rst: fix a PDF output issue
  media: ext-ctrls-codec-stateless.rst: fix an H-264 table format
  media: pixfmt-yuv-planar.rst: fix PDF OUTPUT
  media: docs: uAPI: fix table output in LaTeX/PDF format

 Documentation/conf.py                         |   3 +-
 .../media/cec/cec-ioc-adap-g-caps.rst         |   4 +-
 .../media/cec/cec-ioc-adap-g-conn-info.rst    |   6 +-
 .../media/cec/cec-ioc-adap-g-log-addrs.rst    |  12 +-
 .../media/cec/cec-ioc-dqevent.rst             |  10 +-
 .../media/cec/cec-ioc-g-mode.rst              |   4 +-
 .../media/cec/cec-ioc-receive.rst             |   8 +-
 .../userspace-api/media/dvb/fe-type-t.rst     |   2 +-
 .../media/mediactl/media-ioc-device-info.rst  |   2 +-
 .../mediactl/media-ioc-enum-entities.rst      |   2 +-
 .../media/mediactl/media-ioc-enum-links.rst   |   6 +-
 .../media/mediactl/media-ioc-g-topology.rst   |  12 +-
 .../media/mediactl/media-types.rst            |   4 +-
 .../userspace-api/media/rc/rc-tables.rst      |   2 +-
 .../userspace-api/media/v4l/buffer.rst        |  22 +-
 .../media/v4l/colorspaces-details.rst         |  31 --
 .../userspace-api/media/v4l/control.rst       |  13 +-
 .../userspace-api/media/v4l/dev-meta.rst      |   2 +-
 .../userspace-api/media/v4l/dev-raw-vbi.rst   |   4 +-
 .../userspace-api/media/v4l/dev-rds.rst       |   4 +-
 .../userspace-api/media/v4l/dev-sdr.rst       |   2 +-
 .../media/v4l/dev-sliced-vbi.rst              |  50 +--
 .../userspace-api/media/v4l/dev-subdev.rst    |   6 +-
 .../userspace-api/media/v4l/diff-v4l.rst      |  10 +-
 .../media/v4l/ext-ctrls-camera.rst            |  14 +-
 .../media/v4l/ext-ctrls-codec-stateless.rst   | 181 +++++++++--
 .../media/v4l/ext-ctrls-codec.rst             | 288 ++++++++++++------
 .../userspace-api/media/v4l/ext-ctrls-dv.rst  |   2 +-
 .../media/v4l/ext-ctrls-flash.rst             |   7 +-
 .../media/v4l/ext-ctrls-jpeg.rst              |  13 +-
 .../userspace-api/media/v4l/field-order.rst   |   2 +-
 .../media/v4l/pixfmt-compressed.rst           |  12 +-
 .../media/v4l/pixfmt-packed-yuv.rst           |  26 +-
 .../media/v4l/pixfmt-reserved.rst             |  10 +-
 .../userspace-api/media/v4l/pixfmt-rgb.rst    |   9 +-
 .../media/v4l/pixfmt-srggb10-ipu3.rst         |  12 +-
 .../media/v4l/pixfmt-srggb10p.rst             |   2 +-
 .../media/v4l/pixfmt-srggb12p.rst             |   2 +-
 .../media/v4l/pixfmt-srggb14.rst              |   2 +
 .../media/v4l/pixfmt-srggb14p.rst             |   6 +-
 .../media/v4l/pixfmt-srggb16.rst              |   2 +
 .../userspace-api/media/v4l/pixfmt-srggb8.rst |   3 +-
 .../media/v4l/pixfmt-v4l2-mplane.rst          |   4 +-
 .../userspace-api/media/v4l/pixfmt-v4l2.rst   |   4 +-
 .../media/v4l/pixfmt-yuv-luma.rst             |  10 +
 .../media/v4l/pixfmt-yuv-planar.rst           |  34 ++-
 .../media/v4l/subdev-formats.rst              |  26 +-
 .../media/v4l/v4l2-selection-flags.rst        |  14 +-
 .../media/v4l/v4l2-selection-targets.rst      |  12 +-
 .../media/v4l/vidioc-create-bufs.rst          |   2 +-
 .../media/v4l/vidioc-cropcap.rst              |   4 +-
 .../media/v4l/vidioc-dbg-g-chip-info.rst      |   6 +-
 .../media/v4l/vidioc-dbg-g-register.rst       |   4 +-
 .../media/v4l/vidioc-decoder-cmd.rst          |   8 +-
 .../media/v4l/vidioc-dqevent.rst              |  21 +-
 .../media/v4l/vidioc-dv-timings-cap.rst       |   6 +-
 .../media/v4l/vidioc-encoder-cmd.rst          |   6 +-
 .../media/v4l/vidioc-enum-dv-timings.rst      |   2 +-
 .../media/v4l/vidioc-enum-fmt.rst             |  10 +-
 .../media/v4l/vidioc-enum-frameintervals.rst  |   7 +-
 .../media/v4l/vidioc-enum-framesizes.rst      |   8 +-
 .../media/v4l/vidioc-enum-freq-bands.rst      |   4 +-
 .../media/v4l/vidioc-enuminput.rst            |   8 +-
 .../media/v4l/vidioc-enumoutput.rst           |   6 +-
 .../media/v4l/vidioc-enumstd.rst              |   6 +-
 .../userspace-api/media/v4l/vidioc-expbuf.rst |   2 +-
 .../media/v4l/vidioc-g-audio.rst              |   6 +-
 .../media/v4l/vidioc-g-audioout.rst           |   2 +-
 .../userspace-api/media/v4l/vidioc-g-crop.rst |   2 +-
 .../userspace-api/media/v4l/vidioc-g-ctrl.rst |   2 +-
 .../media/v4l/vidioc-g-dv-timings.rst         |  20 +-
 .../userspace-api/media/v4l/vidioc-g-edid.rst |   2 +-
 .../media/v4l/vidioc-g-enc-index.rst          |   6 +-
 .../media/v4l/vidioc-g-ext-ctrls.rst          |  20 +-
 .../userspace-api/media/v4l/vidioc-g-fbuf.rst |   6 +-
 .../userspace-api/media/v4l/vidioc-g-fmt.rst  |   2 +-
 .../media/v4l/vidioc-g-frequency.rst          |   2 +-
 .../media/v4l/vidioc-g-jpegcomp.rst           |   4 +-
 .../media/v4l/vidioc-g-modulator.rst          |   9 +-
 .../userspace-api/media/v4l/vidioc-g-parm.rst |  11 +-
 .../media/v4l/vidioc-g-priority.rst           |   2 +-
 .../media/v4l/vidioc-g-selection.rst          |   2 +-
 .../media/v4l/vidioc-g-sliced-vbi-cap.rst     |  29 +-
 .../media/v4l/vidioc-g-tuner.rst              |  12 +-
 .../media/v4l/vidioc-querycap.rst             |   8 +-
 .../media/v4l/vidioc-queryctrl.rst            |  21 +-
 .../media/v4l/vidioc-reqbufs.rst              |  14 +-
 .../media/v4l/vidioc-s-hw-freq-seek.rst       |   2 +-
 .../v4l/vidioc-subdev-enum-frame-interval.rst |   2 +-
 .../v4l/vidioc-subdev-enum-frame-size.rst     |   2 +-
 .../v4l/vidioc-subdev-enum-mbus-code.rst      |  14 +-
 .../media/v4l/vidioc-subdev-g-crop.rst        |   2 +-
 .../media/v4l/vidioc-subdev-g-fmt.rst         |   4 +-
 .../v4l/vidioc-subdev-g-frame-interval.rst    |   2 +-
 .../media/v4l/vidioc-subdev-g-selection.rst   |   2 +-
 .../media/v4l/vidioc-subdev-querycap.rst      |   4 +-
 .../media/v4l/vidioc-subscribe-event.rst      |   4 +-
 97 files changed, 802 insertions(+), 446 deletions(-)

-- 
2.29.2

[PATCH 01/13] docs: conf.py: fix sphinx version detection for margin set

[Mauro Carvalho Chehab]

The PDF generator has a logic to detect the proper way to
setup the page margins. By default, the page has about
14.8 cm, which is too short to display some tables and literal
blocks. So, previous patches changed it to be around 17.5 cm,
but the logic only works with Sphinx version 1.x.x.

Fix it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/conf.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/conf.py b/Documentation/conf.py
index ed2b43ec7754..66e121df59cd 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -375,9 +375,10 @@ if cjk_cmd.find("Noto Sans CJK SC") >= 0:
 if major == 1 and minor > 3:
     latex_elements['preamble']  += '\\renewcommand*{\\DUrole}[2]{ #2 }\n'
 
+# Set page margins
 if major == 1 and minor <= 4:
     latex_elements['preamble']  += '\\usepackage[margin=0.5in, top=1in, bottom=1in]{geometry}'
-elif major == 1 and (minor > 5 or (minor == 5 and patch >= 3)):
+elif (major == 1 and (minor > 5 or (minor == 5 and patch >= 3))) or (major > 1):
     latex_elements['sphinxsetup'] = 'hmargin=0.5in, vmargin=1in'
     latex_elements['preamble']  += '\\fvset{fontsize=auto}\n'
 
-- 
2.29.2

Re: [PATCH 01/13] docs: conf.py: fix sphinx version detection for margin set

[Jonathan Corbet]

On Thu, 10 Dec 2020 11:55:40 +0100
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:

> The PDF generator has a logic to detect the proper way to
> setup the page margins. By default, the page has about
> 14.8 cm, which is too short to display some tables and literal
> blocks. So, previous patches changed it to be around 17.5 cm,
> but the logic only works with Sphinx version 1.x.x.
> 
> Fix it.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Acked-by: Jonathan Corbet <corbet@lwn.net>

Feel free to route this one with the rest of the set.

Someday it might be nice to isolate all of the latex stuff into
conf-latex.py or some such so most of us don't have to look at it..:)

Thanks,

jon

[PATCH RFC] docs: experimental: build PDF with rst2pdf

[Mauro Carvalho Chehab]

Add an experimental PDF builder using rst2pdf

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/Makefile                     |  5 +++++
 Documentation/conf.py                      | 21 +++++++++++++++------
 Documentation/userspace-api/media/Makefile |  1 +
 Makefile                                   |  4 ++--
 4 files changed, 23 insertions(+), 8 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 61a7310b49e0..c3c8fb10f94e 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -115,6 +115,10 @@ pdfdocs: latexdocs
 
 endif # HAVE_PDFLATEX
 
+rst2pdf:
+	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,pdf,$(var),pdf,$(var)))
+
 epubdocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var)))
@@ -140,6 +144,7 @@ dochelp:
 	@echo  '  htmldocs        - HTML'
 	@echo  '  latexdocs       - LaTeX'
 	@echo  '  pdfdocs         - PDF'
+	@echo  '  rst2pdf         - PDF, using experimental rst2pdf support'
 	@echo  '  epubdocs        - EPUB'
 	@echo  '  xmldocs         - XML'
 	@echo  '  linkcheckdocs   - check for broken external links'
diff --git a/Documentation/conf.py b/Documentation/conf.py
index 66e121df59cd..6f2788aac81e 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -123,6 +123,12 @@ if (major == 1 and minor > 3) or (major > 1):
 else:
     extensions.append("sphinx.ext.pngmath")
 
+# Enable experimental rst2pdf, if available
+try:
+    extensions.append("rst2pdf.pdfbuilder")
+except:
+    sys.stderr.write('rst2pdf extension not available.\n')
+
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
 
@@ -614,12 +620,15 @@ epub_exclude_files = ['search.html']
 #
 # See the Sphinx chapter of https://ralsina.me/static/manual.pdf
 #
-# FIXME: Do not add the index file here; the result will be too big. Adding
-# multiple PDF files here actually tries to get the cross-referencing right
-# *between* PDF files.
-pdf_documents = [
-    ('kernel-documentation', u'Kernel', u'Kernel', u'J. Random Bozo'),
-]
+
+# Add all LaTeX files to PDF documents as well
+pdf_documents = []
+for l in latex_documents:
+    doc = l[0]
+    fn = l[1].replace(".tex", "")
+    name = l[2]
+    authors = l[3]
+    pdf_documents.append((doc, fn, name, authors))
 
 # kernel-doc extension configuration for running Sphinx directly (e.g. by Read
 # the Docs). In a normal build, these are supplied from the Makefile via command
diff --git a/Documentation/userspace-api/media/Makefile b/Documentation/userspace-api/media/Makefile
index 81a4a1a53bce..8c6b3ac4ecb0 100644
--- a/Documentation/userspace-api/media/Makefile
+++ b/Documentation/userspace-api/media/Makefile
@@ -59,6 +59,7 @@ all: $(IMGDOT) $(BUILDDIR) ${TARGETS}
 html: all
 epub: all
 xml: all
+pdf: all
 latex: $(IMGPDF) all
 linkcheck:
 
diff --git a/Makefile b/Makefile
index 43ecedeb3f02..db4043578eec 100644
--- a/Makefile
+++ b/Makefile
@@ -264,7 +264,7 @@ no-dot-config-targets := $(clean-targets) \
 			 cscope gtags TAGS tags help% %docs check% coccicheck \
 			 $(version_h) headers headers_% archheaders archscripts \
 			 %asm-generic kernelversion %src-pkg dt_binding_check \
-			 outputmakefile
+			 outputmakefile rst2pdf
 no-sync-config-targets := $(no-dot-config-targets) %install kernelrelease
 single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.s %.symtypes %/
 
@@ -1654,7 +1654,7 @@ $(help-board-dirs): help-%:
 
 # Documentation targets
 # ---------------------------------------------------------------------------
-DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs epubdocs cleandocs \
+DOC_TARGETS := xmldocs latexdocs pdfdocs rst2pdf htmldocs epubdocs cleandocs \
 	       linkcheckdocs dochelp refcheckdocs
 PHONY += $(DOC_TARGETS)
 $(DOC_TARGETS):
-- 
2.29.2

Re: [PATCH 01/13] docs: conf.py: fix sphinx version detection for margin set

[Mauro Carvalho Chehab]

Em Thu, 10 Dec 2020 07:48:45 -0700
Jonathan Corbet <corbet@lwn.net> escreveu:

> On Thu, 10 Dec 2020 11:55:40 +0100
> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> 
> > The PDF generator has a logic to detect the proper way to
> > setup the page margins. By default, the page has about
> > 14.8 cm, which is too short to display some tables and literal
> > blocks. So, previous patches changed it to be around 17.5 cm,
> > but the logic only works with Sphinx version 1.x.x.
> > 
> > Fix it.
> > 
> > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>  
> 
> Acked-by: Jonathan Corbet <corbet@lwn.net>
> 
> Feel free to route this one with the rest of the set.
> 
> Someday it might be nice to isolate all of the latex stuff into
> conf-latex.py or some such so most of us don't have to look at it..:)

Yeah, makes sense.

Btw, just gave another trial of rst2pdf (sent the RFC patch
adding support for it)...


Still doesn't work: lots of documents are produced with
zero size :-(


Thanks,
Mauro

Re: [PATCH RFC] docs: experimental: build PDF with rst2pdf

[Mauro Carvalho Chehab]

Em Thu, 10 Dec 2020 17:01:19 +0100
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> escreveu:

> Add an experimental PDF builder using rst2pdf
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

I opened an issue at:

https://github.com/rst2pdf/rst2pdf/issues/958

Let's hope someone at rst2pdf could help fixing this ;-)

Regards,
Mauro


> ---
>  Documentation/Makefile                     |  5 +++++
>  Documentation/conf.py                      | 21 +++++++++++++++------
>  Documentation/userspace-api/media/Makefile |  1 +
>  Makefile                                   |  4 ++--
>  4 files changed, 23 insertions(+), 8 deletions(-)
> 
> diff --git a/Documentation/Makefile b/Documentation/Makefile
> index 61a7310b49e0..c3c8fb10f94e 100644
> --- a/Documentation/Makefile
> +++ b/Documentation/Makefile
> @@ -115,6 +115,10 @@ pdfdocs: latexdocs
>  
>  endif # HAVE_PDFLATEX
>  
> +rst2pdf:
> +	@$(srctree)/scripts/sphinx-pre-install --version-check
> +	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,pdf,$(var),pdf,$(var)))
> +
>  epubdocs:
>  	@$(srctree)/scripts/sphinx-pre-install --version-check
>  	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var)))
> @@ -140,6 +144,7 @@ dochelp:
>  	@echo  '  htmldocs        - HTML'
>  	@echo  '  latexdocs       - LaTeX'
>  	@echo  '  pdfdocs         - PDF'
> +	@echo  '  rst2pdf         - PDF, using experimental rst2pdf support'
>  	@echo  '  epubdocs        - EPUB'
>  	@echo  '  xmldocs         - XML'
>  	@echo  '  linkcheckdocs   - check for broken external links'
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index 66e121df59cd..6f2788aac81e 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -123,6 +123,12 @@ if (major == 1 and minor > 3) or (major > 1):
>  else:
>      extensions.append("sphinx.ext.pngmath")
>  
> +# Enable experimental rst2pdf, if available
> +try:
> +    extensions.append("rst2pdf.pdfbuilder")
> +except:
> +    sys.stderr.write('rst2pdf extension not available.\n')
> +
>  # Add any paths that contain templates here, relative to this directory.
>  templates_path = ['_templates']
>  
> @@ -614,12 +620,15 @@ epub_exclude_files = ['search.html']
>  #
>  # See the Sphinx chapter of https://ralsina.me/static/manual.pdf
>  #
> -# FIXME: Do not add the index file here; the result will be too big. Adding
> -# multiple PDF files here actually tries to get the cross-referencing right
> -# *between* PDF files.
> -pdf_documents = [
> -    ('kernel-documentation', u'Kernel', u'Kernel', u'J. Random Bozo'),
> -]
> +
> +# Add all LaTeX files to PDF documents as well
> +pdf_documents = []
> +for l in latex_documents:
> +    doc = l[0]
> +    fn = l[1].replace(".tex", "")
> +    name = l[2]
> +    authors = l[3]
> +    pdf_documents.append((doc, fn, name, authors))
>  
>  # kernel-doc extension configuration for running Sphinx directly (e.g. by Read
>  # the Docs). In a normal build, these are supplied from the Makefile via command
> diff --git a/Documentation/userspace-api/media/Makefile b/Documentation/userspace-api/media/Makefile
> index 81a4a1a53bce..8c6b3ac4ecb0 100644
> --- a/Documentation/userspace-api/media/Makefile
> +++ b/Documentation/userspace-api/media/Makefile
> @@ -59,6 +59,7 @@ all: $(IMGDOT) $(BUILDDIR) ${TARGETS}
>  html: all
>  epub: all
>  xml: all
> +pdf: all
>  latex: $(IMGPDF) all
>  linkcheck:
>  
> diff --git a/Makefile b/Makefile
> index 43ecedeb3f02..db4043578eec 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -264,7 +264,7 @@ no-dot-config-targets := $(clean-targets) \
>  			 cscope gtags TAGS tags help% %docs check% coccicheck \
>  			 $(version_h) headers headers_% archheaders archscripts \
>  			 %asm-generic kernelversion %src-pkg dt_binding_check \
> -			 outputmakefile
> +			 outputmakefile rst2pdf
>  no-sync-config-targets := $(no-dot-config-targets) %install kernelrelease
>  single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.s %.symtypes %/
>  
> @@ -1654,7 +1654,7 @@ $(help-board-dirs): help-%:
>  
>  # Documentation targets
>  # ---------------------------------------------------------------------------
> -DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs epubdocs cleandocs \
> +DOC_TARGETS := xmldocs latexdocs pdfdocs rst2pdf htmldocs epubdocs cleandocs \
>  	       linkcheckdocs dochelp refcheckdocs
>  PHONY += $(DOC_TARGETS)
>  $(DOC_TARGETS):



Thanks,
Mauro

[PATCH RFC v2] docs: experimental: build PDF with rst2pdf

[Mauro Carvalho Chehab]

Add an experimental PDF builder using rst2pdf

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---

Please notice that 18 documents (of a total of 71) won't build with 
rst2pdf. There's an opened issue about that at:

    https://github.com/rst2pdf/rst2pdf/issues/958

v2: usage of SPHINXDIRS was fixed.


 Documentation/Makefile                     |  5 +++++
 Documentation/conf.py                      | 21 +++++++++++++++------
 Documentation/sphinx/load_config.py        | 12 ++++++++++++
 Documentation/userspace-api/media/Makefile |  1 +
 Makefile                                   |  4 ++--
 5 files changed, 35 insertions(+), 8 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 61a7310b49e0..c3c8fb10f94e 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -115,6 +115,10 @@ pdfdocs: latexdocs
 
 endif # HAVE_PDFLATEX
 
+rst2pdf:
+	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,pdf,$(var),pdf,$(var)))
+
 epubdocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var)))
@@ -140,6 +144,7 @@ dochelp:
 	@echo  '  htmldocs        - HTML'
 	@echo  '  latexdocs       - LaTeX'
 	@echo  '  pdfdocs         - PDF'
+	@echo  '  rst2pdf         - PDF, using experimental rst2pdf support'
 	@echo  '  epubdocs        - EPUB'
 	@echo  '  xmldocs         - XML'
 	@echo  '  linkcheckdocs   - check for broken external links'
diff --git a/Documentation/conf.py b/Documentation/conf.py
index 66e121df59cd..6f2788aac81e 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -123,6 +123,12 @@ if (major == 1 and minor > 3) or (major > 1):
 else:
     extensions.append("sphinx.ext.pngmath")
 
+# Enable experimental rst2pdf, if available
+try:
+    extensions.append("rst2pdf.pdfbuilder")
+except:
+    sys.stderr.write('rst2pdf extension not available.\n')
+
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
 
@@ -614,12 +620,15 @@ epub_exclude_files = ['search.html']
 #
 # See the Sphinx chapter of https://ralsina.me/static/manual.pdf
 #
-# FIXME: Do not add the index file here; the result will be too big. Adding
-# multiple PDF files here actually tries to get the cross-referencing right
-# *between* PDF files.
-pdf_documents = [
-    ('kernel-documentation', u'Kernel', u'Kernel', u'J. Random Bozo'),
-]
+
+# Add all LaTeX files to PDF documents as well
+pdf_documents = []
+for l in latex_documents:
+    doc = l[0]
+    fn = l[1].replace(".tex", "")
+    name = l[2]
+    authors = l[3]
+    pdf_documents.append((doc, fn, name, authors))
 
 # kernel-doc extension configuration for running Sphinx directly (e.g. by Read
 # the Docs). In a normal build, these are supplied from the Makefile via command
diff --git a/Documentation/sphinx/load_config.py b/Documentation/sphinx/load_config.py
index eeb394b39e2c..8266afd438aa 100644
--- a/Documentation/sphinx/load_config.py
+++ b/Documentation/sphinx/load_config.py
@@ -43,6 +43,18 @@ def loadConfig(namespace):
 
             namespace['latex_documents'] = new_latex_docs
 
+            new_pdf_docs = []
+            pdf_documents = namespace['pdf_documents']
+
+            for l in pdf_documents:
+                if l[0].find(dir + '/') == 0:
+                    has = True
+                    fn = l[0][len(dir) + 1:]
+                    new_pdf_docs.append((fn, l[1], l[2], l[3]))
+                    break
+
+            namespace['pdf_documents'] = new_pdf_docs
+
         # If there is an extra conf.py file, load it
         if os.path.isfile(config_file):
             sys.stdout.write("load additional sphinx-config: %s\n" % config_file)
diff --git a/Documentation/userspace-api/media/Makefile b/Documentation/userspace-api/media/Makefile
index 81a4a1a53bce..8c6b3ac4ecb0 100644
--- a/Documentation/userspace-api/media/Makefile
+++ b/Documentation/userspace-api/media/Makefile
@@ -59,6 +59,7 @@ all: $(IMGDOT) $(BUILDDIR) ${TARGETS}
 html: all
 epub: all
 xml: all
+pdf: all
 latex: $(IMGPDF) all
 linkcheck:
 
diff --git a/Makefile b/Makefile
index 43ecedeb3f02..db4043578eec 100644
--- a/Makefile
+++ b/Makefile
@@ -264,7 +264,7 @@ no-dot-config-targets := $(clean-targets) \
 			 cscope gtags TAGS tags help% %docs check% coccicheck \
 			 $(version_h) headers headers_% archheaders archscripts \
 			 %asm-generic kernelversion %src-pkg dt_binding_check \
-			 outputmakefile
+			 outputmakefile rst2pdf
 no-sync-config-targets := $(no-dot-config-targets) %install kernelrelease
 single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.s %.symtypes %/
 
@@ -1654,7 +1654,7 @@ $(help-board-dirs): help-%:
 
 # Documentation targets
 # ---------------------------------------------------------------------------
-DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs epubdocs cleandocs \
+DOC_TARGETS := xmldocs latexdocs pdfdocs rst2pdf htmldocs epubdocs cleandocs \
 	       linkcheckdocs dochelp refcheckdocs
 PHONY += $(DOC_TARGETS)
 $(DOC_TARGETS):
-- 
2.29.2

Re: [PATCH RFC v2] docs: experimental: build PDF with rst2pdf

[Jonathan Corbet]

On Fri, 11 Dec 2020 09:33:32 +0100
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:

> Add an experimental PDF builder using rst2pdf
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
> 
> Please notice that 18 documents (of a total of 71) won't build with 
> rst2pdf. There's an opened issue about that at:
> 
>     https://github.com/rst2pdf/rst2pdf/issues/958
> 
> v2: usage of SPHINXDIRS was fixed.
> 
> 
>  Documentation/Makefile                     |  5 +++++
>  Documentation/conf.py                      | 21 +++++++++++++++------
>  Documentation/sphinx/load_config.py        | 12 ++++++++++++
>  Documentation/userspace-api/media/Makefile |  1 +
>  Makefile                                   |  4 ++--
>  5 files changed, 35 insertions(+), 8 deletions(-)

So I would dearly love to have rst2pdf working.

I applied this, then tried to see what would happen if I ran a build
without having rst2pdf installed:

> 1108 meer kernel: make htmldocs
>   SPHINX  htmldocs --> file:///stuff/k/git/kernel/Documentation/output
> make[2]: Nothing to be done for 'html'.
> WARNING: The kernel documentation build process
>         support for Sphinx v3.0 and above is brand new. Be prepared for
>         possible issues in the generated output.
>         enabling CJK for LaTeX builder
> 
> Extension error:
> Could not import extension rst2pdf.pdfbuilder (exception: No module named 'rst2pdf')
> make[1]: *** [Documentation/Makefile:91: htmldocs] Error 2
> make: *** [Makefile:1663: htmldocs] Error 2

Methinks it's perhaps not quite ready for linux-next yet :)

With rst2pdf installed I get a bunch of zero-length files, as promised.
Pretty much none of the larger "books" make it through.  It's a start,
though.  I'll happily apply this as a step forward once it doesn't break
the docs build if rst2pdf is missing.

Thanks,

jon

Re: [PATCH RFC v2] docs: experimental: build PDF with rst2pdf

[Mauro Carvalho Chehab]

Em Fri, 11 Dec 2020 13:48:59 -0700
Jonathan Corbet <corbet@lwn.net> escreveu:

> On Fri, 11 Dec 2020 09:33:32 +0100
> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> 
> > Add an experimental PDF builder using rst2pdf
> > 
> > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> > ---
> > 
> > Please notice that 18 documents (of a total of 71) won't build with 
> > rst2pdf. There's an opened issue about that at:
> > 
> >     https://github.com/rst2pdf/rst2pdf/issues/958
> > 
> > v2: usage of SPHINXDIRS was fixed.
> > 
> > 
> >  Documentation/Makefile                     |  5 +++++
> >  Documentation/conf.py                      | 21 +++++++++++++++------
> >  Documentation/sphinx/load_config.py        | 12 ++++++++++++
> >  Documentation/userspace-api/media/Makefile |  1 +
> >  Makefile                                   |  4 ++--
> >  5 files changed, 35 insertions(+), 8 deletions(-)  
> 
> So I would dearly love to have rst2pdf working.
> 
> I applied this, then tried to see what would happen if I ran a build
> without having rst2pdf installed:
> 
> > 1108 meer kernel: make htmldocs
> >   SPHINX  htmldocs --> file:///stuff/k/git/kernel/Documentation/output
> > make[2]: Nothing to be done for 'html'.
> > WARNING: The kernel documentation build process
> >         support for Sphinx v3.0 and above is brand new. Be prepared for
> >         possible issues in the generated output.
> >         enabling CJK for LaTeX builder
> > 
> > Extension error:
> > Could not import extension rst2pdf.pdfbuilder (exception: No module named 'rst2pdf')
> > make[1]: *** [Documentation/Makefile:91: htmldocs] Error 2
> > make: *** [Makefile:1663: htmldocs] Error 2  
> 
> Methinks it's perhaps not quite ready for linux-next yet :)

Well, I haven't test this.

I'm not an usual python programmer, so, don't know much about its 
specifics... Yet, I would be expecting that something like this:

	try:
	    extensions.append("rst2pdf.pdfbuilder")
	except:
	    sys.stderr.write('rst2pdf extension not available.\n')
	

Would avoid it to crash, if the extension is not available.
Silly me :-)

Still, I suspect that it should not be hard to modify the above to
avoid the crash. 

I shouldn't be doing much development those days, as I'm taking
some vacations, after sending media stuff for 5.11. 

So, if you have a better idea about how to optionally probe an
extension, feel free to modify my patch.


> With rst2pdf installed I get a bunch of zero-length files, as promised.
> Pretty much none of the larger "books" make it through.  

Yeah. I guess one of the issues is with tables that don't fit into
a single page.

Yet, devicetree book is empty. That sounds really weird, as there are
few files on it, and I didn't see anything uncommon on the rst files.

> It's a start,
> though.  I'll happily apply this as a step forward once it doesn't break
> the docs build if rst2pdf is missing.

Sounds like a plan.

Thanks,
Mauro

Re: [PATCH RFC v2] docs: experimental: build PDF with rst2pdf

[Jonathan Corbet]

On Sat, 12 Dec 2020 00:54:35 +0100
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:

> I'm not an usual python programmer, so, don't know much about its 
> specifics... Yet, I would be expecting that something like this:
> 
> 	try:
> 	    extensions.append("rst2pdf.pdfbuilder")
> 	except:
> 	    sys.stderr.write('rst2pdf extension not available.\n')
> 	
> 
> Would avoid it to crash, if the extension is not available.
> Silly me :-)

No, that's not going to do it, for a couple of reasons.  First being that
all it's doing is appending a string to a list, which pretty much always
succeeds.  The attempt to actually import the module happens later.

...and you won't catch that either because it isn't actually throwing an
exception, it's just noting the problem and giving up.

The right solution is probably something like this:

	try:
	    import rst2pdf
	    extensions.append('rst2pdf.pdfbuilder')
	except ModuleNotFoundError:
	    pass # no rst2pdf for you

This is totally untested, of course.

[Incidentally, a blank "except:" clause like the one you had is, in my
experience, a bad idea.  That will catch *anything*, leading to hiding all
kinds of bugs.  Not that I've ever committed such a faux pas and suffered
the consequences myself...no...never...honest...]

I'll mess with this a bit more later.

Thanks,

jon